Skip to content

feat(openai): update model YAMLs [bot]#1496

Merged
harshiv-26 merged 2 commits into
mainfrom
bot/update-openai-20260617-093116
Jun 17, 2026
Merged

feat(openai): update model YAMLs [bot]#1496
harshiv-26 merged 2 commits into
mainfrom
bot/update-openai-20260617-093116

Conversation

@models-bot

@models-bot models-bot Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Auto-generated by poc-agent for provider openai.


Note

Medium Risk
Wide metadata changes affect model pickers, deprecation warnings, and param validation for many OpenAI models; incorrect dates or supportedValues could misroute users or reject valid API params.

Overview
Bulk refresh of OpenAI provider model YAMLs (auto-generated) to align lifecycle, pricing notation, and request-parameter metadata with current OpenAI docs.

Deprecation lifecycle: Many models move from active/preview to deprecated, with isDeprecated: true, earlier deprecationDate values, and a new retirementDate where the old single date used to live. Affected families include legacy GPT-4 variants, search/TTS/computer-use previews, GPT-5 dated snapshots, Codex lines, o1/o3-pro/o4-mini, gpt-image-1, and sora-2.

Active model tuning: Current GPT-5.x entries gain explicit reasoning_effort supportedValues (including xhigh where applicable). gpt-5.4-pro adds priority_pricing. gpt-4.1 / gpt-5.2 snapshots add json_output where missing. Several audio/moderation/realtime configs removeParams now strip reasoning_effort; gpt-4.1-2025-04-14 and transcribe YAMLs adjust params/pricing/sources.

Formatting: Token costs are largely normalized from decimal literals to scientific notation (equivalent values); minor list formatting (e.g. "n"n).

Reviewed by Cursor Bugbot for commit 6b1a21e. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

Copy link
Copy Markdown
Contributor

/test-models

@harshiv-26

Copy link
Copy Markdown
Collaborator

Gateway test results

  • Total: 162
  • Passed: 132
  • Failed: 0
  • Validation failed: 0
  • Errored: 0
  • Skipped: 30
  • Success rate: 100.0%
Provider Model Scenarios
openai computer-use-preview skipped: skip-check
openai gpt-3.5-turbo-instruct skipped: skip-check
openai gpt-4 skipped: skip-check
openai gpt-4-0613 skipped: skip-check
openai gpt-4-turbo skipped: skip-check
openai gpt-4.1 success: json-output:text-text, params:text-text, structured-output:text-text, parallel-tool-call:text-text, tool-call:text-text, structured-output:text-text:stream, parallel-tool-call:text-text:stream, params:text-text:stream, json-output:text-text:stream, tool-call:text-text:stream
openai gpt-4.1-2025-04-14 success: parallel-tool-call:text-text, parallel-tool-call:text-text:stream, json-output:text-text, structured-output:text-text, tool-call:text-text:stream, json-output:text-text:stream, params:text-text:stream, structured-output:text-text:stream, params:text-text, tool-call:text-text
openai gpt-4.1-nano skipped: skip-check
openai gpt-4o-2024-05-13 success: json-output:text-text:stream, tool-call:text-text:stream, json-output:text-text, params:text-text, structured-output:text-text:stream, structured-output:text-text, params:text-text:stream, tool-call:text-text, parallel-tool-call:text-text, parallel-tool-call:text-text:stream
openai gpt-4o-mini-search-preview skipped: skip-check
openai gpt-4o-mini-transcribe-2025-03-20 success: params
openai gpt-4o-mini-transcribe-2025-12-15 success: params
openai gpt-4o-mini-tts-2025-03-20 skipped: skip-check
openai gpt-4o-search-preview skipped: skip-check
openai gpt-4o-search-preview-2025-03-11 skipped: skip-check
openai gpt-5 success: params:text-text:stream, params:text-text, json-output:text-text, json-output:text-text:stream, parallel-tool-call:text-text, parallel-tool-call:text-text:stream, structured-output:text-text:stream, tool-call:text-text:stream, tool-call:text-text, structured-output:text-text, reasoning:text-text:stream, reasoning:text-text
openai gpt-5-2025-08-07 skipped: skip-check
openai gpt-5-chat-latest skipped: skip-check
openai gpt-5-codex skipped: skip-check
openai gpt-5-mini-2025-08-07 skipped: skip-check
openai gpt-5-nano skipped: skip-check
openai gpt-5-nano-2025-08-07 skipped: skip-check
openai gpt-5-pro skipped: skip-check
openai gpt-5-pro-2025-10-06 skipped: skip-check
openai gpt-5.1 success: tool-call:text-text:stream, parallel-tool-call:text-text, structured-output:text-text:stream, tool-call:text-text, params:text-text, params:text-text:stream, structured-output:text-text, parallel-tool-call:text-text:stream, reasoning:text-text:stream, reasoning:text-text
openai gpt-5.1-chat-latest skipped: skip-check
openai gpt-5.1-codex skipped: skip-check
openai gpt-5.1-codex-mini skipped: skip-check
openai gpt-5.2 success: parallel-tool-call:text-text, params:text-text:stream, params:text-text, tool-call:text-text, structured-output:text-text:stream, tool-call:text-text:stream, structured-output:text-text, parallel-tool-call:text-text:stream, reasoning:text-text, reasoning:text-text:stream
openai gpt-5.2-2025-12-11 success: tool-call:text-text:stream, structured-output:text-text, params:text-text, parallel-tool-call:text-text, parallel-tool-call:text-text:stream, structured-output:text-text:stream, tool-call:text-text, params:text-text:stream, json-output:text-text, json-output:text-text:stream, reasoning:text-text, reasoning:text-text:stream
openai gpt-5.2-codex skipped: skip-check
openai gpt-5.2-pro success: params, params:stream, parallel-tool-call, tool-call, tool-call:stream, parallel-tool-call:stream, reasoning, reasoning:stream
openai gpt-5.2-pro-2025-12-11 success: params, params:stream, tool-call, tool-call:stream, reasoning:stream, reasoning
openai gpt-5.3-codex skipped: skip-check
openai gpt-5.4 success: params:text-text, json-output:text-text, parallel-tool-call:text-text:stream, structured-output:text-text:stream, params:text-text:stream, tool-call:text-text:stream, reasoning:text-text, parallel-tool-call:text-text, tool-call:text-text, json-output:text-text:stream, structured-output:text-text, reasoning:text-text:stream
openai gpt-5.4-pro success: params:stream, params, tool-call, tool-call:stream, reasoning, reasoning:stream
openai gpt-5.5 success: params:text-text:stream, parallel-tool-call:text-text:stream, json-output:text-text, structured-output:text-text, params:text-text, tool-call:text-text:stream, parallel-tool-call:text-text, tool-call:text-text, structured-output:text-text:stream, json-output:text-text:stream, reasoning:text-text:stream, reasoning:text-text
openai gpt-5.5-2026-04-23 success: params:text-text, parallel-tool-call:text-text:stream, json-output:text-text:stream, tool-call:text-text, tool-call:text-text:stream, structured-output:text-text, params:text-text:stream, structured-output:text-text:stream, json-output:text-text, parallel-tool-call:text-text, reasoning:text-text, reasoning:text-text:stream
openai gpt-image-1 skipped: skip-check
openai gpt-realtime skipped: skip-check
openai o1 skipped: skip-check
openai o3-pro skipped: skip-check
openai o4-mini skipped: skip-check
openai omni-moderation-2024-09-26 skipped: skip-check
openai sora-2 skipped: skip-check
Successes (132)

openai/gpt-4o-2024-05-13 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-4o-2024-05-13 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-4o-2024-05-13 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-4o-2024-05-13 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-4o-2024-05-13 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-4o-2024-05-13 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-4o-2024-05-13 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-4o-2024-05-13 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-4o-2024-05-13 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-4o-2024-05-13 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-4.1 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-4.1 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-4.1 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-4.1 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-4.1 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-4.1 — structured-output:text-text:stream (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-4.1 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-4.1 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-4.1 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-4.1 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-4.1-2025-04-14 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-4.1-2025-04-14 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-4.1-2025-04-14 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-4.1-2025-04-14 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-4.1-2025-04-14 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-4.1-2025-04-14 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-4.1-2025-04-14 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-4.1-2025-04-14 — structured-output:text-text:stream (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-4.1-2025-04-14 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-4.1-2025-04-14 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.5 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-5.5 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.5 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-5.5 — structured-output:text-text (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.5 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-5.5 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.5 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.5 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.5 — structured-output:text-text:stream (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.5 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.5 — reasoning:text-text:stream (success)

Output
By standard math convention, exponentiation is evaluated **from the top/right first**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the
... (truncated, 404 chars omitted)

openai/gpt-5.5 — reasoning:text-text (success)

Output
By standard convention, exponentiation is evaluated **right-to-left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top exponent:


... (truncated, 321 chars omitted)

openai/gpt-5.5-2026-04-23 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-5.5-2026-04-23 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.5-2026-04-23 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.5-2026-04-23 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.5-2026-04-23 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.5-2026-04-23 — structured-output:text-text (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.5-2026-04-23 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-5.5-2026-04-23 — structured-output:text-text:stream (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.5-2026-04-23 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-5.5-2026-04-23 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.5-2026-04-23 — reasoning:text-text (success)

Output
By the usual convention, exponentiation is evaluated **right to left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top exponent:

... (truncated, 481 chars omitted)

openai/gpt-5.5-2026-04-23 — reasoning:text-text:stream (success)

Output
By the usual convention, exponentiation is evaluated **from right to left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Start at the top/righ
... (truncated, 541 chars omitted)

openai/gpt-5.1 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.1 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.1 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.1 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.1 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-5.1 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-5.1 — structured-output:text-text (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.1 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.1 — reasoning:text-text:stream (success)

Output
The key point is how to *interpret* the expression, because exponentiation is **right‑associative**.

So

\[
3^3^3^3
\]

means

\[
3^{(3^{(3^3)})}
\]

... (truncated, 679 chars omitted)

openai/gpt-5.1 — reasoning:text-text (success)

Output
We interpret the expression using the usual convention that exponentiation is **right-associative**:

\[
3^{3^{3^{3}}} = 3^{\left(3^{\left(3^3\right)}
... (truncated, 1005 chars omitted)

openai/gpt-5.2 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.2 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5.2 — params:text-text (success)

Output
Paris.

openai/gpt-5.2 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.2 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2 — structured-output:text-text (success)

Output
{"name":"science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.2 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.2 — reasoning:text-text (success)

Output
By standard convention, exponentiation is **right-associative**, so

\[
3^3^3^3 \;=\; 3^{\left(3^{\left(3^3\right)}\right)}.
\]

Compute from the top 
... (truncated, 488 chars omitted)

openai/gpt-5.2 — reasoning:text-text:stream (success)

Output
Exponentiation is **right-associative** in standard math notation, so

\[
3^3^3^3 \;=\; 3^{\left(3^{\left(3^3\right)}\right)}.
\]

Step by step:

1) C
... (truncated, 710 chars omitted)

openai/gpt-5.4 — params:text-text (success)

Output
Paris.

openai/gpt-5.4 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-5.4 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.4 — structured-output:text-text:stream (success)

Output
{"name":"science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.4 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5.4 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.4 — reasoning:text-text (success)

Output
Exponentiation is evaluated right to left, so:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Compute in stages:

1. \[
3^3 = 27
\]

2. \[
3^{27} = 7,\!625,\!597,\
... (truncated, 351 chars omitted)

openai/gpt-5.4 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.4 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.4 — json-output:text-text:stream (success)

Output
{
  "colors": [
    { "name": "red", "hex": "#FF0000" },
    { "name": "green", "hex": "#00FF00" },
    { "name": "blue", "hex": "#0000FF" }
  ]
}
VAL
... (truncated, 36 chars omitted)

openai/gpt-5.4 — structured-output:text-text (success)

Output
{"name":"science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.4 — reasoning:text-text:stream (success)

Output
By standard convention, exponentiation is evaluated from the top down:

\[
3^{3^{3^3}} = 3^{3^{27}}
\]

Step by step:

1. Compute the top exponent:
  
... (truncated, 427 chars omitted)

openai/gpt-5.2-2025-12-11 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2-2025-12-11 — structured-output:text-text (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.2-2025-12-11 — params:text-text (success)

Output
Paris.

openai/gpt-5.2-2025-12-11 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.2-2025-12-11 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.2-2025-12-11 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.2-2025-12-11 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2-2025-12-11 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5.2-2025-12-11 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-5.2-2025-12-11 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.2-2025-12-11 — reasoning:text-text (success)

Output
Exponentiation is evaluated **right-to-left** (it’s right-associative), so

\[
3^{3^{3^{3}}} \neq (((3^3)^3)^3)
\]
but instead
\[
3^{3^{3^{3}}}=3^{\le
... (truncated, 493 chars omitted)

openai/gpt-5.2-2025-12-11 — reasoning:text-text:stream (success)

Output
Exponentiation is **right-associative**, so

\[
3^3^3^3 \;=\; 3^{(3^{(3^3)})}.
\]

Compute from the top:

1. \[
3^3 = 27
\]
2. \[
3^{27} = 7{,}625{,}5
... (truncated, 701 chars omitted)

openai/gpt-5 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5 — params:text-text (success)

Output
Paris.

openai/gpt-5 — json-output:text-text (success)

Output
{
  "colors": [
    { "name": "Red", "hex": "#FF0000" },
    { "name": "Green", "hex": "#00FF00" },
    { "name": "Blue", "hex": "#0000FF" }
  ]
}
VAL
... (truncated, 29 chars omitted)

openai/gpt-5 — json-output:text-text:stream (success)

Output
{
  "colors": [
    { "name": "Red", "hex": "#FF0000" },
    { "name": "Green", "hex": "#00FF00" },
    { "name": "Blue", "hex": "#0000FF" }
  ]
}
VAL
... (truncated, 36 chars omitted)

openai/gpt-5 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5 — reasoning:text-text:stream (success)

Output
I can’t share my detailed internal reasoning, but here’s the correct way to evaluate it and the result.

- Convention: exponentiation is right-associa
... (truncated, 409 chars omitted)

openai/gpt-5 — reasoning:text-text (success)

Output
I can’t show my detailed chain-of-thought, but here’s a concise solution outline and the result.

- Exponentiation is right-associative, so 3^3^3^3 me
... (truncated, 446 chars omitted)

openai/gpt-4o-mini-transcribe-2025-03-20 — params (success)

Output
Hey there, what's going on?

openai/gpt-4o-mini-transcribe-2025-12-15 — params (success)

Output
Hey there, what's going on?

openai/gpt-5.2-pro-2025-12-11 — params (success)

Output
Paris.

openai/gpt-5.2-pro-2025-12-11 — params:stream (success)

Output
Paris.

openai/gpt-5.2-pro-2025-12-11 — tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2-pro-2025-12-11 — tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2-pro-2025-12-11 — reasoning:stream (success)

Output
By standard convention, exponentiation is **right‑associative**, so

\[
3^3^3^3 \;=\; 3^{\left(3^{\left(3^3\right)}\right)}.
\]

### Step 1: Evaluate 
... (truncated, 958 chars omitted)

openai/gpt-5.2-pro-2025-12-11 — reasoning (success)

Output
Exponentiation is **right-associative**, so

\[
3^3^3^3 \;=\; 3^{(3^{(3^3)})}.
\]

Now compute from the top:

1) \[
3^3 = 27
\]

2) \[
3^{(3^3)} = 3^{
... (truncated, 665 chars omitted)

openai/gpt-5.2-pro — params (success)

Output
Paris.

openai/gpt-5.2-pro — params:stream (success)

Output
Paris.

openai/gpt-5.2-pro — parallel-tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
Function: get_weather
Arguments: {"location":"Paris"}
VALIDATION: parallel-tool-call SUCCESS

openai/gpt-5.2-pro — tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2-pro — tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2-pro — parallel-tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

Function: get_weather
Arguments: {"location":"Paris"}

VALIDATION: parallel-tool-call stream 
... (truncated, 8 chars omitted)

openai/gpt-5.2-pro — reasoning (success)

Output
Exponentiation is **right-associative**, so

\[
3^{3^{3^{3}}}=3^{\left(3^{(3^{3})}\right)}.
\]

Compute from the top down:

1) \[
3^{3}=27
\]

2) \[
3
... (truncated, 513 chars omitted)

openai/gpt-5.2-pro — reasoning:stream (success)

Output
By convention, exponentiation is **right-associative**, so

\[
3^3^3^3 = 3^{\left(3^{\left(3^3\right)}\right)}
\]
(not \(((3^3)^3)^3\)).

### Step 1: 
... (truncated, 820 chars omitted)

openai/gpt-5.4-pro — params:stream (success)

Output
Paris.

openai/gpt-5.4-pro — params (success)

Output
Paris.

openai/gpt-5.4-pro — tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.4-pro — tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

VALIDATION: tool-call stream SUCCESS

openai/gpt-5.4-pro — reasoning (success)

Output
Using the standard math convention, exponentiation is evaluated **from the top down**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the
... (truncated, 740 chars omitted)

openai/gpt-5.4-pro — reasoning:stream (success)

Output
By standard convention, exponentiation is evaluated **right to left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top exponent:
 
... (truncated, 601 chars omitted)
Skipped (30)

openai/computer-use-preview — skip-check (skipped)

Skip reason
Requires the Computer Use tool to be enabled

openai/gpt-3.5-turbo-instruct — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4-0613 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4-turbo — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4.1-nano — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-mini-search-preview — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-mini-tts-2025-03-20 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-search-preview — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-search-preview-2025-03-11 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-2025-08-07 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-chat-latest — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-mini-2025-08-07 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-nano — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-nano-2025-08-07 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-pro — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-pro-2025-10-06 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.1-chat-latest — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.1-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.1-codex-mini — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.2-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.3-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-image-1 — skip-check (skipped)

Skip reason
unsupported mode 'image'

openai/gpt-realtime — skip-check (skipped)

Skip reason
unsupported mode 'realtime'

openai/o1 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/o3-pro — skip-check (skipped)

Skip reason
deprecated or retired model

openai/o4-mini — skip-check (skipped)

Skip reason
deprecated or retired model

openai/omni-moderation-2024-09-26 — skip-check (skipped)

Skip reason
unsupported mode 'moderation'

openai/sora-2 — skip-check (skipped)

Skip reason
unsupported mode 'video'

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 706f238. Configure here.

Comment thread providers/openai/gpt-4o-mini-tts-2025-03-20.yaml
@harshiv-26 harshiv-26 enabled auto-merge (squash) June 17, 2026 13:24
@github-actions

Copy link
Copy Markdown
Contributor

/test-models

@harshiv-26 harshiv-26 merged commit 1aa097f into main Jun 17, 2026
8 checks passed
@harshiv-26 harshiv-26 deleted the bot/update-openai-20260617-093116 branch June 17, 2026 13:25
@harshiv-26

Copy link
Copy Markdown
Collaborator

Gateway test results

  • Total: 162
  • Passed: 132
  • Failed: 0
  • Validation failed: 0
  • Errored: 0
  • Skipped: 30
  • Success rate: 100.0%
Provider Model Scenarios
openai computer-use-preview skipped: skip-check
openai gpt-3.5-turbo-instruct skipped: skip-check
openai gpt-4 skipped: skip-check
openai gpt-4-0613 skipped: skip-check
openai gpt-4-turbo skipped: skip-check
openai gpt-4.1 success: tool-call:text-text, json-output:text-text:stream, tool-call:text-text:stream, params:text-text:stream, parallel-tool-call:text-text, json-output:text-text, parallel-tool-call:text-text:stream, params:text-text, structured-output:text-text:stream, structured-output:text-text
openai gpt-4.1-2025-04-14 success: params:text-text:stream, structured-output:text-text:stream, tool-call:text-text, structured-output:text-text, tool-call:text-text:stream, parallel-tool-call:text-text:stream, parallel-tool-call:text-text, json-output:text-text, params:text-text, json-output:text-text:stream
openai gpt-4.1-nano skipped: skip-check
openai gpt-4o-2024-05-13 success: parallel-tool-call:text-text, params:text-text:stream, structured-output:text-text, structured-output:text-text:stream, parallel-tool-call:text-text:stream, params:text-text, json-output:text-text, tool-call:text-text, tool-call:text-text:stream, json-output:text-text:stream
openai gpt-4o-mini-search-preview skipped: skip-check
openai gpt-4o-mini-transcribe-2025-03-20 success: params
openai gpt-4o-mini-transcribe-2025-12-15 success: params
openai gpt-4o-mini-tts-2025-03-20 skipped: skip-check
openai gpt-4o-search-preview skipped: skip-check
openai gpt-4o-search-preview-2025-03-11 skipped: skip-check
openai gpt-5 success: parallel-tool-call:text-text, params:text-text:stream, json-output:text-text, params:text-text, json-output:text-text:stream, structured-output:text-text, structured-output:text-text:stream, tool-call:text-text:stream, tool-call:text-text, parallel-tool-call:text-text:stream, reasoning:text-text, reasoning:text-text:stream
openai gpt-5-2025-08-07 skipped: skip-check
openai gpt-5-chat-latest skipped: skip-check
openai gpt-5-codex skipped: skip-check
openai gpt-5-mini-2025-08-07 skipped: skip-check
openai gpt-5-nano skipped: skip-check
openai gpt-5-nano-2025-08-07 skipped: skip-check
openai gpt-5-pro skipped: skip-check
openai gpt-5-pro-2025-10-06 skipped: skip-check
openai gpt-5.1 success: parallel-tool-call:text-text:stream, parallel-tool-call:text-text, structured-output:text-text:stream, params:text-text, tool-call:text-text, structured-output:text-text, tool-call:text-text:stream, params:text-text:stream, reasoning:text-text, reasoning:text-text:stream
openai gpt-5.1-chat-latest skipped: skip-check
openai gpt-5.1-codex skipped: skip-check
openai gpt-5.1-codex-mini skipped: skip-check
openai gpt-5.2 success: tool-call:text-text, tool-call:text-text:stream, params:text-text, structured-output:text-text:stream, parallel-tool-call:text-text, parallel-tool-call:text-text:stream, params:text-text:stream, structured-output:text-text, reasoning:text-text, reasoning:text-text:stream
openai gpt-5.2-2025-12-11 success: tool-call:text-text, params:text-text, json-output:text-text, structured-output:text-text:stream, params:text-text:stream, json-output:text-text:stream, structured-output:text-text, parallel-tool-call:text-text, tool-call:text-text:stream, parallel-tool-call:text-text:stream, reasoning:text-text, reasoning:text-text:stream
openai gpt-5.2-codex skipped: skip-check
openai gpt-5.2-pro success: params:stream, params, tool-call, parallel-tool-call:stream, parallel-tool-call, tool-call:stream, reasoning, reasoning:stream
openai gpt-5.2-pro-2025-12-11 success: params, params:stream, tool-call, tool-call:stream, reasoning:stream, reasoning
openai gpt-5.3-codex skipped: skip-check
openai gpt-5.4 success: json-output:text-text:stream, params:text-text:stream, structured-output:text-text, params:text-text, structured-output:text-text:stream, tool-call:text-text:stream, json-output:text-text, tool-call:text-text, parallel-tool-call:text-text:stream, parallel-tool-call:text-text, reasoning:text-text:stream, reasoning:text-text
openai gpt-5.4-pro success: params, params:stream, tool-call:stream, tool-call, reasoning:stream, reasoning
openai gpt-5.5 success: parallel-tool-call:text-text:stream, json-output:text-text:stream, tool-call:text-text:stream, tool-call:text-text, structured-output:text-text, parallel-tool-call:text-text, params:text-text, structured-output:text-text:stream, params:text-text:stream, json-output:text-text, reasoning:text-text:stream, reasoning:text-text
openai gpt-5.5-2026-04-23 success: parallel-tool-call:text-text, structured-output:text-text:stream, tool-call:text-text:stream, json-output:text-text, params:text-text, params:text-text:stream, parallel-tool-call:text-text:stream, tool-call:text-text, structured-output:text-text, json-output:text-text:stream, reasoning:text-text, reasoning:text-text:stream
openai gpt-image-1 skipped: skip-check
openai gpt-realtime skipped: skip-check
openai o1 skipped: skip-check
openai o3-pro skipped: skip-check
openai o4-mini skipped: skip-check
openai omni-moderation-2024-09-26 skipped: skip-check
openai sora-2 skipped: skip-check
Successes (132)

openai/gpt-4o-2024-05-13 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-4o-2024-05-13 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-4o-2024-05-13 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-4o-2024-05-13 — structured-output:text-text:stream (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-4o-2024-05-13 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-4o-2024-05-13 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-4o-2024-05-13 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-4o-2024-05-13 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-4o-2024-05-13 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-4o-2024-05-13 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.2-2025-12-11 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2-2025-12-11 — params:text-text (success)

Output
Paris.

openai/gpt-5.2-2025-12-11 — json-output:text-text (success)

Output
{
  "colors": [
    { "name": "Red", "hex": "#FF0000" },
    { "name": "Green", "hex": "#00FF00" },
    { "name": "Blue", "hex": "#0000FF" }
  ]
}
VAL
... (truncated, 29 chars omitted)

openai/gpt-5.2-2025-12-11 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.2-2025-12-11 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5.2-2025-12-11 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.2-2025-12-11 — structured-output:text-text (success)

Output
{"name":"science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.2-2025-12-11 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.2-2025-12-11 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2-2025-12-11 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.2-2025-12-11 — reasoning:text-text (success)

Output
Exponentiation is **right-associative**, so

\[
3^3^3^3 = 3^{\left(3^{\left(3^3\right)}\right)}.
\]

Step by step:

1) Compute the top exponent:
\[
3^
... (truncated, 642 chars omitted)

openai/gpt-5.2-2025-12-11 — reasoning:text-text:stream (success)

Output
Exponentiation is **right-associative**, so

\[
3^3^3^3 \;=\; 3^{\left(3^{\left(3^3\right)}\right)}.
\]

Now evaluate from the top:

1. \[
3^3 = 27
\]
... (truncated, 517 chars omitted)

openai/gpt-5.4 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.4 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5.4 — structured-output:text-text (success)

Output
{"name":"science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.4 — params:text-text (success)

Output
Paris.

openai/gpt-5.4 — structured-output:text-text:stream (success)

Output
{"name":"science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.4 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.4 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "red",
      "hex": "#FF0000"
    },
    {
      "name": "green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-5.4 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.4 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.4 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.4 — reasoning:text-text:stream (success)

Output
By standard math convention, exponentiation is evaluated right-to-left:

\[
3^{3^{3^3}} = 3^{3^{27}}
\]

Step by step:

1. Compute the top exponent fi
... (truncated, 449 chars omitted)

openai/gpt-5.4 — reasoning:text-text (success)

Output
By standard math convention, exponentiation is evaluated from the top down:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

So the steps are:

1. Compute the top ex
... (truncated, 575 chars omitted)

openai/gpt-4.1-2025-04-14 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-4.1-2025-04-14 — structured-output:text-text:stream (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-4.1-2025-04-14 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-4.1-2025-04-14 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-4.1-2025-04-14 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-4.1-2025-04-14 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-4.1-2025-04-14 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-4.1-2025-04-14 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-4.1-2025-04-14 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-4.1-2025-04-14 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-4.1 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-4.1 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-4.1 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-4.1 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-4.1 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-4.1 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-4.1 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-4.1 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-4.1 — structured-output:text-text:stream (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-4.1 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.5 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.5 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.5 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.5 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.5 — structured-output:text-text (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.5 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.5 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-5.5 — structured-output:text-text:stream (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.5 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-5.5 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-5.5 — reasoning:text-text:stream (success)

Output
By standard exponent rules, exponentiation is evaluated **right-to-left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top exponen
... (truncated, 516 chars omitted)

openai/gpt-5.5 — reasoning:text-text (success)

Output
By the usual convention, exponentiation is evaluated **from right to left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top expon
... (truncated, 331 chars omitted)

openai/gpt-5.2 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2 — params:text-text (success)

Output
Paris.

openai/gpt-5.2 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.2 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.2 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.2 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5.2 — structured-output:text-text (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.2 — reasoning:text-text (success)

Output
Exponentiation towers like \(3^3^3^3\) are evaluated **right-to-left** (i.e., \(a^{b^{c}} = a^{(b^{c})}\)), not left-to-right.

### Step 1: Parenthesi
... (truncated, 633 chars omitted)

openai/gpt-5.2 — reasoning:text-text:stream (success)

Output
Exponentiation is **right-associative**, so

\[
3^{3^{3^{3}}} \;=\; 3^{\left(3^{\left(3^{3}\right)}\right)}.
\]

Compute from the top:

1) \[
3^3 = 27
... (truncated, 824 chars omitted)

openai/gpt-5.1 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.1 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.1 — structured-output:text-text:stream (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.1 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-5.1 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.1 — structured-output:text-text (success)

Output
{"name":"Science fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.1 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.1 — params:text-text:stream (success)

Output
The capital of France is Paris.

openai/gpt-5.1 — reasoning:text-text (success)

Output
Exponentiation associates to the right, so

- \(3^{3^3^3}\) means \(3^{\left(3^{(3^3)}\right)}\), **not** \(((3^3)^3)^3\).

Let’s outline it without g
... (truncated, 695 chars omitted)

openai/gpt-5.1 — reasoning:text-text:stream (success)

Output
The expression \(3^{3^{3^{3}}}\) is interpreted using the standard convention that exponentiation is **right-associative**:

\[
3^{3^{3^{3}}} = 3^{\le
... (truncated, 894 chars omitted)

openai/gpt-5.5-2026-04-23 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5.5-2026-04-23 — structured-output:text-text:stream (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5.5-2026-04-23 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5.5-2026-04-23 — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 77 chars omitted)

openai/gpt-5.5-2026-04-23 — params:text-text (success)

Output
The capital of France is Paris.

openai/gpt-5.5-2026-04-23 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5.5-2026-04-23 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5.5-2026-04-23 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.5-2026-04-23 — structured-output:text-text (success)

Output
{"date":"Friday","name":"science fair","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5.5-2026-04-23 — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex": "#FF0000"
    },
    {
      "name": "Green",
      "hex": "#00FF00"
    },
    {
      "name"
... (truncated, 84 chars omitted)

openai/gpt-5.5-2026-04-23 — reasoning:text-text (success)

Output
By standard convention, exponentiation is evaluated **right to left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

\[
3^3 = 27
\]

So:

\[
3^{(3^
... (truncated, 516 chars omitted)

openai/gpt-5.5-2026-04-23 — reasoning:text-text:stream (success)

Output
By the usual convention, exponentiation is evaluated **right-to-left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top exponent:

... (truncated, 560 chars omitted)

openai/gpt-4o-mini-transcribe-2025-12-15 — params (success)

Output
Hey there, what's going on?

openai/gpt-5 — parallel-tool-call:text-text (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

openai/gpt-5 — params:text-text:stream (success)

Output
Paris.

openai/gpt-5 — json-output:text-text (success)

Output
{
  "colors": [
    { "name": "Red", "hex": "#FF0000" },
    { "name": "Green", "hex": "#00FF00" },
    { "name": "Blue", "hex": "#0000FF" }
  ]
}
VAL
... (truncated, 29 chars omitted)

openai/gpt-5 — params:text-text (success)

Output
Paris.

openai/gpt-5 — json-output:text-text:stream (success)

Output
{
  "colors": [
    { "name": "Red", "hex": "#FF0000" },
    { "name": "Green", "hex": "#00FF00" },
    { "name": "Blue", "hex": "#0000FF" }
  ]
}
VAL
... (truncated, 36 chars omitted)

openai/gpt-5 — structured-output:text-text (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output SUCCESS

openai/gpt-5 — structured-output:text-text:stream (success)

Output
{"name":"Science Fair","date":"Friday","participants":["Alice","Bob"]}
VALIDATION: structured-output stream SUCCESS

openai/gpt-5 — tool-call:text-text:stream (success)

Output
{"location":"London"}
VALIDATION: tool-call stream SUCCESS

openai/gpt-5 — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5 — parallel-tool-call:text-text:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

openai/gpt-5 — reasoning:text-text (success)

Output
I can’t share my step-by-step internal reasoning, but here’s the correct evaluation and a concise method.

- Exponentiation is right-associative, so 3
... (truncated, 525 chars omitted)

openai/gpt-5 — reasoning:text-text:stream (success)

Output
I can’t share my internal chain-of-thought, but here’s a clear, step-by-step solution and the key results.

- By convention, exponentiation is right-a
... (truncated, 572 chars omitted)

openai/gpt-4o-mini-transcribe-2025-03-20 — params (success)

Output
Hey there, what's going on?

openai/gpt-5.2-pro — params:stream (success)

Output
Paris.

openai/gpt-5.2-pro — params (success)

Output
Paris.

openai/gpt-5.2-pro — tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2-pro — parallel-tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

Function: get_weather
Arguments: {"location":"Paris"}

VALIDATION: parallel-tool-call stream 
... (truncated, 8 chars omitted)

openai/gpt-5.2-pro — parallel-tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
Function: get_weather
Arguments: {"location":"Paris"}
VALIDATION: parallel-tool-call SUCCESS

openai/gpt-5.2-pro — tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2-pro — reasoning (success)

Output
Exponentiation towers are evaluated **from the top down** (i.e., `^` is **right-associative**), so

\[
3^3^3^3 \;=\; 3^{\left(3^{\left(3^3\right)}\rig
... (truncated, 783 chars omitted)

openai/gpt-5.2-pro — reasoning:stream (success)

Output
Exponentiation is **right-associative** by convention, so

\[
3^3^3^3 \;=\; 3^{(3^{(3^3)})}.
\]

Step by step:

1) Compute the top exponent:
\[
3^3 = 
... (truncated, 828 chars omitted)

openai/gpt-5.2-pro-2025-12-11 — params (success)

Output
Paris.

openai/gpt-5.2-pro-2025-12-11 — params:stream (success)

Output
Paris.

openai/gpt-5.2-pro-2025-12-11 — tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.2-pro-2025-12-11 — tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

VALIDATION: tool-call stream SUCCESS

openai/gpt-5.2-pro-2025-12-11 — reasoning:stream (success)

Output
Exponentiation is **right-associative**, so

\[
3^3^3^3 = 3^{\left(3^{\left(3^3\right)}\right)}.
\]

Now evaluate from the top (right) inward:

1) Com
... (truncated, 597 chars omitted)

openai/gpt-5.2-pro-2025-12-11 — reasoning (success)

Output
Exponentiation towers are evaluated **right-to-left** (i.e., \(a^b^c = a^{(b^c)}\)).

So
\[
3^3^3^3 \;=\; 3^{(3^{(3^3)})}.
\]

Step by step:

1) Compu
... (truncated, 786 chars omitted)

openai/gpt-5.4-pro — params (success)

Output
Paris.

openai/gpt-5.4-pro — params:stream (success)

Output
Paris.

openai/gpt-5.4-pro — tool-call:stream (success)

Output
Function: get_weather
Arguments: {"location":"London"}

VALIDATION: tool-call stream SUCCESS

openai/gpt-5.4-pro — tool-call (success)

Output
Function: get_weather
Arguments: {"location":"London"}
VALIDATION: tool-call SUCCESS

openai/gpt-5.4-pro — reasoning:stream (success)

Output
Using standard math convention, exponents are evaluated **right to left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top exponen
... (truncated, 672 chars omitted)

openai/gpt-5.4-pro — reasoning (success)

Output
Using the standard math convention, exponents are evaluated **right to left**:

\[
3^3^3^3 = 3^{(3^{(3^3)})}
\]

Step by step:

1. Compute the top exp
... (truncated, 670 chars omitted)
Skipped (30)

openai/computer-use-preview — skip-check (skipped)

Skip reason
Requires the Computer Use tool to be enabled

openai/gpt-3.5-turbo-instruct — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4-0613 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4-turbo — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4.1-nano — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-mini-search-preview — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-mini-tts-2025-03-20 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-search-preview — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-4o-search-preview-2025-03-11 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-2025-08-07 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-chat-latest — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-mini-2025-08-07 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-nano — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-nano-2025-08-07 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-pro — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5-pro-2025-10-06 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.1-chat-latest — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.1-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.1-codex-mini — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.2-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-5.3-codex — skip-check (skipped)

Skip reason
deprecated or retired model

openai/gpt-image-1 — skip-check (skipped)

Skip reason
unsupported mode 'image'

openai/gpt-realtime — skip-check (skipped)

Skip reason
unsupported mode 'realtime'

openai/o1 — skip-check (skipped)

Skip reason
deprecated or retired model

openai/o3-pro — skip-check (skipped)

Skip reason
deprecated or retired model

openai/o4-mini — skip-check (skipped)

Skip reason
deprecated or retired model

openai/omni-moderation-2024-09-26 — skip-check (skipped)

Skip reason
unsupported mode 'moderation'

openai/sora-2 — skip-check (skipped)

Skip reason
unsupported mode 'video'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant