Skip to content

Commit a82e04f

Browse files
Handle max_completion_tokens via extra_body for newer models on old SDK
When the API rejects max_tokens, retry with max_completion_tokens sent via extra_body, which works even on older openai library versions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 40c81bd commit a82e04f

1 file changed

Lines changed: 13 additions & 6 deletions

File tree

queries/util/openai_gpt.py

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -82,19 +82,26 @@ def _create(cls, *args: Any, **kwargs: Any) -> ChatCompletion:
8282
"""
8383
Creates a new completion while handling formatting and parsing.
8484
Newer OpenAI models require max_completion_tokens instead of
85-
max_tokens; try the new parameter first and fall back to the old.
85+
max_tokens. We send max_tokens (compatible with the installed
86+
openai library) but catch API rejection and construct a raw
87+
request with max_completion_tokens if needed.
8688
"""
8789
if "max_tokens" in kwargs:
8890
max_val = kwargs.pop("max_tokens")
8991
try:
90-
return client.chat.completions.create(
91-
*args, model=MODEL, max_completion_tokens=max_val, **kwargs
92-
)
93-
except TypeError:
94-
# Older openai library doesn't support max_completion_tokens
9592
return client.chat.completions.create(
9693
*args, model=MODEL, max_tokens=max_val, **kwargs
9794
)
95+
except openai.BadRequestError as e:
96+
if "max_completion_tokens" in str(e):
97+
# Model requires the new parameter name
98+
return client.chat.completions.create(
99+
*args,
100+
model=MODEL,
101+
extra_body={"max_completion_tokens": max_val},
102+
**kwargs,
103+
)
104+
raise
98105
return client.chat.completions.create(*args, model=MODEL, **kwargs)
99106

100107
@classmethod

0 commit comments

Comments
 (0)