Skip to content

Invalid Schema Error when Using examples Field in Pydantic Model for OpenAI Python Client #2274

@DavidSanSan110

Description

@DavidSanSan110

Confirm this is an issue with the Python library and not an underlying OpenAI API

  • This is an issue with the Python library

Describe the bug

When you add the examples field in the Field definition of a Pydantic model (as shown in the code below) while using the OpenAI Python client, it raises a BadRequestError with error code 400. The error message indicates that the examples field is not permitted in the schema for the response_format parameter.

This is the error message:

raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid schema for response_format 'MathReasoning': In context=('properties', 'final_answer'), 'examples' is not permitted.", 'type': 'invalid_request_error', 'param': 'response_format', 'code': None}}

Question:

Is this behavior due to a limitation in the OpenAI Python client or API, or is it something that could potentially be patched in the future?

To Reproduce

from openai import Client
from pydantic import BaseModel, Field
from dotenv import load_dotenv

load_dotenv()

client = Client()

class Step(BaseModel):
    explanation: str
    output: str

class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str = Field(
        title="Final Answer",
        description="The final answer to the math problem",
        examples=["x = -3", "x = 2"]
    )

completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step."},
        {"role": "user", "content": "how can I solve 8x + 7 = -23"}
    ],
    response_format=MathReasoning,
)

math_reasoning = completion.choices[0].message

# If the model refuses to respond, you will get a refusal message
if (math_reasoning.refusal):
    print(math_reasoning.refusal)
else:
    print(math_reasoning.parsed)

Code snippets

OS

Windows

Python version

Python v3.11.9

Library version

openai v1.68.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions