Structured Outputs
Generate type-safe, structured responses from AI models
Introduction
When building AI applications, you often need responses in a specific format—whether that's extracting data fields, classifying into different categories, or generating different fields. JSON has become the universal language for data exchange between applications, but getting AI models to consistently produce valid, well-structured JSON can be challenging.
Structured Outputs solves this problem by ensuring AI models always generate responses that perfectly match your defined JSON Schema. Instead of hoping the model follows your formatting instructions or writing complex validation logic, you get guaranteed compliance with your data structure requirements.
For example: imagine you want to extract the name, age, and email of a user from a text.
from pydantic import BaseModel
class User(BaseModel):
name: str
age: int
email: strimport { z } from "zod";
const User = z.object({
name: z.string(),
age: z.number(),
email: z.string()
});{
"type": "json_schema",
"json_schema": {
"name": "User",
"strict": true,
"schema": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
},
"email": {
"type": "string"
}
},
"required": ["name", "age", "email"],
"additionalProperties": false
}
}
}Key benefits of Structured Outputs:
- Reliable type-safety: No need to validate or retry incorrectly formatted responses
- Explicit refusals: Safety-based model refusals are now programmatically detectable
- Simpler prompting: No need for strongly worded prompts to achieve consistent formatting
Supported models
A key benefit of using AnotherAI's inference API is that structured outputs are supported across all available models. While some providers only support structured outputs on specific models, AnotherAI ensures you get properly formatted data from any model you choose to use. This means you can reliably extract structured data regardless of which underlying model powers your application.
How to use
The openai Python library offers a highly convenient way to achieve this by directly providing a Pydantic model definition.
To get structured output using the openai Python library with AnotherAI:
- Define your desired output structure as a Pydantic
BaseModel. - Use the
client.beta.chat.completions.parse()method (note the.parse()instead of.create()). - Pass your Pydantic class directly to the
response_formatparameter. - Access the parsed Pydantic object directly from
response.choices[0].message.parsed.
Example: Basic Usage with Deployments
Let's redefine the get_country example using a Pydantic model:
from pydantic import BaseModel
# Assuming `openai` client is configured as `client`
class CountryInfo(BaseModel):
country: str
population: int
def get_country(city: str):
# Use the `.parse()` method for structured output with Pydantic
completion = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a helpful assistant that extracts geographical information."},
{"role": "user", "content": f"What is the country and population of {{city}}?"}
],
# Pass the Pydantic class directly as the response format
response_format=CountryInfo,
extra_body={
"input": {
"city": city
}
},
metadata={
"agent_id": "country-extractor",
}
)
parsed_output: CountryInfo = completion.choices[0].message.parsed
return parsed_outputThis approach leverages the openai library's integration with Pydantic to abstract away the manual JSON schema definition and response parsing, providing a cleaner developer experience.
The openai TypeScript library provides structured output support using Zod schemas for type-safe validation.
To get structured output using the openai TypeScript library with AnotherAI:
- Define your desired output structure using Zod schema.
- Use the
client.beta.chat.completions.parse()method. - Pass your Zod schema using
zodResponseFormat()helper to theresponse_formatparameter. - Access the parsed object directly from
response.choices[0].message.parsed.
Example: Basic Usage with Deployments
Let's redefine the get_country example using a Zod schema:
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";
// Assuming `openai` client is configured as `client`
const CountryInfo = z.object({
country: z.string(),
population: z.number().int()
});
async function getCountry(city: string) {
// Use the `.parse()` method for structured output with Zod
const completion = await client.beta.chat.completions.parse({
model: "gpt-4o",
messages: [
{role: "system", content: "You are a helpful assistant that extracts geographical information."},
{role: "user", content: `What is the country and population of {{city}}?`}
],
// Pass the Zod schema using zodResponseFormat helper
response_format: zodResponseFormat(CountryInfo, "CountryInfo"),
input: {
city: city
},
metadata: {
agent_id: "country-extractor",
}
});
const parsedOutput = completion.choices[0].message.parsed;
return parsedOutput;
}This approach leverages the openai library's integration with Zod to provide type-safe structured outputs with TypeScript.
For direct API integration, you can use CURL with raw JSON Schema definitions.
To get structured output using CURL with AnotherAI:
- Define your desired output structure using JSON Schema format.
- Use the
/v1/chat/completionsendpoint. - Pass your JSON schema in the
response_formatparameter withtype: "json_schema". - Access the parsed JSON from the response.
Example: Basic Usage with Deployments
Let's redefine the get_country example using raw JSON Schema:
curl https://api.anotherai.dev/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_API_KEY" \
-d '{
"model": "gpt-4o",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant that extracts geographical information."
},
{
"role": "user",
"content": "What is the country and population of {{city}}?"
}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "CountryInfo",
"strict": true,
"schema": {
"type": "object",
"properties": {
"country": {
"type": "string"
},
"population": {
"type": "integer"
}
},
"required": ["country", "population"],
"additionalProperties": false
}
}
},
"input": {
"city": "Paris"
},
"metadata": {
"agent_id": "country-extractor"
}
}'This approach uses raw JSON Schema definitions to ensure structured output compliance at the API level.
When using structured output, the prompt does not need to explicitly ask for JSON output, as AnotherAI automatically handles the formatting. Simply focus on describing the task clearly and let AnotherAI take care of ensuring the response matches your defined schema.
Description and examples
You can significantly improve the LLM's understanding of the desired output structure by providing description and examples directly within your response_model schema. By using pydantic.Field, you can annotate each field with a clear description of its purpose and provide a list of illustrative examples. These descriptions and examples are passed along to the LLM as part of the schema definition, helping it grasp the expected data format and content for each attribute.
Here's an example:
from typing import Optional, List
from pydantic import BaseModel, Field
class CalendarEvent(BaseModel):
title: Optional[str] = Field(
None,
description="The event title/name",
examples=["Team Meeting", "Quarterly Review"]
)
date: Optional[str] = Field(
None,
description="Date in YYYY-MM-DD format",
examples=["2023-05-21", "2023-06-15"]
)
start_time: Optional[str] = Field(
None,
description="Start time in 24-hour format",
examples=["14:00", "09:30"]
)
...By providing these details, you make the task clearer for the LLM, reducing ambiguity and leading to better, more reliable structured data extraction.
Migrating to Structured Outputs
Using Claude Code
Convert @customer-support-agent.py to use structured outputs instead of returning plain text responses.Examples
Here are common examples showing how to migrate from traditional JSON prompting to using structured outputs:
How is this guide?