The foundational pattern for AI models to interact with external tools and APIs
Function calling is the core mechanism that enables large language models to interact with external systems by generating structured requests to invoke predefined functions. Rather than being limited to generating text, a model with function calling capabilities can express its intent to call specific functions with properly formatted arguments, enabling applications to bridge the gap between natural language understanding and programmatic action.
OpenAI introduced function calling as a first-class API feature in June 2023, and it has since been adopted by virtually every major AI provider including Anthropic (as "tool use"), Google, and open-source model frameworks. The pattern has become so fundamental that it underpins higher-level protocols like MCP and A2A -- these protocols standardize how functions are discovered and managed, but function calling remains the underlying mechanism models use to express tool invocation intent.
The function calling workflow is elegant in its simplicity. Developers define available functions using JSON Schema, specifying each function's name, description, and parameter structure. These definitions are included in the API request alongside the conversation messages. When the model determines that calling a function would help address the user's request, it generates a structured response containing the function name and arguments instead of a text response. The application executes the function, sends the result back to the model, and the model incorporates the result into its final response.
This pattern supports increasingly sophisticated interaction modes. Parallel function calling allows the model to request multiple independent function calls simultaneously, reducing latency for tasks that require information from several sources. Sequential function calling enables multi-step tool usage where the output of one call informs the next. Forced function calling constrains the model to call a specific function, useful for structured data extraction. Strict mode guarantees that function arguments exactly conform to the provided schema, eliminating parsing errors.
Function calling has transformed AI application development by enabling a clean separation of concerns: the model handles natural language understanding and decision-making, while the application handles actual execution. This means models do not need to know how to query a database, call an API, or perform a calculation -- they just need to know when such actions are needed and what parameters to provide.
Function calling is implemented as an extension of the chat completion API. The request includes a messages array (the conversation) and a tools array (the available functions). Each tool definition contains a type ("function"), and a function object with name, description, and parameters (a JSON Schema object).
During inference, the model processes the conversation and tool definitions together. If the model determines a function call is appropriate, it returns a message with a tool_calls array instead of (or in addition to) text content. Each tool call contains an id, the function name, and a JSON string of arguments conforming to the function's parameter schema.
The application then executes the indicated function(s) and constructs tool result messages, each containing the tool_call_id and the function's return value. These results are appended to the conversation, and the API is called again so the model can generate a final response incorporating the function results.
For parallel function calling, the model may return multiple entries in the tool_calls array within a single response. The application can execute these calls concurrently since they are independent by definition. All results are returned together in the subsequent API call.
Strict mode adds a compilation step where the JSON Schema is pre-processed to enable constrained decoding during generation. This ensures that every generated argument string is valid JSON conforming exactly to the schema -- correct types, required fields present, enum values respected, and no extra fields. This eliminates the class of runtime errors caused by malformed function arguments.
Function calling is supported by all major AI providers with slight variations in API design. OpenAI's implementation is the most widely referenced, with Anthropic offering an equivalent "tool use" feature, Google providing function calling in the Gemini API, and open-source frameworks supporting it through libraries like Outlines and Instructor.
The ecosystem includes extensive tooling for working with function definitions: schema generators that create tool definitions from code (TypeScript types, Python dataclasses, Pydantic models), testing frameworks for validating function calling behavior, and observability tools for monitoring tool use patterns in production.
Function calling serves as the foundation layer for higher-level abstractions. MCP uses function calling internally when models invoke MCP tools. Agent frameworks use function calling to enable their agents to take actions. Structured output features are essentially function calling applied to the response format itself. Understanding function calling is essential for working with any of these higher-level tools and protocols.
Start by defining a simple function (like getting weather data or looking up a user) as a JSON Schema tool definition. Include clear descriptions for both the function and its parameters -- the model uses these descriptions to decide when and how to call the function.
Make an API call including your tool definition alongside a user message that would benefit from the tool. Observe the model's response: it should return a tool_calls array with the function name and arguments.
Implement the function execution logic in your application, construct the tool result message, and make a follow-up API call to get the model's final response incorporating the function result.
Gradually add complexity: multiple functions, parallel calling, error handling for failed function executions, and validation of function arguments. The provider documentation (OpenAI, Anthropic, Google) includes comprehensive guides and examples for each feature.
MCP (Model Context Protocol)
The universal standard for connecting AI models to tools and data
A2A (Agent-to-Agent Protocol)
Enabling AI agents to discover, communicate, and collaborate across frameworks
RAG (Retrieval-Augmented Generation)
Grounding AI responses in real-world data through intelligent retrieval
Context Engineering
The systems discipline of designing optimal information flow into AI models