Agentic AI
Tool Use and Function Calling
Tool use enables models to interact with external software by outputting structured data instead of conversational text. This capability bridges the gap between static language understanding and dynamic execution, allowing models to retrieve data or perform actions. Developers implement this when a model needs to access real-time information, perform calculations, or interface with APIs that are outside its training data.
The Mechanism of Tool Declaration
At the core of tool use is the concept of structured schema definition. When you provide a model with a tool, you are not simply giving it a name; you are providing a formal contract in the form of a JSON schema. The model inspects this schema to understand what arguments are required for a function and what data types they expect. This works because LLMs are trained on vast amounts of code documentation, making them adept at predicting the correct parameters to fill based on a natural language prompt. When the model determines it needs information outside its training window, it generates a specific request format that matches your schema, rather than generating free-form text. Your code then intercepts this intent, executes the function locally, and returns the result back to the model context. This decoupling ensures the model maintains control over the logic flow while the execution remains secure and restricted to defined boundaries.
# Define a schema for a function that the model will learn to call
weather_tool = {
"name": "get_current_weather",
"description": "Get the current weather for a specific location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City and state, e.g. San Francisco, CA"}
},
"required": ["location"]
}
}The Execution Loop and Context Handling
Once the model identifies that a tool call is necessary, it produces a structured block instead of a standard string response. As a developer, you must treat this as a control signal. The application flow is interrupted to parse this structured output and actually execute the corresponding function in your environment. Crucially, the process does not end there. You must append both the tool call and the subsequent output returned by your function back into the conversation history. This feedback loop is essential because it allows the model to 'see' the output of its own requested action. Without providing this observation, the model would remain blind to the result of the function call. By feeding this data back, you enable the model to synthesize the final natural language answer that incorporates the real-world data it just retrieved, completing the agentic cycle of reasoning, observation, and response generation.
# Simulate the agentic loop
function_call = {"name": "get_current_weather", "args": {"location": "Seattle, WA"}}
def execute_tool(call):
# This function safely executes the logic requested by the model
return {"temperature": "55F", "condition": "Rainy"}
result = execute_tool(function_call)
print(f"Model observed: {result}") # This is passed back into the LLM contextManaging Multiple Tool Choices
When dealing with complex applications, you may provide the model with a collection of diverse tools, ranging from database queries to mathematical calculators. The model acts as a dispatcher, evaluating the incoming user prompt against the descriptions of all available tools to select the most appropriate one. This selection process relies heavily on the quality of your tool descriptions. If a description is vague, the model will struggle to differentiate between similar tools. When providing multiple options, it is helpful to think of the model as a software architect; it evaluates the intent of the user and matches it against the capabilities defined in your registry. By carefully crafting the 'description' field for every tool, you guide the model's decision-making process. Providing clear, concise, and distinct descriptions ensures that the model correctly invokes the right logic at the right time, minimizing errors and improving the overall accuracy of the agentic system.
# Providing a list of available tools to the model
available_tools = [weather_tool, stock_price_tool, calculator_tool]
# The model will internally weigh these based on the user's intent
# E.g., User: 'What is 5 plus 5?' -> Model picks calculator_toolAdvanced Reasoning and Chained Calls
Sophisticated agentic behaviors often require chaining multiple tool calls together to solve a complex request. For instance, if a user asks for information about a company that requires both a database search and a stock price lookup, the model can generate a sequence of calls. The key to enabling this is the model's ability to maintain state across multiple turns. By feeding the result of the first tool back to the model, it can perform an intermediate reasoning step to determine if enough information exists or if another tool call is required. This multi-step process allows for high-level problem solving where the model decomposes a high-level goal into actionable, sub-atomic operations. Because the model understands the dependencies between functions, it will continue to output structured tool calls until it has sufficient data to provide a comprehensive final answer, effectively turning the LLM into a sophisticated programmable engine.
# Chained reasoning example
# 1. Model identifies tool 'get_company_ticker'
# 2. Model identifies tool 'get_stock_price' using the result of step 1
# 3. Model synthesizes final answer
# This loop continues until 'finish_reason' is 'stop'Error Handling and Guardrails
Since LLMs are probabilistic, they might occasionally attempt to use a tool incorrectly, such as by passing malformed arguments or calling a function that does not exist. Implementing robust guardrails is non-negotiable for production systems. You must implement input validation on the arguments returned by the model before passing them to your actual execution functions. If a validation fails, the application should return a descriptive error message to the model rather than crashing. This 'error feedback' allows the model to attempt to correct its mistake in subsequent iterations. Furthermore, you should restrict tool access based on user permissions or environmental context to ensure the model cannot perform unauthorized actions. Treat the model's tool calls as untrusted input, just as you would treat data from a browser or API client. Proper validation ensures that your system remains resilient even when the model encounters edge cases or ambiguous prompts.
# Validate input before execution
def safe_execute(call):
if "location" not in call["args"]:
return "Error: Missing required argument 'location'"
# Proceed to perform actual system action
return perform_action(call["args"])Key points
- Tool use allows language models to execute external functions by returning structured data instead of conversational text.
- Functions are defined using JSON schemas that explain the purpose and parameters of the tool to the model.
- The model identifies the need for a tool by mapping user intent to the provided descriptions of available functions.
- Successful tool calling requires a loop where the developer executes the code and feeds the output back into the conversation context.
- Effective descriptions are the most important factor in ensuring the model selects the correct tool for a given task.
- Models can perform complex, multi-step reasoning by chaining multiple tool calls until the desired outcome is achieved.
- All model-generated arguments must be validated as untrusted input to prevent system errors or unauthorized actions.
- The agentic loop relies on the model being able to process the feedback from its own function calls to refine its final answer.
Common mistakes
- Mistake: Expecting the model to execute the tool code directly. Why it's wrong: The model is a text generator, not an execution engine; it only generates the schema-compliant arguments. Fix: Ensure your application logic parses the model's output and performs the actual function execution.
- Mistake: Overloading the model with too many tool definitions. Why it's wrong: Large numbers of tools can confuse the model and increase token usage. Fix: Group relevant tools and only provide the subset necessary for the current task.
- Mistake: Failing to provide clear docstrings for function parameters. Why it's wrong: The model relies entirely on the function signature and parameter descriptions to determine intent. Fix: Write detailed descriptions for every parameter so the model knows exactly what data is required.
- Mistake: Assuming the model handles tool errors gracefully. Why it's wrong: If a tool fails or returns invalid output, the model may get stuck in a loop. Fix: Implement robust error handling in the integration layer and feed the error messages back to the model for correction.
- Mistake: Forgetting to include the tool result in the conversation history. Why it's wrong: Without the tool output, the model has no context to generate a final answer. Fix: Append the function output as a new message to the chat history to allow the model to finalize the response.
Interview questions
What is the fundamental purpose of tool use in Generative AI systems?
The fundamental purpose of tool use, often referred to as function calling, is to overcome the inherent limitations of a Large Language Model. While models are excellent at reasoning and language generation, they lack real-time access to private data and the ability to execute external actions. By allowing a model to invoke specific tools, we bridge the gap between static training data and dynamic real-world environments. For example, if a user asks for the current weather, the model can invoke a `get_weather(city)` tool, receive structured JSON data, and synthesize a natural language response based on that live information, effectively expanding the model's utility beyond its internal memory.
How does an LLM know when and how to call a specific function?
An LLM determines when to call a function through its system instructions and the provided schema definition. During the prompt engineering phase, we define available tools in a structured format, such as JSON Schema, which details the function's name, description, and parameters. The model analyzes the user's intent; if the intent maps to an available tool, the model pauses generation and outputs a specific signal, usually a structured request containing the function name and arguments. This process relies on the model's semantic understanding, where the 'description' field is crucial because it acts as the semantic anchor telling the model exactly what the function achieves.
Can you explain the workflow of a multi-step tool-use execution cycle?
A multi-step tool-use cycle is an iterative loop where the AI agent acts as a planner. First, the model receives a complex query and identifies the necessary tools. It then generates a call request, which the application code executes, returning the result to the model. The model receives this output as a new message in the conversation history and then decides whether to call another tool, perform additional processing, or provide a final answer to the user. This 'Reasoning-Acting-Observing' loop allows complex tasks, like 'fetch flight data, check calendar, and book a hotel,' to be broken down into discrete, manageable steps by the model itself.
What are the primary differences between zero-shot tool calling and ReAct prompting?
Zero-shot tool calling is the most direct approach where the model predicts the tool call in a single pass based on immediate context. It is fast and efficient but struggles with complex, multi-dependency tasks. In contrast, ReAct—short for Reasoning and Acting—is a framework where the model explicitly generates 'thought' traces alongside its tool calls. By requiring the model to articulate its plan before invoking a tool and then observing the outcome before deciding the next step, ReAct significantly improves performance on ambiguous queries. While zero-shot is a 'react-and-respond' mechanic, ReAct is a 'reason-then-act' methodology designed to increase reliability in reasoning-heavy environments.
How do you handle potential errors or invalid arguments when an LLM attempts to call a function?
Handling errors requires a robust middleware layer between the model and the function. When the model generates a tool call, the system should validate the arguments against the schema before execution. If the model generates a hallucinated parameter or an invalid type, the function execution will fail. The best practice is to catch these exceptions, format the error into a descriptive message, and feed it back to the model as a new message. This allows the model to 'see' the failure, correct its logic, and retry the function call with the necessary adjustments, effectively creating a self-healing loop for API interactions.
What challenges arise when scaling function calling to a large number of available tools?
Scaling to many tools creates a 'context pollution' problem. When you provide the model with a large library of function definitions, you consume significant context window space, which can degrade the model's attention accuracy and increase latency. Furthermore, when the number of tools grows, the risk of the model selecting the wrong function increases due to overlapping semantic descriptions. To mitigate this, developers should use 'tool routing,' where an agent first classifies the user query to determine which subset of tools is relevant, and then presents only that curated list to the model. This keeps the prompt concise and ensures that the model operates with high precision even in complex, enterprise-grade ecosystems.
Check yourself
1. What is the primary role of an LLM during the function calling process?
- A.Executing the underlying function code on the server
- B.Determining if a function should be called and generating valid arguments
- C.Validating the database schema of the requested tool
- D.Managing the hardware resources for the tool execution
Show answer
B. Determining if a function should be called and generating valid arguments
The LLM is responsible for deciding whether a tool can solve the user's request and producing a structured object (JSON) that fits the required schema. It cannot execute code itself, validate database schemas, or manage hardware.
2. When should you include a 'description' field for a function parameter in your schema?
- A.Only when the parameter is a complex object
- B.Only when the parameter is optional
- C.Always, to guide the model on what data to extract or provide
- D.Never, as it consumes unnecessary context window space
Show answer
C. Always, to guide the model on what data to extract or provide
Descriptions are essential for the LLM to understand what each parameter represents. Without them, the model may hallucinate values or choose incorrect parameters. Complex/optional status are important, but descriptions are universal requirements for reliable tool use.
3. After the model outputs a function call, what is the required next step in a standard agentic flow?
- A.Directly display the JSON output to the user
- B.Prompt the user to confirm the function call
- C.Execute the function, then pass the result back to the model
- D.Terminate the process and start a new session
Show answer
C. Execute the function, then pass the result back to the model
The agent loop requires the output of the function to be fed back into the model's context so it can process the result and turn it into a natural language answer. Displaying raw JSON is poor UX, and manual confirmation isn't always required.
4. Why is it important to keep function definitions concise and specific?
- A.To minimize the number of tokens used for the function schema
- B.Because models have a strict limit on the number of arguments
- C.To prevent the model from becoming distracted by irrelevant features
- D.Because long definitions cause the model to ignore user input
Show answer
A. To minimize the number of tokens used for the function schema
While keeping the model focused is good, the primary technical constraint is token budget. Large, bloated schemas eat up context window space, which can decrease the space available for conversation history and function results.
5. If an LLM consistently provides the wrong arguments for a tool, what is the best strategy to fix this?
- A.Increase the temperature to encourage more creative arguments
- B.Update the tool's parameter descriptions to be more explicit
- C.Rewrite the tool code to be more efficient
- D.Add more functions to the list to give the model more options
Show answer
B. Update the tool's parameter descriptions to be more explicit
The model's behavior is driven by the prompt and the schema descriptions. Clarifying descriptions helps the model map the user intent to the correct parameters. Increasing temperature makes output less predictable, code changes don't affect output, and more functions increase confusion.