Tool Use & Function Calling

Beyond Text Generation: Activating LLMs

Standard LLMs are confined to outputting text tokens inside a chat box. They cannot check live weather, query SQL databases, or book flights on their own.

Tool Use & Function Calling transform static language models into Actionable Agents.

  USER QUERY: "What is the weather in Paris?"
        │
        ▼
  [ LLM GENERATOR ] ──► Detects Tool Need! Outputs JSON Function Call:
                        {"name": "get_weather", "arguments": {"city": "Paris"}}
        │
        ▼
  [ EXTERNAL APP RUNTIME ] ──► Executes get_weather("Paris") API call ──► Returns {"temp": "22C"}
        │
        ▼
  [ LLM GENERATOR ] ◄── Appends API result to context ──► Generates Final Text: "The weather in Paris is 22C."

Notice that the LLM does NOT execute the code itself. It acts as an Intelligent Planner, outputting structured JSON payload requests for an external application runtime to execute.

Step-by-Step Function Calling Flow

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ STEP 1: DEFINE TOOLS     │ STEP 2: LLM CALL         │ STEP 3: API EXECUTION    │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Define tools using JSON  │ LLM inspects tools and   │ Application executes     │
│ Schema (name, desc,      │ outputs tool_calls JSON  │ function, feeds result   │
│ parameter types).        │ with extracted arguments.│ back as 'tool' role msg. │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

Step 1: Tool Definition (JSON Schema)

Pass a list of available tools inside the API payload:

{
  "name": "get_stock_price",
  "description": "Retrieves real time stock ticker price.",
  "parameters": {
    "type": "object",
    "properties": {
      "ticker": { "type": "string", "description": "Stock symbol (e.g. AAPL)" }
    },
    "required": ["ticker"]
  }
}

Step 2: Model Tool Selection & Argument Extraction

The LLM reads the user prompt ("What is Apple stock trading at?") alongside available tool schemas, detecting that get_stock_price should be called:

{
  "tool_calls": [
    {
      "id": "call_9842",
      "function": { "name": "get_stock_price", "arguments": "{\"ticker\": \"AAPL\"}" }
    }
  ]
}

Step 3: Local Execution & Final Synthesis

Your Python backend parses the JSON, executes fetch_stock("AAPL") $\to$ gets $220.50, and appends a tool role message to the message history:

messages.append({
    "role": "tool",
    "tool_call_id": "call_9842",
    "content": "{\"price\": 220.50}"
})

Pass the updated message history back to the LLM. The model reads the API result and outputs a friendly natural language response: "Apple stock is currently trading at $220.50."

Best Practices for Tool Engineering

1. Write Descriptive Tool Descriptions: The LLM uses tool descriptions to decide which tool to pick. Clear descriptions prevent incorrect tool selection.
2. Handle Invalid JSON via Constrained Decoding: Ensure 100 percent valid JSON parameter parsing using JSON schema constrained decoding (Outlines or Instructor).
3. Handle API Execution Errors Gracefully: If a tool call fails (e.g. HTTP 500), feed the error message back as a tool response content string so the LLM can explain the failure or retry with alternative arguments.

Say this out loud

Function Calling allows LLMs to interact with external APIs, databases, and code environments. The LLM does not execute code directly; it outputs structured JSON payloads specifying function names and argument parameters defined via JSON Schemas. An external application runtime executes the call and returns the result back to the LLM context for final response synthesis.

Followups to expect

1. What is Parallel Function Calling? Generating multiple independent tool calls in a single response step (e.g. calling get_weather("Paris") and get_weather("London") simultaneously) to execute API requests in parallel.
2. What is Model Context Protocol (MCP - Anthropic 2024)? An open standard protocol that unifies how applications expose data sources, tools, and prompts to LLM client applications.