Function Calling and Tool Use: Connecting AI to APIs

Function calling lets a model emit structured tool calls instead of plain text. Your runtime executes the call, hands the result back, and the model continues. It is the productionised, schema-enforced cousin of ReAct — and the foundation of every serious AI integration shipping today.

1. Introduction

Modern AI work falls into two halves. The first half is generating text. The second half is making things happen — sending an email, querying a database, charging a card, looking up a flight, updating a record. Plain text outputs cannot do those things. They can describe what the user wants, but a human or another piece of code has to translate that description into an actual API call.

Function calling closes that gap. The model produces a structured object — typically JSON — that names the function it wants to call and supplies the arguments. Your runtime parses that object, executes the corresponding code, and feeds the result back so the model can continue the conversation. The model never touches your APIs directly; your code does. The model just picks which tool to use and with what arguments.

2. The Concept Explained

A function-calling system has three moving parts: a tool catalog (JSON Schema definitions of every function the model is allowed to call), a model that supports function calling (every major provider does today), and a runtime loop that detects when the model wants to call a tool, executes it, and continues the conversation.

Think of it as hiring an assistant and giving them a small set of well-labelled forms to fill in. They cannot do anything you have not given them a form for. When they fill in a form, you process it. The forms are the schema; the assistant filling them in is the model; you executing the form is the runtime.

3. The Problem Without Function Calling

You are a CRM assistant. The user might ask you to look
up customers or create notes. If they do, respond with
something like:
LOOKUP_CUSTOMER name=John Doe
CREATE_NOTE customer_id=12345 text="..."

Use exactly these formats.

4. The Solution: Schema-Defined Tools

// Tool catalog passed to the API alongside the prompt
[
  {
    "name": "lookup_customer",
    "description": "Find a customer by name or email.
                    Returns id, name, email, plan, status.",
    "parameters": {
      "type": "object",
      "properties": {
        "name":  { "type": "string", "description": "Full name" },
        "email": { "type": "string", "description": "Email address" }
      },
      "anyOf": [ {"required":["name"]}, {"required":["email"]} ]
    }
  },
  {
    "name": "create_note",
    "description": "Attach a note to a customer record.",
    "parameters": {
      "type": "object",
      "properties": {
        "customer_id": { "type": "string" },
        "text":        { "type": "string", "minLength": 1, "maxLength": 2000 }
      },
      "required": ["customer_id", "text"]
    }
  }
]

// System prompt
You are a CRM assistant. Use the provided tools to look
up customers and create notes. Never invent customer ids
— always look up first, then act on the returned id. If
unsure, ask the user before calling a tool that modifies
data.

5. Step-by-Step Breakdown

1. Design a small tool surface. Three to eight tools per assistant is usually right. More than that and the model starts mis-routing — picking the wrong tool for the job.
2. Write rich descriptions. The description field is what the model reads to decide when to call a tool. Treat it as marketing copy aimed at the model — what does it do, when should it be used, what are the constraints?
3. Make schemas precise. Use enums, min/max lengths, required fields. The tighter the schema, the more reliable the calls. Loose schemas invite the model to invent arguments.
4. Separate read tools from write tools. Reads are safe to retry; writes are not. Mark destructive tools clearly and require user confirmation in the UI before executing them — never trust the model to confirm.
5. Validate before executing. Even with schema-typed arguments, run business-rule checks: is the customer in this user's tenant? Is the amount under the daily limit? Treat every tool call as untrusted input.
6. Handle tool errors as data. When a tool fails, return a structured error in the tool result rather than throwing. The model can read the error and recover — "Customer not found, try a different name" — which is dramatically more useful than crashing the conversation.
7. Cap the loop. Set a maximum number of consecutive tool calls (5–10 is normal). Without a cap, a confused model can keep calling tools forever and rack up costs.

Tip: Tool descriptions are prompts in disguise. The same model that needs careful prompt engineering also needs careful tool description engineering. Spend time on these — they are the contract between the model and your APIs, and they pay back forever.

6. Practice Exercises

7. Key Takeaways