How to Use Tool Calling With a Local LLM in Ollama

A local model that can only talk is half a system. Tool calling is what lets it check today's weather, query your database, or run a calculation instead of guessing, and Ollama supports it natively over the same chat endpoint you already use. The short version: pass your functions in the tools list to ollama.chat, run whatever the model asks for, then send the result back as a message with role: "tool" so it can finish the answer.

Key Takeaways

The short answer: give Ollama a list of functions in the tools parameter, execute the tool_calls it returns, and append each result as a role: "tool" message before asking for the final response.
The Python library (0.4+) reads your type hints and docstring to build the JSON schema, so you pass the function itself, not a hand-written spec.
The model never runs your code. It returns a name and arguments; your program decides what to execute.
Tool calling needs a tool-capable model such as Llama 3.1, Qwen 3, or Mistral. A base model without that training will ignore the tools.
It works with streaming too, so you can show tokens live and still catch tool calls mid-stream.

What You Need Before You Start

Three things, all free and local.

Ollama installed and running. If ollama --version prints a version, the daemon is serving on http://localhost:11434. New to this? Start with running LLMs locally with Ollama.
A tool-capable model pulled locally, for example ollama pull llama3.1.
The Python client: pip install ollama (version 0.4 or later for function-as-tool support). You can do everything with plain curl too; both are shown below.

Wire a Tool Into a Local Model in Five Steps

This is the whole loop. Define a function, hand it to the model, run what it asks for, and return the result. You can finish the task from this section alone.

Write a normal Python function with a clear docstring. The library turns your type hints and docstring into the JSON schema the model sees, so the description matters as much as the code.

def get_current_weather(city: str) -> str:
    """
    Get the current weather for a city.

    Args:
      city: The city name, for example "Tokyo"

    Returns:
      A short weather description.
    """
    # Replace with a real API call in production
    return f"18 degrees celsius and clear in {city}"

Pass the function directly to ollama.chat. No separate schema. The model decides whether the request needs the tool.

import ollama

messages = [{'role': 'user', 'content': 'What is the weather in Tokyo?'}]

response = ollama.chat(
    model='llama3.1',
    messages=messages,
    tools=[get_current_weather],
)

Read the tool calls the model returns. When it wants a tool, response.message.content is usually empty and response.message.tool_calls holds the name and parsed arguments.

available_functions = {'get_current_weather': get_current_weather}

for tool in response.message.tool_calls or []:
    fn = available_functions.get(tool.function.name)
    if fn:
        result = fn(**tool.function.arguments)
        print('Tool output:', result)

Append the assistant turn and each tool result to the conversation. The result goes back as a message with role: "tool" and the tool's name, so the model can tie it to its request.
```
    messages.append(response.message)
    messages.append({
        'role': 'tool',
        'name': tool.function.name,
        'content': str(result),
    })
```

Call the model again to get the grounded answer. With the tool output in context, it now writes a normal reply instead of another tool call.

final = ollama.chat(model='llama3.1', messages=messages)
print(final.message.content)
# -> "It's currently 18 degrees celsius and clear in Tokyo."

What the Raw API Looks Like

If you are calling the endpoint directly instead of through the Python client, you supply the JSON schema yourself. This is the same request under the hood.

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1",
  "stream": false,
  "messages": [{"role": "user", "content": "What is the weather in Tokyo?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_current_weather",
      "description": "Get the current weather for a city",
      "parameters": {
        "type": "object",
        "properties": {
          "city": {"type": "string", "description": "The city name"}
        },
        "required": ["city"]
      }
    }
  }]
}'

The response mirrors the OpenAI shape. The assistant message carries a tool_calls array, each entry with function.name and a parsed function.arguments object. You send the result back as another message: {"role": "tool", "content": "...", "tool_name": "get_current_weather"}.

Which Local Models Actually Support Tools?

Tool calling is a trained behavior, not a wrapper you can bolt onto any checkpoint. Pull one of these, or filter the library by the tool tag before you download.

Model	Pull command	Good for
Llama 3.1 (8B)	`ollama pull llama3.1`	General-purpose default, reliable tool use on modest hardware
Qwen 3 (8B)	`ollama pull qwen3`	Strong reasoning and multi-tool calls; supports streaming tools
Mistral (7B)	`ollama pull mistral`	Lightweight, fast, single-tool workflows
Qwen2.5-coder	`ollama pull qwen2.5-coder`	Code-heavy agents that call shells, linters, or test runners

The full, current list lives behind the tool capability filter on Ollama's model search. A base or instruct model without tool training will simply answer in prose and never populate tool_calls.

How Do You Handle Tool Calls While Streaming?

You do not have to choose between live tokens and tools. Set stream=True and Ollama interleaves content chunks with tool-call chunks, parsing the call tokens out as they arrive.

from ollama import chat

stream = chat(
    model='qwen3',
    messages=[{'role': 'user', 'content': 'What is the weather in Toronto?'}],
    tools=[get_current_weather],
    stream=True,
)

for chunk in stream:
    if chunk.message.content:
        print(chunk.message.content, end='', flush=True)
    if chunk.message.tool_calls:
        for tool in chunk.message.tool_calls:
            print('\nCall:', tool.function.name, tool.function.arguments)

Collect the calls as they stream, run them after the chunk that contains them, and continue the same append-and-recall loop from the steps above.

Troubleshooting Common Failures

Most tool-calling problems trace back to one of four causes.

The model answers in text and tool_calls is always empty. The model is not tool-capable, or your function description is too vague for it to know when to call. Switch to a tool-tagged model and rewrite the docstring to state exactly what the tool does and when to use it.
Arguments come back malformed or with the wrong keys. Tighten the schema. Mark fields required, name them the way you name them in the function signature, and describe each one. Vague parameter names produce vague guesses.
The model calls the same tool in a loop. You likely never appended the role: "tool" result before calling again, so the model still thinks the request is unanswered. Always add both the assistant message and the tool result to messages before the next call.
Nothing responds and the client hangs. Confirm the daemon is up with curl http://localhost:11434/api/tags. If that fails, start it with ollama serve and pull the model again.

Treat every argument the model hands you as untrusted input. It chose the values, so validate types, clamp ranges, and never pass them straight into a shell, a filesystem path, or a SQL string.

What to Do Next

The single tool above is the primitive. Real systems chain it.

Give the model several functions at once and let it pick, then loop until it stops calling tools. That loop is the core of an agent; building a stateful agent with LangGraph shows how to add memory and control flow around it.
When you need the model to return data rather than call a function, use structured outputs instead. Tools act; structured outputs describe.
Expose your tools to any client over a standard protocol by wrapping them in an MCP server with FastMCP, so the same functions serve Ollama, Claude, and your own apps.
Read the request and response fields in full in the Ollama API docs and the Python client repository.