How Does an LLM Use Tokens to Decide When to Call External Tools Like a Calculator or Python Code?

Here is a question that reveals how tool use really works: if a model can only generate tokens, how does it run a calculator or execute Python? The answer is that it never runs anything directly. It only ever does the one thing it can do, which is produce tokens, and some of those tokens are a signal the application knows how to act on. Understanding this removes almost all the mystery from tool use.

The model only ever emits tokens

Everything a model produces is a stream of tokens, whether it is prose, code, or a tool request. It has no hands and no direct access to your computer, so it cannot literally call a function. What it can do is generate a specific, structured piece of text that means please run this tool with these inputs. That text is made of the same tokens as anything else it writes. The trick is entirely in how the application interprets those particular tokens.

A tool call is structured output

When a model wants a tool, it emits the request in a structured format the application can parse, typically naming the tool and its arguments. Instead of writing four times seven is twenty-eight, it might output a small structured block that names a calculator tool with the inputs four and seven. The application is watching the model’s output for exactly this shape. When it sees a valid tool request, it stops treating the text as a message to show you and starts treating it as an instruction to execute. The format is the whole signal.

The application detects and runs it

Once the application spots a tool call in the token stream, it takes over. It pauses generation, extracts the tool name and inputs, runs the actual calculator or code, and captures the result. None of that execution happens inside the model, which is exactly why it is safe to give the model a calculator without trusting it to do arithmetic itself. The generation loop is briefly interrupted so the real work can happen outside the model. The model asked in tokens, and the app answered in the real world.

The result comes back as tokens

After the tool runs, its output is turned back into text and inserted into the model’s context. The model then resumes, now able to read the tool’s result as part of its input, and continues generating from there. So the calculator’s answer enters the model the same way your original question did, as tokens in the context. This is the same mechanism by which conversation history is fed back to a stateless model. The whole exchange is tokens out, then tokens back in.

How the model decides to call at all

The deeper question is why the model chooses to emit a tool call rather than just answer. It learned during training to recognize situations where a tool would help, and it is told at runtime which tools exist through their descriptions. When the current context looks like a job for a calculator or code runner, the pattern that produces a tool call becomes the most likely next output. So the decision is the same next-token prediction as everything else, just shaped toward a tool request. The model is not consulting a rulebook, it is predicting that calling a tool is the right move.

Tool descriptions steer the choice

Because the decision is learned and context-driven, the descriptions you provide matter enormously. A clear description of when and how to use a tool makes the model reach for it at the right moments, while a vague one leads to missed or misused calls. This is why guides to building with tools and function calling put so much weight on precise tool definitions. You are effectively programming the model’s judgment through natural language. Better descriptions produce better decisions.

Why offload to a tool at all

It might seem odd to make a powerful model defer to a humble calculator, but it is exactly the right design. Models are unreliable at exact arithmetic and cannot know today’s data, so offloading those jobs to a deterministic tool produces correct results instead of confident guesses. The model is great at deciding that a calculation is needed and terrible at doing it precisely, so it delegates. This division of labor, judgment in the model and precision in the tool, is what makes the combination trustworthy. Use each part for what it is actually good at.

Chaining tools across steps

Real tasks often need several tool calls in sequence, and the token mechanism handles that naturally. The model can call a tool, read the result, decide a second tool is now needed, and emit another call, building up a solution step by step. Each result reshapes the context and therefore the model’s next prediction, so the sequence adapts as it goes. This ability to chain tool calls is what lets agents tackle genuinely complex, multi-step work. One tool call is useful, but a chain of them is where real capability appears.

Why this design is safe and flexible

Keeping execution outside the model has real advantages. The application can validate a tool request before running it, refuse dangerous ones, and log everything, all because the call is just text it inspects first. It also means new tools can be added without retraining the model, since the model only needs their descriptions. This clean separation between deciding and doing is part of why the whole approach scales so well. The model proposes in tokens, and the trusted application disposes.

The mental model to keep

So picture tool use as a conversation in tokens. The model writes call this tool, the application reads that, runs the tool, and writes the result back, and the model reads on. There is no magic reach-out, only structured text that the application has agreed to act on. Hold that picture and function calling, tool use, and agent behavior all stop being mysterious. It is tokens all the way down, with a trusted app in the middle doing the real work.

Common questions