What Is LLM Reasoning, and Why Does “Thinking Step by Step” Produce Better Outputs?

One of the strangest and most useful facts about language models is that they get better at hard problems when you simply ask them to think out loud. Give a model a tricky question and it may blurt a wrong answer, but ask it to work through the steps first and it often lands correctly. This is the heart of what people mean by LLM reasoning. It looks like a trick, but it reflects something real about how these models actually compute an answer.

Reasoning is thinking on the page

For a language model, reasoning is not a hidden faculty, it is text the model writes before its final answer. When it reasons, it produces a chain of intermediate steps, working through a problem in the open rather than leaping to a conclusion. Those steps are generated the same way as any other output, one piece at a time. The visible working is not a report of some separate thought process, it is the thinking itself. The model reasons by writing, not by writing up reasoning it did elsewhere.

Why writing the steps helps

The reason this works comes down to computation. A model does a fixed amount of work per token it generates, so forcing a hard answer into a single step gives it very little room to compute. Letting it produce many intermediate tokens gives it far more total computation to spend on the problem. Each step it writes also becomes part of the context it reads for the next step, so it can build on its own partial work. More visible steps means more thinking room and more chances to catch itself, which is exactly why the answers improve.

Thinking step by step in a prompt

The simplest way to trigger this is to ask for it directly. Adding a phrase like work through this step by step nudges the model to lay out its reasoning before committing to an answer. This technique, often called chain-of-thought prompting, reliably improves performance on math, logic, and multi-step coding problems. It costs almost nothing to try and frequently turns a wrong answer into a right one. When a model fumbles a hard question, asking it to slow down and show its work is the first thing worth reaching for.

Reasoning models bake it in

Newer models take this further by being trained to reason extensively on their own. So-called reasoning models will spend a long internal stretch working through a problem before they answer, without you having to ask. They have learned that generating a thorough chain of thought produces better results, so they do it by default on hard tasks. This is why some models pause noticeably before responding to a difficult prompt. The step-by-step habit that used to be a prompting trick has become a built-in capability.

It is still prediction underneath

It is worth staying grounded about what is happening here. The reasoning steps are still generated by the same next-token prediction that drives everything an LLM produces. There is no separate logic engine switching on, only the model conditioning each step on the ones before it. That is powerful, because good reasoning patterns learned from training genuinely help, but it also means the reasoning can look sound and still be wrong. The appearance of careful thought is not a guarantee of a correct conclusion.

When it helps and when it wastes effort

Step-by-step reasoning is not free and not always worth it. On genuinely hard, multi-step problems it can be the difference between success and failure, and it earns its cost. On simple lookups or trivial tasks it just burns extra tokens and time for no benefit, and can even talk the model into overcomplicating something easy. The skill is matching the amount of reasoning to the difficulty of the task. Reserve the deep thinking for problems that actually have depth.

Reasoning can be confidently wrong

A chain of reasoning that reads beautifully can still reach a false conclusion. The model can make an early mistake and then reason fluently from it, producing a convincing but flawed argument. This is why visible reasoning is helpful for you as a reviewer but is never a substitute for verification. The same discipline of tests and review applies to a reasoned answer as to any other. Read the steps to catch errors, but confirm the result rather than trusting the confidence.

Reasoning powers agent planning

This matters enormously for agents, because planning is reasoning applied to actions. When a coding agent decides how to tackle a task, break it into steps, and choose which tool to use, it is reasoning its way through the problem. The quality of that internal reasoning largely determines how well the agent performs across a long run. Better reasoning means better plans, fewer dead ends, and less wasted work. It is the difference between an agent that flails and one that makes steady progress.

How to prompt for better reasoning

You can actively help a model reason well. Break a big request into clear parts, ask it to plan before it writes code, and give it room to lay out its approach rather than demanding an instant answer. Providing the relevant context up front also helps, since the model reasons better when it is not guessing at missing facts. Small changes to how you ask can produce a large change in the quality of the thinking you get back.

The cost of thinking

All this reasoning is made of tokens, and tokens cost time and money. A model that thinks extensively before answering is slower and more expensive than one that replies immediately, which is a real tradeoff on high-volume work. For a one-off hard problem the cost is trivial and worth it, while for millions of simple calls it adds up fast. Deciding when to pay for deep reasoning is part of using these models economically. Think of thinking as a dial you turn up for hard problems and down for easy ones.

The takeaway

Reasoning turns a language model from a fast guesser into a slower, more careful problem-solver, simply by giving it room to work. Ask for steps on hard problems, expect it on modern reasoning models, and always verify the conclusion rather than trusting the polish. Used well, thinking step by step is one of the cheapest and most reliable ways to get more out of any model.

Common questions