Reasoning models and reasoning engines

Learn in your browser · Chapter 01 of 06

Some models start answering at once. Others are built to work through the problem first. The difference is where the computation goes.

A chain of thought is a series of intermediate steps written out before the final answer. In 2022 researchers showed that prompting a large language model to produce such steps improved its results on arithmetic, commonsense and symbolic reasoning tasks.

A reasoning model goes further. It is trained to produce a long chain of thought by itself, without being asked, and to use it. Published descriptions of how this is done commonly involve reinforcement learning: the model attempts many problems and is rewarded when its final answer is correct, and sometimes for sound intermediate steps. The rewarded tasks are usually ones whose answers can be checked automatically, such as maths answers or code that passes tests. Over training, models tend to learn longer working, self-checking and trying alternatives. Details differ between models and are not always published.

Test-time compute means computation spent while the model is answering, rather than while it was trained. A 2024 study asked how much a fixed amount of extra answering-time computation can improve results. It found that the best way to spend it depends on how hard the prompt is, and that on problems a smaller model could already sometimes solve, spending it well could beat a much larger model. On the most challenging problems it helped little, and a larger model did better.

Scroll sideways to see every column.

Learn in your browser · Chapter 02 of 06

A reasoning model is a tool with a price. Use it where the price buys something.

Reasoning helps most where a mistake early in the problem ruins everything after it, and where the model can check its own working: multi-step maths and logic, code with several interacting parts, debugging, and planning with constraints. It also helps when the answer can be checked, because the model can test candidate answers against the rules.

It helps little, or hurts, on simple chat, rewriting, summarising, extracting facts that are stated in the text, and lookups where knowledge is the limit. There the extra working adds delay and cost for almost no gain, and thinking cannot conjure information the model does not have.

Scroll sideways to see every column.

Cost matters. Many hosted services bill thinking tokens as output tokens, even when you see only a summary of them or nothing at all (see Sources), so a hard question can cost many times an easy one. Check your provider's pricing page for exactly how reasoning tokens are counted.

A common pattern is routing: send easy requests to a fast model, escalate hard ones to a reasoning model, or start at low effort and raise it only if a check fails.

Learn in your browser · Chapter 03 of 06

Most reasoning services give you a dial. Turn it deliberately.

Providers expose the dial differently: an on and off switch, named levels such as low, medium and high, a numeric scale, or a cap on the number of thinking tokens. Some also restrict or ignore sampling settings such as temperature in reasoning mode. Read your provider's documentation rather than assuming.

1. Try the task with reasoning off.
2. If the answer fails your check, try low effort.
3. If it still fails, try high effort.
4. Note the lowest level that passes. Use it as your default for that kind of task, and re-test when the model changes.

Prompting changes too, because the model already works step by step.

• State the goal, the constraints and the format of the answer. Leave the method to the model.
• You rarely need to say "think step by step". It is usually redundant.
• Put every fact the model needs in the prompt. Thinking cannot fetch what is missing.
• Say what success looks like: the tests to pass, the units, the rules. That gives the model something to verify against.
• Try zero-shot first. Published evaluations of at least one reasoning model found that few-shot examples reduced its performance, so test yours before adding examples.

Learn in your browser · Chapter 04 of 06

Many reasoning models show their working. It is tempting to read it as a window into the model's mind. Be careful.

A reasoning trace is the model's visible working. It is useful: it often reveals where a wrong answer went astray. But a fluent story about how an answer was reached is not the same as an accurate account of what actually drove it. This property is called faithfulness.

A 2023 study of chain-of-thought prompting found that explanations could be heavily influenced by a bias the model never mentioned, for example reordering the multiple-choice options in the prompt's examples so the right answer was always option A. When models were pushed towards a wrong answer, they often wrote convincing working that justified it. A 2025 study of reasoning models added hints to prompts and found that the traces often did not mention a hint the model had used.

• Read traces to find errors, not to certify correctness.
• A trace can contain mistakes when the answer is right, and look flawless when the answer is wrong.
• Some services show only a summary of the working, or none at all. Do not build a process that depends on seeing it.
• For audit and accountability, log inputs, outputs and the checks performed, not just the model's story.

Learn in your browser · Chapter 05 of 06

The most reliable way to use a reasoning model is to pair it with something that checks.

A verifier is anything that judges whether an answer is acceptable. In 2021 researchers trained verifiers to judge model solutions to maths problems, then generated many candidate solutions and picked the one the verifier ranked highest. A related technique, self-consistency, samples many reasoning paths and takes the most common answer. A 2023 study compared feedback on each step (process supervision) with feedback on the final result only (outcome supervision).

Scroll sideways to see every column.

def solve_with_check(task, max_tries=3):
    prompt = task
    for attempt in range(max_tries):
        effort = "low" if attempt == 0 else "high"
        answer = ask_model(prompt, effort=effort)
        ok, problems = check(answer)   # tests, schema, arithmetic, rules
        if ok:
            return answer
        prompt = (task + "\nYour last answer was:\n" + answer +
                  "\nA check failed: " + problems + " Fix it.")
    return None                        # hand over to a person

Learn in your browser · Chapter 06 of 06

Long before language models, an inference engine or reasoning engine meant something quite different.

In classical symbolic AI, a knowledge base holds facts and rules written in a precise form, and an inference engine applies the rules to derive new facts. Expert systems built this way encoded specialist knowledge, such as diagnostic rules, and could typically show which rules had produced a conclusion. This is a different meaning from the inference engine in the workbook Inference engines explained, which is software that executes a neural network.

Two strategies are classic. Forward chaining starts from the known facts and fires every rule that applies until nothing new appears. Backward chaining starts from a goal and works back to see whether known facts can prove it.

Facts:  penguin(p1)
Rules:  IF penguin(X) THEN bird(X)
        IF bird(X) THEN has_wings(X)
Query:  has_wings(p1)?

Forward:  penguin(p1) gives bird(p1), which gives has_wings(p1). Yes.
Backward: to show has_wings(p1) I need bird(p1).
          To show bird(p1) I need penguin(p1). It is a fact. Yes.

Scroll sideways to see every column.

Neuro-symbolic systems combine the two. Surveys describe fusing neural, symbolic and probabilistic approaches to improve interpretability, robustness and trustworthiness. Four practical patterns stand out.

• The model reads messy input and produces structured facts. A rule engine makes the decision.
• The model writes code or logic, and an interpreter or solver does the exact computation.
• A rule engine or validator checks the model's output before it is used.
• The model proposes and a formal checker verifies, which is the verifier pattern from the last chapter.