The loop harness: how AI agents actually run

Learn in your browser · Chapter 01 of 06

On its own, a language model can only produce text in response to its input. It cannot open a file, search the web or send an email. Something has to sit around it and do those things on its behalf. That something is the harness.

A loop harness (also called an agent harness) is the program that runs a model in a loop. It is ordinary software, written by people, wrapped around a model. Together they make an agent: a system that pursues a goal over many steps by choosing an action, seeing what happened and choosing again.

In plain words, the harness builds what the model sees and calls it. If the model asks for a tool, the harness checks the request, runs the tool and adds the result to the conversation. Then it goes round again until a stop condition is met.

Scroll sideways to see every column.

Two agents on the same model can behave very differently: one has sensible limits and a good log, the other runs up a bill or changes the wrong file. Published research has explored interleaving a model's reasoning with actions in the outside world (see Sources).

Learn in your browser · Chapter 02 of 06

Read the loop first as a story with six beats, then as code.

1. Assemble. Build the full prompt: system prompt, tool definitions, the task and the conversation so far.
2. Call. Send it to the model. The reply is either plain text or one or more tool requests.
3. Parse. Read the reply. A tool request names a tool and gives its arguments, usually as structured data.
4. Check and run. Confirm the tool exists, the arguments are valid and the action is allowed. Ask a human if the rules say so. Then run it.
5. Feed back. Add the tool's result, or its error, to the conversation as a new message.
6. Repeat or stop. Check the budgets and stop conditions. If none has triggered, go round again.

Here is a minimal loop in illustrative Python. It is vendor-neutral: client.chat stands for whichever model access you use.

# Illustrative pseudocode: every helper is a placeholder you would write.
import time

def run_agent(task, client, tools, limits):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": task},
    ]
    steps, tokens_used = 0, 0
    started = time.monotonic()
    recent = []

    while True:
        # 1. Check budgets BEFORE spending anything.
        if steps >= limits.max_steps:
            return finish("step_limit", messages)
        if tokens_used >= limits.max_tokens:
            return finish("token_limit", messages)
        if time.monotonic() - started >= limits.max_seconds:
            return finish("time_limit", messages)

        # 2. Ask the model what to do next.
        reply = client.chat(messages=messages, tools=describe(tools))
        steps += 1
        tokens_used += reply.usage_tokens
        messages.append(reply.as_message())
        audit_log("model_reply", steps, reply)

        # 3. No tool request: the model believes it is done.
        if not reply.tool_calls:
            return finish("final_answer", messages, reply.text)

        # 4. Run each requested tool through the safety checks.
        for call in reply.tool_calls:
            signature = (call.name, str(call.arguments))
            recent.append(signature)
            if recent[-3:].count(signature) == 3:
                return finish("stuck_repeating", messages)
            result = run_tool_safely(call, tools)
            audit_log("tool_result", steps, call, result)
            messages.append(tool_message(call.id, result))

        # 5. Keep the context inside its limit.
        messages = compact_if_needed(messages, limits.max_context_tokens)

def run_tool_safely(call, tools):
    tool = tools.get(call.name)
    if tool is None:
        return "error: no such tool"
    problem = check_arguments(call.arguments, tool.schema)
    if problem:
        return "error: bad arguments: " + problem
    if tool.needs_approval and not ask_human(call):
        return "error: a human declined this action"
    try:
        return tool.run(call.arguments, timeout_seconds=tool.timeout)
    except Exception as exc:
        return "error: " + str(exc)[:300]  # short; never put secrets in errors

Notice what the model never does: it never runs anything and never sets its own limits. A failing tool does not crash the loop, because run_tool_safely turns problems into short error text the model can react to. And every planned exit goes through finish, which records why the run ended.

Task: What do the three invoices in the folder "march" come to?

Turn 1  model   -> tool call: list_files(folder="march")
        harness -> ["inv1.txt", "inv2.txt", "inv3.txt"]
Turn 2  model   -> tool call: read_file(path="march/inv1.txt")
        harness -> "Total due: 120.00 GBP"
Turn 3  model   -> tool call: read_file(path="march/inv2.txt")
        harness -> "Total due: 75.50 GBP"
Turn 4  model   -> tool call: read_file(path="march/inv3.txt")
        harness -> "Total due: 42.25 GBP"
Turn 5  model   -> text: "The invoices come to 237.75 GBP."
        (no tool call, so the loop ends)

Learn in your browser · Chapter 03 of 06

The model is stateless. Each call starts from nothing, so on every turn the harness must rebuild everything the model needs to know. Getting that assembly right is most of the craft.

The system prompt is the standing set of instructions at the start of every call. A good one states the role, the goal, the rules, when to ask for help and when to stop. Keep it short, specific and testable. If you cannot imagine a test that would show a rule was broken, the rule is too vague.

Tool definitions tell the model what it may ask for. Each has a name, a plain description of what it does and when to use it, and a schema of its arguments. The model knows a tool only through this description, so a vague one produces wrong calls. Prefer a few narrow tools such as search_orders and read_file over one broad tool such as run_command that can do anything.

Context assembly builds the prompt for each call from the system prompt, the tool definitions, the task, the conversation so far and any saved notes. Memory comes in two kinds. Short-term memory is the conversation held in the context window. Long-term memory is anything saved outside it, such as notes in a file, and loaded back only when relevant. The model remembers nothing by itself. The harness decides what is saved and reloaded.

Every turn adds to the conversation, and every call pays again, in cost and delay, for all of it. Sooner or later the history nears the context limit. Compaction is how the harness copes: it replaces older turns with a short summary and keeps recent turns word for word.

• Pin the system prompt, the task and any hard constraints so they are never summarised away.
• Trim bulky tool results. Keep the useful part, or save the full output to a file and give the model only a reference.
• Test after compaction. A summary can silently drop a detail or a rule, so try tasks that depend on early information.

Learn in your browser · Chapter 04 of 06

A harness earns its keep by saying no. These controls decide what an agent may do, how much it may spend and when it must stop.

Permissions. Give the agent the least power it needs (the principle of least privilege) and sort every tool by the damage it could do.

Scroll sideways to see every column.

A good approval request shows the exact action in plain words, such as the recipient and full text of the email, not just 'Allow?'. Save approvals for actions that matter. If people must approve everything, they soon click yes without reading.

Sandboxing. Run tools that touch files, code or the network inside a boundary: a limited folder, no network unless needed, no secrets in the environment, capped memory and time, and a disposable environment. If the model writes code, it runs inside the sandbox. Never pass model output to eval.

Budgets. Set four kinds of limit and check them before every call, not after.

Scroll sideways to see every column.

When a budget runs out, stop and report what was done and what is left. Do not carry on quietly.

Stop conditions. A run should end for a named reason: a final answer, a passed check (such as a test now succeeding), a spent budget, an agent stuck repeating itself, a person pressing stop, an unrecoverable error, or an attempt to break policy. Always return the reason.

Errors and retries. Send tool errors back to the model as short text so it can adapt. Retry only failures likely to be temporary, such as a timeout or a rate limit, waiting a little longer each time with a small random variation (backoff with jitter) and giving up after a few attempts. Never retry permanent errors, and never blindly retry an action that changes the world, such as taking a payment. Use an idempotency key, or check first whether it already happened.

Learn in your browser · Chapter 05 of 06

You will not watch every step. Logs, tests and defences against hostile text are how you keep trust in a system that acts on its own.

Logging and the audit trail. A log is how you find out afterwards what an agent did and why. For every step record a run identifier, a timestamp, the prompt version, each model reply, each tool request with its arguments, each approval or refusal, each result, token counts and the stop reason. Redact secrets and personal data before writing.

A log you can rely on needs tamper-evidence. In a tamper-evident log, each entry carries a fingerprint (a hash) worked out from its own content plus the previous entry's fingerprint. Change or delete an old entry and every later fingerprint stops matching, so the change is detectable. This does not stop anyone editing the file. It makes an edit visible, provided you keep the latest fingerprint somewhere the editor cannot reach.

import hashlib, json

def fingerprint(previous, event):
    body = json.dumps(event, sort_keys=True)
    return hashlib.sha256((previous + body).encode("utf-8")).hexdigest()

def append_entry(log, event):
    previous = log[-1]["hash"] if log else "start"
    log.append({"event": event, "hash": fingerprint(previous, event)})

def chain_is_intact(log, kept_hash):
    # kept_hash: the latest fingerprint, stored where the editor cannot reach
    previous = "start"
    for entry in log:
        if entry["hash"] != fingerprint(previous, entry["event"]):
            return False
        previous = entry["hash"]
    return previous == kept_hash

Evaluation and testing. Agents vary from run to run, so test them like a process, not a single function.

• Write scenarios that check the end result (a file changed, a record created), not exact wording. Run each several times and record how often it passes, its steps and its cost.
• Include scenarios that go wrong on purpose: a failing tool, a tool returning nonsense, hostile text.
• Test the harness with a fake model that returns scripted replies. It is fast, free and repeatable, and it checks that budgets, stops and approvals work.
• Re-run everything whenever you change the prompt, a tool or the model.

Prompt injection. A model reads instructions and data through the same channel: text. So when it reads a web page, email or tool result that contains instructions, it may follow them as if they came from you. That is prompt injection. It is direct when a user types it and indirect when it hides in content the agent fetches while working.

The UK National Cyber Security Centre advises that prompt injection is a risk to reduce by design rather than one that can be fully removed, and OWASP lists it first among its top ten risks for language model applications (see Sources). So build in layers.

• Treat everything the agent reads from outside as untrusted data, and label it so in the prompt.
• Use least privilege, so that a hijacked agent can do little.
• Be wary of any agent that holds private data, reads untrusted content and can send information out. Split those powers across agents, or add approvals.
• Check tool arguments in code with allow-lists, such as permitted recipients or web domains.
• Log everything, so an attack can be seen and studied afterwards.

Learn in your browser · Chapter 06 of 06

The best way to make the parts stick is to design a whole harness for a real task before writing any code.

Pick a real task from your own life or work: triaging a shared inbox, updating a stock sheet from supplier emails, or researching a topic and producing a briefing with sources. Then work through this sheet.

HARNESS DESIGN SHEET
1. Task, and the test for "done and correct"
2. Tools: name, purpose, read only or not
3. Permissions: free, or needs a human
4. Sandbox: what the agent can reach
5. Context plan: pinned, trimmed, saved
6. Budgets: steps, tokens, time, money
7. Stop conditions and error rules
8. Log: recorded, redacted, fingerprint kept where
9. Tests: five scenarios, two that go wrong
10. Injection: where untrusted text enters

Keep your sheet: when you build, it becomes your specification, test plan and safety case. To go further, read AI agents explained, RAG and knowledge bases, and Reasoning models.