Running AI Agents Safely in Production

Learn in your browser · Chapter 01 of 09

A demo agent proves an idea works once, with you watching. A production agent has to keep working, unattended, against real systems and real people, carrying whatever access you gave it. This chapter sets out that shift and the five boundaries the rest of this course builds, one at a time.

A demo usually runs on a laptop, uses one developer's own broad credentials, touches a handful of fake or low-stakes records, and stops the moment the terminal window closes. Production is the opposite on every count: the agent runs repeatedly and unattended, it needs its own scoped identity rather than borrowing yours, it reaches real customer data and real money, and it keeps running long after you have stopped watching it. None of that is a reason to avoid shipping agents. It is a reason to design deliberately for what changes.

Five boundaries recur across nearly every serious agent incident: identity (who is the agent, provably, and can that be revoked on its own), tools (exactly which actions it may take, and with what limits), network and data (what it can reach, and what it can read or write), cost and time (what stops it if something goes wrong and it keeps trying), and visibility (whether anyone can reconstruct what it did afterwards). Chapters 2 to 9 take these in turn, plus how you decide an agent is ready to ship and how you recover when something still goes wrong.

1. Identity: could this agent's access be revoked on its own, without touching anyone else's account?
2. Tools: does it hold any permission its actual task does not need, kept just in case?
3. Network and data: can its runtime reach anything beyond the specific systems and data it was built to use?
4. Cost and time: what concretely stops it if it starts calling a tool in a loop?
5. Visibility: could you reconstruct exactly what it did and decided last week, from the logs alone?

The rest of this course follows one invented example throughout: Thornfield Logistics, a fictional parcel courier, and Wayfinder, the AI agent it built to handle customer delivery queries, reschedule deliveries and issue small goodwill credits. Every name, figure and log line attached to Thornfield and Wayfinder is invented for teaching. Read them as a template to hold your own agent against, not as a real company's data.

Scroll sideways to see every column.

Learn in your browser · Chapter 02 of 09

An agent that runs as whoever happened to be logged in cannot be scoped, logged or revoked on its own. Give it an identity of its own before you give it anything to hold.

A service identity is a distinct identity issued to the agent itself, separate from any human user and separate from any other agent or service. It matters because access, logs and revocation all attach to an identity: if Wayfinder shares an engineer's personal token, disabling Wayfinder means disabling that engineer, and every action Wayfinder ever took shows up in that person's own audit trail, indistinguishable from their own work.

Secrets are the credentials, keys and tokens an identity uses to prove itself and reach other systems. The house rule is simple to state and easy to get wrong under deadline pressure: a secret lives in a vault or a short-lived token issuer, never in a prompt sent to a model, never committed to a script, and never printed to a log in plain form. A model that has read a secret in its context can repeat it in an output; a log that stored one in plain form is a second copy sitting wherever that log is stored, backed up and, eventually, forgotten about.

1. Issue the agent its own service identity, distinct from any person's account.
2. Scope that identity's credentials to exactly the systems the agent's tools need to reach.
3. Source secrets from a vault or a short-lived token issuer at runtime, never from a file the agent's own code or prompt can read wholesale.
4. Set an expiry and an automatic rotation on every credential the identity holds.
5. Write down, in one place, how to revoke that identity alone, without touching any other identity.

Thornfield gives Wayfinder the service identity svc-wayfinder-prod. It authenticates to the orders and credit APIs with a token issued for fifteen minutes at a time by Thornfield's internal token service; the token itself is never stored anywhere Wayfinder's own logs or prompts could echo it, only a short, non-secret reference such as token-ref: 8f2c-... appears in a log line. When a security review later needs to revoke Wayfinder's access on its own, one operator disables svc-wayfinder-prod, and nothing else in Thornfield's systems is affected.

Learn in your browser · Chapter 03 of 09

Every tool you give an agent is a door. Least privilege means the agent should be able to open exactly the doors its task needs, and nothing else, no matter how convenient it would be to leave a few more unlocked.

Least privilege is an old computer-security idea, older than AI agents: give any user, program or agent the smallest set of permissions that lets it do its actual job. Applied to an agent, this means scoping three things separately: which tools it can call, what arguments or ranges it may pass to each one, and which of those technically-reachable actions it must never take without a person's approval.

• Is the action reversible, such as a read, a draft or a reschedule, or hard to undo, such as a delete, a payment or a message sent to a real person?
• Could a single call, on its own, cause serious harm if the agent's reasoning happens to be wrong that one time?

A never-list is different from a permission list. A permission list says what the agent CAN reach. A never-list says which of those reachable actions it must never actually carry out alone, even though it technically has the access. An agent might genuinely need broad read access to serve customers well, and still be told, in a way the system enforces rather than merely documents: never issue a credit above a stated limit, never delete a record, never send a message outside an approved template, without a person approving first.

1. List every tool the agent can call, including ones inherited from a shared library it happens to import.
2. For each tool, write the narrowest scope that still lets it do its job: a maximum amount, a specific record type, a read-only mode where writing is not needed.
3. Write a never-list: actions the agent must never take without a person approving, enforced in the tool itself, not only in a prompt instruction.
4. Set the approval threshold: the point above which a person must say yes before the action runs.
5. Review the list again once the agent has been live for a month; real use reveals tools nobody remembers granting.

Scroll sideways to see every column.

Learn in your browser · Chapter 04 of 09

Identity says who the agent is; tool scope says what it may do. This chapter is about what it can physically reach: which systems its network traffic can touch, and which data it is allowed to read or write.

Blast radius is the extent of damage a single failure, a bug or a compromised credential can cause, because of what it can reach. An agent with narrow tool scope can still have a huge blast radius if its runtime sits on a network that can reach everything, or if the data it can write to includes things far beyond its job.

Egress control restricts which external addresses or systems a running agent's network traffic may reach at all, regardless of what its own code tries to do. Pair that with environment isolation: an agent built and tested against staging data should not be able to reach production systems by accident, and a production agent should not be able to reach the wider internet unless a specific tool genuinely needs it.

1. Write an egress allow-list: the specific systems the agent's runtime may reach, nothing implied or inherited from the wider network.
2. Isolate environments: staging credentials and data must not be reachable from a production agent, and vice versa.
3. Classify the data the agent can read or write, and redact or exclude anything outside what its task needs before it reaches the model.
4. Store backups behind a different identity and, where possible, a different resource than the live data they protect.
5. Test the boundary by trying to reach something outside it deliberately, in a safe environment, rather than assuming the configuration works.

Wayfinder's runtime can reach only two internal addresses: the orders API and the credit-issuing API, both over Thornfield's internal network. It cannot reach the customer database directly; it reads orders through the API, which enforces its own field-level limits. It cannot reach the public internet, and it cannot reach Thornfield's backup storage at all, because backups are managed by a separate system under a separate identity that Wayfinder's service identity was never granted.

Learn in your browser · Chapter 05 of 09

Every other boundary in this course limits what an agent can do. This one limits how much and how long, so a bug or a bad loop cannot turn a small mistake into an expensive one.

Four controls work together here. A budget caps total spend, in money, tokens or tool calls, over a session or a day. A rate limit caps how often a tool can be called in a given window, independent of total spend. A timeout caps how long any single step, or the whole loop, is allowed to run before it is stopped regardless of what it is doing. A circuit breaker is the mechanism that actually enforces a limit once it is crossed, killing the session rather than logging a warning and continuing.

A budget without a circuit breaker is just a number in a dashboard: useful for reporting after the fact, useless for stopping damage while it happens. The breaker has to be able to end the session, not merely flag it, and it has to fire before the limit is exceeded by a wide margin, not after a bill arrives.

1. Set a per-call timeout: the longest any single tool call may run before it is treated as failed.
2. Set a per-session limit: total calls, tokens or spend allowed before the session ends automatically.
3. Set a per-day or per-organisation budget as a second, wider net above individual sessions.
4. Build the circuit breaker to end the session on breach, not just to log or alert.
5. Alert a person immediately when a breaker fires, with enough detail, such as the session identifier and what was being attempted, to investigate without reproducing the loop.

Scroll sideways to see every column.

Learn in your browser · Chapter 06 of 09

A log is how you find out what an agent did after the fact, which matters most exactly when something has gone wrong and you were not watching live. It only helps if it exists, is structured, and does not itself become a second place sensitive data leaked to.

Log enough to reconstruct a decision: what the agent was asked, which tools it called and with what arguments, what each tool returned, what it decided, and when, tagged with the identity that acted. Log too little and an incident review becomes guesswork; log too much, in the wrong form, and the log itself becomes a liability.

Redaction means removing or masking sensitive values before they are stored or displayed: a card number becomes a masked string, a full address might become a postcode, a secret token never appears at all, only a short non-secret reference to it. Redact before the value is written to the log, not after: a log that briefly held the real value and was cleaned up later has already been copied to backups, forwarded to a monitoring tool, or read by someone, by the time the clean-up runs.

1. Decide what every log line must contain to support an incident review: actor identity, action, arguments, result, timestamp.
2. List the fields that must never appear in plain form: secrets, full payment details, health data, anything your organisation classifies as sensitive.
3. Redact or hash those fields before the line is written, not in a later clean-up pass.
4. Set a retention period matched to purpose: incident-response fields kept longer, raw sensitive content kept only as long as genuinely needed, or not at all.
5. Restrict who can read the logs to those who need them for operations or investigation, separate from broader system access.

{
  "ts": "2026-09-15T14:02:11Z",
  "identity": "svc-wayfinder-prod",
  "tool": "issue_credit",
  "args": {"order_ref": "TL-88213", "amount_gbp": 15.00},
  "result": "approved",
  "token_ref": "8f2c-...",
  "card_field": "REDACTED"
}

Learn in your browser · Chapter 07 of 09

A demo that worked once is not evidence an agent is safe to ship. A release gate is a stated, checked bar a new version has to clear before it reaches real users, and it only works if the evaluation behind it includes the cases that actually break things.

An evaluation set is a fixed collection of realistic inputs, with a known good, or known unacceptable, outcome for each, run against every candidate version before release. A useful set mixes ordinary cases, most of real traffic, with adversarial ones deliberately built to provoke a bad response: an attempted prompt injection, a request just outside the agent's remit, an ambiguous instruction a careless version might over-comply with.

A release gate turns that evaluation into a decision rule: a stated pass threshold, for example a minimum overall score and zero tolerance for any critical-severity failure, that a version must clear before it is allowed to reach real traffic. An evaluation without a gate is just a report nobody has to act on; a gate without adversarial cases in the set behind it mostly proves the happy path still works, which is the part least likely to have broken.

1. Collect or write realistic ordinary-case inputs covering the agent's actual range of use.
2. Add adversarial cases deliberately: hidden instructions, edge-of-remit requests, ambiguous phrasing that invites over-compliance.
3. Score each case by severity, not just pass or fail: a wrong tone is not the same severity as an unapproved credit.
4. Set the gate: a stated overall threshold, plus zero tolerance for any critical-severity failure, both checked automatically.
5. Stage the rollout after the gate: a small percentage of real traffic first, a canary rollout, widened only once that stage looks clean too.

This is also where NIST's AI Risk Management Framework is most directly useful, not as a certification but as a structure. NIST's framework describes four functions: GOVERN (culture, policies, accountability), MAP (understanding context and risks), MEASURE (evaluating them, which is exactly what a release gate does) and MANAGE (treating them). It is voluntary guidance, not a legal requirement or a certification, and NIST's own site noted on 29 September 2026 that AI RMF 1.0 is being revised, so check the current framework text before treating any specific control inside it as final.

Learn in your browser · Chapter 08 of 09

A release gate checks a version before it ships. This chapter is about the agent you already shipped: watching it for change you did not make, and reviewing the things it depends on before they change under you.

Drift is a live agent's behaviour changing over time because the model, an upstream API, or the data it reads has changed, without the agent's own code changing at all. It is quieter than a bug: nothing failed loudly, the agent simply started behaving a little differently, and by the time someone notices, it may have been happening for weeks.

Monitoring an agent in production means watching metrics that would show drift or a developing problem before a person happens to read a transcript: error and escalation rates over time, latency, cost per session, and the share of sessions that hit a boundary from earlier chapters, such as a never-list rule triggered or a budget breaker fired. A sudden change in any of these, even without an outright failure, is worth investigating.

1. Track error rate, escalation rate, latency and cost per session over time, not just as a single current number.
2. Alert on a meaningful change in any of those, not only on an outright failure.
3. Make every external tool call fail closed by default: an unexpected response escalates to a person rather than being guessed at.
4. Set a freshness check on any data source the agent relies on that changes outside its own release process.
5. Review each dependency, the model provider, every tool API, every library, on a fixed schedule: read its changelog, check its terms have not changed, confirm the version in use is still supported.

Learn in your browser · Chapter 09 of 09

Every boundary in this course can still fail. This chapter is about what happens next: failing in a way that limits harm, containing an incident fast, and recovering cleanly, then applying all of it in one capstone exercise.

Graceful failure means choosing, in advance, whether an agent should fail closed, stop and escalate to a person, or fail open, continue with reduced function, for each kind of failure, rather than discovering the answer live during an incident. As a rule of thumb: fail closed for anything hard to undo, a payment, a delete, a message to a real person; fail open only for things that are safe to get slightly wrong and easy to correct later, a slightly delayed response, a lower-confidence suggestion clearly marked as such.

Incident containment is the immediate response once something has gone wrong: stop the specific harm from continuing, before investigating why it happened. For an agent, that usually means one of: pausing the agent entirely, revoking or rotating the specific credential involved, or lowering an approval threshold to zero so nothing proceeds without a person, exactly as the compromised-tool response in an earlier chapter did.

1. Detect: an alert, a report or a monitoring signal says something is wrong.
2. Contain: stop the specific harm continuing, pause the agent, revoke the credential, drop the threshold, before anything else.
3. Diagnose: read the logs to reconstruct what happened, only once containment is in place.
4. Recover: roll back to the last known-good version, or fix forward if rolling back is not possible, and confirm normal operation before wider release.
5. Review: write down what happened, what caught it, what did not, and what changes as a result, and keep that record.

Rollback and change control work together: rollback only works cleanly if the previous version is genuinely still available and known to have been good, which is why the release gate matters: a version that never passed the gate should never be the last known good. Change control is the written process that decides, approves, records and reviews any change before it goes live, so there is always a specific prior version to roll back to.

Thornfield and Wayfinder in this course are invented, but the shape of the problem is not: a real production agent needs its identity, tools, network, cost and visibility boundaries actually built and actually tested, not sketched once and left. If your organisation is past the point of learning this and into the point of needing an agent's boundaries, logging and release gates actually built, tested and maintained, Mickai builds bespoke AI workflows and code enhancements for companies, running on Mickai's own sovereign operating system with zero data egress: your systems, your data and your agent's own logs never leave your own environment. See what that looks like at mickai.co.uk/bespoke.