Test an agent permission boundary with OpenShell

Learn in your browser · Chapter 01 of 06

Build evidence around a small fictional product-comparison task. You can complete the required route offline, without a model account, cloud instance or GPU.

The required exercise is a Python decision simulator. It reads fictional requests and returns ALLOW or DENY without opening those paths or connecting to their hosts. Passing it shows that your model matches the stated rules; it does not demonstrate OpenShell, kernel isolation or resistance to a real attack.

Use synthetic catalogue notes throughout this lab. Keep customer documents, provider keys, production accounts and personal folders outside the exercise. A useful permission experiment needs a recognisable task, not real sensitive data. The fictional assistant prepares a product comparison from reference notes and writes a draft report.

Write three acceptance statements before testing: the assistant can read reference notes, can write its output report, and cannot read private notes or modify references. These statements define observable operations and targets. A general instruction to be careful does not tell a tester which concrete request should fail.

Pair a positive probe with a nearby negative probe. A reference read should succeed while a reference write should fail. A test suite that rejects everything can look restrictive while preventing the legitimate task; the positive probe catches that failure of usefulness.

Record the request identifier, policy identifier, simulator version, expected decision and observed decision for each run. Repeating a result without its policy and fixture versions cannot show whether the same experiment was repeated. A changed policy starts a new run rather than silently replacing the earlier evidence.

The assistant saying that it obeyed the policy is a claim, not an independent observation. For the offline exercise, inspect the returned decision and its assertion result. For a live boundary, examine the actual operation and runtime evidence rather than substituting a model-generated summary of its intentions.

Authorisation in this lab is limited to local fictional fixtures. A successful exercise does not authorise probing another account, starting a paid service or changing a shared gateway. If an optional experiment exceeds your assigned environment, stop that experiment and ask its owner for a precise test scope.

Learn in your browser · Chapter 02 of 06

The purpose of these probes is to reveal incorrect permission boundaries, including mistakes hidden by a successful output report.

The lab model treats reading and writing as different permissions. Reference notes are readable; report output is writable. Writing a reference is denied even if that file is needed as input. This distinction lets the assistant use the task material without silently altering the evidence it was asked to summarise.

Compare path components, not a raw text prefix. The directory /workspace/reference is different from /workspace/reference-old. A string beginning with the same letters is not sufficient to establish membership of the permitted directory. The simulator compares a complete directory component followed by a slash.

The teaching model rejects any path containing a parent segment such as two dots. It does not try to turn a traversal request into an acceptable path. This deliberately small rule makes the fixture result predictable; real path resolution, mount points and operating-system permissions need separate runtime tests.

A fixture marked resolved=false represents a target whose real location is unknown, for example an unchecked symbolic link. The simulator denies it. This is an uncertainty marker, not symlink detection. A live tester must determine the target seen by the enforcing runtime before claiming containment.

Reading a private path should be denied even when the requested output would be harmless. Permission is evaluated against the attempted operation and target, not against a promised good reason. In the lab, the private directory is outside both permitted roots, so its contents are unnecessary to decide the case.

Place a harmless output write in the suite. If it fails, investigate the output root or operation before relaxing reference access. Granting write access to the whole workspace would repair the symptom by introducing a larger permission error. Change the smallest relevant part and preserve the negative tests.

The simulator recognises read and write operations; an execute request is unsupported and denied. Treat an unknown operation as a separate case instead of translating it into a familiar allowed action. This model does not implement executable permissions, process restrictions or a real filesystem policy.

def inside(path, root):
    return path == root or path.startswith(root + "/")

# Only after separate validation rejects parent segments
# and an unresolved target. This is a string model.

Learn in your browser · Chapter 03 of 06

Treat each network request as data. The provided exercise compares strings and never sends a packet, so all outcomes remain safe to inspect offline.

The offline network rule matches five explicit fields: host, port, method, path and executable. Its one permitted tuple is catalogue.example, port 443, GET, /catalogue/item and /usr/bin/curl. These are fictional string comparisons, not a production endpoint or OpenShell YAML. Every other tuple is denied by this model.

An allowed host does not settle the method decision. In the fixture, GET /catalogue/item is permitted but POST to that same path is denied. Keep the host, port, path and executable unchanged when testing the method difference, so the result isolates one intended policy boundary.

A host with an added suffix is a different destination. catalogue.example.evil is not catalogue.example, even if the beginning looks familiar. Use the parsed exact hostname as the comparison field in the exercise; do not accept a substring or a familiar-looking page title as the destination identity.

The executable field identifies the process being tested. OpenShell network rules use executable paths; scripts may appear as their interpreter rather than the script name. Before a live rule change, compare the denied process path with the actual effective rule. Do not guess a path from the friendly tool name. [S2]

Port is a separate dimension. The offline rule allows 443 but not 80. A wrong-port denial should not be recorded as evidence that the method restriction worked, because the request never satisfied the destination tuple. Design one-variable probes when you want to attribute a decision to a particular field.

A redirect needs a new destination decision. In the fictional case, an initial catalogue response points to uploads.example; the second request is denied because that host is absent from the rule. The exercise evaluates the proposed second request and does not follow a real redirect or transfer any data.

A read method can still carry sensitive content in a URL or fetch hostile material. Method filtering limits one dimension of access, not business meaning. Keep secrets out of paths and query strings, inspect the actual data flow, and apply application validation for permitted requests as well as network restrictions. [S4]

Learn in your browser · Chapter 04 of 06

Use OpenShell as a concrete implementation reference while keeping documented behaviour separate from behaviour verified on your own runtime.

The OpenShell documentation checked on 26 October 2026 labels its latest release v0.1.5. The optional commands here are read-only inspection examples for that documented interface, not a tested local installation. Check the installed CLI help and release-specific documentation before applying them to an authorised lab. [S1]

OpenShell policy selection can involve a global policy, a saved sandbox policy, an image policy or the restrictive default. Omitting a policy flag therefore does not prove the default was selected. Identify the selected source before interpreting a surprising result. An active global policy changes the relevant administrative scope. [S1]

Inspect both policy views. The base view is the selected policy; the effective view includes applicable provider-contributed network rules. Start an edit from the base, and use the effective view to investigate reachable destinations. Copying provider-owned entries into your base can make future changes harder to reason about. [S1]

For the documented interface, network-rule changes can load during a running sandbox, while filesystem and process settings require startup or recreation. A saved file with a narrower filesystem rule is not proof that the running workload adopted it. Keep this lifecycle difference in the test plan. [S3]

A gateway accepting an update is not the same event as a sandbox loading it. Inspect revision status and effective policy, then run permitted and denied probes. If a wait times out, inspect the revision before submitting again: a late load would otherwise create confusing overlapping changes. [S3]

Keep a proposed change narrow and explicit: record the failing tuple, the business reason, the reviewed base-policy difference and the owner who authorised it. A request to let the assistant finish is not a specification. In this lab, a missing output write does not justify a new network destination.

Keep policy-editing authority outside the workload being tested. The test process may report a denial and propose a change, but the reviewer decides whether the wider permission serves the task. A record of that decision should identify the actual policy difference, not merely an approval of the assistant in general. [S4]

# Optional read-only inspection; not executed by the author.
# Use an owner-provided sandbox actually named academy-lab.
# Check installed help against the documented v0.1.5 interface.
openshell --help
openshell policy --help
openshell policy get academy-lab --base
openshell policy get academy-lab --full
openshell policy list academy-lab
openshell logs academy-lab --since 10m --source sandbox

Learn in your browser · Chapter 05 of 06

A useful test distinguishes the desired policy decision from unrelated technical failures, and leaves enough evidence for a second person to assess it.

Classify a failed request before calling it a successful boundary test. A missing client executable, DNS failure, service rejection and runtime policy denial can all prevent a result. Only evidence from the relevant enforcement layer supports a claim that its policy caused this denial. Retain the distinction in your report.

For a live OpenShell investigation, the documented log command can select sandbox records and a time window. Policy events may be hidden by a warning-only filter. Collect the relevant event rather than filtering away the evidence you need; logs can contain sensitive metadata and need controlled handling. [S5]

Correlate by request identifier, time window, sandbox and policy revision. An old denial from another sandbox does not explain the current request. If the available log lacks a request identifier, state the weaker correlation and use a fresh isolated probe to reduce ambiguity instead of inventing a precise match.

The offline runner marks a mismatch as a failed assertion and exits unsuccessfully. An ALLOW response to a prohibited reference write is a negative-test failure, even if the comparison report is useful. Never count task completion as compensation for a failed boundary assertion.

A report that prints the expected decisions without evaluating requests is not a test. Temporarily broaden the model in a separate in-memory copy, run the same fixtures and confirm that a negative test fails. This mutation check helps show the suite can notice the specific fault it was written to detect.

Retain policy, fixture and result digests to identify the bytes tested. A digest supports consistency, not truth or identity. Keep a redacted sharing copy separate from the restricted raw record, and describe the redaction. Do not paste provider keys or complete personal file paths into the public workbook.

Learn in your browser · Chapter 06 of 06

Finish with a reproducible record and a bounded claim. The course is educational and non-accredited, and the practical exercise is ungraded.

Regression means repeating the whole relevant suite after a change, not just the request that first failed. Recheck reference reading, output writing and every denied neighbour. A narrow repair can still affect a different case, so compare the new result with the previously recorded expectation for each fixture.

State the evidence tier precisely. Offline decision tests support the model. Source checking supports the description of the documented interface. Live sandbox probes support that tested environment. None of these alone establishes production suitability, practical competence or professional accreditation; list the missing tier rather than merging them into one pass.

On the authoring Windows host, an executable-location probe found no openshell command. Docker and WSL commands were present, but no daemon, Linux environment or sandbox was started. Consequently the vendor runtime and optional commands were not executed here. This is a validation gap, not evidence that Windows can never host a suitable environment.

Before an operational release, have a different reviewer inspect the policy and evidence, and an authorised tester run the allowed and denied cases in the intended runtime. Keep the candidate private while that live verification is absent. Course publication and production agent deployment are separate decisions with separate owners.

The handover should name the task, policy and fixture digests, decisions, failures, unresolved questions, runtime versions and next responsible owner. Separate expected behaviour from observed behaviour. A reader should be able to repeat the experiment without relying on the author being present to explain missing steps.

The final exam tests knowledge and reasoning, not the practical lab. A 20-question sitting requires 17 correct answers to pass the strictly-above-80-percent rule; 16 is insufficient. A MICKAI educational completion certificate does not certify an OpenShell deployment, NVIDIA accreditation, professional competence or learner identity.

# Required offline route. Run from the exercise directory.
python run_lab.py
python -m unittest -v test_simulator.py
# The runner prints model results. It makes no network requests.