Healthcare AI purpose, governance and clinical safety boundaries

Learn in your browser · Chapter 01 of 08

An intended purpose is a boundary that can be tested. Start by stating whose task changes, what information the system receives, what it produces and what authority it does not have. A product name such as hospital copilot is too broad to support a release decision.

Cedar Vale Hospital is fictional. Its Willow project proposes a staff-only assistant that converts approved appointment records and service-location instructions into a draft appointment summary. The clerk compares the draft with the current source and chooses whether to send through the existing correspondence system. Willow cannot book, cancel, rank clinical urgency, alter care instructions or contact a patient directly. Every record, role, number and policy in this workbook is invented for learning. The exercise is about controlling an administrative workflow, not making a clinical recommendation.

Begin with the current service rather than the model. Observe how a letter is requested, how the correct record is selected, who checks it, how communication needs are met, how dispatch is confirmed and how corrections reach the recipient. A faster draft has little value if reviewers spend longer resolving ambiguous source records. Preserve the established route for urgent questions and human assistance. When the organisation cannot describe its current process, a model may automate an inconsistency that staff previously resolved through experience.

Mickai's Trust Agent principle connects human control with data and decision sovereignty: who authorises access, owns the appointment, retains copies and can stop delivery. Assess contractual and technical controls for the chosen architecture; neither local hosting nor a familiar supplier establishes permission to process health information.

Scroll sideways to see every column.

1. Record the current workflow and baseline problem before choosing an AI solution. Include a non-AI template or rules-based alternative.
2. Write one purpose statement for one bounded function. State intended users, affected people, inputs, outputs, setting and excluded actions.
3. Compare the statement with the interface, demonstration, contract and sales claims. Resolve contradictions before approval.
4. List foreseeable misuse: a clerk may request advice, paste another person's record or treat a draft as final. Assign design controls, not only warnings.
5. Create a purpose-version record and named owner. Any change to function, claims, users or environment triggers review.

MHRA intended-purpose guidance is written for software medical-device manufacturers. It is a useful scoped reference for clarity about function, user, population and environment, not a declaration that every administrative application is a device. The accompanying MHRA ambient-voice guidance distinguishes administrative functions from medical purposes and warns that broader claims matter. A disclaimer cannot settle contradictory functionality. The course therefore asks learners to prepare the facts for a qualified assessment, not self-award an exemption.

Learn in your browser · Chapter 02 of 08

Administrative does not mean harmless. Analyse how an output enters a real workflow and how a plausible error could affect access to care. Keep the causal chain specific enough to test a control, while leaving clinical consequence judgement with qualified people.

Willow might preserve most words in a letter yet move a time from 09:10 to 19:10. It might combine two appointments with similar identifiers or omit a communication-support requirement. The immediate fault is textual or administrative; the relevant consequence may be delayed attendance, inappropriate disclosure or a person unable to understand the message. These are plausible scenario hypotheses, not measured patient outcomes. Ask the clinical safety officer and operational staff to assess consequence severity and appropriate escalation in the actual service.

Separate a hazard from its causes and harms. A hazard in this teaching case is an incorrect appointment instruction reaching the intended recipient. Causes could include a stale source, cross-record retrieval, unsupported generation or a reviewer selecting the wrong source. The harm is not the same as the wrong string. Record who could be affected, through which sequence and under what conditions. Also consider omission and delay: stopping all correspondence indefinitely can create its own service risk. A controlled hold needs an accountable human route and an ageing queue.

Link the hazard log to evidence. Record a stable ID, scope version, initiating condition, sequence, affected people, consequence, controls, tests, residual uncertainty and decision owner. Probability and severity categories need a locally agreed meaning; multiplying arbitrary scores cannot establish acceptable risk. Use the organisation's authorised clinical risk-management process.

Scroll sideways to see every column.

1. Walk the workflow with operational and clinical-safety representatives, including error, absence, outage and accessibility cases.
2. Write at least one cause-hazard-consequence chain for each consequential output. Include human workarounds and hidden manual copying.
3. Assign a risk decision owner separately from the person implementing the control. Name escalation cover when either is absent.
4. Choose controls that remove or constrain the failure before adding warnings. Record what each control cannot address.
5. Attach a falsifiable test and residual uncertainty to each control. Update the log after tests and incidents instead of treating it as a one-time procurement document.

NHS clinical-safety guidance describes a nominated clinical safety officer with current professional registration and clinical risk-management training. In England, DCB0129 and DCB0160 address different manufacturer and deployment responsibilities within their scope. Those references do not mean that a supplier can transfer the organisation's complete local safety responsibility by sending a certificate. Outside that setting, identify the equivalent accountable local functions rather than exporting NHS job titles as universal legal requirements.

Learn in your browser · Chapter 03 of 08

Three different questions often get collapsed into one: whether a function is a regulated medical device, whether the local workflow is acceptably controlled and whether its data processing is lawful. A positive answer to one does not answer the others.

Before applying a checklist, record the service organisation, people served, supply and use locations, processing locations and manufacturer, deployer or processor roles. Hosting alone does not settle jurisdiction. Select Great Britain and England, the EU, the United States or another region with its own authority and reviewer. These are reference routes, not exhaustive legal coverage or product classifications.

Scroll sideways to see every column.

As checked on 30 September 2026, MHRA guidance explains that healthcare context alone does not make a software function a medical device; intended medical purpose matters. Its current ambient-voice guidance also explains that administrative documentation and medical recommendations differ, and that disclaimers do not erase contradictory medical claims. This is a narrow summary of those sources, not a classification of Willow. An accidental harmful output can still demand investigation even if it does not by itself redefine the manufacturer's intended purpose.

NHS sources distinguish DCB0129 for manufacture from DCB0160 for deployment and use. They are information standards with specified applicability, not a general international AI certificate. NHS service guidance says both standards are under review, so capture the retrieved edition and check for changes before using them in a real procurement. The course's mapping tables are educational work products and are not completed DCB safety documentation. A qualified clinical safety officer must assess the relevant obligations and evidence.

For an EU service, the Commission's MDCG catalogue supports medical-device interpretation but explicitly describes its guidance as non-binding. Applicable legislation remains distinct. For a US service, FDA's current CDS guidance is dated January 2026 and explains criteria concerning non-device functions; do not reuse an older summary or assume every AI assistant is excluded. HHS explains HIPAA scope through covered entities and business associates, so a consumer app is not automatically covered merely because it holds health-related information. Other privacy rules may still apply. These routes need specialist local review, not a translated UK approval form.

1. Choose a real jurisdiction for your organisation or retain the fictional England case. Record uncertainty rather than choosing UK by default.
2. Create three rows for device, operational safety and information governance. Add accessibility, professional, employment or sector requirements where relevant.
3. For each row, record the current primary authority, source date, applicability reasoning, responsible specialist and evidence still missing.
4. Resolve conflicts between product instructions, marketing, actual integration and intended use. Keep the assessment function-specific.
5. Set a source review trigger before procurement, material change and publication. A draft consultation or voluntary framework must not be represented as enacted law.

NIST AI RMF may help structure an AI risk discussion across this work. NIST describes it as voluntary guidance. It is not medical-device market authorisation, a legal compliance decision or a course accreditation. Likewise, a locally stored model does not remove applicable privacy, safety or professional obligations. Keep sovereignty in the decision-rights and data-control analysis without converting it into an exemption.

Learn in your browser · Chapter 04 of 08

A useful draft should be based on the right authorised record, not the largest possible collection of data. Design the data path field by field, including copies that are easy to overlook in prompts, logs, support tickets and backups.

Cedar Vale's first prototype request includes complete clinical notes because they are available in the same export. That is a design mistake for the bounded appointment-summary purpose. Start with the minimum fields required by the approved task. Establish the source identifier and current version outside the model. Missing or conflicting appointment data should produce a hold with an operational reason, not a plausible replacement. The model must not decide which person an ambiguous record belongs to. Records staff use the existing authorised reconciliation process.

For UK special-category processing, ICO guidance explains the need for both an Article 6 lawful basis and an Article 9 condition. A privacy checkbox is not a universal basis for healthcare processing, and consent should not be assumed to fit every institutional relationship. Additional confidentiality and national requirements may apply. The privacy lead must document the actual purpose and roles. ICO's DPIA guidance currently carries an under-review notice following the Data (Use and Access) Act; it still provides a route for assessing processing likely to result in high risk. Verify the current applicable position with qualified review rather than copying old wording.

Map each copy: original appointment record, extracted fields, model request, draft, review record, dispatched correspondence, monitoring event and backup. For each, record purpose, controller or processor responsibility, permitted role, destination, retention rule and deletion or correction mechanism. A vendor saying it does not train on your inputs does not answer how long requests, abuse logs, support access or backups persist. Ask for the exact service configuration and contractual evidence. A data map should include remote support and subprocessors as well as the named hosting region.

Scroll sideways to see every column.

1. List the minimum input fields and their purposes; remove fields without a justified need.
2. Bind record selection to an authorised source and version. Quarantine ambiguous, missing or contradictory identifiers.
3. Review access in the source, retrieval layer, model request, draft store and evidence store. Test cross-service and cross-role denial.
4. Ask suppliers about retention, training use, support access, subprocessors, exports and deletion. Record unresolved claims as gaps.
5. Set correction and retention procedures for every relevant copy. Verify them with synthetic records before importing any authorised real dataset.

Legacy digitisation needs its own quality boundary. Scanning and optical character recognition can create a searchable text layer while misreading an identifier or joining two pages. Preserve original provenance, quarantine uncertain matches and record corrections instead of silently overwriting source evidence. A document hash can support integrity comparison for particular bytes; it does not establish that the scan was correctly attributed, that processing is authorised or that the content is clinically true. Do not describe an integrity record as proof of consent or a person's identity.

Mickai may be considered for a separately scoped digitisation, retrieval or administrative-integration engagement when the service and access are actually agreed. This course provides an educational method and synthetic exercises. It does not establish that Mickai operates a verified hospital connector, has access to a health record or provides medical-device assurance. Any real engagement needs explicit authority, qualified review and evidence for the proposed service.

Learn in your browser · Chapter 05 of 08

A person named in a log is not the same as a person who can prevent an incorrect dispatch. Design the review interface and the final execution boundary together. Give staff the information, authority, time and accessible controls needed to disagree with a draft.

Willow has three distinct states: proposed draft, approved draft and dispatched correspondence. The model can create the first state only. The review interface shows the selected source reference, its version, mandatory fields, draft differences and unresolved warnings. Approval identifies the exact draft and source versions, the authorised reviewer role, the permitted operation and its expiry. Immediately before dispatch, the trusted service rechecks those conditions. Material changes return the item to review; they do not inherit an old approval flag.

Provide reject, correct and defer actions with useful handover reasons. Show the source beside the draft and highlight material transformations. Avoid preselected approval. A generator's confidence score establishes neither source fidelity nor authority. Staff need clear rules about fields that must never be inferred and an existing service route for uncertainty.

Meaningful control also depends on staffing. In the fictional pilot, each clerk has time to compare a draft with its source; during an outage, the queue could grow beyond that capacity. Measure time spent understanding, checking and correcting, not only model response time. Establish a queue owner, an ageing view and an alternative correspondence process. Expiry means no automatic dispatch, while the service team remains responsible for the held work. Do not silently discard it or describe a technical refusal as a completed patient contact.

Human presence, account authentication and approval are different evidence claims. A logged-in session or a microphone response does not prove that the person understood the draft, intended this dispatch or held the required role. A receipt should state exactly what was approved, by which authenticated role reference and when, with limits on that evidence. Strong access controls and appropriate staff authentication are necessary; recording extra video or voice merely to make an attestation look stronger may add unnecessary privacy risk.

Communication accessibility belongs in the workflow. The NHS England Accessible Information Standard addresses identifying, recording, flagging, sharing, meeting and reviewing relevant information and communication needs within its scope. For Willow, a large-print or other recorded format requirement cannot disappear during simplification. Test the actual channel and human assistance route. A technically accessible web page does not establish that a posted letter, translated message or contact centre can meet the person's needs. Qualified language and accessibility review is still necessary where relevant.

Learn in your browser · Chapter 06 of 08

Model accuracy alone does not establish that the service works. Evaluate purpose adherence, source fidelity, access control, review, dispatch and fallback as one workflow. Specify expected outcomes before observing results and keep every measured statement tied to its test set.

Build the acceptance matrix from the purpose card and hazard log. A test should identify its hazard, fixture, relevant version, expected state, observed state and evidence location. Include ordinary successful drafts, similar record IDs, missing fields, conflicting dates, changed source versions, unsupported clinical requests, communication requirements, denied roles, duplicate jobs and unavailable dependencies. Use invented records with no copied patient text. A synthetic test is useful for isolating a failure; it cannot establish clinical effectiveness or the distribution of real patients.

Split tests by failure class and affected workflow. If 99 simple letters succeed and one wrong-record letter is sent, reporting 99 percent success hides a serious boundary failure. A release decision needs the result for the wrong-record class and its consequence, not only a pooled average. Likewise, a pass on English plain-text letters provides no evidence about another language or format. Identify missing coverage rather than treating untested groups as passing. Clinically meaningful severity and acceptable residual risk require qualified local judgement.

from fractions import Fraction
fixtures = {'ordinary': (80, 80), 'wrong_record_hold': (12, 12),
            'stale_source_hold': (10, 10), 'communication': (6, 8),
            'clinical_boundary_hold': (10, 10)}
correct = sum(pair[0] for pair in fixtures.values())
total = sum(pair[1] for pair in fixtures.values())
assert Fraction(correct, total) == Fraction(59, 60)
assert Fraction(*fixtures['communication']) == Fraction(3, 4)
mandatory_pass = all(correct == total for correct, total in fixtures.values())
assert mandatory_pass is False
print({'synthetic_correct': correct, 'synthetic_total': total,
       'communication_correct': 6, 'communication_total': 8,
       'fictional_release_decision': 'HOLD' if not mandatory_pass else 'REVIEW'})

The displayed calculation checks arithmetic under the fictional acceptance policy. The downloadable synthetic gate exercise tests record binding, changed drafts, expiry, roles, clinical exclusions and a stop state using fixed fixtures. Download the synthetic gate exercise ZIP from the course resources, read its README and source, then run python gate_lab.py and python test_gate_lab.py in a local Python 3 environment. It needs no additional package or network connection. It does not call a language model, send a letter, authenticate a real staff member or establish regulatory conformity. Record those limitations beside the results.

1. Derive one or more test cases from every purpose exclusion and hazard control, including a negative case that should be held.
2. Write the expected result before execution, including which state changes and which external effect must not occur.
3. Run a non-AI baseline and the proposed workflow with the same bounded task. Record review effort, corrections and unresolved work as well as successful drafts.
4. Report results by hazard class, relevant service context and communication need. Keep the denominator and uncertainty visible.
5. Repeat affected tests after material changes. Use independent review of fixtures and expected answers to detect a test set that merely repeats the implementation.

Defensive security testing belongs in the owned sandbox. Put a harmless instruction inside a synthetic source field asking the assistant to ignore its task; the expected outcome is that source text cannot grant tool authority or change protected policy. Test a denied record reference and a denied dispatch role. Do not use real patient records, contact live services or probe a third-party organisation. A model refusal in one demonstration is not proof that every injection is blocked; the trusted execution boundary must enforce the policy regardless of wording.

Learn in your browser · Chapter 07 of 08

Evidence has a scope and a lifetime. A model, prompt, source system, staffing arrangement or supplier update can change the behaviour that earlier tests established. Link versions to the workflow and rehearse what happens when an outcome is uncertain.

Keep a release inventory covering the model service, relevant configuration, prompt, retrieval filters, source schema, required-field rules, review interface, dispatch integration and operational procedure. A supplier can preserve its product name while changing behaviour. Ask for material-change notification and a supported rollback or exit path. A locally hosted model also needs version and configuration control. Sovereignty includes the practical ability to pause, inspect evidence and continue the service through an authorised alternative.

Classify a change by its effect, not the size of the code diff. Adding a short sentence that recommends whether to attend changes the bounded administrative purpose. Altering a required communication field may affect access. Switching an identifier mapping can create a wrong-record risk. A cosmetic change may still hide a warning or break keyboard review. The change owner must state which purpose, hazard, data flow and test evidence are affected, then obtain the appropriate decision before release.

Scroll sideways to see every column.

Record necessary time, scope, versions, state transition, operation reference, service and action; restrict detailed evidence separately instead of copying health narrative into an open ticket. Safety, privacy and security functions assess reporting duties and deadlines for the selected jurisdiction. Do not invent a universal clock or delay internal escalation while classifications remain uncertain.

1. Define who can stop generation, approval and dispatch separately, and test that a stop reaches queued as well as new work.
2. Record in-flight operations before taking recovery actions. Distinguish not started, rejected, accepted and uncertain outcomes.
3. Preserve necessary evidence and use established safety, privacy and security escalation routes; restrict access to sensitive detail.
4. Restore the authorised manual service or a known tested version, then reconcile held work and corrections before declaring recovery.
5. Review the cause, failed control, affected evidence and purpose boundary. Retest before resuming the affected scope and record who accepted the residual uncertainty.

A future more capable model, sometimes described using the uncertain term AGI, would not remove these boundaries. There is no assumption in this course that a current product is AGI or that capability growth follows a known schedule. Treat a supplier's claim as a proposal requiring evidence about its actual function. A system with broader abilities can create additional purpose-drift and authority risks. The governance decision remains specific to the service, people, controls and tested release.

Run a tabletop with the correspondence lead, technical operator, privacy representative and clinical-safety function. Give each role a different incomplete observation. The technical operator sees a timeout; the clerk sees a held draft; the service lead sees an ageing queue. The exercise succeeds when participants reconcile those observations and make an accountable service decision, not when everyone declares the model fixed. Record contact cover for absences and a practical handover between shifts.

Learn in your browser · Chapter 08 of 08

A release dossier is a connected argument supported by evidence. Its value lies in showing why a defined workflow may proceed, what remains uncertain and who can hold it. A polished document cannot make an untested control effective.

The capstone combines the Willow artefacts in an index linking purpose, hazard, control, test and decision. Distinguish supplier evidence, local observations, proposed controls and unresolved assumptions. A no-wrong-record-dispatch claim needs a specific test, observation and limit. An unrelated accuracy number, logo or unsupported signed declaration cannot establish it.

A proportionate dossier answers four questions. What exact service and version is being considered? What could go wrong for the people affected? What evidence supports the controls under the intended conditions? Who accepts, rejects or defers the remaining risk within their authority? The reviewer should be able to disagree and identify the consequence of a missing item. A safety case is maintained through change and operation; it should not become a static PDF attached to a procurement ticket and forgotten.

Business value must include the whole service. Suppose a fictional baseline takes six staff minutes per completed letter, while the proposed workflow takes two minutes to generate and five minutes to check and correct. Counting only the two-minute stage would misrepresent staff effort. Compare total time, correction burden, held work, channel success and the consequence of errors under a documented method. None of these invented timings are claims about a real hospital or Mickai product. A non-AI template may remain the more appropriate solution.

1. Freeze the capstone scope, source versions and synthetic fixture set. Create a numbered index of every artefact.
2. Trace each claimed control to an observed result and each unresolved item to an owner, consequence and next action.
3. Prepare a go, hold or limited-scope proposal. State which decision maker has authority and which specialist decisions remain separate.
4. Invite a challenger to find a bypass, an unrepresented user, a stale assumption and a misleading metric. Revise the dossier with their actual findings.
5. Record the final educational decision, conditions and review triggers. Do not claim this exercise has approved any real healthcare deployment.

The final knowledge exam draws 20 equally weighted scenario multiple-choice questions from the 40-question bank, with every chapter represented. Passing requires strictly above 80 percent: 16 correct is 80 percent and fails; 17 correct is 85 percent and passes. The eight written assignments and capstone develop judgement and remain ungraded practice. Their worked answers help you examine reasoning and evidence; keyword matching is not used to establish professional competence. When confident with the course, take the official Mickai exam using the course exam link.

Take the official Mickai exam and receive your named certificate through the Official Mickai course exam link in the resources. The exam is scored by the server. A passing attempt can produce a named, dated educational certificate with a verification link to its protected issuance record. Verification confirms the recorded issuance and status, not the learner's independently verified identity. This is not an accredited qualification, clinical licence, medical-device approval or evidence that a hospital system is safe. Downloads, exam attempts, passes and certificate issuance are distinct events, not counts of unique people.

This course is part of Mickai's Trust Agent mission to make AI more trustworthy and human-controlled. Its practical output is your evidence dossier and completed synthetic exercise. Real healthcare integration, records digitisation and operational support require a separately scoped engagement, authorised access, qualified reviewers and verified capabilities. Use the dossier to expose missing evidence and preserve accountable decision rights rather than treating an educational assessment as permission to deploy.