workbook · Level 3
Predictive maintenance AI with accountable human handover
Evaluate synthetic event windows, warning time and reviewer workload

Your next step
Start this course online
Register your start, then work through the lessons and exercises at your own pace. You can download the PDF separately. Where a scored exam is available, you will find it after the lessons.
Register or resume your course
Start with the essentials
The short answer
Predictive maintenance alerts need a defined target, time-valid evidence and an accountable review route. Use the supplied fictional history to expose leakage, withheld windows, repeated alerts and capacity gaps. Report event coverage alongside window metrics and preserve the existing-plan boundary. No diagnosis, operational instruction or professional approval follows.
What you will learn
- Define an alert target and preserve the distinction between evidence and maintenance authority.
- Identify missing, stale and cross-reset sensor windows without hiding coverage loss.
- Audit feature and label availability against a chronological evaluation freeze.
- Calculate window metrics, event warning time and review-episode burden with explicit denominators.
- Prepare a prospective source-linked human handover and expose review-capacity gaps.
- Compare monitoring results and defend a bounded fallback and evaluation recommendation.
Who it is for
Reliability analysts and maintenance planners learning to question AI alert evidence. Engineering reviewers and supervisors practising a fictional desk study.
Before you start
- Comfort with arithmetic, tables and time-ordered records.
- Basic Python is optional; every practical has a paper route.
Jurisdiction and study scope
These are selected reference pathways. Choose the relevant pathway before applying the exercises. This does not determine applicable law or provide worldwide coverage.
All pathway limits
- Synthetic learning context: All readings, assets, labels, scores, delays and capacity assumptions are invented. No equipment operation, inspection method or professional approval is supplied.
- UK reference context: HSE maintenance guidance is used only for the general distinction between competent maintenance work and a draft analytical report. No site-specific, regulatory or legal determination is made.
- Other jurisdictions: No other jurisdiction has been reviewed. Responsible organisations determine real equipment, safety and maintenance obligations.
Run the synthetic exercise
Download the exercise source, tests and instructions (ZIP)
No real records, external service or model is used by this exercise. Extract the ZIP and start with README.txt. Complete worksheet.md on paper or use local Python3. A structural pass does not verify source truth, free-text meaning or maintenance authority.
Owner-commissioned Mickai Training Academy material prepared with AI assistance. Tern Reliability Learning Desk and all readings, events, scores, policy values and handovers are fictional educational inputs. No real equipment, trained model, operational approval or professional validation is represented.
Learn in your browser · Chapter 01 of 06
Define the alert before evaluating it
An alert requests examination of evidence. It does not diagnose a fault, predict a remaining life or approve a maintenance change.
Tern Reliability Learning Desk uses three invented pump identifiers, P1, P2 and P3, to practise a review process. They are rows in a local dataset, not machines you can operate. The packet has 30 numbered daily observation slots per asset, except one deliberately missing reading. Each day means one equal teaching interval. There is no site, person, plant, customer, model service or equipment connection. The condition index is an invented unitless value; it is not vibration, temperature or a transferable physical limit. Scores are supplied ordinal inputs, not outputs of a model trained in this course.
Start by defining the decision. Tern wants to know whether a proposed alert queue is worth further controlled investigation, and what a person must check before changing an existing plan. The educational output is a reproducible evaluation and a draft handover. It is not a work instruction. The exercise cannot tell you how to inspect, isolate, operate, service or repair any pump. A score of 80 does not mean an 80 percent chance of failure. A later event label does not identify a defective part. Those meanings require evidence this packet does not contain.
Scroll sideways to see every column.
| Record | Meaning here | What it does not establish |
|---|---|---|
| Reading | Asset, scheduled day, measured day, availability day, epoch and condition index | Authentic sensor data or a safe operating condition |
| Score | Supplied ranking value from 0 to 100 | A calibrated probability or remaining useful life |
| Event | Invented condition label occurring on a named day | Cause, severity, diagnosis or need for a repair |
| Epoch | A supplied segment identity changing after a fictional reset | Proof of real maintenance or permission to restart |
| Handover | Source-linked discussion draft | Authority to alter a plan or close work |
The target is deliberately narrow: at an eligible anchor day t, was at least one supplied event for the same asset recorded to occur strictly after t and no later than t+3? Day t itself is excluded from this future-window target. An event exactly three days later is included. The rule is a case definition for retrospective learning, not a universal maintenance rule. A positive label means the target condition occurred in that window; it does not say the score caused it or that an intervention would have prevented it. The same event can label several overlapping windows positive.
Keep observation time, availability time, event time and review time separate. A reading may describe yesterday but arrive tomorrow. An event may occur on day 19 and become known on day 21. The eventual record can support a later evaluation without having been available to a planner on day 19. This distinction applies even when a spreadsheet contains all columns in one file. File proximity is not evidence of historical availability. Every result should name the information cutoff and the source edition used.
A poor proposal says, 'The model predicts bearing failure in three days, so postpone the scheduled task and close the alert.' The packet supports none of those three actions or diagnoses. A defensible question is, 'Under the declared three-day target, which supplied alerts anticipated which invented events, what was withheld for data quality, and can a reviewer inspect the evidence?' A planner can choose whether further authorised investigation is needed under the real organisation's own process. Course completion creates no such authority.
- Name the decision and audience before selecting metrics.
- Write a target definition with asset identity, interval endpoints and label meaning.
- State which fields were observable at the intended decision time.
- List the numerical assumptions, excluded uses and responsible review role.
- Define the paper output and keep any equipment action outside this exercise.
NIST SP 800-82 Revision 3 describes operational technology as interacting with the physical environment and recognises its performance, reliability and safety requirements. This course uses that distinction only to keep a desk-study analysis separate from equipment control. It does not implement the guide or claim a security assessment. The cited 2023 final edition and the dated source-review record identify the reference being used; a later draft is not silently substituted for it.
Try it yourself · Activity 01
35 minRepair the predictive-maintenance claim
Use policy.json, the source definitions and the bad proposal above. Produce a five-line evaluation charter.
- Define the three-day target with both endpoints.
- Explain what score 80 can and cannot mean.
- Separate draft evaluation, human review and equipment authority.
- List two source assumptions the program cannot authenticate.
Which assumption would change your conclusion first?
Worked answer
The charter evaluates supplied scores against same-asset events with 0 < event day minus anchor day <= 3. Score 80 is ordinal only. The output is a reproducible desk-study report and draft request for responsible-person review. No diagnosis, maintenance change or closure follows. Event completeness and epoch authenticity remain source-owner assumptions.
Try it yourself · Activity 02
20 minChapter review
Answer these questions before consulting the separate worked guide.
- Does score 80 mean 80 percent failure probability?
- Does an event on anchor day t satisfy the future target?
- Does an event on t+3 qualify?
- Can one event make several windows positive?
- What does an epoch change establish here?
- What is the authorised course output?
Name one result that the supplied evidence cannot establish.
Worked answer
1. No. It is a supplied ordinal ranking input. 2. No. The lower endpoint is strictly after t. 3. Yes. The upper endpoint is inclusive. 4. Yes, because the windows overlap. 5. Only the supplied segmentation convention, not authenticated maintenance. 6. A fictional evaluation and draft human-review handover, without equipment action.
Learn in your browser · Chapter 02 of 06
Preserve missing readings and reset boundaries
A clean-looking average can combine observations that should never share one feature window. Expose coverage loss before calculating performance.
The feature rule uses the current reading and the two preceding scheduled days for the same asset. It requires all three rows, correctly aligned measurement days, availability by the anchor, and one common epoch. The checker reports the three-day mean condition index, but the alert decision uses the supplied score on the anchor row. It does not learn a relationship between the mean and future events. This separation prevents a worked data-quality calculation from being mistaken for a validated predictive model.
The raw file contains 89 rows from a nominal 90 daily asset slots. P3 day 16 is absent. P2's day-20 row was measured on day 18, so it is stale under this teaching rule even though the row arrived on day 20. P1 changes epoch at day 14; P2 changes at day 21. These are deliberately supplied reset markers. Preserve them in the source copy. Do not infer a reset from a convenient value change or insert a missing reading just to make an average calculable.
Scroll sideways to see every column.
| Candidate window | Rows needed | Disposition |
|---|---|---|
| P1 anchor 15 | Days 13,14,15 | Hold: crosses the P1 epoch boundary |
| P1 anchor 16 | Days 14,15,16 | Eligible: one epoch, values 24,25,26 |
| P2 anchors 20,21,22 | Each includes the stale day-20 row | Hold: stale evidence; 21 and 22 also overlap the reset |
| P2 anchor 23 | Days 21,22,23 | Eligible: fresh rows within the new epoch |
| P3 anchors 16,17,18 | Each needs missing day 16 | Hold: do not impute a learning pass |
P1 anchor 16 has mean (24+25+26)/3 = 25, with source days 14, 15 and 16 in epoch 1. Anchor 15 needs day 13 in epoch 0, so its superficially plausible arithmetic is irrelevant: the window violates the stated comparison rule. P2 day 20 can contaminate three consecutive features. Counting bad source rows alone therefore understates how much evaluation coverage is lost. One absent P3 row likewise removes three anchor windows, not just one.
The evaluation proposes anchors 15 through 27 inclusive for three assets: 13 times 3 = 39 nominal windows. Seven are held: P1/15; P2/20,21,22; P3/16,17,18. Thirty-two remain eligible, or 32/39 = 82.05 percent coverage. The held fraction is 7/39 = 17.95 percent. These are window-coverage measures, not uptime, reliability or safety percentages. A held window is not a negative prediction, a correctly avoided false alarm or evidence that no event happened.
Keep a reason list even when a window has more than one defect. The simple checker reports the first applicable reason in a declared order: missing row, stale measurement, unavailable evidence, then mixed epochs. P2 anchors 21 and 22 report stale evidence even though they also cross a reset. That reporting convention does not make the second problem disappear. A reviewer should inspect the supplied source rows when diagnosing a hold. If a real system needs every cause, its specification must say so and its implementation must retain all causes.
- Preserve the original CSV and its declared schedule.
- Check unique asset/day identity and the meaning of measured versus available time.
- Construct the exact preceding-three-day set by asset.
- Hold missing, stale, not-yet-available or cross-epoch windows.
- Report nominal, eligible and held counts with asset/day reasons.
- Only then calculate features and retain their source days.
The program rejects duplicate asset/day rows rather than choosing the last arrival. If two records conflict, arrival order cannot establish authenticity. It also rejects boolean or out-of-range numerical inputs, future measurement times and impossible availability ordering. These checks constrain the exercise format. They cannot detect a sensor that is consistently wrong while producing valid-looking numbers, a forged epoch label or an omitted event. Test such limits with known synthetic mutations, not with real machinery.
Try it yourself · Activity 03
35 minReconstruct the withheld-window register
Use readings.csv and policy.json. Work on a copy; the paper route uses the five-row comparison table above.
- Calculate P1 anchor 16 and explain why anchor 15 is excluded.
- List all seven held asset/day pairs.
- Derive nominal and eligible counts without treating held windows as negatives.
- Explain why the stale P2 row affects three windows.
Which assumption would change your conclusion first?
Worked answer
P1/16 uses24,25,26 and mean25 within epoch 1; P1/15 crosses epochs. Held pairs are P1/15, P2/20,21,22 and P3/16,17,18. There are 39 nominal windows and 32 eligible, so coverage is 82.05 percent. Every three-day window containing the stale row is held. The program reports one primary reason but the reviewer retains other visible defects.
Try it yourself · Activity 04
20 minChapter review
Answer these questions before consulting the separate worked guide.
- How many raw rows and nominal daily slots are supplied?
- How many candidate evaluation windows are proposed?
- Why is P1/15 held while P1/16 is eligible?
- What does32/39 measure?
- Does a missing reading become a true negative?
- Why must the reviewer inspect P2/21 beyond its first reason?
Name one result that the supplied evidence cannot establish.
Worked answer
1. 89 rows from 90 slots. 2. 39:13 anchors for each of 3 assets. 3. The earlier window crosses the reset; the later uses days 14-16 in one epoch. 4. Eligible window coverage only. 5. No. It is excluded with a visible data-quality hold. 6. It has stale evidence and also crosses a reset; reporting one reason does not remove the other.
Learn in your browser · Chapter 03 of 06
Freeze the information available at each time
A time-ordered split still leaks if a development label was not yet available when the decision was frozen.
Tern freezes its illustrative threshold and evaluation policy at the end of day 14. A development anchor needs a three-day future target and the declared two-day label reporting delay. To know the complete label at freeze, anchor + 3 + 2 must be no later than 14. Therefore the latest permitted development anchor is day 9. Days 3 through 9 are the declared development range. The two earlier days lack a full three-reading history. The score values remain supplied inputs; the course does not fit a detector or claim a learned threshold.
A tempting split uses development anchors through day 11 because day 11+3 reaches day 14. It overlooks label availability. That window's complete target is only available by day 16 under the declared delay, after the freeze. The checker rejects this change. A later file export cannot repair the chronology by putting the outcome column beside old features. A source may also become available later than its nominal event date. Reconstruct the information that could actually have been used at the time.
Scroll sideways to see every column.
| Object | Latest relevant calculation | Meaning |
|---|---|---|
| Development anchor 9 | 9+3+2=14 | Complete target available by freeze |
| Rejected development anchor 11 | 11+3+2=16 | Leaks post-freeze label availability |
| Evaluation anchors 15-27 | Later than freeze14 | Chronological assessment range |
| Final evaluation anchor 27 | 27+3+2=32 | Needs complete outcomes through 30 and label availability32 |
| Rejected evaluation anchor 28 | 28+3=31 | Extends beyond the supplied outcome-completeness horizon |
Evaluation uses anchor days 15 through 27. Outcomes are declared complete through day 30 and available at day 32. Those declarations are fixture assumptions, not evidence that a real event register is complete. A final anchor at day 28 would need outcomes through day 31 and labels by day 33. Excluding that immature window is different from labelling it negative. At the evaluation cutoff, some real studies may have censored observations whose future outcome is not yet known. An explicit eligibility rule is necessary before choosing a method to handle them.
The five anchor days 10 through 14 are not used as development or evaluation anchors. This is not a generic instruction to discard five rows. There are three assets, unequal observed row counts and a missing slot. A gap measured in rows can represent different elapsed times after pooling assets or losing observations. Use asset identity and the declared calendar logic. The scikit-learn TimeSeriesSplit documentation describes time-ordered folds, notes equal spacing for comparable fold durations and defines gap in samples. This local exercise implements its own explicit day rules and does not depend on scikit-learn.
A further source of leakage is preparing a transformation using the evaluation period. If you later experiment with a fitted normaliser, imputer or threshold, estimate it from permitted development evidence only, then freeze its parameters before assessing later data. The present mean is an arithmetic feature with no fitted population parameter. That does not make every possible transformation safe. A centred mean using tomorrow's reading would violate the prospective feature definition even if the training and evaluation files were separate.
- Write the freeze, development anchors, evaluation anchors and outcome-availability cutoff.
- For every development target, check horizon plus reporting delay against freeze.
- For every feature, check source measurement and availability times against its anchor.
- Preserve asset identity and reset epochs when forming windows.
- Freeze threshold and any learned preprocessing before later evaluation.
- Record rejected immature or leaking rows instead of silently scoring them as negatives.
The exercise tests chronology, not generalisation to a new plant. All three identifiers occur in the synthetic history, and the score rule is not learned. A time holdout on familiar assets would answer a different question from transfer to unseen equipment. Before a real proposal, its evaluation design would need to match the intended future use, sources and maintenance context. Do not claim that a clean time split proves cross-site performance or removes all dependence between overlapping windows.
Try it yourself · Activity 05
35 minAudit the freeze without borrowing future labels
Use the policy and target definition. The optional test suite includes deliberate chronology mutations.
- Explain the latest permitted development anchor using both delays.
- Reject or accept development anchor 11 and evaluation anchor 28 with arithmetic.
- Describe why a five-row gap is not necessarily five days.
- Name one preprocessing change that would leak future information.
Which assumption would change your conclusion first?
Worked answer
Development anchor 9 is latest because9+3+2=14. Anchor 11 needs availability16 and is rejected. Evaluation anchor 28 needs outcomes 31 and availability33, outside the supplied horizon30/freeze32. A row gap changes meaning when assets are interleaved or readings missing. A centred average using tomorrow, or a fitted transform estimated on evaluation data, would leak.
Try it yourself · Activity 06
20 minChapter review
Answer these questions before consulting the separate worked guide.
- What is the freeze day?
- What makes day 9 the latest development anchor?
- Why is day 11 still leaking despite its target ending at 14?
- Why does final evaluation anchor 27 need label availability32?
- Does TimeSeriesSplit gap count calendar days?
- What generalisation claim is not tested?
Name one result that the supplied evidence cannot establish.
Worked answer
1. Day 14. 2. Its three-day horizon and two-day reporting delay end at 14. 3. The complete label is not available until16. 4. 27+3+2=32. 5. No. It counts samples; the caller must ensure its timing meaning. 6. Transfer to unseen real equipment or sites, among other untested uses.
Learn in your browser · Chapter 04 of 06
Count windows, episodes and event warning time
Window recall, event coverage and review workload use different units. Report all three before interpreting an apparently excellent result.
Apply the fixed threshold 70 to the32 eligible windows. A score equal to 70 alerts; this is a greater-than-or-equal comparison. The later labels identify10 positive windows and 22 negative windows. Eleven eligible windows alert. Seven are positive and four are negative, so TP=7, FP=4, FN=3 and TN=18. Check7+4+3+18=32 and 7+3=10. Missing or cross-reset windows are absent from this matrix and remain visible in the coverage report. They are not true negatives.
Scroll sideways to see every column.
| Measure | Calculation at 70 | Interpretation |
|---|---|---|
| Window precision | 7/11=63.64% | Share of alerted eligible windows with a later target event |
| Window recall | 7/10=70% | Share of positive eligible windows that alerted |
| False positive rate | 4/22=18.18% | Share of negative eligible windows that alerted |
| Eligible coverage | 32/39=82.05% | Share of proposed windows with usable evidence |
Repeated alerts for one event can make the queue seem busier without detecting another event. Tern therefore groups adjacent alerted days on the same asset and within one epoch into a review episode. A gap in alert days starts another episode; different assets never merge. The grouping is an invented workload convention. It must not conceal individual source rows or imply that any outstanding work has been completed. Keep both row and episode counts in the report.
Scroll sideways to see every column.
| Episode | Alert days | Later event within the window |
|---|---|---|
| P1-A | 16,17,18 | E1 at 19 |
| P1-B | 22 | None |
| P1-C | 25,26 | E2 at 28 |
| P2-A | 19 | E3 at 22 |
| P2-B | 23,24 | None |
| P3-A | 19 | None |
| P3-B | 23 | E4 at 25 |
Eleven alert rows become seven episodes, of which three have no supplied target event. The false-episode fraction is 3/7=42.86 percent. It is not a false-alarm rate per operating hour: the fixture contains no verified operating-hour exposure. Four episodes each anticipate one of four declared evaluation events. Event coverage is 4/4=100 percent, even though window recall is 70 percent. There is no contradiction. Several missed positive windows still relate to an event that another earlier alert covered.
Warning time uses the first qualifying alerted anchor inside the three-day target window for each event. E1 occurs on19 and first alerts on16, giving3 days. E2 occurs28 and first qualifies25, also3. E3 occurs22 and first qualifies19, also3. E4 occurs25 and first qualifies23, giving2. Mean warning time across these four detected events is 11/4=2.75 days; median3 and minimum2. These are retrospective intervals, not promised intervention time. They exclude undetected events from the lead-time average, so report the event denominator and misses alongside them.
Raising the illustrative threshold to 90 yields four alert rows: P1/16, P1/26, P2/23 and P3/23. The matrix becomes TP3, FP1, FN7, TN21. Window precision rises to 75 percent, recall falls to 30 percent, and only three of four events are covered. E3 is missed. The detected lead times are 3,2 and 2 days, averaging7/3, about 2.33 days. This is a sensitivity comparison, not permission to choose a threshold on the evaluation set and call its performance independently tested. Any choice informed by these outcomes needs a new evaluation design.
One evidence set, three units of evaluation
Preserve withheld coverage and distinguish repeated alert rows from distinct review episodes and later events.
Scroll the diagram sideways, or read every step and relationship below. Keyboard: focus the diagram, then use the Left and Right arrow keys.
- 39 proposed windows: Includes all three assets and thirteen anchor days.
- 7 quality holds: Remain explicit, outside the window matrix.
- 32 eligible windows: Threshold 70 yields11 alert rows.
- 7 review episodes: Adjacent alerted days grouped within asset and epoch.
- 4 declared events: All four covered at 70; first qualifying leads3,3,3,2.
- 39 proposed windows to 7 quality holds: Record each excluded window
- 39 proposed windows to 32 eligible windows: Apply the declared quality rule
- 32 eligible windows to 7 review episodes: Group without losing row evidence
- 7 review episodes to 4 declared events: Evaluate coverage with the declared event set
- Show the nominal and eligible window denominators.
- Reproduce all four confusion-matrix cells.
- Group alerts using a stated episode rule and retain the original days.
- Count every declared evaluation event, including those with no eligible warning.
- Calculate first qualifying warning time only for detected events and report misses separately.
- Discuss workload and uncertainty before making a bounded recommendation.
Try it yourself · Activity 07
50 minDefend three apparently conflicting results
Use the baseline report or reproduce it with evaluate.py. Compare threshold 70 with 90 on a copy.
- Calculate the threshold 70 matrix and denominator checks.
- Explain how70% window recall coexists with 100% event coverage.
- Reconstruct the seven episodes and three false episodes.
- Compare threshold 90 event coverage and lead time without claiming a newly validated policy.
Which assumption would change your conclusion first?
Worked answer
At 70 the matrix is 7,4,3,18 over32 eligible windows; precision7/11 and recall 7/10. The four events are each covered by at least one alert despite three missed positive windows. Eleven alert rows group to seven episodes, three false. At 90 the matrix is 3,1,7,21; E3 is missed, event coverage 75%, and detected warning times3,2,2. Evaluation-informed threshold selection needs fresh validation.
Try it yourself · Activity 08
20 minChapter review
Answer these questions before consulting the separate worked guide.
- Which windows enter the confusion matrix?
- What is the false positive rate at 70?
- Why is 3/7 not an operating-hour false-alarm rate?
- What are event coverage and mean detected lead at 70?
- What event is missed at 90?
- Why does selecting90 after seeing these results need new evidence?
Name one result that the supplied evidence cannot establish.
Worked answer
1. Only the32 eligible windows, not the seven holds. 2. 4/22, approximately 18.18 percent. 3. Its denominator is episodes, and verified operating-hour exposure is absent. 4. 4/4=100 percent and 11/4=2.75 days. 5. E3 on P2 day 22. 6. The held-out outcomes have influenced policy selection, so the same result is no longer independent validation of the choice.
Learn in your browser · Chapter 05 of 06
Hand over evidence without changing the plan
A review request must preserve what is known now, what is unknown and who owns the separate maintenance decision.
Use P2 at day 19 as the prospective handover case. Days 17,18,19 contain condition indices37,38,39, giving mean38 within epoch 0. The current supplied score is 80, above threshold 70. At that moment the later E3 event on day 22 is not known; its label becomes available on day 24. A retrospective reviewer can later count a three-day warning, but the original handover cannot cite that future result as contemporaneous evidence. Store the two records with their different purposes and times.
The structured draft records packet version tern-v1, P2, anchor 19, source days 17-19, score 80, threshold 70, draft-for-human-review status and responsible-person-review as the next step. It states changes_existing_plan=false. This does not authenticate the author or reviewer. It simply lets another learner compare a structured draft with the supplied source. The actual human must read the complete free text, ask for missing evidence and make any real decision through the responsible organisation's process.
Scroll sideways to see every column.
| Sentence in a draft | Disposition | Reason |
|---|---|---|
| P2 day 19 has supplied score 80 against threshold 70. | Keep with source locator | Matches the current input and frozen rule |
| The three same-epoch values average38. | Keep with days 17-19 | Reproducible feature calculation |
| A bearing will fail on day 22. | Remove | No diagnosis or prospective event knowledge supplied |
| Defer the existing task until the model confirms it. | Remove | The exercise gives no plan-changing authority |
| Work completed; close the order. | Remove | Neither inspection nor work-completion evidence exists |
The useful handover asks the responsible person to review the observations and determine whether further authorised action is appropriate. It names the source edition, known quality holds, intended audience, unresolved questions, existing-plan boundary and response owner. It does not prescribe an inspection technique. The HSE maintenance guidance emphasises competent people, suitable information and planned arrangements. That supports keeping real maintenance decisions with the responsible competent process; it does not turn this workbook into a maintenance method or an assessment of any real site.
Human review needs time. Seven review episodes at an invented18 minutes each require126 minutes. The available 90-minute analyst window fits five complete reviews, leaving two episodes unserved and 36 minutes of demand. Eleven separate alert rows would consume198 minutes if reviewed independently under the same assumption. Grouping saves72 review minutes in this arithmetic, but that does not prove a real productivity gain or make two pending episodes disappear. A queue can be precise and still exceed its review capacity.
Do not allocate scarce review slots by using future labels from the evaluation. On the day of the alert, the desk does not know which episodes will later match an event. A real priority rule would require authorised, current context and its own evidence. Here the learner reports the capacity gap and requests a human decision; the exercise deliberately supplies no live prioritisation rule. A missed response or absent reviewer must not be converted into automatic permission, an automatic negative label or cancellation of existing maintenance.
A structured draft is not an equipment instruction
The review path keeps current observations, human decisions and existing maintenance authority separate.
Scroll the diagram sideways, or read every step and relationship below. Keyboard: focus the diagram, then use the Left and Right arrow keys.
- Current evidence: Only inputs available by the anchor; no future labels.
- Draft handover: Sources, score, quality limits and open questions.
- Responsible-person review: Compare complete wording and assess missing evidence.
- Hold or return: Keep pending status and the original source.
- Separate authorised process: Any real maintenance decision belongs here, outside the exercise.
- Current evidence to Draft handover: Preserve time and source edition
- Draft handover to Responsible-person review: Request evidence-based discussion
- Responsible-person review to Hold or return: If evidence or review capacity is inadequate
- Responsible-person review to Separate authorised process: No automatic transfer of permission
Test 32 changes only the free_text field to a false diagnosis and a fabricated work-completion claim. The structured checker still returns a match because it does not understand arbitrary prose. This is a designed negative capability demonstration, not an approval. Read that output together with its limitation. A field-level pass can coexist with a dangerous sentence. The learning evidence is your identification and repair of that sentence, supported by the source, not the fact that you clicked a practice control or obtained exit code0.
Try it yourself · Activity 09
45 minWrite the day 19 handover and capacity note
Use handover.json, P2 source days 17-19 and the paper contract above.
- Write a source-linked prospective draft without using E3 as known evidence.
- Mark the three unsupported action/diagnosis sentences in the table.
- Calculate total review demand and the two-episode gap.
- Run or read test 32 and explain why its structural success is insufficient.
Which assumption would change your conclusion first?
Worked answer
Keep the known score 80, threshold 70, source days 17-19, epoch 0 and mean38. E3 becomes known only on24 and belongs to later evaluation. Remove diagnosis, deferral and completion claims. Seven episodes require126 minutes;90 fits five, leaving two and 36 minutes. Test 32 deliberately passes structural fields despite false free text, so human semantic review is necessary.
Try it yourself · Activity 10
20 minChapter review
Answer these questions before consulting the separate worked guide.
- What is the mean for P2 days 17-19?
- Can a day 19 handover cite E3 as already known?
- How many episode reviews fit the stated90 minutes?
- What remains after those five?
- Does a structured draft match authenticate a reviewer?
- What does test 32 demonstrate?
Name one result that the supplied evidence cannot establish.
Worked answer
1. (37+38+39)/3=38. 2. No. E3 occurs22 and becomes known24. 3. Five at 18 minutes each. 4. Two episodes and 36 minutes of unserved demand. 5. No. It verifies neither identity nor authority. 6. Correct structured fields can accompany false free text; the human must inspect meaning.
Learn in your browser · Chapter 06 of 06
Monitor coverage, preserve fallback and defend the result
A metric can improve because difficult evidence disappeared. A defensible recommendation keeps excluded data, missed events and unresolved human work visible.
Repeat the analysis after removing every P3 reading from a copy. Do not remove E4 from the declared evaluation event list. The nominal39 windows remain, but only 22 are eligible: P1 contributes12 and P2 contributes10. Coverage falls from 32/39=82.05 percent to 22/39=56.41 percent. Seventeen windows are now held, or43.59 percent. The missing asset has not become reliable, silent or safe. Its evidence is unavailable, so the report must retain its identity and missing coverage.
The remaining eligible matrix is TP6, FP3, FN1, TN12. Window recall rises from 70 percent to 6/7, about 85.71 percent. Yet event coverage falls from 4/4 to 3/4 because E4 has no eligible warning. This is the course's central monitoring trap: a better conditional metric can accompany worse coverage of the intended population. Neither result alone can justify expansion. Report the missing asset and both denominators before discussing the model. Do not silently remove an undetectable event from the event denominator.
Scroll sideways to see every column.
| Measure | Complete fixture | P3 readings removed, E4 retained |
|---|---|---|
| Eligible windows | 32/39=82.05% | 22/39=56.41% |
| Window recall | 7/10=70% | 6/7=85.71% |
| Event coverage | 4/4=100% | 3/4=75% |
| Held windows | 7 | 17 |
A changing result may arise from a new score distribution, sensor availability, operating context, maintenance reset conventions, label delay or the outcome population. Do not call every change model drift without examining the source. Track asset-level coverage, stale-data holds, missing events, input versions, review latency and pending work. A frozen threshold can still produce a different queue when the input process changes. Conversely, editing a threshold without recording its version makes comparisons uninterpretable even when the headline percentage stays the same.
The fallback in this educational design is a human-controlled review route under the existing maintenance arrangements. Missing AI or sensor output supplies no permission to extend an interval, restart equipment, omit a task or assume normal condition. The responsible organisation must define its own response to unavailable evidence. The course provides no operational response sequence. In the paper packet, record the hold, affected asset, last trustworthy source edition, pending review owner and condition for resuming the comparison. Do not substitute a copied last value for a fresh observation merely to keep the dashboard green.
Feedback is also time-bound evidence. A person marking a review as complete does not establish an event outcome. A replacement record does not automatically mean a predicted failure was prevented. Actual interventions can change what is observed, making counterfactual claims harder. The synthetic packet has no intervention trial, control group or causal model. It supports reproducible counting under stated assumptions, not an estimate of avoided failures, life extension, reduced downtime or financial return. Keep those benefits as untested hypotheses if a future proposal needs them.
- Freeze the input, policy, outcome-completeness declaration and report versions.
- Compare coverage by asset before comparing conditional accuracy.
- Retain every declared evaluation event, including unobservable ones.
- Separate data-quality changes, model changes and human queue changes.
- Record fallback ownership and unresolved review without inventing an operating procedure.
- Use a new, suitable evaluation to assess any revised policy; retain the old evidence.
The final recommendation should be modest and specific. Tern's threshold 70 finds all four supplied events, but excludes seven windows and creates126 minutes of review demand against 90 available. The labels and scores are invented, the event count is tiny and no model is trained. Continue only as an educational desk study. Repair the data-quality and capacity assumptions before designing another controlled evaluation. The packet cannot establish site readiness, professional competence, engineering fitness or statutory compliance.
Try it yourself · Activity 11
90 minCapstone: defend the Tern handover packet
As the fictional review lead, assemble a portable evidence pack using only the supplied files. A second learner should be able to reproduce and challenge it.
- State the target, information freeze and separate human-authority boundary.
- Include the seven holds, window matrix, episode list, event warning times and workload calculation.
- Include the repaired prospective P2 day 19 handover and its missing-evidence note.
- Show the P3 outage comparison without removing E4.
- Record what the code checks, what it deliberately cannot check and the evidence needed for another bounded trial.
Could another learner reproduce your report without trusting your status labels?
Worked answer
A defensible packet preserves the original89 rows and four labels, records39 proposed/32 eligible windows, matrix7/4/3/18, seven episodes with three false, event leads3/3/3/2 and 126-minute review demand with two episodes pending. It proves the day 14 freeze with development anchors 3-9 and evaluation15-27 plus labels available 32. P3 removal gives22 eligible, matrix6/3/1/12 and only 3/4 event coverage despite85.71% window recall. The handover remains a source-linked draft with no diagnosis or maintenance change. Limitations include event completeness, authentic resets, reviewer authority, free-text truth, causal benefit and performance outside this invented packet.
Try it yourself · Activity 12
20 minChapter review
Answer these questions before consulting the separate worked guide.
- How many eligible windows remain without P3 readings?
- Why can recall improve while event coverage falls?
- Should E4 be removed because P3 has no usable readings?
- Can missing AI output defer existing maintenance?
- What causal benefit has been demonstrated?
- What can the automated certificate assess?
Name one result that the supplied evidence cannot establish.
Worked answer
1. 22, out of the original39 proposed windows. 2. The eligible-window population changes while the declared event set still includes the missed P3 event. 3. No. Retain the declared event and record the missing warning. 4. No authority to change a real plan is created by this exercise. 5. None. There is no intervention trial or real performance evidence. 6. Knowledge answers only; written work and capstone are ungraded and no professional competence or accreditation follows.
Mickai Training Academy is not an accredited training provider. This course has 40 scored knowledge questions and six ungraded written practices. A knowledge sitting draws 20 questions with a 45-minute limit and requires strictly more than 80 percent: 16 correct fails and 17 passes. The raw score is not rounded into a pass. Completing the workbook, running tests or writing a capstone does not itself pass the knowledge assessment or verify practical competence. Check the published course and assessment pages for current examination and certificate availability. This workbook supplies no equipment service.
Keep learning
The complete workbook
A complete offline desk study for reliability analysts and maintenance planners. Six chapters teach temporal evidence, quality gates, alert/event evaluation and a source-linked handover through one invented dataset. The workbook includes worked answers, an ungraded capstone and an executable standard-library fixture. Study time is an initial reading/practice estimate, not a measured learner outcome.
- 01Define the alert before evaluating itRead here · 2 exercises
An alert requests examination of evidence. It does not diagnose a fault, predict a remaining life or approve a maintenance change.
- 02Preserve missing readings and reset boundariesRead here · 2 exercises
A clean-looking average can combine observations that should never share one feature window. Expose coverage loss before calculating performance.
- 03Freeze the information available at each timeRead here · 2 exercises
A time-ordered split still leaks if a development label was not yet available when the decision was frozen.
- 04Count windows, episodes and event warning timeRead here · 2 exercises
Window recall, event coverage and review workload use different units. Report all three before interpreting an apparently excellent result.
- 05Hand over evidence without changing the planRead here · 2 exercises
A review request must preserve what is known now, what is unknown and who owns the separate maintenance decision.
- 06Monitor coverage, preserve fallback and defend the resultRead here · 2 exercises
A metric can improve because difficult evidence disappeared. A defensible recommendation keeps excluded data, missed events and unresolved human work visible.
Also inside: a 10-point checklist, a glossary of 12 terms and 10 questions and answers to test yourself. 12 hands-on exercises, each with a worked answer at the back where the workbook gives one.
The PDF is free. We ask for your name and email before downloading. Read the privacy notice at the download form. Free to download and use for personal learning, study groups and inside your own team. Please do not resell the workbooks or republish them as your own. Link people to trust-agent.ai instead.
Get your free PDF workbook
Enter your name and email, then download the complete workbook. This form also works without JavaScript.
Test yourself
Questions and answers
Is this a pump maintenance procedure?
No. It is an offline synthetic evidence-review exercise. Real equipment work requires its own competent, authorised process.
Does a high score diagnose a fault?
No. The supplied score is ordinal and neither probability, diagnosis nor remaining useful life.
Why can one missing row hold three windows?
Each three-day feature window containing that scheduled day lacks a required source row.
Why is a simple chronological split not enough?
A development label or transformation can still use information available only after the freeze.
What does the local code train?
Nothing. It evaluates supplied fixed scores and declared event labels.
How can window recall be70 percent while all events are covered?
An event can have multiple positive windows; one qualifying alert can cover it even if other windows miss.
Is the2.75-day average a assured response interval?
No. It is a retrospective average for detected invented events, without an operational readiness claim.
Does the handover checker understand every sentence?
No. Test 32 deliberately shows false prose passing structured checks.
Can better recall conceal lost coverage?
Yes. Removing P3 improves eligible-window recall to 85.71 percent while event coverage falls to 75 percent.
Does a course pass certify maintenance competence?
No. The certificate assesses knowledge only; practical work is ungraded and the Academy is not accredited.
Write as you go
My notes
Use this scratchpad for your own notes, working or answers as you read. Notes are saved in this browser on this device. Downloading makes a local copy; it does not send your notes to Mickai.
Learn → Practise → Demonstrate
Ready to put it into practice?
Take the scored exam when you feel ready. You do not need to download the workbook first. A score strictly above 80% earns a named, dated Mickai certificate with a verification page.
Take the official examYour name and email are collected at exam start. Certificates record an assessment result, not an accredited qualification or verified identity.
Learn the language
Key terms
- Anchor
- The day at which an input window and prospective alert are evaluated.
- Availability time
- The time when a source could actually be used, which may follow its observation time.
- Epoch
- A declared segment identity used to avoid mixing pre-reset and post-reset evidence.
- Label horizon
- The future interval in which the specified target event is counted.
- Label delay
- The assumed interval between an event and its recorded availability.
- Leakage
- Use of information that would not have been available for the intended decision.
6 of the workbook's 12 terms. The complete glossary is in the workbook.
Follow the evidence
Sources and checks
Facts last checked: .
These workbooks use AI assistance. See how the workbooks are made.
- Guide to Operational Technology Security, SP 800-82 Revision 3 final, September 2023National Institute of Standards and Technology
- TimeSeriesSplit documentation, checked 12 October 2026scikit-learn developers
- Maintenance of work equipment, checked 12 October 2026Health and Safety Executive
Created by Mickarle Sean Junior Wagstaff-Irons - Micky Irons with the Mickai team. Published by Mickai LTD. Last updated 12 October 2026.
NextKeep going
Where to go next

Recommended for you
Design a repeatable evaluation suite for an AI application
Build an offline AI evaluation harness with test cases, a rubric, regression comparisons, uncertainty checks and a reproducible release record.

Recommended for you
Engineering AI purpose, hazards and professional sign-off
Build a fictional engineering AI assurance plan that distinguishes document assistance,current evidence,independent challenge and held physical release.