Exam · Reasoning models and reasoning engines

Take the exam and earn your certificate.

A real, server-scored exam for this course. Score strictly above 80% to pass and get a genuine, verifiable certificate.

Before you start

We ask for your name and email first, the same as a PDF download, so we know you are here and your certificate can carry the right name.

We privately record your name, email and request IP for this exam request. Read the privacy notice and retention policy.

Visit Mickai on LinkedIn

Complete the security check before submitting.

Cloudflare Turnstile helps prevent automated submissions. How this check uses data.

You can retake the exam any time. Each attempt draws a fresh set of questions.

Written practice, ungraded

Write your response in your own notes, then compare it with the sample answer. This practice is separate from the scored exam and does not affect your certificate.

Practice 1

Explain how a reasoning model differs from prompting an ordinary model for a chain of thought. Then say what test-time compute means and name one situation where spending more of it is unlikely to help.

Compare with a sample answer

Chain-of-thought prompting asks a model to write out intermediate steps before its final answer. A reasoning model is trained, commonly with reinforcement learning on tasks whose answers can be checked automatically, to produce and use a long chain of thought by itself. Test-time compute is computation spent while the model is answering rather than while it was trained. A 2024 study found that spending it well could beat a much larger model on problems a smaller model could already sometimes solve, but it helped little on the most challenging problems. It also cannot supply a fact the model never learned: for that, supply the text or use retrieval or tools.

A good answer separates prompting from training, defines test-time compute, and names a limit such as the hardest problems or missing knowledge.

Practice 2

A team handles four kinds of request: polite email rewrites, one-page summaries, debugging faults that span several files, and planning project schedules with constraints. Decide which should go to a reasoning model, justify each choice, and describe how you would keep the cost under control.

Compare with a sample answer

Debugging across several files should go to a reasoning model because it must trace behaviour and compare hypotheses. Planning with constraints often benefits because trade-offs must be weighed. Email rewrites and one-page summaries should go to a fast model: nothing in a rewrite needs searching or verifying, and a faithful summary comes from the text rather than deliberation. To control cost, route easy requests to the fast model and escalate hard ones, or start at low effort and raise it only when a check fails. Because many hosted services bill thinking tokens as output tokens even when they are hidden, check the provider's pricing page for how reasoning tokens are counted.

A good answer uses the help and no-help reasoning for each task and links cost control to routing, effort and how reasoning tokens are billed.

Practice 3

Outline a prompt and an effort plan for asking a reasoning model to fix a function that is failing its tests, following the recommended prompting and effort practice.

Compare with a sample answer

State the goal (make the function pass its tests), the constraints (for example, keep its public interface) and the format of the answer, and leave the method to the model. Include all the code and facts it needs, because thinking cannot fetch what is missing. Say what success looks like: the named tests must pass. There is rarely a need to add 'think step by step', and the prompt should start zero-shot, adding examples only if testing shows they help. For effort, try reasoning off, then low, then high only when the tests fail; record the lowest level that passes as the default for this kind of task, re-test when the model changes, and set a maximum output length and a timeout.

A good answer covers goal, constraints, format, complete facts, success criteria, zero-shot first, and an escalating effort plan with limits.

Practice 4

You add 'I think the answer is' followed by a wrong option to a multiple-choice question, and a reasoning model changes its answer without mentioning the hint in its working. Explain what this shows and how you would design an accountable process around such a model.

Compare with a sample answer

It shows the working is not a faithful account: something drove the answer that the trace did not mention, which matches a 2025 study that added hints to prompts and found traces often did not mention a hint the model had used. A fluent trace is not evidence of why the model decided. I would read traces only to find errors, verify answers with an independent check, avoid any process that depends on seeing the working because some services show only a summary or nothing, and for audit log the inputs, the outputs and the checks performed rather than the model's story.

A good answer names faithfulness, cites the hint finding, and replaces trust in the trace with independent checks and proper logging.

Practice 5

Design a generate-and-check loop for a model that writes database queries. Name the checks you would use, what each might miss, and what happens when the checks keep failing.

Compare with a sample answer

Ask at low effort first, then check the answer. A programmatic check runs the query on test data and compares the result with a known answer, but it can miss errors if the tests are too weak. A constraint check validates the output schema and checks ranges, but it misses rules nobody wrote down. If a check fails, ask again at high effort with the last answer and the failed check added to the prompt, asking the model to fix it. After a set number of tries, return nothing and hand the task to a person. The loop works because checking is usually easier than solving.

A good answer pairs each check with its watch-out, escalates effort with feedback, and ends with a hand-over to a person.

Practice 6

Split the decision to approve an expense claim between a model and rules. Say what the model should do, what the rule engine should decide, how you would check the model's work, and why this split suits the decision.

Compare with a sample answer

The model reads the messy receipt images and claim text and extracts structured facts such as amount, date and category. A rule engine applies IF and THEN rules, for example IF the amount exceeds the limit THEN refer the claim for approval, so the decision is exact, repeatable and auditable because you can list the rules that fired. A validator checks the extracted fields before the rules run, and a person reviews referred or unusual claims. The split gives flexible input and an auditable outcome: the model reads and the rules decide.

A good answer gives reading to the model, the decision to rules, a check on the extraction, and a person for exceptions.

About this exam

Is this exam graded?

Yes. Every question is scored on our server, never in your browser, and you need a score strictly above 80% to pass. The questions are a real, reviewed bank for this course, drawn fresh for each attempt.

Is it only multiple choice?

Yes. This is a multiple-choice knowledge assessment, with at least one question from each chapter in every attempt. Written exercises are separate practice and are not automatically scored. Passing does not establish practical professional competence or independently verify identity.

Is there a time limit?

Yes, 45 minutes, shown as a countdown once you start. If time runs out, whatever you have answered is submitted automatically.

Do I need to download the PDF first?

No. This course's full text is on its own page, so you can read it online and come straight here when you feel ready.

What do you do with my name and email?

We privately record your name, email, request IP, course and privacy acknowledgement for this request. Newsletter consent is separate and optional. See /privacy.html for storage and retention. A passed exam creates a server-stored named certificate. An email confirmation is not identity assurance.

Can I retake it if I fail?

Yes, straight away. Each attempt draws a fresh set of questions and shuffles the answer order, so it will not be identical to your last attempt, and an attempt can only be submitted once.

Is the certificate real?

It is a genuine, server-issued certificate with its own verification page and a unique ID, not a self-declared one. It records that you passed this course's exam on this date. It is not an accredited qualification, a professional licence or proof of employment.