Exam · Transformers: attention explained and implemented

Take the exam and earn your certificate.

A real, server-scored exam for this course. Score strictly above 80% to pass and get a genuine, verifiable certificate.

Before you start

We ask for your name and email first, the same as a PDF download, so we know you are here and your certificate can carry the right name.

We privately record your name, email and request IP for this exam request. Read the privacy notice and retention policy.

Visit Mickai on LinkedIn

Complete the security check before submitting.

Cloudflare Turnstile helps prevent automated submissions. How this check uses data.

You can retake the exam any time. Each attempt draws a fresh set of questions.

Written practice, ungraded

Write your response in your own notes, then compare it with the sample answer. This practice is separate from the scored exam and does not affect your certificate.

Practice 1

Write the shape contract for L = 3, S = 5, dk = 2 and dv = 4: give the Q, K, V, weight and output shapes, and say which dimensions must agree.

Compare with a sample answer

Q is 3 by 2, K is 5 by 2 and V is 5 by 4. Scores and weights are 3 by 5 and the output is 3 by 4. Q and K must share the feature width 2 so that dot products are defined, and K and V must share the 5 source positions; the value width 4 may differ. Different query and source lengths are valid here because the call is noncausal.

A good answer uses the L by dk, S by dk and S by dv convention and names both agreement rules.

Practice 2

For Q and K both [[1, 0], [0, 1]] and V = [[10, 0], [0, 20]], calculate the first noncausal attention row, then explain why a row-sum check would not catch a missing scale.

Compare with a sample answer

The dot products are 1 and 0, so the scaled scores are 1/sqrt(2) (about 0.707107) and 0. The first weight is exp(1/sqrt(2)) / (exp(1/sqrt(2)) + 1), about 0.669762, and the second is about 0.330238. The output is about [6.697615, 6.604769]. Without the scale the first weight becomes about 0.731059, yet both versions sum to one, so only comparison with a known calculation reveals the defect.

A good answer keeps full precision, shows the scale and contrasts the scaled and unscaled weights.

Practice 3

Evaluate the two-position fixture with allow=[[False, True], [False, True]] first with causal=False and then with causal=True, and explain each result.

Compare with a sample answer

With causal=False both queries can see only key one, so each has weights [0, 1] and output [0, 20]. With causal=True the first query cannot see position one and allow excludes position zero, so its effective visibility is empty and the function raises ValueError instead of returning a uniform or zero row.

A good answer combines the allow mask with the causal condition and reports the error case rather than inventing an output.

Practice 4

Trace the third row of X, [1, 1, 0, 0], through both fixed heads, explain the uniform row in head two, and list the operations after head concatenation in order.

Compare with a sample answer

Head one selects features zero and one, giving the query [1, 1]; head two selects features two and three, giving [0, 0]. The zero query has a zero dot product with every key, so head two gives equal weights of one third to its three visible positions. After concatenation the block multiplies by WO, adds X, normalises, multiplies by W1, applies ReLU, multiplies by W2, adds the hidden representation and normalises again.

A good answer gives both projected queries, explains the zero dot products and lists the post-normalisation sequence.

Practice 5

Describe the future-independence test for the complete block and explain why per-row normalisation matters to it.

Compare with a sample answer

The test changes only the last row of X, runs the causal block on the original and altered inputs, and checks that the first two output rows are identical while the third differs. It runs the whole block, including both residual paths and normalisation, because a passing attention function alone would not catch a later step that mixes positions. Normalising within each row keeps that boundary; normalising across positions could let the later row change earlier outputs. The test checks information flow only, not language quality or gradients.

A good answer describes the perturbation, the expected comparison and why a passing attention primitive alone is not enough.

Practice 6

Write a short plan for extending the fixture into a tiny language model: give shifted targets for a four-token example, list the missing trainable components and name three separate checks.

Compare with a sample answer

For [a, b, c, d], use inputs [a, b, c] and targets [b, c, d] with causal diagonal visibility. Add token embeddings, a positional scheme, distinct trainable projections, normalisation parameters as chosen, a vocabulary output layer and a next-token loss with optimisation. First reproduce this fixed forward fixture in the chosen framework, then check gradients on a tiny example, then evaluate a trained model on held-out data. None of these steps has been executed in the current lab, so each needs its own evidence.

A good answer separates what has been executed from proposed work and names each evidence stage.

About this exam

Is this exam graded?

Yes. Every question is scored on our server, never in your browser, and you need a score strictly above 80% to pass. The questions are a real, reviewed bank for this course, drawn fresh for each attempt.

Is it only multiple choice?

Yes. This is a multiple-choice knowledge assessment, with at least one question from each chapter in every attempt. Written exercises are separate practice and are not automatically scored. Passing does not establish practical professional competence or independently verify identity.

Is there a time limit?

Yes, 45 minutes, shown as a countdown once you start. If time runs out, whatever you have answered is submitted automatically.

Do I need to download the PDF first?

No. This course's full text is on its own page, so you can read it online and come straight here when you feel ready.

What do you do with my name and email?

We privately record your name, email, request IP, course and privacy acknowledgement for this request. Newsletter consent is separate and optional. See /privacy.html for storage and retention. A passed exam creates a server-stored named certificate. An email confirmation is not identity assurance.

Can I retake it if I fail?

Yes, straight away. Each attempt draws a fresh set of questions and shuffles the answer order, so it will not be identical to your last attempt, and an attempt can only be submitted once.

Is the certificate real?

It is a genuine, server-issued certificate with its own verification page and a unique ID, not a self-declared one. It records that you passed this course's exam on this date. It is not an accredited qualification, a professional licence or proof of employment.