Exam · Build a tiny language model from scratch

Take the exam and earn your certificate.

A real, server-scored exam for this course. Score strictly above 80% to pass and get a genuine, verifiable certificate.

Before you start

We ask for your name and email first, the same as a PDF download, so we know you are here and your certificate can carry the right name.

We privately record your name, email and request IP for this exam request. Read the privacy notice and retention policy.

Visit Mickai on LinkedIn

Complete the security check before submitting.

Cloudflare Turnstile helps prevent automated submissions. How this check uses data.

You can retake the exam any time. Each attempt draws a fresh set of questions.

Written practice, ungraded

Write your response in your own notes, then compare it with the sample answer. This practice is separate from the scored exam and does not affect your certificate.

Practice 1

Before running any code, write a short model card for the lantern-workshop experiment: state the context length and output units, calculate the parameter count for V = 28, give the document counts in each split and name two claims the corpus cannot support.

Compare with a sample answer

The context is one previous character, plus an input-only beginning context for the first character. Outputs are the character tokens, <eos> and <unk>. The table has (28+1)*28 = 812 stored scores. The split is 12 training, 3 validation and 3 test documents, fixed before the vocabulary is built. Because every note comes from one small fictional workshop setting, the experiment cannot establish general language competence or frontier-model training costs.

A good answer separates what the experiment fixes (context, outputs, table size, split) from what it cannot show.

Practice 2

For the training documents ab and a, list every target event, give the beginning, a and b count rows over [<eos>, <unk>, a, b], and explain why no b-to-a pair appears.

Compare with a sample answer

The vocabulary is [<eos>, <unk>, a, b] and the beginning context has input ID 4. There are five targets: beginning-to-a twice, a-to-b once, a-to-end once and b-to-end once. The beginning row is [0, 0, 2, 0], the a row is [1, 0, 0, 1] and the b row is [1, 0, 0, 0]; the end and unknown context rows are all zero. There is no b-to-a pair because each document starts from a fresh beginning context and the end token never connects to the next document.

A good answer resets the context at each document and gives each document its own end target.

Practice 3

Using the beginning row of the ab and a fixture at zero logits, compute N, T and the four probabilities, derive the gradient and the updated row with learning rate 1, and explain why adding the gradient would be the wrong direction.

Compare with a sample answer

N = 2 because the beginning context appears twice, T = 5 targets in total, and every probability is 0.25 at zero logits. Using (N*p - C)/T with counts [0, 0, 2, 0] gives [0.1, 0.1, -0.3, 0.1]. Subtracting it with rate 1 gives [-0.1, -0.1, 0.3, -0.1], which raises the probability of a to exp(0.3)/(exp(0.3)+3*exp(-0.1)), about 0.332120. Adding the gradient would lower the score of a, the target that was observed, and the one-update test checks that the fixture loss really falls.

A good answer applies (N*p - C)/T, subtracts the gradient and checks the direction against the observed target.

Practice 4

Describe what must stay fixed when comparing generated text between runs, explain the two stopping reasons, and identify one checkpoint check and one input boundary that check does not protect.

Compare with a sample answer

Keep the vocabulary order, weights, sampler code, seed, character cap and runtime fixed. The sampler stops either when it draws <eos> or when it reaches the explicit character cap, and a cap stop is not evidence that a sentence finished. restore() rejects a weight row whose length does not match the vocabulary, but its two-million-character string check does not bound an earlier file read. The source files and training configuration are kept separately because the checkpoint stores only inference state.

A good answer separates inference state and sampling settings from training configuration and states the limits of the string-size check.

Practice 5

Write an honest result note for the fixed run: give the training, validation and test target counts and final losses, report the test perplexity with its definition, and explain why it cannot be compared directly with a differently tokenised model.

Compare with a sample answer

The fixed run uses 535 training, 138 validation and 131 test targets, each count including document ends. Final losses are 2.031755 (training), 2.076519 (validation) and 1.967426 nats per target (test). Test perplexity is exp(mean test loss), printed as 7.152244, under this exact character vocabulary, split and end-token rule. A model with word or byte tokens has a different denominator and prediction task, so the numbers are not directly comparable. Include one printed sample unchanged and note that the model sees only the previous character.

A good answer reports measured numbers exactly and scopes perplexity to its token and end-target definition.

Practice 6

Explain what this one-character model cannot learn about language, why simply lengthening the character context of a direct score table is costly, and what a trainable attention model would need that the score table lacks.

Compare with a sample answer

The model predicts from the previous character alone, so it gives the same next-character distribution after any two histories ending in the same character; it cannot track an earlier subject, reason about the workshop or answer questions from the notes. A longer character context would distinguish more histories, but a direct table grows rapidly as context combinations increase. A trainable attention model would need embeddings, positions, projections, a vocabulary head, gradient propagation and an aligned next-token objective, none of which is present in this score table.

A good answer names concrete context limits and lists the missing components without implying they are hidden in the table.

About this exam

Is this exam graded?

Yes. Every question is scored on our server, never in your browser, and you need a score strictly above 80% to pass. The questions are a real, reviewed bank for this course, drawn fresh for each attempt.

Is it only multiple choice?

Yes. This is a multiple-choice knowledge assessment, with at least one question from each chapter in every attempt. Written exercises are separate practice and are not automatically scored. Passing does not establish practical professional competence or independently verify identity.

Is there a time limit?

Yes, 45 minutes, shown as a countdown once you start. If time runs out, whatever you have answered is submitted automatically.

Do I need to download the PDF first?

No. This course's full text is on its own page, so you can read it online and come straight here when you feel ready.

What do you do with my name and email?

We privately record your name, email, request IP, course and privacy acknowledgement for this request. Newsletter consent is separate and optional. See /privacy.html for storage and retention. A passed exam creates a server-stored named certificate. An email confirmation is not identity assurance.

Can I retake it if I fail?

Yes, straight away. Each attempt draws a fresh set of questions and shuffles the answer order, so it will not be identical to your last attempt, and an attempt can only be submitted once.

Is the certificate real?

It is a genuine, server-issued certificate with its own verification page and a unique ID, not a self-declared one. It records that you passed this course's exam on this date. It is not an accredited qualification, a professional licence or proof of employment.