Write your response in your own notes, then compare it with the sample answer. This practice is separate from the scored exam and does not affect your certificate.
Practice 1
Use three fictional strings: `naïve`, `£10` and `a b `. Predict each string's Unicode-character count, whitespace-split word count and UTF-8 byte count. Then identify which count loses the repeated or trailing spaces when the text is split and joined.
Check your answer against these points:
- Distinguishes Unicode-character count from UTF-8 byte count.
- Counts the examples with their spaces as written.
- Identifies that whitespace splitting can lose exact spacing.
Compare with a sample answer
`naïve`: 5 Unicode characters, 1 word, 6 UTF-8 bytes. `£10`: 3 characters, 1 word, 4 bytes. `a b `: 5 characters, 2 words, 5 bytes. Splitting and joining the last example normalises its spaces, so it does not reproduce the original string.
Ungraded paper practice. Counts are for these supplied strings and UTF-8; the task does not compare commercial tokenisers.
Practice 2
On paper, run three BPE merges for this fictional corpus: `cat` appears 4 times, `cats` 2 times, and `scat` 1 time. Start with single characters, count adjacent pairs weighted by word frequency, and record each winning pair and its count. State one tie rule you would need if two pairs shared the highest count.
Check your answer against these points:
- Weights each adjacent pair by the number of times its word occurs.
- Recounts pairs after each merge rather than choosing the longest string.
- States a deterministic tie rule.
Compare with a sample answer
Initially `c+a` occurs 7 times, `a+t` 7 times, `t+s` 2 times and `s+c` once. With a stated first-counted-pair tie rule, `c+a` wins first. Recount after the merge before choosing the next pair, and repeat for three merges. A different documented tie rule can produce a different ordered merge list.
Ungraded paper practice. The sample illustrates the starting counts and the need to recompute after each merge; learners should show their own ordered recounts.
Practice 3
A teammate runs the same tiny BPE training corpus twice and gets different merge lists. Write a short diagnostic checklist using the source code's behaviour: include pair counting, tie handling and the order in which learned merges are replayed. Suggest one controlled check before changing the implementation.
Check your answer against these points:
- Checks corpus and chunk inputs before attributing a result to the algorithm.
- Mentions the deterministic tie rule and learned merge order.
- Proposes a controlled comparison with the same inputs.
Compare with a sample answer
Check that both runs use the identical corpus and chunk boundaries, that pair counts weight repeated chunks, that ties use the same first-counted rule, and that the merge list is replayed in learned order. First compare the exact corpus and printed first few merges from two clean runs with the same requested merge count.
Ungraded paper practice. This is a reproducibility diagnostic for the supplied toy implementation, not a claim about every tokeniser.
Practice 4
For a fictional message containing `café`, a tab and the typed text `<|end|>`, describe two checks that distinguish exact text round-tripping from a real special-token id. Explain why decoded text alone cannot prove that the end marker was inserted as a special id.
Check your answer against these points:
- Checks equality against the original text, including whitespace.
- Inspects ids separately from their decoded spelling.
- Explains that typed marker text and a special id can look alike after decoding.
Compare with a sample answer
Compare the decoded text with the original string exactly, including the tab. Separately inspect the encoded id sequence and verify that the special id is added only by the document-wrapping function. A typed `<|end|>` can be ordinary text that decodes to the same visible spelling, so printed text alone cannot distinguish the ids.
Ungraded paper practice. Treat the example as a test-design exercise; malformed byte streams and actual model interfaces are outside its scope.
Practice 5
For a fictional model with width 64, compare embedding-table entry counts at vocabulary sizes 256 and 452. Then explain in one or two sentences why the shorter sequence from the larger vocabulary is only one side of the trade-off.
Check your answer against these points:
- Calculates vocabulary size multiplied by width for both cases.
- States at least one sequence-side and one vocabulary-side cost.
- Limits the conclusion to the supplied calculation.
Compare with a sample answer
At width 64, the tables contain 256 x 64 = 16,384 and 452 x 64 = 28,928 numbers. A larger vocabulary can shorten sequences and reduce sequence-related work, while making embedding and output tables larger and giving rare tokens fewer examples. These toy counts alone do not establish the cost or quality of a complete model.
Ungraded paper practice. The calculation uses the course's toy vocabulary sizes and a fictional width; it is not a model price estimate.
Practice 6
Design a five-row probe sheet for one toy tokeniser. Include an ordinary English string, a string with changed spaces, a number, text in another writing system or with a combining mark, and a short code fragment. For each row record the exact input, corpus and tokeniser version, observed token count, and one limit on what the result can show.
Check your answer against these points:
- Includes all five requested input categories.
- Records the tokeniser and corpus conditions beside each observed count.
- States a limit that prevents a broad language or task-performance claim.
Compare with a sample answer
A useful sheet keeps the exact text unchanged in each row and records the same corpus, pre-tokenisation rule and merge count. It reports counts for the specific toy implementation only, notes that five examples are not a language benchmark, and does not infer arithmetic ability from digit counts.
Ungraded paper practice. Use invented or public example text; a small probe set cannot establish general language or model performance.