Write your response in your own notes, then compare it with the sample answer. This practice is separate from the scored exam and does not affect your certificate.
Practice 1
Explain why the row vector with shape (3,) can be added to a matrix with shape (2, 3), and why subtracting a row (3,) from a column (3, 1) produces a (3, 3) result.
Check your answer against these points:
- Compares dimensions from the right.
- Treats a missing dimension as size 1.
- Explains why the column and row create every pairing in a 3 by 3 result.
Compare with a sample answer
Compare dimensions from the right. For (2, 3) and (3,), the last dimensions match and the missing leading dimension acts like 1, so the row is reused for both rows. For (3, 1) and (3,), the shapes act like (3, 1) and (1, 3); each column value pairs with each row value, producing (3, 3).
Ungraded paper practice. It checks shape reasoning from the course examples and does not run code.
Practice 2
Using the course's mean-squared-error gradient, work one update by hand for x = [0, 1], y = [2, 5], starting at w = 1 and b = 0 with learning rate 0.1. Show the errors, loss, two slopes, updated parameters and new loss.
Check your answer against these points:
- Uses prediction minus target for the errors.
- Computes the weight and bias slopes separately.
- Moves each parameter against its slope and recomputes the loss.
Compare with a sample answer
The predictions are [0, 1], so errors prediction minus target are [-2, -4]. The mean squared error is (4 + 16) / 2 = 10. The weight slope is mean(2 * error * x) = (0 - 8) / 2 = -4; the bias slope is mean(2 * error) = (-4 - 8) / 2 = -6. The update gives w = 1.4 and b = 0.6. New predictions are [0.6, 2.0], giving errors [-1.4, -3] and loss (1.96 + 9) / 2 = 5.48.
Ungraded paper practice. It applies the source's stated error, loss, gradient and update formulas to a small supplied example.
Practice 3
Classify the supplied 50-step runs at learning rates 0.01, 0.3 and 1.1. Which is slow, which gives the best line fit in this example, and which overshoots? State one limit on generalising the best rate.
Check your answer against these points:
- Identifies 0.01 as too slow in the supplied run.
- Identifies 0.3 as the best of these rates for this line and data.
- Identifies the 1.1 run as diverging by overshooting and notes the dataset-specific limit.
Compare with a sample answer
At 0.01 the loss remains 2.104, moving in the right direction but too slowly. At 0.3 it reaches 0.004496, the best a line can do for these data. At 1.1 the loss grows to 4.55e+08 and the parameters become very large in magnitude, showing repeated overshooting. The course says the best rate and the edge are for this data alone, so the number is not a universal setting.
Ungraded paper practice. These are the supplied course runs and do not establish a learning rate for another dataset or model.
Practice 4
A learner calls loss.backward() twice before an optimiser update. Explain what happens to the stored gradients, why the second values are doubled in the course output, and when the loop clears them.
Check your answer against these points:
- Says gradients accumulate rather than reset automatically.
- Connects a repeated backward pass to the doubled shown values.
- Places opt.zero_grad() before calculating the next batch loss.
Compare with a sample answer
Backward gradients accumulate in the parameter .grad values. Calling backward a second time without clearing them adds the same gradients again, which is why the shown values double. The training loop calls opt.zero_grad() before its forward and loss calculation for the next update.
Ungraded paper practice. The task describes gradient accumulation and the loop's clearing step from the supplied source.
Practice 5
For the supplied XOR network, training loss continues to fall while validation loss is 0.269 near epoch 400 and later reaches 0.364 at epoch 2000. Which checkpoint would you keep, what pattern does this show, and what remedy does the course name?
Check your answer against these points:
- Chooses the checkpoint near epoch 400 using validation loss.
- Identifies the split between falling training loss and rising validation loss as overfitting.
- Names early stopping or another remedy listed by the course.
Compare with a sample answer
Keep the checkpoint around epoch 400 because it has the lower reported validation loss. The later rise in validation loss while training loss keeps falling is overfitting. The course names early stopping at the best validation epoch as a remedy; it also mentions adding data or shrinking the model.
Ungraded paper practice. It uses the supplied experiment to distinguish training fit from validation performance; this small synthetic run is not a general performance claim.
Practice 6
Compare the workbook's toy training loop with the cited frontier-scale reports on data, hardware and evaluation. Give one specific figure or detail for each and explain the limit the course puts on this comparison.
Check your answer against these points:
- Compares data scale, hardware and evaluation using details from the cited table.
- Keeps workbook and research-report examples distinct.
- States that the toy is an illustration and the reports are specific historical research runs.
Compare with a sample answer
The workbook uses 220 invented points, while the cited reports describe much larger token sets, including about 180 billion tokens for OPT. The workbook runs on one CPU for seconds, while the OPT report used 992 GPUs. The workbook checks 50 held-out points, while the OPT report lists 16 language tasks plus hate-speech, stereotype and toxicity benchmarks. These are descriptions of particular published research runs; the toy teaches the idea, not frontier training practice or the properties of every current product.
Ungraded paper practice. The figures are the source's descriptions of particular reports, not current vendor or product claims.