Part 08 — Handwritten Code
What you write on a whiteboard.
Everything in this part is something an interviewer can ask you to produce, live, with no autocomplete. The chapters are therefore written as drills: the problem statement, the shape of a correct answer, the details that earn or lose points, and the mistakes that are common enough to be a signal.
- Multi-Head Attention from Scratch — the single
most-asked implementation question. Includes the masking convention, the
numerical-stability trap, the
.contiguous()trap, and a compact reference implementation you can reproduce from memory.
How to practise: write it out, then diff against the reference. If you cannot explain why each line is there — not what it does, why it is there — you have memorised it rather than learned it, and a follow-up question will find that out.
Depends on: Part 01 for the mechanism and Part 04 for why its shape matters when served.
Where this leads: Part 09 turns all of the above into answers.