Question Index

TL;DR — Every interview question the book answers, one line each, linked to the chapter that answers it. Use it as a question bank: sample at random, answer aloud in ninety seconds without notes, and score yourself against the four-axis rubric in The Interview Playbook section 7. If you cannot answer an entry, the link tells you exactly which chapter to reopen.

This is a lookup table, not a chapter to read. Use it two ways. Audit coverage before an interview: scan the Part you expect to be questioned on, mark every question you cannot answer cold, and reopen only those chapters. Practise: work down a Part in order, or sample at random so you do not rehearse in a predictable sequence. Each entry keeps the interviewer’s wording, so reading it aloud is the drill; the strong answer, the common wrong answer, and the follow-up chain are in the linked chapter’s ## 8. Interview drill. A question that genuinely has two different answers — one conceptual, one implementation-level — is listed under both chapters, and both links are worth reopening.

Part 01 — Foundations

• “Explain a Transformer block in 60 seconds.” — The Transformer
• “Why is the residual connection load-bearing?” — The Transformer
• “Why did the Transformer replace the RNN? Be specific.” — The Transformer
• “What does the feed-forward network do that attention can’t?” — The Transformer
• “Which is more expensive, attention or the MLP?” — The Transformer
• “Why layer norm and not batch norm?” — The Transformer
• “The 2017 paper scaled embeddings by sqrt(d) — why?” — The Transformer
• “What would you change in the block for a 2026 model?” — The Transformer
• “Why divide by sqrt(d_k)?” — Attention
• “Why multiple heads instead of one big one?” — Attention
• “Why is attention O(n^2)?” — Attention
• “How does the causal mask work?” — Attention
• “Self-attention vs cross-attention?” — Attention
• “What is the difference between your implementation and nn.MultiheadAttention?” — Attention
• “Why does a Transformer need positional encoding at all?” — Positional Encoding
• “Walk me through the sinusoidal encoding.” — Positional Encoding
• “Explain RoPE to me as if I have 90 seconds.” — Positional Encoding
• “Why is RoPE applied to q and k but not v?” — Positional Encoding
• “I trained a 2k-context model and I need 32k. What do you do?” — Positional Encoding
• “ALiBi vs RoPE — why did RoPE win?” — Positional Encoding
• “Does RoPE change the 1/sqrt(d_k) scaling?” — Positional Encoding
• “Why does everyone use pre-norm instead of post-norm?” — Modern Architecture Deltas
• “What does RMSNorm drop relative to LayerNorm, and why is that allowed?” — Modern Architecture Deltas
• “Derive the 8d/3 hidden size.” — Modern Architecture Deltas
• “Why does adding a gate improve the FFN?” — Modern Architecture Deltas
• “How does GQA reduce memory, exactly?” — Modern Architecture Deltas
• “Is FlashAttention approximate?” — Modern Architecture Deltas
• “Then what limits long-context inference?” — Modern Architecture Deltas
• “What is MoE and when is it the wrong choice?” — Modern Architecture Deltas
• “Why can’t you generate with BERT?” — BERT vs GPT vs Encoder-Decoder
• “Why 15% masking?” — BERT vs GPT vs Encoder-Decoder
• “Why did decoder-only win?” — BERT vs GPT vs Encoder-Decoder
• “Which objective trains on the most positions per sequence?” — BERT vs GPT vs Encoder-Decoder
• “Why are encoder models still used for embeddings?” — BERT vs GPT vs Encoder-Decoder
• “What is cross-attention, and which axis is masked?” — BERT vs GPT vs Encoder-Decoder
• “Why does T5 use span corruption instead of MLM?” — BERT vs GPT vs Encoder-Decoder
• “Can you fine-tune BERT as a generator?” — BERT vs GPT vs Encoder-Decoder

Part 02 — Pretraining and Fine-Tuning

• “Where does pretraining data come from, and what do the filters remove?” — Pretraining Pipeline
• “Why 6ND, and how do you turn it into GPU-hours?” — Pretraining Pipeline
• “Why did Chinchilla disagree with Kaplan et al.?” — Pretraining Pipeline
• “Why do modern models use thousands of tokens per parameter?” — Pretraining Pipeline
• “Explain MinHash and LSH. What do the parameters control?” — Pretraining Pipeline
• “What is the most expensive part of the pipeline?” — Pretraining Pipeline
• “Why does SFT change behaviour so much with only a thousand examples?” — Supervised Fine-Tuning (SFT)
• “Why mask the prompt tokens in the loss?” — Supervised Fine-Tuning (SFT)
• “What is a chat template, and why are the role markers special tokens?” — Supervised Fine-Tuning (SFT)
• “How many epochs should SFT run?” — Supervised Fine-Tuning (SFT)
• “Does SFT teach the model new facts?” — Supervised Fine-Tuning (SFT)
• “You enable packing and the model starts producing off-topic continuations. What happened?” — Supervised Fine-Tuning (SFT)
• “SFT vs RLHF/DPO — which do you reach for?” — Supervised Fine-Tuning (SFT)
• “Why is B initialised to zero?” — LoRA and QLoRA
• “Why divide by r and not just use alpha?” — LoRA and QLoRA
• “How many parameters does LoRA train on a 7B model?” — LoRA and QLoRA
• “What does QLoRA actually quantise, and what stays in 16-bit?” — LoRA and QLoRA
• “When is LoRA the wrong choice?” — LoRA and QLoRA
• “You merged a QLoRA adapter into the 4-bit base and quality dropped. Why?” — LoRA and QLoRA
• “Your adapter trains but the loss plateaus above the baseline. First three things you check?” — LoRA and QLoRA
• “What does the zero-point do, and when do you set it to zero?” — Quantization
• “Why is 4-bit hard for LLMs when 8-bit is free?” — Quantization
• “How is NF4 different from INT4?” — Quantization
• “QLoRA, GPTQ, AWQ — which would you pick and why?” — Quantization
• “Does 4-bit quantization make inference 4x faster?” — Quantization
• “You quantized to 4-bit: perplexity is fine but math and code are much worse. What do you do?” — Quantization
• “When must you pay for QAT?” — Quantization

Part 03 — Alignment and Reinforcement Learning

• “What does each stage of RLHF contribute, and what happens if you remove it?” — RLHF: The Three Stages
• “Where does the preference data come from, exactly?” — RLHF: The Three Stages
• “Why is the KL penalty mandatory?” — RLHF: The Three Stages
• “What is the alignment tax, and how do you fix it?” — RLHF: The Three Stages
• “InstructGPT calls this a bandit. So why does your PPO code have a value function and a discount factor?” — RLHF: The Three Stages
• “Would you run RLHF for a coding assistant?” — RLHF: The Three Stages
• “Derive the reward-model loss from the preference assumption.” — Reward Modeling
• “Why only the difference of rewards?” — Reward Modeling
• “The RM’s validation accuracy peaked at epoch 2. Do you stop?” — Reward Modeling
• “How do you detect reward hacking?” — Reward Modeling
• “Would an ensemble of reward models solve this?” — Reward Modeling
• “You have 500 preference pairs and a 7B model. Now what?” — Reward Modeling
• “What is the difference between the RM and DPO’s implicit reward?” — Reward Modeling
• “Name a failure of the RM that is not over-optimisation.” — Reward Modeling
• “Why clip the ratio at all?” — PPO for LLMs
• “Why min and not max?” — PPO for LLMs
• “Where exactly does the KL term enter?” — PPO for LLMs
• “Why does PPO need a critic, and what does it cost?” — PPO for LLMs
• “lambda = 0 versus lambda = 1?” — PPO for LLMs
• “Why is gamma = 1 here when RL always uses gamma < 1?” — PPO for LLMs
• “Is PPO still what people use in 2026?” — PPO for LLMs
• “What exactly does GRPO change versus PPO?” — GRPO
• “Why is the group mean a valid baseline?” — GRPO
• “How much memory does removing the critic save?” — GRPO
• “What does GRPO cost instead?” — GRPO
• “Why did it matter for reasoning models?” — GRPO
• “What are GRPO’s known biases?” — GRPO
• “Why not just set G very large?” — GRPO
• “Derive DPO.” — DPO
• “What is the implicit reward, and why is it called a reward?” — DPO
• “Why does beta appear twice and is it the same beta?” — DPO
• “What does the DPO gradient actually do?” — DPO
• “Does DPO beat PPO?” — DPO
• “Why can you skip the reward model but not the reference model?” — DPO
• “When would you choose DPO over GRPO in 2026?” — DPO
• “Why do you need an advantage instead of just using the return?” — Monte Carlo, TD, and Search
• “Monte Carlo or TD?” — Monte Carlo, TD, and Search
• “Walk me through GAE. What is lambda doing?” — Monte Carlo, TD, and Search
• “Why is gamma = 1 in LLM RL?” — Monte Carlo, TD, and Search
• “What is importance sampling for, and what is the problem with it?” — Monte Carlo, TD, and Search
• “Derive the PUCT formula’s terms and tell me what happens if I remove one.” — Monte Carlo, TD, and Search
• “Why does MCTS beat minimax with a neural evaluator?” — Monte Carlo, TD, and Search
• “Should you use MCTS for LLM reasoning?” — Monte Carlo, TD, and Search
• “Is RLVR teaching models new reasoning, or just sharpening what they had?” — Monte Carlo, TD, and Search
• “Why does generation dominate RL training time when it is only a forward pass?” — RL Frameworks
• “Why can’t you just use TRL’s PPOTrainer?” — RL Frameworks
• “How do updated weights reach the rollout engine?” — RL Frameworks
• “Colocated or disaggregated — which do you choose?” — RL Frameworks
• “Why is Ray in every one of these frameworks?” — RL Frameworks
• “What actually changed by 2026?” — RL Frameworks

Part 04 — Inference and Serving

• “Show me the memory arithmetic for serving a 70B at long context.” — KV Cache
• “Why is the factor 2 there, and not 3?” — KV Cache
• “Does GQA reduce the compute or the memory?” — KV Cache
• “Why is decode memory-bound while prefill is compute-bound?” — KV Cache
• “Can batching make decode compute-bound?” — KV Cache
• “What breaks at 128k context, and what do you do about it?” — KV Cache
• “Why was naive KV-cache management wasting 60-80% of the cache?” — Serving: vLLM and PagedAttention
• “What does continuous batching buy you over static batching?” — Serving: vLLM and PagedAttention
• “How does a block table not destroy performance?” — Serving: vLLM and PagedAttention
• “How do you choose block size?” — Serving: vLLM and PagedAttention
• “Recompute or swap on preemption?” — Serving: vLLM and PagedAttention
• “Throughput or goodput?” — Serving: vLLM and PagedAttention
• “What is the difference between a task and an actor?” — Ray
• “What is an ObjectRef and why is it not just a future?” — Ray
• “Where do Ray objects actually live, and when does a copy happen?” — Ray
• “What is a placement group, and why must a bundle fit on one node?” — Ray
• “I made a placement group with pure {"GPU": 1} bundles and my actor hangs forever. Why?” — Ray
• “My task raised an exception and Ray didn’t retry it. Why not?” — Ray
• “I lost a worker node. Which of my objects are gone?” — Ray
• “Why do veRL, OpenRLHF, and SkyRL all use Ray instead of torchrun?” — Ray
• “Does Ray Serve LLM give me anything over plain vLLM?” — Ray
• “How does a KV-aware router decide which replica is least loaded?” — Ray

Part 05 — Agents

• “What is an agent, precisely?” — What an Agent Actually Is
• “Agent or workflow?” — What an Agent Actually Is
• “Why is the context window the binding constraint?” — What an Agent Actually Is
• “How do you guarantee an agent terminates?” — What an Agent Actually Is
• “Where do agents actually fail in production?” — What an Agent Actually Is
• “What makes a tool safe to put in an agent?” — What an Agent Actually Is
• “Why does interleaving reasoning and acting beat either alone?” — ReAct
• “How does the loop terminate?” — ReAct
• “Is ReAct better than chain-of-thought?” — ReAct
• “Why does ReAct hallucinate less than CoT?” — ReAct
• “What is ReAct’s most common failure?” — ReAct
• “How does ReAct relate to function calling?” — ReAct
• “When would you not use ReAct at all?” — ReAct
• “What does strict: true actually guarantee?” — Tool / Function Calling
• “Why are arguments a string instead of a JSON object?” — Tool / Function Calling
• “How do you stop the model calling a tool it should not?” — Tool / Function Calling
• “How do you handle a malformed call?” — Tool / Function Calling
• “When do you retry, and when do you not?” — Tool / Function Calling
• “Why is parallel_tool_calls sometimes off?” — Tool / Function Calling
• “Tool calling or structured outputs?” — Tool / Function Calling
• “How does the model know which tool to pick?” — Tool / Function Calling
• “What does LangChain actually give you?” — LangChain
• “What is LCEL?” — LangChain
• “Why did the agent API change twice?” — LangChain
• “When would you not use LangChain?” — LangChain
• “How do you make an agent stop?” — LangChain
• “Where does the money go in an agent?” — LangChain
• “Why a graph instead of a while loop?” — LangGraph
• “What is a reducer, exactly?” — LangGraph
• “What is a super-step, and why should I care?” — LangGraph
• “Walk me through interrupt.” — LangGraph
• “Does replay read from cache?” — LangGraph
• “Where does the memory go as a thread grows, and why is the default recursion limit a problem?” — LangGraph
• “Should you split this into multiple agents?” — Multi-Agent Patterns
• “What does multi-agent actually buy you?” — Multi-Agent Patterns
• “You get a 90% improvement from multi-agent. Is it real?” — Multi-Agent Patterns
• “How do errors behave as you add steps?” — Multi-Agent Patterns
• “Why does majority voting help sometimes and not others?” — Multi-Agent Patterns
• “What’s the most common multi-agent failure?” — Multi-Agent Patterns
• “Why does the lead agent block the whole system?” — Multi-Agent Patterns
• “The conversation exceeds the window. What do you do?” — Memory and Context Management
• “Why not just use a bigger context window?” — Memory and Context Management
• “What is the cost of compaction?” — Memory and Context Management
• “How does prompt caching change your context strategy?” — Memory and Context Management
• “A summary said 300 tests passed but the real number was 347. How do you prevent that?” — Memory and Context Management
• “What is the difference between short-term and long-term memory in an agent?” — Memory and Context Management
• “What problem does MCP solve that ad-hoc tool code does not?” — MCP (Model Context Protocol)
• “What is the current revision, and what changed?” — MCP (Model Context Protocol)
• “Tools, resources, prompts — what is the difference?” — MCP (Model Context Protocol)
• “How is a tool error reported?” — MCP (Model Context Protocol)
• “Where is the security boundary, and why does removing sessions matter?” — MCP (Model Context Protocol)
• “When would you not use MCP?” — MCP (Model Context Protocol)
• “How do you stop prompt injection?” — Guardrails
• “Why is a 95%-effective guardrail not good enough?” — Guardrails
• “Which actions need human approval, and where does a sandbox belong?” — Guardrails
• “Which OWASP entries matter most for agents?” — Guardrails

Part 06 — Retrieval-Augmented Generation

• “Our RAG system hallucinates. What do you do?” — The RAG Pipeline
• “How do you know your retriever is good enough?” — The RAG Pipeline
• “Why two stages — why not just retrieve the top 5 with the best model?” — The RAG Pipeline
• “When would you not use RAG?” — The RAG Pipeline
• “Your system is correct but users do not trust it. Why?” — The RAG Pipeline
• “How do you handle a corpus that updates hourly?” — The RAG Pipeline
• “How do you pick a chunk size?” — Chunking
• “What does overlap actually buy, quantitatively?” — Chunking
• “Too-small chunks — what actually breaks?” — Chunking
• “How does late chunking differ from just using overlapping chunks?” — Chunking
• “How would you handle a corpus of tables?” — Chunking
• “Your manager says to double the overlap because recall is low. What do you say?” — Chunking
• “Why can’t we just use the LLM’s embeddings for retrieval?” — Embedding Models
• “Cosine, dot product, or Euclidean — does it matter?” — Embedding Models
• “What does Matryoshka truncation do to the geometry?” — Embedding Models
• “What is the contrastive objective optimising, in words?” — Embedding Models
• “How do you use MTEB?” — Embedding Models
• “When is fine-tuning an embedding model worth it?” — Embedding Models
• “Why is BM25 still around when we have embedding models?” — Retrieval: Hybrid Search and Reranking
• “Why RRF rather than averaging normalised scores?” — Retrieval: Hybrid Search and Reranking
• “How many candidates should you rerank?” — Retrieval: Hybrid Search and Reranking
• “Cross-encoder vs bi-encoder — what is the architectural difference, and why does it cost more?” — Retrieval: Hybrid Search and Reranking
• “Your hybrid search made things worse. What happened?” — Retrieval: Hybrid Search and Reranking
• “A user asks a follow-up question and retrieval returns junk. Where do you look first?” — Query Transformation
• “What does HyDE actually do, and why can a wrong hypothetical document still retrieve the right real one?” — Query Transformation
• “You add multi-query expansion with m=4 and recall barely moves. Why?” — Query Transformation
• “Why is multi-hop retrieval not just a better retriever?” — Query Transformation
• “What is the cost, and when do you turn it on?” — Query Transformation
• “How would you evaluate a rewrite step honestly?” — Query Transformation
• “How do you know a change helped?” — RAG Evaluation
• “Explain nDCG to me.” — RAG Evaluation
• “An LLM judge says the new system is better. What do you check?” — RAG Evaluation
• “Your retriever recall@10 is 0.92 and users still say the answers are wrong. What now?” — RAG Evaluation
• “How large does your eval set need to be?” — RAG Evaluation

Part 07 — Applications

• “How do you choose the confidence threshold?” — Intent Recognition and Routing
• “Your classifier says 0.92. What does that mean?” — Intent Recognition and Routing
• “When would you not use an LLM router?” — Intent Recognition and Routing
• “The router is 95% accurate and the handlers are 95% accurate. Is the system 95% accurate?” — Intent Recognition and Routing
• “Which metric for text-to-SQL?” — NL2SQL (Text-to-SQL)
• “Where does generated SQL go wrong?” — NL2SQL (Text-to-SQL)
• “How do you make it safe to run against production?” — NL2SQL (Text-to-SQL)
• “The schema is 3,000 columns. What do you do?” — NL2SQL (Text-to-SQL)
• “You add execution feedback as a repair loop. How much does it help?” — NL2SQL (Text-to-SQL)
• “How do you evaluate correctness when the database is dirty?” — NL2SQL (Text-to-SQL)
• “How would you evaluate a coding assistant?” — AI-Assisted Coding
• “What does pass@k actually measure?” — AI-Assisted Coding
• “The tests pass. Is the patch correct?” — AI-Assisted Coding
• “Where would you not use an AI coding assistant?” — AI-Assisted Coding
• “You have a failing test and an agent that can retry. How many rounds?” — AI-Assisted Coding
• “How do you keep the review process sane?” — AI-Assisted Coding

Part 08 — Handwritten Code

• “Why the transpose?” — Multi-Head Attention from Scratch
• “Why divide by sqrt(d_k) and not d_k?” — Multi-Head Attention from Scratch
• “How do you apply the causal mask?” — Multi-Head Attention from Scratch
• “What does .contiguous() do here?” — Multi-Head Attention from Scratch
• “How would you test it?” — Multi-Head Attention from Scratch
• “How does this differ from nn.MultiheadAttention?” — Multi-Head Attention from Scratch
• “Add GQA.” — Multi-Head Attention from Scratch

Part 09 — Interview

These are the meta-questions: they are about your answer rather than about a topic, and they are the ones that decide whether the rest of the index helps you. The templates they are drawn from are in The Interview Playbook.

• “Why did you choose RAG over fine-tuning?” — The Interview Playbook
• “How would you evaluate that?” — The Interview Playbook
• “Your system is giving wrong answers. Where do you look?” — The Interview Playbook
• “What is the biggest cost in your design?” — The Interview Playbook
• “Tell me about a failure.” — The Interview Playbook
• “What are you weakest on?” — The Interview Playbook

Coverage audit

Using this with the rubric. Score each sampled question on the four axes from The Interview Playbook section 7, one point each: did you name the problem before the technique; did you name a constraint or a rejected alternative; did you give one number, correctly labelled derived, cited, or example; and did you say how you would know it worked. Anything below three gets rewritten, not reread — reopen the linked chapter, rewrite the answer, and say it aloud again. A candidate who scores four out of four on eight of ten sampled questions, and can name the chapter to reopen for the other two, is prepared. That is the bar this book is written to.