The Interview Playbook

TL;DR — Knowing the material and passing the interview are different skills. The interview for an AI-application role is usually project-driven: the interviewer picks one thing off your résumé or out of the job description and drills until they find your depth. The structure that survives drilling is problem → constraint → options → decision → metric → failure: what you were trying to do, what made it hard, what you considered, what you chose and why, how you knew it worked, and how it broke. Answer at the decision level, not the tool level — “I used LangChain” is not an answer, “I used a state graph because I needed to resume mid-flow after a human approval” is. This chapter gives the templates, the decision ladder for the “how would you build X” question, and the rubric to practise against.

1. Why this exists

Everything in Parts 01–06 is knowledge. An interview is a performance of knowledge under time pressure, against an interviewer whose goal is to find the edge of your understanding. Those are different failure modes:

The knowledge failure is not knowing what LoRA is.

The interview failure is knowing what LoRA is and answering “LoRA freezes the weights and trains low-rank matrices, so it’s more memory-efficient”, which invites the follow-up “why is it more memory-efficient — which part of the memory?” and produces silence.

The second failure is much more common, and it is entirely fixable, because it is structural: the answers that invite no follow-ups are the ones built from decisions rather than definitions.

What the interviewer is actually measuring. For an AI-application engineer, three things, in roughly this order:

1. Can you scope a problem? Given a vague request (“we want an AI assistant for our support docs”), can you turn it into a system with an evaluation criterion?
2. Can you defend a trade-off? Every choice in this book has a cost. Can you name what you gave up?
3. Do you know how you would know it worked? This is the question that separates candidates, and the one most people have no answer for.

Depth of theory matters most as evidence for those three — which is why the rest of this book is written the way it is.

2. How it works

2.1 The six-part answer

Any answer that describes a system — yours, or a hypothetical one — should have these six parts. They take about ninety seconds in total.

Two parts do most of the work. Constraint is what makes the rest of the answer necessary: if there is no constraint, there is no engineering, only configuration. Options is what proves you chose rather than pattern-matched.

Contrast:

Weak (tool-level). “For the support assistant I used RAG with a vector database and LangChain, and it worked pretty well.”

Strong (decision-level). “Support answers had to be grounded in our own docs and cite a source, because a wrong answer was worse than no answer. The docs were small — about 400 pages — so the first option was to put them straight in the context window; that was ruled out because it cost roughly 100k tokens per request (example) and blew the latency budget. Fine-tuning was ruled out because the docs changed weekly and we had no labelled answer set. So: retrieval over chunked docs with a citation requirement in the prompt. We measured answer faithfulness against a 120-question set we labelled by hand, and retrieval recall@10, which we tracked separately — because our first version had a faithfulness problem that turned out to be a chunking problem. It still fails on questions that require joining two documents, which is what I would fix next.”

The second answer is longer but not padded: every sentence either names a constraint, a rejected option, or a measurement.

2.2 The four-layer answer

For a conceptual question (“explain LoRA”), answer in layers and stop when the interviewer stops asking. This maps directly onto how the chapters in this book are structured.

1. Definition — one sentence, no follow-up bait.
2. Mechanism — how it actually works. This is where the formula terms come in: name each one and say what it does.
3. Implementation — the parameters that matter and their realistic ranges.
4. Trade-off — what it costs, and when you would not use it.

Most candidates live entirely in layer 1. The mistake is not going deeper voluntarily; it is stopping at layer 1 and then being unable to go deeper when asked. Practise layers 1→4 as one continuous ninety-second answer.

2.3 The “how would you build X” decision ladder

This is the single most common design question, in some form. Walk the ladder in order and stop at the first rung that satisfies the requirement. The point of the ladder is that it is ordered cheapest and most reliable first — and saying so out loud is itself part of the answer.

Three things to say while walking it:

• Facts live at rung 3, form lives at rung 4. This is the most useful sentence in the whole framework. Fine-tuning to teach a model facts is a common and expensive mistake; retrieval to teach a model a house style is similarly wasteful.
• Each rung needs the previous rung’s evaluation. You cannot tell whether fine-tuning helped if you never measured the prompt baseline.
• The ladder is also the debugging order. When the system is wrong, check the rungs below before blaming the one above — see §5.

2.4 When you do not know

There are exactly three acceptable moves, and one unacceptable one.

Acceptable:

1. Say what you do know. “I haven’t worked with QLoRA’s NF4 quantization directly, but the way it’s described, it quantizes the frozen base weights to 4-bit normal-float and keeps the LoRA adapters in higher precision — so the memory saving comes from the frozen weights, and the adapters stay trainable.” You have now demonstrated understanding of a thing you claimed not to know. This scores well.
2. Reason from first principles, out loud, and flag the uncertainty. “I don’t know the exact number, but I can bound it: the KV cache is 2 bytes per element per key and value, so for 32 layers at 4096 width it’s roughly half a megabyte per token — call it a gigabyte per two thousand tokens.” Then check the assumption with them.
3. Say you don’t know. Briefly, without apology, and move on.

Unacceptable: inventing a mechanism, a number, or a library feature. An experienced interviewer has heard it before, will ask one follow-up, and every subsequent answer loses credibility. The cost of “I don’t know” is one point; the cost of a bluff is the interview.

2.5 Talking about a project you have not built

There is a specific situation worth handling explicitly: you understand a technique deeply but have no production project behind it yet. Two honest options, both of which score:

• Frame it as design, not history. “I haven’t deployed this, but here is how I’d approach it, and here’s the part I’d expect to be hard.” Interviewers routinely ask candidates to design something they have never built — that is what a system design round is.
• Frame it as an exercise you ran. “I implemented multi-head attention from scratch and verified it against scaled_dot_product_attention; the first version passed the equivalence test and failed a causality test, so I learned the mask ordering the hard way.” This is a real, defensible story with a real detail in it.

What you must not do is present an example number as your own measurement. If asked “how much did it improve?”, the answer is either a real number you have, or “I don’t have production numbers for this; here’s the ablation I’d run and the metric I’d use.” Every number in this book is labelled (example) for exactly this reason.

3. Implementation

3.1 Answer templates you can actually rehearse

“Walk me through a project.”

The situation was [user-facing problem]. The hard part was [constraint — pick the sharpest one]. I considered [option A, option B, option C]. I chose [decision] because [which constraint it satisfies], accepting [the cost]. We measured [metric] against [baseline]; it went from [X] to [Y] (example — replace with your own). The thing that still breaks is [failure], and the next thing I’d try is [specific fix].

“Why did you use X?” The answer is never “because it’s popular” or “because it’s the standard”. It is: which requirement did the alternatives fail? Name at least one alternative and the specific way it failed.

“How would you evaluate this?” Two levels, always: an offline set you own and can iterate against (recall@k, faithfulness, exact-match on a labelled set), and an online signal you would actually watch in production (task completion, escalation rate, thumbs-down rate). Then say the number you would consider a success, and that deciding it before building is the point.

“How would you make it faster / cheaper?” Name the budget first: latency (time-to-first-token vs total), throughput, or cost per request — they trade against each other. Then the levers, cheapest first: prompt caching, smaller model for easy traffic, retrieval to shorten context, quantisation, then batching, then speculative decoding.

“What would you do differently?” Have one real answer. “Nothing” reads as no reflection; a recital of everything reads as no judgement.

“How do you keep it safe?” For agents: least-privilege tool scopes, human approval on irreversible actions, sandboxed execution, input/output validation, and treating every tool result as untrusted input. See Guardrails.

3.2 The whiteboard skeleton

For a design question, draw the same five boxes every time and fill them in; this is the diagram that makes you look organised even while thinking.

flowchart LR
    U["User / caller"] --> G["Guardrail + router"]
    G --> R["Retrieval (if knowledge)"]
    R --> M["Model call"]
    G --> M
    M --> T["Tools / actions (if agentic)"]
    T --> M
    M --> O["Output validation"]
    O --> U
    M -.-> L["Logging + eval capture"]
    O -.-> L

Then narrate it left to right, naming the failure mode at each box and where the latency goes. The dotted line to logging is the part most candidates omit and the part that makes the next interview answer possible — you cannot report a metric you never captured.

4. Numbers and trade-offs

4.1 The cost ladder in numbers

The ladder from §2.3, with magnitudes. These are orders of magnitude, not quotations — say them that way.

The column that decides most real arguments is iteration cycle. A pipeline whose loop takes minutes will be debugged; one whose loop takes days will be shipped at whatever quality it happens to reach. This is a legitimate and under-used argument for retrieval over fine-tuning, and it is worth saying out loud.

4.2 The latency budget

For an interactive product, “fast” is not a number — split it:

The key structural insight: an agent with $k$ tool calls pays $k$ prefill latencies, not $k$ decode latencies, because everything before the last hop is re-processed. That is why prompt caching and context trimming matter disproportionately in agentic systems — see Memory and Context.

4.3 What “good” means, per application

There is no universal accuracy target. The defensible move is to attach the metric to the consequence:

5. Pitfalls

Tool-level answers. “I used LangChain and a vector database.” This is a description of shopping, not engineering. Symptom: every follow-up question starts with “why”. Fix: lead with the constraint.

Claiming results you cannot defend. Any number in an interview is a promise to answer the next question about it. Symptom: you are asked how the eval set was built and you do not have an answer. Fix: prepare the eval story first, because the number is worthless without it.

Optimising the wrong rung. The most expensive version of this is fine-tuning a model to fix a retrieval problem. Symptom: the fine-tuned model still fails on the same questions, but more confidently. Fix: walk the ladder from the bottom before reaching up.

Over-answering. A four-minute answer to a one-line question reads as uncertainty, and it consumes time you will need later. Symptom: the interviewer interrupts to redirect. Fix: answer at layer 2, then stop and offer the next layer — “I can go into the memory arithmetic if that’s useful.”

Arguing with a correction. If the interviewer proposes a different design, the right move is to take it seriously: “that would avoid the reranker latency — the reason I didn’t is X; does X not apply here?” You may be wrong, and finding out costs nothing. Symptom: you are defending a decision you made up ninety seconds ago.

Reciting the book. Recognisable answers to a question the interviewer just asked differently. Symptom: you answer the question you prepared rather than the one you were asked. Fix: repeat the question back in your own words before answering — it is not filler, it is scoping.

6. Using it in a project

The playbook is not only for interviews. The six-part answer is the structure of a good design document, and the decision ladder is the structure of a good technical proposal. Concretely, when you are asked at work to “add AI to X”:

1. Write the problem and the constraint down, in one sentence each, and get agreement on them before proposing anything. Most failed AI projects failed at this step, not at the model.
2. Build the eval set first — even a hundred hand-labelled examples. It is the only thing that makes the rest of the project measurable, and it is cheap.
3. Walk the ladder explicitly in the proposal, including the rejected rungs and why. This is what prevents a fine-tuning project that should have been a prompt.
4. Ship the cheapest rung, measure, and let the measurement justify the move up.

And when not to use any of it: if the task has a deterministic specification, write code. String matching, arithmetic, and lookups are not AI problems, and saying so in an interview is a strength, not a dodge.

7. How you would prove it worked

For interview preparation, “it worked” means you can answer the follow-up — so that is what to measure. A practice loop that actually moves the needle:

1. Build a question bank and score, do not reread. Use the Question Index to sample questions at random. Answer aloud, in ninety seconds, without notes. Score each on four axes, one point each:

• Did I name the problem before the technique?
• Did I name a constraint or a rejected alternative?
• Did I give one number, correctly labelled as derived, cited, or example?
• Did I say how I would know it worked?

Anything scoring below 3 gets rewritten, not reread. This is the same discipline as fixing a bug rather than staring at it.

2. Record yourself once. People consistently underestimate their filler and overestimate their structure. One recording is enough to fix both.

3. Rehearse the follow-up, not the answer. For each topic, write down the three questions you would ask if you were trying to find the candidate’s edge, and answer those. The interview drill sections at the end of every chapter are built for this: each common wrong answer is a follow-up you will be asked.

4. Do one full mock, out loud, against the clock. The failure mode this catches is not knowledge — it is running out of time because you spent four minutes on the warm-up question.

Self-assessment rubric (example). A candidate who scores 4/4 on eight of ten sampled questions, and can name the chapter to reopen for the other two, is prepared. That is the bar this book is written to.

8. Interview drill

“Why did you choose RAG over fine-tuning?” Strong answer: name what each fixes — retrieval supplies facts that change and must be cited; fine-tuning changes form and behaviour. Then the constraint that decided it (docs change weekly; no labelled set; citation required). Common wrong answer: “RAG is cheaper.” True but incomplete; it invites “and if it weren’t cheaper, what then?”

“How would you evaluate that?” Strong answer: an offline labelled set with a named metric and a baseline, plus an online signal, plus the threshold you decided in advance. Common wrong answer: “we’d look at user feedback.” Not a measurement — no baseline, no denominator, no way to compare two versions.

“Your system is giving wrong answers. Where do you look?” Strong answer: walk the ladder upwards: is the answer in the retrieved context at all (recall@k)? If not, it is a retrieval problem — chunking, embedding, query formulation. If it is in the context and the answer is still wrong, it is a generation problem. Measure which before changing either. Common wrong answer: “I’d try a better model.” This is the expensive rung chosen before the cheap diagnosis.

“What is the biggest cost in your design?” Strong answer: name it and quantify it — usually context tokens, or multi-hop latency in an agent. Then name the lever you would pull first. Trap: answering “the GPU” when the dominant cost is re-processing the same prompt on every agent hop.

“Tell me about a failure.” Strong answer: a specific, technical failure with a cause and a fix — mask ordering, a chunking scheme that split tables, a prompt cache invalidated by a timestamp in the system prompt. Common wrong answer: a people problem, or a success disguised as a failure. The interviewer is testing whether you debug systems or narrate them.

“What are you weakest on?” Strong answer: a real gap, with the specific thing you are doing about it, and the boundary of what you do know adjacent to it. Trap: “I work too hard”, or a fake weakness that is obviously a strength.

Next: Question Index — every question in the book in one place, so you can audit your coverage.