Embedding Models

TL;DR — A retrieval embedder compresses a whole passage into one vector without seeing the query, so it cannot rely on the generator’s next-token objective to make that vector useful. It is trained with a contrastive objective: pull the true query–passage pair together, push every other pair in the batch apart. That objective is why a great language model is a terrible embedder until it has been trained, and why training data and hard negatives matter more than parameter count. Two implementation facts dominate real systems: cosine, dot product, and Euclidean rank identically once vectors are unit-normalised (and not otherwise), and prefix losses let you truncate an embedding to 256 of 768 dimensions for a few points of quality — but only if the model was trained that way (Matryoshka). MTEB is a starting filter, not a decision; the benchmark’s own authors found no method that dominates across tasks.

1. Why this exists

You need to retrieve passages. Someone on the team reasons: “we already paid for a good LLM — let’s just use its hidden states.” They take the last-layer hidden state at the final token, mean-pool it, and compute cosine similarity against queries.

Every query returns roughly the same ranking. Cosine similarities cluster between 0.93 and 0.97, and the ordering is close to noise.

This is not a bug in the code. It is a property of representations that were never trained to be compared: they live in a narrow cone of the space, so all pairwise angles are small and the differences between similarities — the only thing ranking depends on — are dominated by noise. The phenomenon is called anisotropy. A model trained to predict the next token has no reason to spread its sentence representations across the sphere.

Retrieval asks for something fundamentally different from generation:

That last row is the crux. At index time you have no query, so the embedder must commit, in advance, to a single point in ${\mathbb{R}}^d$ that will be close to the unknown query, far from every other passage’s query, and stable across paraphrase. That is a far more constrained problem than language modelling, and it is why retrieval embedders are trained on pairs rather than on raw text.

The one-sentence answer to “why do retrieval and generation need different models?” Because retrieval is a ranking problem over points chosen without seeing the query, and the loss that produces good points is a contrastive loss over pairs, not a likelihood over tokens. The empirical corollary is in the MTEB paper itself (Muennighoff et al. 2022, arXiv:2210.07316): across 8 tasks, 58 datasets, and 33 models, the authors found that “no particular text embedding method dominates across all tasks.” There is no best embedder; there is a best embedder for your distribution.

In 2026 the decoder/encoder line has blurred — instruction-tuned decoder LLMs are used as embedders and rerankers — but the objective distinction has not gone away.

2. How it works

2.1 The geometry: three metrics, and when they are the same metric

Let $q\in {\mathbb{R}}^d$ be the query embedding and $d_i\in {\mathbb{R}}^d$ the $i$-th document embedding. Three similarity functions are in common use:

$$\text{cosine:}\cos (q,d)=\frac{q\cdot d}{\Vert q\Vert \Vert d\Vert },\text{dot:}q\cdot d=\sum _{k=1}^d{q}_k{d}_k,\text{Euclidean:}\Vert q-d\Vert =\sqrt{\sum _{k=1}^d(q_k-d_k{)}^2}$$

Expand the squared Euclidean distance:

$$\Vert q-d{\Vert }^2=\Vert q{\Vert }^2+\Vert d{\Vert }^2-2(q\cdot d)$$

Now assume both vectors have unit length, $\Vert q\Vert =\Vert d\Vert =1$. The expansion collapses to

$$\Vert q-d{\Vert }^2=2-2\cos (q,d)$$

which is a strictly decreasing function of the cosine. Therefore:

On unit-normalised vectors, ranking by dot product, by cosine similarity, and by negative Euclidean distance are the same ranking. On unnormalised vectors they are three different rankings. This explains a class of bug that produces nearly right results.

What normalisation removes, and why removing it matters. Since $q\cdot d=\Vert q\Vert \Vert d\Vert \cos (q,d)$, the dot product mixes the angle with both magnitudes. Suppose the query is unit-length and the documents are not:

The dot product ranks B first: B is less similar in angle by a wide margin and wins purely because its vector is longer. Norms vary because mean pooling over more tokens pulls a vector towards the corpus mean, and because output norm correlates with text length and topic centrality.

So: is magnitude a signal or a bug? For retrieval, treat it as a bug. Length is a property of the chunker, not of relevance — if you want to rank by length, do it explicitly as a metadata filter. Normalise once at index time with normalize_embeddings=True, index with metric="ip", and query with the same flag, the same model, and the same version. Mixing is the bug: normalised documents searched with an unnormalised query, or a vector store applying its own normalisation you did not account for. The metric you report may be cosine while the metric you rank by is dot — see §5.

2.2 What the contrastive objective actually optimises

A bi-encoder’s training signal comes from pairs. The canonical form is InfoNCE: a $K$-way classification where the positive is the correct document and everything else in the batch is a distractor.

$$\mathcal{L}=-\log \frac{\exp (s(q,d^{+})/\tau )}{\exp (s(q,d^{+})/\tau )+{\sum }_{j=1}^K\exp (s(q,d_j^{-})/\tau )}$$

Every symbol, and what happens if you remove it:

• $q$ — the query embedding, produced by the same encoder (or a query-specific prefix/projection) as $d$. The point of a bi-encoder is that $q$ and $d$ are computed independently, which is what lets every document vector be precomputed. Remove that independence and you have a cross-encoder: more accurate, unusable as an index. The cross-encoder/reranker treatment belongs to Retrieval: Hybrid Search and Reranking §2.4; this chapter only needs the contrast.
• $d^{+}$ — the positive, the passage that answers $q$. It is the entire supervision; with no positives there is no signal about what should be close.
• $d_j^{-}$ — the $j$-th negative. In the common implementation these are in-batch negatives: the positives of the other $(B-1)$ examples in the same batch, reused as each other’s distractors. This is why contrastive training wants large batches — batch size is the negative count, and it is free supervision. Remove the negatives and the objective becomes “make all these pairs similar,” whose global optimum is to map every input to the same point — total collapse, cosine 1.0 everywhere, retrieval is random. Negatives are not regularisation; they are what creates the geometry.
• $K$ — the number of negatives. Larger $K$ means a harder $K$-way problem and better representations, which is why strong embedders report their batch size.
• $s(\cdot ,\cdot )$ — the similarity function, typically cosine (equivalently, a dot product on normalised vectors). It is the only place retrieval quality is defined during training, which is why §2.1’s normalisation convention must match what the model was trained with.
• $\tau$ — the temperature, a positive scalar setting how sharply the softmax concentrates. As $\tau \to 0$ the loss approaches a hard max and gradients come only from the hardest negative; as $\tau \to \infty$ the distribution flattens, every negative contributes equally, and the gradient shrinks towards zero. It is the most sensitive hyperparameter in contrastive training; values in the 0.01–0.1 range are typical (example), and the exact value is tuned per model.

Hard negatives and their failure mode. In-batch negatives are random, so most are easy — the model quickly learns “retrieval” versus “sourdough recipes” and stops improving. Hard negatives are documents a first-stage retriever scored highly that are not the answer; mining them (BM25 or dense top-$k$, excluding the gold) sharpens the boundary where it matters. The failure mode is false negatives: a mined “negative” that actually answers the query, of which there are often several. Training then pushes apart two correct answers. Symptom: a fine-tune that helps single-answer questions and hurts multi-answer ones; recall@$k$ looks fine while recall of all gold passages drops. Mitigate by hand-labelling a sample of mined negatives, or by using an objective that discounts high-similarity negatives.

flowchart LR
    QE["query encoder"] -->|q| L["InfoNCE loss / tau"]
    DE["doc encoder (shared weights)"] -->|"d+, d1-, d2-, ..."| L
    L --> U["update shared weights"]
    C[("corpus")] --> E2["doc encoder, offline"] --> IX["index (N, d), unit norm"]
    Q["user query"] --> QE2["query encoder"] --> QV["q, unit norm"]
    IX --> DOT["dot product"] --> TOP["argpartition top-N"]
    QV --> DOT

The asymmetry between the two halves is why this works at all: the expensive part happens once, offline, over the corpus.

2.3 Matryoshka: making a prefix of the vector usable

An ordinary embedding has no meaningful ordering of its coordinates. Truncating 768 dimensions to 256 gives you a projection that the model never optimised, and quality falls off a cliff.

Matryoshka Representation Learning (Kusupati et al. 2022, arXiv:2205.13147) changes the loss so that prefixes are valid embeddings. Instead of one loss on the full vector, apply the same contrastive loss at several nesting levels and sum:

$${\mathcal{L}}_{\text{MRL}}=\sum _{m\in \mathcal{M}}{w}_m\mathcal{L}(f(x{)}_{1:m},y)$$

• $f(x)\in {\mathbb{R}}^D$ — the full embedding of input $x$; $D$ is the nominal dimension (768, 1024, 3072).
• $\mathcal{M}\subseteq \{1,\dots ,D\}$ — the set of nesting dimensions. e.g. $\{64,128,256,512,768\}$. Each is a separate “granularity” the model must serve.
• $f(x{)}_{1:m}$ — the first $m$ coordinates, not a learned projection, not the top-$m$ principal components. This is the detail that makes MRL a training-time constraint rather than a compression trick: the model must put coarse information in low indices and fine information in high indices, because only a prefix is ever truncated.
• $w_m$ — the weight on the loss at dimension $m$, setting how much the model cares about small prefixes versus the full vector. Uniform weights ($w_m=1/|\mathcal{M}|$) are the simple default; if you drop $w_m$ for small $m$, the low-index coordinates lose their special status and truncation degrades back to ordinary projection. The paper’s framing is that MRL “minimally modifies existing representation learning pipelines” and imposes no extra inference cost.
• $y$ — the target for $x$ (the positive and the negatives), exactly as in §2.2. MRL does not change what is learned; only at which granularities.

What it does to the geometry: coarse structure — topic, domain, language — lives in the first few dozen coordinates, fine distinctions in the tail. The paper reports up to 14× smaller embeddings at the same accuracy on ImageNet-1k classification and up to 14× real-world speed-ups on large-scale retrieval, though the numbers that matter for text are model-specific.

The honest text-side numbers come from a released model — Nomic AI publishes the MTEB score of nomic-embed-text-v1.5 at each recommended truncation¹:

The curve is smooth and monotone, which ordinary embeddings are not. Implementing it is a prefix slice followed by re-normalisation — the step people forget, because dot-product indexes do not do it for you (see §3 for the code).

The storage implication: at 256 dims in fp16 the per-vector cost is $256\times 2=512$ bytes, versus $3,072\times 2=6,144$ bytes for a 3072-dim fp16 vector — a 12× reduction. At 10M chunks, 5.1 GB versus 61.4 GB.

OpenAI’s third-generation embedders expose the same idea as an API parameter: the dimensions parameter reduces output size, and OpenAI reports that even below the 1536 dimensions of text-embedding-ada-002, “performance remains slightly better” on their MTEB testing (Azure AI Foundry model docs, which mirrors OpenAI’s tables):

Matryoshka is not PCA. PCA finds the directions of highest variance in a fitted model and needs a separate projection per corpus; MRL orders coordinates during training and needs nothing at inference — so you can truncate per-query, at runtime, with no refitting.

2.4 Asymmetry: queries and documents are not the same distribution

A query is typically 3–15 tokens and interrogative; a passage is 100–500 tokens and declarative. A single symmetric encoder must produce comparable vectors for both, which is a compromise. Three ways models handle it:

E5 made this explicit: Wang et al. 2022, arXiv:2212.03533 trained on weakly supervised text pairs with query: / passage: prefixes, and reported being “the first model that outperforms the strong BM25 baseline on the BEIR retrieval benchmark without using any labeled data” zero-shot, and beating “existing embedding models with 40x more parameters” when fine-tuned. BGE’s README documents a single query instruction and states that the -v1.5 update was specifically to “enhance its retrieval ability without instruction” — newer models are more forgiving, not indifferent.

The rule: copy the prefix discipline from the model card, character for character, and apply it to the correct side. It is the highest recall-per-minute fix in a RAG system, and it is invisible in every metric you are not computing.

3. Implementation

# Deps: pip install "sentence-transformers>=3.0" "numpy>=1.26"
# The SentenceTransformer / encode(normalize_embeddings=...) / model.fit surface
# has been stable across the 3.x-5.x line [as of 2026-10].
import math
import numpy as np
from sentence_transformers import InputExample, SentenceTransformer, losses
from torch.utils.data import DataLoader

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
model.max_seq_length = 512     # CHECK THIS against your chunk size in tokens

# 1. Asymmetry: the task type is part of the API, not an afterthought.
def encode_documents(texts):   # documents get search_document:
    return model.encode([f"search_document: {t}" for t in texts],
                        normalize_embeddings=True, batch_size=64,
                        convert_to_numpy=True)

def encode_queries(texts):     # queries get search_query: — different prefix
    return model.encode([f"search_query: {t}" for t in texts],
                        normalize_embeddings=True, convert_to_numpy=True)

# 2. The diagnostic that catches untrained representations. If this is a narrow
#    spike near 1.0, no amount of index tuning will help.
docs = encode_documents(corpus_chunks)
q = encode_queries(["how do I rotate an API key?"])[0]
sims = docs @ q
print(np.percentile(sims, [1, 25, 50, 75, 99]).round(3))
# Healthy, in-domain: broad spread, e.g. [0.21 0.48 0.61 0.72 0.89]   (example)
# Collapsed:         everything in 0.93-0.97                          (example)
assert sims.max() - np.percentile(sims, 50) > 0.05, "similarities are degenerate"

# 3. Matryoshka truncation, correctly: prefix slice + RE-NORMALISE.
def truncate(vecs, m):
    out = np.ascontiguousarray(vecs[:, :m])
    return out / np.linalg.norm(out, axis=1, keepdims=True)

# 4. Fine-tuning: this IS InfoNCE with in-batch negatives.
#    MultipleNegativesRankingLoss uses the other positives in the batch as
#    negatives for each anchor, so `batch_size` is the negative count.
#    https://sbert.net/docs/package_reference/sentence_transformer/losses.html
train = [InputExample(texts=[a, p]) for a, p in pairs]   # (query, gold passage)
loader = DataLoader(train, shuffle=True, batch_size=64)  # 64 -> 63 negatives
loss = losses.MultipleNegativesRankingLoss(model)        # scale ~ 1/tau
model.fit(train_objectives=[(loader, loss)], epochs=1,
          warmup_steps=math.ceil(len(loader) * 0.1), output_path="embedder-ft-v1")

Four things in that code decide whether it works:

• model.max_seq_length matters more than the model choice. Anything past it is silently truncated. Set it deliberately and compare it to your chunk size in tokens.
• The prefix/task-type must match the query side and the document side separately. Applying search_document: to a query, or omitting both, degrades recall without raising anything.
• The similarity histogram is your cheapest health check. A collapsed representation — the §1 failure — shows up immediately as a narrow spike. Run it before you build the index.
• batch_size is a modelling hyperparameter, not just a throughput knob. In MultipleNegativesRankingLoss, batch size 64 gives 63 negatives per anchor. Dropping to 16 for memory reasons materially weakens the training signal.

The knobs

4. Numbers and trade-offs

Storage and search cost scale linearly in $d$; quality does not

Per-vector storage is exactly $d\times \text{bytes-per-element}$. For 10M chunks that is 61.4 GB at $d=1536$ in fp32, 30.7 GB in fp16, 15.4 GB in int8 — and 10.2 / 5.1 / 2.6 GB at $d=256$. Storage scales linearly in $d$; quality does not. Using §2.3’s nomic curve, 768 → 62.28 MTEB, 256 → 61.04, 64 → 56.10: losing 1.2 points to shrink storage 3× is usually a good trade; losing 6 points to shrink it 12× is a decision, not a default.

ANN search cost is the second-order effect. A graph index such as HNSW stores a fixed number of neighbour links $M$ per node, so its graph memory is $O(N\cdot M)$ and independent of $d$ — but each distance computation during traversal is $O(d)$, so query latency scales roughly linearly in $d$ until you are memory-bandwidth-bound rather than compute-bound. That crossover is why 256-dim embeddings are a genuine latency optimisation, not just storage.

The benchmark landscape, and how to use it

• MTEB (Muennighoff et al. 2022, arXiv:2210.07316): 8 task types, 58 datasets, 112 languages. The finding to quote is that no method dominates across tasks, which is why a single leaderboard average poorly predicts your retrieval quality. The leaderboard was rebuilt in June 2026 (MTEB blog) — record which revision a number came from.
• BEIR (Thakur et al. 2021, arXiv:2104.08663): 18 datasets, zero-shot. Uncomfortable reading for anyone who skipped hybrid search: BM25 is a robust baseline, and dense and sparse retrievers “often underperform other approaches” out of distribution.
• C-MTEB for Chinese; domain sets for code, law, finance.

So use MTEB as a filter, not a decision: shortlist the top handful of open models, check licence, dimension, and max sequence length against your constraints, then evaluate three or four on your own golden set with your own chunking. [as of 2026-10] the board churns monthly; the method is what is stable.

5. Pitfalls

Treating cosine similarity as a calibrated score. Cosine is a ranking device, not a probability: its absolute value depends on the model, the dimension, and the domain, so a hard threshold like “relevance ≥ 0.8” is not portable. Symptom: a threshold that worked in development filters nothing, or everything, after an embedder upgrade. Compare against a distribution — a percentile of random query–document pair similarities on your corpus — not an absolute number.

Anisotropy / collapsed representations. Used straight out of a language model or after a bad fine-tune. Symptom: the similarity histogram is a narrow spike (§3, step 2); all queries return the same ranking. Fix: use a retrieval-trained model; if fine-tuning made it worse, you over-trained.

Forgetting the query prefix or the task type. Symptom: mediocre, model-size- independent retrieval quality. Nothing raises an error. Check the model card before you check anything else.

Truncating without Matryoshka training, or without re-normalising. Symptom: recall falls off a cliff between the full dimension and anything smaller (the opposite of §2.3’s smooth curve), or retrieval after truncation is worse than the MTEB curve predicts and worse for dot-product indexes than for cosine. Fix: use a model with MRL weights, and re-normalise the prefix.

Domain shift. A general embedder on legal, medical, code, or another language can lose to a smaller in-domain model. Symptom: the vector store returns chunks that are topically adjacent but never the exact clause or identifier. Compare recall@$N$ on domain vs general queries; the gap is the domain shift.

Assuming a reranker can fix a bad embedder. It cannot: reranking reorders the candidates it is given. Symptom: you add a reranker, recall@10 improves modestly, recall@100 does not move, and end-to-end accuracy stays flat. This is the candidate-set ceiling derived in Retrieval: Hybrid Search and Reranking §1.

6. Using it in a project

Choosing a model without over-thinking it

1. Check the constraints first, because they eliminate most models: licence (commercial use), dimension (storage budget), max sequence length (≥ your chunk size in tokens), language coverage, and whether it needs a prefix you will remember to apply.
2. Shortlist three or four off MTEB/BEIR and the open ecosystem (BGE, E5, GTE, Nomic), plus your provider’s API model if you already pay for one.
3. Decide on your data. Build the golden set from The RAG Pipeline, then measure recall@100 for each candidate with identical chunking — a day’s work, and the only evidence that matters. Only then consider fine-tuning.

When fine-tuning an embedding model is worth it

Fine-tuning is the right move when all of the following hold:

• Recall@100 is the binding constraint, you have measured it, and the gap is not explained by chunking (fix that first) or a missing lexical retriever (add hybrid search before retraining anything).
• You have or can get pair data. Clicks, question→answered-document pairs, merge history, tickets with the article that resolved them. A few thousand clean pairs beat 100k noisy ones. With no pairs, synthetic pairs from an LLM are possible, but hand-validate a sample.
• The gap is domain shift, not capability. Failing on jargon, identifiers, or another language is a domain problem a fine-tune can fix. Failing on reasoning-shaped queries is not a retriever problem at all.
• You can maintain it. A fine-tuned embedder is an artifact: version, re-index pipeline, evaluation record, owner.

When fine-tuning is the wrong tool: the corpus fits in the prompt (no retriever, no fine-tune); you want the model to know facts rather than rank them (a knowledge problem — RAG or full fine-tuning, not a low-rank embedding update); or a hybrid BM25 + reranker stack already hits your recall target, which needs no training data and leaves no artifact to maintain.

7. How you would prove it worked

The evaluation that isolates the embedder. Fix the chunk set, the retrieval depth $N$, the normalisation, and the query prefix; vary only the model. Report $Recall@k=|\text{gold}\cap \text{retrieved@}k|/|\text{gold}|$ and, because retrieval is about where the gold landed, not just whether it did,

$$\mathrm{MRR}=\frac{1}{|Q|}\sum _{q=1}^{|Q|}\frac{1}{{\mathrm{rank}}_q}$$

• $|\text{gold}|$ — the number of passages that could answer the query. Including all of them (rather than just one) is what catches the false-negative problem in §2.2.
• ${\mathrm{rank}}_q$ — the 1-based position of the first gold passage for query $q$; MRR is right when exactly one passage answers the question.

Ablation table (example — replace with your own numbers):

Three lessons are visible only when the table is laid out this way. The prefix row is the cheapest gain in the whole system. The model-size row shows diminishing returns on generic text. And the last row is the only one that moves recall@100 after the prefix fix — precisely why you should not attempt it before measuring.

Ablate the metric too. Build the index twice, with and without normalisation, and report recall under both cosine and dot ranking. If it makes no difference, your documents happen to have similar norms; if it does, you have proved §2.1 on your own data in twenty minutes.

Reporting template. “On a 200-question golden set with 512-token chunks we compared four retrieval embedders from MTEB’s top open models, with identical chunking and no reranking. bge-base-en-v1.5 reached recall@100 of 0.81 with its documented query prefix and 0.71 without (example) — a 10-point swing from one line of code. Fine-tuning on 2,000 click-derived pairs for 1 epoch raised it to 0.93; a second epoch reduced it to 0.90 (example). Matryoshka truncation from 768 to 256 dims cost 0.7 recall@100 points and cut index memory 3× (example).”

8. Interview drill

“Why can’t we just use the LLM’s embeddings for retrieval?” Strong answer: the LLM was optimised to predict the next token, not to place texts at comparable points in a space. Its representations are anisotropic — they occupy a narrow cone — so pairwise cosine similarities are all high and their differences are noise. Retrieval needs a contrastive objective over pairs, which spreads representations out. The tell-tale symptom is a similarity histogram centred near 0.95 with almost no spread. Common wrong answer: “the vectors are too high-dimensional / too slow.” A small contrastively trained model beats a large untrained one on retrieval.

“Cosine, dot product, or Euclidean — does it matter?” Strong answer: if and only if the vectors are unit-normalised. On unit vectors $\Vert q-d{\Vert }^2=2-2\cos (q,d)$, so minimising Euclidean distance, maximising the dot product, and maximising cosine produce the same ranking. Without normalisation the dot product mixes in magnitude, and a long generic chunk can outrank a short precise one purely because its vector is longer. Normalise, then pick dot product for speed. Common wrong answer: “they’re all just distance metrics, it’s a preference.” It is a correctness question, and the failure is subtle rather than loud.

“What does Matryoshka truncation do to the geometry?” Strong answer: it changes the loss, not the architecture. You sum the same contrastive loss over nested prefixes $f(x{)}_{1:m}$ for $m\in \mathcal{M}$, which forces coarse information into low indices and fine information into high ones. Truncation is then a prefix slice plus re-normalisation, and quality degrades smoothly instead of falling off a cliff — Nomic’s model goes from MTEB 62.28 at 768 dims to 61.04 at 256 and 56.10 at 64. It is not PCA: PCA fits a projection post hoc, MRL builds the ordering in during training. Common wrong answer: “it’s just dimensionality reduction.” A post-hoc projection onto an untrained prefix destroys the representation.

“What is the contrastive objective optimising, in words?” Strong answer: an InfoNCE loss — a softmax classification over the positive plus $K$ negatives, with temperature $\tau$. It raises the similarity of the true pair and lowers it to every other passage in the batch. With no negatives the objective is trivially satisfied by mapping everything to one point, so negatives are what create the geometry. Batch size is the negative count; hard negatives matter more than easy ones; false negatives actively teach the model to separate correct answers. Follow-up: “what does $\tau$ do?” → it controls softmax sharpness. Small $\tau$ focuses the gradient on the hardest negatives; large $\tau$ flattens the distribution and the gradient vanishes.

“How do you use MTEB?” Strong answer: as a shortlist filter, not a decision. It covers 8 task types and 58 datasets, and its own authors found no method dominates across tasks, so a leaderboard average poorly predicts retrieval quality on your corpus. Filter by licence, dimension, and max sequence length; shortlist three or four; decide on your own golden set with your own chunking. Record the leaderboard revision — [as of 2026-10] it is versioned and churns. Common wrong answer: picking the current top-1 and shipping it; being wrong costs a full re-index.

“When is fine-tuning an embedding model worth it?” Strong answer: when recall@100 is measured and binding, the gap is domain shift rather than capability, and you have real pair data (clicks, tickets, merge history). Fix chunking and add hybrid search first — both are cheaper and often sufficient. Expect 1–3 epochs; more on small data overfits. Budget the ongoing cost: an artifact, a version, a re-index pipeline. Common wrong answer: “fine-tune as soon as retrieval underperforms.” Without a measured recall@100 you cannot tell whether the embedder is the bottleneck, and you will spend GPU hours fixing a chunking bug.

Next: Retrieval: Hybrid Search and Reranking — dense vectors miss exact strings, so the retriever needs a lexical partner and the ranking needs a second, more expensive pass.

1. Nomic Atlas docs, “Text Embedding” (as of 2026-10). The same page lists non-Matryoshka nomic-embed-text-v1 at 768 dims / MTEB 62.39 — the baseline: Matryoshka costs ~0.1 MTEB points at full dimension, and buys a 12× smaller embedding at a cost of ~6 points. ↩