What are embeddings?

A dense vector that positions text in a space where geometric closeness approximates semantic similarity. That single property is what turns “find things that mean the same” into a nearest-neighbour lookup, which is a solved engineering problem.

"cancel my order"    ─▶ [0.12, -0.44, ...]
"how do I refund"    ─▶ [0.15, -0.41, ...]  close
"reset my password"  ─▶ [-0.61, 0.22, ...]  far

The numbers mean nothing individually. Only distances between them mean anything, and only within one model’s space.

This is what semantic search means in practice, and the problem it solves has a name: vocabulary mismatch — the question says “linter”, the document says “lint”; the user asks to “cancel”, the policy says “refund”. Keyword search scores those at zero. Embeddings score them as near-identical, which is the entire reason retrieval moved to vectors.

Static vs contextual

The distinction interviewers use to date your knowledge.

Static embeddings gave one vector per word regardless of use, so bank was a single average of two unrelated meanings. Contextual models produce the vector from the surrounding tokens, so the two senses separate.

Word2Vec is worth knowing as history — the king - man + woman ≈ queen arithmetic is where the intuition comes from — but proposing it for a retrieval system in 2026 is a red flag.

Text embeddings are not token embeddings

A modern embedding model produces one vector per input, by pooling token vectors. When someone says “embedding model” for RAG, they mean this pooled, whole-passage vector.

Similarity: pick one and normalise

import numpy as np

# Normalise once, at index time.
v = v / np.linalg.norm(v, axis=1, keepdims=True)
q = q / np.linalg.norm(q)

# Now a plain dot product IS cosine similarity,
# and it is a single fast matrix multiply.
scores = v @ q

Raw dot product mixes direction (semantics) with magnitude, and magnitude often tracks passage length rather than meaning — so a long, vaguely related chunk can outrank a short exact one. Unit-normalising removes that, and makes dot product and cosine identical.

Gotcha: some vector databases default to Euclidean. If your relevance looks subtly wrong and long documents win too often, check the index metric before you change models.

Dimensions cost real money

Dimension count drives storage, memory and query latency roughly linearly. A million chunks at 1536 float32 dimensions is about 6 GB before index overhead; at 384 it is 1.5 GB.

Modern models trained with Matryoshka representation learning let you truncate the vector and keep most of the quality — ask for 512 dimensions from a 3072 model and renormalise, rather than switching models. That is the current answer to “how do I make this cheaper”, and knowing it signals recent hands-on work.

Quantisation before dimension cuts

Storing vectors as int8 instead of float32 cuts memory 4× for a small recall loss, and binary quantisation goes further with rescoring. Try quantisation before you accept a weaker model.

Choosing a model

Choose on your own retrieval eval, not on a leaderboard. MTEB scores are averaged over tasks that are probably not your task, and public benchmarks leak into training sets.

In priority order:

1. Recall@k on 30 of your real queries. Nothing else predicts production.
2. Max input length against your chunk size. A 512-token model silently truncates a 900-token chunk, and the tail is simply not indexed.
3. Dimension, for the cost reasons above.
4. Multilingual, if your corpus is.
5. Where it can run — an API model means your documents leave the boundary, which is decided for you in regulated work.

Symmetric vs asymmetric

Retrieval is asymmetric: a short question has to match a long passage. Models trained for that expect a prefix (query: / passage:) and lose real accuracy without it. Check the model card — this is a common silent misuse.

The trap: changing the model

Vectors from two different models are not comparable. There is no conversion. Changing the embedding model means re-embedding and re-indexing the entire corpus, and until that finishes, mixed vectors give meaningless distances.

Plan for it: version the index, build the new one alongside, and cut over. On a large corpus this is hours of compute and a migration, not a config change.

Where the depth is

• Vector Databases — how nearest-neighbour search is actually done
• Chunking Strategies and Retrieval Techniques — what you embed matters more than the model
• Hybrid search and reranking — where embeddings alone fall short
• Linear algebra for ML — the geometry underneath