Meaning as Geometry: Embeddings, Retrieval, and RAG

The assistant we assembled across Lessons 4–6 has a strange epistemic profile. It writes beautifully, defaults to helpfulness, and knows a compressed, blurry copy of everything its pretraining corpus contained — as of the day that corpus was collected. It knows nothing else. Not your hospital's antimicrobial guidelines. Not the trial published last month. Not the allergy documented in the notes of the patient in front of you. And, as Lesson 6 established, it will answer questions about all of these anyway — fluently.

You could try to fix this by retraining. For facts, that's the wrong tool: pretraining is a nine-figure industrial process, and Lesson 6's fine-tuning relocates defaults, not reliably-recallable knowledge. What you actually want is something more surgical: at the moment a question is asked, find the right passages from documents you control and hand them to the model as context. The model then does what it has always done — continue a transcript — except the transcript now contains the evidence.

That is retrieval-augmented generation — RAG — and it rests on one genuinely beautiful idea: that meaning itself can be given coordinates.

🎬 Watch: Meaning as Geometry — the video companion to this lesson.

🎧 Listen to this post: How RAG prevents AI hallucinations — the audio companion to this lesson.

Meaning as geometry

You met embeddings in Lesson 4 as the transformer's front door: each token gets a learned vector, and tokens used similarly end up near each other. The idea scales up. An embedding model — itself a transformer, trained for exactly this job — reads a whole sentence, paragraph, or document chunk and outputs a single vector, typically a few hundred to a few thousand numbers, such that texts with similar meaning land near each other in the vector space.

Not similar wording — similar meaning. A well-trained embedding model places "the patient developed an itchy rash after the first dose of amoxicillin" close to "penicillin allergy — urticaria" and far from "the patient's rash improved after stopping the statin," even though the surface vocabulary overlaps more with the latter. How? Training by contrast: the model sees millions of pairs known to be related (question and its answer, title and its abstract, two translations of one sentence) and is optimised to pull related pairs together and push unrelated ones apart. Geometry is sculpted until distance means dissimilarity.

Figure 1. Embedding space, cartooned in two dimensions (real spaces have hundreds to thousands). Note the crucial win over keyword search: "co-amoxiclav" doesn't appear anywhere in the allergy documentation, and "rash" appears in the wrong cluster — yet meaning-distance routes the query correctly. This is retrieval by concept, not by string.

💡 Key idea. An embedding is a fixed address for a piece of text in a space where distance ≈ dissimilarity of meaning. Once text is geometry, "find documents relevant to this question" becomes "find the nearest points" — a problem computers have been fast at for decades. That single reduction powers semantic search, RAG, memory systems, deduplication, and clustering alike.

Two practical notes before we build with it. Similarity is usually measured by cosine similarity — the angle between vectors, ignoring length. And nearest-neighbour search over millions of vectors is made fast by approximate indexes (the "vector databases" you've heard of are, at heart, this index plus bookkeeping). At clinic scale — thousands of documents — you don't even need that: brute-force comparison is milliseconds.

The RAG pipeline

With embeddings in hand, retrieval-augmented generation is two pipelines meeting at a context window:

Figure 2. The full RAG pipeline. The model itself is unmodified — same weights as Lesson 6, doing Lesson 4's next-token prediction. All the new engineering lives outside the model, in what gets placed into its context window. Fresh knowledge is one re-index away, no retraining involved.

In code, the question-time lane is almost embarrassingly short. Given any embedding function (every major model provider offers one; good open-weight models run locally):

import numpy as np

# ingest once: chunks = list of (text, source) pairs
vecs = np.stack([embed(text) for text, _ in chunks])       # (N, d)
vecs /= np.linalg.norm(vecs, axis=1, keepdims=True)        # unit length

def retrieve(question, k=5):
    q = embed(question)
    q /= np.linalg.norm(q)
    sims = vecs @ q                     # cosine similarity with every chunk
    top = np.argsort(sims)[-k:][::-1]   # k best matches
    return [chunks[i] for i in top]

def answer(question):
    passages = retrieve(question)
    context = "\n\n".join(f"[{src}] {text}" for text, src in passages)
    return llm(f"Answer using ONLY these passages. Cite sources. "
               f"If they don't contain the answer, say so.\n\n"
               f"{context}\n\nQuestion: {question}")

Twenty lines, and every one of them is a concept you already own: the embedding is Lesson 4's front door scaled up, the llm() call is Lesson 6's assistant, and the prompt assembly is Lesson 3's context management. This is also, incidentally, exactly how the Memory system in Lesson 3's agent works: memory notes are chunks, session start embeds the task, and the nearest notes get injected into context. You have now closed that loop.

Why this beats retraining for knowledge

It's worth being crisp about why retrieval, not fine-tuning, is the default answer to "make the model know our stuff" — because vendors regularly sell the opposite.

Updateable in minutes. Guideline changed Tuesday? Re-index Tuesday. A fine-tuned model is a snapshot; an index is a living document set.

Auditable provenance — the thing Lesson 6 said was missing. A RAG answer can carry real citations to real passages that a human can open and check. The chain from claim to source is inspectable at last. Post-training alone could only ever produce citation-shaped text.

Access control that actually works. Retrieval respects permissions: the index can serve different users different document sets. Knowledge baked into weights is available to every user of the model, always — you cannot revoke a fact from a fine-tune.

Honest failure. When retrieval finds nothing relevant, the system can say so — and a well-prompted model asked to answer only from provided passages will decline far more reliably than one asked to search its own parametric memory, where (Lesson 5) something plausible-sounding is always available.

Where RAG fails — and how to catch it

RAG is the best answer we have to grounding, which is precisely why you should know its failure modes cold. They come in two families.

Retrieval fails. The right passage exists but doesn't come back. Causes: bad chunking (the answer straddles two chunks, or a chunk lost its context — "the dose is 5 mg" of what?); vocabulary mismatch that even embeddings can't bridge (your guidelines say "NOAC," the query says "DOAC," older embedding models may separate them); or the answer requiring synthesis across many documents when only five chunks were retrieved. And one failure mode subtle enough to deserve its own picture:

Figure 3. The failure mode I test for first in any clinical RAG product. Contradictory guidance on the same topic embeds close together — topical similarity is what the geometry encodes. Currency, population, and polarity must be handled by metadata filters, index hygiene, and the model's reading of the retrieved text — not by the vector search, which is blind to all three.

Generation fails despite good retrieval. The passages arrive and the model still gets it wrong: it blends retrieved facts with its parametric memory (answering from the blur of pretraining rather than the document in front of it); it over-trusts a passage that keyword-matches but doesn't apply; or the answer sits in the middle of a long stuffed context where models demonstrably attend least — recall from Lesson 4 that attention can reach everything in the window, but post-training taught models habits about where to look.

The evaluation discipline follows directly, and it's the same one we apply to any diagnostic cascade: measure the stages separately. Score retrieval on its own (for a set of test questions, did the right passages come back? — recall@k, checkable without any LLM), then score generation given correct passages (faithful? cited? complete?). A single end-to-end accuracy number confounds the two exactly the way "overall mortality" confounds case-mix with quality of care.

⚕️ Two clinical notes before you deploy anything. First: an embedding of a patient note is derived from the note and can be partially inverted — treat vector indexes of clinical text as PHI, with the same storage, access, and audit obligations as the source documents. Second: when a RAG product demo shows you a beautiful cited answer, ask to see the retrieved passages it was not shown — the ones ranked 6th through 20th. Whether the truth was sitting just below the cutoff is the single most informative thing you can learn about the system's margin of safety.

What's next

Look at what's now on the bench: a model you understand down to the attention head (Lessons 4–5), shaped into an assistant whose failure modes you can name (Lesson 6), grounded in documents you control (this lesson), wrapped in an agent loop with tools, memory, and permissions (Lesson 3). Every component of a production clinical AI system, understood end to end.

Lesson 8 — now live — puts it all together with our hands: we'll build a working retrieval assistant over your own documents — a folder of guidelines or papers, a local embedding model, the twenty-line pipeline above, and an honest evaluation of where it succeeds and fails. Like Lesson 5, it will run on your laptop, and like Lesson 5, the point is calibration: after you've watched your own retriever return a superseded protocol with a confident citation, no product pitch will ever sound quite the same.

Until then: next time someone shows you a clinical AI tool and says "it's grounded in your guidelines," you know the three questions to ask. What's in the index? What came back below the cutoff? And what happens when the passages contradict each other? The answers will tell you whether you're looking at Figure 2 — or at Lesson 6's fluent confidence wearing a lab coat.

— Neal