// Deep Learning
Backpropagation by Hand: Build It With a Pencil, Check It With a Spreadsheet
Backpropagation is the algorithm that lets a neural network learn, and it is usually taught either as a wall of subscripts or as a vague story about errors flowing backwards. It is neither. It is one idea — if I nudge this, how much does that move? — applied over and over. This post derives every piece from scratch, assuming no calculus at all, and works a complete network end to end with numbers you can check on paper. By the end you will have built backpropagation in a spreadsheet and confirmed each derivative against a nudge.
Read article →The Glass Box Transformer: A Language Model With the Lid Off
Every explanation of a transformer eventually reaches a line like loss.backward() and stops. This is the one that does not. I take a complete decoder-only language model written in NumPy — no framework, no autograd, every gradient derived by hand and checked against finite differences — and walk the whole of it: byte-pair tokenisation, layer normalisation, causal multi-head attention, the residual stream, the backward pass through attention, Adam, nucleus sampling, and the formula that connects a 27,861-parameter toy to a 175-billion-parameter frontier model without changing a single line of the mathematics.
Read article →The Agent: When a Model Learns to Act
Chapter Eight — the finale — of a ground-up account of how large language models work. Everything so far built a mind that answers. This chapter builds a system that acts: give the model tools and a loop, let it decide for itself which to use and when, and the oracle you consult becomes an agent that works on your behalf. We assemble the last piece, weigh the new care it demands, and stand back at last to see the whole machine — from a single split token to a system that reaches into the world.
Read article →Meaning You Can Search: Embeddings, Retrieval, and Grounding
Chapter Seven of a ground-up account of how large language models work. The assistant we finished building is sealed inside the moment its training ended — it knows nothing of your documents, and it bluffs when it does not know. This chapter gives it a way to reach outside itself: to turn text into geometry, search a body of knowledge by meaning rather than by keyword, and ground its answers in sources a human can open and check. It is the machine's first honest connection to the living world.
Read article →Manners for a Mind: From Predictor to Assistant
Chapter Six of a ground-up account of how large language models work. The last chapter left us with a base model — vast, fluent, knowledgeable, and useless: a mind without a manner, that answers your question with three more questions. This chapter is about the second, smaller, stranger training that gives it a manner — and about why the seams of that final shaping are exactly where a deployed model's most important behaviours, and its most dangerous failures, are quietly decided.
Read article →How Noise Becomes Knowledge: Training a Language Model
Chapter Five of a ground-up account of how large language models work. We built the whole engine in the last chapter — and admitted it was empty, a magnificent tower full of random numbers that would output pure gibberish. This chapter fills it. It is the story of how a single measure of one wrong guess can reach back through a hundred layers and correct every weight that caused it — and how, repeated across a large fraction of everything humans have written, that one procedure turns noise into something that knows the world.
Read article →The Tower: How a Transformer Turns Attention into Thought
Chapter Four of a ground-up account of how large language models work. We have the single beating part — attention, a token's glance across its neighbours. Now we build the whole body around it: many glances at once, the private thinking step that turns gathered context into inference, and the deep tower that refines meaning layer upon layer until a prediction can be read off the top. By the end, the full transformer — the architecture that has intimidated readers for years — will be a machine you understand from the inside.
Read article →Reading the Room: The Idea at the Heart of Every Language Model
Chapter Three of a ground-up account of how large language models work. In the last chapter we left every token stranded — rich with meaning but frozen, wearing the same face in every sentence. This chapter builds the single mechanism that lets a token turn its head, look at the words around it, and become a different thing in every context. It is called attention, it is the beating heart of every modern language model, and we are going to derive it from nothing.
Read article →The Prediction Game: How Tokens Learn to Mean Something
Chapter Two of a ground-up account of how large language models work. A model is handed a stream of tokens that mean nothing — arbitrary ID numbers — and a single, almost insultingly simple task: guess the next one. This is the story of why that one task is enough to summon everything an LLM can do, and of the quiet trick that turns a meaningless number into something that behaves like understanding: giving every token a place in space.
Read article →The Grain of Language: How a Machine Reads the Internet
The opening chapter of a ground-up account of how large language models actually work. Before a model can think, it must read — and reading, for a machine, means something stranger and more consequential than most people imagine. This is the story of how the raw text of the internet becomes the tokens a model sees, why the model builds its own alphabet to do it, and why that single design choice explains so many of an LLM's strangest habits.
Read article →The Capstone: Build a Grounded Assistant
Lesson 8 of Learning With Dr Neal — the capstone. Build a working retrieval assistant over a folder of your own documents: local embeddings, the twenty-line retriever, grounded answers with citations, and an honest recall@k evaluation that will teach you more about deployed clinical AI than any product demo ever will.
Read article →Meaning as Geometry: Embeddings, Retrieval, and RAG
Lesson 7 of Learning With Dr Neal. The assistant from Lesson 6 knows nothing about your hospital's guidelines, your patient's notes, or anything published since its training data was collected. Closing that gap without retraining is the job of embeddings and retrieval — turning documents into geometry, searching by meaning, and grounding answers in sources you control.
Read article →From Predictor to Assistant
Lesson 6 of Learning With Dr Neal. The raw next-token predictor you trained in Lesson 5 is the 'before' picture. This is the 'after' — instruction tuning, feedback-based refinement, why the same weights can host such different behaviours, and what the post-training pipeline means for the failure modes you'll meet in deployed clinical tools.
Read article →Train Your Own GPT
Lesson 5 of Learning With Dr Neal. Stop reading and train one: take the TinyGPT from Lesson 4, feed it a corpus small enough for a laptop, write the training loop with your own hands, and watch generated text condense out of randomness — from noise, to word-shaped noise, to sentences.
Read article →Inside the Transformer
Lesson 4 of Learning With Dr Neal. How a language model actually turns a transcript into the next token: tokens and embeddings, why attention was the breakthrough, what a context window physically is — and a complete tiny GPT in PyTorch, small enough to read in one sitting and train on your laptop.
Read article →Learning With Dr Neal
Before Chapter 4 throws you into the deep end, build a working neural network from scratch — in a spreadsheet. No code. No libraries. Just arithmetic you can see and touch.
Read article →Learning With Dr Neal
Professional differentiation. In medicine, law, finance, engineering, and research — every field that processes data — practitioners who can build and evaluate AI tools will command significant advantages over those who can only consume them.
Read article →