How Noise Becomes Knowledge: Training a Language Model

Let me begin by making the problem vivid, because its difficulty is easy to underestimate and the solution is easy to take for granted once you have seen it.

At the end of the last chapter we had assembled a complete transformer — the embedding table, the attention heads with their query, key, and value projections, the feed-forward layers, the residual stream, the whole tower, and the final projection that reads a prediction off the top. It is an engine of real intricacy. And I told you the uncomfortable truth: if you built it today and ran a sentence through it, it would produce gibberish. Every one of its parameters — and there may be billions of them — begins life as a random number. The embedding that we later admired for placing aspirin near ibuprofen starts as buckshot. The attention heads that we said track pronouns start attending to nothing in particular. The feed-forward layers that hold the model's knowledge start holding noise.

So here is the question this chapter answers, and I want you to feel how audacious it is. We have a machine with a billion knobs, all set randomly. We want to turn those random settings into precisely the configuration that makes the machine fluent, knowledgeable, and startlingly capable. Nobody can set a billion knobs by hand, and nobody knows what the right settings even are. How, then, does the machine find them itself?

The answer is one of the most beautiful procedures humans have ever devised, and it has just three moving parts: a way to measure how wrong the machine currently is, a way to work out which direction to nudge every knob to make it less wrong, and the patience to do this a truly staggering number of times. Measure, nudge, repeat. That is all training is. Let us build each part.

Part I — Measuring wrongness

You cannot improve what you cannot measure, so everything starts with turning "the model is wrong" into a single, honest number.

We have exactly what we need, and we have had it since Chapter One. Recall that language modelling is self-supervised: we take real text, and at every position the "correct answer" is simply the token that actually came next. So we can play the following game against any piece of writing. Show the model the text up to some point. Let it produce its forecast — the full probability distribution over the vocabulary from Chapter Two. Then look at the token that truly came next, and ask a pointed question: how much probability did the model put on the truth?

If the model was confident and right — it put 70% of its forecast on the token that actually followed — it was barely wrong at all. If it was confident and wrong — it put 70% on some other token and a measly 0.1% on the truth — it was badly, embarrassingly wrong. The measure of wrongness we use, called the cross-entropy loss, captures exactly this: it is large when the model assigned low probability to the true next token, and small when it assigned high probability. You can think of it, without losing anything important, as a number that measures the model's surprise at the correct answer. A good model is rarely surprised by what comes next; a bad one is constantly astonished.

Figure 1. The loss turns "how wrong is the model right now?" into one number, by asking how much probability it placed on the token that actually came next. It is high for a confidently-wrong model and low for a confidently-right one. This single number — averaged over many predictions — is the compass for everything that follows.

So now we have a number. For any setting of the billion knobs, we can compute a loss — a measure of how badly, on average, the machine currently predicts real text. And the entire problem of training has been transformed into something almost tractable-sounding: find the setting of the knobs that makes this number as small as possible. We have turned "teach a machine to understand language" into "minimise a number." That reframing is the whole ballgame.

Part II — Which way is downhill?

Here is a way to picture the task that will carry you a long way. Imagine the loss as a landscape. Every possible setting of the billion knobs is a location, and the height of the terrain at that location is the loss — how wrong the model is with those settings. A random starting point is somewhere high up on a hillside, in the fog. The configuration we want — low loss, fluent model — is down in a valley we cannot see. Training is the search for that valley. And the rule for the search is the simplest imaginable: always step downhill.

But downhill in a landscape of a billion dimensions? You cannot see it, cannot picture it, cannot survey it. What you can do, remarkably, is feel the slope directly under your feet. For each of the billion knobs, calculus lets us ask a precise local question: if I nudged this one knob a hair in the positive direction, would the loss go up or down, and how steeply? Collect that answer for every knob at once and you have the gradient — a vector that points in the direction of steepest increase of the loss. And if the gradient points most steeply uphill, then its exact opposite points most steeply downhill. So we take a small step in that opposite direction, and we have, with certainty, reduced the loss a little. Every knob moves a touch toward "less wrong."

Figure 2. Gradient descent, the engine of all learning here. The loss is a landscape over the space of all parameters; the gradient reveals the downhill direction; we take a small step that way and repeat. The size of each step is the learning rate — one of the few dials a human sets — and getting it wrong in either direction (crawling, or overshooting and bouncing) is one of the most common ways training fails.

If gradient descent feels familiar it should — it is the same "small step downhill on a loss" that underlies essentially all of machine learning, and it is exactly what you would do, blindfolded, to find the bottom of a valley: feel which way the ground slopes beneath your feet, and shuffle that way, over and over. What makes it feel like magic in a language model is only the dimensionality. You are not on a hillside; you are on a surface with a billion independent directions, feeling the slope in all of them at once, and stepping downhill in that unimaginable space. The idea is humble. The scale is not.

⚕️ The trainee analogy, and where it holds. Think of the loss as the feedback a trainee gets on every single prediction they make — not a term-end exam, but a correction after each guess. "You said antibiotics; it was antibiotics — good, barely adjust." "You said giraffe; it was antibiotics — badly wrong, adjust sharply." A human trainee integrates such feedback slowly and holistically. The machine does something more literal and more thorough: it works out, for every internal parameter that contributed to the guess, the precise direction that parameter should move to have made the guess a little better — and moves all of them at once. It is feedback turned into a billion simultaneous, individually-tailored corrections.

Part III — The astonishing part: assigning the blame

I have been gliding over the hardest question, and now we face it, because it is the genuine miracle at the centre of this chapter.

Computing the gradient means working out, for each of a billion knobs, how the loss would change if that knob moved. But the loss is computed at the very top of the tower — after the token has passed up through a hundred layers of attention and feed-forward — while the knobs are scattered throughout every one of those layers, deep below. How can a single number at the summit possibly tell a particular weight, buried in the twelfth attention head of the fortieth block, which way it should move? The mistake happened at the top; the causes are everywhere underneath. This is the problem of credit assignment — apportioning responsibility for the error back to every part that contributed — and for decades it was the barrier that made deep networks seem hopeless.

The solution is an algorithm called backpropagation, and its essential idea is one every clinician will recognise as root-cause analysis. When something goes wrong at the end of a long process, you do not shrug and blame everything equally. You trace backward. You ask: given the final error, how much did the last step contribute? Then, holding that accountable, how much did the step before it contribute to that? And so on, backward, apportioning a share of the blame at each stage to the stages that fed it. Backpropagation does exactly this, mechanically and exactly, using the chain rule of calculus. It starts with the loss at the top, computes how much the topmost layer's weights contributed, then uses that to compute how much the layer below contributed, and so on — a single sweep from the summit all the way down to the embeddings, leaving behind, on every weight, a precise note: nudge yourself in this direction, by this much.

Figure 3. Backpropagation. Text flows up the tower to produce a prediction and a loss (blue). Then the blame flows down (red): the error is traced backward, layer by layer, until every weight in the entire model — billions of them — has received a precise, individual instruction for how to change. What sounds impossible is, mechanically, just the chain rule applied with ruthless bookkeeping, and it costs only about one extra pass through the network.

I want to dwell on how astonishing this is, because familiarity has dulled it for the field and it should not be dulled for you. A single number — the model's surprise at one token — is decomposed, exactly, into billions of individual responsibilities, one for every parameter that had any hand in producing it, no matter how deep it sits or how indirectly it contributed. And it is done efficiently, in a single backward sweep, not by the impossible brute force of testing each knob one at a time. Backpropagation is the reason deep learning works at all. Without it, the tower we built in Chapter Four would be an unteachable monument. With it, the tower can be corrected, wholesale, from a single measure of a single mistake.

💡 The complete recipe, in one breath. Show the model some text. Let it predict the next token (a forward pass up the tower). Measure its surprise at the truth (the loss). Trace that surprise backward to a correction for every weight (the backward pass — backpropagation). Nudge every weight a small step in its correcting direction (gradient descent). That is one training step. Everything else — the fluency, the knowledge, the apparent reasoning — is this one step, repeated.

Part IV — One step is nothing; repetition is everything

Here is the deflating and then re-inflating truth about a single training step: it barely does anything. One nudge, on one small batch of text, moves each of the billion knobs by a hair. Run it once and the model is imperceptibly less terrible than before — still, to any observer, producing gibberish. If training were a single step, or a thousand, it would be a curiosity and nothing more.

The magic is entirely in the repetition, at a scale that is genuinely hard to hold in the mind. The step is repeated millions of times, over batch after batch of text, until the model has been nudged by essentially every sentence in a corpus of trillions of tokens — a large fraction of the readable internet from Chapter One, passed through the tower and turned into corrections. And slowly, then all at once, structure precipitates out of the noise. The embedding vectors drift from random scatter into the meaningful landscape of Chapter Two. The attention heads settle into their specialities. The feed-forward layers fill with fact. Gibberish becomes grammar; grammar becomes fluency; fluency becomes something that argues and codes and diagnoses. Watch the loss fall and the generated text change in lockstep:

Figure 4. Training in one picture. The loss curve falls steeply at first (spelling and word statistics are cheap to learn) then more slowly (grammar, then meaning, then knowledge). The generated samples track it exactly: noise, then word-shaped noise, then fluency. This is the promise from Chapter Two made literal — meaning is not installed, it precipitates out of the single relentless pressure to predict the next token.

This whole phase has a name — pretraining — and it is, by a wide margin, the most expensive thing that happens to a language model. It consumes almost the entire compute budget: months of computation across thousands of specialised processors, at a cost that runs well into the millions. Nearly everything the finished model knows, it learned here, in this long descent, from this one repeated step. When people speak of the staggering resources behind frontier AI, it is overwhelmingly this they are describing — not the clever architecture, which is comparatively cheap to specify, but the brute, patient, enormously scaled application of measure-nudge-repeat.

Part V — What the descent gives you, and what it does not

When the loss finally flattens and pretraining ends, we have something genuinely new in the world: a base model. The random noise is gone. The tower is full of structure — an embedding space that knows medicine and poetry and code, attention heads that parse grammar, feed-forward layers dense with fact. It is fluent in dozens of languages, apparently knowledgeable across most of human writing, and able to continue almost any text you give it with uncanny plausibility. Noise has become knowledge. The promise of this chapter's title is kept.

And yet — this is the twist that sets up everything still to come — a raw base model is not the helpful assistant you talk to. It has been trained to do exactly one thing: continue text in the way its corpus would. So its instinct, faced with your input, is not to help but to continue. Ask a base model a question and you may get, in reply, not an answer but more questions — because on the internet a question is very often followed by other questions, in an exam, an FAQ, a forum thread. It has learned the shape of all text, which is not the same as the shape of helpful text.

Figure 5. The base model's telling limitation. It is not broken; it is doing precisely what training rewarded — continuing text. But "continue the text" and "answer the question helpfully" are different goals, and the gap between them is exactly the work of the next chapter.

⚕️ A base model is a brilliant, unshaped mind. Picture a physician with encyclopaedic knowledge and total fluency who has, however, never been taught the role of a doctor — never learned that when a worried person describes symptoms, the thing to do is respond helpfully rather than, say, continue writing the textbook chapter the symptoms came from. The knowledge is all there. The orientation toward being useful is not. That orientation is not knowledge; it is a posture, and it has to be trained in separately. Which is precisely why the model you actually talk to went through a second, quite different kind of training after this one.

Part VI — What we have, and the one thing left

Stand back and see how far we have come. Five chapters ago we had raw text from the internet. We ground it into tokens. We gave the tokens meaning as geometry. We built attention, the glance that lets a token read the room. We stacked attention and computation into a deep tower. And in this chapter we filled that tower — turning a billion random numbers into fluent knowledge by the humblest of procedures, applied at inhuman scale: measure the surprise, trace the blame, nudge every weight, repeat until the internet has passed through.

That is a complete, trained, knowledgeable language model. If your goal was to understand how the raw intelligence of these systems comes to be, you now understand it, end to end, with no gaps papered over and no magic left unexamined. The engine is built and it is full.

But it is not yet the thing in your pocket. The base model we have made is a mind without a manner — it will complete your sentence, mimic any voice, continue any document, and answer your question with three more questions, because helpfulness was never the target. The final transformation — the one that turns this vast, fluent, faintly feral predictor into something that answers, follows instructions, admits uncertainty, and declines to help with the dangerous things — is not more of the same training. It is a second kind of shaping entirely, smaller and stranger and, in its way, more consequential for how these systems behave in the world.

How a base model becomes an assistant — instruction tuning, learning from human preferences, and why the very same weights can host such wildly different personalities — is Chapter Six.

— Neal