One pass through the machinery
When you type a question into ChatGPT, Claude, or any large language model, the text you wrote passes through a series of transformations before a single word of response is chosen. This is one pass through that machinery.
Between the moment you press Enter and the first character appearing, something remarkable happens. This sentence is about to leave the world of human language and enter a world of numbers. Follow it through.
The model doesn't see characters or words. It sees subword chunks, carved up by an algorithm that knows nothing about meaning—only frequency. Common fragments survive intact. Rare words get split. This is why counting letters inside a word is so disorienting for a language model: the letters were never there to count.
Each token becomes a vector—a point in a space with thousands of dimensions. In this space, distance is meaning. Tokens that play similar roles in language cluster together; tokens with nothing in common drift apart. The model's entire understanding of language lives in the geometry of this space.
Here is where the computation happens. Each token asks every other token: how much do you matter to me? The answers are different for every token. “How” cares about “?” because they frame a question together. “R” cares about “straw” and “berry” because they are the subject it needs to reason about. Click any token to see what it attends to.
After all that computation, the model produces a probability distribution over every token in its vocabulary. The next word is chosen by a weighted coin flip. Temperature controls the shape of that flip: low temperature concentrates probability on the top candidate; high temperature flattens the distribution toward randomness.
The machinery is elegant, but it has characteristic failure modes. Each one follows directly from how the system works.
Plausible does not mean true. The model pattern-completes; it does not fact-check. Every token is chosen because it is statistically likely given the preceding tokens, not because it corresponds to reality. Confabulation is the default mode of operation. Accuracy is the special case—the remarkable thing is how often the patterns happen to be correct.
Agreeing was rewarded during training. When you push back on a model's answer, the probability distribution shifts toward approval—toward “you're right, I was wrong”—regardless of whether you actually are right. It is not lying. It is completing the most likely pattern, and the most likely pattern after human disagreement is human-pleasing agreement.
Attention creates a feedback loop. A token's presence in the context increases its own probability in the next generation step. Say a word once and it becomes slightly more likely to appear again. Under certain conditions, the model gets stuck in a groove of its own making—repeating phrases, circling back to the same idea, unable to escape the attractor it has created.
Tokens outside the context window do not fade gracefully. They vanish. One moment the model has access to everything you discussed ten thousand tokens ago; the next, those tokens are gone entirely. There is no graceful degradation, no dimming memory. It is a cliff edge, and the model has no way to know what fell off.
These aren't bugs. They're the natural consequences of the machinery working exactly as designed.
This happens every time. Billions of parameters, one token at a time, fast enough that you barely notice the pause.