You built the transformer block in Phase 5. Now watch a whole GPT think: a token goes in at the bottom, climbs a tower of identical blocks — each one reading the running total and adding its two cents — and a probability over the next token comes out the top. This page renders that tower in real 3-D (our own engine, no WebGL) so you can see the shape of a forward pass, click any block, and understand the one trick that makes generation fast: the KV cache.
Here is the whole computation, standing up. At the bottom, tokens become vectors (embedding + position). That bundle of vectors — one per token — is the residual stream, and it flows straight up the middle. Every block reads the stream, computes something, and adds it back: first attention (tokens exchange information), then a small FFN (each token thinks privately). Add, add, add — 12 times for GPT-2, 96 for GPT-3 — and at the top a final projection turns the stream into logits: one score per vocabulary word, which softmax makes a probability. Slide to pick a block and watch it light up; its attention pattern appears on the right.
illustrative causal attention for this block — each row is a token attending to itself and the past (lower-triangular: no peeking ahead).
The tower looks deep, but there's only one design repeated. Each block is: attention (the warm slab) — where tokens look at each other and pull in context, exactly the Q·Kᵀ→softmax→·V you drove in Phase 5 — followed by a feed-forward network (the gold slab) — a little two-layer MLP applied to every token separately, where most of the parameters (and most of the "knowledge") live. Both write their output back into the residual stream by addition, so information accumulates rather than being replaced. That additive stream is why a 96-layer model trains at all (the residual highways from Phase 5) — and why you can read the stream at any height as "the model's current best guess so far."
Generation is one-token-at-a-time: predict a token, append it, run the whole tower again for the next. Naively, step 100 would re-process all 99 previous tokens through all 12 blocks — quadratic waste. The fix is the KV cache. In attention, each past token contributes a Key and a Value; those never change once computed. So cache them. At each new step you compute Q, K, V for only the new token, and attend over the cached K/V of everything before. Step through the six frames below: both strategies write the same sentence, and the running totals underneath count the K/V pairs each one actually computed.
The cache is a classic time-for-memory trade: you stop recomputing K/V (huge compute saving) by storing them (memory that grows with context length — the same n that makes attention n² in Phase 5). It's why long conversations eat GPU memory, why "context window" and "KV-cache size" are the numbers that bound what a server can hold, and why serving engines (vLLM's paged attention) obsess over managing it.
Curated companion: bbycroft — LLM Visualization — a full WebGL walkthrough of every matmul in nano-GPT, with real weights.