Inside an LLM

You built the transformer block in Phase 5. Now watch a whole GPT think: a token goes in at the bottom, climbs a tower of identical blocks — each one reading the running total and adding its two cents — and a probability over the next token comes out the top. This page renders that tower in real 3-D (our own engine, no WebGL) so you can see the shape of a forward pass, click any block, and understand the one trick that makes generation fast: the KV cache.

residual streamstacked blocks KV cachelogits → token

The forward pass is a tower

Here is the whole computation, standing up. At the bottom, tokens become vectors (embedding + position). That bundle of vectors — one per token — is the residual stream, and it flows straight up the middle. Every block reads the stream, computes something, and adds it back: first attention (tokens exchange information), then a small FFN (each token thinks privately). Add, add, add — 12 times for GPT-2, 96 for GPT-3 — and at the top a final projection turns the stream into logits: one score per vocabulary word, which softmax makes a probability. Slide to pick a block and watch it light up; its attention pattern appears on the right.

A GPT forward pass — ▶ play it, block by block

Block 1 of 12

illustrative causal attention for this block — each row is a token attending to itself and the past (lower-triangular: no peeking ahead).

GPT-2 small = 12 · GPT-3 = 96 · every block is identical, just more of them — press ▶ to run the pass

Every block does the same two things

The tower looks deep, but there's only one design repeated. Each block is: attention (the warm slab) — where tokens look at each other and pull in context, exactly the Q·Kᵀ→softmax→·V you drove in Phase 5 — followed by a feed-forward network (the gold slab) — a little two-layer MLP applied to every token separately, where most of the parameters (and most of the "knowledge") live. Both write their output back into the residual stream by addition, so information accumulates rather than being replaced. That additive stream is why a 96-layer model trains at all (the residual highways from Phase 5) — and why you can read the stream at any height as "the model's current best guess so far."

The KV cache — why the 100th token is cheap

Generation is one-token-at-a-time: predict a token, append it, run the whole tower again for the next. Naively, step 100 would re-process all 99 previous tokens through all 12 blocks — quadratic waste. The fix is the KV cache. In attention, each past token contributes a Key and a Value; those never change once computed. So cache them. At each new step you compute Q, K, V for only the new token, and attend over the cached K/V of everything before. Step through the six frames below: both strategies write the same sentence, and the running totals underneath count the K/V pairs each one actually computed.

KV cache: generate “The cat sat on the mat”, one token per step

The cache is a classic time-for-memory trade: you stop recomputing K/V (huge compute saving) by storing them (memory that grows with context length — the same n that makes attention n² in Phase 5). It's why long conversations eat GPU memory, why "context window" and "KV-cache size" are the numbers that bound what a server can hold, and why serving engines (vLLM's paged attention) obsess over managing it.

⚠️ Traps & honesty: the attention maps here are illustrative (deterministic per block, causal-masked) — a real model's patterns are learned and far messier · the KV cache stores K and V, not Q (Q is only ever needed for the current token) · caching saves compute, not memory — it spends memory to buy speed · "residual stream" isn't a special tensor, it's just the running sum every block reads and writes · depth ≠ intelligence linearly: more blocks help, but returns diminish and training gets harder.
Takeaways: a forward pass is a tower — embed → (attention + FFN, added back) × N → logits · the residual stream is the running total every block edits · attention mixes tokens, the FFN holds the knowledge · the KV cache makes generation cheap by never recomputing past Keys/Values, at the cost of memory that grows with context. Next: shrink it to fit — LoRA & quantization.

Curated companion: bbycroft — LLM Visualization — a full WebGL walkthrough of every matmul in nano-GPT, with real weights.