Neural net playground

This page trains a real neural network — forward pass, backprop, mini-batches, the lot — live in your browser, on data you choose. No library, no server: ~60 lines of the same math you'll write in BYO-1. Your job: discover why hidden layers exist by watching a net with none fail, then watching one hidden layer bend the impossible into the easy.

perceptron → MLPXOR epochs & batcheshidden features

A single neuron draws a line

Strip the mystique: one neuron computes w·x + b — a dot product and a shift — then squashes it. From Phase 2 you know exactly what that is: a line (in 2-D) with a smooth yes/no gradient across it. A network with no hidden layers is just that one line-drawer, however long you train it. So here's the experiment the whole field was stuck on for a decade: give the line-drawer the XOR dataset — blue in two opposite corners, green in the other two. No single line can ever separate them. Press train and watch it flail. Then add one hidden layer and watch the problem dissolve.

Train a net — pick data, pick a brain, press ▶

hidden layers folding stages: 0 = one straight line (perceptron); each layer bends what the last built

neurons / layer folds per stage: too few can't bend enough (underfits); more = slower, eventually overkill

learning rate step size downhill: too high → loss bounces or explodes; too low → crawls forever

activation the bend between layers: tanh = smooth S-curve; ReLU = kink at 0 (fast, but neurons can "die")

epoch 0 · loss — · acc —

What each hidden neuron learned (first hidden layer)

Run the script of history yourself: (1) XOR + 0 hidden layers → the loss jams near 0.69 (that's −log 0.5 — the net is literally guessing) and the background stays a helpless single gradient. (2) Slide hidden layers to 1 → within a few hundred epochs the background folds into a curved region and accuracy hits ~100%. Nothing about the training algorithm changed — only the shape of the function family the net can express. That's why an MLP is defined by having at least one hidden layer: the hidden layer is what upgrades "a line" to "a composition of lines", and compositions can bend.

Hidden neurons are lines too — the output combines their folds

Look at the gallery under the board while a 1-layer net trains on XOR. Each mini-map is one hidden neuron's view of the plane — and every one of them is still just a line (a straight color boundary). The magic is one level up: the output neuron takes a weighted vote of the folds, and "above line A but below line B" is a corner — something no single line can say. Depth stacks this: layer 2 draws lines in the feature space layer 1 built, which look like curves in pixel space. Try the spiral with 1 layer (struggles, wobbles) then 2 × 8 with tanh — features of features.

🎛 Things worth breaking (beyond the guided experiments): crank the learning rate to 3 on the spiral — the loss wobbles high forever instead of descending (the overshoot you met in the optimizer explainer; small nets on easy data often survive lr 3, big nets on hard data don't) · set ReLU with 2 layers and retrain a few times — some runs are faster than tanh, some neurons flatline to a dead constant color (dying ReLU) · train blobs with 2×8 — massive overkill works, but watch how weirdly confident the background is far from any data. That's overparameterization, and it's Phase 3's overfitting story wearing a new coat.

Epochs, batches, iterations — the vocabulary of a training loop

This playground holds 240 points and trains on mini-batches of 24, so one pass through the data — one epoch — takes exactly 10 iterations (weight updates). That's the whole relationship tests: iterations per epoch = samples ÷ batch size (1000 samples at batch 100 → 10 iterations). Batches exist because the full-data gradient is expensive and the single-sample gradient is noisy; 24-at-a-time is the compromise that also happens to love GPU parallelism. The "epoch" counter above is honest — it increments every 10 updates.

⚠️ Traps: an MLP has at least one hidden layer — zero hidden layers is a plain perceptron/logistic regression · a single-layer perceptron cannot learn XOR; one hidden layer can · ReLU is non-linear — piecewise-linear still bends, and bending is all you need · more layers is not automatically better: harder to train (vanishing gradients), easier to overfit · 1000 samples, batch 100 → 10 iterations/epoch, not 100.

Check yourself

Takeaways: a neuron = w·x+b = a line · no hidden layers = only lines, hence the XOR wall · hidden neurons are lines whose combination bends — depth = features of features · epoch = full pass; iteration = one batch update; iterations/epoch = N ÷ batch · training = the Phase 2 descent loop, watched through a loss curve. Next: what the squashing functions actually do — activations.

Curated companion: TensorFlow Playground — the original browser net, with feature engineering knobs ours deliberately hides.