Generators & `yield`

A function that can pause. Where a normal function runs to return and forgets everything, a generator yields a value, freezes its place, and resumes from exactly there next time you ask. That one capability gives you lazy, memory-cheap sequences — and it's the idea behind iterating huge datasets without loading them.

yieldlazy evaluation one at a timeconstant memory iterator

return forgets; yield remembers

A normal function is one-shot: you call it, it computes, it returns a single result, and its local variables vanish. To produce a sequence it must build the whole thing first — a list of all the values — and hand it back at once.

A function containing yield is a generator. Calling it doesn't run the body at all; it hands you a generator object, paused before the first line. Each time you call next() on it (which is what a for loop does under the hood), it runs until it hits a yield, emits that value, and freezes — local variables and all — until you ask again. Step through one and watch the function pause and resume at the yield:

the generator function:

frozen state between calls:

values produced (one per next):

The payoff: memory

Because a generator only ever holds one value at a time, it costs essentially constant memory no matter how long the sequence is. The eager list version must materialise every value first. For a one-pass computation like a sum, that difference is enormous:

sum([n*n for n in range(N)]) # builds a list of N numbers, then sums
sum(n*n for n in range(N)) # generator: one number at a time

Drag N and compare what each approach keeps in memory at its peak:

list […]

generator

When to use one (and when not)

Use a generator when the sequence is large or infinite, and you consume it once — feeding sum, any, max, a for loop, or streaming rows from a file. It's the natural tool for "process this 10 GB log line by line."
Use a list when you need to index it (x[5]), get its len(), or iterate it more than once. A generator is exhausted after one pass — ask for the next value past the end and you get StopIteration (the loop simply stops).

The lightest way to make one is a generator expression — a comprehension with parentheses: (n*n for n in range(8)). Same laziness, no def needed.

Takeaways: a function with yield is a generator; calling it gives a paused object that runs to the next yield on each next(), remembering its locals in between. That makes sequences lazy — one value at a time, ~constant memory — ideal for big or infinite data consumed once. Lists are eager but re-iterable and indexable.

Watch a generator pause and resume frame-by-frame in Python Tutor.

Generators & yield

return forgets; yield remembers

The payoff: memory

When to use one (and when not)

Generators & `yield`