Iterators & Generators
The iterator protocol, generators, yield, and lazy sequences.
12 questions
JuniorTheoryVery commonHow do [x for x in y] and (x for x in y) differ?
How do [x for x in y] and (x for x in y) differ?
[...] is a list comprehension: it eagerly builds a full list in memory holding every element. (...) is a generator expression: it returns a lazy generator that yields items one at a time, keeping only the current state.
Common mistakes
- ✗Believing
(...)builds atuple— it produces a generator, not a tuple - ✗Assuming both forms materialize a full collection, when the generator expression is lazy
- ✗Swapping which form is lazy —
[...]is eager,(...)is the lazy generator
Follow-up questions
- →How would you sum a million squares without ever building a million-element list?
- →Can you index into the result of
(x for x in y), and why or why not?
JuniorTheoryVery commonWhat is an iterable in Python?
What is an iterable in Python?
An iterable returns its elements one at a time — anything defining __iter__ (or legacy __getitem__ from index 0). iter() turns it into an iterator. list, tuple, str, dict, set and files all qualify.
Common mistakes
- ✗Believing every iterable must be a sequence with indexing —
setanddictare iterable yet unordered/unindexed - ✗Confusing an iterable with an iterator — an iterable yields a fresh iterator via
iter(), it is not itself the cursor - ✗Thinking custom classes cannot be iterable, when defining
__iter__makes any class usable infor
Follow-up questions
- →What does
iter()return when called on alist, and is it the same object each time? - →How can you make a custom class iterable without writing a separate iterator class?
JuniorCodeCommonWhat does listing a generator twice print?
What does listing a generator twice print?
(1) [0, 1, 2], (2) []. Generators are lazy and single-pass — once exhausted, they yield nothing on a second iteration. Re-create the generator, or materialize it into a list, if you need to iterate more than once.
Common mistakes
- ✗Expecting a generator to restart on a second iteration
- ✗Thinking exhaustion raises StopIteration to the caller of list()
- ✗Reusing a generator where a list is needed
Follow-up questions
- →Why does
list()return[]rather than raising on an exhausted generator? - →When is a one-pass generator preferable to materializing a list?
JuniorTheoryCommonWhat is a generator function in Python?
What is a generator function in Python?
A function whose body contains yield. Calling it does not run the body; it returns a generator object (an iterator). The body runs lazily, advancing one step each time you call next() on the returned generator.
Common mistakes
- ✗Thinking calling a generator function runs its body — it only returns a generator object, body runs on
next() - ✗Believing it returns a
listof all values eagerly rather than yielding lazily - ✗Treating
returnandyieldas interchangeable — onlyyieldmakes a function a generator
Follow-up questions
- →What happens to the generator's local variables between two successive
next()calls? - →How do you trigger the function body to start executing after the generator is created?
MiddleTheoryCommonWhat is an iterator, and what protocol must it implement?
What is an iterator, and what protocol must it implement?
An iterator implements __next__ (next item, raises StopIteration when exhausted) and __iter__ (returns self), so every iterator is iterable. for calls iter() once, then next() repeatedly; once exhausted it stays exhausted.
Common mistakes
- ✗Thinking an iterator needs only
__iter__— it must define__next__and raiseStopIterationto end - ✗Assuming an exhausted iterator auto-resets, when it stays permanently exhausted
- ✗Forgetting that
__iter__on an iterator returnsself, which is why every iterator is iterable
Follow-up questions
- →Why does every iterator also need an
__iter__method that returnsself? - →What happens if you call
next()on an iterator that has already raisedStopIteration?
MiddleTheoryCommonWhat does yield do inside a generator?
What does yield do inside a generator?
yield freezes the function's execution state and hands the current value back to the caller. The next next() resumes right after the yield, with all local variables preserved between resumes, so it picks up where it paused.
Common mistakes
- ✗Treating
yieldlikereturn— it pauses and resumes, it does not terminate the function - ✗Thinking
yieldbuilds and returns a fulllistof all values in one call - ✗Assuming local variables are reset between resumes, when they are preserved across each
yield
Follow-up questions
- →What value does the
yieldexpression itself evaluate to when you usegen.send(x)? - →How does a
forloop know the generator is finished after its finalyield?
JuniorCodeOccasionalImplement your own reversed as a generator and an iterator
Implement your own reversed as a generator and an iterator
As a generator, loop the index from len(seq)-1 down to 0 and yield seq[i] — lazy, one element at a time. As an iterator class, store the sequence and a current index in __init__, return self from __iter__, and in __next__ decrement the index and return the element, raising StopIteration when it reaches 0. Both are equivalent; the generator is just shorter.
Common mistakes
- ✗Omitting
__iter__returningself, so the object isn't usable in aforloop - ✗Forgetting to raise
StopIterationwhen the index is exhausted - ✗Materializing a reversed copy instead of yielding lazily by index
Follow-up questions
- →Why must
__iter__returnselffor the iterator to work in aforloop? - →What does
StopIterationsignal, and who catches it?
MiddleCodeOccasionalFlatten an arbitrarily nested list
Flatten an arbitrarily nested list
Recurse with a generator: for each item, if it is a list, yield from flatten(item); otherwise yield item. yield from delegates to the sub-generator, flattening any depth lazily. Materialize the result with list(flatten(...)) when you need a concrete list.
Common mistakes
- ✗Using a flat comprehension that only un-nests one level
- ✗Believing
sum(nested, [])recurses into deeper nesting - ✗Assuming lists have a built-in
flattenmethod
Follow-up questions
- →What does
yield fromdo that a plainfor ... yieldloop would also achieve? - →How would you extend this to flatten tuples and other iterables, not just lists?
MiddleTheoryOccasionalWhat do a generator's send, throw, and close methods do?
What do a generator's send, throw, and close methods do?
send(x) resumes the generator and makes the paused yield evaluate to x, so next(g) equals send(None). throw(exc) raises exc at the yield point so the generator can catch or clean up. close() raises GeneratorExit there to finalize it.
Common mistakes
- ✗Calling
send(x)before the generator has been primed withnext()orsend(None) - ✗Expecting
throwto raise after the generator finishes rather than at the currentyield - ✗Assuming plain iterators like
listorrangesupportsend/throw/close
Follow-up questions
- →Why must a generator be primed with
next()before its firstsend(x)? - →How should a generator handle
GeneratorExitraised byclose()in afinallyblock?
MiddleTheoryOccasionalWhy is a generator more memory-efficient than a list comprehension?
Why is a generator more memory-efficient than a list comprehension?
The list comprehension allocates and holds all n elements at once (O(n) memory), while the generator holds only the current frame's state and produces each element on demand (O(1) memory). This lets a generator process streams larger than RAM or even infinite sequences, where a list would exhaust memory.
Common mistakes
- ✗Thinking peak memory is still O(n) because Python pre-materializes the result before yielding it
- ✗Assuming the generator's footprint can't beat O(n) since the source iterable already sits in RAM
- ✗Quoting O(log n) or O(√n) memory for a generator instead of the true O(1) per-frame state
Follow-up questions
- →How could a generator process a 50 GB log file on a machine with 8 GB of RAM?
- →When would a list comprehension actually be the better choice despite the memory cost?
SeniorTheoryOccasionalWhat does yield from do with a subgenerator?
What does yield from do with a subgenerator?
yield from subgen delegates to it: it yields every value the subgenerator produces, transparently forwards send and throw, and its own result is the subgenerator's return value (delivered via StopIteration). It flattens nested generators.
Common mistakes
- ✗Thinking
yield fromyields the subgenerator object itself rather than its individual values - ✗Believing it is just
for x in sub: yield x— it also forwardssend/throwand thereturnvalue - ✗Forgetting the subgenerator's
returnvalue becomes theyield fromexpression's result
Follow-up questions
- →Where does the subgenerator's
returnvalue surface, and how do you capture it? - →How does
yield fromsimplify writing a recursive tree-flattening generator?
SeniorTheoryRareHow does iter() fall back to __getitem__ when __iter__ is absent?
How does iter() fall back to __getitem__ when __iter__ is absent?
iter(obj) first looks for __iter__. If it is absent but __getitem__ exists, it builds an iterator that calls obj[0], obj[1], … until IndexError (the legacy sequence protocol). If neither method exists, it raises TypeError.
Common mistakes
- ✗Thinking a missing
__iter__always raises immediately, ignoring the__getitem__fallback - ✗Believing the fallback uses
__len__or aNonesentinel instead of stopping onIndexError - ✗Forgetting that with neither
__iter__nor__getitem__,iter()raisesTypeError
Follow-up questions
- →Why must the
__getitem__fallback start at index 0 specifically? - →How does the fallback iterator know to stop, and which exception signals the end?
