Every for loop you've written in this section has relied on a mechanism worth understanding directly: Python's iterator protocol. A generator is a simple, common way to build your own.
Lists, tuples, dictionaries, and strings are all iterable — for x in ... works on all of them because each knows how to produce its items one at a time when asked, via iter() and next() under the hood:
numbers = [1, 2, 3]
iterator = iter(numbers)
print(next(iterator)) # 1
print(next(iterator)) # 2
print(next(iterator)) # 3
# next(iterator) # StopIteration — nothing leftA for loop is really just this pattern, wrapped in convenient syntax — it calls next() repeatedly until it runs out.
Building a full list to loop over it once wastes memory if the list is huge, or impossible if it's infinite. A generator function — using yield instead of return — produces values one at a time, on demand, without ever holding the whole sequence in memory at once:
def count_up_to(n):
i = 1
while i <= n:
yield i
i += 1
for number in count_up_to(5):
print(number) # 1, 2, 3, 4, 5 — computed one at a time, not built as a list first
A generator expression is the lazy cousin of the list comprehension from earlier in this section — swap the square brackets for parentheses:
squares_list = [n ** 2 for n in range(1000000)] # builds all million values right now
squares_gen = (n ** 2 for n in range(1000000)) # builds nothing yet — values come one at a timeList vs. generator — when each makes sense
Use a list when you need to loop over the data more than once, check its length, or index into it. Reach for a generator when you're processing a large or open-ended sequence just once, in order — reading a huge file line by line is a classic real-world case.