Generator

Description

A Generator is a special type of function in programming that allows you to pause and resume execution, producing a sequence of values over time instead of computing them all at once. Unlike regular functions that return a single value and exit, generators use the yield keyword to return values one at a time, preserving their state between calls.

Generators are heavily used in iterators, lazy evaluation, and asynchronous programming, offering efficient ways to process large datasets or streams without memory overhead.

Key Concepts

Concept	Explanation
Yielding	The act of producing a value and pausing the function state
State Preservation	The function’s local variables are saved between calls
Lazy Evaluation	Values are generated only when requested, not all at once
One-Time Use	Most generators can only be consumed once unless re-instantiated

How It Works

Regular Function Example

def square_list(nums):
    return [x*x for x in nums]

Returns: All results at once.

Generator Example

def square_gen(nums):
    for x in nums:
        yield x * x

Returns: One value at a time using next().

Lifecycle of a Generator

Initialization: Function is called but not executed.
First next() call: Execution starts until yield.
Each subsequent next() call: Continues from where it paused.
StopIteration: Raised when the generator completes.

Simple Use Case: Fibonacci Sequence

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

gen = fibonacci()
for _ in range(5):
    print(next(gen))

Output:

Why Use Generators?

Benefit	Explanation
Memory Efficiency	Generates values on the fly—no need to store everything in memory
Performance	Starts yielding results before the full sequence is ready
Infinite Sequences	Easily create endless sequences (e.g., counters, Fibonacci)
Control Flow Tools	Useful in coroutines, pipelines, and async flows

Generator Expressions

Similar to list comprehensions but return a generator:

gen = (x * x for x in range(5))

Use case: Efficient iteration without storing all values.

Behind the Scenes (Python)

When a generator function is called, it returns a generator object that implements the iterator protocol:

__iter__() returns self
__next__() resumes execution until next yield

Generator vs Iterator

Feature	Generator	Iterator
Syntax	Defined using `yield`	Implements `__iter__` and `__next__`
State Tracking	Handled automatically	Handled manually
Readability	Cleaner for sequential data	More flexible but verbose

Advanced: Sending Data into Generators

def echo():
    value = yield
    while True:
        print("Received:", value)
        value = yield

gen = echo()
next(gen)            # Priming the generator
gen.send("hello")    # Prints: Received: hello

Generators in Asynchronous Programming

In Python, asynchronous generators allow await and yield to be used together:

async def async_gen():
    for i in range(3):
        await asyncio.sleep(1)
        yield i

In JavaScript, similar capabilities exist using function* and async function*.

Real-World Use Cases

🗃️ File Streaming

def read_lines(filename):
    with open(filename) as f:
        for line in f:
            yield line

🕹️ Game Loops

Pause and resume game mechanics.

📡 Data Pipelines

Transform or filter data streams lazily.

🔁 Infinite Sequences

def counter(start=0):
    while True:
        yield start
        start += 1

Common Pitfalls

Issue	Explanation
StopIteration Confusion	Generator ends silently unless caught
Can’t Restart	Generator exhausted after full iteration
State Leaks	Variables persist—can cause bugs if reused carelessly
No Random Access	You can’t index like a list

Alternatives

List Comprehensions (if data is small and fits in memory)
Coroutines (if two-way communication is needed)
Async Streams (for IO-heavy use cases)

Copy-Paste Key Formulas Summary

Basic Generator Template

def my_gen():
    yield val1
    yield val2

Infinite Generator

def counter():
    n = 0
    while True:
        yield n
        n += 1

Generator with send()

def gen():
    data = yield
    while True:
        process(data)
        data = yield

Async Generator

async def async_gen():
    yield val

Related Keywords

Async Generator
Coroutine
Event Loop
Generator Expression
Iterator
Lazy Evaluation
Resume Point
Suspend Function
Yield Keyword
Yield From