Dan Imhoff

Developer and Photographer

17 February 2016
Unlocking your Zen with Python Comprehensions

So you’ve been doing some Python. (Great choice, by the way!) You’ve heard of list comprehensions, but the for-loop is just so familiar. Here’s the rub: you’ve been conditioned by dominant languages such as C, Java, and JavaScript. These languages are widespread and very popular, but they all lack the syntactic sugar that makes Python such a pleasure to work in.

Let’s get one thing out of the way real quick. Comprehensions do not completely replace for-loops. They are a way to create data structures. Most of the time, you’ll still want to use a for-loop. It’s up to you to know when to use each one.

Generally, comprehensions can replace a loop such as this:

collection = [1, 2, 3]
l = []
for n in collection:
    l.append(n)
# [1, 2, 3]

Comprehensions were made to replace this common, but ugly bit of nonsense. Check it out:

collection = [1, 2, 3]
l = [n for n in collection]  # [1, 2, 3]

Cleaner, more concise, and faster. The comprehension in the example above is almost twice as fast as the loop!

Filtering

You can add a predicate to the comprehension by using an if-statement. If the if-statement is true, the value is added to the resulting list. Otherwise, it is ignored.

[n for n in collection if n % 2 != 0]  # [1, 3]

Transforming

Before the value is added to the resulting list, it can be transformed. Comprehensions have output expressions, so you can transform the value via an expression or even call a function.

[n * 5 for n in collection]  # [5, 10, 15]
def transform(v):
    if v > 1:
        return -1
    return v

[transform(n) for n in collection if n % 2 != 0]  # [1, -1]

Data Structures

Comprehensions come in multiple flavors for various Python data structures. There are list comprehensions, dict comprehensions, set comprehensions, tuple comprehensions (sort of), and generator comprehensions.

For dictionary comprehensions, a slightly different syntax is used. Curly braces are used and a colon separates the key from the value:

mixed = {'foo': 'bar', 'hello': 'world', 'number': 5}
{k: v for k, v in mixed.items() if isinstance(v, str)}  # {'foo': 'bar', 'hello': 'world'}

Like in list comprehensions, the data (both keys and values) can be transformed before being added to the resulting dict.

Set comprehensions also use curly braces, but the lack of the colon tells you and the Python interpreter that this is a set, not a dict!

collection = [1, 1, 2, 2, 3, 3]
{n for n in collection if n < 3}  # {1, 2}

For tuple comprehensions, you actually need to use generator comprehensions and pass the resulting generator into the tuple class to instantiate a new tuple:

collection = [1, 2, 3]
tuple(n for n in collection)  # (1, 2, 3)

Taking this apart, we can inspect what’s happening. We’re first using a comprehension to create a generator (so fucking cool) and then throwing that right into tuple, which takes any iterator and constructs a tuple:

collection = [1, 2, 3]
gen = (n for n in collection)  # <generator object <genexpr> at 0x7f674db5ddc8>
tuple(gen)  # (1, 2, 3)

Whoa. I invite you to try to find a cleaner syntax for creating generators in any other language. This is enlightenment. You are that much closer to achieving your Python Zen.

Read more about comprehensions in the Python docs.


— Dan Imhoff, 2016