Idiomatic Python

Idiomatic use of language enables clear concise expression of intent.

Python provides a sparse and carefully-considered set of primitives that compose nicely. Coding with good taste takes practice in any language. Luckily, the Python community has produced some great resources on writing idiomatic Python code.

• Python core developer Raymond Hettinger’s talks:
  • Transforming Code into Beautiful, Idiomatic Python (old but good)
  • Beyond PEP 8 – Best practices for beautiful intelligible code
  • Preventing, Finding, and Fixing Bugs On a Time Budget
  • Python’s Class Development Toolkit
• The Hitchhiker’s Guide to Python is a handbook of Python best practice.
• Fluent Python by Luciano Ramalho is a well-written intermediate-level programming book. Working your way through its chapters will take you from competence to proficiency.
• Chip Huyen’s Python-is-cool is a quick primer on some of Python’s unique syntax.

Python language constructs

Modules

Modules can hold variables, functions, class definitions, or executable code. They are the preferred tool for namespacing. Avoid deeply nested modules and cyclic dependencies between modules.

Built-in functions and magic methods

Built-in Functions together with magic methods provide a common interface for Python objects. This reduces annoying inconsistencies like: array.length, string.length() and list.size(). In Python, it’s: len(a), len(s) and len(l).

Laziness

Python 3 makes frequent use of lazy evaluation, for example, range, zip, and dict.items. Syntactic support for laziness comes in the form of the generator, which can help your programs run faster in less memory.

For example:

def unique(iterable, key=lambda x: x):
    seen = set()
    for elem, ekey in ((e, key(e)) for e in iterable):
        if ekey not in seen:
            yield elem
            seen.add(ekey)

Comprehensions

Comprehensions are Python’s syntactic sugar for map and filter operations. Comprehensions come in different flavors which evaluate to different types, either containers like lists or sets or generators.

Comprehensions combine well with itertools and functools from the standard library to compose pipelines in the style of functional programming.

List comprehensions

unmoldy_twaddled_turnips = [twaddle(turnip)
    for turnip in turnips if not is_moldy(turnip)]

Generator expressions

The lazy version of a list comprehension is a generator expression.

generator = (twaddle(turnip)
    for turnip in turnips if not is_moldy(turnip))

Set and dict comprehensions

letters = {x for x in 'abracadabra' if x not in 'abc'}
squares = {x: x**2 for x in range(10)}

Conditional expressions

Expressions can be evaluated—turned into a value—in contrast to statements which are executed and don’t return a value. Comprehensions enable iteration in the form of expressions. Conditional expressions do the same for branching.

'ok!' if quux.succeeded else 'failed!'

Unpacking, args, and kwargs

Python supports destructuring in function arguments and assignments.

def foo(*args, **kwargs):
    """
    Accept positional and keyword arguments and print them.
    """
    print('positional args = {}'.format(', '.join(str(arg) for arg in args)))
    print(', '.join(f'{key}={value}' for key, value in kwargs.items()))

Destructuring assignment

a, b, c, d = (1, 2, 3, 4)
head, *tail = (1, 2, 3, 4)

Swap

a, b = b, a

Decorators

Decorators are syntax for creating wrappers around functions, methods or classes. Usually, you’ll import and use them from libraries. Examples from the standard library include:

• property
• classmethod
• lru_cache

Decorators help separate concerns. For example, the purpose of a function is completely orthogonal to caching its result. The lru_cache decorator separates the concept and implementation strategy of caching from whatever function you’re applying it to.

NamedTuple

Named tuples are memory-efficient, immutable records. For a mutable alternative, consider argparse.Namespace which is especially useful for configuration.

Habits to avoid

• putting everything in a class; use modules
• singleton classes; use modules
• static methods; use functions, prefer pure functions
• class hierarchies; prefer composition over inheritence
• record / struct style classes; use namedtuple or dataclass
• getters / setters; use member variables, covert to @property when needed and not before
• methods that don’t use self: use a function
• defining custom exceptions; prefer built-in exceptions

To a first approximation, stop writing classes, as detailed in a talk by python core developer Jack Diederich. Where modules and pure functions serve, they are a better choice.

Simple is better

Simplicity takes work. The judgement to know whether a little extra effort will pay off or become a maintenance headache comes only with experience—specifically the experience of doing it wrong.

In general, don’t develop against imagined future requirements. The YAGNI principle says “you ain’t gonna need it”. If it turns out that we do need it, we can add it later.

Python has some features that help you avoid unnecessary work. Properties mean that getters and setters can be added transparently to the client and therefore never have to be created “just in case we need them later”. In the same way, functions can become callable objects with no impact on client code.

It’s important to realize that everything has a cost. Each line of code, each layer of indirection has a cost. Tricky code using exotic language features has a higher cost. That cost accrues not just when writing a piece of code, but also falls on every reader, caller and maintainer. Minimizing surface area and cognitive load is typically a great investment.

a = 1

…is better than…

vars = {'a': 1}

…which is still better than…

class VariableTwaddler:
    """Twaddles the variable"""
    def __init__(self, a):
        self.a = a
variableTwaddlerA = VariableTwaddler(1)

Testing

Testing tools

• py.test
• pytest-cov
• doctest
• Hypothesis

Style

The Python style guide, known as PEP-8, starts with the admonition, “A Foolish Consistency is the Hobgoblin of Little Minds”. At the same time, “Beautiful is better than ugly” and “Readability counts”.

Rather than burn time developing your own style guide, consider using an autoformatter (e.g. Black). Flake8 checks your code against a Big Ol’ List of Rules covering style as well as more substantial issues like unused imports.

Naming

Naming is hard, but there are a few helpful guidelines.

Use plurals for containers:

for thing in things:
    process(thing)

Single-letter variables are useful in the context of abstract values, for example many Python math functions take x as an input.