Idiomatic Python

Idiomatic use of language enables clear concise expression of intent.

Python provides a sparse and carefully-considered set of primitives that compose nicely. Coding with good taste takes practice in any language. Luckily, the Python community has produced some great resources on writing idiomatic Python code.

Python core developer Raymond Hettinger’s talks:
The Hitchhiker’s Guide to Python is a handbook of Python best practice.
Fluent Python by Luciano Ramalho is a well-written intermediate-level programming book. Working your way through its chapters will take you from competence to proficiency. The first edition is very readable. The second edition, at 850 pages, is more of a reference tome, though still of excellent quality.
Chip Huyen’s Python-is-cool is a quick primer on some of Python’s unique syntax.

Python language constructs

Modules

Modules can hold variables, functions, class definitions, or executable code. They are the preferred tool for namespacing. Avoid deeply nested modules and cyclic dependencies between modules.

Built-in functions and magic methods

Built-in Functions together with magic methods provide a common interface for Python objects. This reduces annoying inconsistencies like: array.length, string.length() and list.size(). In Python, it’s: len(a), len(s) and len(l).

Laziness

Python 3 makes frequent use of lazy evaluation, for example, range, zip, and dict.items. Syntactic support for laziness comes in the form of the generator, which can help your programs run faster in less memory.

For example:

def unique(iterable, key=lambda x: x):
    seen = set()
    for elem, ekey in ((e, key(e)) for e in iterable):
        if ekey not in seen:
            yield elem
            seen.add(ekey)

Comprehensions

Python logo

Comprehensions are Python’s syntactic sugar for map and filter operations. Comprehensions come in different flavors which evaluate to different types like lists, sets or generators.

Comprehensions combine well with itertools and functools from the standard library to compose pipelines in the style of functional programming.

List comprehensions

unmoldy_twaddled_turnips = [twaddle(turnip)
    for turnip in turnips if not is_moldy(turnip)]

Generator expressions

The lazy version of a list comprehension is a generator expression.

generator = (twaddle(turnip)
    for turnip in turnips if not is_moldy(turnip))

Set and dict comprehensions

letters = {x for x in 'abracadabra' if x not in 'abc'}
squares = {x: x**2 for x in range(10)}

Conditional expressions

The if-elif-else construct in Python is a statement. Conditional expressions return a value.

message = 'ok!' if quux.succeeded else 'failed!'

What’s the difference? An expression is something; a statement does something. Prefer the conditional expression when you want a value. Prefer an if-elif-else for code that performs side-effects.

Comprehensions enable iteration in the form of expressions. Conditional expressions do the same for branching.

Unpacking, args, and kwargs

Python supports destructuring in function arguments and assignments.

def foo(*args, **kwargs):
    """
    Accept positional and keyword arguments and print them.
    """
    print('positional args = {}'.format(', '.join(str(arg) for arg in args)))
    print(', '.join(f'{key}={value}' for key, value in kwargs.items()))

Destructuring assignment

a, b, c, d = (1, 2, 3, 4)
head, *tail = (1, 2, 3, 4)

Swap

a, b = b, a

Decorators

Decorators are syntax for creating wrappers around functions, methods or classes. Usually, you’ll import and use them from libraries. Examples from the standard library include:

Decorators help separate concerns. For example, the purpose of a function is completely orthogonal to caching its result. The lru_cache decorator separates the concept and implementation strategy of caching from whatever function you’re applying it to.

NamedTuple

Named tuples are memory-efficient, immutable records. For a mutable alternative, consider argparse.Namespace which is especially useful for configuration.

Habits to avoid

putting everything in a class; use modules
singleton classes; use modules
static methods; use functions, prefer pure functions
class hierarchies; prefer composition over inheritence
record / struct style classes; use namedtuple or dataclass
getters / setters; use member variables, covert to @property when needed and not before
methods that don’t use self: use a function
defining custom exceptions; prefer built-in exceptions

To a first approximation, stop writing classes, as detailed in a talk by python core developer Jack Diederich. Where modules and pure functions serve, they are a better choice.

Simple is better

Simplicity takes work. The judgement to know whether a little extra effort will pay off or become a maintenance headache comes only with experience—specifically the experience of doing it wrong.

In general, don’t develop against imagined future requirements. The YAGNI principle says “you ain’t gonna need it”. If it turns out that we do need it, we can add it later.

Python has some features that help you avoid unnecessary work. Properties mean that getters and setters can be added transparently to the client and therefore never have to be created “just in case we need them later”. In the same way, functions can become callable objects with no impact on client code.

It’s important to realize that everything has a cost. Each line of code, each layer of indirection has a cost. Tricky code using exotic language features has a higher cost. That cost accrues not just when writing a piece of code, but also falls on every reader, caller and maintainer. Minimizing surface area and cognitive load is typically a great investment.

a = 1

…is better than…

vars = {'a': 1}

…which is still better than…

class VariableTwaddler:
    """Twaddles the variable"""
    def __init__(self, a):
        self.a = a
variableTwaddlerA = VariableTwaddler(1)

Testing

Testing tools

Style

The Python style guide, known as PEP-8, starts with the admonition, “A Foolish Consistency is the Hobgoblin of Little Minds”. At the same time, “Beautiful is better than ugly” and “Readability counts”.

Rather than burn time developing your own style guide, consider using an autoformatter (e.g. Black). Flake8 checks your code against a Big Ol’ List of Rules covering style as well as more substantial issues like unused imports.

Naming

Naming is hard, but there are a few helpful guidelines.

Use plurals for containers:

for thing in things:
    process(thing)

Single-letter variables are useful in the context of abstract values, for example many Python math functions take x as an input.

Name	Meaning
i, j, k	indexes
n, m	counting numbers
x, y, z	real numbers
a, b, c	integers
e, ex	exceptions
f, g, h	functions
p	predicate
_	unused variable

Pedantic code reviewers may insist that terse variable names are always bad. Refer them to the standard libraries.

Avoid naming things after your company, your technology choices, or this week’s new hotness. Your company will rebrand, you’ll swap out that framework, and buzzwords age in the general direction of embarrassing.

Write Pythonic code

Python is a pragmatic language with lots of batteries included that helps you get things done. Don’t write Java or C# code in Python. You’ll write fewer classes and, when you do, magic methods to provide bits of standard interface. That’s one reason Python is one of the most readable languages out there. Fluency with idiomatic Python will help you write code with brevity, clarity and unity that readers will appreciate. Happy hacking!