Chapter 4. Decorators

Python supports a powerful tool called the decorator. Decorators let you add rich features to groups of functions and methods, without modifying them at all; untangle distinct, frustratingly intertwined concerns in your code, in ways not otherwise possible; and build powerful, extensible software frameworks. Many of the most popular and important Python libraries in the world leverage decorators. This chapter teaches you how to do the same.

A decorator is something you apply to a function or method. You’ve probably seen decorators before. There’s a decorator called property often used in classes:

class Person:
    def __init__(self, first_name, last_name):
        self.first_name = first_name
        self.last_name = last_name

    @property
    def full_name(self):
        return self.first_name + " " + self.last_name
>>> person = Person("John", "Smith")
>>> print(person.full_name)
John Smith

Note that it is printing person.full_name, not person.full_name().

For another example: in the Flask web framework, here is how you define a simple home page:

@app.route("/")
def hello():
    return "<html><body>Hello World!</body></html>"

The app.route("/") is a decorator, applied here to the function called hello(). So an HTTP GET request to the root URL (“/”) will be handled by the hello() function.

A decorator works by adding behavior around a function​1—meaning, lines of code which are executed before that function begins, after it returns, or both. It does not alter any lines of code inside the function. Typically, when you go to the trouble to define a decorator, you plan to use it on at least two different functions, usually more. Otherwise you’d just put the extra code inside the lone function, and not bother writing a decorator.

Using decorators is simple and easy; even someone new to programming can learn to use them quickly. Our objective in this chapter is different: to give you the ability to write your own decorators, in many different useful forms. This is not a beginner topic; it barely qualifies as intermediate. Writing decorators requires a deep understanding of several sophisticated Python features and how they play together. Most Python developers never learn how to create them. In this chapter, you will.

The Basic Decorator

Once a decorator is written, using it is easy. You just write @ and the decorator name, on the line before you define a function:

@some_decorator
def some_function(arg):
    # blah blah

This applies the decorator called some_decorator to some_function().2 Now, it turns out this syntax with the @ symbol is a shorthand. In essence, when byte-compiling your code, Python will translate the above into this:

def some_function(arg):
    # blah blah
some_function = some_decorator(some_function)

This is valid Python code too; it is what people did before the @ syntax came along. The key here is the last line:

some_function = some_decorator(some_function)

First, understand that a decorator is just a function. That’s it. It happens to be a function taking one argument, which is the function object being decorated. It then returns a different function. In the code snippet above you are defining a function, initially called some_function. (Remember that some_function is a variable holding a function object, because that is what the def statement does.) That function object is passed to some_decorator(), which returns a different function object, which is finally stored in some_function.

To keep us sane, let’s define some terminology:

  • The decorator is what comes after the @. It’s a function.

  • The bare function is what is def‘ed on the next line. It is, obviously, also a function.

  • The end result is the decorated function. It’s the final function that you actually call in your code.3

Your mastery of decorators will be most graceful if you remember one thing: a decorator is just a normal, boring function. It happens to be a function taking exactly one argument, which is itself a function. And when called, the decorator returns a different function.

Let’s make this concrete. Here’s a simple decorator which logs a message to stdout every time the decorated function is called.

def printlog(func):
    def wrapper(arg):
        print("CALLING: " + func.__name__)
        return func(arg)
    return wrapper

@printlog
def foo(x):
    print(x + 2)

This decorator creates a new function, called wrapper(), and returns that. This is then assigned to the variable foo, replacing the undecorated, bare function:

# Remember, this...
@printlog
def foo(x):
    print(x + 2)

# ...is the exact same as this:
def foo(x):
    print(x + 2)
foo = printlog(foo)

Here’s the result:

>>> foo(3)
CALLING: foo
5

At a high level, the body of printlog() does two things: define a function called wrapper(), then return it. Most decorators follow that structure. Notice printlog() does not modify the behavior of the original function foo itself; all wrapper() does is print a message to standard output, before calling the original (bare) function.

Once you’ve applied a decorator, the bare function isn’t directly accessible anymore; you can’t call it in your code. Its name now applies to the decorated version. But that decorated function internally retains a reference to the bare function, calling it inside wrapper().

Generic Decorators

This version of printlog() has a shortcoming, though. Look what happens when I apply it to a different function:

>>> @printlog
... def baz(x, y):
...     return x ** y
...
>>> baz(3, 2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: wrapper() takes 1 positional argument but 2 were given

Can you spot what went wrong?

printlog() is built to wrap a function taking exactly one argument. But baz has two, so when the decorated function is called, the whole thing blows up. There’s no reason printlog() needs to have this restriction; all it’s doing is printing the function name. You can fix it by declaring wrapper() with variable arguments:

# A MUCH BETTER printlog.
def printlog(func):
    def wrapper(*args, **kwargs):
        print("CALLING: " + func.__name__)
        return func(*args, **kwargs)
    return wrapper

This decorator is compatible with any Python function:

>>> @printlog
... def foo(x):
...     print(x + 2)
...
>>> @printlog
... def baz(x, y):
...     return x ** y
...
>>> foo(7)
CALLING: foo
9
>>> baz(3, 2)
CALLING: baz
9

A decorator written this way, using variable arguments, will potentially work with functions and methods written years later—code the original developer never imagined. This structure has proven to be powerful and versatile.

# The prototypical form of Python decorators.
def prototype_decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

We don’t always do this, though. Sometimes you are writing a decorator that only applies to a function or method with a very specific kind of signature, and it would be an error to use it anywhere else. So feel free to break this rule when you have a reason.

Decorating Methods

Decorators apply to methods just as well as to functions. You often don’t need to change anything: when the wrapper has a signature of wrapper(*args, **kwargs), like printlog() does, it works just fine with any object’s method. But sometimes you will see code like this:

# Not really necessary.
def printlog_for_method(func):
    def wrapper(self, *args, **kwargs):
        print("CALLING: " + func.__name__)
        return func(self, *args, **kwargs)
    return wrapper

This wrapper() has one required argument, named self. It works fine when applied to a method. But for the decorator I’ve written here, self is completely unnecessary, and in fact, it has a downside.

Simply defining wrapper(*args, **kwargs) causes self to be considered one of the args; such a decorator works just as well with both functions and methods. But if a wrapper is defined to require self, that means it must always be called with at least one argument. Suddenly you have a decorator that cannot be applied to functions which do not take at least one argument. (The fact that it’s named self does not matter; it’s just a temporary name for that first argument, inside the scope of wrapper().) You can apply this decorator to any method, and to some functions. But if you apply it to a function that takes no arguments, you’ll get a runtime error.

Now, here’s a different decorator:

# Using self makes sense in this case:
def enhanced_printlog_for_method(func):
    def wrapper(self, *args, **kwargs):
        print("CALLING: {} on object ID {}".format(
            func.__name__, id(self)))
        return func(self, *args, **kwargs)
    return wrapper

It could be applied like this:

class Invoice:
    def __init__(self, id_number, total):
        self.id_number = id_number
        self.total = total
        self.owed = total
    @enhanced_printlog_for_method
    def record_payment(self, amount):
        self.owed -= amount

inv = Invoice(42, 117.55)
print(f"ID of inv: {id(inv)}")
inv.record_payment(55.35)

Here’s the output when you execute:

ID of inv: 4320786472
CALLING: record_payment on object ID 4320786472

This is a different story, because this wrapper()’s body explicitly uses the current object—a concept that only makes sense for methods. That makes the self argument perfectly appropriate. It prevents you from using this decorator on some functions. But since you intend this decorator to only be applied to methods, it would be an error to apply it to a function anyway.

When writing a decorator for methods, I recommend you get in the habit of making your wrapper only take *args and **kwargs, except when you have a clear reason to include self. After you’ve written decorators for a while, you’ll be surprised at how often you end up using old decorators on new callables—both functions and methods—in ways you never imagined at first. A signature of wrapper(*args, **kwargs) preserves that flexibility. If the decorator turns out to need an explicit self argument, it’s easy enough to put that in.

Data in Decorators

Some valuable decorator patterns rely on using variables inside the decorator function itself. This is not the same as using variables inside the wrapper function. Let me explain.

Imagine you need to keep a running average of what a chosen function returns. And further, you need to do this for a family of functions or methods. We can write a decorator called running_average to handle this. As you read, note carefully how data is defined and used:

def running_average(func):
    data = {"total" : 0, "count" : 0}
    def wrapper(*args, **kwargs):
        val = func(*args, **kwargs)
        data["total"] += val
        data["count"] += 1
        print("Average of {} so far: {:.01f}".format(
             func.__name__, data["total"] / data["count"]))
        return val
    return wrapper

Each time the function is called, the average of all calls so far is printed out.4 The decorator itself is called once for each bare function it is applied to. Then, each time the resulting decorated function is called in the code, the wrapper() function is what’s actually executed.

So imagine applying running_average to a function like this:

@running_average
def foo(x):
    return x + 2

This executes running_average() as a function once, which creates an internal dictionary, named data, used to keep track of foo​’s metrics.

But when you run foo(), you do NOT execute running_average() again. Instead, each time you run the decorated function foo(), it is calling the wrapper() which running_average() created internally. This means it can access data from its containing scope.

This may not all make sense yet, so let’s step through it. Running foo() several times produces:

>>> foo(1)
Average of foo so far: 3.00
3
>>> foo(10)
Average of foo so far: 7.50
12
>>> foo(1)
Average of foo so far: 6.00
3
>>> foo(1)
Average of foo so far: 5.25
3

The placement of data is important. Pop quiz:

  • What happens if you move the line defining data up one line, outside the running_average() function?

  • What happens if you move that line down, into the wrapper() function?

Looking at the code above, decide on your answers to these questions before reading further.

To answer the first question, here’s what it looks like if you create data outside the decorator:

# This version has a bug.
data = {"total" : 0, "count" : 0}
def outside_data_running_average(func):
    def wrapper(*args, **kwargs):
        val = func(*args, **kwargs)
        data["total"] += val
        data["count"] += 1
        print("Average of {} so far: {:.01f}".format(
             func.__name__, data["total"] / data["count"]))
        return func(*args, **kwargs)
    return wrapper

If you do this, every decorated function shares the exact same data dictionary! This actually doesn’t matter if you only ever decorate just one function. But you never bother to write a decorator unless it’s going to be applied to at least two:

@outside_data_running_average
def foo(x):
    return x + 2

@outside_data_running_average
def bar(x):
    return 3 * x

And that produces a problem:

>>> # First call to foo...
... foo(1)
Average of foo so far: 3.0
3
>>> # First call to bar...
... bar(10)
Average of bar so far: 16.5
30
>>> # Second foo should still average 3.00!
... foo(1)
Average of foo so far: 12.0

Because outside_data_running_average() uses the same data dictionary for all the functions it decorates, the statistics are conflated.

Now, for the other question: what if you define data inside wrapper()?

# This version has a DIFFERENT bug.
def running_average_data_in_wrapper(func):
    def wrapper(*args, **kwargs):
        data = {"total" : 0, "count" : 0}
        val = func(*args, **kwargs)
        data["total"] += val
        data["count"] += 1
        print("Average of {} so far: {:.01f}".format(
             func.__name__, data["total"] / data["count"]))
        return func(*args, **kwargs)
    return wrapper

@running_average_data_in_wrapper
def foo(x):
    return x + 2

Look at the average as we call this decorated function multiple times:

>>> foo(1)
Average of foo so far: 3.0
3
>>> foo(5)
Average of foo so far: 7.0
7
>>> foo(20)
Average of foo so far: 22.0
22

Do you see why the running average is wrong? The data dictionary is reset every time the decorated function is called, because it is reset every time wrapper() is called. This is why it’s important to consider the scope when implementing your decorator. Here’s the correct version again (repeated so you don’t have to skip back):

def running_average(func):
    data = {"total" : 0, "count" : 0}
    def wrapper(*args, **kwargs):
        val = func(*args, **kwargs)
        data["total"] += val
        data["count"] += 1
        print("Average of {} so far: {:.01f}".format(
             func.__name__, data["total"] / data["count"]))
        return func(*args, **kwargs)
    return wrapper

So when exactly is running_average() executed? The decorator function itself is executed exactly once for every function it decorates. If you decorate N functions, running_average() is executed N times, so we get N different data dictionaries, each tied to one of the resulting decorated functions. This has nothing to do with how many times a decorated function is executed. The decorated function is, basically, one of the created wrapper() functions. That wrapper() can be executed many times, using the same data dictionary that was in scope when that wrapper() was defined.

This is why running_average produces the correct behavior:

@running_average
def foo(x):
    return x + 2

@running_average
def bar(x):
    return 3 * x
>>> # First call to foo...
... foo(1)
Average of foo so far: 3.0
3
>>> # First call to bar...
... bar(10)
Average of bar so far: 30.0
30
>>> # Second foo gives correct average this time!
... foo(1)
Average of foo so far: 3.0
3

Accessing Inner Data

What if you want to peek into data? The way we’ve written running_average, you can’t—at least not with ordinary Python code. data persists because of the reference inside of wrapper(), and in a sense is veiled from outside access.

But there is an easy solution: simply assign data as an attribute to the wrapper object. For example:

# collectstats is much like running_average, but lets
# you access the data dictionary directly, instead
# of printing it out.
def collectstats(func):
    data = {"total" : 0, "count" : 0}
    def wrapper(*args, **kwargs):
        val = func(*args, **kwargs)
        data["total"] += val
        data["count"] += 1
        return val
    wrapper.data = data
    return wrapper

See that line wrapper.data = data? Yes, you can do that. A function in Python is just an object, and in Python, you can add new attributes to objects by just assigning them. This conveniently annotates the decorated function:

@collectstats
def foo(x):
    return x + 2
>>> foo.data
{'total': 0, 'count': 0}
>>> foo(1)
3
>>> foo.data
{'total': 3, 'count': 1}
>>> foo(2)
4
>>> foo.data
{'total': 7, 'count': 2}

It’s clear now why collectstats doesn’t contain any print statement: you don’t need one! We can check the accumulated numbers at any time, because this decorator annotates the function itself, with that data attribute.

Nonlocal Decorator State

Let’s switch to another problem you might run into. Here’s a decorator that counts how many times a function has been called:

# Watch out, this has a bug...
count = 0
def countcalls(func):
    def wrapper(*args, **kwargs):
        global count
        count += 1
        print(f"# of calls: {count}")
        return func(*args, **kwargs)
    return wrapper

@countcalls
def foo(x):
    return x + 2

@countcalls
def bar(x):
    return 3 * x

This version of countcalls() has a bug. Do you see it?

Here it is: this version stores count as global, meaning every function that is decorated will use the same variable:

>>> foo(1)
# of calls: 1
3
>>> foo(2)
# of calls: 2
4
>>> bar(3)
# of calls: 3
9
>>> bar(4)
# of calls: 4
12
>>> foo(5)
# of calls: 5
7

No good can come from this, so we need a better way. But the solution is trickier than it seems. Here’s one attempt:

# Move count inside countcalls, and remove the
# "global count" line. But it still has a bug...
def countcalls(func):
    count = 0
    def wrapper(*args, **kwargs):
        count += 1
        print(f"# of calls: {count}")
        return func(*args, **kwargs)
    return wrapper

But that just creates a different problem:

>>> foo(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 6, in wrapper
UnboundLocalError: local variable 'count' referenced before assignment

We can’t use global, because it’s not global. But we can use the nonlocal keyword:

# Final working version!
def countcalls(func):
    count = 0
    def wrapper(*args, **kwargs):
        nonlocal count
        count += 1
        print(f"# of calls: {count}")
        return func(*args, **kwargs)
    return wrapper

This finally works correctly:

>>> foo(1)
# of calls: 1
3
>>> foo(2)
# of calls: 2
4
>>> bar(3)
# of calls: 1
9
>>> bar(4)
# of calls: 2
12
>>> foo(5)
# of calls: 3

Applying nonlocal gives the count variable a special scope that is part way between local and global. Essentially, Python will search for the nearest enclosing scope that defines a variable named count, and use it like it’s a global.

You may be wondering why we didn’t need to use nonlocal with the first version of running_average above—here it is again, for reference:

def running_average(func):
    data = {"total" : 0, "count" : 0}
    def wrapper(*args, **kwargs):
        val = func(*args, **kwargs)
        data["total"] += val
        data["count"] += 1
        print("Average of {} so far: {:.01f}".format(
             func.__name__, data["total"] / data["count"]))
        return func(*args, **kwargs)
    return wrapper

The line count += 1 is actually modifying the value of the count variable itself, because it really means count = count + 1. And whenever you modify (instead of just read) a variable that was created in a larger scope, Python requires you to declare that’s what you actually want, with global or nonlocal.

Here’s the sneaky thing: when you write data["count"] += 1, you are not actually modifying data! Or rather, you’re not modifying the variable named data, which points to a dictionary object. Instead, the statement data["count"] += 1 invokes a method on the data object. (Specifically, __setitem__().5)

This does change the state of the dictionary. But it doesn’t make data point to a different dictionary. In contrast, count += 1 makes count point to a different integer, so use nonlocal there.

Decorators That Take Arguments

Early in this chapter, I showed you an example decorator from the Flask framework:

@app.route("/")
def hello():
    return "<html><body>Hello World!</body></html>"

This is different from any decorator we’ve implemented so far, because it actually takes an argument. How do we write decorators that can do this? For example, imagine a family of decorators adding a number to the return value of a function:

def add2(func):
    def wrapper(n):
        return func(n) + 2
    return wrapper

def add4(func):
    def wrapper(n):
        return func(n) + 4
    return wrapper

@add2
def foo(x):
    return x ** 2

@add4
def bar(n):
    return n * 2

There is literally only one character difference between add2 and add4. Wouldn’t it be better if we can do something like this:

@add(2)
def foo(x):
    return x ** 2

@add(4)
def bar(n):
    return n * 2

We can. The key is to understand that add is actually not a decorator; it is a function that returns a decorator. In other words, add is a function that returns another function—since the returned decorator is, itself, a function.

To make this work, we write a function called add(), which creates and returns the decorator:

def add(increment):
    def decorator(func):
        def wrapper(n):
            return func(n) + increment
        return wrapper
    return decorator

It’s easiest to understand from the inside out:

  • The wrapper() function is just like in the other decorators. Ultimately, when you call foo() (the original function name), it’s actually calling wrapper().

  • Moving up, we have the aptly named decorator. It is a function.

  • At the top level is add. This is not a decorator. It’s a function that returns a decorator.

Notice the closure here. The increment variable is encapsulated in the scope of the add() function. We cannot access its value outside the decorator, in the calling context. But we don’t need to, because wrapper() itself has access to it.

Suppose the Python interpreter is parsing your program and encounters the following code:

@add(2)
def f(n):
    # ....

Python takes everything between the @ symbol and the end-of-line character as a single Python expression—add(2) in this case. That expression is evaluated. This all happens at compile time. Evaluating the decorator expression means executing add(2), which will return a function object. That function object is the decorator. It’s named decorator inside the body of the add() function, but it doesn’t really have a name at the top level; it’s just applied to f().

What can help you see more clearly is to think of functions as things that are stored in variables. In other words, if I write def foo(x): in my code, I can say to myself: “I’m creating a function called foo”.

But there is another way to think about it. I could instead say: “I’m creating a function object, and storing it in a variable called foo”. Believe it or not, this is closer to how Python actually works. So things like this are possible:

>>> def foo():
...     print("This is foo")
>>> baz = foo
>>> baz()
This is foo
>>> # foo and baz have the same id()... so they
... # refer to the same function object.
>>> id(foo)
4301663768
>>> id(baz)
4301663768

Now, back to add. As you realize add(2) returns a function object, it’s easy to imagine storing that in a variable named add2. As a matter of fact, you can do this:

add2 = add(2)
@add2
def foo(x):
    return x ** 2

Remember that @ is a shorthand:

# This...
@some_decorator
def some_function(arg):
    # blah blah

# ... is translated by Python into this:
def some_function(arg):
    # blah blah
some_function = some_decorator(some_function)

So for add, the following are all equivalent:

add2 = add(2) # Store the decorator in the add2 variable

# This function definition...
@add2
def foo(x):
    return x ** 2

# ... is translated by Python into this:
def foo(x):
    return x ** 2
foo = add2(foo)

# But also, this...
@add(2)
def foo(x):
    return x ** 2

# ... is translated by Python into this:
def foo(x):
    return x ** 2
foo = add(2)(foo)

Look over these variations, and trace through what’s going on in your mind, until you understand how they are all equivalent. The expression add(2)(foo) in particular is interesting. Python parses this left-to-right. So it first executes add(2), which returns a function object. In this expression, that function has no name; it is temporary and anonymous. The very next character is “(”. This causes Python to take that anonymous function object, and immediately call it. It is called with the argument foo (which is the bare function—the function which we are decorating). The anonymous function then returns a different function object, which we finally store in the variable called foo, reassigning that variable name.

Study that previous paragraph until you fully get it. This is important.

Notice that in the line foo = add(2)(foo), the name foo means something different each time it’s used. When you write something like n = n + 3, the name n refers to something different on either side of the equals sign. In the exact same way, in the line foo = add(2)(foo), the variable foo holds two different function objects on the left and right sides.

Class-Based Decorators

I lied to you.

I repeatedly told you that a decorator is just a function. Well, decorators are usually implemented as functions; that is true. But it is also possible to implement a decorator as a class. In fact, any decorator that you can implement as a function can be implemented with a class instead.

Why would you do this? For certain kinds of decorators, classes are better suited. They can be more readable, and otherwise easier to work with. In addition, they let you use the full feature set of Python’s object system. For example, if you have a collection of related decorators, you can leverage inheritance to collect their shared code.

Most decorators are probably better implemented as functions, though it depends on whether you prefer object-oriented or functional abstractions. It is best to learn both ways, then decide which you prefer in your own code on a case-by-case basis.

Implementing Class-Based Decorators

The secret to decorating with classes is a magic method: __call__(). Any object can implement __call__() to make itself callable—meaning, it can be called like a function. Here’s an example:

class Prefixer:
    def __init__(self, prefix):
        self.prefix = prefix
    def __call__(self, message):
        return self.prefix + message

We can call instances of this class like a function:

>>> simonsays = Prefixer("Simon says: ")
>>> simonsays("Get up and dance!")
'Simon says: Get up and dance!'

Just looking at simonsays("Get up and dance!") in isolation, you’d never guess it is anything other than a normal function. In fact, it’s an instance of Prefixer.

What’s happening is that any time you type an identifier name, followed by the “(” (left parenthesis) character, Python immediately translates that into the __call__() method. In fact, amazingly enough, this is true even for regular functions!

>>> def f(x):
...     return x + 1
>>> # This...
>>> f(2)
3
>>> # ... is the EXACT SAME as this:
>>> f.__call__(2)
3

When you use __call__(), you are hooking into something very fundamental in how Python works.

You can use __call__() to implement decorators. Before proceeding, think back to the @printlog decorator.Using this information about __call__(), how might you implement printlog() as a class instead of a function? I encourage you to pause and try implementing it yourself, before you proceed.

The basic approach is to pass func (the bare function) to the constructor of a decorator class, and adapt wrapper() to be the __call__() method. Like this:

class PrintLog:
    def __init__(self, func):
        self.func = func
    def __call__(self, *args, **kwargs):
        print(f"CALLING: {self.func.__name__}")
        return self.func(*args, **kwargs)

# Compare to the function version you saw earlier:
def printlog(func):
    def wrapper(*args, **kwargs):
        print("CALLING: " + func.__name__)
        return func(*args, **kwargs)
    return wrapper
>>> @PrintLog
... def foo(x):
...     print(x + 2)
...
>>> @printlog
... def baz(x, y):
...     return x ** y
...
>>> foo(7)
CALLING: foo
9
>>> baz(3, 2)
CALLING: baz
9

From the user’s point of view, @Printlog and @printlog work exactly the same.

Benefits of Class-Based Decorators

Class-based decorators have some advantages over function-based decorators. For one thing, the decorator is a class, which means you can leverage inheritance. So if you have a family of related decorators, you can reuse code between them. Here’s an example:

import sys
class ResultAnnouncer:
    stream = sys.stdout
    prefix = "RESULT"
    def __init__(self, func):
        self.func = func
    def __call__(self, *args, **kwargs):
        value = self.func(*args, **kwargs)
        self.stream.write(f"{self.prefix}: {value}\n")
        return value

class StdErrResultAnnouncer(ResultAnnouncer):
    stream = sys.stderr
    prefix = "ERROR"

Another benefit is when you prefer to accumulate state in object attributes, instead of a closure. For example, the countcalls function decorator (discussed in “Nonlocal Decorator State”) could be implemented as a class:

class CountCalls:
    def __init__(self, func):
        self.func = func
        self.count = 0
    def __call__(self, *args, **kwargs):
        self.count += 1
        print(f"# of calls: {self.count}")
        return self.func(*args, **kwargs)

@CountCalls
def foo(x):
    return x + 2

Notice this also lets us access foo.count, if we want to check the count outside of the decorated function. The function version didn’t let us do this.6

When creating decorators which take arguments, the structure is a little different. In this case, the constructor accepts not the func object to be decorated, but the parameters on the decorator line. The __call__() method must take the func object, define a wrapper function, and return it—similar to simple function-based decorators:

# Class-based version of the "add" decorator above.
class Add:
    def __init__(self, increment):
        self.increment = increment
    def __call__(self, func):
        def wrapper(n):
            return func(n) + self.increment
        return wrapper

You then use it like you would any other argument-taking decorator:

>>> @Add(2)
... def foo(x):
...     return x ** 2
...
>>> @Add(4)
... def bar(n):
...     return n * 2
...
>>> foo(3)
11
>>> bar(77)
158

Any function-based decorator can be implemented as a class-based decorator; you simply adapt the decorator function itself to __init__(), and wrapper() to __call__(). It’s possible to design class-based decorators which cannot be translated into a function-based form, though.

For certain complex decorators, some people feel that class-based decorators are easier to read than function-based ones. In particular, many people seem to find multinested def statements hard to reason about. Others (including your author) feel the opposite. This is a matter of preference,7 and I recommend you practice with both styles before coming to your own conclusions.

In this section, I have capitalized the names of class-based decorators. I did this because I think it makes this section easier to understand. But that is not what I actually do in real code.

We have conflicting conventions here. Python class names are capitalized, and function names start with a lowercase letter. But whether a decorator is implemented as a function or class is an implementation detail. In fact, you may create a decorator as a function, and then later refactor it as a class—or vice versa. People using your decorator probably do not know or care whether it’s a class or a function; to change its casing each time is a waste of effort, and breaks all the code using that decorator already.

So in my own code, I always use function-name conventions for decorators, even if they are implemented as a class. Essentially creating a higher-priority naming convention. I suggest you do this also; it will make it easier for everyone using the decorators you write, including you.

Decorators for Classes

I lied to you again. I said decorators are applied to functions and methods. Well, they can also be applied to classes.

Understand this has nothing to do with the last section’s topic, on implementing decorators as classes. A decorator can be implemented as a function, or as a class; and that decorator can be applied to a function, or to a class. They are independent ideas. Here, we are talking about how to decorate classes instead of functions.

To introduce an example, let me explain Python’s built-in repr() function. When called with one argument, this returns a string, meant to represent the passed object. It’s similar to str(); the difference is that while str() returns a human-readable string, repr() is meant to return a string version of the Python code needed to recreate it. So imagine a simple Penny class:

class Penny:
    value = 1

penny = Penny()

Ideally, repr(penny) returns the string "Penny()". But that’s not what happens by default:

>>> class Penny:
...     value = 1
>>> penny = Penny()
>>> repr(penny)
'<__main__.Penny object at 0x10229ff60>'

You can fix this by implementing a __repr__() method on your classes, which repr() will use:

>>> class Penny:
...     value = 1
...     def __repr__(self):
...         return "Penny()"
>>> penny = Penny()
>>> repr(penny)
'Penny()'

Let’s create a decorator that will automatically add a __repr__() method to any class. You might be able to guess how it works. Instead of a wrapper function, the decorator returns a class:

>>> def autorepr(cls):
...     def cls_repr(self):
...         return f"{cls.__name__}()"
...     cls.__repr__ = cls_repr
...     return cls
...
>>> @autorepr
... class Penny:
...     value = 1
...
>>> penny = Penny()
>>> repr(penny)
'Penny()'

It’s suitable for classes with no-argument constructors, like Penny. Note how the decorator modifies cls directly. The original class is returned; that original class just now has a __repr__() method. Can you see how this is different from what we did with decorators of functions? With those, the decorator returned a new, different function object.

Another strategy for decorating classes is closer in spirit: creating a new subclass within the decorator, returning that subclass in place of the original:

def autorepr_subclass(cls):
    class NewClass(cls):
        def __repr__(self):
            return f"{cls.__name__}()"
    return NewClass

This has the disadvantage of creating a new type:

>>> @autorepr_subclass
... class Nickel:
...     value = 5
...
>>> nickel = Nickel()
>>> type(nickel)
<class '__main__.autorepr_subclass.<locals>.NewClass'>

The resulting object’s type is not obviously related to the decorated class. That makes debugging harder, creates unclear log messages, and throws a greasy wrench into the middle of your inheritance hierarchy. For this reason, I recommend the first approach.

Class decorators tend to be less useful in practice than those for functions and methods. One real use is to automatically add dynamically generated methods. But they are more flexible than that. You can even express the singleton pattern using class decorators:

def singleton(cls):
    instances = {}
    def get_instance():
        if cls not in instances:
            instances[cls] = cls()
        return instances[cls]
    return get_instance

# There is only one Elvis.
@singleton
class Elvis:
    pass

Note the IDs are the same:

>>> elvis1 = Elvis()
>>> elvis2 = Elvis()
>>> id(elvis1)
4410742144
>>> id(elvis2)
4410742144

This singleton class decorator does not return a class; it returns a function object (get_instance). This means that in your code, Elvis will actually be that function object, rather than a class.

This is not without consequences; writing code like isinstance(elvis1, Elvis) will raise a TypeError, for example. This and other effects create code situations which are confusing, to say the least. It illustrates once again why we want to be careful about what is actually returned when we write decorators for classes.

Conclusion

Decorators are a powerful tool for metaprogramming in Python. It is no mistake that some of the most successful and famous Python libraries use them extensively in their codebase. When you learn to write decorators, not only do you add this power tool to your toolbox, but you also permanently level up your deep understanding of Python itself.

1 Or method. When talking about the target of a decorator, “function” will always mean “function or method”, unless I say otherwise.

2 For Java people: this looks just like Java annotations. However, it is completely different. Python decorators are not in any way similar.

3 Some authors use the phrase “decorated function” to mean “the function that is decorated”—what I’m calling the “bare function”. If you read a lot of blog posts, you’ll find the phrase used both ways. (Sometimes in the same article.) This book will consistently use the definitions I give here.

4 In a real application, you’d write the average to some kind of log sink, but we’ll use print() here because it’s convenient for learning.

5 When you write data["count"] += 1, Python reads that as data.__setitem__("count", data["count"] + 1).

6 This would also be a way to expose the data dict of the @collectstats decorator, from earlier in the chapter.

7 I believe it may have more to do with what language you “grew up” with. Someone who first learned to code in Java will tend to think best in terms of classes; someone with a mathematical background, or whose first coding language was more function oriented, will tend to think better in terms of nested functions.

Get Powerful Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial