How to understand Python decorators

Introduction to Decorators

A decorator is a high-level Python syntax. It can be used to process a function, method, or class. In Python, we have several ways to process functions and classes. Compared to other ways, decorator syntax is simple and the code is highly readable. As a result, decorators have a wide range of applications in Python projects. Decorators are often used in scenarios with tangential requirements, some classic ones are inserting logs, performance testing, transaction processing, web permission checking, Cache, etc.

The advantage of decorators is that they can extract a large amount of code from functions that are not related to the function itself and continue to reuse it. That is, functions can be “modified” to behave completely differently, effectively orthogonalizing the business logic. In a nutshell, the role of a decorator is to add additional functionality to an already existing object. For example, for logging, you need to log some functions. The dumb way, adding code to each function, is sad if the code changes. The decorator approach, define a decorator that specializes in logging and decorate the functions that need it.

Python’s Decorator is very similar to Java/C#’s Annotation in that it adds a @XXX annotation in front of the method name to decorate the method with something. However, Java/C#’s Annotation is also prohibitive, and you need to understand a bunch of Annotation library documentation before you can use it, which makes it feel like you’re learning another language. Python uses a very elegant approach compared to Decorator Pattern and Annotation, which doesn’t require you to master any complex OO model or Annotation’s various library rules, it’s all about language level play: a functional programming technique.

The rationale behind decorators

In Python, decorator implementations are very convenient. Here’s why: functions can be thrown around.

Python’s functions are objects

To understand decorators, you must first know that in Python, functions are objects. It’s important to understand this, and let’s see why with an example.

def shout(word="yes"):
    return word.capitalize() + "!"


print(shout())
# outputs : 'Yes!'

# 作为一个对象，你可以像其他对象一样把函数赋值给其他变量
scream = shout

# 注意我们没有用括号：我们不是在调用函数，
# 而是把函数'shout'的值绑定到'scream'这个变量上
# 这也意味着你可以通过'scream'这个变量来调用'shout'函数

print(scream())
# outputs : 'Yes!'

# 不仅如此，这也还意味着你可以把原来的名字'shout'删掉，
# 而这个函数仍然可以通过'scream'来访问
del shout
try:
    print(shout())
except NameError as e:
    print(e)
    # outputs: "name 'shout' is not defined"

print(scream())
# outputs: 'Yes!'

Another interesting feature of Python functions is that they can be defined within another function body.

def talk():
    # 你可以在 'talk' 里动态的(on the fly)定义一个函数...
    def whisper(word="yes"):
        return word.lower() + "..."

    # ... 然后马上调用它！
    print(whisper())


# 每当调用'talk'，都会定义一次'whisper'，然后'whisper'在'talk'里被调用
talk()
# outputs:
# "yes..."

# 但是"whisper" 在 "talk"外并不存在:
try:
    print(whisper())
except NameError as e:
    print(e)
    # outputs : "name 'whisper' is not defined"

Functions references

As you just learned, Python functions are also objects, so.

can be assigned to a variable
can be defined inside another function

So, this means that a function can return another function that.

def get_talk(type="shout"):
    # 我们先动态定义一些函数
    def shout(word="yes"):
        return word.capitalize() + "!"

    def whisper(word="yes"):
        return word.lower() + "..."

    # 然后返回其中一个
    if type == "shout":
        # 注意：我们是在返回函数对象，而不是调用函数，所以不要用到括号 "()"
        return shout
    else:
        return whisper


# 那你改如何使用d呢？
# 先把函数赋值给一个变量
talk = get_talk()
# 你可以发现 "talk" 其实是一个函数对象:
print(talk)
# outputs : <function shout at 0xb7ea817c>

# 这个对象就是 get_talk 函数返回的:
print(talk())
# outputs : Yes!

# 你甚至还可以直接这样使用:
print(get_talk("whisper")())
# outputs : yes...

Since it is possible to return a function, it is also possible to pass it as a parameter.

def shout(word="yes"):
    return word.capitalize() + "!"


scream = shout


def do_something_before(func):
    print("I do something before then I call the function you gave me")
    print(func())


do_something_before(scream)
# outputs:
# I do something before then I call the function you gave me
# Yes!

Decorators in action

All the basics of understanding decorators are now available. Decorators are also known as wrappers, they allow you to execute other code before or after the execution of the decorated function, and they do not require modifications to the function itself.

Handmade decorators

# 一个装饰器是一个需要另一个函数作为参数的函数
def my_shiny_new_decorator(a_function_to_decorate):
    # 在装饰器内部动态定义一个函数：wrapper(原意：包装纸).
    # 这个函数将被包装在原始函数的四周
    # 因此就可以在原始函数之前和之后执行一些代码.
    def the_wrapper_around_the_original_function():
        # 把想要在调用原始函数前运行的代码放这里
        print("Before the function runs")

        # 调用原始函数（需要带括号）
        a_function_to_decorate()

        # 把想要在调用原始函数后运行的代码放这里
        print("After the function runs")

    # 直到现在，"a_function_to_decorate"还没有执行过 (HAS NEVER BEEN EXECUTED).
    # 我们把刚刚创建的 wrapper 函数返回.
    # wrapper 函数包含了这个函数，还有一些需要提前后之后执行的代码，
    # 可以直接使用了（It's ready to use!）
    return the_wrapper_around_the_original_function


# Now imagine you create a function you don't want to ever touch again.
def a_stand_alone_function():
    print("I am a stand alone function, don't you dare modify me")


a_stand_alone_function()
# outputs: I am a stand alone function, don't you dare modify me

# 现在，你可以装饰一下来修改它的行为.
# 只要简单的把它传递给装饰器，后者能用任何你想要的代码动态的包装
# 而且返回一个可以直接使用的新函数:

a_stand_alone_function_decorated = my_shiny_new_decorator(a_stand_alone_function)
a_stand_alone_function_decorated()
# outputs:
# Before the function runs
# I am a stand alone function, don't you dare modify me
# After the function runs

Syntactic sugar for decorators

Let’s rewrite the previous example using the syntax of a decorator.

# 一个装饰器是一个需要另一个函数作为参数的函数
def my_shiny_new_decorator(a_function_to_decorate):
    # 在装饰器内部动态定义一个函数：wrapper(原意：包装纸).
    # 这个函数将被包装在原始函数的四周
    # 因此就可以在原始函数之前和之后执行一些代码.
    def the_wrapper_around_the_original_function():
        # 把想要在调用原始函数前运行的代码放这里
        print("Before the function runs")

        # 调用原始函数（需要带括号）
        a_function_to_decorate()

        # 把想要在调用原始函数后运行的代码放这里
        print("After the function runs")

    # 直到现在，"a_function_to_decorate"还没有执行过 (HAS NEVER BEEN EXECUTED).
    # 我们把刚刚创建的 wrapper 函数返回.
    # wrapper 函数包含了这个函数，还有一些需要提前后之后执行的代码，
    # 可以直接使用了（It's ready to use!）
    return the_wrapper_around_the_original_function


@my_shiny_new_decorator
def another_stand_alone_function():
    print("Leave me alone")


another_stand_alone_function()
# outputs:
# Before the function runs
# Leave me alone
# After the function runs

Yes, that’s it, it’s that simple. @decorator is just a shortcut of the following statement (shortcut).

`1`	`another_stand_alone_function = my_shiny_new_decorator(another_stand_alone_function)`

Decorator syntactic sugar is actually a Python-ified variant of the decorator pattern. To facilitate development, Python has several classic design patterns built in, such as iterators. Of course, you can also use decorators in a stack:

def bread(func):
    def wrapper():
        print("</''''''\>")
        func()
        print("<\______/>")

    return wrapper


def ingredients(func):
    def wrapper():
        print("#tomatoes#")
        func()
        print("~salad~")

    return wrapper


def sandwich(food="--ham--"):
    print(food)


sandwich()
# outputs: --ham--
sandwich = bread(ingredients(sandwich))
sandwich()
# outputs:
# </''''''\>
# #tomatoes#
# --ham--
# ~salad~
# <\______/>

Expressed in Python’s decorator syntax.

def bread(func):
    def wrapper():
        print("</''''''\>")
        func()
        print("<\______/>")

    return wrapper


def ingredients(func):
    def wrapper():
        print("#tomatoes#")
        func()
        print("~salad~")

    return wrapper


@bread
@ingredients
def sandwich(food="--ham--"):
    print(food)


sandwich()
# outputs:
# </''''''\>
# #tomatoes#
# --ham--
# ~salad~
# <\______/>

The order in which the decorators are placed is also important:

def bread(func):
    def wrapper():
        print("</''''''\>")
        func()
        print("<\______/>")

    return wrapper


def ingredients(func):
    def wrapper():
        print("#tomatoes#")
        func()
        print("~salad~")

    return wrapper


@ingredients
@bread
def strange_sandwich(food="--ham--"):
    print(food)


strange_sandwich()
# outputs:
##tomatoes#
# </''''''\>
# --ham--
# <\______/>
# ~salad~

Passing parameters to decorator functions

# 这不是什么黑色魔法(black magic)，你只是必须让wrapper传递参数:
def a_decorator_passing_arguments(function_to_decorate):
    def a_wrapper_accepting_arguments(arg1, arg2):
        print("I got args! Look:", arg1, arg2)
        function_to_decorate(arg1, arg2)

    return a_wrapper_accepting_arguments


# 当你调用装饰器返回的函数式，你就在调用wrapper，而给wrapper的
# 参数传递将会让它把参数传递给要装饰的函数
@a_decorator_passing_arguments
def print_full_name(first_name, last_name):
    print("My name is", first_name, last_name)


print_full_name("Peter", "Venkman")
# outputs:
# I got args! Look: Peter Venkman
# My name is Peter Venkman

Decorators with parameters

In the above decorator call, such as @decorator, the decorator defaults to the function that follows it as the only argument. The decorator syntax allows us to call decorator with other arguments, such as @decorator(a). This provides more flexibility in the writing and use of decorators.

# a new wrapper layer
def pre_str(pre=''):
    # old decorator
    def decorator(F):
        def new_F(a, b):
            print(pre + " input", a, b)
            return F(a, b)

        return new_F

    return decorator


# get square sum
@pre_str('^_^')
def square_sum(a, b):
    return a ** 2 + b ** 2


# get square diff
@pre_str('T_T')
def square_diff(a, b):
    return a ** 2 - b ** 2


print(square_sum(3, 4))
print(square_diff(3, 4))

# outputs:
# ('^_^ input', 3, 4)
# 25
# ('T_T input', 3, 4)
# -7

The pre_str above is a decorator that allows parameters. It is actually a function wrapper around the original decorator and returns a decorator. We can think of it as a closure with environment parameters. When we call it with @pre_str(’^_^’), Python is able to discover this layer of wrapping and pass the arguments to the decorator’s environment. The call is equivalent to.

`1`	`square_sum = pre_str('^_^') (square_sum)`

Decorating “methods in classes”

One of the great things about Python is that methods and functions are really the same, except that the first argument to a method should be a reference to the current object (i.e. self). This also means that you can create decorators for methods in the same way, as long as you remember to take self into account.

def method_friendly_decorator(method_to_decorate):
    def wrapper(self, lie):
        lie = lie - 3  # very friendly, decrease age even more :-)
        return method_to_decorate(self, lie)

    return wrapper


class Lucy(object):

    def __init__(self):
        self.age = 32

    @method_friendly_decorator
    def say_your_age(self, lie):
        print("I am %s, what did you think?" % (self.age + lie))


l = Lucy()
l.say_your_age(-3)
# outputs: I am 26, what did you think?

Of course, if you want to write a very generic decorator that can be used to decorate arbitrary functions and methods, you can ignore the specific arguments and just use *args, **kwargs.

def a_decorator_passing_arbitrary_arguments(function_to_decorate):
    # The wrapper accepts any arguments
    def a_wrapper_accepting_arbitrary_arguments(*args, **kwargs):
        print("Do I have args?:")
        print(args)
        print(kwargs)
        # Then you unpack the arguments, here *args, **kwargs
        # If you are not familiar with unpacking, check:
        # http://www.saltycrane.com/blog/2008/01/how-to-use-args-and-kwargs-in-python/
        function_to_decorate(*args, **kwargs)

    return a_wrapper_accepting_arbitrary_arguments


@a_decorator_passing_arbitrary_arguments
def function_with_no_argument():
    print("Python is cool, no argument here.")


function_with_no_argument()
# outputs
# Do I have args?:
# ()
# {}
# Python is cool, no argument here.

@a_decorator_passing_arbitrary_arguments
def function_with_arguments(a, b, c):
    print(a, b, c)


function_with_arguments(1, 2, 3)
# outputs
# Do I have args?:
# (1, 2, 3)
# {}
# 1 2 3

@a_decorator_passing_arbitrary_arguments
def function_with_named_arguments(a, b, c, platypus="Why not ?"):
    print("Do %s, %s and %s like platypus? %s" % (a, b, c, platypus))


function_with_named_arguments("Bill", "Linus", "Steve", platypus="Indeed!")
# outputs
# Do I have args ? :
# ('Bill', 'Linus', 'Steve')
# {'platypus': 'Indeed!'}
# Do Bill, Linus and Steve like platypus? Indeed!


class Mary(object):
    def __init__(self):
        self.age = 31

    @a_decorator_passing_arbitrary_arguments
    def say_your_age(self, lie=-3):  # You can now add a default value
        print("I am %s, what did you think ?" % (self.age + lie))


m = Mary()
m.say_your_age()
# outputs
# Do I have args?:
# (<__main__.Mary object at 0xb7d303ac>,)
# {}
# I am 28, what did you think?

Decoration class

In the above example, the decorator takes a function and returns a function, thus having the effect of processing the function. In Python 2.6 onwards, decorators are extended to classes. A decorator can take a class and return a class, thus having the effect of processing the class.

def decorator(aClass):
    class newClass:
        def __init__(self, age):
            self.total_display = 0
            self.wrapped = aClass(age)

        def display(self):
            self.total_display += 1
            print("total display", self.total_display)
            self.wrapped.display()

    return newClass


@decorator
class Bird:
    def __init__(self, age):
        self.age = age

    def display(self):
        print("My age is", self.age)


eagleLord = Bird(5)
for i in range(3):
    eagleLord.display()

In the decorator, we return a new class, newClass, in which we record the objects generated by the original class (self.wrapped) and attach a new attribute, total_display, to record the number of calls to display. We also changed the display method at the same time. With this change, our Bird class can now display the number of calls to display.

Built-in decorators

Python has three types of decorators that we often use, property, staticmethod, and classmethod, and they all have one thing in common: they act on top of class methods.

property Decorator

property decorators are used for functions in classes, allowing us to access the return value of a function in the same way as a property.

class XiaoMing:
    first_name = '明'
    last_name = '小'
    @property
    def full_name(self):
        return self.last_name + self.first_name
xiaoming = XiaoMing()
print(xiaoming.full_name)

In the example, we get the return value of the full_name method like a property, which is the point of using property decorators, both to get the value like a property and to do some operations while getting the value.

staticmethod decorator

The staticmethod decorator is also used for methods in classes, which means that the method will be a static method, meaning that the method can be called directly without instantiation, but it also means that it has no self argument and no access to the instantiated object.

class XiaoMing:
    @staticmethod
    def say_hello():
        print('同学你好')


XiaoMing.say_hello()

# 实例化调用也是同样的效果
# 有点多此一举
xiaoming = XiaoMing()
xiaoming.say_hello()

classmethod Decorator

classmethod is still a method used in a class, which means that it will be a class method, meaning that it can be called directly without instantiation, but it also means that it does not have a self argument and cannot access the instantiated object. The difference compared to staticmethod is that it takes a cls argument to the class itself.

class XiaoMing:
    name = '小明'

    @classmethod
    def say_hello(cls):
        print('同学你好， 我是' + cls.name)
        print(cls)

XiaoMing.say_hello()

wraps decorators

A function has not only its execution statement, but also attributes such as __name__(function name), __doc__ (description document), etc., which our previous example will cause to change.

def decorator(func):
    def wrapper(*args, **kwargs):
        """doc of wrapper"""
        print('123')
        return func(*args, **kwargs)

    return wrapper

@decorator
def say_hello():
    """doc of say hello"""
    print('同学你好')

print(say_hello.__name__)
print(say_hello.__doc__)

Since the decorator returns the wrapper function to replace the say_hello function, the function name and help documentation becomes the wrapper function. The solution to this problem is to use the wraps decorator under the functools module.

from functools import wraps

def decorator(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        """doc of wrapper"""
        print('123')
        return func(*args, **kwargs)

    return wrapper

@decorator
def say_hello():
    """doc of say hello"""
    print('同学你好')

print(say_hello.__name__)
print(say_hello.__doc__)

Decorator Summary

The core role of decorators is name binding. this syntax is another manifestation of Python’s multi-programming paradigm. Most Python users don’t really need to define decorators, but are likely to use them. Given the widespread use of decorators in Python projects, it is useful to understand this syntax.

Common error: “decorator” = “decorator pattern”

Design patterns are a big word in the computer world. If you’re a Java programmer and you don’t know anything about design patterns, I bet you’ll have a tough time getting a job interview.

But when writing Python, we rarely talk about “design patterns”. Although Python is also an object-oriented programming language, its duck type design and excellent dynamics dictate that most design patterns are not necessary for us. So many Python programmers may not really apply several design patterns after working with them for a long time.

The exception is Decorator Pattern. Because Python’s “decorator” and “decorator pattern” have the same exact name, I’ve heard more than once that people treat them as the same thing and think that using “decorator " is practicing the “decorator pattern”. But in fact, they are two completely different things.

“The Decorator Pattern is a programming technique based entirely on Object-Orientation. It has several key components: a unified interface definition, a number of classes that follow the interface, and a layer of wrapping between classes. Together, they create a “decorative” effect.

The “decorator” in Python has no direct connection to “object-oriented”, and **it can be a trick that happens between functions and functions. In fact, a “decorator” doesn’t provide some irreplaceable functionality; it’s just a “syntactic sugar”. Here is the code that uses a decorator.

1
2
3

@log_time
@cache_result
def foo(): pass

Essentially the exact equivalent of.

1
2

def foo(): pass
foo = log_time(cache_result(foo))

The biggest credit to decorators is that they allow us to write more intuitive, easy-to-read code for certain scenarios. It is just a “candy”, not a complex programming pattern for some object-oriented domain.

Table of Contents