Free sharing platform for various workplace information and recruitment needs across the country

Python artisans: tips for using decorators

release: 05-22 Category: Job Search Tips


This is the eighth article in the "Python Craftsman" series. [See all articles in the series]

Decorators are a special tool in Python that provide us with the flexibility to modify functions outside of functions. It's a bit like a magic hat with a unique @ symbol. As long as you wear it on top of a function, you can silently change the function's behavior.

You may have dealt with decorators a lot. When doing object-oriented programming, we often use the two built-in decorators @staticmethod and @classmethod. In addition, if you have touched the click module, you will not be unfamiliar with decorators. The most well-known parameter definition interface for click, @ click.option (...), is implemented using decorators.

In addition to using decorators, we often need to write some decorators ourselves. In this article, I will share some little knowledge about decorators from two aspects of best practices and common mistakes.

Best Practice 1. Try using classes to implement decorators

Most decorators are implemented based on functions and closures, but this is not the only way to make decorators. In fact, Python has only one requirement for whether an object can be used in the form of a decorator (@decorator): the decorator must be a "callable" object .

#Using callable can detect whether an object is "callable" >>> def foo (): pass ... >>> type (foo) >>> callable (foo) True

Functions are naturally "callable" objects. But in addition to functions, we can also make any class "callable". The method is very simple, just customize the __call__ magic method of the class.

class Foo: def __call __ (self): print ("Hello, __call___") foo = Foo () # OUTPUT: True print (callable (foo)) # call foo instance # OUTPUT: Hello, __call__ foo ()

Based on this feature, we can easily use classes to implement decorators.

The following code will define a decorator named @delay (duration). The function decorated with it will wait for additional duration seconds before each execution. At the same time, we also want to provide users with an eager_call interface that does not need to wait to execute immediately.

import time import functools class DelayFunc: def __init __ (self, duration, func): self.duration = duration self.func = func def __call __ (self, * args, ** kwargs): print (f "Wait for {self.duration } seconds ... ") time.sleep (self.duration) return self.func (* args, ** kwargs) def eager_call (self, * args, ** kwargs): print (" Call without delay ") return self .func (* args, ** kwargs) def delay (duration): "" "Decorator: Delay the execution of a function. Also provide the .eager_call method to execute immediately" "" # In order to avoid defining additional functions, use directly functools.partial help construct # DelayFunc instance return functools.partial (DelayFunc, duration)

Sample code on how to use decorators:

@delay (duration = 2) def add (a, b): return a + b # This call will be delayed for 2 seconds add (1, 2) # This call will execute add.eager_call (1, 2) immediately

@delay (duration) is a class-based decorator. Of course, if you are very familiar with functions and closures in Python, the delay decorator above can actually be implemented using only functions. So why do we use classes to do this?

Compared to pure functions, I think that decorators implemented using classes have several advantages in specific scenarios :

When implementing stateful decorators, manipulating class attributes is more intuitive and less error-prone than manipulating variables in closures. When implementing a decorator that extends a function's interface, using a class wrapper is easier and easier to maintain than adding attributes directly to a function object Implement an object compatible with both the decorator and the context manager protocol (see unitest.mock.patch) 2. Write a flatter decorator using the wrapper module

In the process of writing decorators, have you encountered any unpleasant things? Whether you have them or not, I have anyway. When I write code, I often get particularly uncomfortable with the following two things:

When implementing a decorator with parameters, the layered function code is particularly difficult to write and difficult to read. Because of the differences between functions and class methods, the decorators written for the former often cannot be applied directly to the latter.

For example, in the following example, I implemented a decorator that generates random numbers and injects them as function parameters.

import random def provide_number (min_num, max_num): "" "Decorator: randomly generates an integer in the range [min_num, max_num] and appends it as the first position parameter of the function" "" def wrapper (func): def decorated ( * args, ** kwargs): num = random.randint (min_num, max_num) # The function return func (num, * args, ** kwargs) return decorated return wrapper @provide_number (1 after appending num as the first parameter , 100) def print_random_number (num): print (num) # print a random integer from 1-100 # OUTPUT: 72 print_random_number ()

The @provide_number decorator function looks great, but it has two problems that I mentioned earlier: deep nesting levels and inability to use it on class methods. If you use it to decorate class methods directly, the following situation will occur:

class Foo: @provide_number (1, 100) def print_random_number (self, num): print (num) # OUTPUT: <__ main __. foo> Foo (). print_random_number ()

The print_random_number method in the Foo class instance will output the class instance self instead of the random number num we expected.

The reason for this result is that the method and function of the class have slightly different working mechanisms. If you want to fix this, the provider_number decorator must intelligently skip the class instance self variable hidden in * args when modifying the position parameter of the class method, in order to correctly inject num as the first parameter.

At this time, it should be time for the wrap module to debut. The wrapt module is a library of tools specifically designed to help you write decorators. Using it, we can very easily transform the provide_number decorator, and perfectly solve the two problems of "deep nesting levels" and "uncommon",

import wrapt def provide_number (min_num, max_num): @ wrapt.decorator def wrapper (wrapped, instance, args, kwargs): # Parameter meaning: # #-wrapped: the function or class method being decorated #-instance: #-If it is Decorator is a normal class method, the value is a class instance #-If the decorator is a classmethod class method, the value is class #-If the decorator is a class / function / static method, the value is None # #-args: Positional parameters when calling (note that there is no * symbol) #-kwargs: keyword arguments when calling (note that there is no ** symbol) # num = random.randint (min_num, max_num) # No need to pay attention to wrapped is a class method or ordinary function, Append the parameter args = (num,) + args return wrapped (* args, ** kwargs) return wrapper <...> # OUTPUT: 48 Foo (). Print_random_number ()

Decorators written using the wrapper module have the following advantages over the original:

@ wrapt.decorator instance Common Mistakes 1. "Decorator" is not "Decorator Pattern"

"Design pattern" is a word that is famous in the computer world. If you are a Java programmer and you don't understand any design patterns, then I bet you will have to go through a difficult job interview.

When writing Python, we rarely talk about "design patterns." Although Python is also an object-oriented programming language, its duck-type design and excellent dynamic characteristics determine that most of the design patterns are not necessary for us. Therefore, many Python programmers may not have actually applied several design patterns after working for a long time.

The "Decorator Pattern" is an exception. Because Python's "decorator" and "decorator pattern" have exactly the same names, I have heard more than once that someone thinks of them both, thinking that using "decorators" is the practice of "decorator pattern". But in fact, they are two completely different things.

The "decorator pattern" is a programming technique based entirely on "object-oriented". It has several key components: a unified interface definition , several classes that follow the interface , and a layer-by-layer wrapper between classes . In the end they form a "decorative" effect.

In Python, there is no direct connection between "decorators" and "object-oriented", it can be just a trick that happens between functions. In fact, the "decorator" does not provide some kind of irreplaceable function, it is just a "syntactic sugar". The following code uses decorators:

@log_time @cache_result def foo (): pass

Basically identical to the following:

def foo (): pass foo = log_time (cache_result (foo))

The biggest contribution of decorators is that we can write more intuitive and easy-to-read code in certain specific scenarios. It is just a "sugar", not a complex programming model in an object-oriented domain.

Hint: There is an example on the Python website that implements the decorator pattern. You can read this example to understand it better.

2. Remember to decorate the inner function with functools.wraps ()

Here is a simple decorator specifically designed to print the time taken for a function call:

import time def timer (wrapped): "" "Decorator: record and print function time consuming" "" def decorated (* args, ** kwargs): st = time.time () ret = wrapped (* args, ** kwargs) print ("execution take: {} seconds" .format (time.time ()-st)) return ret return decorated @timer def random_sleep (): "" "Random sleep for a while" "" time.sleep ( random.random ())

Although the timer decorator is error-free, after you use it to decorate a function, the original signature of the function is destroyed. In other words, you can no longer get the name and document content of the random_sleep function correctly, all signatures will become the value of the inner function decorated:

print (random_sleep .__ name__) # prints "decorated" print (random_sleep .__ doc__) # prints None

Although this is only a minor issue, it can also lead to undetectable bugs at some point. Fortunately, the standard library functools provides a solution for it. You only need to decorate the inner decorated function with another decorator when defining the decorator.

It sounds a bit confusing, but it's just a new line of code:

def timer (wrapped): # assign the true signature of the wrapper function to decorated @ functools.wraps (wrapped) def decorated (* args, ** kwargs): # <...> return decorated has been omitted

After processing in this way, the timer decorator will not affect the function it decorates.

print (random_sleep .__ name__) # output "random_sleep" print (random_sleep .__ doc__) # output "random sleep for a while" 3. Remember to use nonlocal when modifying outer variables

Decorators are an advanced application of function objects. In the process of writing decorators, you often encounter situations where the inner function needs to modify the outer function variable. Just like this decorator:

import functools def counter (func): "" "Decorator: record and print the number of calls" "" count = 0 @ functools.wraps (func) def decorated (* args, ** kwargs): # count up + + 1 print (f "Count: {count}") return func (* args, ** kwargs) return decorated @counter def foo (): pass foo ()

In order to count the number of function calls, we need to modify the value of the count variable defined in the outer function inside the decorated function. However, the above code is problematic and the interpreter will report an error when executing it:

Traceback (most recent call last): File "counter.py", line 22, in foo () File "counter.py", line 11, in decorated count + = 1 UnboundLocalError: local variable "count" referenced before assignment

This error is caused by the scope of the counter and decorated functions nested inside each other.

When the interpreter reaches count + = 1, it doesn't know that count is a variable defined in the outer scope. It treats count as a local variable and looks in the current scope. In the end, it didn't find any definitions about the count variable, and then threw an error.

,之前的错误就可以得到解决。 In order to solve this problem, we need to tell the interpreter through the nonlocal keyword: "The count variable does not belong to the current local scope. Look for it outside." The previous error can be resolved.

def decorated (* args, ** kwargs): nonlocal count count + = 1 # <...>

Hint: If you want to learn more about the history of nonlocal keywords, check out PEP-3104

to sum up

In this article, I share with you some tips and small knowledge about decorators.

Some key points to summarize:

functools.wraps nonlocal

After reading this article, do you have any thoughts about it? Please leave a message or let me know in the project Github Issues.

Source of appendix map: Photo by Clem Onojeghuo on Unsplash More series of articles address: http://github.com/piglei/one-python-craftsman
如有转载或引用以上内容之必要,敬请将本文链接作为出处标注,谢谢合作! Kind reminder If it is necessary to reprint or quote the above content, please mark the link of this article as the source, thank you for your cooperation!

Welcome to visit this site with mobile phone scanning