Decorators are closely tied to concepts such as closures, first-class functions, and more. So by understanding these, it’ll make it intuitive not only to write elegant and reusable solutions but also to make it easy to debug things.

If you do already understand what is a first-class function and what is a closure then please feel free to jump to the decorators section of this article.

First-class functions (FCF)

FCF or first-class functions are functions that we can pass around and manipulate just like any other variable. This means a FCF:

• Can be used as parameters
• Can be used as a return value
• Can be assigned to variables
• Can be stored in data structures such as hash tables, lists, …

Note: All functions in python are first-class functions!

Example:

 """A simple function to greet."""
def say_hello():
    print("Hello!")

greet = say_hello
greet()  # Output: Hello!

If we print greet function but without executing it, it’ll show us something like

>>> print(greet)
<function say_hello at 0x7fdb228b7ce0>

greet refers to the function itself or a reference to the function object. It does not execute the function directly, but rather allows us to invoke or call the function when followed by parentheses ().

Hold on! What is a function object ?

Well …

Objects

An object is a fundamental building block of an object-oriented language. Integers, strings, floating point numbers, even arrays and dictionaries, are all objects 🤯.

In a little more technical term, “everything” is inheriting from the object class also called type

>>> type(object)
<class 'type'>

And as you know inheritance is the process in which you inherit or derive functionality from the “mother” class (and also extends it). Read more …

>>> dir(object)
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

This will list all the methods and attributes available for the class object and so as you guessed!

• For int:

>>> dir(int)
['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__getstate__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'as_integer_ratio', 'bit_count', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

Will also list some or all of the methods & attributes from the object class along with other methods and attributes that define what is an integer in Python. Read more about the python data model.

Note:

Important ❗:

• Attributes? Methods? Functions, Variables … What’s going on here 🤔?

I’m glad you asked! In Python, any function and variable stored in an instance or a class is referred to as a method or an attribute, respectively. I mentioned “instance” and “class” because they have slight differences, but for now, we can use them interchangeably.

Technical 🐍:

• The double leading and trailing underscore means that it’s a reserved Python methods (also called dunder methods).
• The ability of an object to know about its own attributes at runtime, is called Introspection.

>>> int.__name__
'int'

Now what is a function object ? It’s also “something” that inherits from the object class and extends it to incorporate the characteristics of a function. In this case being a function means being a callable entity (has a special __call__ method).

The concept may be overwhelming initially, but remember that objects consist of methods and attributes. First-class functions are highly versatile, allowing you to easily use and pass them around within your code.

Closures

Nested functions

To understand closures let’s first take this nested function:

def outer_function():
    message="Hello" 
    def inner_function():
        print(message)
    return inner_function # no '()' --> no execution

We are simply returning the inner_funciton and NOT executing it!

>>> outer_function()
<function outer_function.<locals>.inner_function at 0x7f11b067bd80>

The notation <locals> is used to indicate that inner_function is a local function, meaning it is defined within another function (outer_function in this case).

>>> func = outer_function()
>>> func() # this executes the inner_function
'Hello'

The key note here is that the inner_function can access and use the message variable even after the outer_function has finished executing. This behavior is called a closure.

So in short closures in Python allow inner functions to access and “remember” variables from their outer functions.

Once you understand this, decorators will be a piece of cake 😋.

We can also pass arguments! Which are also “Remembered” by the inner_function

def outer_function(x):
    def inner_function(y):
        print(x + y)
    return inner_function 

my_func = outer_function(1) # Executes outer_function and returns inner_function with x already defined
my_func(2) # inner_function know that x = 1 so it prints the the sum
3

Note: You can use this to pass the arguments instead: outer_function(1)(2)

Click to see more

The ability to add an argument to the function each at a time is called currying

def multiply(a):
    def multiply_inner(b):
        def multiply_innermost(c):
            return a * b * c
        return multiply_innermost
    return multiply_inner

# We add args in a partial way
multi_1 = multiply(1)
multi_2 = multi_1(2)
multi_3 = multi_2(3)

>>> print(multi_3)
6
>>> print(multiply(1)(2)(3))
6

We probably won’t use closures by passing in variables … but you know by now that we have first-class functions and instead of x or y we can pass in a function and do little more.

def outer_function(func): # we can name func and x, y whatever we want!
    def inner_function(x, y):
        return func(x, y)
    return inner_function

def add_numbers(x, y):
    return x + y

my_func = outer_function(add_numbers)
result = my_func(1, 2)

>>> print(result)
3

Question: Alright cool! I understand closures but WHY do we need those nested functions anyways?

• Answer: Well this is where decorators comes in it’ll help us modify and alter the behaviours of functions! For example:

def outer_function(func):
    def inner_function(x, y):
        print("Doing something before the function ...")
        result = func(x, y)
        print("Doing something after the function ...")
        return result
    return inner_function

If you want to calculate the time the function took to execute or log something, etc., it’s so easy to do it this way! We’ll explore this more details in the decorators section!

*args and **kwargs

What if the function we want to pass in has many arguments ? Should we hard-code them all ? No!

def outer_function(func):
    def inner_function(*args): # * matches all the function positional arguments!
        return func(*args)
    return inner_function
    
def add_numbers(*args):
    return sum(args)
    
result = outer_function(add_numbers)(1, 2, 3, 4, 6, 7)
>>> print(result)
23

Sometimes We want to have keywords arguments! For example:

"""
a, b, c are positional arguments
d, e, f are keyword arguments
"""
def func(a, b, c, d="string", e=1.5, f=None):
    print(a, b, c, d, e, f)
    
func(1, 2, 4)

Output: # 1 2 4 string 1.5 None  

Or we can also use the ** Python notation to unpack a dictionary!

my_dict =  {'d': 'string', 'e': 1.5, f: None}

func(a, b, c, **my_dict) # equivalent to func(a, b, c, d="string", e=1.5, f=None)

Output: # 1 2 4 string 1.5 None  

Let’s see another example that uses closures

def outer_function(func):
    def inner_function(*args, **kwargs):
        print(f"The keywords arguments are: {kwargs}")
        print(f"The positional arguments are {args}")
        return func(*args, **kwargs)
    return inner_function

def add_numbers(*args, **kwargs):
    return sum(args) + sum(kwargs.values())

result = outer_function(add_numbers)(1, 2, 3, 4, 6, 7, bananas=12, orange=1.5, apple=4.5)

>>> print(f"The result is {result}")
The keywords args are: {'bananas': 12, 'organge': 1.5, 'apple': 4.5}
The positional arguments are (1, 2, 3, 4, 6, 7)
The result is 41.0

Why is this useful ? Well *args collect all the positional arguments and **kwargs will also collect all the keyword arguments without having to hard-code any of them!

The use of args and kwargs as variable names is a convention commonly seen in programming. However, the real magic lies in the * and ** symbols that precede them, which have special meanings in Python. The * is used to unpack a list or tuple, while the ** is used to unpack a dictionary.

Decorators

Function decorators

Now decorators are easy to understand It’s just a matter of naming conventions 😀!

Naming convention: We’ll change outer_function to a meaningful decorator name. And inner_function to wrapper just because it happens to wrap the original function 🤷

➡️ The only goal of decorators is to dynamically add or modify any object! A common way to use them would be to modify function behaviours.

Let’s add a little functionality to our previous decorator!

import time

def calculate_execution_time(func):
    def wrapper(*args):
        start_time = time.time()
        result =  func(*args)
        end_time = time.time()
        print(f"The function finished executing in {end_time - start_time} seconds")
        return result
    return wrapper

def heavy_computation(x, y):
    """
    This function computes the sum from 0 to x^y.
    """
    result = 0
    for i in range(x ** y):
        result += i
    return result

decorator_func = calculate_execution_time(heavy_computation)

>>> decorator_func(12, 7)
The function finished executing in 1.2244946956634521 seconds

Now simply instead of writing:

decorator_func = calculate_execution_time(heavy_computation) we can use the Python @ notation for it!

@calculate_execution_time
def heavy_computation(x, y):
    result = 0
    for i in range(x ** y):
        result += i
    return result
heavy_computation(12, 7)

The function finished executing in 1.2223949432373047 seconds

We can even stack many decorators!

@opitmize
@cache
def function()
    pass

It’s simply equivalent to optimize(cache(function)). The Order of the decorators matter!

• Okay now that we understood the notion! Let’s use a more real example!

def admin_required(f):
    @wraps(f)
    def decorated_function(*args, **kwargs):
        if not current_user.is_admin():
            abort(403) # returns an HTTP 403 code
        return f(*args, **kwargs)
    return decorated_function

This is a code snippet taken from a flask application that uses the Flask-admin extension.

The admin_required decorator protects decorated routes by returning a 403 HTTP response if the user lacks admin privileges.

from flask import Flask

app = Flask(__name__)

@app.route("/admin")
@admin_required
def admin():
    return "Hello admin :=) !"

if __name__ == "__main__":
    app.run()

Note: the route method from flask is also a decorator! That for instance takes an argument (We’ll see how to do that too!).

There is still one thing however that is maybe confusing to you! What is @wraps and why are we using it inside of our decorator 😦 ?

• Each object (a function in this case) has some kind of Metadata for example the function name, the module it’s part of etc.

So when you run:

>>> help(heavy_computation)

Help on function heavy_computation in module __main__:

heavy_computation(x)
    This function computes the sum from 0 to x^y.

Python displays the content of the __doc__ attribute of the specified object (function, module, class, etc.) in the console.

Let’s reuse the previous function!

def heavy_computation(x):
    """
    This function computes the sum from 0 to x^y.
    """
    result = 0
    for i in range(x**7):
        result += i
    return result

>>> heavy_computation.__name__
'heavy_computation'

Or

>>> heavy_computation.__doc__

This function computes the sum from 0 to x^y.

But after passing this function into our decorator it loses its identity 😮!

import time

def calculate_execution_time(func):
    def wrapper(*args):
        start_time = time.time()
        result =  func(*args)
        end_time = time.time()
        print(f"The function finished executing in {end_time - start_time}")
        return result
    return wrapper
    
@calculate_execution_time
def heavy_computation(x):
    """
    This function computes the sum from 0 to x^y.
    """
    print(heavy_computation.__doc__)
    result = 0
    for i in range(x**7):
        result += i
    return result

>>> heavy_computation.__name__
'wrapper'
>>> heavy_computation.__doc__
None

The name of our function heavy_computation changes to wrapper and we also lost our docstring! This actually makes sens! because the heavy_computation function changed to calculate_execution_time(heavy_computation)! Again @ is just a syntactic shortcut…

>>> calculate_execution_time()
<function calculate_execution_time.<locals>.wrapper at 0x7f11b067bd80>

• But this is probably not what we want!

➡️ This is where the wraps decorator comes in handy.

from functools import wraps
import time

def calculate_execution_time(func):
    @wraps(func)
    def wrapper(*args):
        start_time = time.time()
        result =  func(*args)
        end_time = time.time()
        print(f"The function finished executing in {end_time - start_time}")
        return result
    return wrapper

>>> my_func.__name__
'heavy_computation'
>>> my_func.__doc__
This function computes the sum from 0 to x^y.

Much better now 🙂! The function is still itself after decoration.

Boilertemplate Function decorators

As a summary here is the boilerplate that is used to create decorators!

from functools import wraps

def decorator_function(original_function):
    @wraps(original_function)
    def wrapper_function(*args, **kwargs):
        # Do stuff
        return original_function(*args, **kwargs)
    return wrapper_function

And for decorator that except arguments, like the flask route one @app.route("/home")

def argument_decorator(arg1, arg2)
    def decorator_function(original_function):
        @wraps(original_function)
        def wrapper_function(*args, **kwargs):
            # Do stuff
            # Do more stuff with arg1 and arg2
            return original_function(*args, **kwargs)
        return wrapper_function
    return decorator_function

functools.partial(fn, /, *args, **kwargs)

from functools import partial, wraps
def argument_decorator(original_function=None, arg1=None, arg2=None):
    if not func:
       return partial(argument_decorator, arg1=arg1, arg2=arg2)
    @wraps(original_function)
    def wrapper_function(*args, **kwargs):
        # Do stuff
        return original_function(*args, **kwargs)
    return partial(wrapper_function, arg1=arg1, arg2=arg2)

@argument_decorator('value1', 'value2')
def my_function():
    # Function body
    pass
#Or
@argument_decorator
def my_function():
    # Function body
    pass

Class Decorators

In Python 2.6 and higher class decorators were introduces! Now we can write decorators with classes 😀!

Before we jump to the syntax let’s take a moment to see WHY class decorators ? Because honestly if you can do the same with function decorators why the complexity ?

Here are some points where class Decorators shines more:

• It’s easy to keep the state with classes.
• Use inheritance to implement similar but different decorators.
• Add methods and properties to the decorated callable object, or implement operations on them.

After you know how each one works under the hood you’ll make the best choice between function and class decorators.

Let’s implement our calculate_execution_time function decorator to be a class decorator.

class CalculateExecutionTime:
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        start_time = time.time()
        result = self.func(*args, **kwargs)
        end_time = time.time()
        print(f"The function finished executing in {end_time - start_time} seconds.")
        return result

@CalculateExecutionTime
def heavy_computation(x, y):
    """
    This function computes the sum from 0 to x^y.
    """
    result = 0
    for i in range(x ** y):
        result += i
    return result

Let’s call the function!

>>> heavy_computation(12, 7)
The function finished executing in 1.2205712795257568 seconds.

All good! But remember @CalculateExecutionTime is just a shortcut for CalculateExecutionTime(heavy_computation) Which for instance is not exactly our heavy_computation function:

>>> print(heavy_computation)
<__main__.CalculateExecutionTime object at 0x7f7de7c6b450>

Python won’t store the metadata of the function you decorate for you 🙁.

We have to explicitly do that ourselves. For classes we’ll use functools.update_wrapper() function.

# from https://github.com/python/cpython/blob/main/Lib/functools.py
def wraps(wrapped,
          assigned = WRAPPER_ASSIGNMENTS,
          updated = WRAPPER_UPDATES):
    return partial(update_wrapper, wrapped=wrapped,
                   assigned=assigned, updated=updated)

from functools import update_wrapper
import time

class CalculateExecutionTime:
    def __init__(self, func):
        update_wrapper(self, func) 
        self.func = func

    def __call__(self, *args, **kwargs):
        start_time = time.time()
        result = self.func(*args, **kwargs)
        end_time = time.time()
        print(f"The function finished executing in {end_time - start_time} seconds.")
        return result

Boilertemplate Class Decorators

class DecoratorFunction:
    def __init__(self, func):
        update_wrapper(self, func) 
        self.func = func
        # Add stuff

    def __call__(self, *args, **kwargs):
        # Do stuff
        return self.func(*args, **kwargs)

Or a decorator that excepts arguments:

class DecoratorFunction:
    def __init__(self, *decorator_args, **decorator_kwargs):
        update_wrapper(self, func) 
        self.decorator_args = decorator_args
        self.decorator_kwargs = decorator_kwargs
    def __call__(self, func):
        # Do stuff
        return self.func(*args, **kwargs)

Those boilertemplate are only meant as a starting point. So feel free to customize them as needed.

Built-in decorators

There are many built-in decorators in python which are super helpful and are in my opinion a must know.

Getter and Setter methods

Getter and setter are a quite popular methods in object-oriented programming.

• Getter methods as the name apply are used to get or access the attributes from a class.
• Setter methods allow you to set or mutate the value of an attribute in a class.

without a decorator

Let’s do it first without a decorator so we can better see the difference:

class Person:
    def __init__(self, name):
        self.name = name
    
    def get_name(self):
        return self.name
    
    def set_name(self, name):
        self.name = name

To get the name:

person = Person("John")
>>> person.get_name()
'John'
>>> person.set_name("Harry")
>>> person.get_name()
'Harry'

with a decorator

The pythonic way to do it would be by using the property and setter decorators: when you decorate a method with property decorator, Python executes it for you:

class Person:
    def __init__(self):
        self.name = name
        
    @property
    def name(self):
        return self._name

    @name.setter
    def name(self, name):
        if not name:
            raise ValueError("Oooh the person must have a name :/")
        self._name = name

Let’s start by explaining the syntax and the weird _name instead of simply using name the name of decorated method should have the same name as the stored attribute.

• If you only want to access attributes, you don’t need to pass them. But you still have to have access to the object itself that’s why you keep passing the self keyword.

Similarly to set the attribute you have to have the same method name for it!

@attribute.setter or in this case @name.setter.

Note : will still pass the name attribute in order to store it!

Now why are we using _name instead of name ? Well let’s try that !

class Person:
    def __init__(self):
        self.name = name
        
    @property
    def name(self):
        return self.name # changed self._name to self.name

    @name.setter
    def name(self, name):
        if not name:
            raise ValueError("Oooh the person must have a name :/")
        self.name = name # changed self._name to self.name

        
person = Person("John")

If you try to execute the above code you’ll get the following error:

Traceback (most recent call last):
  File "tmp/tutos/test.py", line 14, in <module>
    person = Person("John")
             ^^^^^^^^^^^^^^
    ...
    
  File "tmp/tutos/test.py", line 12, in name
    self.name = name
    ^^^^^^^^^
  [Previous line repeated 994 more times]
RecursionError: maximum recursion depth exceeded

Recursion Error ? What! But we just created a person instance 😮

Python executes the decorated setter and getter methods whenever you want to access & assign an attribute even for the __init__ method.

When the name "John" is passed to the __init__ method, the assignment statement within the method triggers the name.setter method. However, at this point, the name.setter method does not have access to the attribute self.name because it was never stored or initialized.

➡️ we store a different attribute that has by convention a leading underscore. It also means that it’s a non public attribute.

• There are still many other built-in decorators they serve different purposes but we won’t go through them all. So feel free to check out the following links to read more about them.

• @classmethod

• @staticmethod

Hope you enjoyed the post 😊!

Have a question or spotted a mistake? Please leave Comment below!