How to Use Python Decorators to Retry Code Blocks

1. Importance of retry Decorators
2. Use @retry to Retry Code Blocks in Python
3. Use tenacity to Retry Code Blocks in Python

We can modify a function or class with a decorator to extend the function’s behaviour without permanently changing it. This article discusses how to use retry decorators to modify an existing function without making changes to the said function.

In this case, the modification retries the function several times in a given situation where its return value could be different from what we want.

Importance of retry Decorators

We can use decorators to extend the behaviour of a specific function, and we can easily create decorators to modify that function even when we don’t have access to it or don’t want to change it.

We might often need that function rather specific manner, and that’s where Python decorators come in. So let’s create a simple function to show how decorators work.

The simple function, quotient(), takes two arguments and divides the first argument by the second argument.

def quotient(a, b):
    return a / b


print(quotient(3, 7))

Output:

0.42857142857142855

However, if we want the division result always to be such that the bigger number is what’s divided (so the result will be 2.3333333333333335), we can either change the code or make use of decorators.

With decorators, we can extend the function’s behaviour without changing its code block.

def improv(func):
    def inner(a, b):
        if a < b:
            a, b = b, a
        return func(a, b)

    return inner


@improv
def quotient(a, b):
    return a / b


print(quotient(3, 7))

Output:

2.3333333333333335

The improv() function is the decorator function that takes the quotient() function as its argument and holds an inner function that takes the argument of the quotient() function and brings in the additional functionality that you need to add.

Now, with decorators, we can add a retry functionality on a particular function, especially with functions we don’t have access to.

retry decorators are helpful in scenarios where unpredictable behaviours or errors might exist, and you want to retry that same operation again when they occur.

A typical example is handling a failed request within a for loop. In such scenarios, we can use retry decorators to manage retrying that specific request a specified number of times.

Use @retry to Retry Code Blocks in Python

For retry decorators, there are different libraries that provide this function, and one of such libraries is the retrying library.

With it, you can specify wait and stop conditions from Exceptions to expected return results. To install the retrying library, we can use the pip command as follows:

pip install retrying

Now, let’s create a function that randomly creates numbers between 0 and 10 but raises a ValueError when we have a situation where the number is greater than 1.

import random


def generateRandomly():
    if random.randint(0, 10) > 1:
        raise ValueError("Number generated is greater than one")
    else:
        return "Finally Generated."


print(generateRandomly())

The code output will look as follows if the number generated at the time is greater than one.

Traceback (most recent call last):
  File "c:\Users\akinl\Documents\Python\SFTP\test.py", line 11, in <module>
    print(generateRandomly())
  File "c:\Users\akinl\Documents\Python\SFTP\test.py", line 6, in generateRandomly
    raise ValueError("Number generated is greater than one")
ValueError: Number generated is greater than one

We can’t have a situation where a ValueError is thrown into our code, so we can introduce a retry decorator to retry the generateRandomly() function until it doesn’t raise a ValueError.

import random
from retrying import retry


@retry
def generateRandomly():
    if random.randint(0, 10) > 1:
        raise ValueError("Number generated is greater than one")
    else:
        return "Finally Generated."


print(generateRandomly())

Output:

Finally Generated.

Now, the retry decorator retries the random operation till it doesn’t have a ValueError, and we only have the string Finally Generated..

We can see the number of times the code retried the generateRandomly() by introducing a print() statement within the if block.

import random
from retrying import retry


@retry
def generateRandomly():
    if random.randint(0, 10) > 1:
        print("1")
        raise ValueError("Number generated is greater than one")
    else:
        return "Finally Generated."


print(generateRandomly())

Output:

1
1
1
1
1
1
1
Finally Generated.

Here, 8, but it could be different when you run the code. However, we can’t have a situation where the code keeps retrying for a long time. So, we have arguments like stop_max_attempt_number and stop_max_delay.

import random
from retrying import retry


@retry(stop_max_attempt_number=5)
def generateRandomly():
    if random.randint(0, 10) > 1:
        print("1")
        raise ValueError("Number generated is greater than one")
    else:
        return "Finally Generated."


print(generateRandomly())

Output:

1
1
1
Finally Generated.

The function is retried only 5 times, but either it gets successful before the fifth time, returns the value Finally Generated., or it doesn’t and throws the ValueError.

1
1
1
1
1
File "C:\Python310\lib\site-packages\retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "C:\Python310\lib\site-packages\six.py", line 719, in reraise
    raise value
  File "C:\Python310\lib\site-packages\retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "c:\Users\akinl\Documents\Python\SFTP\test.py", line 9, in generateRandomly
    raise ValueError("Number generated is greater than one")
ValueError: Number generated is greater than one

Use tenacity to Retry Code Blocks in Python

The retrying library can be a little quirky and is no longer maintained, but the tenacity library provides all its features with more tools at its disposable.

To install tenacity, use the following pip command:

pip install tenacity

We can try the same code with a stop attempt at 3.

import random
from tenacity import retry, stop_after_attempt


@retry(stop=stop_after_attempt(3))
def generateRandomly():
    if random.randint(0, 10) > 1:
        print("1")
        raise ValueError("Number generated is greater than one")
    else:
        return "Finally Generated."


print(generateRandomly())

The output of the code if the random number generated is less than 1 when within the three attempt timeframe.

1
Finally Generated.

However, if it isn’t, the below output will be thrown.

1
1
1
Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\tenacity\__init__.py", line 407, in __call__
    result = fn(*args, **kwargs)
  File "c:\Users\akinl\Documents\Python\SFTP\test.py", line 9, in generateRandomly
    raise ValueError("Number generated is greater than one")
ValueError: Number generated is greater than one

The above exception was a direct cause of the following exception:

Traceback (most recent call last):
  File "c:\Users\akinl\Documents\Python\SFTP\test.py", line 14, in <module>
    print(generateRandomly())
  File "C:\Python310\lib\site-packages\tenacity\__init__.py", line 324, in wrapped_f
    return self(f, *args, **kw)
  File "C:\Python310\lib\site-packages\tenacity\__init__.py", line 404, in __call__
    do = self.iter(retry_state=retry_state)
  File "C:\Python310\lib\site-packages\tenacity\__init__.py", line 361, in iter
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x29a75442c20 state=finished raised ValueError>]