Python Unittest vs Pytest

This article’s primary aim is to discuss two of the most used frameworks for unit testing in Python, unittest and pytest, their pros & cons, and when to prefer which over the other.

Python unittest vs Pytest

While writing any software, we must maintain the process of error checking throughout the development process. It ensures that once the software reaches the release stage, a minimal number of bugs are encountered during its usage.

Python also has a variety of testing frameworks that allow the testing of written code by giving it varied inputs to check its behaviour.

In case any error is encountered, it can be rectified during the development stages as opposed to hotfixes after the initial release of the application.

Example Code:

class Calculate:
    def CheckPrime(self, a):
        for i in range(a):
            if a % i:
                return False
        return True

    def CalcFact(self, a):
        if a == 1:
            return a
        else:
            return a * self.fact(a - 1)

The code shown above contains two functions named CheckPrime and CalcFact, which, as evident from their names, check for prime numbers and calculate factorials.

To ensure that the Calculate methods work smoothly, checking for errors that may arise by giving varied outputs is essential.

So, how can we do that? To ensure that our code is error-free, we can use different testing frameworks to write test cases and test our code on top of them to check the integrity of our code.

Although there are many testing frameworks, two of the most widely used ones are unittest and pytest. Let’s explore them one by one below.

Unit Test by unittest Framework

unittest is a unit testing framework included in the Python Standard Library. This framework was inspired by JUnit, a Java framework for unit testing.

Before discussing the workings of unittest, it is essential to know commonly used terms in unittest (also used in other related frameworks).

• Test Case – Smallest unit of testing – Usually consists of a single
• Test Suite – Test Cases grouped – Usually executed one after another
• Test Runner – Coordinates and handles the execution of test cases and suites

Use unittest Framework to Write Test Cases

Since Python’s standard library already contains unittest, there is no need to download any external module to start writing unit tests using unittest.

We can start after importing the unittest module. Now, let’s focus on the code we have gone through before.

Example Code:

class Calculate:
    def CheckPrime(self, a):
        for i in range(a):
            if a % i:
                return False
        return True

    def CalcFact(self, a):
        if a == 1:
            return a
        else:
            return a * self.fact(a - 1)

To write test cases using unittest, we have to follow a specific syntax, namely that the test class is a child of unittest.TestCase, and its methods must start with test_.

Consider the following code:

import unittest


class Calculate:
    def CheckPrime(self, a):
        for i in range(2, a):
            if a % i == 0:
                return False
        return True

    def CalcFact(self, a):
        if a == 1:
            return a
        else:
            return a * self.CalcFact(a - 1)


class TestCalc(unittest.TestCase):
    def test_CheckPrime(self):
        calc = Calculate()

        # Passing different outputs
        self.assertEqual(calc.CheckPrime(2), True)
        self.assertEqual(calc.CheckPrime(3), True)

        self.assertEqual(calc.CheckPrime(4), False)
        self.assertEqual(calc.CheckPrime(80), False)

    def test_CheckFact(self):
        calc = Calculate()

        # Passing different outputs
        self.assertEqual(calc.CalcFact(2), 2)
        self.assertEqual(calc.CalcFact(3), 6)

Output:

PS D:\Unittest> python -m unittest a.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK 

Judging from the output, we can see that all the test cases passed because all assertions were successful.

Now let’s try a case where the test case fails.

def test_CheckFact(self):
    calc = Calculate()

    # Passing different outputs
    self.assertEqual(calc.CalcFact(2), 2)
    self.assertEqual(calc.CalcFact(3), 6)
    # Supposed to throw an error
    self.assertEqual(calc.CalcFact(0), 0)

Output:

PS D:\Unittest> python -m unittest a.py
======================================================================
ERROR: test_CheckFact (a.TestCalc)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\Python Articles\a.py", line 34, in test_CheckFact
    self.assertEqual(calc.CalcFact(0), 0) # Supposed to throw an error
  File "D:\Python Articles\a.py", line 15, in CalcFact
    return a * self.CalcFact(a-1)
  File "D:\Python Articles\a.py", line 15, in CalcFact
    return a * self.CalcFact(a-1)
  File "D:\Python Articles\a.py", line 15, in CalcFact
    return a * self.CalcFact(a-1)
  [The previous line is repeated 974 more times]
  File "D:\Python Articles\a.py", line 12, in CalcFact
    if (a == 1):
RecursionError: maximum recursion depth exceeded in comparison

----------------------------------------------------------------------
Ran 2 tests in 0.004s
FAILED (errors=1)

As evident from the code, we execute the script using python -m unittest <name_of_script.py>.

This code works without calling the test class’ methods because the unittest module handles script files given to it in a particular format.

Since our script contained TestCalc, the child class of unittest.TestCase is automatically instantiated by the Test Runner.

After the instantiation, test methods are found inside the class and executed in order. For a method to be considered a test method, it must start with a test_.

Once the test methods are found, they are called in order; in our case, both test_CheckPrime and test_CalcFact are called. Assertions are checked in our implementation, and an error is thrown into the output in case of unexpected behaviour.

From our test case, which contained an error, one can deduce that due to how the code is written, an infinite recursion started occurring in the CalcFact method, which now can be fixed thanks to the test case.

In case you are wondering why the error is occurring, it is due to the initial condition not checking for numbers less than one.

Pros and Cons of unittest Framework

Some of the advantages of using unittest are listed below:

• Included in Python Standard Library
• Promotes related test cases into a single test suite
• Speedy test collection
• Precise test time duration

The unittest comes with the following disadvantages:

• Can be hard to understand
• No coloured output
• Can be too verbose

Unit Test by Pytest Framework

Unlike unittest, Pytest is not a built-in module; we have to download it separately. However, installing Pytest is relatively easy; to do so, we can use pip and execute the following command.

pip install pytest

Use Pytest to Write Test Cases

Let’s write some test cases using Pytest. Before starting, however, let’s look at how Pytest differs from unittest in writing test cases. For unit tests written in Pytest, we have to:

• Create a separate directory and place the scripts to be tested in the newly created directory.
• Write tests in files that either start with test_ or end with _test.py. An example would be test_calc.py or calc_test.py.

Consider the following code written for test cases using Pytest.

def test_CheckPrime():
    calc = Calculate()

    # Passing different outputs
    assert calc.CheckPrime(2) == True
    assert calc.CheckPrime(3) == True

    assert calc.CheckPrime(4) == False
    assert calc.CheckPrime(80) == False


def test_CheckFact():
    calc = Calculate()

    # Passing different outputs
    assert calc.CalcFact(2) == 2
    assert calc.CalcFact(3) == 6
    # assert calc.CalcFact(0) == 0 # Supposed to throw an error

Output:

============================================================== test session starts ==============================================================
platform win32 -- Python 3.10.7, pytest-7.1.3, pluggy-1.0.0
rootdir: D:\Unittest
collected 2 items

test_a.py
[100%]

=============================================================== 2 passed in 0.04s ===============================================================

Now, with a failed test case:

============================================================== test session starts ==============================================================
platform win32 -- Python 3.10.7, pytest-7.1.3, pluggy-1.0.0
rootdir: D:\Unittest
collected 2 items

test_a.py .F
[100%]

=================================================================== FAILURES ====================================================================
________________________________________________________________ test_CheckFact _________________________________________________________________

    def test_CheckFact():
        calc = Calculate()

        # Passing different outputs
        assert calc.CalcFact(2) == 2
        assert calc.CalcFact(3) == 6
>       assert calc.CalcFact(0) == 0 # Supposed to throw an error

test_a.py:50: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_a.py:13: in CalcFact
    return a * self.CalcFact(a-1)
test_a.py:13: in CalcFact
    return a * self.CalcFact(a-1)
test_a.py:13: in CalcFact
    return a * self.CalcFact(a-1)
.
.
.
.
.
RecursionError: maximum recursion depth exceeded in comparison

test_a.py:10: RecursionError
============================================================ short test summary info ============================================================ 
FAILED test_a.py::test_CheckFact - RecursionError: maximum recursion depth exceeded in comparison
========================================================== 1 failed, 1 passed in 2.42s ========================================================== 

The test cases written using Pytest are a bit simpler than unittest; instead of creating a class that was the child of unittest.TestCase, we can write our test functions just with test_ at the start of the method.

Pros and Cons of Pytest Framework

Following are some advantages of using the Pytest framework in Python.

• Compact test suites

• Minimal boilerplate code

• Plugin Support

• Neat and proper output presentation

  It also comes with a disadvantage, which is listed below.

• Often incompatible with other frameworks