---
title: "Test-Driven Development in Python with Pytest"
description: "How to setup and use Pytest to test Python code"
author: "Bartosz Mikulski"
author_bio: "Principal AI Engineer & MLOps Architect. I bridge the gap between \"it works in a notebook\" and \"it works for 200 million users.\""
author_url: https://mikulskibartosz.name
author_linkedin: https://www.linkedin.com/in/mikulskibartosz/
author_github: https://github.com/mikulskibartosz
canonical_url: https://mikulskibartosz.name/tdd-with-pytest
---

How can you write tests using `pytest`? How does `pytest` work? The questions seem simple, but while teaching Python workshops, I saw many people confused by `pytest`.

We will start with an empty directory. All you need is `pytest` installed in your working environment.

## Why do we write tests?

> I've just realised that TDD is simply hypothesis-driven development.
>
> This is what I think my code should be doing, let's create a test to express that, followed by the code that validates that hypothesis.
>
> [Jakub Jurkiewicz](https://www.linkedin.com/posts/jakubjurkiewicz_tdd-technicalagility-modernsoftwareengineering-activity-6995884704088436736-OU6Z)

If you need convincing why you should practice Test-Driven Development, I have written several articles on TDD:

* [Why should you practice TDD?](https://www.mikulskibartosz.name/why-should-you-practice-tdd/)
* [How to learn TDD](https://www.mikulskibartosz.name/how-to-learn-tdd/)
* [How to teach your team to write automated tests?](https://www.mikulskibartosz.name/how-to-teach-testing/)
* [Testing legacy data pipelines](https://www.mikulskibartosz.name/testing-legacy-data-pipelines/)
* [4 reasons why TDD slows you down](https://www.mikulskibartosz.name/4-reasons-why-tdd-slows-you-down/)

## Directory structure and test naming conventions

We need to start with naming because `pytest` follows the "convention-over-configuration" principle. If we name our directories and tests in the right way, we can run them without configuring anything. What is the "right way" in this case?

First, we must **create a `tests` directory**. We can have as many subdirectories as we want. I suggest having a separate directory for every feature you test.

In the `tests` directory (or its subdirectories) we store the Python scripts with the tests. **`pytest` will find them if the script's name starts with `test_`**. We can put other files in the `tests` directory as long as their names have different prefixes. For example, we can have the following structure:

```
tests/
    test_some_name.py <- pytest will look for tests here
    another_script.py
    feature_A/
        test_feature_A_does_X.py <- and here
        mock_dependencies_for_A.py
```

When the directory structure is ready, `pytest` knows where to look for tests. But which tests will it run?

## What does a test look like in `pytest`?

**A `pytest` test is a Python function whose name starts with `test_`**. By default, those functions accept no parameters (I'll show you later how to use parameters in tests). Optionally, we can put those functions in a class. When we do it, the class name must begin with `Test`. Of course, all test method names must begin with `test_`.

Both of the following examples are valid (but empty) `pytest` tests:

```python
def test_do_nothing():
    pass
```

and with a test class:

```python
class TestDoingNothing:
    def test_do_nothing(self):
        pass
```

We have our first test! It doesn't test anything, but we can run it. How do we do it?

## How to run `pytest` tests

We have two options: the command line and the test runner in the IDE (PyCharm, IntelliJ, etc.).

When we want to use the command line, we should navigate to the directory with the `tests` directory. (don't open the test directory!) and type the `pytest` command. `pytest` will automatically find all of the tests and run them. When you want to run a single test, type `pytest -k name_of_the_test`.

In your IDE, it's even easier. Right-click the `tests` directory and select the `Run tests` command. If you want to run a single test, open the file containing the test and click the green triangle next to the test function. In the menu, choose the `Run test` option.

What's the difference between both options? There is no difference when you run all of the tests at once. However, when you run a single test in the IDE, the working directory changes to the location of the test! Check the working directory path twice if your test needs to load data from files &mdash; the correct path will differ depending on how you run tests.

## How to test something in `pytest`?

Generally, **a test should consist of three parts: setup, action, and assertions**. For example, if we test the append function of a Python list. The test may look like this:

```python
def test_should_append_an_element_to_an_empty_list():
    element_to_be_added = 123
    object_under_test = []

    object_under_test.append(element_to_be_added)

    assert len(object_under_test) == 1, 'The list should contain one element'
    assert object_under_test[0] == element_to_be_added
```

What does happen in the function above? First, we prepare the test data and the object we want to test (by convention, we tend to name it `object_under_test` to distinguish the tested code from other variables in the test). After that, we use the tested object. In the end, we verify the object's properties to check whether the action had the desired effect.

### When do a test pass, and when do they fail?

**A test passes when the test function doesn't raise an exception**. The test may fail for two reasons:

* AssertionError - if the condition in the `assert` line evaluates to False. If we add an optional comment to the `assert` statement (like in the first assertion in the example above), the comment will be returned as the error message.
* any other error - if the tested code raised an exception (and we don't catch it), the test fails too.

## How to read `pytest` test results?

Let's assume I put the test from the previous example in a test file called `test_example.py`. When I run the `pytest` command, I'll see the following result:

```
======================================== test session starts ========================================
platform darwin -- Python 3.10.0, pytest-7.2.0, pluggy-1.0.0
rootdir: THE_DIRECTORY_WITH_THE_PROJECT
collected 1 item

tests/test_example.py .                                                                       [100%]

========================================= 1 passed in 0.03s =========================================
```

The `rootdir` tells us where `pytest` looks for the tests. If your tests don't run, the `rootdir` is the first thing you should check. Perhaps, you run the command in the wrong directory.

After that, we see the number of tests found by `pytest`: `collected 1 item`.

Next, `pytest` lists all of the test files one by one. In this example, I have only one file. Every test run by `pytest` is denoted as a single dot next to the test file name.

In the end, `pytest` summarizes the number of passed/failed tests and the elapsed time.

Let's break the test. We add a single element to the list, but we'll check if the list contains two elements in our assertion. Of course, our new assertion makes no sense, but we want to see what happens when a test fails:

```python
def test_should_append_an_element_to_an_empty_list():
    element_to_be_added = 123
    object_under_test = []

    object_under_test.append(element_to_be_added)

    # the following line will fail
    assert len(object_under_test) == 2, 'The list should contain two elements'
    assert object_under_test[0] == element_to_be_added
```

Now, the test results look like this:

```
======================================== test session starts ========================================
platform darwin -- Python 3.10.0, pytest-7.2.0, pluggy-1.0.0
rootdir: THE_DIRECTORY_WITH_THE_PROJECT
collected 1 item

tests/test_example.py F                                                                       [100%]

============================================= FAILURES ==============================================
__________________________ test_should_append_an_element_to_an_empty_list ___________________________

    def test_should_append_an_element_to_an_empty_list():
        element_to_be_added = 123
        object_under_test = []

        object_under_test.append(element_to_be_added)

>       assert len(object_under_test) == 2, 'The list should contain two elements'
E       AssertionError: The list should contain two elements
E       assert 1 == 2
E        +  where 1 = len([123])

tests/test_example.py:7: AssertionError
====================================== short test summary info ======================================
FAILED tests/test_example.py::test_should_append_an_element_to_an_empty_list - AssertionError: The list should contain two elements
========================================= 1 failed in 0.05s =========================================
```

We see an `F` instead of a dot in the tests list. `F`s denote failing tests. Below the tests, `pytest` reports every failure separately. It  prints the test code, marks the failing line with `>`, prints the errors in lines starting with an `E`, and reports which code line failed (it's the `:7` next to the file path).

The end of the test report contains the `short test summary info` with names of the failed tests and the error message from the raised exception.

## How to write a readable test?

`pytest` will run any code we put in the test function. Nothing stops us from creating a terrible, messy test code. Ideally, the test code should be easy to read and understand. We have several best practices governing writing tests. For a start, you should know about two of them.

### Single business case per test

A single test may contain multiple `assert` statements, but **all of those `assert` statements should test the details of a single business use case**. For example, if you write code to generate an invoice, you should write separate tests for testing the tax calculations, data retrieval, tax identification number validation, etc.

When you have multiple assertions per test, consider encapsulating them in an [assert object](https://www.mikulskibartosz.name/assert-object-pattern/) to separate the business meaning from the technical implementation.

### Given/When/Then

Our test consists of three parts. I prefer explicitly separating them by putting the Given/When/Then headers as code comments. For example, our test could look like this:

```python
def test_should_append_an_element_to_an_empty_list():
    # Given
    element_to_be_added = 123
    object_under_test = []

    # When
    object_under_test.append(element_to_be_added)

    # Then
    assert len(object_under_test) == 1, 'The list should contain one element'
    assert object_under_test[0] == element_to_be_added
```

It's a controversial practice. Some people loathe adding such comments because they seem redundant. After all, you are supposed to have three blocks of code in a test. Do you need to name them when you always have the same three-block structure? Others enjoy alliterations and name their blocks Arrange/Act/Assert.

Whatever you choose, remember to **have three separate parts of the test code. Don't mix them.**

## How to setup test dependencies in `pytest`

What if multiple tests require the same data preparation code? Should we copy/paste the `Given` block between test functions? Do we extract the `Given` block to a separate function and call the setup function in every test?

In `pytest`, we have yet another option, which seems to be better than copying/pasting code or calling a setup function in every test. We can use `pytest` fixtures.

### What is a fixture?

**A fixture is a function we implement to prepare the test data.** `pytest` runs them automatically and caches the values returned by the function. If we use the same fixture in multiple tests, `pytest` reuses the returned values without calling the setup function every time.

Assume we load a Pandas DataFrame in the given section of our test:

```python
import pandas as pd

def test_something_here():
    # Given
    test_data = pd.read_csv('input_data.csv')
```

Instead of opening the file in every test (which takes time), we can read it once and pass the DataFrame to every test function where we need the same input data:

```python
import pandas as pd
import pytest

class TestClass:
    @pytest.fixture(scope="class")
    def input_data(self):
        return pd.read_csv('input_data.csv')

    def test_something_here(self, input_data):
        # we don't need to load the input_data anymore
```

In the test function, we have added a parameter `input_data`. When we run the test, `pytest` will look for a fixture function with the same name and pass its returned value to the test function.

We can also use fixtures when we don't care about the returned value. For example, when our setup function creates a file. We can also pass another fixture to a fixture. In the example below, we use a fixture to create the test directory. Then another fixture writes the input data in the directory. Ultimately, we have a test with the fixture name as a parameter. The variable contains `None`. We use it only to instruct `pytest` what setup functions to run:

```python
import os
import pandas as pd

class TestLoadCsv:
    # this creates the `data/csv/` directory if it doesn't exist
    @pytest.fixture(scope='class')
    def test_directory(self):
        os.makedirs('data/csv/', exist_ok=True)

    # it writes a test file in the `data/csv` directory
    @pytest.fixture(scope="class")
    def csv_without_header(self, test_directory):
        df = pd.DataFrame(
            {
                "col1": [1, 2, 3],
                "col2": [4, 5, 6],
                "col3": [7, 8, 9],
            }
        )
        df.to_csv('data/csv/without_headers.csv', index=False, header=False)

    # it needs the test file to exist, hence the fixture parameter
    # note that the fixtures don't return anything
    def test_load_csv_without_headers(self, csv_without_header):
        # here is the test
```

### What are the fixture scopes?

**The `scope` parameter tells `pytest` how to reuse the value returned by the fixture.** By default, the fixture function is called before every test function. However, we can change the scope if we want to reuse the returned value (or a side-effect created by the fixture). In my example, I want to run the setup code only once for every test in the test class.

You can find all supported `scope` values in the [`pytest` documentation](https://docs.pytest.org/en/6.2.x/fixture.html#scope-sharing-fixtures-across-classes-modules-packages-or-session).

### How to run a cleanup code after a test?

What if we wanted to remove the test file after the tests?

We need a slightly different fixture code if we want to run a cleanup code after the tests (when `pytest` exits the fixture scope). Instead of returning a value, we will use the `yield` keyword. **Everything we put before the `yield` runs before the fixture scope. Everything after the `yield` runs after all of the tests in the fixture scope.**

Suppose I wrote a fixture to start a Selenium session before every test. I need to stop the browser at the end of the test. We can use a `function`-scoped fixture to do so:

```python
from selenium import webdriver
import pytest

@pytest.fixture(scope='function')
def driver():
    driver = webdriver.Safari()
    yield driver
    driver.quit()

def test_using_selenium(driver):
    pass
```

The fixture will setup the Selenium driver and pass the instance to the test. The test will run with the driver available in the parameter. After the test finishes, the fixture runs the `quit` function.

## Reusing test code with different input data

Sometimes, instead of reusing the setup code, we want to reuse the `When`/`Then` blocks with different data. We don't need to copy/paste test code! because`pytest` supports parameterized tests. We can run the same test code with different input data values.

If we wanted to verify whether a Python list can accept values of multiple types, we could write our first example like this:

```python
import pytest

@pytest.mark.parametrize(
    "element_to_be_added,number_of_elements",
    [(123, 1), ("123", 1), ("this is my test", 1)]
)
def test_should_append_an_element_to_an_empty_list(element_to_be_added, number_of_elements):
    object_under_test = []

    object_under_test.append(element_to_be_added)

    assert len(object_under_test) == number_of_elements
    assert object_under_test[0] == element_to_be_added
```

In the test function, we have two parameters. `pytest` will pass them there automatically from the `@pytest.mark.parametrize` fixture.

In the fixture, we have to define the parameter names &mdash; the fixture's first parameter. Note that it's a single string with names separated by commas (without spaces!).

We have a list of parameter values in the fixture's second parameter. Each of those tuples replaces the  `test_should_append_an_element_to_an_empty_list` parameters listed in the first argument of the fixture.

In the test report, we see three test runs (but we had to implement only one test):

```
======================================== test session starts ========================================
platform darwin -- Python 3.10.0, pytest-7.2.0, pluggy-1.0.0
rootdir: /Users/myszon/Projects/blog_posts/pytest
collected 3 items

tests/test_example.py ...                                                                     [100%]

========================================= 3 passed in 0.03s =========================================
```

If the test fails, we get a detailed error log telling us which variant of the test failed. Note the square brackets after the test name. Those brackets contain the parameter values separated by `-`.

```
====================================== short test summary info ======================================
FAILED tests/test_example.py::test_should_append_an_element_to_an_empty_list[123-10] - assert 1 == (1 + 1)
FAILED tests/test_example.py::test_should_append_an_element_to_an_empty_list[123-11] - AssertionError: assert 1 == (1 + 1)
FAILED tests/test_example.py::test_should_append_an_element_to_an_empty_list[this is my test-1] - AssertionError: assert 1 == (1 + 1)
========================================= 3 failed in 0.07s =========================================
```