221: How to get pytest to import your code under test
Test & Code
Deep Dive
What are the two main options discussed for enabling pytest to import code under test?
The two main options are: 1) Installing the code under test as a pip installable package using `pip install -e /path/to/local/package`, and 2) Using the `pythonpath` pytest setting to specify the location of the code under test.
Why might creating an installable package be beneficial even if you don't plan to distribute it?
Creating an installable package simplifies the process of importing and testing code, as it allows the code to be installed into a virtual environment using pip. This approach is increasingly easier to implement and can streamline testing workflows, even for local projects not intended for distribution.
How does the `pythonpath` setting in pytest help with importing code under test?
The `pythonpath` setting in pytest allows you to specify the directory where the code under test is located. By adding a line like `pythonpath = src` in the `pytest.ini` file, pytest can locate and import the code from the specified directory, even when tests are run from subdirectories.
What is the advantage of using the `pythonpath` setting when running tests from subdirectories?
The `pythonpath` setting ensures that pytest can still locate the source code and test files, even if you run tests from deep within subdirectories. It maintains the relative path to the top-level directory, making it easier to organize and run tests across complex project structures.
What additional functionality does the `pythonpath` setting provide beyond importing code under test?
The `pythonpath` setting can also be used to import test helper functions or utility modules. By specifying the path to a directory containing these helpers, you can reuse them across multiple test files, improving code organization and reducing redundancy.
What will the next episode focus on regarding imports in Python projects?
The next episode will focus on importing different parts of a project into other parts of the project, particularly addressing how the command-line interface can access the API of a project. This will include discussing non-standard import approaches and their implications.
- Code under test can be a single file, module, directory package, or installable package.
- Tests and source code are typically in different directories.
- The challenge is enabling tests to import code from a different directory.
Shownotes Transcript
In this episode and the next, I'm going to talk about imports a bit. In this episode, we're talking about importing our code under test. In the next episode, we're going to talk about importing the API of a project from within the project. This is partly for testing purposes and partly for example purposes. But we'll get into that in the next episode. In this episode, setting up everything right so that our tests can import our code under test. Now let's get to it.
Welcome to Test and Code. This episode is brought to you by HelloPyTest, the new fastest way to learn PyTest, and by the Python Test community. Find out more at courses.pythontest.com. So we've got some code we want to test and some tests. The tests need to be able to import the code under test, or at least the API of the code under test, in order to test it. Let's be more specific.
Let's say our code under test is either a single Python file or a module with functions in it that need testing, or a directory full of Python files and modules with a dunderinit.py file. This is sometimes called a directory package or just a package, but it's a little confusing because we also have pip installable packages. That's something else we could be testing.
And if we're installing an installable package, I'm going to assume that it's a package that's local to our machine. It's not something that's up on PyPI. So that's our code we want to test. It's either a single file module or a directory package or an installable package.
Now our test code. Let's assume our test code and our source code are in different directories. So for the sake of this discussion, we've got a top-level project directory, and then we've got a source subdirectory, usually spelled SRC in my case, and a tests directory.
And what's one of our test files look like? In our test file, it's just going to be test underscore something dot pi. And inside of it, it'll have something like import project or import project name or import project dot module or from project dot module import func name. You get it.
Somehow I've got the thing that I'm testing or the part of the project I'm testing within this file. I've got to import that function or the module or something so that I can access it from the test code. But the test code is in a different directory from the source code. So how does that import work? Well, really, we have to do one of two things. We either have to take the code under test and install it.
using pip. Or we need to tell pytest where to find the code under test. So again, our code under test is either a single file or a directory package or an installable package. If it's an installable package, that's the
using pip to install our code under test. So we just say pip install dot local dirname, and that will install into our virtual environment. Hopefully we're using virtual environments. So what are the other cases? The other cases are the tricky part. So if it's a single file module or a directory package, or even just a directory of stuff, that's the hard part. And that's probably why you're listening to this episode.
However, let's not dismiss the packaging thing right away. Packaging has gotten so much easier in the last few years that even if you don't intend to distribute your project via a package index like pip or a local repository or your work repository, let's take...
Taking the extra few steps to create an installable package from your directory package might be pretty easy, and it's something you might want to look into. But let's say you didn't. Let's say you just want to have this directory, and normally you're going to access it. You know how you're accessing it, but the tests don't see it that way. So we need to tell PyTest where to find this code.
Luckily, there's a setting specifically for this. It's called Python Path. And I'll put a link in the show notes to the documentation around this. And the gist of this is in your settings file, like say your pytest.ini or your tox.ini or pyproject.toml, let's say for the sake of this argument, let's say your pytest.ini file, that's going to be in your top level directory. So your top project directory, the one at the same level as your source and your tests.
In that file, you're going to have a line that says PythonPath equal SRC or source or source slash subter or however I get to my code relative to that top level. That's the setting that you want to put in PythonPath. And that's really all we have to do to tell PyTest where to find that import and make that import work.
What's cool about this is even when we're testing, if you test from that directory, if you're in PyTest from the top level, PyTest will look in the test directory, actually look in all directories to find all the tests. At your top level, it'll find the tests and it'll find the code under test because of our Python path setting. Even if we've CD'd into our change directories into one of the sub-directory of the test directory,
like maybe even like several layers deep, PyTest will still find that PyTest any file. It'll find that Python path setting and set that relative to that top level thing. And it'll still find the source code. So that's really all you need to do to get your test code to find your source code. Another thing you can do with the Python path setting is you can set up a path to your helper functions. So let's say your tests have some
some utility test helper functions that you've defined. You can stick those in, say, tests slash helper or really lib or really anything and reference that from your Python path setting. And now you can use test helper functions. So what are we going to talk about in the next episode? In the next episode, we're going to look at specifically different parts of your project and how to import things
those different parts of your project into other parts of your project. Specifically, I'm going to address a question where I have taken a possibly non-standard way for some of my, for the command line interface to access the API for a project.
Thank you for listening and thank you to everyone who has supported the show through purchases of the courses, both Hello PyTest, the new fastest way to learn PyTest, and the complete PyTest course, if you'd like to really become an expert at PyTest. Both are available at courses.pythontest.com and there you can also join the Python test community. That's all for now. Now go out and test something.