222: Import within a Python package
12:02 Share

Today we're talking about importing part of a package into another part of the same package. We'll look at from.importModule and from.module.importSomething and also importPackage to access the external API from within the package. Why should we use importPackage if from.importAPI would work just fine? Well, we'll get into that in this episode.

Welcome to Test & Code. This episode is brought to you by HelloPytest, the new fastest way to learn pytest, and by the PythonTest community. Find out more at courses.pythontest.com. Although the techniques I'm going to talk about in this episode apply to lots of situations, this topic is due to a question I received about the cards application used in both the book Python Testing with PyTest, second edition, and the complete PyTest course.

So I'll talk about the imports within cards. If you want to take a look at the code, it's at github.com slash Aachen slash cards. The actual source code for the project is pretty small. The source code is in source cards, actually SRC slash cards. There's four files, db.py, api.py, cli.py, and dunderinit.py.

These are really layers. The user interface is a command line interface implemented in CLI.py. The CLI is intentionally as thin as possible, leaving most of the logic to the API layer. API.py holds both the intended API of the project

as well as much of the business logic as possible. API.py calls into db.py, so the API calls into the database layer for database concerns. db.py contains a DB class that acts as an interface to a storage service.

The layering of cards is intentional and strict. CLI uses API but doesn't directly access the database. API uses the database out of db.py but doesn't directly access the database and doesn't access the CLI code. And the DB, the layer db.py, doesn't know anything about API or CLI at all. I guess at the top is Dunder and it also.

This file holds a version string, imports the app object from CLI, and has a from.api import star. The everything that's grabbed by this star in the import statement is defined by the dunder all list defined in API.py, which isn't that much. And now that I'm looking at this, I could have structured the dunder in it.py differently, and maybe including the CLI app is not really necessary.

Not important right now, though. Okay, that's the project in question. Now let's take a look at the imports, the specific import statements. We're trying to get to a discussion of importing within a project, right? So looking at db.py, the bottom layer, the database, doesn't need to import any of the other parts of the project. api.py uses the database, so it imports it.

It imports it with the line from.db import capital DB because the capital D capital B is the class name that I'm using to encompass the database object. The .db means look for DB in the current package and in there import the DB class thing. So that's the import statement within the API.py from.db import DB.

The CLI uses the API, so it needs to import it. But it's a little different. I could have done something similar. I could have written from dot API import whatever. Or more likely, since the CLI uses everything in the API, from dot import API. But I didn't. Instead, I wrote import cards. Why did I do that? Now we get to the question that I got from a reader. It's mostly this.

I believe the explanation for this has something to do with dunderinit.py, which you helpfully label the top-level package of cards, but I don't really understand what's going on there. Why not just import API from the CLI? And also, how do the imports in dunderinit allow the CLI to access the API? When CLI imports cards, what is cards? And that is essentially the question I got from a reader of the book, which is essentially,

what is the benefit of import cards over from dot import API?

And once I've done that, what does this cards thing hold within that file? So let's answer that. The reason I chose import cards instead of from dot import API is to allow the CLI file to be used as an example for how someone could use the API. The cards package is both a command line tool that you can use as a rudimentary project management tool and also a library that can be used by some other program.

So let's say that I want to write a different UI for cards as a different Python project, maybe a curses or a textual based thing, or maybe a GUI. When I'm building this thing, I can use the CLI.py as an example of how to use the cards API. I wanted this to be true. So I intentionally wrote the CLI implementation to import the cards package as import cards, because that's why what a third party thing would do.

That's really my reasoning. I wanted the CLI.py to be used as an example of how to use the Cards API. So how does this all work? If I've installed Cards through pip install cards, then I have access to both Cards as an application, but I also have access to Cards as a package that can be imported by another application. So this new application, when it does import Cards, what's the end result? What is Cards?

In other words, the variable cards, what does that value look like? This can be easily tested with a REPL. If I start it with Python or Python-i or something for interactive, it launches the REPL.

And if I run import cards within that, I can look at what this thing is. I can just say print cards. And that does the trick and shows me what it is. And what it is, is it says module cards from blah, blah, blah, something cards, dunderingit.py. So yep, that cards thing, whatever that variable is, it holds whatever the stuff is that I put in dunderingit.py. Awesome.

So that's the inside details. And you can take a look yourself. And again, the reason why I chose import cards over from dot import API is that I really wanted to the CLI.py to be used as it be able to be used as an example. But there's also other benefits. The API has to be designed, implemented, tested and used.

If your test code covers the entire API, then you have your test code as one user of the API. That's good. If it's painful to write the tests against the API, it might be painful to use the API in real work. And finding that out as soon as possible is good. So you can change it, improve it, make it more comfortable to work with before you submit it for other people to use. At least that's one of the theories around test-driven development.

But for some reason, people sometimes don't notice a bad API even when they're writing tests for it. So now you can have a second example of using the API with the command line interface. If we write the CLI or GUI or something else that utilizes the API, it's one more client of the API.

One more chance for us to try the API out, take it for a spin, see if it's fun to use. Okay, if you're still not convinced that I did the right thing, how much work would it be to change it and just have the CLI import the API? I went ahead and tried this and it's not much work.

And since I'm confident in my test suite, I can use the test suite to see if I've got everything covered, pun intended, with the change. So to test this with minimal code changes, I can first make sure that I've installed an editable version of cards in my virtual environment using pip install-e.

essentially, or the directory. So pip install -e and then whatever you're installing, the dot for current directory from the cards project directory. And then I run pytest for sanity's sake to make sure that the test passed before any changes, of course.

And then I change the import cards in the line CLI.py to from.importAPI as cards. The reason why I did the as cards is so that I don't have to change any other code. So the from. is from the current package. And from.importAPI is import the API thing from the current package. That would import the API as a local variable named API.

Then I can go through and change all of the files to use, or everything in the file, to use API instead of cards. But that's messy, so instead I used from.importAPI as cards. That way I can leave all of the cards.whatever as is in the code. Now run pytest. I get one failure. So one failure in this. What is it?

I have a test for version and Dunder version is in the Dunder init and it's not part of the API. That's easily fixed. I can move the version from Dunder init to the API and add version to the all list. So I tried that also just to make sure that would work and it passes. Is that better? I don't actually really think it's better.

I don't really like having the version in the API file. I just have a kind of a tradition of putting that either in dunderinit.py or putting it in pyproject.toml and then having the dunderinit.py look that version up. Another reason why I don't like this is now it's not a good example. My CLI isn't a true example of using the API.

So that from dot import API as cards works, but that's not how someone else would use the cards as a package. They would just say import cards. I could put a comment in the import line saying that normally you would just say import cards instead, but why not just go ahead and do that? Why not just say import cards? And also the longer form doesn't really buy us anything.

I also don't think having the code that says API.whatever is easier to read than cards.whatever. I think actually using the cards interface is good. Anyway, that's what's going on with this import and my thinking around it and why I think that it's good to have an example of using your API within an application, especially if it's an open source application.

Thank you for listening and thank you to everyone who has supported the show through purchases of the courses, both Hello PyTest, the new fastest way to learn PyTest, and the complete PyTest course, if you'd like to really become an expert at PyTest. Both are available at courses.pythontest.com and there you can also join the Python test community. That's all for now. Now go out and test something.