pytest plugins - a full season
11:52 Share

I started using PyTest primarily for the fixture system. PyTest fixtures are kind of like setup and teardown in other test systems, but way more modular and flexible and usable across a test suite, and you can share them with other projects through the plugin system. Welcome to Test & Code. This episode is brought to you by HelloPytest, the new fastest way to learn PyTest, and by the Python Test community. Find out more at courses.pythontest.com.

The PyTest plugin system allows plugins to share fixtures, but also allows them to share hook functions. So what's a hook function? A hook function is a way to modify part of the PyTest behavior. A couple of common examples are, for instance, adding a command line flag to be able to pass in information to the test run. You do that with a hook function.

Another example would be to deselect or skip some tests based on some runtime factor. You would do that in a hook function as well. This plugin system that allows sharing of fixtures and hook functions is so powerful and popular that there are at least 1,487 itest plugins on PyPI.

Where did I get that number? If you go to the pytest.org, there is a site, there is a pytest plugin list that lists 1,487 plugins. This is done by searching PyPI for pytest. Actually, projects that begin with pytest- or pytest- since that is a common convention. If you go directly to PyPI, you can just search for pytest, and it comes up with

10,000 plus projects. That's a little overwhelming. So while I appreciate the restriction down to 1,487, that's still kind of a lot. So what should we do? An alternative list is on pythontest.com. There's a list I put together called Top PyTest Plugins, and that is based on the counts for the number of downloads over the past 30 days-ish, and

The list in there has currently 133 plugins. Although popularity isn't everything, I was curious about the most downloaded Bytes plugins, and that's why I put this list together.

I actually used another data set from a website from Hugo VK called Top PyPI Packages. And this is regularly updated about once a month to dump the 8,000 most downloaded packages from PyPI. And I grabbed all 8,000 and searched for pytest.py.

in that list and I do some exclusions. So there are some packages that are not supported anymore so there's no point in looking at them. And there's also PyTest itself which isn't a plugin so those are removed from the list. And that's why right now there's 133 items but that could change over time. So this list and PyTest plugins in general is what we're going to focus on for this season.

We're not going to go through all 133, but we're going to go through quite a few. And we're also not going to restrict ourselves to just this. There are other plugins that are super cool that might not be on this list. I'd like you to share your favorite plugins, whether on this list or not, that you would like to, or plugins that you'd like to have me take a look at, and we can talk about it on the show. If you're a plugin fan,

author and you really think that your plugin should you'd want to talk about either being an author of pytest plugins or this a particular plugin or two or three let me know i'd be happy to try to get some people on the show

So I've got this list of the top plugins from 1 to 133, and it's not going to be ordered by top to bottom, but I might start with an ordering and then jump around a bit, depending on popularity of myself. The things that I use, I can talk about easily. Some things I might have to do a little bit of research with. And...

Some, I just am not interested. Also, I really don't want to do 133 episodes in this season, but we'll see. Who knows? Might be longer, but probably not. For instance, let's take a look at the top.

The number one downloaded plugin is PyTestCov. And that is a PyTest plugin for measuring coverage. It actually uses Coverage.py and it provides some conveniences. And I won't get too much more into it because I want to talk about that in the next episode. So we'll use PyTestCov in the next episode. But there's a whole bunch of great ones in here. Let's take a look down the list.

Currently number four is PyTest AsyncIO. That sounds cool. I don't write a lot of async code, but that'd be one that I'd love to talk to somebody about, either the plugin author or anybody that uses it on a regular basis and wants to talk to me. That'd be great. PyTest Metadata I use all the time, and that's along with PyTest HTML, but Metadata is usable by itself. And those are great. Those are number five and six. PyTest Timeout is fun.

skim down a bit. PyTest ordering, there's various ones for ordering that I'd like to kind of like talk about those together that might make sense. There is PyTest repeat for repeating tests. That's one that I've, I have helped with in maintenance before. PyTest check is my plugin. That's number 24. That's one of mine. It allows multiple features, multiple failures per test case.

PyTest Playwright. This is on my list of things I want to try because I'm working on a web project. So I do definitely want to try PyTest Playwright, but it'd be fun to talk to somebody else about that. What else? Let's go down. What's near the bottom? There is PyTest MD Report at 133. Now that's not the least downloaded one. That's just the least downloaded in my list. But

It still gets, looks like, 112,000 downloads per month. So PyTest plugins are downloaded a lot. So this one is a PyTest plugin to generate list test outcome reports with markdown table format. Cool. So reporting the outcomes in a markdown format. This would be cool if you're piping the output to somewhere where you can read it better or...

That can, yeah, maybe to a website or something. That's cool. I'll definitely check this one out. And one of the things that if I, so I'm looking at this list and what I've done with this list is it's got the number, the order number, the name of the plugin, plus a link to the plugin in the name and the number of downloads. And, and the, so I'm getting that all right from the top PyPI packages.

And then I also wanted to have like the description. So this one says a PyTest plugin to generate test outcome reports with Markdown table format. And I'm getting that directly from PyPI. So there's an API to PyPI to pull that out. So all of those descriptions and the name and links, those are all on the top PyTest plugin list.

And yeah, that's partly where I'm getting this. Like I said, I will cover other stuff too. Popularity isn't everything. The top of the list is going to be not too surprising. Let's actually talk about some of the other, the top. Let's go through the top nine because these are all ones that I'm definitely going to talk about. Probably the top. Yeah, more than that. But top nine are PyTestCov, PyTestMock, XDist, AsyncIO, Metadata, HTML, Timeout, RerunFailures,

And number nine is PyTest Django. And I definitely will talk about that because I'm working on a Django project. So these will all get covered, but I'm not going to. And then I don't know if I will do it all right away or jump around. Who knows what I will think about doing. So that's covering pre-existing plugins, which is, there's a few reasons why I want to do this. One, I want to let people know about these without having to go find them themselves.

Two, I want to plug my top PyTest plugin list, but I'm not making any money from it, so I don't know why.

Three, it's just good to like sort of hear about some other things. And the episodes might be short. So I might have some short episodes because I don't know much about something. But I don't want to like lump things together too much. If there's like there's a whole bunch of random ones, we'll probably do those together. We might do those together. And then there's different ordering ones that might do those together. Because it makes sense when we're thinking about

If you're going to reorder stuff, which ordering plugin to use, that might make sense to lump those into one episode. But a lot of them might be just individual plugins that we'll take a look at. Now, there's a lot of plugins that you should take a look at, even if you're not going to use, because...

they either will show you some cool use of fixtures or some cool use of the hook functions. So I find that learning how to use the hook functions from examples using the hook functions is the best way. Now there's the PyTest documentation and it's really good, but around hook functions and how to use hook functions, I think it might still be lacking. And this is something that like,

I think how to write PyTest plugins, that would be actually a decent course or a book. I do have a section of the Python Testing with PyTest and the Complete PyTest course, so both in book form and in video course form. I do cover building plugins, but I don't go through all of the hook functions, just a few.

So, opportunities there for people to jump on board. Anyway, this introduction is getting long. I'm going to talk about not just some plugins that you need to know about, but plugins that you might want to look at for their source code, and also some

Also might talk about some of these hook functions and how to utilize those and what they do. That might be a fun thing to do in this series. And I'm not sure where this series will go, but I thought it'd be fun to do a season of PyTest plugins. So, welcome. Thanks.

Thank you for listening and thank you to everyone who has supported the show through purchases of the courses, both Hello PyTest, the new fastest way to learn PyTest, and the complete PyTest course, if you'd like to really become an expert at PyTest. Both are available at courses.pythontest.com and there you can also join the Python test community. That's all for now. Now go out and test something.