#407 Back to the future, destination 3.14
28:26 Share

Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 407, recorded October 28th, 2024. And I am Brian Ocken. And I'm Michael Kennedy. And this week we're sponsored by ourselves. So please check out our offerings. There's all of the fabulous courses at TalkPythonTraining.com.

And then over at PythonTest.com, we've got the complete PyTest course and also the short course, Hello PyTest. And also thank you to our Patreon supporters who we don't do a shout out enough to, but thank you very much. We appreciate it. If you'd like to connect with the show, we'd love to hear from you. You can connect with us on Fostedon. We're at mkedney, at Brian Ocken, and at Python Bytes, all at Fostedon.

And also those links are in the show notes, as well as in at PythonBytes.fm. You can find out when the next upcoming episode is going to be. And I was going to mention this later. I'll just go ahead and mention it right now. Next week on November 4th, we are going to be recording early. So it'll be at 730 in the morning instead of our normal time. And that's Pacific time instead of our normal 10 a.m. Pacific time.

because of work for me. Anyway, so that's what's going on next week. But at PythonBytes.fm, you can sign up for our newsletter, you can get the show notes, you can sign up and look to see when the next live recording is. But right now, I'd like to hear from Michael to see what you have to talk about. - Shall we teleport, transport to the future? - Yeah. - Brian, I want you to imagine a time when it is not so gloomy and rainy.

about 11 months from now when we have a new version of Python that comes onto the scene 3.14 and that future can be for you now in a limited sense because 3.14 alpha 1 is now available the pi version of python that's right so uh

That's the first Alpha 1 release. And there will be seven, in theory, by plan at least, unless something goes wrong. This is the first of seven Alpha releases before we go beta, before we go release candidate, and then we finally ship. So it has...

It has some of the new ideas, but not all of the new ideas. It comes with performance improvements and actually quite a bit more, which is pretty interesting. So if you read the blog post announcement, well, it looks like, okay, so we've got this new PEP called Deferred Evaluation of Annotations. Now, I think this was meant to land in 3.11 or 3.12, something like that. And all of the frameworks and libraries that thought,

typing should matter. Like Fast API. Yes, and Typer and Pydantic and so on. And I count myself as a big fan of all of those libraries.

All those people are like, hey, hold on. This is not just a thing for MyPy. These types are not just for MyPy. And if you take away their evaluation, all of our awesome frameworks go away. And I think that would be a massive step back for Python. A lot of people agreed, made some changes. So that wasn't done as far, at least in a limited way. But having these type annotations does have a performance cost of, say, an import potentially. It does...

have weird issues where you've got indirect or sort of half-defined types that you want to talk about. I have a function or a method on a class and I want to say it returns one of those classes or, you know, it's like an equals. It takes another one of these classes and it tells you if it's equal to it.

So you had to do weird tricks like typing .self or you could have quotes the name of a type that's not defined yet, but eventually someday it'll be defined if you get around to looking for it. So this will allow you to not worry about those types of things. And there's a new API if you consume them. If you were Pydantic or FastAPI or whatever, there's a new way to say...

let me read the annotation metadata about this. That's slightly different, but it's still there. Improved error messages. We all love improving further improved error messages. So there's that. And then, so that sounds like when you read it, like, okay, that's the thing. But actually, if you go and look, where's the link? The release notes are in here somewhere. So you click here and

Why are there's a release notes? Do, do, do, do, do, do, do, do. Really? Where did it go? Maybe I haven't noticed. But anyway, Brian, there's a ton of changes, a ton of changes from there. I think. Oh, I know where it is. They don't call it release notes. Here's how you do it.

you click on the pep and then you scroll down until you get past the pep and it just has other changes. So, yeah. So there's, if you go to either of those links, it'll tell you about their message, tell you about the pep, but then there's, if you look at the scroll bar, a bunch of stuff below that. For example, incorrect use. Yeah, I think that's actually the news, not the stuff that was called out. Incorrect usage of await and

And asynchronous comprehensions is now detected. I don't know exactly what you could have done wrong before, but if you're doing it wrong, I guess, anyway, if you're doing it wrong. Well, apparently, like, assert await one. Yeah, so I think what the issue is, you could await a thing that is not awaitable, potentially, in the list comprehension. So now you'll get a warning. Is it warning or error? Syntax error. Yeah, syntax error.

But you can disable the syntax error because apparently it used to work. If you want to just keep doing that, that's fine. Let's see. Some stuff about dunder debug. The numerical things now have a float.from number and complex.from number, which converts to a float type or a complex type, which is...

somewhat interesting. So the thing is, this is not a parsing operation. It takes one number and converts it. And if you give it a string, it'll be an error. So this is like, I just want to make sure I'm always working in numbers and convert them in probably a more efficient way. Oh, better than like a cast thing. Yeah. Yeah. Where the cast is just... Where the...

A type annotation type of deal, whereas this is a true transformation, right? Nice. So we get the new annotation lib, which talks about, which provides those features that I talked about for PEP 749, the deferred type annotations. ArgPars gets some updates. AST, concurrent features, C types, decimal, daytime, disks, fractions, functuals, HTTP. It's funny. You know, HTTP, this was something nice. You can say Python dash M HTTP dash,

server I think is it whatever so if you have just a directory and you're like I need to have this as a web view for example if you have a an HTML file there and it says I want to open up forward slash some CSS file or forward slash some JavaScript file or something along those lines

If you just open it, it goes, I don't know what these files are, right? But if this web server were to serve it, it would easily work. So you could just type Python-M space acp.server, enter, and it'll pull up a little web, like a no-op web server that you can then actually interact with that stuff with. So now it has a dark mode. Oh, you should have led with that, man. That's cool. Yeah.

Yeah, this is contributed by Yorick Henson, and I thought, okay, Yorick must be a huge fan of dark mode to want... As am I, yes. Right on, okay. JSON, get some updates. OperatorOps, Pathlib. Pathlib, actually, the changes to Pathlib are interesting. I'm going through this list to point out, like, there's a lot of changes just in this Alpha 1 release. But Pathlib now has a...

Add methods, like new functionality to Pathlib to recursively copy and move files and directories. Oh, yes. I'm here for this. Copy. Copy is a file or directory tree to a destination. Copy into. Copy is into a destination directory. Move and then move into. That's awesome. That's good. So Barney Gale did that. Thank you. PDB, Pickle, Pydoc, SimTable, Sys, and Unit Test. It ain't dead. No, it ain't.

Anyway, so there's also optimizations. For example, async IO is now implemented using doubly linked list implementations. Hat tip for the comp sci data structures level up there for native tasks, which speeds up execution by 10% on standards, standard high performance benchmarks and reduces memory usage. So who wouldn't want that? That sounds great.

Exactly. All right. There's more. You can read about it. And I just touched on it. Like each thing I said had a paragraph or list of bullet points you could go into if you wanted. So that's it. Now, back to the future. Back to the present. Whatever. Back to the current time. Yeah. Or the fast time. So just last week on October 21st, episode 406, we talked about...

We talked about a lot of stuff, but one of the things was PEP 735 with dependency groups in PyProject.toml. And I didn't know when we could play with them. And the answer is now because UV decided to do that right away. So UV just had to put a change in in 2021.

In version 0.4.27, in the changelog, they say the dependency groups are supported, which is super cool. There's even support for PEP 735. I couldn't believe this was so fast. It was like just a few days after we talked about it. They must listen to the podcast. That's pretty awesome. Yeah, probably. And this was like, oh yeah, three days ago. It was merged in. So this is pretty cool. So there's a whole bunch of added stuff.

And so I went off and played with it. So there's dash dash group is added to UV add and UV remove. So when you add dependencies or remove dependencies, you can say group and it puts them in groups. The dash dash dev and things like that. There's a couple of those. So dash dash dev used to be an extra dependency. Now it's a dependency group called dev and...

And so --dev is the same as saying --group dev. It's more clear in the list, in the documentation, if the documentation is updated. So there's a couple of things where it isn't quite updated. So there's a lot of, it was very clear on how to add dependencies to dependency groups with UV now. You can say uvadd or

or UV remove, that works great. But how do you install them then? So that's what I tried to play with. So also the documentation for default groups is up. So you can click around in the UV. Talks about default groups and dependency groups. And here we have like the dev for PyTest. Of course, you're going to have PyTest in your dev dependencies. And there's the little tip for the documentation

dash dash dev flags. Okay. So I'm like, okay, so how do I install stuff? So I expected what I expected to have work is to say UV pip install dash dash group dev and be able to just install those and not the project. Well, that doesn't work yet. So there is a issue filed by Hinnick and the

of this. This should work and it doesn't. Pip install group. But the issue is that they want to be like the same compatibility with the interface for pip. So they're just waiting for PyPA to define what that interface is. And that's fair. So we've got all these dependencies and you can't install stuff, right? No, not quite. So I also asked on LinkedIn if anybody knew

And Tushar responded saying, hey, the docs, which they don't, but it says UV sync group will work. So I checked out the docs and I can't find it. The sync doesn't talk about groups, but it does work. I tried it. You could, so you,

Essentially, this works. So what do you do? You create, you have a, you say you added a dependency group, then you can say uvsync. And if you add the group, like dash dash group, and then the group name, and you can add a bunch of those, then your

Virtual environment will sync to match what your project says. So basically that's how you can install. You can use sync to install your stuff. Long-winded, but that's how you can do it. Nice. Oh, that's awesome. Anyway. Yeah. So this lets you say, unlike a requirements.dev, it lets you say, here's a separate set of dependencies that I need for a certain action or set of tasks on my project. It doesn't necessarily pull in the base dependencies, right? That's the big difference. Yeah.

Well, that's the idea, but with UV, it's different. So with UV currently, with utilizing sync, it will pull in your base dependencies and the extras. You can't do just the group yet. Got it, because it's not defined. It's not agreed upon yet how to say it. Right. So the syntax is there, though, so you can play with the syntax. And it's very comfortable. Actually, there would be very seldom...

Like for me, I'm okay with this being, having to install the project and then the dependencies. But there's, there are times where you just want to install a group like to build the documentation or something like that. Yeah, I'm fine with it as well. You know, one that might be

on most people wouldn't guess for, but in the ML space, some of these dependencies are massive. For example, if you want to use the large English model for Spacey, you specify that as, you specify it as a dependency. You don't Python, maybe you run it as a Python call. Anyway, that thing is like 500 megs or something. All right. So you might want to just

not download and install that as well. Right, especially if you have like a modular CI system where one of the build stages is like building documentation or linting, static linting or something like that. You might not want to pull in the world just to do one of your stages. Yeah, that's a great case. Okay, let's dive into the next thing. Okay. Dive comes to us unknowingly by Mike Fiedler. Let's see. Yeah, Mike Fiedler. So he...

mentioned this in a group discussion that I was also looking at. I don't know that he sent it in specifically for us, but here it is. I'm taking it because it's a cool idea. So here's the thing. If you have some sort of container system, like a Docker image, you

have a bunch of steps. And when you apply, the way you make the containers is you go to a Docker file and you put in lines like, do this action, then this action, then this action. And they can be, they're usually some kind of Linux command or maybe you'll copy these files or run this Python command once you've copied the right stuff over. And so they can be big, like apt install this one thing, but this one thing has 20 dependencies or

Or you might be running install the development requirements or a compiler that you didn't need. So you might end up with slow builds and large images. What do you do about that? You run this dive thing on it. So this thing's pretty cool. So what it does is there's a little animated graphic you can have here if you go to the GitHub page. Hat tip for anything UI, please, animated GIF would be awesome, wouldn't it? So anyway, you can have this here.

And the idea is it basically allows you to see every layer. So each line of code that you've run, like make a directory, copy these files, UV pip install, dash R requirements or something along those lines. And each line you click on, it gives you all the files, like a tree that you can explore with the file size of the directory.

of the image, the resulting image, and then also kind of a delta. So as you click around, it'll say these files were added, these files were changed, these files were deleted on this step. So you can see this is the reason this file is here because on line four, we ran this command which added these files. Why is that?

Oh, that's cool. Isn't it, right? Yeah. So if you were shipping containers around, shipping images around, you know, like pushing it to Docker Hub and then pulling it somewhere else, like you should definitely look at this. It's not something I'll probably care about because I just build my containers on the server and then use them, which is also a way to do it. But if you ship them around and you care about their size or they're building slow or whatever, this is awesome. Check this out.

Also for security concerns to make sure things don't get added that you don't know about. Right. Or even just, I want to make sure that there's no compilers on here because if somebody were to break into the container and they could use the compiler, then they could say, upload some C++ code, compile it and run it. Yeah. Right.

you know, the so-called living off the land sort of issues after somebody breaks into a thing. So you just say dive space your image or you can alias it to, this is a, this is pretty nice actually. So there's something I've started doing with some of the tools if I'm already on a system that has Docker. So you can do this with glances, you can do this with dive, you can do this with other things as one way to do this is I could brew installed dive onto my Mac, but then

Who knows what that's doing to my computer? I'm running arbitrary code off the internet. I probably trust it, but you know, eh.

Right? So what you can do is you can alias, so you can run this from Docker, and then you can alias dive to just be the sequence of Docker commands that you want to run. So even dive itself runs in Docker, and then it talks to the other Docker container that you pointed at, and that's it, which is a really nice pattern that's getting to be more popular. It's Docker inception. It's Dockers all the way down. But yeah, you just create an alias, and then it's just Docker run dash IT, remove the image, map the volume it needs over, and then off it goes. Yeah.

Yeah, it's really cool. All right, what else? Real quick, I'll wrap this up. So it'll show your image broken down by layer, like I said. What's changed? It has a metric, like cyclomatic complexity equivalent, but for Docker, they made this up, I believe. An estimated image efficiency. The lower left pane of the base layer shows an experimental metric that'll guess how much wasted space your image contains. That's kind of interesting. You could do a quick build analysis.

quick build test so instead of going and building the thing and then running dive you can say dive build and it'll actually build the thing and then test it instantly so it's like one line and then there's CI integration for you know basically doing some of the stuff in CI and ways in which you can make it fail the CI if it fails some metrics and so on

Oh, that's cool. Yeah, Pat Decker out there says, I've been using Dive for a while. Neat. Awesome. Glad to hear the testimony. So, yeah, people can check us out. Obviously, it's open source, written in Go, 46,000 GitHub stars. So it's pretty well known and used. That's pretty nice. Also, I like the CI integration because if I can't get it to a feature from the command line, it doesn't exist. That's my concern.

Anyway, cool. Nice. I've got just, I guess, a little blast. We talked about the future. Let's talk about the past a little bit. I had a question. I haven't talked about PyTest metadata for a while, but I had a question just this last week. Somebody got a hold of me and said, hey, I've got these PyTest CI system put together. I'd like to send some extra data from the test environment to PyTest

to the report server. Is there a way to get metadata, like extra metadata in PyTest to add it to the results and then put it somewhere else in the result system? And I said, well, have you checked out the plugin called PyTest Metadata? It does exactly that.

So I guess I'm just reminding everybody that PyTest metadata is a really cool plugin. It does exactly what you would think it would do. It bundles metadata with your results, or at least that's exactly what I think it would do. It isn't testing your metadata. It's adding metadata to the results. So some of the things it does right off the bat, which is kind of fun, is it automatically stuffs in four, did I count them right? Yep, four different keys. It's key value pairs for the metadata.

It does the Python version, the platform. So Python version is like, you know, whatever Python version. The platform is a string that we get out from Python to say what kind of like, you know, for Mac OS, it's Darwin or something else. And then the packages, it lists all the PyTest packages that are there. And then...

Actually, it says the description of PyTest packages. I think it's... I'm going to have to check this out. I'm not sure if it's all packages that are available or just the ones associated with it. Yeah, the name says PyTest, but the example lists more than PyTest. Yeah, so I'm not sure. But then there's plugins and it lists all the PyTest plugins that are there that you have bundled, but

I don't really use it for that. I mean, that's interesting, but really I add stuff. So we've got like my, I'm often the target environment that I'm testing. What version is that? What extra packages are on that? So if I'm testing, you know, embedded systems, so I'm checking information about the system I'm testing and adding that to the metadata frequently, but you can do whatever you want. But this is a very,

But this adding is cool and it's really easy to do. You can add it during, you can even add it at the beginning, like on command line, if you know it at command line. But within the test, you can do it within a fixture or within a test as well. You can add metadata. And it's reported in the command line if you do the dash dash verbose that gets or dash B will add like reports all the metadata. But it also comes out in the JUnit XML. And a

And a lot of JSON results and other plugins will pull this metadata out also because it's a very widely used plugin. So very useful if you want to pass that along. It was, I believe it was generated specifically for the PyTest HTML plugin because it's the same contributor, Dave Hunt and others. But...

but it's, uh, the metadata is used even without the HTML, um, in the, in our top pie test plugins list. Um, the metadata is number five and HTML is number six. So they're both very popular, but, uh, metadata is popular by itself. So yeah, very cool. I love it. That's a great idea anyway. So that's, those are our items. Um, uh,

I have a couple extras or one extra. Do you have any extras? Hit them. Okay. Yeah, I got a few, but they're quick as well. It makes sense to start right here because I did make some changes to the plugin list. So just the other day, I was looking through the PyTorch

the top PyTest plugins list over on pythontest.com, and noticed that there was stuff in there that I don't like talking about because they're deprecated, and even the developers don't want you to use them. So I've started to start filtering those. So I'm going to go through the list and check out to see if there are deprecated ones

Because you can already find this information out if you want to find out with the list of which ones I'm pulling out. But why would you want to do that? We want to know the top PyTest plugins that I should look at. And if there's ones you shouldn't.

So yeah, those are coming out. And then I thought, you know, I've been wondering to do a new series on the Testing Code podcast. And I thought, why not look at these plugins? So I'm not going to go through the entire list. Plus it changes like every month a little bit. But I'm going to go through some of these and pull out some that I think people should know about. So I'm going to do like a series of plugin episodes. Yeah.

Max, I'll be coming up. How about you? Any extras? Uh,

- A couple. Let's see what we got. So let's start here. So Hugo Van Camande said, "Guess what? Pillow," you know, the Python image library, 11.0.0 is out with not just support for Python 3.13, but free-threaded Python wheels. - Oh, nice. - So we're doing a bunch of image processing and you want to speed that up, the free-threaded Python T or whatever it is you run is good. Drop support for 3.8 'cause that's so old school. Removed a bunch of stuff.

And then added many things. So if you go look here, I believe, you look at all the changes in this. So pretty awesome how much has changed here. So if you do stuff with images in Python, use Pillow, there's a new one that's a big release. But if you're like, what could I do with images in Python? Well, check out Pillow. It does a lot. So that's cool. We have...

You know, you've heard of pip install anti-gravity. You've heard pip install this, or sorry, import this and import anti-gravity. But what about pip install Deutschland? What does that do? Hmm.

So this comes from Grand Knapp. It says, hello, Michael and Brian. Here's a recommendation for the 14% of your listeners who are in Germany. I remember I talked about that. It's analytics from Imami a while ago. And it says, just pip install Deutschland for lots of useful German data sets, regularly updated. And so you go over here and it's got things like, do you want to go and...

Find addresses, barrier lines, building areas, building borders, I suppose, and so on. My German construction terms are not that good. But anyway, something along these lines, right? And you basically, it just has data for all these different components or things you might want to know about in Germany, which is pretty cool. Yeah, that's pretty cool. A lot more than just buildings. There's quite a bit there, isn't there? Yeah, yeah. Addresses...

history points, underground stuff. It's got Autobahn information. Yeah. Yeah. We need a pip install for a lot of countries like pip install USA. It chants a little USA, USA, then it installs it. All right. America, America, America's got boards. Okay. So another big thing that's fun is I just released a talk podcast.

Python blog. So I've got my personal blog and there's tons of RSS at TalkPython like for the podcast and stuff as you would guess. But a dedicated blog so I can post interesting things and series that I plan on working on that are more TalkPython related, not just for my personal blog. So anyway, it's got a couple posts already and yeah, I tried to go for the super clean, super readable version here, Brian. So trying to make it just not. That's nice. Yeah, thanks. So this is built. Yeah, go ahead. Is it

Go ahead. I was going to ask you what it's built with. Well, it's built with Hugo, which is a glorious static site builder. But what's interesting is it's hosted under talkpython.fm slash blog, not like blog.talkpython.fm. And I'm using Nginx to point different sub-urls to different sections. So like most of TalkPython is a PyTorch,

Python app, obviously. But then this part is Hugo, and they just kind of coexist under the same URL structure, which is fun. Okay, nice. Yeah, I might write about that actually at some point. But yeah, it's built with Hugo. Kind of like our own stuff, right? Our own personal things. Yeah, I'm just having some Hugo. I'm having Hugo issues, but... Are you? Well, I'm still having Hugo love, so it's okay. Well...

Even it out there. All right. I got a joke for you, but I can't find, this is a cartoon I saw, but I can't find it again. I thought I bookmarked it in our thing that bookmarks it and the title of it is there, but the URL to it is not. It just says dog joke. I'm like, huh? Wonder what,

where that is. And I've tried to search for it and I can't find it. But I can tell you the joke, okay? Okay. Are you ready? Yes. Imagine, imagine if you will, a little girl just getting her first puppy. The dad has brought it home and they're out in the backyard and...

He's about to share some wise advice with his daughter to help her become accustomed to having a puppy and all the new responsibilities and so on, right? Yeah. So she's just sitting there like gleefully petting the new puppy and says, now they're thinking of a name for it, right? He says, now, sweetie, you need to think very carefully about the name for this puppy because

because you're going to have to remember it for the rest of your life and enter it on every webpage that has a stupid security question. Yeah. So that's the advice of the dad. Remember this name of your first pet because you're going to have to remember that for the rest of your life. Yeah. That was a joke. That's good. No picture because I can't find it. I've got a dog joke. So let's add another joke. Let's do it. Okay.

So kind of a dog joke. So a guy walks into a library and he's looking for a book and he's, and he walks up to the, the librarian says, says there's a particular book I'm looking for, but I can't remember the name of it. It's a, it's, it's a, it's a book about Pavlov's dog and Schrodinger's cat. And the librarian says that rings a bell, but I'm not sure whether we have it here or not.

Perfect. Pretty good. A lot of science in there. Some social science, some hard science. I love it. Yeah. Yeah. All right. Some uncertainty. Yeah. Yeah. Might be there. Might be there. And it rings a bell. Cool. Well, awesome episode again, I think. So thank you, Michael. Yeah. And thank you, everybody, for listening. So I guess we'll talk next week. Bye. Yep. Bye all.