AI Video Killed the Video Star
27:49 Share

A few weeks ago, Google dropped VO3, Generative AI Video, but now with Generative AI Sound to go with it. This is video from VO3. What do you think about the idea that we're just a bunch of prompts? If I'm generated from a prompt, how come I don't have six fingers? So is this. About to do the first plunge into an active volcano. Let's send it.

And this. Breaking news, the Secretary of Defense Pete Hegseth has died after drinking an entire liter of vodka on a dare by RFK. But how are the reviews? A slotmonger's dream, says The Verge. It might actually take my job, says YouTuber Matthew Berman. The world is not ready, says Mashable. We're so cooked, says The Verge.

Thousands of people on social media. But are we? Maybe not. That's our take at Today Explained. Support for this show comes from Monday.com. Use Monday CRM's AI to speed up your sales and cut the busy work. Less admin, more closing. Try it free at Monday.com slash CRM. Avoiding your unfinished home projects because you're not sure where to start? Thumbtack knows homes, so you don't have to.

Don't know the difference between matte paint finish and satin? Or what that clunking sound from your dryer is? With Thumbtack, you don't have to be a home pro. You just have to hire one. You can hire top-rated pros, see price estimates, and read reviews all on the app. Download today. This is an artificial intelligence version of Drake and UL is named Toto the X-Player.

Joanna Stern is a personal technology columnist at The Wall Street Journal. She is not a filmmaker, but that didn't stop her from trying to harness all the latest AI tools to make a short film. So I worked on this film with a close friend of mine and producer named Gerard Cole. He works here at The Wall Street Journal, and he's a seasoned audio and video journalist who just really has become obsessed with testing and playing with AI video tools. ♪

We started on this project probably at the end of March. I sort of challenged Gerard. I said, hey, we'll make a film. I think we should try to make something that's like a real film. We come to this place for magic. I'm going to make him an offer. I mean, like a two minute short film. You're not sure it's not Spielberg here. And what was so crazy about it is that every week there would be new tools that would come out.

The companies keep getting in touch and saying, well, actually, we have a new update next week. So you might want to hold off on publishing that video or you might want to hold off because we have a new tool that you can test. And so in May, Google announced VO3, which is their third version of their video model. They also announced a new tool called Flow, which makes it easier to edit with AI video. And so we kind of had to uproot the project a little bit to get this going. But...

This stuff is moving so fast that every night we'd go to sleep, we'd wake up in the morning and there'd be a new AI video tool that we thought we should try. The one that has gotten a ton of buzz over the last couple of weeks is Google VO. And this is from Google. This is VO3. What they did here with VO3 is they just created a new model that really blew people away.

Previously with AI video, not only did you kind of have some weird wonkiness to some of the visuals and maybe things didn't look as realistic, but also there was no audio to them. And now with VO3, you can put in a prompt. You can say, a woman working out alongside a robot. And now with VO3, you have audio. Go for the burn, sweat.

When I see the woman boxing with a robot, you hear... Sprunting. You hear sounds of the robot's mechanics. You hear... Punching sounds. You hear all kinds of audio to make the scene come to life. Which is to say, it just took, like...

a massive jump, this technology, because it just feels a lot more real. Yeah, it feels a lot more real. And I said, okay, well, what if we can see if we can actually tell a story here? We really wanted to see if that was possible. And we learned very quickly, it is possible. It's just really hard and time consuming. ♪

The film itself is about my, if we want to say that I'm me in this film, getting a sort of a humanoid robot. And these robots were designed to make people and humans more efficient. Time for your coffee IV drip.

Because I thought, okay, maybe we can have some fun playing off of this idea that AI is all about making us efficient in our jobs and in everything else. I'll let people watch, but the robot lives with me. We have some good times together. We have some not-so-good times together. He really wants me to keep working. My ultra-sensitive microphones indicate you are not engaged in elimination activities. And then I can't ruin the end, but, you know, let's just say I come out on top at the end. Wait, can you ruin the end?

Okay, fine, I'll do it, but I don't know. It's not usually how it works with movie interviews. But yeah, I mean, in the end... Spoiler alert. Spoiler alert. I get frustrated with this robot, and I had no other choice, but I have to reprogram him. Joanna, please don't do this. Oh. Yeah. Yeah.

But I will say there was a lot of constraints to making this. And so you'll notice when you watch everybody, you'll see like the robot doesn't talk, right? The robot has a voice, but it doesn't have mouth movements. And so that was one of the constraints we had. And you'll see I never talk. Like my mouth never moves in the piece.

Because we had that technical constraint. You can't really have the dialogue work very well between two people. You can't really make that consistent. And so when you watch with an eye for the technical constraints, you can really see like, oh, yeah, they kind of had to make something that was like this.

Tell people exactly how you made this short film. What exactly are you doing to make this? Because this isn't like shooting a little short film on your phone where you hit record, you capture some footage, then you edit it. Yeah. No, and I'll take you through as simply as I can, but it is pretty complicated. So we decided we wanted to have two characters, me and...

And I exist in real life. And this robot, which does not exist in real life. And so we created these digital versions of the characters. The robot named Max or OptiMax 5000, we created using an AI image generator called MidJourney. And so we kind of iterated in that. We worked through, okay, what does he look like? What does he look like? And so we finally landed on some images we liked.

As for me, I took a bunch of photos of myself, different angles. And so then we went into Runway, which is an AI video generation tool, and we uploaded those photos. And then we said, okay, create a scene where you see the robot working out alongside Joanna and make it in a suburban background with houses on a paved street.

And so then the runway would spit out what we would call the first frame of that runway.

And so we'd have an image, and then we would take that image and we'd put it into VO, Google's tool, and say what we wanted the motion to look like. And here's where things got really complicated, and Gerard really did a lot of this work. But you really have to give the model very specific instructions on what you want to be done. And so he worked alongside Google's Gemini, which is their large language model, to really craft detailed prompts of what we wanted the videos to look like.

And so these were long texts, like hundreds of words that you would put in with the photo and the text into Google VO, tell it what we'd want it, and out we would get a bunch of videos. And we'd pick from those videos what would look the best for the scene. When your video dropped, what did people think of it?

What was crazy was how mixed the reviews were. A lot of people wrote in saying they were blown away and they could not believe how real it looked. They could not, they laughed because we played a lot of bloopers. So there was a lot of people that really enjoyed watching this. Joanna is so good at doing these and brings in the mainstream in such a great way. But then there was a very loud and vocal group that just hated this.

Here are some of the reviews that I read on X or on TikTok. Wow, that was just awful. Ugly, soulless, nonsensical. Garbage. This is an abomination, and you should feel ashamed for making it. Absolute soulless shit. Wow. Shit from the butt. Shit from the butt? That's my favorite. Why are people so mad at your video?

They're mad at AI video. They're not mad at my AI video. They're mad at AI video in general for existing. Can you trust what you see? Because people don't let you know they're using AI. Deep fakes and misinformation could get a serious upgrade. Synthetic video evidence might become harder to distinguish from the real thing. You can also see in the quality right now, it's not really Hollywood level. Is that where there's like a more practical use for this technology right now?

That's the goal of many of these AI companies. I mean, yeah, I mean, that's where it really gets interesting. So some will say, like, look, this is a moment to democratize video tools, right? Those folks who aspire to be filmmakers, well, they can now just do this. They can sit in front of their computer and they can make things that they once never would have been able to make before.

But then you have the other side of this where what might we see on the big screen that might actually be AI generated. And so we've seen a bunch of AI film studios and production houses start popping up. The goal is for the makers, the Googles, the runways of the world to be working with Hollywood. Their hope is to start working with film studios to generate stuff that will end up in the films we see on the big screen or the small screen, whatever you watch your Netflix on. ♪

You can watch Joanna Stern's short film at WSJ.com or on YouTube where it's called We Tested Google VO and Runway to Create This AI Film. It was wild. We're heading to Hollywood in a minute at Today Explained. Today Explained

It's impossible to find more time in the day until now with HubSpot suite of AI powered tools. You can get more done way faster, speed up your lead generation and create attention grabbing lead driving quota crushing campaigns in an instant, which will give you more than enough time to listen to podcasts like this one.

HubSpot. Impossible growth made impossibly easy. Get started today at HubSpot.com slash AI. Support for today's show comes from Bombas. Bombas wants to make your summertime in the sun a little more comfortable with socks that they say are perfect for your next marathon or just your next trip down to the bodega. Bombas says their running socks help wick, sweat, keep you cool and fight blisters and...

They don't just stop at socks. Bombas says they also offer those white tees, those waterproof slides, and those sweat-wicking mudans. Nisha Chichal is our colleague here at Vox, and she's tried Bombas herself. I am part of a whole family of Bombas wearers. My daughter, who's three, also wears Bombas. She has several pairs in different colors.

toddler, kid sizes, and they're great. The kids' ones have little grips on them, which is great because she runs around a lot, so the grips help her to make sure she's not slipping on wood floors. So she's a fan, too. Bombas also wants you to know about their mission, which is for every comfy pair you purchase, they say they donate another comfy pair to someone facing homelessness. You can head over to bombas.com slash explain.

And use code EXPLAINED for 20% off your first purchase. That's B-O-M-B-A-S dot com slash EXPLAINED. Code EXPLAINED at checkout. Bombas dot com slash EXPLAINED and use the code EXPLAINED. Support for the show today comes from Jerry and Ben's nowhere to be seen. This is not them. Jerry is an app that says they can make finding the right car insurance a breeze. From comparing quotes to getting you covered, everything can be found in the Jerry app.

Just answer a few quick questions and then they can instantly pull quotes from like over 50 top rated insurers. You guys, you can stop needlessly overpaying for car insurance. Jerry says drivers who save with Jerry save over $1,300 a year on average. Before you renew your policy, you can download the Jerry app online.

or head to jerry.ai slash explained. In just a few minutes, you can compare quotes and coverages from up to 50 top insurers. Jerry says they make car insurance simple, smart, and finally, on your side. Based on drivers who switched and saved with Jerry over the past 12 months, over 20% of drivers who switched with Jerry found a monthly premium of $87 or less. Not all drivers find savings. ♪

We come to this place today explained for magic because we need that. Devin Gordon wrote a big piece titled What if AI is actually good for Hollywood for the New York Times magazine late last year. We asked him, how dare you? Here's what he had to say. The premise and starting point was this.

My sense that if you were listening to the discourse about AI in Hollywood, you would either hear that it was going to be the end of Hollywood and wipe out everyone's jobs and turn the future of cinema over to robots, or it was going to be the greatest creative unlocking event

magical wand ever handed to creative filmmakers in the history of humankind. And I had also been hearing and reading these stories in places like The Hollywood Reporter,

Everyone is using AI, but they're scared to admit it. It's the dirty little secret. AI is being used for scripting, for shooting and producing movies. You go into a little booth that's 360 degree camera and you're asked to do 30 different expressions. And so I was like, OK, well, what are people actually using it for? What is actually happening with AI? So I started with a

visual effects company that works with AI called Metaphysic. The reason why I wanted to start with them is because everything I kept hearing was that when AI descended upon Hollywood, it was going to hit visual effects first and hardest. So I wanted to start with a visual effects company. And this particular special effects company, visual effects company, Metaphysic, their specialty was sort of taking

the deep fake logic and of digitally creating a photorealistic copy of a famous person's face and applying that to all sorts of aspects of the filmmaking industry from special effects to dubbing to reshoots, animation, aging and de-aging, et cetera. And so I went and spent time with them

Tom Hanks the actor? My mama always said

Life was like a box of chocolates. I could see my face. It was still me. And if I talked, it was moving. But I was also very recognizably Tom Hanks. You never know what you're going to get. The reason I was Tom Hanks is because the film project that Metaphysic was then working on was a movie called Here that starred Tom Hanks and Robin Wright. Hey, Dad, I couldn't meet Margaret.

Nice to meet you, Margaret. Nice to meet you, Mr. Young. It was directed by Robert Zemeckis. The team from Forrest Gump reunited again using AI technology to a degree that it had not been deployed in a Hollywood movie before. In fact, it was central to the making of it.

She's pregnant. She's what? She's pregnant. Margaret is pregnant. You're just 18 years old. In this case, they were using it to enable Tom Hanks to play the same character from the age of 18 to the age of 80. And the way they were able to do that was using metaphysics, AI technology. And one of the reasons why I wanted to focus on this movie here

which is not a particularly good movie. I wouldn't necessarily recommend you Netflix and chill with it. Get the fuck out of my house. But I was interested in this movie because this movie is probably the first mainstream Hollywood movie that would not have existed without AI technology. And the reason why is because it's effectively a small domestic company

emotional, serious drama. The only reason why this movie could happen is because the visual effects that it required were cheap enough with AI. It's as good as CGI now, and it's a lot cheaper, and it's a lot faster, and it gives directors a lot more creative control on the set. So that's why in the visual effects space, there's such this expectation that AI

AI is very quickly and already is in a lot of ways transforming that industry in good ways, but also in ways that's probably going to cost a lot of people their jobs. I mean, and let's talk about all those people for a moment here. Let's start with Tom Hanks, because one thing that really surprised me about your piece was that you asked Tom Hanks how he felt about the potential for AI to enable him to star in movies 100 years after his death. And he was like,

Bring it on, right? Surprisingly unconcerned. Wow. He was just sort of like, well, let's just get the paperwork sorted out. Amazing. And I was a little surprised, to be honest, about how cavalier he was. For instance, I mean, isn't it easy to imagine a scenario, maybe not in the Hanks family. I'm sure the Hanks family is going to, I trust Chet. Do you trust Chet? Big up, big up the whole island, man.

No, but I do trust Colin. I trust Colin. I trust Colin. You always want to work with good people. And obviously, I think my dad's good people. But OK, what about Colin's grandkids? And they're down on their luck. And all of a sudden, 100 years from now, Tom Hanks is legend.

His imagery is being sullied because he's being, you know, his image is being used to make bucks in porn or whatever. He's not thinking that far in advance. Let's put it that way. I think the takeaway for me, no shots at Tom Hanks, was that it did sort of reflect a class divide in AI culture.

worriedness and how worried you should be. Right, because not everyone is Tom Hanks. I mean, what did you learn about all the people in VFX or, you know, costumes or makeup or what have you that are terrified about what's about to happen to their industry? You know, one of the things that I kept hearing on the makeup front with AI is a director going to have to have a makeup department do a character's makeup every single day.

Or can the makeup department do it once, right? At the start of the production, that becomes a file that gets saved and mapped onto the character's face later. And now instead of having a makeup artist for the entire run of the set, you've only got the makeup artist for one day.

You go from makeup artists being paid by the day to some sort of almost license or copyright for how many days that makeup work gets used, right? The entire economics of the industry has to change. Does it mean that we're not going to need makeup artists? Of course not. We're still very much going to need makeup artists. They're going to need them as much as ever. But how they work and how they get compensated is going to radically transform. And you can go through every department in the filmmaking process.

And each of them would have different ways in which AI will disrupt how they work. The thing about all these ways is that none of them are as grandiose as the worst of our imagining, right? You know, the people who were the most skeptical about AI's ability to overtake human creativity that I spoke with are the people who understand AI the most and use it the most. They understand its limitations and

and also how to best use it. They understand how to use this tool. When we're talking generative AI, when we're talking creative orientations or applications of AI, they understand how indispensable the human mind is to that equation. It just doesn't work without it. The notion, the theory, who knows if this will come to pass, but the positive theory, the flip side of this,

is that AI lowers the barrier of entry to so many more films that even though the size of the crew and production is shrinking because of AI, the amount of productions that can exist grows because more people can afford to make more movies. You can accuse that of being sanguine and overly sunny. I would say in the defense of the sanguine people,

The indie film movement does provide an interesting parallel here, right? When filmmaking went from very, very expensive, limited film in the 90s to small, handheld digital filmmaking where anybody could make cinema-quality movies.

All of a sudden, you did have a lot more movies, right? You had a lot more movies being made for a lot less money. So there is a test case, right? Can AI do that? Well, I feel like in some ways that brings us back to our friend Joanna Stern at the Wall Street Journal.

To her haters out there, I think you're missing the point. I don't think that Joanna Stern is in any way trying to make a film that could go air on ABC or air in the movie theater. What she's trying to demonstrate is how easy it is for even someone like her to effectively make

sit there and make something that looks at the worst, the bad knockoff, but look at all the things that she can do without having anybody. Exactly. Or experience. Anybody. Or experience. Yeah. Right? Right. And now take that capacity out of her hands and put it in the hands of people who actually do this for a living. Right? And the question is how do

dangerous does this get? How many people is this going to replace? And I just don't think we know. I don't think we really know. In some ways, what Joanna's film leaves me with is both fear and relief.

Read Devin Gordon's great piece on Hollywood and AI at NYTimes.com. This episode was made by humans. Their names, Peter Balanon Rosen and Gabrielle Berbe. Amina Alsadi and Abishai Artsy. Patrick Boyd

and Andrea Christen's daughter. And I'm Sean Ramos-Verm. And here are some more humans who didn't work on today's show. Noelle King, Miranda Kennedy, Jolie Myers, Hadi Mawagdi, Miles Bryan, Victoria Chamberlain, Devin Schwartz, Denise Guerra. We used music by Breakmaster Cylinder and Laura Bullard as our senior researcher. Today Explained is distributed by WNYC. The show is a part of Vox. You can listen to this podcast ad-free

free by signing up for a membership at Vox.com slash members. Right now, you can pay 30% less than normal for that membership. So get in there while you can. Thank you and have a weekend. Thanks to Smartsheet for their support.

Learn more at Smartsheet.com slash Vox.

The new McCrispy Strip is here. Dip approved by ketchup, tangy barbecue, honey, mustard, honey mustard, Sprite, McFlurry, Big Mac sauce, double dipped in buffalo and ranch, more ranch, and creamy chili McCrispy Strip dip. Now at McDonald's.