Meta Announces Movie Gen AI With Realistic Sounds
08:06 Share

Meta has just released what they are calling MovieGen. This is a new video model out of Meta. This is exciting for me. I'm going to be showing demos, talking about it, breaking down everything that they're working on because this is the first time that we've seen Meta seriously jump into the video.

video industry. Before this, we've seen them working on a bunch of stuff, most importantly, a lot of open source stuff. So it's pretty exciting what they're working on, but this is the first big jump for them into the video space. And so we're going to be covering everything that they're working on. Before we get into that, if you are interested in AI and side hustles and using some of these tools we talked about in the podcast to make money, whether that's to grow your business or to make money from side hustles, I would love for you to join the AI Hustle School community. This is something where I create...

exclusive videos you won't see anywhere else every single week breaking down the exact tools, strategies, and things I'm doing to make money with AI tools online. It's at AI Hustle School Community. Link is in the description. It's $19 a month. In the future, I'll change that probably to be $100. But for now, if you lock in the price, it'll never raise on you. I would love to have you as a member of the community. Okay, let's talk about what we're seeing out of Meta. So the first thing I have to start this off with is an absolutely ridiculous comment that

in an article from TechCrunch on this. TechCrunch said, no one really knows what generative video models are useful for just yet, but that hasn't stopped companies like Runway, OpenAI, and Meta from pouring millions

into developing them. Okay, the first thing I want to say is this is absolutely ridiculous. There's plenty of things to criticize with these video models. Primarily, you can criticize where they're getting their data from. You can criticize maybe they're not that good. You can criticize they don't understand physics yet. There's a lot of things to criticize. I wouldn't criticize. I think this is the most ridiculous take ever.

that no one knows what these are useful for yet. Okay, let me tell you what they're useful for. Everyone in Hollywood is secretly or openly trying to use them to save hundreds of millions of dollars in film costs. These things can generate amazing videos. And sometimes it's not just about like the video on the screen that you're looking at, but you might imagine like where there's a shot where there's a TV in the back

and they want to have licenses to whatever the images or the videos are on that TV in the background, they could generate something with AI. They don't have to worry about getting any sort of licenses. Like there's all sorts of little things like that

But, and that's especially when the models aren't as good, but as these models are getting better and better, we're seeing actual film studios embedding them into, you know, $300 million films are using snippets from some of these AI tools because some of them are pretty decent. And if they can even get one or two shots in there, that's saving hundreds of thousands of dollars. So that's like the big Hollywood. Then of course we have the entire side of this where it's like YouTubers are

B-roll. Hello, people spend thousands of dollars buying B-roll. I used to work in a marketing department and we had to get licenses for the B-roll of our videos. Okay. So obviously this is very useful. I'll get off of my, uh, my rant on that. Let's talk about what they're actually doing here. So Meta had a whole, um, breakdown of this where they're talking about, you know, what this can, what these things can actually do. They shared a video of, of course, a hippo swimming around in the water. Um,

viral hippo. So yeah, anyways, it's interesting though, because it's like swimming in the water and the shots from taken from under the water. So it's showing a couple of different physics properties. Now, the thing that I find the most impressive about what Meta is doing is beyond just the video, which is like, okay, this isn't an incredible video. Um, they're doing something new that I've not seen a lot of other people do. And that is that they're doing, um,

video and audio generation. So they said in their release, they said, finally, we trained a 13 billion parameter audio generation model that can take a video and optional text prompts and generate high quality, high fidelity audio up to 45 seconds, including ambient sounds, sound effects, and instrumental background music, all synced to the video's content.

So to me, this is amazing. And they showed a demo where they actually had a person riding a quad in the desert. And you could hear like the actual sound of the quad.

Okay, so the prompt for that was an ATV engine roars and accelerates with guitar music. So they have the guitar music, they have the quad, they have the ATV. Is it like a perfect video? No, hopefully they get there. We'll see. But all this to say, it's really impressive when they're starting to pair these two models is becoming so much more useful. They showed a bunch of really interesting demos as well.

where essentially they're able to take a video and change the background. So they have a kid releasing a lantern, the background's changing, they have a dog chewing on a stick, all of a sudden the dog's wearing a pink hat and pink clothes. So they're dynamically changing these videos, which is really impressive. The other thing that I think is really interesting is how they're able to personalize some of these videos. They have this thing where they support video generation that takes a person's image and is able to

make the girl all of a sudden she's playing music. She's a DJ and there's a cheetah in the background, but it was just taken from the girl's photo. So I think that's really interesting. Um, you're able to change the style. So they have some penguins that are in the desert or in the, uh, in the Arctic and they say changes to a pencil style. And all of a sudden the background becomes a pencil style. So, um, I think that the, the demos are really impressive. There's a lot of really interesting things that they're actually able to, to do with this. Um,

Beyond just making, you know, some sort of video, they're able to do a lot of different styles. They're adding the audio. A lot of exciting things are happening in this. They highlighted something interesting, which they said is kind of like talking about behind the curtains. They said, as the most advanced and immersive storytelling suite of models, MovieGen has the capabilities of creating video generation, personalized video generation, precise video editing, and audio generation. We've combined these

models, um, or we've trained these models on a combination of licensed and publicly available data sets. Okay. This is what everyone wants to talk about is the data set and where they're actually getting this from, because people are criticizing runway of using YouTube and other, uh, and also open eyes. Sora was heavily criticized for using YouTube. And when they were asked point blank about it, Miria, who now has left the company, uh,

by the Wall Street Journal was like, oh, I don't know. I'll have to get back to you when it comes to where we're actually getting our data from. So that's sort of dubious. One thing that I did think was interesting that TechCrunch speculates when it comes to where they're actually training this from

because they say that they're coming from a combination of licensed and publicly available data sets. TechCrunch says, quote, we can only guess this means a lot of Instagram and Facebook videos plus some partner stuff and lots of other things that are inadequately protected from scrapers, aka, quote, unquote, publicly available. So YouTube is one that comes to mind. It'd be interesting, but YouTube themselves have said that they don't want people to actually do this. So it'll be interesting to see if they're actually getting stuff from YouTube

YouTube. There's a bunch of other competitors in this space, like we mentioned, Runway, OpenAI, and Runway is the one that really is quite publicly available and is making, you know, they have all sorts of interesting, all sorts of interesting things where they're, you know, giving away $5 million to fund 100 films using AI-generated videos. They're really pushing the industry. And Runway is a great tool if you want to test out an AI generation tool. I would highly recommend

getting your feet wet with runway as it's publicly available. It's exciting when meta releases this. The thing that I don't love is, is all the look, we have this cool functionality, but you can't use it yet. So it's hard to know if this is actually true. Google kind of burned us back in the day by releasing some fake doctored demos where you don't actually know if this is cherry picked or if their models are generally capable of, of making some of this stuff. So I think we're,

Meta does have a bit of a leg up where they have the Facebook and Instagram data set to pull from that other people do not. So I'd be very impressed. I'd be very interested to see what they can continue to actually push out to the public. And I think we're going to get some impressive stuff out of Meta, you know, with the hundreds of millions of dollars that they're going to be spending on all of these models.

So if you enjoyed the episode today, I would appreciate it if you left a review, leave a like or comment on YouTube. And if you're interested in making money with AI tools, make sure to go join the AI Hustle School Community. The link is in the description for that. I would love to have you as a member of that and on the journey with us making money with AI tools.