Fortnite, Darth Vader and the use of AI voices in gaming
13:39 Share

Luke, I am your father. If you can't tell, I'm trying to embody the steely menace of the Star Wars villain Darth Vader. He was originally voiced by James Earl Jones in the movies.

I'm pretty confident that artificial intelligence would do a better job mimicking Darth Vader's intimidating tone and distinct rhythm. The fact is, machine learning now allows for highly realistic voice clones, and that partly explains why the makers of the wildly popular video game Fortnite have chosen to use AI to recreate Darth Vader's voice from the movies, rather than having a living actor do it. That's angered some folks in Hollywood, though, and they've filed an official complaint.

On today's episode, we're going to find out why, and we're going to look at how generative AI might impact the world of gaming. I'm William Lee Adams, and this is What in the World from the BBC World Service. We're going to hear a bit more about that complaint in a minute, but I want to explain who filed it. It comes from SAG-AFTRA, which stands for the Screen Actors Guild and the American Federation of Television and Radio Artists.

It's a union that speaks for more than 160,000 members who include actors, stunt coordinators, and perhaps most important for this conversation, voiceover artists. So let's get into it. I'm speaking to BBC reporter Liv McMahon. Liv, hi. Hi, William. How are you doing? Really good. Thank you for joining us.

First things first, in the Star Wars franchise, Darth Vader's voice was done by the late actor James Earl Jones. He, of course, passed away in September 2024. I'm assuming the actor's estate gave permission for his voice to be used in Fortnite. Do we know their position on the use of his voice and AI? Yeah, so they have indeed given this permission to...

for fortnight to use his voice and in fact they said in a statement attached to fortnight's announcement of the ai dove vader feature that he had always felt the iconic supervillain's voice was inseparable from the story of star wars and he always wanted fans of all ages to continue to experience it so they've also said as well they thought the collaboration would allow

both longer and younger fans of the Sith Lord to actually enjoy the character in Fortnite. So they've been pretty positive around that collaboration from the get-go. How have they used the voice in the game?

I think the way that they have used it is to allow people to interact with Darth Vader, who is a battle royale boss within the game. He's featured in Fortnite before. They brought him back, but with a twist. This time, he's a version which people can not only fight against as this very powerful boss, but also they can recruit him to their team. And more importantly, if they want to, they can interact with him

by chatting with him with their own voice and they will get voice responses. And that's the element that has been powered by AI. That's that generation of James Earl Jones's voice using AI software, which allows fans to interact with him.

And SAG-AFTRA, of course, they have a different perspective on this. Can you tell us why they issued a complaint against Fortnite for the use of AI? So SAG-AFTRA has alleged that Llama Productions, which is a subsidiary of Epic that makes Fortnite, they've...

said that essentially, Lama Productions chose to replace the work of human performers with AI technology, but more to the point, did so without giving the union any notice of its plans to use AI or an opportunity to enter into bargaining over terms around that use.

So it also said as well, it really welcomed its members and their estates exercising their right to control how digital replicas might be used and welcomed their kind of adoption of new technology. But it wanted to be involved so that actors more widely and its members could also benefit.

They said that we must protect our right to bargain terms and conditions around uses of voice that replace the work of our members, including those who previously did the work of matching Darth Vader's iconic rhythm and tone in video games. So there have been successive actors who have voiced Darth Vader in several video games of the big, massive Star Wars franchise. And I think it's looking for those protections for Darth

those actors and more games across the board. And you mentioned the word replacing earlier in that answer. Is this tapping into a broader fear that AI might put voice actors out of their jobs?

It is, yeah. We've seen this kind of bubbling away across the AI industry and also across like so many creative fields where I think creatives are particularly worried that companies will look to cut costs by adopting kind of faster, very easy and straightforward technology that doesn't involve as much payment. It's cheaper to do things like kind of

recreate a voice or recreate audio or images. And it's become a huge concern, particularly for the gaming sector, which over the last few years has been struggling around the world and also has been affected by sweeping layoffs. We've seen both actors and also developers, so people on the front end of video games, both getting very concerned that

that their jobs will be at risk as companies look to make savings and also adopt new technology like generative AI. I want to pause for a second and look at how using generative AI to create a voice for a non-player character or NPC actually works. Here's Larry Cutler from Baobab Studios, an Emmy award-winning animation studio.

An NPC character is typically controlled in a game by an AI system. And that AI system is in real time gathering inputs from the user. So it's taking into account user action. It may in fact be taking into account also dialogue or speech. If it's speech, it's actually going to run a speech to text converter, which will convert the speech into text. And if it's speech, it's actually going to run a speech to text converter, which will

and it may also be taking into account other aspects of the game as context. Then the AI system needs to decide what to do next and what the character might say. In the past, this AI system would actually be really complicated and really be a bespoke system to that individual character. And with generative AI, this actually becomes a lot simpler. We can actually embed a large language model such as ChatGPT

into the AI system and furthermore, we can actually fine-tune or customize that large language model to fit a specific character personality. Now, the LLM is actually deciding and creating completely on-the-fly responses based on user inputs.

And those responses are in text. And so then we have to convert that into a voice performance. And that's done with a model called text-to-speech. And text-to-speech, as its name implies, is actually taking the text response from an LLM and it's actually converting it into a synthetic voice. And so in many cases, this is choosing between a number of different preset voices and

But more recently, we've been able to clone voices from actors. So we can have an actor actually train a model by saying lines of dialogue. And over time, the amount that you need to train a text-to-speech model is actually becoming much smaller.

And then on the fly, that text-to-speech model will actually sound like that actor. Generative AI has enabled a much richer range of NPC characters that you're going to start to see in games. And I think we've only just scratched the surface. And what's even more exciting is that the speed at which these changes are happening is even more rapid than it was a couple of years ago.

Liv, back to you now. What have Fortnite said about SAG-AFTRA's complaint? Well, we've contacted Epic for comment. They haven't yet come back. It will be interesting to see what they...

But we know as well that there are these kind of broader kind of conversations and discussions happening between both unions like SAG-AFR and gaming companies like Epic over those terms and around kind of protecting employees and businesses.

protecting actors from the use of generative AI in particular, that's been a really big sticking point. So we haven't heard back yet, but I'm sure it will be interesting to hear what they have to say. ♪

Liv, let's zoom out for a bit. What are the wider implications of gaming companies using AI voices? Well, we touched on this earlier, but performers are really concerned that gaming studios could use generative area to reproduce their voices or their physical appearances to animate video game characters without ultimately paying them properly or providing them with fair compensation or credit. There is also as well that idea of not

not being able to fully control your digital replica or this version of an AI voice and losing control over how that is used in years to come, where it's used, what it does or does not say. There is also, I think, this continued backlash to the idea of companies across a wealth of sections just turning to AI

AI to make things that would typically be done by a human. I think people like to feel that they are having that human touch from content they're consuming and also within gaming as well. I think games now, especially blockbuster ones, they are more expensive than they ever have been. And for that, I think audiences are expecting quite a lot of time and effort and consideration to be put into them that they don't often feel is there with

using generative AI that can just do it in a moment and also can't think, can't feel anything

Can't have that same emotional connection that a human might bring. And just piggybacking off what you've said, perhaps the greatest defense for actors is the simple fact that they are human and they can more fully inhabit a character. And unlike a computer, they can actually perform. I want to play you this clip of Alistair McGowan. He's an impressionist and actor here in the UK. You have to know the character. It's not just the voice. So whether AI is able to replicate...

the person's soul because the voice reflects everything about your soul, a person's soul. I had a great voice teacher called Patsy Rodenberg when I was at drama school and she said everything is in the voice.

And we know this, Sita, particularly because of the hints that we pick up about people's character. Even if somebody is slightly unwell or slightly down in the days when you would talk to people on the telephone, you would hear them and within five seconds you'd say, are you all right? You sound different. Are you all right? You sound low. Are you all right? You sound like you've seen a ghost or whatever else. So everything is reflected in the voice. Now, can AI replicate that human soul?

So Alistair states a really obvious but also very important point. Humans are not robots. And generative AI, you can program it, you can try to teach it, you can use machine learning to get it to generate something. But ultimately, the viewer, the listener, the person on the receiving end, they have a heart. And it can be hard for an algorithm or science to kind of penetrate the heart. ♪

So Liv, let's look forward now. What about regulation? How do you see regulation of this generative AI space changing in the future? So I think regulators are starting to kind of take more notice of this debate around AI and particularly how it affects people's livelihoods and concerns around that. And I think that

While a lot is still up in the air, something that we have been seeing a lot of is voice actors and gaming industry workers trying to work with unions to negotiate better terms and protections to help ensure that any AI use is with their consent and it has clear limitations and clear language around it.

There's still a lot to be hammered out in terms of what that looks like. So there's a lot still up in the air and I expect that it's going to be quite some time before we get to a point where there is that clear kind of consensus and feeling from particularly from artists and creatives that their rights and their kind of views are being reflected fully in language.

legislation. Liv McMahon, a BBC reporter in Glasgow, thank you for explaining that. No worries, thanks for having me. That's all for today. If you're interested in gaming, we have several other episodes you might want to check out. They include Why Did Saudi Arabia Capture Pokemon Go? and Gaming Disorder, What Are the Signs to Look For? You can find those wherever you're listening to us now. I'm William Lee Adams, and this is What in the World from the BBC World Service. We'll see you again soon.