The Future of AI Avatars and DeepFake Technology w/ Joshua Xu & Steve Brown | EP #161
35:38 Share

Where does this go? What's your vision for where this is going? How will I use these avatars? Very, very soon, we're actually going to see everybody be able to have a meeting with the agent in the video interface. Joshua Xu. Joshua Xu. Joshua Xu. The mind behind HeyGen. He is redefining how we create and communicate.

You will be able to have your agent talk to people in different languages. 175 languages, that's insane. I'm very excited about the mission that we have enabling, you know, visual storytelling for all. The technology has gotten so extraordinarily powerful.

And yet we're just at the very beginning. We take trust and safety seriously on the platform. Every single avatar created on HeyGen will require the first party consent. This is the future which is available when? Now. Now that's the moonshot, ladies and gentlemen. So Joshua, you have built an extraordinary company. I mean, I think truly if you have know anything about digital avatars, HeyGen is the top of the game.

Congratulations. Thank you. So how long ago did you start? Hey, Jen, tell me a little bit about your journey. You know, I want to understand where your technology is in our conversation, understand where it's going. Sure, definitely. The journey of HeyGen started back in 2020.

Long ago in 2020. Yeah, I think that was the time that we didn't have the term called generative AI. And I was working at very similar technology, enabling using AI technology to enhance the camera screens at Snap.

And I just felt that the technology can continue to evolve and potentially generate something that does not exist in the world, but still feel high realistic. So that's how we started the company. And in fact, I think the funny thing about the mission of a company in the beginning, we call it replacing the camera.

- Replace your camera, yeah. - Because I think if you look back the past 15 years, mobile camera is really the thing that enable a lot of people creating content. But still there's so many people are not being able to create a good content using the camera, either because camera shine

or camera being very expensive in business. And we found that if we can replace the camera, we can remove the barrier for the visual storytelling for the business. And that's how we started the journey. Amazing. Well, you've come a very far way. I think you've got a video to kick us off. Yeah. To give us a sense. Let's run that video so folks can get a sense of your technology. Yep.

Hey everyone, I'm Joshua. And yes, I'm here on stage with you, but I'm also right here as his digital twin. Do we look alike? This is Heijen's avatar technology in action. I can deliver video messages without a camera or production crew. This ranges from video ads to how-to videos to trainings and so much more. I can also speak in over 170 languages and dialects. For example, hola,

Besides just speaking different languages, I can even engage in real-time conversations, whether it's on your website, product, or platform. Imagine what this could mean for your business. Joshua and Peter, I'll let you take it from here. Okay. Nice. Love that. The technology has gotten so extraordinarily powerful.

And yet we're just at the very beginning. So where does this go? What's your vision for where this is going? How will I use these avatars? Yeah, so I think things are happening in two ways, I would say. One thing is that

Every individual will be able to easily visualize their ideas and every business will be able to access more videos creation. Every business will be creating hundreds, you know, 100x more videos in the business communication.

And I think the business content is going to be more engaging compared to the old days of email, images, taxes. That's one. And that's happening very, very fast today. We have seen customers like creating tons of videos, like promoting their business.

I think on the other side, which is also very exciting, I think everybody knows the AI agent role is happening very fast as well. And I think the next interface for us to really interrupt the AI agent may be actually we started from chatting by typing the text.

And then I think it's going to very, very soon we're going to interrupt the agent with voice interface. And then very, very soon we're actually going to see everybody be able to like meet, you know, have a meeting with the agent and in the video interface. And I think that's something that we are seeing, you know, very exciting for the future. Yeah, I mean, I think you've been building these avatars and you can have your avatar go attend a Zoom meeting for you.

Yeah. Right? Yeah. I mean, I love that. I don't know about you guys, but some Zoom meetings, I would much rather have my avatar do the work. I think about this in the future where I can build a really rich, uh, identical agent. In other words, an agent that knows everything about me, knows everything about my company, that if you ask the agent a question and ask me a question, you're going to get the same answer.

Imagine being able to create not one, but like a thousand agents and send them out to a thousand different conferences on your behalf, go and cut deals and bring back, you know, business for you. Is that possible? Is that where we're going? Yeah, I think, you know, yeah.

Not only about that, I think there's another thing is that you will be able to have your agent talk to people in different languages, which was not possible before. And, you know, it is always hard to touch customers in different regions, different languages. And now you can really enable that.

And that's one. I think the second thing is that you can really run your agent meeting in parallel and having your customer interacting with you at the same time, like hundreds, thousands of different sessions happening at the same time. Yeah. And the other thing is, I don't know about you, if you are working on new products or services online,

I know my team will bring me in and say, hey, can we get you to record a video at this particular time? But you put the video up and it's not hitting properly. So we want to change the pricing, want to change the name, want to change all these things. Instead of pulling me into the, you know, into camera or hiring someone to come in and talk about your new longevity clinic or whatever it might be.

you can spin up your, your Hey, John avatars to iterate rapidly as quickly as you want. Right. Yeah. Yeah. I think that's exactly how, how we, you know, um, um, start the company and solve the camera piece problem.

And in fact, even for myself, I already created probably more than 300 different avatars in the past four years because really just testing out the technology. But even today, if you have me to really, like you said, I want to film something in front of camera,

I still feel like, all right, I need to do some preparation for that. I might sometimes need one or two reshoots and I need to pick up a good lighting, things like that. But Avatar really helps us solve that problem for people and being able to like create content anytime. Yeah, I remember last year, Steve Brown, my chief AI officer, a member of our community here,

who will be coming out and joining us in a couple minutes, he created a Peter Bott avatar of myself, right? Which uploaded all of my knowledge, all of my books, all of my podcasts, everything known about me. And I did a podcast with Peter Bott. And I was blown away. Peter Bott was much more eloquent, much more knowledgeable, much smoother. I mean, honestly, I felt very deficient in front of Peter Bott.

It didn't stammer to remember something in its memory. And it was just very compelling. And so there's this incredible power that you have. Yeah, yeah. And that's very exciting. Just getting started. Yeah, it's just getting started. I know you've got a marketing video, which I've asked you to play because I want people to see all the different ways that...

that people can utilize this. I really want this to be one of the technologies everybody plays with this year. It's easy. You just have to make a commitment to actually try it. So please, you know, one of the things I'm looking for you to do is commit to certain things. This is one of them. Let's, let's play that marketing video if we could. Yeah, sure. Today, video is everything. You need speed, creativity, and consistency.

With HeyGen, you can create the most lifelike avatar videos, 10 times faster than traditional production. Scale across 175 different languages, looks, and platforms, all in 4K. Your clients want more, more videos, more engagement, and more results. HeyGen gives you the tools to deliver. Need personalized videos that feel custom? Done.

Launch interactive, engaging campaigns that deliver more for your clients and do it all without blowing the budget. No expensive studios, no costly talent, just beautiful, effective videos that drive results. And here's a secret. We used HeyGen to create me. Join the future of storytelling. Become a HeyGen certified agency partner.

Hey everybody, I hope you're enjoying this episode. Did you know that we're likely to see as many as 10 billion humanoid robots by 2040? And that Brett Adcock, the CEO of Figure, anticipates they'll have robots in our home in the next two to three years. How about Max Hodak's new form of BCI called BioHybrid Interfaces that could offer millions of connections between your neocortex and the cloud?

Then there's Michael Andrej, whose efforts at Eon is focusing on uploading the human connectome to the cloud by 2030. These aren't science fiction scenarios. They're serious efforts underway today. I've distilled the most powerful insights and roadmaps from this year's Abundance 2025 Summit into a comprehensive report that will transform how you see the future.

Get your free copy of the Abundance 2025 Summit Summary at diamandus.com slash breakthroughs. That's diamandus.com slash breakthroughs. I find that extraordinarily impressive. We're going to talk about in a few minutes how each of you can walk away with your own Hey Jen avatar this week, right? It should be something that all of you commit to do, including the teens. I would love you guys to do this.

So let's talk about the two parts. There's the voice and then there's the video persona. And one of the improvements in the video persona is hand movements, backgrounds. I mean, that's relatively recent. Can you speak about that? Yeah. So... And...

If you look at the technology itself, what we're truly building is a human-centric video model. So I think the difference is about human-centric video model versus a general video model is that we need to be able to...

and engaging effect, such as gesture and the way how you move and you move your body, body languages and expression, things like that. And there's a couple of elements out there. Voice is definitely one. I think video is another one. And what we usually require is that we'll take roughly two minutes of training footage from you. Two minutes of video. Yeah, two minutes of video. And then our AI model will try to learn that from the way how you speak.

how your muscle move on your face and your expression, and as well as very importantly, how your body language match your expression. Because everybody's body language is different.

I think the two-minute footage will help us to really personalize your model behavior after that. And that's roughly how it works. And obviously, if we build our own AI agent behind it, you need to input the knowledge base that you have and whatever documents or keynotes, interviews you might have before, and we enable that as an LLM layer behind it. Yeah, it's truly amazing. Let me...

suggest two things to everybody here if you've not done this. And I think they're very important in this age of, you know, photorealistic visual and auditory avatars. The first is you're going to receive a phone call from somebody you think you know. And it may be a FaceTime video from someone you think you know. And they may ask you to do something incredibly crazy like, can you please wire money here? Or can you please send money so I can get out of jail?

I'm serious. I've had lots of friends who've had that happen to their aging parents and so forth. Everybody here, one thing that you should do is choose a word, a password or something that if you have any doubt about the authenticity of the phone call or the video call you're getting, ask for it. Right? Makes sense. It's super easy to do. Have that conversation at dinner.

The second thing is if you're lucky enough to have your parents or grandparents around, do two things. If you can, record their avatar. If you can't do that, sit them down for a couple of hours with your iPhone and ask them for their story, for their history. Record their video, record their voice. And this is for your kids and their grandkids, right?

Please do that. It's extremely valuable. You'll be able to bring them back as your family history. Make sense? Great. So, Joshua, let's talk about what you have enabled for us here at the Google Tech Hub.

Yeah, sure. So we have our booth in the back. We are giving the opportunity for all the attendees to create your own digital twin over there right at the conference. And it's for free. And you're going to be able to record your footage, the training footage for the AI model training, and then going to be able to get your avatar today.

And is this a glimpse to the future of AI video creation? And then, you know, we'll be able to test it out on your HL account. Yeah. So I want you to see how good this is. We had a few members do this last night, and I want to show you one of them that got processed overnight. Rikard, are you here in the audience? Where are you? Okay, Rikard, good to see you, my friend.

I'm waiting for my flying Jetson as soon as I can get it. Let's watch Reichardt, his avatar. So he's not backstage, he's over there. Let's play the video. From now on, I can speak in any language with my own avatar and my own voice. This means that I can communicate with anyone,

Amazing. Let's give it up.

That was extraordinary. And Rekha, how easy was that to do? Four minutes of your time. And then an infinity, 175 languages. That's insane. So this, again, please take this home with you. Play with this. Your whole goal here is to do something different.

you know, at least show it to your kids or your, your, you know, your spouse when you get back. Yeah. It was about 13 years ago. I had my two kids, my two boys. And I remember at that moment in time, I made a decision to double down on my health.

Without question, I wanted to see their kids, their grandkids. And really, you know, during this extraordinary time where the space frontier and AI and crypto is all exploding, it was like the most exciting time ever to be alive. And I made a decision to double down on my health. And I've done that in three key areas. The first is.

is going every year for a fountain upload. You know, fountain is one of the most advanced diagnostics and therapeutics companies. I go there, upload myself, digitize myself about 200 gigabytes of data that the AI system is able to look at to catch disease at inception. You know, look for any cardiovascular, any cancer, neurodegenerative disease, any metabolic disease.

These things are all going on all the time and you can prevent them if you can find them at inception. So super important. So Fountain is one of my keys. I make that available to the CEOs of all my companies, my family members, because health is a new wealth.

But beyond that, we are a collection of 40 trillion human cells and about another 100 trillion bacterial cells, fungi, viri. And we don't understand how that impacts us. And so I use a company and a product called Viome. And Viome has a technology called Metatranscriptomics. It was actually developed recently.

in New Mexico, the same place where the nuclear bomb was developed as a biodefense weapon. And their technology is able to help you understand what's going on in your body to understand which bacteria are producing which proteins. And as a consequence of that, what foods are your superfoods that are best for you to eat? Or what food should you avoid? What's going on in your oral microbiome?

So I use their testing to understand my foods, understand my medicines, understand my supplements. And Viome really helps me understand from a biological and data standpoint what's best for me. And then finally, you know, feeling good, being intelligent, moving well is critical, but looking good. When you look yourself in the mirror,

Saying, you know, I feel great about life is so important, right? And so a product I use every day, twice a day is called One Skin, developed by four incredible PhD women that found this 10 amino acid peptide that's able to zap senile cells in your skin and really help you stay youthful in your look and appearance.

So for me, these are three technologies I love and I use all the time. I'll have my team link to those in the show notes down below. Please check them out. Anyway, I hope you enjoyed that. Now back to the episode. What's most exciting for you in the future here? I'm very excited about...

the mission that we have enabling visual storytelling for all. I think today when we talk about video, video is the main format for people to consume on the internet.

But still, I think less than 1% of the audience, less than 1% of our people are able to actually create great videos. And I think camera is the first problem, and we are actively solving that. And also with a lot of agent technology from the AI model advancement, I think video editing is going to be solved with AI very soon as well.

And think about the world like this. Every time you want to send out an email, instead, you can actually turn that email into a video and send it to your whole company and give the video format to the customer so they can watch it on their own. I think those are the features that we are very excited about. I agree. I know that some people are...

concerned or scared about their identity, their image being stolen and being used to promote something they're not involved in. But you have protections against that, don't you? Yeah, we do. Definitely. So first of all, we take trust and safety seriously on the platform.

Every single avatar created on HeyGem will require the first-party consent where you have to give a video consent format so that we can verify that. So you have to go on video and say, this is Peter Diamandis, and I give consent to create this avatar, et cetera. Yeah, and we are proud to say that in the entire history of HeyGem, we never compromised on that consent. Amazing. And no matter who is coming to create avatar on HeyGem, we always require that.

And meanwhile, we also have a moderation system behind it just serve as a safeguard. There's a lot of things not allowed on the hate gem platform, things like hate speech, fraudulent, and any political campaign. Today, we have a strategic moderation team in the back, combined with AI and human moderator to help to make sure that no

the technology has been at good use. Yeah, I'm so, so impressed. Last year, I tried to get a video acknowledgement from Aristotle to get that video made and he never responded. But thank you for allowing it to happen. Last question here. Let's discuss the difference between integration of an AI agent and my digital clone. Yeah, the way how I look at it is in two ways, right?

when we talk about AI agent today, we're mainly talking about there's a AI assistant that can help us to do something, not only, you know, at the first action, but also a sequence of action, right? And

And I think when we talk about digital twin, it's the enablement about myself. It's like, let's say, Peter, you want to have three meetings happening at the same time. And it's kind of like agentize yourself so that other people can get access to you.

And I think there's a combination overlap of these two roles where, you know, okay, the question would be, how does a human interact with agent in the future? I think the most natural way to interact with agent would be a video format, would be a video call, would be a video chat. So I think, you know, you can kind of like

attach your digital twin in your AI agent so that other people can interact with it. And meanwhile, you can also get access to other people's agent in a video chat format. I don't know if you can feel the speed, the future speeding up in this very moment, right? So when you hire 20 marketing agents online,

AI agents we've been talking about, those AI agents will show up as you'll give them names. They'll show up on Slack. You can Slack them. You can email them. You can hop on a video conference with them. You can have a Zoom call with your marketing force of 20 AI agents and they will look unique. You'll have conversations with them. This is the future which is available when?

Now. Now. All right. The following segment is a presentation from Steve Brown on exactly how to use these AI avatars in your business and life. One thing I wanted to ask you is like, how many of you are at this point confused about the difference between avatar and agent? A little bit. Okay. I'm going to give you a little shortcut to figuring it out. Avatar is AV, audio visual. Agent is agency. And if you're going to give

agency to your agent, it needs to know you and you need to trust it. So how do you make that happen? It's one thing to have that emotional connection because it looks fantastic and it acts like you, it looks like you visually, but we need to train the brain.

So that's what I've been working on now. This past 12 months have been the 12 months of avatars, and now all we're going to talk about is agents from here on out. So first of all, how do you train your agent? So you're going to all go and make a great avatar downstairs, and I did a great one last night, so I'm not going to show you. We already saw Ricard's. But...

I actually built agents to help you train your agent. So asking you all about your life story, asking you about your values, asking you about your purpose, asking you about your personal anecdotes and all these things so you can talk to your agent forever to train your story about...

so that your actual legacy agent or clone will know all about you. So these are means by which, in just interacting, a voice conversation, you build the corpus of data that makes that agent actually meaningful to you. So you have a lot of books and you have a lot of things to upload, but a lot of people don't. You have your Facebook and your LinkedIn and your social media, but what's really important is your values, what you care about, your personal story, the depth, right?

And that might be hundreds of pages. If you were to write a biography, that's hundreds of pages. So anyways, I'm building agents that help you actually train your agent. So now, beyond that, what can you use your agent for? Another thing I've been using agents for, this is a medical agent, and we'll hope that he works. He's actually connected to my medical records. So just talking a little bit about your story. So knowing your story for future generations. So

My story has been one of entrepreneur, and then I meet this fabulous entrepreneur and entrepreneur mentor, Peter, which has stimulated even more ideas to go start new ventures. But in the last couple of months, actually, my own story took just a quite radical turn. It started off with the fires in L.A. two months ago, which happened to burn our house down. For the second time.

Well, the first one was a rental, so it didn't really count. But this was like, you know, the dream home. Counted for the owners. And yeah, I mean, well, anyways, yeah, we lost our stuff before, but this was a much more traumatic event. But it actually turned out to be something else. Lee Stein, who's here, said, Steve, you need to look for the gift. If you don't look for the gift, you're not going to find the gift. So we were displaced from Los Angeles and...

My wife and I and our 170-pound Great Dane, we ended up staying with friends in Palm Springs. And I ended up in the emergency room because of a steak dinner pain, thinking that I was obstructed. I went to the ER and got a CAT scan. And the ER doctor is like, hey, there's something going on here. I got admitted to the hospital. And for many, many more tests,

And biopsies and everything. And I ended up getting diagnosed with multiple myeloma, blood cancer. And while I'm in the hospital for eight days... This is about six weeks ago? Yeah. So I'm in the hospital and I'm like, of course, I'm just talking to AI the whole time. What's going on here? What does all this mean? And I had just had, not Fountain Life, I'm sure Fountain Life would have figured this out, but I had...

gone to the doctors, and I had had everything under the sun tested, I thought, to specialists. And they were like, this is a fishing expedition, we can't find anything.

You know, maybe you're just stressed out and overworked. I'd been working very hard for a very long time. But anyways, you couldn't find anything. So I end up with a young ER doctor in Palm Springs who didn't know me, didn't have any really any background in me at all, not a specialist. And he ends up piecing these things together. So I went back to my while I was in the hospital. I went back

to my medical record, all the tests that I'd been done. This is his story over the last few months. Yeah. And I said, when would AI have caught this? I took all the data, not from Palm Springs, but all the prior data. And I said, is there anything in there that might've helped, um,

find this earlier. I was extremely lucky in that we found it very early before much damage had been done. And I'm actually on chemo right now and immunotherapies. And, you know, I think I'm going to be OK. But let's talk to Hippocrates and say, hey, Hippocrates, I was not feeling well for a while and I lost a bunch of weight. So I went to my specialist, two different specialists, and I had everything under the sun tested and

and they really couldn't find anything, and they thought maybe I'm just stressed out or overworked or depressed, kind of the default answer that doctors sometimes give if they don't know what's going on. I wondered if you could look at my chart and tell me what you see. I see your concern, Steve.

Looking at your chart, I noticed several significant findings that warrant further investigation. Your low immunoglobulins, IgA and IgM, low gamma globulins, mild anemia, elevated ferritin, and unexplained weight loss together suggest a possible plasma cell disorder. I recommend discussing with your physician about bone marrow evaluation to rule out multiple myeloma or related conditions. So first off the bat, all right,

So that's pretty extraordinary because that's actually what the diagnosis ended up being. But my specialist didn't find it, even though they were supposedly testing for everything. But a generalist...

put the pieces together and found it, and you found it even earlier. I'm trying to understand how that could be. Steve, I'm not surprised. Your pattern of hypergammaglobulin-mea with weight loss, fatigue, and anemia creates a classic constellation suggesting plasma cell disgracia. Specialists sometimes focus narrowly on their area while missing broader patterns.

The immunoglobulin abnormalities were the key clue that needed integration with your symptoms. This pattern recognition is fundamental to Hippocratic medicine, seeing the whole person, not just isolated test results.

So what's the test that I should have gotten that might have figured this out? Steve, you should have received serum-free light chain assay testing, which measures kappa and lambda light chains, crucial for detecting plasma cell disorders even when immunofixation is negative. A bone marrow biopsy would have been definitive. Also, a 24-hour urine protein electrophoresis could have detected Bence-Jones proteins. These tests are standard when plasma cell disorders are suspected.

So one last thing here. So actually I'm on chemotherapy and I'm getting this new immunotherapy called daratumumab. I don't know if you guys can figure that out, what that is on the voice transcription. So anyways, after two weeks of chemo, my light chains went down by 50%, over 50%. So what do you think of that? Steve, that's excellent progress.

So one of the things that I'm working on is that

All of our specialists get narrow and narrow and more narrow in their silos because there's so much medical knowledge. There's no way that anybody can know everything. So you end up getting siloed. The AI is not siloed. The AI can understand everything. It can absorb everything. So, you know, you showed a video at the beginning of Minecraft with the thousand agents running around and collecting things and collaborating and doing things. And you mentioned, like, can I have 100...

of my agents out there going to Zoom meetings and doing business deals. One of the things I'm working on is, well, why not have 100 doctors with 100 different points of view and 100 different ways of dipping into this messy, fragmented data of our health that is very, very hard to infer what's going on, and everybody has a different point of view. So this is just one agent, and he's kind of trained to be kind of a God agent, you know, be Hippocrates, know everything.

But what's actually really interesting is when your agents have different points of view because those different points of view are different search strategies into this vast knowledge base that's captured in the LLM. So this is an example of how agents are not just going to do some of your work for you, not just represent you to future generations if you really get your story in there, but your agents...

might be saving your life. This is really powerful stuff. And what Joshua and Heijen are doing just at the top of the game here is giving that human interface because our ability to trust our agents and our ability to connect with our agents and to tell our stories and interact with them, that empathetic connection actually is very, very kind of dependent on

on that feeling of connection that I'm really dealing with an entity that I know, that I trust, that cares about me. So that's one part of the equation and it's extraordinarily important. And HeyGen is doing just an extraordinary job with it. The other side of it is the brain and what can you train it to do and what can it do for you? And that's what I'm working on next, Peter.

Amazing. Guys, give it up for Steve and Joshua. Guys, come on out here. If you enjoyed this episode, I'm going to be releasing all of the talks, all the keynotes from the Abundance Summit exclusively on exponentialmastery.com. You can get on-demand access there. Go to exponentialmastery.com.