#398 – Mark Zuckerberg: First Interview in the Metaverse
01:12:04 Share

The following is conversation with mark sock berg inside the metaverse. Mark and I are hundreds of miles apart from each other in physical space, but IT feels like we in the same room because we appear to each other as foto. Realistic code avatars in three d with spatial audio this technology is incredible, and I think is the future of how human beings connect to each other in the deeply meaningful way on the internet.

These avatars can capture many of the nuances of facial expressions that we use. We humans used to communicate emotion to each other. Now I just need to work on upgrading my emotion expressing capabilities of the underlying human.

And now a quick you second mention of the sponsor, check them out in the description is the best way to support the spot as we got an element for delicious electricities inside track of a biological data, eight sleep for delicious, perhaps a one for delicious health, and next week for probably delicious. But I haven't a tasted IT business measurement software, choose lize in my friends. Also, if you want to work with our team, world was hiring last three, and outcomes as hiring.

And now onto the full ad reads, as always, no ads in the middle. I try to make this interesting, but if you must keep them friends, please do check out the sponsors. I enjoy their stuff.

Maybe you will too. You never know until you try. This episode is brought to you by the thing I consume many times a day.

The thing i'm consuming currently, as I am speaking the words on speaking element, a lecture, drink, spell, L M N T. I do a low cb diet, sometimes a very low cb diet, and I usually eat only once a day. So that's fasting and low cards.

And for that you have to get your electoral ze right. Electrodes include sodium pattani, magnesium. The proportions are really important. The deliciousness of the proportions will combine with water is also important.

IT, turns out, is also just important to consume water and making water a little bit more delicious, which is what everyone does, is a win, win. That's what we call on the business of win. win.

Friends, get a sample pack for free with any purchase. Try IT join me and try IT a drink element to come flashes. Get a sample pack for free with any purchase at drink element to slash leaks.

This shows also about to buy inside tracker, a service I used to listen to my body through the power of data science. So somebody comes signals through blood tests to give information about whatever gone on my body. And that data is used for machine learning algorithms to make suggestions about what you should do in your life.

Now, in this conversation about the metaverse where mark I are, incredibly, in a whole other plane of existence, we have teleported into this place while we miles apart from each other. In physical space, in the virtual space, were right there together, shrouded in darkness. I wonder if those entities, they're driven by the content of our minds, if they have biological lake signals, if they need to make life style diet decisions, I wonder one day we should be able to ask them.

Actually, that's one of the things that matter announced, which is really cool, which are the A I. personalities. And I do think for just from a technical perspective, of super exciting and difficult to incapable a particular personality like snoop dog in an A I agent where IT reflects all the corks, the warehouses, but also the beauty of the character of that particularly human being.

I love conversation, that task, doing that thing technically super excitement and think super difficult. So glad they're taking IT on anyway, get special savings for a limited time when you go to inside track and that come flash lex, this epsom is also brought you by asleep, and it's part three matches. You can cool down or heat up the two sizes.

But separately, this actually makes me think of that mean what the two kids are scared in the corner and they're looking at a bunny and. I am those two scared kids in the corner thinking about the bunny or the bony represents the people that want their bed heat IT up. I don't know who does this, but I want to meet you, maybe even an asleep, like in the antartica or somewhere in northern canada and bays's don't have heat and maybe you're out campaign.

I can't possibly construct the situation which you reduce he and only half kitching, of course. Anyway, I prefer to cool down the bed and is an incredible experience, a cold bed with a warm blanket. It's, it's another place.

I tell the poor two in my mind when I go to the nap. It's not a universe. It's a universe.

And there I find a deep peace from the chaos of life. And I returned refresh, ready to take the chaos on once again. Anyway, check IT out and get special savings when you go to asleep.

That consulate legs. This shows also batty by ag. one. Another delicious thing I consume every day I just drink IT like an hour ago is delicious, is Green and makes me feel like poppy.

I grew up, I don't know how many years ago washing poppy. There was a stretch of time where I wanted to be pop. I, I think I admired pop I for being strong.

Maybe because I wanted to be strong. I would thought a man is supposed to be strong. And so if I just eat my spinach like pop I did, I will be strong.

So hence the color Green forever associated with the, with strength. And i'm also playing the other now. And that's one of the things you can increase its strength, texture, ity will power, intelligence, all that of stuff, stringent, still has that sexy sound to IT.

And so every time I played the abble actually, and I drink A G one and IT just, uh, makes me feel stronger and Better. And like, I got my nutritional bases covered. Anyway, they'll give you one month supply a fiscal.

When you sign up a drink age, you want to come slash lex. This shows also butty by that sweet and all in one cloud business manageress system. I still holding me a little bit of a flame that turns hopefully into a fire, which is a desire to contribute by building stuff with its working at a company or starting a company.

And i'm very coders and of how difficult is to run the company successfully, whether you are looking at a huge company like matter or a tiny start up, all of IT and actually, yes, all of IT is a source of chAllenge in complex citizen and fear and anxiety and uncertainty, all of that ultimately the comradery of the people working together as a deeply fulfilling, deeply meaningful pursuit, especially when there's a big vision that you're reaching for. I love people, ad companies, working anything, chasing that big, impossible vision. Actually, the metaverse is once such thing.

Just the technical complexity of everything involved is just really incredible, from the headset software in the hardware, the cameras, the scanning for the average or at all of IT, just incredible. Anyway, if you want to figure out how to run a business, smaller, large, successful ly should be using the bus tools like that. Sweet, you can start now with no payment or interest for six months.

Got a nets week console's the access there. One of a kind financing program that's net sweet dog com slash legs. This is elect treatment back guests. And now, dear friends, he is mark soaker G.

This is so great. Writing change, put the .

light anywhere.

And doesn't feel awkward to be really close to you.

No, he does. I actually move, move you back a few feet before you got into that. You were like, right here.

I don't know if people can see this, but this is incredible. The realism here is just incredible. Where am I? Where are you, mark? where? where? Where are we?

You're an Austin, right?

Not mean, in this place we're shrouded by darkness with alter real alister face and just feels like win the same room. This is really the most incredible thing of ever in and started being a personal space. I mean, we have done to justice before.

yeah. Now I was also commenting to the team before that even that I feel like we've choke each other from further distances, then IT feels like we are right now.

I mean, this is just really incredible. I don't know how to describe IT with words. IT really feels like IT feels like going in the same room yeah feels like the future.

This is truly, truly incredible. I just wanna take IT in. I'm still getting used to is like it's you, it's really you, but you're not here with me, right? You're there wearing a headset, not wearing it's it's really, really incredible.

So what are he described? What IT takes currently for us to appear? So for alister each other .

yeah so in for for background. We both did these scans for this research project that that we haven't met a called kodak avatars. And the idea is that instead of actually, instead of our avatars being cartoony, and instead of actually transmitting a video, what he does is we've stand ourselves and a lot of different expressions, and we ve built a computer model of sort of each of our faces and and bodies and the different expressions that we make, and collapse that into A A codec that then, when you have the headset on your head, IT can IT IT seize your face, sees your expression, and IT can basically send um an encoded version of what you're supposed to look like over the wire. So in addition to being photorealistic, it's also actually much more band with efficient than transmitting A A full video or specially three d immersive video of a whole seen like this.

And he captures everything like the flaws, like to me, the thirties of the human face are given the flaws. That's like, that's all amazing. IT makes you IT makes you so much more immersive. IT makes you realized that, like, perfection isn't the thing that leads to immersion. It's like the little son of flaws, like france and like variations in color and just all the roses yeah a and just a different like the corners of the eyes, like what your eyes do when you smile.

all that kind of stuff. Yeah eyes. There are a huge far of IT. Yes, there's all the studies is that most of communication, even when people are speaking, is not actually the words that they're saying, right? It's it's kind of the expression and all that.

So and we try to capture that with the kind of classical um expressive avatar AR system that we have, that the kind of more cartoon design when you can you can kind of put those kind of expressions on those faces as well. But there's obviously certain realism that comes with delivering kind of this photo realistic experience that I I just think it's really magical. And this gets to kind of the core of what the vision around virtual and augmented reality is like, delivering a sense of presence as if you are there together, no matter where you actually are in the world. And I mean, this this a experience, I think a good evidence of that where it's like coming where in two completely different states, half fare across the country and IT just like looks like you're just sitting right in front of me. It's it's pretty ild.

Yeah, yeah, I can. It's almost getting emotional like he feels like us are totally it's fundamental new experience like for me to have this kind of conversation with loud ones that would just change everything, maybe just to elaborate. So I went to puzzle an went to the whole scanning procedure, which has so much incredible uh, technology. So software and hardware going on um but IT is a likely process. So what's your vision for the future of this uh, in terms of making this more accessible to people?

You know IT starts off with a small number of people doing these are very detailed scans, right, which is this that the version that you did in that I did in, you know before, there are a lot of people who we've done this kind of a sand for. We probably need to have over collect expressions um when we're doing the scanning because we haven't figured out how much we can reduce that down to a really streamline process. I'm an extrapolate from the the the scans that already been done.

But the goal and we have a project that's working on this already is IT just to do a very quick scan with your cell phone or you just take your phone can waive IT in front of your face for a couple of minutes um you know say a few sentences, make a bunch of expressions, but overall， have the whole process just be two to three minutes and then produce something that of the quality of what we have right now. So I don't know that one of the big chAllenges that remains and right now, we have the ability to do the scans of you, you have hours to sit for one. And with today's technology and you're using A A meta headset that exists, it's a product that's kind of for sale now you can drive this with that.

Um but the production of of these scans and in a very efficient way is one of the last pieces that we still need to really nail. And then obviously, there's all the experiences around IT right now. We're of sitting in a dark room, which um is in a familiar for your podcast.

But I think part of the division for this over time is um is you know not just having this speed like a video call. I mean, that's fine. It's it's cool IT feels like it's mercie. But um you can you can do a video call on your phone. The thing that you can do in the metaverse that is different from what you can do on the phone is like doing stuff where you're physically there together and and participating a thing together.

And we could play games like this um we could have meetings are like this in in in the future once you mix um once you get mixed reality and augmented reality, we could have code habitants like this and go to a meeting and have some people physically there and have some people show up in this photo realistic form uh superimposed on the on the physical environment. Stuff like that is going to be super powerful. So we got to still build out all those kind of applications and the use cases around IT. But I don't know think it's going to be a prety wild next few years around us.

I mean, I just i'm actually almost at a loss of words. This is just so incredible. This is truly incredible. I I hope that people like watching this can get a glimpse of like how incredible IT is IT really feels like going in same room like there is that um I guess that is on canny valley that seems to have been crossed here like IT .

looks like you yeah like the you there are still a bunch of tuning that I think we want to do where different people a mote to different extents, right? So I think one of the big questions is, you know like when you smile, how wide is your smile and how wide do you want your smile to be?

Um and I think getting that to be tuned on a car person basis is um is going to be one of the things that we that we're going to need to figure out um even it's like to what extent you want to give people control over that. Um some people might try to might prefer a version of themselves that's more emotive than their avatar AR than their actual faces. So for example, you know I I always get a lot of um critique and shit for um for for having like a relatively stiff expression。 But you know I mean, I might I might feel pretty happy, but just make a pretty small smile.

I am maybe you know for me I would I it's actually you know I think i'd wanted have my habitual really able to Better express um like how i'm feeling than than what how I can do physically. So I think there's a question about how you want to that. But overall, yeah and we want to start from baseline of capturing how people actually emoting express themselves and and I think that the initial version of this is has been pretty impressive. Like you said, um I do think we're kind of beyond the the on Candy valley here where IT IT does not feel like you IT doesn't feel doesn't feel weird or anything like that.

I mean, that's gonna the meme that the two most monitoring people are in in a metaverse together. But I think that actually makes a more difficult. The the the amazing thing here is that the subtle ties of the expression of the eyes, you know, people say I monitoring and emotion is, but i'm not.

Is just this maybe my expression? Emotion is more subber, usually like with the eyes. And that's one of things i've noticed, is just how expressed of the subtle movement of the corners of the eyes are in terms of displaying happiness, are bored and more all that kind of stuff.

I am curious to see, just we've never done of this before. I've never done a podcasts as is one of these cottage avatar ars. Um I am curious see what what how what people think of IT because you know one of the issues that we've had in some of the VR and and mixed work is IT tends to feel a lot more profound when you're in IT than the two d videos s capturing the experience. So I think that this one because it's photorealistic um may look kind of as amazing in tube for people watching IT as IT as IT feels nothing to be in IT. But we've certainly had this this um this issue where a lot of the other things just it's like you feel the sense of emersion when you're in IT that that doesn't quite translate to a two d screen but .

I don't curious to see to see what people think i'm good to see people could see that um like my heart is actually beating fast now has a super interesting like the that such intimacy y of conversation will be achieve remotely this thing that you know I don't do remote podcast for this reason and this is like breaks all that this seems like just an incredible transition to something else, to the different kind of communication.

Breaks all barriers like geographic physical barriers. Um what you mention, what you have a sense of timely in turns of human difficult things have to be solved. To make this more accessible to like scanning with a smart phone.

Yeah I mean, I think we will probably roll this out progressively over time. It's not to be rolled out one day. Everyone has a coat avatar. Um we want to get more people scanned into the system and then we want to start a integrating IT into each one of our apps, right, making IT so that you know, I think that for a lot of the workers yle things productivity, I think that this is going to make a ton of sense.

And a lot of game environment saying this could be fine, but games can have their own style right, where you almost want to fit more with the aesthetic tic style of the of the game. Um but I think for doing meetings and one of the things that we get a lot of feedback on workrooms, where the people are pretty blown away by the experience of this feeling that you can like be remote but feel like you're physically there around a table with people, but then we get some feedback to people. Have a hard time with the fact that the avatar ars are so expressive and and and don't feel as realistic and the environment.

So I think something like this um could make a very big difference for those remote meetings and especially with quest three coming out, which is going to be the first mainstream mixed reality product, right where you are really taking digital um you know expressions either a person or or objects and overlying them on the physical world. Um I think the ability to do kind of remote meetings and and things like that, where you're like just remote hang sessions with friends, and I think that can be very exciting. So rolling IT out over the next over the next few years, it's not ready to be like a kind of mainstream product yet. But um we just want will keep tuning and keep getting more scans and rents keep know kind of rolling IT out into more of the features. But you I mean definitely in the next in the next few years, you'll be seeing a bunch of more experiences like this.

Yeah, we love to see some celebrities sand and some non celebrities. I just just more people to experience this. I would love to see that this is something that, I mean, i'm my mind, but i'm literalist work because it's very difficult to to just convey credible.

This is how, how, like, how I feel the emotion, how I feel the presence, how I feel like the subtleties of the emotion. Uh, in terms of like work meetings or any kind of in terms of podcast, this is like, this is awesome. I don't even need your .

arms or legs got to get that. I mean, that's its own chAllenge. And in part of the question is also, so you have the scan then IT takes a certain of compute to go drive that both for the sensors on the headset and um and then rendering IT. So one of the things that were working through is what is the level of fidelity that is optimal ready? You can do the full body and in kind of a codec and that can be quite intensive. But um for one of the things that we're thinking about is like maybe you can kind of stitches somewhat lower fidelity version of your body but still still have the main of the major movements um but but but your face is really the thing that we have the most resolution on right in terms of being able to read and express emotions and like you said, if you move your you know eyebrow is like a millimeter, I mean that really changes the expression and what your you're moving yes I mean moving your your arms like of an inch probably doesn't matter quite as much. So so yeah so I think that will we do want to get all of that into here and and that'll be some of the work over the experience as well.

So you mentioned quest three that's coming out. I've god just to try that to that awesome. So the huge you pull off the mixture is not just virtual mixed reality.

Yeah I think it's to it's going to the first mainstream um mixed device. Obviously we ship quest pro um last year, but IT was fifty hundred dollars um and all part of i'm super proud of as you know, we try to innovate not just on pushing the state of the art, delivered new capabilities, but making IT so I can be available to everyone. And we have this and it's coming out.

It's five hundred dollars and um in in some ways I think the mixture is actually Better. Inquest three than IT was um then then what we're using right now in quest row. So I am really proud of the team for being able to deliver that kind of an innovation and get out.

Now some of this is just um the software you tune over time and and get to be Better. Part of IT is you put together a product and you figure out what are the bottle next in terms of making you a good experience. So we got the resolution for the mixed reality cameras and centers to be multiple times is Better in quest three.

And we just figure that, that, that made a very big difference when we sell the experience that we were able to put together for request pro. Um and part of IT is also that no calm just came out with their next generation chip set for for V R and M R. And that we worked with them on on a kind of custom version of IT. Um but that was available this year for quest three and IT wasn't available in quest pro.

So in a way, i'm quest three, even though it's not the the pro product um actually has a stronger chips that and then the pro line add a third the cost so um so i'm really excited to get the some people's hands um IT IT does all of you our stuff that that quest to and the others are done to IT doesn't Better because the display is Better um and and the chip is is Better so you'll get Better graphics. It's forty percent thinner. So it's um such more comfortable uh as well.

But but the mr is really the big capability shift. And and part of what's exciting about the whole space right now is you know this season like smart phones where the companies put out a new smart phone every year and you can almost barely tell the difference between that the one year before IT. Now for this, each time we put out a new headset, IT has a major new capability.

And in the big one now um is is mixed reality, the ability to basically take digital representations of people are or objects and and superimpose them on the world and basically, you know I mean one version of this is you're gonna a kind of have these augments or or holograms and and experiences that you can to bring into living room or a meeting space or office. Um another thing that I just think is gonna a much kind of simper innovation is that there a lot of V R experiences today, they don't need to be fully immersive. And in if you're playing a shooter game or you're doing a fitness experience, sometimes people get worried about swinging their arms around like I going to hit a line or or something, you know it's and and you I going to run into something.

So having that a mixed reality actually just a lot more comfortable for people, right? You you kind of still get the emerge in three d experience and you you can have experience that wouldn't be possible in the physical world alone. But by being anchored to and being able to see the physical world around you, it's like IT just feels so much safer and more secure.

And I think a lot of people are really going to enjoy that too. So yeah I am really excited to see how people use IT. But yeah great three coming out um later this .

fall yeah and I get to experienced with other people sitting around and there's a lot of furniture and so you get to see that furniture and get to see those people and get to see those people like enjoy the ridiculous of you like swing your arms. I mean persons will be their friends of yours even if they make fun of you. There's a lot of love behind that. And you get the experience that it's a really fundamental different experience in just pure of you. R would like with .

zombie ing out of walls and shooting at you, you hide behind your real couch in order duck fire.

it's incredible integration. Da stuff like in a room with no windows, you can add windows to IT and you can look outside as the zombies run towards you but like it's still nice view outside yeah, it's really and so that's pulled off by having cameras on the outside of the had that do the past through uh in that technology is incredible to do that ah yeah had said.

yeah, it's not the cameras. You basically need to you need multiple cameras to capture the different angles and and sort of the three dimensional space and then it's a pretty complex compute problem and A I problem to matter that to your perspective, right?

Because the cameras aren't exactly with your eyes are because no two people's yes or you know going to be an exactly the same place you can need to to get Better to to line up um and then do that basically in real time and then generate something that looks that kind of feels natural um and then superimpose whatever digital objects you want to put there. So it's yeah it's a very interesting technical chAllenges and will continue tuning this for for the years to come as well. But but i'm pretty excited to to get this sound because I think quest three is going to be the first device like this with that millions of people are gna get that mixed reality in. It's only when you have millions of people using something that you're start getting the whole development community really starting to experiment and build stuff because now they're going to be people actually use IT. Um something will get if we got some of that fly will going with quest pro, but I think it'll really get accelerated once quest three gets out there.

Plus and tracking without the need of a control. So this can the cameras aren't doing the pass through uh of the entire of physical reality around you is also tracking the details of your hands in order to use that for a gesture recognition this kind of stop yeah .

we're been able to get away further on hand recognition in a shorter period time that I expected. So that's been pretty cool. I don't know did you see the the demo um experience that we built around um pia like yeah the piano learning .

to take piano ble your basically playing piano on a table and is that without in your controller and like how well in matches physical reality with no latency and is not as tracking your hands with no latency and stacking all the people around you with no lencs.

Integrating physical reality and digital reality, obviously, that connects exactly this uh court avatar which is in parallel allows us to have auto realistic copies of ourselves in this mixed reality. It's uh they so think this all converging toys like an incredible digital experience in the metaverse, to me, obvious. I love the intimacy y of conversation. So even this is awesome. But they have other old ideas of what this unlocks of, like something like a garvet on logs in terms of applications, in terms of things were able to do well.

There's what you can do, advertise overall in terms of superimposing digital objects on the physical world um and then there's kind of psychologically what is having photorealistic do. Um yes. So I I think we're moving towards the world where you we're going to have something that looks like Normal glasses, where you can just see, you see the physical world, you will see holograms. And in that world, I think that they're gonna be, you know, not too far off.

Maybe, you know, by the end of the decade, we'll be living in a world where the, can I ask many holograms when you walk into a room is there are physical objects and IT IT really raises the interesting question about what are um about, you know, a lot of people have this phrase where they they they call the physical world the real world. And I kind of think increasingly then, the physical world is super important. But I actually think the the real world is the combination of the physical world and the digital world coming together.

But until this technology, they were sort of separate, right? I like you access the digital world through a screen, right? And no, maybe it's a small screen that you Carry around. There is a bigger screen, you sit down your desk, strap in for a long session. But um but they're kind of fundamentally divorced and disconnected.

And part of what this technology is going to do is bring those together into a single coherent experience of what the modern real world is, which is it's got to be physical because that we're physical beings. So the physical world is is always going to be super important. But but increasingly, I think a lot of the things that we kind of think of um can be digital hologram.

Some in any screen that you have can be hologram in any media um in any book art um you know can basically be just as effective as a hologram, as a physical object, any game they are playing, a board game or um or any kind of physically game cards um pink punk things like that there there often a lot Better as holograms because you could just snap your fingers and stand tied them and have them show up. You know it's like you have a pink punk table show up in your living room, but then you can snap your fingers and I would be gone so that's super powerful. Um so I think that it's it's actually amazing thought experiment of like how many physical things we have today that could actually be Better as interactive holograms.

But then beyond that, I think the the most important saying obviously is people. So the ability to you have these mixed hangouts, whether they're social or meetings where you show up to a conference room you're wearing glasses or a heads set in the very near term. But you hopefully by over the next five years, glasses or so and um and you know you're they're physically some people are there physically um but other people are just there as holograms and IT feels like it's them um who are right there.

And and also, by the way, another thing that I think it's thing about how to blend together the digital and physical al worlds in this way is were also going to be to somebody um A S as well. So I think you'll also have meetings in the future where you're basically, you know maybe you're sitting there physically and then you have a couple of other people who are there is holograms and then you have like bob the A I who is an engineer on your team who is helping with things and he can now be in bodied as a um you know as as A A realistic advice or as well and just join the meeting um in that way. So I think that that that's can be pretty compelling um as well.

So then okay, so what can you do with photo realistic gavitt compared to kind of the more expensive ones that we have today? Well, I think a lot of this actually comes down to acceptance of the technology um and because all the stuff that we're doing, I mean the the motion of your eyebrows, the motion of your eyes, the cheeks and and all, there's actually no reason why you couldn't do that on an expressive avatar too. I mean, I wouldn't look exactly like you, but you can make a cartoon version of yourself and still have been um almost as expressive.

But I I do think that there's this bridge between the current state of most interactions in the physical world and where we're getting in the future with this kind of hybrid physical and digital world, where I think it's can be a lot of easier for people to kind of take some these experiences seriously with the photo realistic avatar as to start. And then i'm actually really curious to see where IT goes long or term. I could see a world where people stick to the hot realistic and maybe they modify them to make them a little bit more interesting.

But maybe fundamentally we like photorealistic things um but I can also see a world that once people get used to the photo the photo realistic avatars and they get used to these experiences that uh I actually think that there could be a world where people actually prefer um being able to express themselves in kind of none you know ways aren't so tied to their physical reality. And so that's one of things i'm really curious about. And I know in a bunch of our internal experiments on this, one of the things that has I thought was psychologically pretty interesting as people have no issues blending photorealistic stuff and not so you know, we can have you know, for the specific scene that we're in now.

We will happen to sort of being in a dark room. Um I think part of the atheistic sion, I think, was based on the way you like to do your podcast. But we've done experiences like this um where you have like a cartoony background but photorealistic people who are talking to and we seem to like people just seem to just think that that is completely Normal, right? IT doesn't bother you, doesn't feel like it's weird.

Another thing that that we have experienced with this is basically you have a photorealistic avatar they are talking to and then right next to them, you have an expressive kind of cartoon avatar that actually is pretty Normal too. But it's it's like it's not that weird, right to basically been interacting with with different people in different modes like that. So i'm not sure I think I will be an interesting question to what extent these photorealistic avatar ars are like A A key part of just transitioning from being comfortable in the physical world to this kind of new modern real world that that kind includes both the digital and physical? Or if this is like the long term way that IT stays, that, that I I think they're going to be uses for both the expressive and the photorealistic over time. I just don't know what the baLance is going to be.

Yeah, it's a really good, interesting philosopher question. But to me, in the short term, the photo realistic is amazing to where I would prefer, like you said, the worker, but like on a beach with the beer to to see a body of mine remotely on a chair next to me drinking a beer. I mean, that as realistic as possible is an incredible experience.

So I don't want any fake cats on here. I don't want a just chilling with with a friend. Drink beer looking at the ocean while not being in the same place together.

I mean that yeah that experience is just a fundamentally uh it's just a high quality experience of friendship whatever we seek in friendship. He seems to be present there in the same kind of realism seeing right now. This is total game changer. So to me this is I can see myself sticking with this for a long time.

Yeah and I mean, it's also its novel and it's also a technological feet, right? It's like being able to police office like it's it's like a pretty impressive and I I think to some degree, it's kind of like awesome experience.

Yeah um but i'm already sorry to drop i'm already forgetting that you're not real like this really so novel it's a deep .

Sophia question I but hear some of I put the song this morning .

and like A A psychic OK this .

my hair is little shoulder in this then my physical hair is right now I prime to go get a haircut and like and I actually I did happen to have this morning but but if I hadn't you know I could still have this hot realistic habita that is that is more cleanly shave and for right even if i'm you know a few days in um physically so I do you think that they're gna start to be these subbed questions that sep in where the the evita is realistic um in in the sense of this is kind of what you looked like at the time of capture, but it's not necessarily temporarily accurate to exactly what you look like at this moment. And they're going to end up being um a bunch of questions that come from that over time that I think I could be fascinating too.

You mean just like the nature of identic of who we are. Are we the year people? You know how people do like like summer beach body.

Would people be for the school? We'll try to lose some way and look their best and sexist with the my hair like that IT does a IT does raise the question of, you know, if a lot of people interacting with the digital version of ourselves, who are we really? Are we the the entity driving the ata or with the avatar?

Well, I mean, I think our physical bodies also fluctuate. Change your time too. So there's a similar question of like which version of that are we?

There's there's like the I mean, it's interesting identity question because alright, it's like I don't know. It's like weight fluctuates or are things like that. It's like I think most people don't tend to think of themselves as the a lot of I is an interesting psychological question.

Some maybe some people and maybe lot of people do think about themselves as the kind of worst version. Um but but I think a lot of people, I think about themselves the best version and and and then at what you are in a da basis doesn't necessarily ly map to to um to either of those. So I know that yeah there will definitely be a bunch of a bunch of social scientists and in fox will have to. In in psychologist or really there's going be a lot of understand about how our perception of ourselves and others um the shifted from this.

Well, this might be a bit of a complicated dark question but one of the first feelings I had experiencing this is I would love to talk to loved ones. And the next question I have is I would love to talk to people who are no longer here. There are loved ones.

So like, if you look into the future, is that something you think about who people pass away, but they can still exist in the metaverse. You can still have, you know, talk to your father, talk to your grandfather and grandmother and a mother once they pass away. The power of that experience as one of the first things my mind jump to because it's like, this is so real yeah I .

think that there are a lot of norms and things that people have to figure out around that. There's probably some baLance where you know someone has has lost a loved one and is grieving there. There maybe ways is in which being able to interact or relieve certain memories could be helpful, but then there's also probably an extent to which I could become una healthy.

And i'm not i'm not an expert in that. So I think we'd have to study that in an understanding more detail. We have a fair amount of experience with how to handle death and identity and people's digital content through social media already.

Unfortunately, right with you know, there are unfortunately know people who use our services die every day and their families, you are often want to have access to their profiles. And we have whole protocols that we go through where in there are certain parts of IT that um that we try to memories ze that with the the family can get access to IT with the accounts just go away. But then there are other things that are they are important kind of private things that that person has like we're not to give the family access to someone's messages, you know for example.

So um see, I think that there there are some best practices, I think from the current digital world that will Carry over. But um but yeah I think that this will enable some different things. Another version of this is is is how this sex with their eyes, right because and one of the the things that that were really focused on is if we want there, we want the world to evolve in a way where there isn't like a single A I suber intelligence, but where, you know, a lot of people empowered by having A I tools to to do their jobs and to to make their lives Better.

And if you're creator, right? And if you run a podcast like you do, then you have a big community of people who are super interested to talk to you. I know you'd love to cultivate that community and you interact with them online outside the the podcast as well. But I mean, there's way more demand to both to interact with you and i'm sure you'd love to interact with the community more, but you just start limited by the number of hours in the day.

So in at some point, I think making IT so that you could build a an A I version of yourself that could interact with people, you know not if you die, but but while you're here to help um you know help help people kind of fulfill this desired interact with you and your desire to build a community. And there is a lot of interesting questions around that. Um and you know that's obviously it's not just in in the metaverse. I think we would want to make that work you across all the messaging platforms to know what's up and messenger an instrument direct.

But there are certainly know a version that where if you could have an habitual version of yourself in the medium SE that people can interact with, and you can define that sort of an A I um version where, you know, people know that they are interacting with an A I that it's not know the kind of physical version of you, but maybe that A I even if they know it's an A I is the next best thing because not necessarily to act to do directly. I I think that, that could be a really compelling experience. There's a lot of things that we need to get right about IT. Um that you know we're not ready to release the the version that a creator can can can to build the version of themselves yet, but were starting to experiment with that in terms of releasing a number of their eyes that people can interact with different ways. Um and I I think that that is is also can be a very powerful um instead of capabilities that people have over time.

So you've made major strides in developing these early A I personalities um with the idea where you can talk to them across the matter apps and have the interesting, unique kind of conversations. What can you describe your vision air in these early strides? And what are some technical chAllenges there?

Yeah so i'm a lot of the vision comes from this idea that yeah I I don't I don't think we necessarily want there to be like one big super intelligence. We went to empower everyone to both you have more fun accomplish their business goals and everything that they're trying to do. And you know we don't tend to have one person that we work with on everything. And I don't think in the future we're onna have you know one eye that we work with. I think you're going to want a um a variety of the is um so there are bunch of different uses.

Um if some will be kind of more assistant to oriented, there's a so the kind of plain and simple one that we that we're building is called just met A I simple that you can chat with IT in any of your threads um IT doesn't have a face, right IT it's just it's um it's just kind of more vanilla and and neutral and kind of factual can help you with A A bunch of stuff. Then there are bunch of cases that are more kind of business oriented. So so you want to contact A A small business.

Um you know similarly, you know that business pro doesn't anna have to stab someone to man the phones and you I don't want to wait on the phone to talk to someone, but is having someone who you can just like talk to in a natural way who can didn't help you if you're having an issue with the product or if you want to make a reservation or a you buy something online um having the ability to to do that, have a natural conversation, rather navigates website or have to call someone and wait on hold. Um this would be really good both for the businesses and for for moral people who wanted to act with businesses. I think stuff I got make sense um they're going to be a bunch of use cases that I think you're just fun, right? So I think people are gonna.

I think there will be a eyes that I can tell jokes so you can put them into chat, read with friends. I don't think a lot of this because what we're like a social company, right, fundamentally around helping people connect in different ways. And part of what what i'm excited about is how do you enable these kind of a to facilties connection between two people or more, have put them in a group chat, you to make the group chat more interesting around whatever your interest are， sports, fashion um trivia.

video games. I love the idea of playing. I think you mention borders gay, an incredible game, just having an A I that you play together with you. I mean, that could that seems like a small thing, but he could deeply enrich the like gaming experience.

I do think that eyes will will make the NPC is a lot Better in games too. So that's A A separate thing that i'm pretty excited about. But um yeah I mean one of the one of the is that we've built that just not internal testing. People have loved the most is like like a adventure text based um get like a dunder master nice and and I I think part of what what has been fun and we talk about this a bit, but we've gotten some like real kind of cultural figures to play a bunch of these folks and be the ebola ment in the avatara of them. So a snoop dog is the dune master, which I think .

is just yes in terms of the next steps of, you know, if you you matched snows to create a snowman, I so basically A I personality replica, a copy or not a copy, maybe a inspired by smoke. What what are the some of the technical chAllenges of that? What does that experience look like for snake to be able to? Yes.

so that a starting off creating new personas is easier because he doesn't need to stick exactly to what you know, that physical person would want, how theyd want to be represented, right? It's like it's just a new character that we create IT.

So even though this is snoop in that cases, you know he's um you know he's basic an actor, right? He's playing the the dune master, but it's not snoop dog, right? It's it's and ever the the dungeon master is um if you want to actually make IT so that you have an A I and body ying a real creator, there's a whole set of things that you need to do to make sure that that A I is not gonna say things that the creator doesn't want bright and um in that the A I is going to you know know things and build a represent things in the way that the creator would want.

I'm with the crater no. Um so I think that it's less of a it's a lots of a question around like having the avatar AR expressed them. I think that I think we're well, we have our kind of be one of that, that will really soon um uh after connect, but you know that'll get Better over time.

But a lot of this is really just about continuing to make the the models for these day eyes that there are just more and more, I don't know you could say like reliable or predictable in terms of with those communicates that way. You know when you want to create the lex um a system A I that that your community can talk to um you can you know you don't program them like Normal computers. You're training them the A I models not not not kind of Normal computer program, but um but you want to get IT to be predictable enough so that way you can set some parameters for IT.

And even if IT doesn't perfect all the time, um you wanted to generally be able to stay within those bound. And so that's a lot of what what I think we need to nail for further creators. That's why that one's actually a much harder problem.

I think that starting with. With with new characters they are creating from scratch so that when I think we'll probably start releasing sometime next year, um not this year, but experimenting with existing characters and the assistant and games and a bunch different personalities and experimenting with some small businesses. Um I I know that stuff will be ready to do this year and we're rolling IT out you basically right after connect.

i'm deeply entertained by the possibility of you sitting down one myself and saying, hey, man, like you need to stop the dad jokes or .

what the idea of a podcast between you and A I assistant lex podcast. I mean.

there is just even experience of a kota being to freeze yourself like IT basically first mimic yourself so everything you do, you get to see yourself do IT. That's a surreal experience that feels like a if I was like an able looking in a mirror for the first time, realizing like oh, that's you, but then freezing that and being able to look around like i'm looking at you.

It's it's A I don't know how to put them towards, but I just feels like a fundamental in new experience that i'm been seeing maybe calling for the first time of sing. I'm experiencing a new we are seeing the world for the first time because it's physical reality, but it's digital like and realizing that that's possible is just blowing my mind is just really exciting because I live most of my life, you know, before the internet and and experience in the internet, experiencing voice communication, video communication. You think like, well, there's a ceiling to this, but this is make me feel like there might not be there might be that blend of physical reality. Digital is actually what the future is yeah .

think it's .

a weird experience. It's 是 IT feels like the early days like a totally new way of living and like there's a lot of people that kind of complain what you know the internet is not that's not reality。 You need to turn all that off and go, you know, in nature. But this feels like this will make those people happy. I feel like because this feels real, the flaws and everything yeah well.

i'm in a big part of how are trying to design this these new computing products is that they should be physical. But I think part that's a big part of the issue with computers and tvs and even phones is like, yeah, maybe you can interact them in different places, but they're fundamentally like you're sitting, you're still and I mean, people are just not meant to be that way.

I mean, I think you and I have this shared passion for sports and martial arts and doing stuff that are moving around. It's like somebody what makes us people is like now you you move around, you're not or not just like a brain in a tank or it's the where know the human experience is a physical one. Um so it's not just about having the immersive expression of the digital world, it's about being able to really natively bring that together.

And and I do really think that the the real world is this mix of the physical and the digital where the digital is this too much digital this point for you to just be solo to a small screen, but the physical is too important. So you you don't want to just sit down all day long um at a desk so I I think that this is uh ah yeah I do think that this is the future. This is I think, the kind of philosophical way that I would want the world to work in the future is a much more coherently blended .

physical and digital world. There are some difficult philosopher, unethical questions where to figure out society. Ah, maybe you can comment on this. So the the metaphor seems to enable sort of unlock a lot of experiences that we don't have in the physical world.

And the question is like what isn't isn't allowed in the metaverse? You know, video games, we allow, uh, all kinds of crazy stuff. And in physical reality in a lot that is illegal.

So where's that line? Where's that great area between video game and physical reality? You have a sense of that.

Well, I think I mean, there there are content policies and things like that write in in terms of what what people are allowed to create. But i'm a lot of the rules around physical. I think we try to have a society that is as free as possible, mean that people can do as much of what they want unless you're going to do damage to other people and and infringe on on their rights.

And the idea of damage is somewhat different in a in a digital environment. I know when I get into some world with my friends, the first thing we start doing shooting each other, which obviously we will not do in the physical world because you hurt each other. Um but in in a game that's like just up, it's almost you know it's like just fun and um and even like the lobby of the game, right it's like it's just it's not even bearing on the game which kind of like a funny um sort of humorous thing to do.

So it's like is that is that problematic? I don't think so because it's fundamentally it's not you're causin harm in that world. So I think that the part of the question that I think we need to figure out is what are the ways where things could have been harmful in the physical world that will now be freed from that and therefore, there should be fewer restrictions in the digital world um and then there might be new ways in which there could be harm in the digital world that there weren't the case before.

So is more anonymity. But it's know when you show up to uh you restaurant or something, it's like all the norms where you pay the bill at the end. It's because you know you you are you have one identity and the I know you if you stiff them, then like you know like this repeat game and and that's not going to work out well for you.

But in a digital world where you can be anonymous and show up and in different ways, um I think the incentive to act like a good citizen can be a lot less and that causes a lot of issues and toxic behavior. So that is get shorted out. Um so I think in terms of what is a loud I think you wanted just look at what what what are the the damages.

But then there's also other things, things that are not related to kind of harm, less about what should be allowed and more about what will be possible that are more about the laws of physics. So I is up. If you wanted to travel up to see me in person, you'd have to get on a plane and and that was like, take a few hours to get here.

Or as, and we could just jump in a conference room. And you have put on these headsets and we're basically teleported into a space where were and IT feels like we're together. That's a very novel experience that um that IT IT breaks down some things that previously would have defy the laws of physics for what I would take to get together.

And I think that that will create a love new opportunities right? So um one of the things that i'm curious about IT is there are all these debates right now about remote work or people being together and I think this gets us a lot closer to being able to work physically in different places but actually have to feel like we're together. Um so you think the dream is that is that people will one day build to just work wherever they want um but will have all the same opportunities, be able to feel like you're physically together.

I think we're not there today with with with just video conferencing and the basic technologies that we have. But nothing part of the idea is that with something like this over time, you can get a closer that and that would open up a lot of opportunities, right? Because then people could live physically where they want while still being able to get the benefits of being physically or kind of feeling like you're together um with people work all the ways that that helps to build more culture and build Better relationships and build trust.

Um which real issue is that if you if you're not seeing people you know in in in person ever. So ah I don't know. I think it's going to be it's very hard from first principles to think about all the implications of of of a technology like this.

You know all the good and and and the things that you need to mitigate. So you know you try to do your best to kind of vision what things are going to be like and accentuate things that they're going to be awesome and hopefully mitigate some of the the the downside things. But I you know the reality is that we can be building the sw one year at a time. It's gonna a while. Um we're going to just get to see how how how would evolve them and what develops and different folks do with that.

Uh if you can comment, this might be a bit of a very specific technical question but later to is incredible. It's the released that recently um there's already been a lot of exciting developments around IT. Uh is there what what's your sense about this release? And is there lama three in future?

Yeah and I think on the last podcast that we did together, we were talking about the debate that we were having around open sourcing lama too, and i'm glad that we did. Um you know I think at this point is there's the the value of open sourcing a foundation model like lama to is significantly greater than um than the than the risks in in my view view. And we did we spent a lot of time to get very rigorous assessment of that and red teaming IT.

Um but I am am very glad that we really slam IT to I think the reception has been it's just been really exciting to see how excited people have have been about IT. And it's gone way more downloads and usage than I than I I would have even expected. I was pretty optimistic about IT.

Um that's that's been great. Um lama three uh I mean, there's always another model that were training. So I mean, it's for right now if we build, we we train lama to and we released as an open source model.

And right now, the priority is building that into a bunch of the consumer products, all the different ai um in a bunch of different um products that that we're basically building a consumer products because lamu by itself is not a consumer product, right? It's more of a piece of infrastructure that people could could build things with. So that's been the big priority is kind of continuing to find tune and um and in kind of just get lama to and and it's um and it's little the branches that we built off of IT.

Ready for consumer products that hopefully you know hundreds of millions of people will will enjoy using those those products and billions one day. yeah. I mean, we're also working on on the future foundation models. And um I don't have anything new or news news on that. I don't know and I don't know exactly when this can be ready.

Um I think just like we had a debate around lama u and open sourcing IT, um I think we will need to have a similar debating process to red him this and make sure that this is safe. But in my hope is that will be able to to source this next version when it's ready to. But um but that's not that we're not we're not you know close to doing that this month and this is um that's just it's a thing that were worth still somewhat early and worked on well in general.

Thank you so much for all in sourcing lama to in for being transparent about all the exciting developments around A I I feel that contributing to really awesome conversation about where we go with the I and obviously is really interesting to see all the same kind technology integrated into these personalized A I systems of with with the A I personas, which I think when you put in people's hands and they get to have conversations with these A I personas, you get to see like interesting failure cases, like where the things are done, or they go into weird directions, or and we get to learn the society together, what's, what's too far, what's interesting, what's fun, how, how much personalization is good, how much generic is good. We get to learn all of this and you probably don't notice yourself like we have to all figured out by using IT.

right? Yeah I mean, part of what we're trying to do with the initial A I S launch is um having a diversity of different use cases just so that people can try different things because I don't know it's going to work come in our people gonna playing in the text based adventure games. Are they going to know like having a comedian who who can add jokes um to to thread or or they can want interact with historical figures. Um you know we made we made one of jane Austin and in one of market areas and i'm curious to see azores.

I'm excited for both. Yeah is a big fan. I'm excited for both. They have conversation for them. I mean, yeah that's yeah no. And I am also excited to see you know the internet I know if you heard can get kind of weird um and I applaud them for IT so I get yeah so IT be nice to see how we are. They take IT what kind of means I generate from this. And I think all of IT, especially this early stages of development as we progress towards A G I, is good to learn by by playing with those systems and interacting with them and like a large scale.

like he said, yeah totally. I mean, that's why what's we're starting out with a set. And then um we're also working on this platform that we call A I studio that's going to make IT so that every time anyone will be able to create if one of these C I is almost like they create any other ugc content across the platform.

So i'm i'm excited about that. I think that to some degree, we're not going to see the full potential of this until then. You just have the full creativity of the whole community being able to build stuff.

But there's a lot, a lot of stuff we need to get right. So um so i'm excited to take this in stages. I don't I don't think anyone out there is really doing what we're doing here. I think there are there are people who are who are doing kind of like fictional or consumer oriented character type stuff. But the extent to which were building IT out with the um avatar ars and expressiveness and making is that they can interact across um know all the different apps, have profiles you know people to engage people, instagram, facebook, K I I think it's it's just that it's going to be be really fund well.

i'm still so we're talking about A F, and I still blown away the entire time talking a mark zucker burg. And you're not here, but you feel like you're here. I've done quite a few intimate conversations with people alone in a room, and this feels like that. So I keep forgetting for long stretches at time that, like, we're not in the same room.

And for me to imagine a future where I can, with a snap of a finger, do that with anyone in my life, the way we can just call right now and have this kind of shallow to the experience, uh, to have this experience that are sitting next to each other is like, I I don't I don't think I can. I don't think we can even imagine what how that changes things where you can immediately have intimate one of one conversations with anyone that might like, in a way we might not even predict change civilization. I mean.

this is a lot of the thesis behind the whole metaverse, giving people the ability to feel like you're present with someone. I think this is like the main thing I talk about all the time. But I I do think that there's a lot to to process about that. I mean, for microstructure, I mean, I am definitely here. We are just not we're we're not physically in the same place.

It's not like you you're not talking to an A I right here you know this is um so I think that the thing that novel is the ability to convey through technology sense of almost physical presence um so the the thing that is not physically real is um is as being in the same physical place but um but but kind of everything else is and I think that decades to this uh somewhat philosophical question about what is the nature of kind of the modern real world and I I just think that that's IT really is this combination of uh physical world and the presence that we feel, but also being able to combine that with this increasingly rich and powerful and capable digital world that we have and in all of the the innovation that's getting created there. So I I think it's super exciting because I mean, the general world is is just increasing um in in its capability and our build to do awesome things. But the physical world is so profound and that's a lot of what makes us human is, is that we're where we're physical being.

So I don't think we want to run away from that and just spend all day on a screen. And that's like it's one of the reasons why I care so much about about helping to shape and accelerate the these future computing platforms. I just think this is so powerful and it's it's you know, even though the current version this is like you're wearing a headset um I just think this is can be by far the most human and social computing platform that has ever existed and that that's what what makes me excited .

yeah I I think just a linger on this kind of changing nature of reality like of what is real, maybe shifting IT towards the so consciousness. So what is real is the subjective experience of a thing that makes you feel real verses necessarily being in the same physical space as IT feels like we're in the same physical space. And the conscious experience of IT, that's probably what is real.

Not like that the space time, like the physics of IT, I get bracing, a breaking physics and are focusing on the consciousness. That's what's real. This whatever is going outside my head.

But there are a lot of social and psychological things that go along with that experience that was previously only physical presence, right? Anything there is I can intimacy a trust. Um there's A A level of communication because so much of communication is non verb based on expressions that you're kind of you're you're sharing with with someone when you're in this kind of environment.

And before those things would have only been possible, you had, you know, I got on a plane and flown off and and said in physically with you in the same place. So I I think we were basically short cutting those laws of physics and delivering the social and psychological benefits of being able to be present and and feel like you're there with another person which have got real benefits. Um twenty one of the world.

And I I think that that like you said, and I think that is going to be a very profound thing and that a lot of that is that's the promise of of the metaverse and what you know why you I I just I think that that's the next frontier for for what we're working on. You know, I started working on social networks when they were primarily text, where the first version of facebook, your profile, you know, you had one photo and the rest of IT was like lists of things they were interested in. And and then we can went to the period we're doing photos and now we're kind of in the period where most of the content as video.

But there is a clear trend where in over time, the way that we want to express ourselves and and kind of get inside and content about the world around us gets increasingly just richer and more vivid. And I think the ability to be immersed and feel present with the people around you are the people who you care about is, from my perspective, clearly the next frontier is just so happens that it's incredibly technologically difficult. Read requires building up these new computing platforms and completely new software stacks to deliver that. But am I kind of feel like that's what we're here to do as the company?

Well, I really love the connection you have the conversation. And so for me, this photo realism is really, really exciting. I'm really excited for this future.

And um thank you for building and thanks to you and thanks to the amazing matter teams that have met the the engineers and just everybody have met here. Thank you for helping to build this future and thank you, mark, for talking to me inside the metaverse. This is blowing my mind. I can't quite express. I would love to measure my heart right this whole time would be hilarious if you actually like sitting and the beach right .

now in the conference room.

Okay, well, i'm at the beach. And if if and not wearing any pants, someone drive about that for anyone else. He was watching me in physical space.

Anyway, thank you so much for talking today. This, this really boom, my mind. So on the most .

incredible experience, thank you for giving a IT out. I like that you soon.

这 不是 so so main， 不是 so amazing。