Reid riffs on if AI can become conscious
18:11 Share

Hey, everyone. So excited for our discussion today. We are going to be responding and talking about our conversation with Yuval Harari. So if you have not listened to our Yuval Harari episode from last week, go listen to that first. That will make this make a lot more sense. And here we go. Hey, Reid, it is great to be with you here today. Always great to be here.

There's obviously a lot of agreement between you and Yuval about how you see the world, about your care for humanity. You want to make sure to be shooting in the right direction, but there was also plenty of disagreement, sort of both philosophical and also like what exactly is going to happen. So I'd love to hear your reflections. Tell me about the conversation and where did you disagree? Well, Yuval, obviously, always an amazing delight to talk to and one of the most

leading public intellectuals of our time and a very clear-headed thinker. I mean, part of what he tries to do is say, look, I'm seeing the world as clearly and without illusion or self-delusion as possible, and I report what I see.

For him, he looks at the exponential, massive acceleration, not even massive acceleration, the word exponential is used, the word exponential is an increasing capability.

And he goes, well, it's going to be the species. We as a species, planet, planet, why shouldn't we just further? And boy, I hope there's a role for me. Continuing to be a metatube being in this mixture. And

I think that it's actually going to... We make lots of things. We make industries. We make... We make... We make... We make... We make... And I think it's a much higher probability that this elongates the kind of...

the capabilities of Homo sapiens, it's kind of like saying, you know, again, in a simple metaphorical sense, you know, is it, is, is it more likely to be creating, um, you know, kind of drug resistant, uh, or sorry, uh, new forms of, uh,

antibiotics that will help against drug resistant tuberculosis? Or is it more likely to be creating the tuberculosis? And I think it's more likely to be creating the therapeutics to tuberculosis. So it's kind of, so there's kind of a probability set. Now, then when you get into kind of details, he says, well, but one of the problems is, is the learning machine. And if you're learning from

you know, kind of human beings, look at all this bad stuff we do. You know, look, we're cruel. You know, we torture people. We engage in war. We're lying and self-deceptive. And, you know, da-da-da-da isn't just going to, just like a child of ours, isn't just going to learn, you know, because it isn't just what we say. It, like, watches what we do.

And, you know, I think obviously there is, you know, kind of definitely some truth to that. But also, by the way, just as when we're teaching children ourselves,

even though we have some faults, we actually try to direct children towards our more aspirational selves. The fact that we're compassionate, the fact that we're wise, the fact that we have these kind of humanism values, these humanity values, it's kind of what's in depth encoded in religions and where religions kind of share a context of how do you aspire to being our better selves?

And, you know, that, you know, and this is the half full and half empty. I go, well, actually, in fact, I think we can shape it that way. I think we do that with our children as it is. It's part of what we try to make progress in society. And I think that's the thing that we can potentially drive to.

What I also think to your point, a lot of it, of course, is optimism and is let's figure out not to complain. Let's figure out actually action so we can get to a better place. But some of it is just recognition of some of the bad things that actually can happen. Like you said, sort of the tuberculosis versus the antibiotics for tuberculosis.

the same people often who are gloomers or saying this is going down a bad route would probably also tell you that climate change is going to do irreparable harm, or it's going to say that we're unprepared for the next pandemic. And so you're not just saying that, you know, AI is great compared to some utopia. You're saying AI is great compared to the world we have now, because we have real challenges and AI like has to be here to help solve them. So

A lot of people are concerned. It's like, oh, you have these five AI labs who are just, you know, they're deciding the future of the world. Like, first of all, how do we make sure that they have the right values? Like, is there anything we can do to affect what they're doing? And then also, like, who else needs to be involved in this? Like, who else needs to be a part of this decision making? Well, a couple of things. One is, you know, and this causes...

existential risk people some heartburn, it's probably five labs heading to 15, heading to 30, or at least 10 to 15. And why is that good? For people who get heartburn, why do you think that's actually a great outcome for the AI space? Well, it's a complicating outcome for the AI space. One of the places where I tend to find, you know, kind of many critics to be

kind of, you know, really yelling about like, well, I should be in charge and it should be me. And it's like, well, not clear. Right. Even though you go, well, I don't want just the large tech companies and just the heads of these labs to be in charge. And you're like, well, but some people will be like, there will be a limit, like take, there will be a limited set of people who will be in charge. And then you go, given that limited set, what's the, what's the, the reasonability of which limited set. And it's like, well,

folks who are investing and building this stuff and have the capabilities of doing it and are putting their own lives, missions, and their economics into this and who are held accountable by various networks of accountability, which includes critics, includes press, includes government, includes customers, includes shareholders, includes family and community members, includes teammates. There's many, many different things. And we try to make those

channels of accountability to be as healthy and as inclusive as possible. Like the people who say it's only these five and what gives them the right is like, well, actually, in fact, it's, it's a growing number. So if you're, if your particular thing is, is that, you know, what do we, what do we think about these five? You know, it's like, well, but it's not only going to be these five and a little bit of that as a kind of a gesture of the people who get on the

you know, kind of blindly or for their own self, their own, their own reason they get on the antitrust horse. And so they tend to go, our really important thing is to limit these five. And it's like, well, it's only really important to limit these five. If you think these are only these, these are the, the laws of physics say it's only these five. And it's actually not the case because, you know, since,

um since i've been doing the kind of western democracy things both there's been new uh entrance on the western democracy side and there's at least three to five that we're tracking in china and it's probably actually in fact you know 10 to 20 in china so you've got you know this this this swarm of it so you get a larger number of people so in as much as it's people um

with different points of view, you know, kind of playing for this, that answers the, or that, that gestures at the, you know, that there's a number of people working on this. Now, this is not a monopoly question. Yes, not a monopoly question. But this is because other people heartburn because they go, well, actually, in fact, these 10 to 20 people are going to compete that as opposed to getting to our highest virtue selves, it'll be because of competition and because of divergence between people that

We're going to have all like, you know, for example, we'll have AI weapons created. And why won't someone, you know, either deliberately or accidentally of this much larger set, you know, be targeting, creating kind of the equivalent of the Terminator. So, you know, I think that the question is, again, is to say, look, if we presume that

The fundamental thing about human beings is we divide into groups. We have different perspectives about what the risks are, but what the important things to achieve are. And we compete with each other. And part of our competition is manifesting those values. The thing that we must do as thoughtful human beings who are trying to be great for humanity is we say, all right, how do we help

those efforts that, you know, care about the same kind of, you know, perspective of, you know, being compassionate towards human beings of being, you know, kind of elevating more better selves or having a human future that is the continued evolution that we've had over centuries and millennia, you know, and a very broad brush to continue to do that to be, you know, kind of

of the virtues of kind of wisdom and empathy and other kinds of things. And how do we do that? And the answer is make those projects that are deeply trying to embed those values, embed the values in the AI systems they're creating, in the products that they're deploying, and to make those kind of the more accelerated projects

you know, efforts. And as you know, that's of course what I've been doing. But there is a possibility that we're creating creatures, we're creating new entities and it's mind bending. But it doesn't mean it's a certainty. And it doesn't mean that the way that we should steer is, well, because that's a possibility, we should steer about that being the certainty.

we actually have to be iterating and seeing because it's completely possible that X years from now we'll say, oh, just like when in the 80s we were talking about this with AI. And yes, this is a much better technology. Yes, it's achieved so much more. Yes, it's on a better curve. But just when we're talking about that and we said, okay, that will completely change. It's like we may very easily discover in as few as a few years and maybe even five to 10, it's like, well, actually, in fact,

These aren't really entities. They're actually tools in the following way and they have the following shape. And this is the way we're going to integrate it in. And so it's a, it's a discovery of, of what's possible because not everything is possible at every moment. And another thing I appreciated about the conversation, and this is probably true of any conversation where you and Yuval are in the same room, is that we get to talk about more, more philosophy, more philosophical questions.

questions about sort of what it means to be human. And, you know, he had an argument that was focused on the critical distinction between intelligence and consciousness. And he said that while AI might become vastly more intelligent than humans, intelligence alone doesn't guarantee a pursuit of truth.

or even the capacity to reject falsehood. And I know truth-seeking is something you talk about a lot. It's like, how do we get to this place? How do we teach truth-seeking behavior? What are the ways that we can model that world? And so I'll ask you, is...

Is consciousness a necessary foundation for this truth seeking? And if so, can this non-conscious AI like ever really be truly aligned with our deepest values? Or are we missing something fundamental with asking them to define truth when they don't have the consciousness piece, or at least not yet? You know, I haven't had a chance to have this conversation with Yuval. So if Yuval's listening to this, this is the next move in the conversation, but

What I've realized is I think where it's helpful to think about is, well, what kinds of truth seeking are necessary from conscious entities? And how does that illuminate what we think about consciousness? And what kinds of truth seeking are doable without consciousness, with just intelligence or fitness functions? Because you clearly have a set of truth functions that are where consciousness is not necessary.

Like it's just like, for example, you can train in lots of different systems, even less sophisticated than AI systems.

to actually, in fact, run a truth-seeking process. I mean, heck, when we have deep research in ChatGVT and Copilot and Gemini and Claude and others, it actually says, go cross-check your work. Go pull out documents and cross-check your thing. And then when two things disagree, go look for other information and privilege these kinds of sources of information, just the same kind of

cross-checking we do in our group truth-seeking, whether it's science or judicial processes or academic work or else. It's that process for journalism to do truth-seeking. Clearly, you can do a lot of that with just intelligence. Then you get to this interesting question around...

Obviously, if you had an enlightened being that said, I am in touch with how difficult suffering is, and I view the importance is to reduce suffering across more things other than just me, and to have quality, sentient life as doing that.

Then that's a very good thing. Is consciousness necessary for some component of that thing? Is the fact that we experience reflection, meditation, potential empathy, compassion, sympathy with each other through the recognition of that consciousness, is that an essential component?

And if that's an essential component, can we keep it essential in various ways for how we operate with the future of the world is? And I think that that's that's a research that that's a that's an ongoing discovery question as we as we get there. And I'm still thinking about it. Now, I would say in kind of a final close, this is we clearly can demonstrate better and better alignment with human values and

Like one of the things that, again, if I were to make the argument out of evidence for optimism versus just a, I'm hoping for the best is when you look at kind of the evolution of the open AI systems of GBD two and three and four and 4.5, they've much as they get more sophisticated, they much more naturally and easily align with a set of human considerations. They much more naturally understand, you know, kind of,

What potentially our better selves are, what those things are. And they actually do actually, in fact, show better ability to go like I have some level of understanding, comprehension of of what the kind of human goal set is in the aspirational side. I've been I've learned and been trained.

to preference like oh you're trying to figure out how to write poetry or you're trying to figure out how to how to have a productive conversation with your friend your child your spouse you know and i can help with that and then when you say hey i'd like to you you know to make a bomb says no i'm not going to help you with that right right and it naturally you know kind of i

aligns better in those ways doesn't mean there's a ton of work, doesn't mean there isn't risk, but that's like a positive where we're getting that alignment, even though I think it's only...

the you know kind of the very extreme crazy fringe that think these systems are conscious today right and there are people who think that they are conscious today yep i think there's a whole bunch of good arguments as to why they're not but uh and will they ever be that's one more mind-bending question

Well, I cannot wait for our next conversation with Yuval to tease into a lot more of these ideas. Reid, thank you so much. Likewise. Possible is produced by Wonder Media Network. It's hosted by Ari Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Katie Sanders, Nidhi Allard, Sarah Schleed, Vanessa Handy, Malia Yates, Paloma Moreno-Jimenez, and Malia Agudelo.

Jenny Kaplan is our executive producer and editor. Special thanks to Surya Yalamanchili, Saida Sepieva, Benassi Dilos, Ian Ellis, Greg Beato, Parth Patil, and Ben Rellis.