The future of AI
01:28:49 Share

Welcome to the LSE events podcast by the London School of Economics and Political Science. Get ready to hear from some of the most influential international figures in the social sciences. Welcome to the LSE for this evening's public event, The Future of AI.

My name is John Cardoso Silva. I'm an assistant professor of education at the Data Science Institute here at the LSE. And I will shortly be leading this conversation about all things AI with Professor Gary Marcus here on my left. Following the discussion, there will be the chance for you to put your questions to Gary and I.

And when the time comes, please raise your hands and wait for the stewards, you see them in red, to come to you with a roving microphone, and then you can pose your question. Right now, I'd like to remind you all to please put your phones on silent, and so as not to disturb the proceedings. This event will be recorded, audio recorded, there will be no cameras, and it will be available as a podcast subject to no technical difficulties.

Finally, there's no fire alarm scheduled. So if we do hear one, it's actually for real. And we have to make our way outside from where we came from. And that's about it. We can get started with our event. I'll introduce our guest. I'll pose him a few questions and then I'll hand back to all of you in the audience.

Professor Gary Marcus is a leading voice in cognitive science and artificial intelligence. He's a co-founder of the Center for the Advancement of Trustworthy AI. He's well known for his challenges to contemporary AI, anticipating many of the current limitations decades in advance. He was trained by Steven Pinker. He received his PhD from MIT at the age of 23.

and was a professor at NYU for 20 years before becoming founder and CEO of Geometric Intelligence, a machine learning company which was acquired by Uber in 2016. As an emeritus professor of psychology and neuroscience at NYU, he's also known for his research in human language development and cognitive neuroscience. He's the author of six books,

And here's worth noting that in The Algebraic Mind, a book from 2001, he foreshadowed the hallucination problems that plague the current AI systems. So you've been talking about that for some time. Quarter century. In Rebooting AI, a 2019 book with Ernest Davis.

Professor Marcus calls for fundamental changes in how we approach artificial intelligence and that book was listed amongst Forbes seven must-read books in AI. His 2022 article, "Deep Learning is Hitting a Wall," initially enraged many AI researchers around the world, but it was ultimately named one of Pocket's best tech articles of 2022.

More recently, he published Taming Silicon Valley. How can we trust that AI will do good for us?

Among this wealth of knowledge and publications and scholarly endeavors, his conclusions that current AI systems face serious limits in truth, comprehension, and reliability are some of the topics we'll be discussing today in this event. So I ask that you join me with a round of applause to welcome our guest, Professor Gary Meyers. Thank you. Thank you.

Thanks for coming. Thanks for having me. And thanks for selling out the crowd. Yes. Thanks to you all for coming. We're fully booked.

I want to start with a big team and you know, we've been promised artificial general intelligence and we hear about it now quite a lot in the media, in the tech press, AI tech leaders talk about it and in some of the ways people portray this is that there will come a time where AI will be able to perform any task that a human can do. So this is where we'll meet this AGI and some say that it's just around the corner.

Is it just around the corner? No, it's not. But there is a kind of redefinitional Olympics going on where people keep trying to redefine it. And I think that we will see in 2025 or 6 a lot of people saying, hey, I achieved AGI. It's certainly the sexy thing to say right now. But you should, I guess, start by asking what it means.

I don't think it means literally doing every task a human does. So if we had some AI system that could do most things that a human could do, could do them pretty well, but it couldn't taste wines and give us a good answer, I wouldn't want to say, "Aha, you dumb AI, you can't give us good answers on wine tasting, so that's not really AGI."

You're supposed to capture the spirit of what humans do, especially the flexibility, the resourcefulness, and so forth. And if you go back to the original definitions from people like Shane Legg, who I guess lives here in London, who was one of the co-founders of DeepMind, Ben Goertzel, who pushed AGI before anybody else really was, they were talking about the flexibility of human cognition, the ability to be at least as good as humans at most problems.

You could say there's this special aspect like wine tasting or whatever, but you should generally have that flexibility. There are ways in which certain AI systems have been better than all of the best humans for a long time, right? And we call those narrow intelligences, like playing chess. So Garry Kasparov lost in the 90s, I guess if I remember, to Deep Blue. Nobody would say that was artificial genital intelligence. You could say it was super intelligence in a narrow domain.

but it's not general intelligence. Now, until recently, we didn't have anything that even looked vaguely like general intelligence. We now have something that looks, I left something out, we call it artificial general intelligence. The stress is on the generality of it, that you should be able to do lots of different things, not just play chess. That's sort of opposite of now intelligence. So now we have these large language models

and they give the superficial appearance of being able to do essentially anything. You type something in, you get an answer to it. I have suggested a different term, which is broad, shallow intelligence. The clever people will think about what the first two letters might mean there. BSI. And

In BSI, you can BS your way through anything, and that's what large language models do. It doesn't necessarily mean they're reliable. So I give you one of many examples. They can BS their way through a chess game.

Turns out they can play better than the average human being. But they also have a tendency to make illegal moves, and they're nowhere near as good as Garry Kasparov was or Magnus Carlsen or something like that. They play a decent game of chess, but they make a lot of illegal moves. That is pathetic.

Let me tell you why it's pathetic. They've been trained on millions, maybe tens of millions of chess games. That's a lot of data. It's many more than most human experts. You would expect them to work. You would expect them. But the illegal move part is particularly troubling because they are trained on the rules of chess. These things are trained on the entire internet. Do you know how many copies of the rules of chess there are? Thousands? Millions? I don't know. You go to Wikipedia, so you have high quality sources that give you the rules of chess, and

And they'll still do things like they'll have a bishop jump over a rook. Like my daughter, when she was six years old, would have laughed hysterically and be like, you can't do that. So you give them all of this data about chess and they still can't reliably...

Same thing with multiplication. People have been doing studies. The new bigger models are a little bigger. We should talk about scaling and this notion of adding more data. But still, you give them like 12 digits times 12 digits, and the calculator that I had in 1979 or whatever would always get it right. And you'll have these little charts, and they'll be like green for they get it 100% right. So they're 100% right on one digit times one digit, which my daughter did when she was six years old.

- And what is the value in that? So what is the point in creating-- - The value in creating a system that is unreliable at all of these things?

I mean, that's an interesting question itself. So what do we do with this thing? So this thing that we've built, this broad, shallow intelligence, was pitched very successfully in the media as basically tantamount to artificial and general intelligence. And Sam Altman kept flirting with the idea of AGI. Like once he said, AGI has been achieved internally in a Reddit post. And every time he speaks, like,

Hundreds of thousands of people listen to his every tweet and try to parse it and they write stories. There's a whole genre of fan fiction where people write about, I've seen GPT-5 and it's going to be amazing and whatever. And then GPT-5 isn't actually...

It's still not here. I wrote an essay once actually about that. How many people ever saw the movie Airplane? It's one of my favorite comedy movies of all time. So at the end of the movie, it doesn't give away too much to say that the airplane finally lands and it doesn't land very well. And so the announcer goes, now arriving gate five, now arriving gate seven, now arriving gate 10 as it skitters across the airport. And I think it ends at 17. And so I have an essay called like GPT-10.

I can't remember the exact word, GPT-5 now arriving, Gate 5 or something like that. And, you know, it's been promised over and over again. Altman keeps hinting at it. People keep writing these things. Like, I remember Super Bowl 2024, I guess it was. People were like, wow, it's so cool. He's going to drop it after, you know, release it, GPT-5, right after the Super Bowl. Well, that didn't happen. And then there are more memoirs like this. It still hasn't come out. But so Altman has cultivated the notion that,

AGI is imminent. We're going to have systems that can do all kinds of things. The reality is they still get tripped up on basic things. The most notorious is how many R's are in the letter. Strawberry. Are in the word strawberry. Sometimes they put band-aids on these things. The latest system might actually get that one right, but then you change the example to, I don't know, how many...

O's are there in economics and you're still going to see failure. So they always put these patches that are so superficial. So we're not actually close to it. But people have pretended it. And I know I'm going to get to your question eventually. Just bear with me. So

Now we have this artifact in the world, and it is a little like an alien intelligence. If we had aliens descend on planet Earth or whatever, we would talk to them and they would do some things the way that we do and some not, and we would use the tools that linguists use to speak with an informant and get information. And we would find that this intelligence is very different from ours. It does some things very well and some things very poorly.

What it does very poorly are things like planning and reasoning and general purpose abstraction. What it does really, really well is mimicry. And so you can say we have this spectacular mimic. It doesn't really know what it's talking about. What can I use that for? And it turns out there are some things. So one of the things that it can do is it can mimic

computer programming languages and the use of them. And so that allows it to be kind of autocomplete. I've often called LLMs autocomplete on steroids. So you have autocomplete on steroids for computer programs. And there's a lot of kind of repetitive, tedious stuff in computer programming where not only do you have to kind of

type the same kinds of whatever, if then conditionals or for next loops or whatever, but you also have to look things up. What is the function call that I use in order to display this thing in the newest version of HTML or with this, yes, or whatever. And so you can type a few keystrokes and it effectively will figure that out for you. And in fact, you can do some cooler things. You can say, just write me a function that does this. And

used in the hands of a programmer who knows what they're doing, and I'm gonna come back to that in a second, it can really save them a bunch of time. Not apparently as much as people think. So people are like, oh my god, I can program 10 times faster. - You can vibe code your way into a project. - Let's get to vibe coding in a second. So seriously, let's get there. So people said, well, it makes me program 10 times faster or whatever.

There's no study that actually supports that. People maybe can code like 30% faster, but when you really think about coding, how many people here code?

or have coded a little bit right the easy part is typing stuff up i'm not that good i've been coding since i was eight years old um i will make mistakes when i when we type things up but that's still it's the easy part the hard part is debugging partly because you make typing errors and you left something out and you don't realize it to later whatever and debugging is like sherlock holmes you're sitting there like what is this system supposed to do

you know what is actually coming out what can explain the differences you're doing this very logical analysis which the systems like can help with but aren't that great at um and so what you wind up with is yeah you write the code faster but sometimes it takes you like two days to fix the bug that you introduced because somebody else or some not really a somebody but something else exactly has written this code and you don't understand what it's thinking um and so

what you gain on the typing faster you sometimes lose on the debugging later, and sometimes the code that it writes isn't secure, so there's more problems. So now I get to buy coding. - There is actual research on that showing insecurities in code that has been put on GitHub and places like that because of-- - Yeah, I mean there are crazy examples there. These systems will tend to call for functions that don't exist.

And so malicious people have discovered that and they will write malicious things, packages that the systems will then call in order to get their devious things installed. So there are some very specific kind of attack vectors that you're vulnerable to. And so when you actually do studies like how much better

how much more productivity you get from your coders, you get like 30%. You don't get 1,000% at all. No study supports that. Although somebody on Twitter literally told me today, I'm programming 10 times faster. I'm like, buddy, do you have some data on that? Maybe they're just vibe coding new Pac-Man stuff. So let's tell people what vibe coding is. We need to introduce. So vibe coding, the term was invented by Andre Kapathy, who's a very influential machine learning person, was

was that open AI and Tesla and back at open AI and now has quit them all. Um,

The vibe coding is like you type into these systems and vibe coding could be done by people who know what they're doing or not I guess but people like Andre Kaparthi could probably get away with it. I just kind of type some stuff in and I actually have enough of a mental model of what I'm trying to create that it spits out code and I just kind of go with the flow and I guess it's sort of like that. The worst version of this is some people, not to mention names, but Kevin Roos at the New York Times

Think that you can basically take a complete amateur and turn that person into a coder such that we don't need coding anymore. He actually wrote an article about this in the New York Times, which I critiqued in great detail in my Substack, garymarcus.substack.com.

And he gave an example of... He was so excited. He was somewhere between sad and pathetic and cute. He said, I...

built a system that could look in a refrigerator and give recipes so my kid could make his or her lunch. And, you know, it's kind of cool that he feels empowered to do that, but he doesn't understand coding, which he said in the first line of his thing. And it shows because a real coder would be like, okay, this is really cute what you've done. Um,

There's a few things a real coder would do. The first thing a real coder would do is be like, did anybody else do this first and I can maybe modify it? And he didn't do that. So it turns out that when the system gave him some code, it was like in a demo exercise. There were like 100 demo exercises on the web and you could download it from GitHub. So it wasn't as impressive as he thought it might be.

But then a real coder would be like, okay, there are going to be some problems. Like you can't really tell how much milk you have if you have an opaque milk container, which most of them are. Like what are you going to do about that? So a real coder sits there and basically thinks about what's going to go wrong. In fact, there's a term called production code.

And production code is basically about figuring out all the things that go wrong and making sure that there's some defense against them, as opposed to prototype code, which is like, hey, look at the general idea, look and feel of this. But it probably breaks every time somebody does something unexpected. So he made kind of the equivalent of prototype code, but not production code. And he didn't think about all of those kinds of things. And the system was probably actually pretty lousy and wouldn't really be very useful. But there's a demo that you can try on the New York Times he was super excited about.

The Guardian wrote a much better piece. I cannot remember the author's name, I'm sorry. Maybe he's in the room, that'd be really cool. But what the author did is he wrote to like seven people on Twitter that were all excited about vibe coding and whatever, and said, okay, here's what I want you to do. And he did something much easier than what I would have asked for. He said, can you make Pac-Man...

which most of you will know is a game from 1982 where there's a little guy like this going around and there's some ghosts chasing him. There are a bunch of Pac-Man implementations available online. There's a bunch online. So this shouldn't be that hard, but it turns out that they all stank. We went through seven different implementations that were made by non-coders, and they didn't work. The graphics were wrong, the logic...

game logic was wrong. And they didn't even try something harder, which would be like, okay, can you double the number of ghosts or introduce a purple palette that changes the behavior or whatever stuff that I would expect any good undergraduate or high school student or whatever to be able to do. There was no test of that. It was much simpler than that. It was just like replicate Pac-Man, which, you know, you can do probably on my watch if you wanted to. And it didn't work. And it didn't work. So the reality is that,

kids should still be learning to code. Part of the reason I have gone very long on this particular thing, and then we have to come back to the first question, is because there is this myth now, promulgated by people I won't name, like Kevin Roos, which the idea is, but also by Jensen Huang, that kids don't need to learn to code anymore because machines are just gonna take care of that. I think that's an incredibly bad idea. I speak a little bit as an American, don't boo me, it's not my fault.

You know what I'm talking about. As an American who would not like America to be overtaken by China, although it's too late and it's going to happen anyway, but for many reasons, maybe we'll get into some of them. I would like America to still teach its kids how to code. And China's not going to fall for this. China's going to keep teaching their kids to code. They're not going to fall for all this hype. And so I'm actually genuinely worried that we are going to lose...

The kind of momentum we've had towards growing a set of coders in the United States, I think is still going to be important for another 30 years. Because really, and then I'm going to come back to the first question. The critical thing in coding is actually about understanding human needs and translating that into code. So you have to understand, like, what is the person who's using this application going to want to do?

or the person who was purchasing this application, paying me, whatever. And so the human understanding of it, we're not that close to machines. In particular, part of it is because we don't specify everything we want. Like, there are a lot of things in the history of coding that are basically boiled down to, you know, do what I want you to do, not what I've said that you should do. And

To actually code is to understand this complex interplay between a customer's needs and what can be written in code. We're not going to replace that anytime soon. So back to the first question. What can you do with this alien life form? So you can use it to do some coding, but you can't solve the whole code.

coding problem. You can certainly use it for brainstorming. Hey, I'm going to London, give me 10 things I can do with my kids. And you can get reasonable answers out of these systems, especially if you have a human in the loop. Sometimes it will make things up, right? And so, like, you can't trust them too much, but you can do some of that.

High school students use them to write their term papers all the time. I mean, that's not the most pro-social or pro-educational agenda, but that is something you can do with these tools. And we will probably be spending the next 10 years figuring out other positive applications and also negative applications. So cyber criminals have already figured out that you can use them to fish people's credentials by impersonating humans and tricking them and so forth. So the...

Set of applications is still not completely known and you know Better ones come out every day and worse ones come out every day and so forth So those are the kinds of things you can do they all involve mimicry and none of them involve kind of sticking to the fact So even like a study from a couple weeks ago with the most recent Models had these I think like deep research and systems like that. Although not I can't absolutely swear to it. I

generate like research reports and like 60% of the citations were hallucinated we're just made up and so like you can't count on it for precision work and so that limits it and go back to artificial general intelligence and the downscaling of the definitions I didn't tell you how that got downscaled they've gotten downscaled to things like first

that can make $100 billion. I mean, this is a serious proposal. This is ridiculous. This guy made $100 billion. That doesn't mean that this thing is artificially generally intelligent. It's really useful, but I wouldn't want to call it AGI. That's redefinition. And that speaks volumes of how the tech leads, like AI tech leads view AI, right? So it's put as a, if it makes money, then it's intelligent in a way.

So there's some of that. I mean, a back story on that one is that Microsoft made a crazy deal back when they were in the cult of Sam Altman. There was a period in which Microsoft thought Sam Altman was magic. That's not true anymore. I actually wrote a piece in my substack called something like Microsoft and OpenAI, colon, it's complicated.

So their relationship has really deteriorated over the last two years. But early on, they were completely sold on it. And they made a deal, a crazy deal. They put in, I think it was like $11 billion into OpenAI.

And there was profit sharing. So they were going to get the first 49% of OpenAI's profits, which is still true. But there was this crazy clause. The crazy clause said when AGI is achieved, then basically control of the company reverts to OpenAI. But what is AGI? Who's going to decide? Are they going to bring in me, Gary Marcus, to decide? There was a lawsuit with Elon Musk and Alibaba

OpenAI where I thought even if they really should bring me in, that would be like the great irony of all time if they did that. But so like they realized that this was a mess and like they left it up to OpenAI's board

And I think at some point Microsoft said, that's really not going to work for us. We're going to have to renegotiate this. And that was before Sam Altman's ousting, right? So that was... The $100 billion number was floated after he was fired and rehired within the space of a few days. Somebody said Succession is better than the Sam Altman drama. I'm in the third season of catching up on Succession. They might be better.

So, and then another definition which OpenAI had on its website earlier was doing 80% of useful work of humans. And I look at this as an academic, as a kind of somewhat philosophically minded cognitive scientist. And to me that's just the wrong question. The question is do you have the flexibility, not can you do the tasks? Like maybe you're memorizing the task. How are you doing them?

It matters and so to me that was like a kind of functional definition again tied to economics but not really to the underlying cognitive things and not not the way that it had been constructed within the field of computer science and artificial intelligence So all of these things are in play now in part because of this weird Contractual clause that they should not have agreed to in the first place. There's a popular discourse that says that

that you're wrong because, well, it's just about scale. So if we scale up-- - Well, let's clarify which things, I mean, lots of people think I'm wrong about many things, but let's be, let's focus on the, so,

Something that I wrote in what I think you're alluding to is I wrote a piece called deep learning is hitting a wall in 2022 and people hated it and not just like your average fool on Twitter who always hate everything that I say, but like Sam Altman hated it and made fun of it and Yann LeCun made fun of it and Greg Brockman, who was then the president of OpenAI, people came after Elon Musk, you know, named me personally and came after. A lot of people really hated this, but if you go back,

And look at what the paper said. The core of it

was about scaling, this idea that if you just keep adding more data and more compute, we'll get to artificial general intelligence. And the paper was written in the time of GPT-3, and it said, I don't know when this is going to end. Maybe it'll be soon, I'm not sure, but it's not going to keep going. And people are talking about scaling as if it were mathematical law. There are some iconic graphs that came from Kaplan et al. at OpenAI in 2020 or 2021,

in which it appeared you could predict things. And then there was these papers by Google-- I think it was Google Brain at the time-- on the model chinchilla. And people thought, I mean really thought,

Thought to the tune of half a trillion dollars like not just casually that you could predict what a model would do purely by knowing essentially the size of the data set and how many You know GPUs you put on it and what I argued in 2022 was that was not really the case that what we saw these were not physical laws like gravity or something like that that hold throughout the universe but

These were just generalizations that were true for a little while in the way that Moore's Law was like that. So Moore's Law held for a while, and then it stopped. And I said, this is going to stop. And then the next part of the paper was like, here are the problems that's not really solving. And I talked about hallucinations, and I talked about problems in reasoning, maybe a little bit in planning indirectly. And I said, these things are not going to go away simply by adding more data. Like, I understand how these architectures work.

I anticipated hallucinations in 2001 and just acquiring more statistical data is not gonna make this go away. I don't know how many people even read the paper who attacked it. They really didn't like the title, Deep Learning's Hitting Wall. But it has in fact mostly gotten stuck there and people like Lacoon who used to make fun of me now go around giving talks saying exactly the same thing, saying, you know,

we're not going to get to AGI with LLMs alone. He calls them an off ramp. I said wall. He says off ramp. He thinks that's better. Okay, fine. This is a true story. But, you know, cause he's at meta, like he gets more press for what he said. Yeah.

So a bunch of people have actually realized this in fact in November Satya Nadella gave a talk or maybe was early December gave a talk at Microsoft Ignite in which he said almost exactly what I said in the paper he made the same point they're not physical laws that Their empirical laws that that hold for certain period of time and so like a lot of people in the valley or the tech world have come to realize this probably actually the case there's a new thing and

So everybody's kind of realized, hey, we're not even getting to GPT-5 yet, right? We should- We got it at 4.5. Yeah, there's a side note on that, which is,

Why do we have GPT-4.5? It's because GPT-4.5 was supposed to be GPT-5. There's been some writing by Jeremy Kahn, who lives here in London, I think, looking at this history in some Wall Street Journal, and they had a project called Orion that was supposed to be GPT-5, and they did this math on these equations that I said they should not trust. They put in billions of dollars, not trusting me to not trust them, and it didn't work.

They did not get GPT-5 level performance, which by their own measurements should sort of feel like magic. And so they called it something else. And in fact, there have been like seven models since, from OpenAI alone, and another 20 or 30 from other parties. None of them are really what people were hoping for for GPT-5. OpenAI knows that if they release something and they call it GPT-5, and it still hallucinates and makes dumb errors and whatever, there's going to be hell to pay.

And so they don't want to call it that. So they've instead called their things like GPT-4-0, GPT-Turbo, GPT-Turbo-Big, GPT-Turbo-Little. Come on. Computer scientists know that naming things is a very hard thing to do. You could at least just stick with numbered versions. But in any case, they don't have the guts to call any of them GPT-5 because they're not good enough because scaling ran out. So, yeah.

There's going to be an asterisk here. Scaling in the original sense that I was talking about in 2022 that was from the Chinchilla equation and the OpenAI paper simply does not work anymore. They are not getting the expected returns relative to the amount of data that they're putting in. Can you clarify why? So why does increasing models, the size of the models, why throwing more data at it, why won't that get rid of hallucination? So I think there's a few different reasons.

things to say there. The first is these models just aren't going to solve hallucinations because they're doing the wrong kind of thing. They are trying to match a database of text or text and images or whatever. They're trying to get what's qualitatively there, but they explode everything that they see into little bits and then reconstruct them. The reason that they plagiarize sometimes is because the statistically most probable thing is to plagiarize something.

Some of the reconstructions make sense and are true, and some of them aren't. When I gave a TED talk, I gave the example of one of these systems basically saying Elon Musk died in a car crash in 2018. And this is obviously not true. There's lots of data, right? I mean, the man tweets, like, what, hourly or something like that? He's been seen in the Oval Office. And he's in every newspaper on the planet and all the time. Right.

He's been seeing the Oval Office. I mean, the error happened two years ago, but whatever. We imagine these systems to be like us, right? And what a person would do if they were in a newspaper and they were fact-checking, they'd be like, that seems kind of weird. Let me look that up. Let me verify it. These systems don't actually do that. They are just mimicking things that sound kind of plausible in context. They don't actually have the concept of truth at all. Adding more data does not change that at all.

It gets a little bit better because the statistical tracking for the world gets better, but they don't really know how to do things like verify. They don't really know how to do things like reasoning, which is another kind of ledger domain with statistics. And so that doesn't help. That's one issue. That's why fundamentally hallucinations are not gonna be solved in this way. There are different band aids we could talk about if you wanna get into it. Another issue is they're kind of running out of data.

They all convinced themselves that these laws were true as they moved from, like, training these models on making up these numbers. I don't know the real numbers because they're not published. But let's say a billionth of the Internet. They moved from a billionth of the Internet to let's say it's a millionth of the Internet, like of all the text recorded on the Internet. And they got really big leaps from that.

And then they moved, let's say, from a millionth to a thousandth of the internet. And they got really big leaps, and they were very excited. And then they moved from a thousandth to a tenth of the internet. They were really excited. And then they moved from a tenth to the whole internet, and they don't have anywhere else to go. And so there's a great article by Cade Metz and some others in the New York Times. The reporters are more reliable than the commentators. And what he reported was that...

He had a bunch of vivid examples like Greg Brockman, who was then the president of OpenAI, personally downloading a bunch of YouTube videos knowing that he was probably violating licensing because he was so desperate to get more data. What's the word for it?

2022 to 2025 period has been characterized by an absolute greed for data. We should talk about the copyright stuff, especially here in the UK. There's an absolute lust for data, and people stole stuff left, right, and center. You probably saw about how Meta is stealing everybody's books. They stole 101 of my books and articles in one database alone. No permission from me, no

And no repercussion, nothing. So far, no repercussion. Your government is maybe poised on saying, ah, that's all right, steal whatever you want. Greatest land grab in history, if they get away with it. Let's come back to that part.

So there was this lust for data, but they kind of ran out as part of the problem. So one is in principle, it's just the wrong architecture to be doing fact checking. Aside from the hallucinations that are other problems too, those also haven't been solved because there's not really more data. I think that's one of the issues. So it's kind of architecturally wrong. There's not really a whole lot of data. Let's talk about synthetic data now. So the one thing that's very popular, maybe we'll talk about inference time compute later,

is you make up your own data. And lots of people think this is a new idea, and they get very excited about it. It's not actually a new idea. I first read about synthetic data in a paper by a guy named Peter Foldiak in the early 1990s. He created some synthetic data and envisioned having an object pass back and forth, kind of algorithm move it back and forth, in order to make more data for his system. I think that was the first time I saw it. It's not new.

In 2016, I sold my company to Uber. I stuck around not very long, but briefly. There was great interest at that point in the driverless car business. What had happened in the driverless car business, which has so many parallels to what we're talking about, is people made a prototype. It kind of worked a bunch of time, but it didn't work reliably enough.

One thing I vividly remember is Stephen Levy had an article in wire. It was fantastic. He visited the way mo campus where they were driving their cars around all day on private roads and they had just figured out how to get the system to recognize patches of leaves.

And what I said when I saw that, maybe his article was 2015, was we're going to have endless outliers, unusual things like piles of leaves and differently shaped leaves. And it's going to drive these systems insane. So in 2016, I said, you're never going to solve the edge cases. And some people said, ah, we're just going to use synthetic data. We'll make it all up. But there are two problems with synthetic data, or maybe there's just one problem.

The problem is you never anticipate all of the circumstances you need. So my favorite example of how a driverless car has failed is I

a Tesla, they now have this function called summon. You remember when Elon Musk said he was going to have a Tesla drive from New York to Los Angeles, or Los Angeles to New York. This is like 2019. And he promised it in like 2020. And it's still not here. No, you can actually, he actually made a version of it, which is, it could go across a parking lot, which is not, you know, Los Angeles to New York, but it's something right. Give the man some credit. And cause he doesn't get enough attention in the press, give him some credit. And so,

He has this summon function and somebody used it in an unusual circumstance, which is they brought their car to an airplane trade show.

The rich guy with his Tesla uses this app and there are jets on like the airplane trade show is on, you know, an airport that's been closed for the day, a private airport or something like that. And so he says, Hey, look what my thing can do. And it runs his Tesla that he summons runs into a three and a half million dollar jet, which you can see on YouTube video. And you see the jets spinning around after the car ran into it. No human being runs,

would do that. I mean, we're talking about it's a mostly empty airstrip. It would have been easy for it to go around. There wasn't a lot of traffic. There's no obstacles, except for the jet. And a human would be like, huh, I've never run into a jet. None of my friends have had a traffic accident involving a jet. But those things are expensive, and they're big, and they might hurt my car or me. This is a really dumb idea. But from the perspective of a system that does pure mimicry, analogous to an LLM, although not identical,

it's not in the training set. And even though people at that point had been, like there are whole companies making synthetic data for cars, people have been doing this for several years, and these companies have billion dollar valuations or whatever, nobody thought to put in the training set this specific example. And the point is that reality has too many different things. Nobody's ever going to think that an insane, demented, elderly guy is gonna make a trade war that is gonna devastate the world economy.

That's not going to be in your training set because any economist is going to say that's absolutely insane. It doesn't make sense. It's going to cause a recession. Don't do that. And they're going to think if he would do that, that stock market is going to fall and he would realize that that doesn't, you know, that's not good for him. There's just no synthetic data that would ever get there. No synthetic data is going to envision this particular scenario. So,

I wasn't speaking to anybody specific. No, you know. So the real world is complicated. There's never enough synthetic data. So I'm going to sidle into old scaling is dead. It just doesn't really work.

there's a new thing everybody talks about. - And even, like you said, they are also backtracking. - So a lot of them are backtracking, but they're also, there's always hope for another thing. And the hope for the new thing is something called inference time compute. And so you have these new models like O1 and O3 and R1 and all these creative names that don't make a lot of sense. But what these systems do is quote reasoning. And I would keep the quotation,

marks around it. And on some problems, they do better than their predecessors. So there's some real progress. But they're relying very heavily on synthetic data where you can get it. Where can you get synthetic data? Where you can verify it, where you know what the right answer is.

So you don't know what the right answer is for what's going to happen to the world economy if this guy does this thing. But if it's like 12 times 12, you know how to create that data. He's like, okay, it's 144, we're good. And so if you look carefully at all these demos that people have done of these new systems, all the benchmarks they're showing are basically math, coding, things like that, where you can generate new data and you can know that the generated data is correct. And so...

It's a trick to get you more data in a certain context. It doesn't kind of generalize the world. Is just benchmarking that is the trick there? Or is there a trick, a fundamental trick on the model? Because then you could argue that, well, I can go and charge a PGA, I use O1, and it's actually telling me what's reasoning, what it's thinking about. That's a separate issue. That's what a tool would say, would give you as an impression. So there's two separate things. One is when these new reasoning systems work,

it tends to be in places where they can create synthetic data. And where they can't create reliable synthetic data, they're not getting that much boost out of these systems. They're still having all the problems. If you look carefully at the 01 blog when they released it,

On some problems, it was actually worse than GPT-4. So it's much better in these cases where they can do this kind of trick. Now, the second thing is the models are super interesting because they tell you the reasoning that they're doing. But there was an amazing paper that just came out a couple days ago. I don't have all the details in my head. But they basically showed that the systems are, to some extent, lying about what they're doing. And lying is too anthropomorphic, but it's hard to quickly say. They're misrepresenting. It's not a form of hallucination.

They're actually hallucinating what they're doing. So like one example that I do remember is a system was asked on one of these benchmarks to visit a website, do a thing, and report back on it. So it gives you the right answer in this particular case, but the website is down. Hmm.

So it is reported that I have gone to website, gathered data, done this thing, but actually it's probably memorized the data that came from the website, because the website isn't there anymore. So it's not actually doing what it's supposed to do, and it's not doing what it tells you. People have talked a lot about agents, by the way. That's gonna be the big thing in 2025. I'm on record as saying they're not really gonna work reliably.

There was another iteration of agents a couple years ago, something called AutoGPT. People got super excited, and the idea was you would use one GPT, what a horrible name, generalized, whatever, transformer, to...

call a bunch of others. You'd have kind of like an army of GPTs and people got super excited and they're like, I'm going to run my whole life with these things. It's going to take care of my banking and it's going to take care of my airplane reservations and my restaurant reservations and my dating life and blah, blah, blah. And then about a month later, they all found that everybody found the same thing, which is they don't really work because they'll do things like

They'll say, okay, you want me to transfer $100,000 from this account or $10,000 from this account to that? And they'd say, okay, I did.

I did it. But they hadn't, right? You know, they just, okay, I did it is like, you know, something you say in a moment like that when you're helping somebody and it's in the transfers or whatever. But they didn't actually have anybody's passwords, thank God. And like, you know, it's going to be worse when they do, right? Because then they're going to be like, I made the transaction. You're going to think it's okay, but they'll make the wrong transaction. You know, they'll sell when you want it to.

buy or whatever. So the hallucination problem and the lack of kind of a deep semantic understanding of what they're talking about was crippling to auto GPT. Now people are just doing the same thing over again and they're calling in an agent. It's going to have the same problems. And in fact, a bunch of the initial reports are like that, that they're not reliable. They're very cute, but they're telling you what you want to hear because that's what other people have said in those contexts. It doesn't...

mean that they're actually successful. It's sort of like when you get an email and somebody writes back, looks good to me, and you know they haven't really looked at it. It's a little bit of that. Sounds good.

I can relate to a lot of what you're saying because some of my students are in the room. We're doing an experiment in vibe coding in one of my courses, and everyone struggles because, of course, and it's related to that process because if you trust the AI to do one thing that has done amazingly well in the past, even with the Pac-Man example that we're talking about, it doesn't do that well.

you can kind of feel confident that, okay, so it generated stuff that looks credible, it runs in my code or some other applications, but then when you know enough about that concept, when you know enough about our universe,

you can spot the BS right away. So you can see that doesn't work. - There's this psychological effect called the halo effect. I don't know if people recognize that. The halo effect is like you like something about somebody and you think that everything else about them is good. It's a very superficial way of reasoning. We're all vulnerable to it. Somebody's good looking and we'll sort of trust that they're honest or something like that even though there's no evidence that that person

particularly Zanas. People do that with LLMs. In fact, in my book Rebooting AI, Ernest Davis and I coined something we called the gullibility gap, but maybe we should have called it the Eliza effect. So one of the first AI chatbots, or probably the first, was Eliza in 1965. And people would type in stuff about their relationships, and they would be like, I'm having trouble with my boyfriend. And they would say, well, tell me more about your boyfriend. And people would be like, wow, this

I mean, some people actually thought it was a person. In fact, it was dumb as a box of rocks. It just had keyword matching like you used to do in Google. And so if somebody said boyfriend, then it would be like, tell me more about your relationship. So kind of change the word, use a related word. Or you say something about your mother, and it says, tell me more about your family. It was really superficial. It didn't understand anything.

of these concepts but people would get sucked in by it and some of that was the halo effect some of that is like we did not evolve to interrogate AI I mean like I interrogate AI but I you

worked hard at it and studied cognitive science and been looking at these things for 30 years. So it's second nature for me to look at a system and say, okay, I know it looks good, but what are the real limits here? But most people are not trained to do that. They just type it in and it feels real. The most brilliant thing OpenAI did was to have CatGPT type things out one word at a time, which made it look like a human. Now there's a new graphics thing and it draws out a pixel at a time, maybe to try to get a little bit of the same effect. I'm not sure.

People get suckered into those things. They're very gullible. We've just not evolved to do that. We were evolved to spot a lion coming at us and to run. We have lots of shortcuts in our brains to recognize certain problems, but not others. Kids don't know that electrical outlets are dangerous because we didn't evolve to do that. We didn't evolve to try to understand what's AI and what kind of AI is it.

People don't have that training. We really should have AI literacy campaigns that teach people these things hallucinate. It is incredibly baffling to me that I still, two and a half years after ChatGPT, still see academics triumphantly saying, I discovered that this thing hallucinates. I'm like, how did you just discover that now? Maybe they just started using it. They just used it for the first time. They read nothing that was been written. This is amply...

It's at least sampling the technical literature has been a lot of the popular but somehow the message doesn't percolate enough and everybody goes into these systems Initially thinking wow it thinks like me and it doesn't it's very difficult to break people of that And that's why you know masa songs about to put 30 billion dollars I mean maybe there multiple reasons, but you're about to put 30 billion dollars into open AI into you know Bigger versions of this doesn't make any sense to me. There's

So a crude characterization of what you're saying is that okay, so these systems they don't work the way we think they are and the way they are portrayed in especially media and the way that AI leaders talk about. Most of what you're saying is that fundamentally they don't work that way, right? So they won't achieve that level of generality that we want of them and that we think that we will have.

But they also have... Let me insert one sentence. But I'm not saying that no AI will do that. I'm saying this approach is not going to get us there. But then we have also real, real consequences and dangers in the widespread use of AI. So I'm reminded of the research that came out recently from Microsoft Research and Carnegie Mellon where they had knowledge workers perform a task and then they saw how they were using AI. And their conclusions in some sort of way is that

that essentially we're getting dumber when we use AI. So we're a bit, we're not thinking critically, so we're not engaging in that kind of thinking. - Yeah, in a way that people who use calculators can stop doing mental arithmetic. People are stopping doing critical reasoning, 'cause they're just typing stuff in, taking whatever the system says,

face value. And in a way this is more damning because the AI can respond confidently about any topic. Is that why, like in a performative level, when as a user I'm performing that, trusting the AI so much,

I'm losing my critical abilities because of this confident tone of the AI. What do you attribute that confidence that a user would have? I mean, there are multiple reasons why the users are seduced. So they sound like people. And this goes back to the previous question, part of it. They sound like people. And we're accustomed to taking a small amount of evidence that something is a person as opposed to anything else and making lots of inferences. They could be a potential mate or they could be a potential business partner or whatever. Like,

you talk to somebody for 10 seconds and you conclude various things about them. You just assume they're human, you get a little bit of data that they're human and you make broad inferences about what that means, that they're human, about what you can trust and what you can't and what kind of question you can ask. You meet somebody on the street and you think, well, that's a person, I can probably ask them for directions. You don't ask for their educational background before you ask for directions, right? And then machines kind of sweep in under that

naive filtering system and exploit it. So people tend to think that I see a little bit of evidence that this is intelligent, it must be intelligent. It's a bad inference but they tend to do it. There's a great example of this from a few years ago by the way. Someone who was running Waymo showed these videos of the early days of Waymo, before it was even called Waymo, when it was still part of Google, driverless car company.

When they first had prototypes that kind of worked half decently but not that well, they let employees use them. These employees were engineers, right? They were smart people. And they said to them, you can take this out. It'll drive you around Menlo Park. But don't trust it. Like, it's demo. And I don't know if it was unbeknownst or not, they had cameras in the car filming the drivers. They wanted to see how drivers reacted. It was an example of the driver seeing a little bit of evidence that this thing is intelligent,

And inferring that it must be like a human driver when it wasn't. And so he's got, I saw this video once. I don't know if it's available on YouTube. He has these engineers like reaching behind to get there. Like he said, always look forward, always drive, do not trust this thing. They'd be like, yeah, yeah, yeah. And then after like 45 minutes in the car, they'd be like reaching for their briefcase in the backseat, not looking forward and so forth. So they had over attributed to use kind of psychological terminology, over attributed intelligence, um,

to these prototype driverless cars. And so that's part of what goes on. Another part of it is they have been

trained with reinforcement learning and so forth to sound confident in their answers. And people take clues about confidence that might work. I don't think they're great even in the real world. This is why people get fleeced all the time and why grifters are so successful. But people will take things that are said with authority as if they were true. And these systems say everything as if they were basically an encyclopedia.

And most people are naive and they think if it sounds like an encyclopedia, it must be true. Of course, it doesn't actually make it true and a lot of what they say is garbage. But there are lots of ways in which at least the untutored human being is psychologically vulnerable. There's a guy named Balder Bjornsson or something like that. I wrote a piece also in my sub-sack that you can find comparing people who like psychic shows...

to people who like LLMs. And he said, look, if you like, he seriously wrote it, it's an interesting piece. He said, you have a self-selection bias. So some people don't go to the psychic shows in the first place. I don't need to see that. And so it's the people who want to believe. And then the people who want to believe, when the psychic gets it right, doing so-called cold reading, they're like, wow. When the psychic gets it wrong, they're like, yeah, okay. Okay.

They're not so good at looking at the numerator and the denominator. You're getting people in the back row, let's be honest, who aren't good at numerators and denominators and want to believe and whatever. And there's some of that going on with LLM. People want to believe that we are in this spiritual moment when machines transcend... There's a word for it, the singularity. I don't know who made this up, but I like to call it the nerd rapture. So...

A lot of people are looking for the Nerd Rapture and want to be part of it. It sounds fun, though. I mean, the problem is that the stuff we actually have is not that. Is that five minutes and then questions? Or did we just... And then questions. So people want to believe that they're part of this special moment. Hi, I'm interrupting this event to tell you about another awesome LSE podcast that we think you'd enjoy.

LSE IQ asks social scientists and other experts to answer one intelligent question. Like, why do people believe in conspiracy theories? Or, can we afford the super rich? Come check us out. Just search for LSE IQ wherever you get your podcasts. Now, back to the event. Okay, so...

What would be an alternative way? What would be a better AI proposal? What would be a better way to build AI? So what we have to do at a minimum

Is we have to marry two traditions in AI that have hated each other for decades So one of them is called symbolic AI and that just looks like classic computer programming lots of knowledge graphs And if then the statements and so forth, it's very out of fashion. We do actually use it in certain places So Google search was mostly until recently made up of that stuff and still has a lot of that I think

GPS systems are just a bunch of if-then statements. They're AI that we actually can trust and are reliable. They help me navigate London. I don't have the knowledge, but I can get around here. No problem with GPS because that's a piece of AI that actually works. So

We need to take some of that and put it together with the neural networks. These are two approaches. Some of you will know Danny Kahneman's system one and system two distinction. So system one is stuff that we do fast and automatic. And system two is more deliberative and reasoning-like. So neural networks that are popular are kind of like system one.

and the classic symbolic AI is more like system two. We need to bring those together. Humans put them together. We don't put them together perfectly. I can talk about that for hours if you want, but we do use both of those systems.

AI needs to have a way of putting that together. The strengths and weaknesses are almost perfectly complementary. So the neural networks are very good at learning, absorbing massive amounts of data, but very poor at reasoning and planning and so forth, and very poor at abstraction. Whereas the classical AI stuff is very good at abstraction, is very good at planning, very good at reasoning. It's not perfect. There's still improvements, but it's very, very good. And some of that stuff gets used all the time. People don't even talk about it anymore. It's just engineering now.

And it's very, but the system two stuff is very poor at learning. Like people don't know how to build a system two system at large scale the way that they know how to build a neural network. And so part of what we need to do is to have a reconciliation. But these guys all, it's mostly guys but not all, you know, hate each other and have hated each other for a long time. You could ask why. It's really about money and resources, right? So, you know, I don't even want to talk about the United States'

current attitude towards scientific funding, but ignoring the last couple of weeks, which are insane. The way things work is you apply for grants, for example, and if you get the grants thing, you can get graduate students, you grow your empire, and like every scientist realizes that this is how it works. And so that...

puts you in competition with people that have other perspectives. And that competition has really skewed things. And so you have Geoff Hinton, who used to live here in London, who has been incredibly hostile towards making any kind of integration between the two. You also have DeepMind, which is based here in London, which has mercifully not listened to his hostility to it and built some systems like AlphaFold that actually do what I'm describing, just effectively won the Nobel Prize. So there's some steps in that direction.

of what I like to call, and several of us like to call, neuro-symbolic AI. I think that's part of the answer, it's not the whole answer. - That was so, the technical challenge, right? Of building an AI, would that lead to AGI, by the way? Would we want to lead to AGI? Do we want that? - That was a lot of questions to pack into two minutes. - We have one minute. - One minute. AGI might be super helpful.

The only way we want anything like that is if we can align it to human interest.

Current AI, we can't really guarantee that it'll do what we want. The most hilarious example of this is the Apple system prompt, if you know what those things are, says don't make stuff up. But it does anyway, right? So we can't even get these systems to do things like follow a simple instruction, like don't make things up, let alone something like Asimov's law, like don't hurt people, which I've oversimplified. We don't know how to make a system that can calculate the consequences of its actions and actually...

obey something, right? They're these, as Bender and Gebru call them, stochastic parrots, right? They're unpredictable. Stochastic means basically unpredictable, statistical.

We just don't know what's going to come out. They're not aligned. So I think if we could make better systems that can do that, they might help a lot with science and medicine. Maybe it would be a good idea. I think just having lots of LLMs everywhere, which is where we are right now in the world, I think is a bad idea. We haven't even talked about all the risks and problems of having unreliable systems. I'll just give you one quickly. If you take these systems that cannot be trusted and you put them into weapons systems,

which is now happening, you know, a lot of innocent people are going to die because these systems are not reliable reasoners and hallucinate things and so forth. So, like, we're headed, I think, to a bad place. Maybe we will move past it. The other thing we haven't talked about, maybe people ask some questions, is there's a whole kind of power and political agenda. I was going to ask that because, like, there's the technical problem and there's, you know, the question of whether we want this kind of AI. But, yeah, how would we mitigate the problems that we see today with AI?

these companies and holding so much power and so greedy for money essentially at a cost. - I mean, I think the only chance we have of that is if the people of the world stand up and say this is not acceptable. We had a dress rehearsal of this with social media and at least in the United States, there's something called Section 230 that basically gave social media companies exemption from responsibility for algorithmic feeds that basically

polarized the world and gave us this mess that we have right now. People didn't stand up and say this is not acceptable. Same thing with privacy. People just decided, okay, I get some free services like Facebook and I'm just going to give away my privacy. Let me give my deeper secrets to Claude. And now people give their deepest secrets to Claude or ChatGPT or whatever. So like, we're the frog that boiled and unless the public stands up, I'm really not optimistic at all because the governments are clearly not going to do that much. More in the EU...

Not so much here and nothing in the United States. My whole last book, "Caming Silicon Valley," people can ask me a question if you want, is like the core of it is 11 proposals for specific things that we should do. For example, we should have a system where if you're going to roll out a big new AI model, you should be required to submit it to outside experts who evaluate the costs and benefits.

Here's something I haven't written about at all. Somebody just told me. The new open AI system that just released, it does graphics, can make porn by superimposing anybody's face you want on a naked female model. This is one of many examples of kind of, there's a nice, I'll just say shitty things that you can make these systems do. And there's no weighing of the

the benefits versus the costs. I mean, you could make an argument that helping students with their homework offsets that particular problem. I would not. But you could-- and I'm being a little unfair there-- but you could make that analysis. But the point is that somebody outside the company should do it. I think it's one more example, and then maybe we'll go to questions, which is that one of the recent models, OpenAI actually said this elevates the risk of bioweapons attacks by people using this stuff to cause harm.

that should somehow pass through government, right? Like that's a serious thing. I think they probably exaggerated so that people would be like, wow, this system is so smart. But putting that aside, at some point we will have a system that is genuinely, like seriously elevates risk of bioweapons, for example.

It shouldn't be up to Sam Altman sitting in his Koenigsegg $5 million car to decide, okay, I don't care. I'm elevating that risk, but I'm going to be able to buy another couple of these cars. It just shouldn't go that way.

That brings back for me Shoshana Zuboff when surveillance capitalism in some of the ways where she expresses that one of the ways that this new form of capitalism operates is by exactly finding the places where it's unregulated and there's a non-contract where they see everything about us and we don't see anything about them.

And there's some of that in there as well. One sentence on that, then take the question. I think that the only hope OpenAI has of being a successful business is to go much more directly into surveillance capitalism. So if you look at what they're doing, they're losing money. They just announced that they made $12

$12 billion, I think, last year in revenue, or this year, I forget the time period. But they didn't mention the costs. And the costs are much higher. They've invested tens of billions. And the field as a whole is putting like $500 billion. Meanwhile, the prices are going down because everybody basically has the same recipe. We just know what we call technical mode, so everybody can do the same thing. So they're actually in a serious financial problem. And DeepSeq was just one big scare recently. DeepSeq was an example of a big scare on this.

And so, and they don't actually know how to build AGI. So the only thing I think they can do to make massive revenue

leveraging what they've got is to sell all of that data of people typing in, you know, my girlfriend did this and that, like really deeply private data, selling it to federal governments and political advertisers and so forth. I will not be at all surprised. Note that they bought a share in a webcam company. They put Nakasone, who used to be in the US NSA three-letter agency, whatever you call it. Full-on surveillance capitalism, for sure. So I think that they're headed towards surveillance capitalism. Like what you saw with Facebook is just an appetizer.

On that cheery note, we're going to take questions. So our viewers will be walking around the room. I'm going to juggle for you at the end. I'll be taking the questions in trios so that we hear a few questions here and there. And then we give our floor to our guest. One thing, mention your name, state your affiliation if you have one, and make it actually a question. Don't make it a comment.

The person in the globe over there first and then we're going up there as well. Uh, hello. My name is pedro I'm currently at king's college london. I was previously at uh, universidade federal de rio canto in brazil, uh and my supervisor that was a big fan of yours One thing like the current ai systems like the database on llms they can use tools now Right with these agents. Yes ish somewhat. They can use them. The the thing is you like

The issue would be that the reasoning behind these models using the tools

your argument would be that they wouldn't work. You need to change how the reasoning is being done so that they can use the tools instead of the AI tools that can't reason, other quotes, using a reasoning tool, for example, because they can't, for example, use an LLM to-- - So just to make it a little more concrete, there are cases, there are a lot of cases now where people try to hook up, and to clarify for the rest of the people, try to hook up LLMs to other pieces

of software like Wolfram Alpha or things like that. - There's a new way of doing that. - Yeah, we don't have to get into those details, but yes. So the problem is you have the LLM on the front end and sometimes it tells the tool the right thing to do and sometimes the wrong thing to do. So when people experiment with this, for example, in math problems, the math on the back end, they've done this with Wolfram software, for example,

works perfectly. But sometimes the system still feeds the wrong thing into the system because it doesn't really understand the language. It still has problems with compositionality, understanding the structures of holes and stuff like that. So I've seen a lot of experiments like that. I've never seen any of them be reliable. In my sub-stack, there's

an essay with the word tools in the title that went through some examples. It's a little bit dated now, but I think it's still the general problem is, yeah, you can hook these things up to tools, but will they work? Or another example is, you can hook them up to writing Python code, but if you don't specify exactly what Python code is to be written, then the thing that is doing the translation just doesn't wind up in the right place. I've got a question from the balcony there, so we had... Yeah, so...

- Hello, I'm David Gerard, I'm a PhD student at UCL in neuromorphic and optical computing.

So tonight we've talked of AI but mainly large language models. So my question and especially since you're also professor of cognition has to do with your take on more biologically plausible framework like spiking or that sort of things. Do you see a future for this or is it just a funny research topic?

For now it's a funny research topic. The problem right now is we don't really know that much neuroscience. I mean we have billions of, depending on your count, billions of billions of data points. But we don't really have a lot of theory about how it all works.

My view has always been that we are going to need to advance AI in order to understand the brain. We have 86 billion neurons and trillions of connections, and we don't even know the right kind of sampling unit, whether it's an individual neuron or the proteins, you know, individual proteins within the nucleus. I mean, there's just, there's a lot of unknowns. And so, you know,

Current AI is taking something from some neuroscience that we knew in the 1950s about hierarchies of feature detectors, but we still don't know that much about how it really works. Neuroscience is data rich and theory poor.

If people get better theories that they can, you know, show really, predict a lot, that might help the field of AI. But so far, I think the value has been limited. I think there's some value in looking at neural spikes for efficiency, but nobody's really... Despite a lot of effort, nobody's really made that much out of it. So, like, yeah, I think in principle that could be super helpful. In reality, neuroscience isn't really there yet. And I think we need...

something with the causal reasoning of people and the computational power of big machines to help us to make sense of the data. There's an article in Wired like in the last few days about the open worm project and it's a really depressing article. You thought I was depressing. The worm has, I think it's 302 in the Hermaphrodite version where we have a full wiring dam diagram. People have been working on it for 15 years

The model that they have now, 15 years old, it can do like, you know, 2% of what a worm actually does. And it runs like 10,000 times slower than an actual worm. And like, that's a worm. That's 302 neurons. Like, we're nowhere near knowing how to build like a detailed model of the brain. Some of you will remember the Blue Brain Project that Henry Markham pushed that was big in Europe. And I wrote stuff in 2014 basically saying that'll never work. And it didn't work.

More questions from the floor? Let's see, let's see. Very deep in there. Yes. I think it's you. I deem that it's you. It's here by you. Hello, my name is Relina and I work as a lead data architect.

and also into data science a bit. So you did say that the traditional Go-Fi has always been at war, there was a resistance in marrying that up with neural networks, which is better at learning but not good at reasoning. Do you think there's anything else other than funding and that, why didn't anyone think of it before because clearly,

I mean, people have thought about it, but you really do have people like Jeff Hinton who have...

have absolutely ridiculed the idea. And Hinton is very influential. So he stood in front of the European Parliament and said, doing that is like building hybrids when you can just build electric cars. It's old fashioned. It's stupid. And his voice has really loomed large over the field of machine learning. But if it's old fashioned, it's just the go-fi bit of it. But you're marrying that up with neural networks. So that doesn't really hold much water, his argument.

I mean, I think his analogy is stupid, but he is, I mean, you can quote me on that. I've said it before. I just, but he is really kind of like, he has really influenced a lot of people to think that working on symbolic AI is backwards and stupid and,

almost evil and he like Constantly attacks him when that constantly he's frequently attacked me personally for having suggested this he's given lectures when I am NOT there talking about you know he'll use his example sentences punching Marcus in the nose like he's the man has

Sorry, I didn't want to dig up bad memory. I mean, I take it with a sort of compliment that this man who has won the Nobel Prize has three Gary Marcus quotes on his web page. Like, he's very threatened by my advocacy of neurosymbolic AI. He has more quotes from me than his...

prize protege, Yann LeCun, on his webpage. I mean, it's weird. But, you know, people pick up on that and that has had an influence. I mean, it's also hard, right? Like, some people have had pretty good results, mostly in narrow domains, but, like, if you look at it, like, alpha fold is neurosymbolic. You know,

To his credit, Demis did not buy the orthodoxy and is considered a lot of different things at DeepMind. And then alpha geometry, alpha proof, these are all neurosymbolic systems that are actually working. But it's like heretical to even look at it given the world that Hinton and Lacoon and to some extent Bengio have built. They've been incredibly hostile. You can watch debates these guys have had with me and stuff. The hostility is palpable.

Thank you. I'm over it. Like, I'm not mourning, but it has really cost the field. Hi, Gary. My name is Lauren, and I am a book reviewer. You are? A book reviewer. Book reviewer. I don't know how to do it.

That might actually work better, although not for the recording, but I'll repeat your question. My name is Lauren, and I am a book reviewer. And every single author that I know, whether traditionally published or indie, have had their work scraped and put into Meta's program. They are floundering. Most of them do not know anything.

about AI. They're mad. You have Jake Kristoff calling Mark Zuckerberg to see you next Tuesday. Others are just despondent. But in general, no one really knows what to do. So if you were speaking directly to those authors, what do they need to know and what should they be doing?

there's a couple things there will be a class action lawsuit i don't know if this is public but i know just to restate the question oh right yeah so what should authors do about the fact that their stuff has been ripped off wholesale by meta and probably by others what meta did is egregious i wrote about it in my sub stack in a piece saying 101 of my books and articles have been ripped off um and that was based on an article in the atlantic by alex reisner

And Alex made available something, and I linked to it, where you can type in your own name and it will give back all the things. You might want to, if you've, as I have published under two different names, Gary Marcus and Gary F. Marcus, then, you know, the search engine's not very smart, so you should type in all of them. So I found 60 here and 40 there, whatever. Okay.

So what can authors do about the fact that Meta has done this? Oh, and not only did Meta do this, but they knew that what they were doing was probably illegal, which juries are going to get a real kick out of. They knew it was probably illegal. They did it to save money and to save time.

It went ahead anyway. Also, I will add, they're probably not alone. Probably all of the big or most of the big AI companies did this. I told you about the lust for data. This was part of the lust for data. So they did it in cold blood. In the U.S., I believe that it's a violation of the fair use rules, which are complicated. There's four different conditions, but...

Making something that is commercially competitive, for example, is bad, and they were doing it. And as I said, juries really care whether something was done knowingly or not. And there are transcripts showing that they did know, and probably Zuckerberg himself signed off on this. Somebody named MZ, I wonder who that is, signed off on it. So what can authors do? One is they can sue.

I think that they have a good chance in a class action lawsuit because it was so flagrant and so forth. The other thing is they need to petition their governments and say this is not cool. And it can't be just the authors. It has to be all of us. We need to all say, look, if we let them rip off artists, authors, and so forth,

It's going to be everybody. Whatever you do for a living, if you're not a creative type, but you have a keystroke logger, let's say, which your employer has put there, which they probably had if you do anything that's a knowledge work, they're trying to replace you. That is what they're trying to do, and they don't want to compensate you. If you sit around and you let the authors and you let the artists get ripped off,

then you're going to be next. Everybody should be calling their representatives saying this is not cool. And right now, if it hasn't all been decided here and I missed it, you know, the UK is a

critical player in this because the government is contemplating giving all of this copyrighted stuff to the companies for free which is like you know giving away texas for free because somebody asked for it the ridiculous argument that open ai made to your i think it was house of lords was they said we won't be able to build our super fabulous artificial intelligence which is not

as super or fabulous as they say, unless we have all of this stuff for free. And they left out a really important option there. You know, the mind tricks that they played was to make it sound like a dichotomy. You give this stuff for free or we won't be able to build our stuff. And then in the U.S. they add, and China will, right? What they left out is if they...

have valuations in the hundreds of billions or trillions as Microsoft does, they can bloody well pay licensing fees. Some of you will remember Napster. And when Napster was out, basically anybody could steal any piece of music for free. And the artist said, no, that can't possibly be legal. And you know what? The court said, you're right. It isn't legal. And so we just moved to a different business model. The business model is people license things.

So Netflix licenses all of its content. It pays a lot of money so that it can stream it to you. Apple licenses the content and pays a lot of money so it can stream it to you. Spotify, you know, people don't love what it does to the artists, but it at least gives them some compensation. It licenses...

all of this stuff. We have to have a model like that in AI or it will be the most exploitative thing you can possibly imagine. And so I hope that everybody will stand up and say, you know, British government, this is insane. This is stealing from, you know, a large part of the British economy. You have all your rock stars and like, it's just crazy that Britain would even contemplate this. And it's because they've fallen for this Jedi mind trick of, if you don't give us this stuff, we won't be able to build our cool stuff. Bullshit.

Yes, my simple question. What is the role of big data and AI? You don't think big data starts to grow due to big data technology and AI, they start to grow now.

Because of such amount of data that you have to dealing, governing, etc. What is the law around it? The role of big data. Well, I mean, all of this stuff starts with big data, right? I mean, the AI that we are mostly talking about here basically derives whatever power it has from having massive amounts of data. And we have to decide what the laws are around sourcing that data.

you see european union is moving towards the data space building this type of interconnected data at eu level and then there is an ai act at the same time now how you connect the two this means data or ai i mean data is the fuel by which people build generative ai right you generative ai doesn't work without a lot of data

I'm sorry I'm not quite understanding. So we have maybe more on the balcony actually because we didn't get, can you get someone from the side? Yeah, we'll try to get more people. Yeah. Hi, hello. I'm Joe Andrade. I'm a statistician. But since we are at LSE, I would like to ask a little bit about politics and economy and AI.

So you've mentioned something we are being ripped off. So we are the producers of the intellectual content that feeds the internet. But we are also, and most of us, live on our work being really threatened by the possibility of our work being completely devalued. So do we need to move to a more, let's say, socialistic kind of society where wealth is

Given to people just because we are humans and deserve to live. It's actually ties back with the question that the woman Lauren in the book industry asked, right? There is a another piece of well now that I've used the word I will use it again bullshit that these guys put out there which is that they all favor the universal basic income and

The long game has to be that, right? You know, in the short term, the good news is you probably won't lose your job, depending on what it is. But most jobs are going to be safer for longer than I think the average person thinks. So, like, think about car driver. You know, people...

Taxi drivers have been threatened for whatever, a decade, that, oh, driverless cars are going to replace them all next year. The reality is we have, I don't know, 10 million, 100 million drivers in the world, and the number that have been replaced is like 300 in San Francisco and parts of Arizona. And so, like...

It's easy to replace certain parts of people's jobs, but it's typically hard to replace the whole job because we typically need human judgment to do the whole thing. Not that many jobs are threatened in the short term. In the long term, most of them are threatened. 100 years from now, it's not clear how much gainful employment will be.

you know, manual trades will probably be here for a long time because robots don't understand the world well enough to do plumbing and stuff like that. So plumbers might be here for another hundred years or whatever, but eventually even they, you know, be replaced by robots. And so we will need a universal basic income, but the thing that I think is bullshit is that these guys imply that they're going to willingly share the wealth and they're not paying artists. They're not paying writers. They're trying to steal all this stuff. And now you want me to believe that they're just going to, you know, graciously, you know,

And most are graciously going to give us like tiny little bit of subsistence unless the people of the world unite and say, this is not cool. It's not acceptable. And do you think they're going to give up their like beachfront homes? Like everybody talks about universal basic income. It's like things are going to be cheaper. They're not going to give up their beachfront homes. They're not going to give up, you know, their fancy cars and private jets. The rich people are not going to share that much with any of us. And, you know, we have to ask politically. You know, meanwhile, in the United States, we just put in somebody who's,

doing as much as he possibly can to move the wealth to the wealthy people in a way for the poor and the poor people in the United States are hardly protesting aside from you know complaining a bit about Tesla which is you know a fine thing to do I don't support vandalism but I do support protest but there needs to be a lot more

The whole point of my book, Taming Silicon Valley, was if we don't start speaking louder, the tech oligarchs are going to take over the world. Well, you can see how that turned out. Not enough people took my advice, and it's not good. Let me get questions from this side. We haven't heard from people on the left-hand side here. Front row, in green. Hello, I'm Sonia, and I work in the advertising space. I had a question. From my conversation today, it feels like AI, if it stays in this model,

for a long time it was going to kind of go towards a negative trajectory.

Do you actually have hope that we will protest and we will stick up for ourselves and the rich won't stay in their beach homes and steal from us? I'm not at the most optimistic moment in my life. I think that historically, when things get bad enough, people do protest. But you look at what happens in repressive regimes, and it becomes increasingly difficult

to do very much. It doesn't necessarily become impossible, but it becomes very difficult. So you don't see a lot of organized protest in Russia, for example, because people know what's likely to happen if they protest. And I think that this is...

made worse by the surveillance capitalism. I mean, look at what Elon Musk did to the United States kind of information ecosphere. He took over basically the computers and the data of the United States. This means, for example, that he has access to my home phone number, my home address, my income, what I've done, et cetera, et cetera.

they're going to be able to use AI to spot who the protesters might be. The U.S. government seems pretty unafraid of doing things like looking at people's cell phones at the airports. It gets real hard. We waited a little bit too long here. We have time for two quick more questions. I think, gentlemen in the... Yes, over there. Over there, no. In the black suit, yes. And then we can go up.

Hello, my name is Abiyuddin Okutukun, an A-level student at Baskamma School in South East London and an archive scholar with Imperial College London alongside being the VP of a student-led academic publication called The Quarters. What are the limitations of current neural networks and is there any fundamentally different approach that could surpass deep learning? And tying into this, do you believe the future of AI will be dominated by one model or company?

So, I guess two different questions. So, the first question is what are the limits of current neural networks? At some level we don't fully know, but we have a lot of evidence about that. And I would say that the current limits of deep neural networks is they're not very good at abstraction, they're not very good at planning, they're not very good at reasoning, they're not good at fact checking, they're not good at factuality more generally. They're good at mimicry. That's what they do. And if you can

represent your problem as one of mimicry, you can get it to work. And if you can't, then you tend to have problems. Will one approach dominate? I mean, the truth is we already use different approaches for different problems. So, you know, we've mostly been talking about generative AI tonight, but for example, GPS systems are a different form of AI. Nobody is running a GPS system on a large language model. That would be insane. It would hallucinate routes and like, you know,

It's not going to do that. Nobody is actually dumb enough to do that. They are dumb enough to do things like somebody built a startup to try to do driverless cars just with LLMs and the company went out of business fairly quickly. The AI pin is another example of... The AI pin is an example of like...

believing in magic, right? The AI pin was supposed to like stick on your lapel and it would do, you know, whatever you wanted to do. And the only problem was the AI didn't work. It's like the old joke about a perpetual motion machine. I have everything I need except one thing. Oh yeah, what's that? Oh, one part. Oh, there's one part missing. What is it? It's the one that goes back and forth and back and forth. Like all you need for AI pin is AI that actually works, but we don't have that. So...

Eventually we will have many different forms of AI, not one, and we'll use them for different problems and be a little bit smarter about where they apply. - We have one final question in the balcony there, yes. - Hi Dr. Margus, my name's Charlie Fairfax. I'm a general course student here at the LSE, study back in the states at the University of Massachusetts. Would love to talk a little bit more about autonomous driving. You discussed Tesla.

Of course there's ways to limit hallucinations for safer driving, but there are data limitations as well as edge cases that of course make it hard to achieve full autonomous driving, maybe even impossible. I'm curious to hear your opinion or how your opinion might change when you look at multi-fusion approach, so LIDAR-- - Approach? - A multi-fusion approach, so LIDAR, radar, camera, maybe even thermal.

That coupled with driving and maybe a less erratic environment so maybe like strictly just on highway highways or hub-to-hub transportation We already have Waymo which does a lot of sensor fusion which is part of what you're talking about works sort of well, I mean the thing with Waymo is That it only works where they have incredibly detailed maps as you may or may not know so not just like a

Google Maps level detail but much beyond that including I think I speculate cognitive information like what time does a tram run in this place and Like a lot of details about how a city actually works and then exit very expensive for them to start a new city so they have just announced that they're gonna do Washington DC and

which is still a pretty good weather kind of place. And it takes them like years to get going on a new city. So it's expensive. So they're making some progress on this, but it's still...

limited. We will eventually get there. You are right that limited domains is probably the way to go, but what are those limited domains? We've had monorails at airports for a long time. That's like the extreme version of a car on a track. It goes back and forth, totally automated. The real world gets complicated. China can do certain things like build a new city and just route the pedestrians so they always go on bridges and never interact with cars.

you can do that. But if you want to do London, like I don't know what to do. And that brings us to the end of the event. Thank you all very much for coming today. Thank you, Professor Marcus. Thanks so much. Keep looking at LSE events. We also host our own events at the LSE Data Science Institute as well. It was a pleasure to have you all here. Thank you so much. Thanks a lot.

Thank you for listening. You can subscribe to the LSE Events podcast on your favourite podcast app and help other listeners discover us by leaving a review. Visit lse.ac.uk forward slash events to find out what's on next. We hope you join us at another LSE Events soon.