Is Nvidia the most important company in the world? With Stephen Witt
55:06 Share

This episode is sponsored by Indeed. When it comes to hiring, timing is everything. Opportunities pop up fast. We've all had that moment where you're staring at a gap in your team, knowing things are about to get busy and thinking, we need someone now. Indeed cuts through the noise and gets you in front of the people who are actually right for the role.

So, when it comes to hiring, Indeed is all you need. Stop struggling to get your job posts seen on other job sites. Indeed's sponsored jobs helps you stand out and hire fast. With sponsored jobs, your post jumps to the top of the page for your relevant candidates so you can reach the people you want faster.

and it makes a huge difference. According to Indeed data, sponsored jobs posted directly on Indeed have 45% more applications than non-sponsored jobs. We find that Indeed makes hiring so fast.

When you know you're always going to get such a high level of potential candidates, you waste far less time finding a great fit to fill the role. Plus, with Indeed-sponsored jobs, there are no monthly subscriptions, no long-term contracts, and you only pay for results. How fast is Indeed? In the minute I've been talking to you, 23 hires were made on Indeed, according to Indeed data worldwide.

There's no need to wait any longer. Speed up your hiring right now with Indeed. And listeners to this show will get a $75 sponsored job credit to get your jobs more visibility at indeed.com slash intelligence squared.

Just go to indeed.com slash intelligence squared right now and support our show by saying you heard about Indeed on this podcast. Indeed.com slash intelligence squared. Terms and conditions apply. Hiring Indeed is all you need.

The Hoover Dam wasn't built in a day. And the GMC Sierra lineup wasn't built overnight. Like every American achievement, building the Sierra 1500 heavy-duty and EV was the result of dedication. A dedication to mastering the art of engineering. That's what this country has done for 250 years.

and what GMC has done for over a hundred. We are professional grade. Visit GMC.com to learn more. Assembled in Flint and Hamtramck, Michigan and Fort Wayne, Indiana of U.S. and globally sourced parts. Welcome to Intelligence Squared, where great minds meet. I'm head of programming, Conor Boyle. Today's episode features host Carl Miller speaking to author Stephen Witt about the themes of his new book, The Thinking Machine. Let's join Carl now with more.

Hello and very warm welcome everyone to this Intelligence Squared podcast with me, Carmilla. I am delighted to welcome today our guest, Stephen Witt.

So Stephen is an author and a journalist. He wrote How Music Got Free, which was a book that became a finalist for the LA Times Book Prize, amongst many other accolades, actually. And he's also the author of a new book, The Thinking Machine: Jensen Huang, NVIDIA, and the world's most coveted microchip. Stephen, very welcome to you. Thanks so much for joining us today. Yeah, thank you so much for having me. So an easy question to begin: why this book and why now?

Yeah. Well, I was using ChatGPT in 2022. And just like everyone else, I was like, how is this possible? And I was like, well, I have to write about this. But, you know, OpenAI and Sam Altman, it was a very crowded topic. And I was like, well, what? Is there some other way of looking at this? Another way of like understanding this topic?

And what I quickly learned is almost all of the hardware for ChatGPT was made by a single company called Nvidia. Now, I had actually been familiar with Nvidia going way, way, way back into the early 2000s because they used to trade their stock a little bit and they would make video game equipment and they chronically disappointed investors. They were always kind of like going up and down, you know, very erratic, flatlining stock.

And finally, in frustration, I had sold it a long time ago, 2005 and 2006, just thinking, well, I'm done with this. I never bought it again. When I went back and looked at it, it had gone up something like 300 times. I was like, what on earth happened here? And what had happened was that they'd taken the video game equipment that they built and repurposed it as kind of a low-budget supercomputer. And this low-budget supercomputer turned out to be ideal for running neural networks and ideal for AI.

And in fact, this had made their CEO, Jensen Wong, one of the wealthiest men in Silicon Valley and had made Nvidia the same stock. And Jensen amazed me because this was the same guy from the company back in 2005. You know, usually you go and look and let's say, oh, they had different CEOs, change in strategy, whatever. No, same guy who had been running the company for 30 years.

And he just hit upon this brilliant concept of taking these basically consumer video game cards and allowing users to daisy chain them together to build like a kind of jerry-rigged supercomputer. And that's what the first AI researchers used to build these kind of new neural nets. And then a whole industry grew up around it.

So, he was classically the guy selling the shovels during a gold rush. He was the guy selling the hardware to not just open AI, but anyone who wanted to build an AI platform. Most of the AI firms lose money. They still lose money. They're not making money. Nvidia is making a fortune, a huge fortune off this. It's the best place to be from a business perspective. How interesting. So, you're a stockbroker turned author. Yeah.

Yeah, I worked for a hedge fund. I was a stock analyst and trader. I did that for about seven years. And this, NVIDIA, actually, I didn't even trade it at work. I traded in my personal account because I had been, this will really take you back, I had shorted Enron at work.

Congratulations. That turned out to be a very profitable trade. And when Enron got kicked out of the S&P 500 and went bankrupt for being essentially a fraud, they brought in NVIDIA to replace it and it showed up on my radar screen. I wasn't a huge video game player, but I was kind of adjacent to it. And what I had noticed was that there was a certain kind of customer who was kind of buying these NVIDIA cards and then building homebrew computers around them.

So just in the way almost like a hot rodder would take a V12 engine and build a car around it. These guys were taking these video game equipment and building $4,000 or $5,000 personal computers around it. And so that was kind of a new kind of customer. It was very interesting. What NVIDIA saw was that the demand for better 3D graphics from a certain kind of customer was infinite, no matter what.

how good they made the graphics. The customer would always come back and demand better and more graphics. And Jensen's insight was we have to find the next thing with infinite demand and supply that too. We have to find the next thing that no matter how much power, how much computing power we give to the customer, just like 3D graphics, they're always going to come back and demand more and more and more and more. And they looked around for a long time. It took a long time to find something else with that demand profile. But AI turned out to be it.

No matter how intelligent we make the computer, the customer is always going to come back and ask it to be smarter. And with AI, no matter how complex of a task we solve, it only increases the demands that the customers come to with a computer. They just come with increasingly sophisticated demands. And of course, intelligence is an even bigger market than 3D graphics. Well, Stephen, we're going to get to AI in a second. Before we do...

Take us into the early years for Jensen Huang. So this central figure, you've mentioned him already as being one of the richest men in Silicon Valley, ergo the world. But his is actually a rare, is rare in the sense it's a genuine rags to riches story, isn't it? Because most tech billionaires, I feel like it's a already really quite affluent to riches kind of story. But tell us about Oneida Baptist Institute, his early years, you know, and where this figure kind of came from.

So Jensen's dad was, I believe, some kind of HVAC engineer. He did industrial air conditioning and came to the United States to study with the air conditioning giant carrier. And then came back and was like, we have to move to the United States. Our family has to move to the United States. But they weren't immediately able to secure a green card. So they actually sent Jensen and his older brother alone to

ahead of them to the United States. We sometimes call this being a parachute kid who establishes an immigration beachhead, like kind of ahead of the rest of the family, like a paratrooper. So Jensen and his brother show up in, I think, 1973, and the family's looking for a boarding school to put them in. They select what they think is a prestigious preparatory academy in rural Kentucky called the Oneida Baptist Institute.

It sounds pretty seditious. In fact, it's a reform school for juvenile delinquents. And the founder founded it so that the people in the county would stop killing each other in a long-running family feud. So Jensen shows up to this place and the grounds are covered with cigarette butts. Every kid at the school smokes, basically. And he moves in with his roommate. And on his first night there, his roommate lifts up his shirt to show Jensen the numerous places that he's been stabbed.

in a recent knife fight. So Jensen is just thrown into this wild environment. He's probably the only Asian American person around in any direction for about 500 miles. This is rural coal country, tobacco country in Kentucky. Everyone's poor. Most people are on public assistance, even to this day on this county. These kids are tough. They're tough, hardscrabble kids, and they're racist.

And so they start bullying Jensen and his brother for being Asian. He gets called every racial slur you can think of in the book for Chinese people. But the amazing thing is that Jensen kind of adapts to this environment and succeeds and becomes one of the most popular students, actually a leader of the school. He'd also skipped a grade. So he was younger than everyone and he was shorter than everyone. And he was an outsider who barely spoke English coming from Asia. He was born in Taiwan.

and somehow succeeds in this environment. And Jensen has actually talked about this as being one of the most important and formative experiences in his life. One of the best things actually that ever happened to him. And I think that's because he knew how to succeed in an environment in which it was structured. Take a test, do your homework, get a good grade and kind of progress to the next level. For Jensen, that comes very, very naturally.

Now he has to succeed in an environment in which there are no rules whatsoever. And it's entirely up to him to determine, kind of to suss out how he's going to succeed socially and how he's going to succeed in this tough environment away from his parents. And he was able to do so. That's kind of like the world of business. You know, the market segment that Jensen operates in, it was like a knife fight, you know, especially 3D graphics when he started. And so I think this is really instrumental to his later success. He's often said so.

And he's smart and unbelievably capable of doing hard work, isn't he? To this kind of consuming extent. And that kind of propels him into this world, doesn't it, of kind of computer hardware. But it sounds like initially, Stephen, he's quite square. You know, he's got a house and a wife and a desk job, you know, and he doesn't initially strike one as being this like rule-breaking kind of renegade visionary. Yeah.

Yeah, no, I mean, Jensen is absolutely, he's extremely driven. He's extremely intense. He might be the most dutiful, conscientious person I have ever met.

I mean, he got a job when he's 20 designing microchips on paper. That's how they did it back then. He would draft them on paper in 1984 and just has done that ever since. You know, he's done that continuously for 40 plus years. All he's done is design microchips. But the thing that kind of set him apart, he was very dutiful. He was very smart, but he just had this incredible appetite for risk-taking.

So he would go to the most complex customers at his firm, the ones that nobody else could service. And they would make these difficult requests, design us a microchip that's going to do this. And everyone else would say, no, it can't be done. And Jensen would say, yeah, I'll do it. And he wouldn't even have any idea how he was going to do it. And then he would take the request back to his team and work 14 hours a day, seven days a week until he produced something that was functional and kind of accomplished miracles. In some sense, Jensen, this is all he's ever done in his whole career.

He's gone to scientists. He's gone to the most advanced industrial users and said, I will build you the thing that no one else can build. And I'll get it in on time and I'll bring it in on budget. And he's done that his whole career. It's interesting because it seems to come from a place internally of neurosis. And I don't mean this in like a clinical sense, but Jensen is just completely driven by negative emotions. He's terrified of failing.

He's consumed with guilt, actually. He told me this. You know, most Silicon Valley executives, if you believe in this typology, are like type A. They wake up, they're excited to go to work, they're excited to get things done, they're optimistic, looking to the future. Jensen, who works as hard as anyone, is type B. He wakes up and he thinks, oh my god, NVIDIA is going to fail. What can I possibly do today to prevent NVIDIA from going bankrupt?

The company motto in NVIDIA for a long time was, we are 30 days from going out of business. And this was true even when they were making billions of dollars a year. They would just tell everybody we're 30 days from going out of business. And what Jensen meant was, I want you to act with that desperation, that kind of manic anxiety that you feel you're going to fail if you don't show up to work and work your very, very hardest every single day. Jensen has lived his entire life this way.

As far as I can tell, he lived this way when he was like 10 years old. It's just... And what's really interesting about this is that neither of his brothers are this way. So he told me this is not a product of like, you know, tiger parenting or kind of like anxious parenting where you invest a lot in the kid and really expect them to succeed. Jensen didn't actually really have that. It almost all came from within. How interesting. And yeah, it's...

In addition to failure, it sounds like Jensen also thinks a lot about his competitors, a lot about the other companies that he's shoulder to shoulder with. And so he sets up NVIDIA,

And initially he's kind of nestled, isn't he, Stephen, in this like strange niche in the market around the PC video game hardware market. And that sounds like that's the kind of first, you might say in retrospect anyway, big mistake that his competitors give him because they just let him sit there, don't they? Yeah.

Well, yeah, so Intel, Silicon Graphics, and all the other firms at that time, who were the big players, decided to ignore the PC video game market because they thought it wouldn't be profitable.

classic mistake. And the other thing, one of the reasons that Intel ignored it is that at that time you had the concept, this is the early 90s of the PC, was that we'd have a motherboard and then we'd slot in like a sound card and a modem card and a printer card and a graphics card. What Intel believed was, hey, we don't need any of these cards.

In 10 years, our processor is going to keep getting faster and faster and faster because of Moore's Law. It's going to keep speeding up and it's just going to absorb all these functions and all these cards will go away. What they missed was the exact thing that I mentioned before. What they missed was that the demand for one of those cards for 3D graphics was infinite.

The demand for your printer card was not infinite. You only wanted to do so much computing power with your printer. Same with your sound card. As soon as you were producing audio at the same fidelity as a compact disc, there was nothing left to add. But for 3D graphics, the demand was infinite, and that was what Intel missed, and Jensen saw it.

Now, Jensen was not the only person to see it. In fact, in the early days, NVIDIA had a ton of competition. There were 70 different firms trying to build this. And Jensen saw correctly that in the end, only one firm would survive. It was like the Japanese movie Battle Royale, where they're all stuck in an island and they have to fight to the death and only one person will live. That's what it was like, the early PC video gaming market. And Jensen was the winner of that.

He killed, absorbed, or destroyed all of his competitors. It was just incredible. And what he would do is he would go with his lieutenants and he would go into the boardroom of the whiteboard and he'd list out all of his competitors. And then he would say, who's the best engineer at that company? If we could poach one person from this company, who would it be?

And so they had a lift, like a draft of people that they were trying to steal away from the competition and come to get to work for NVIDIA. And they were so successful at doing that, that when they did convince somebody to leave and come to NVIDIA, they called it brain extraction. Because those competitors would fold. They can't live without a brain. Their best talent would leave and then the competitor would cease to exist.

So Jensen hollowed a lot of these guys out from the inside and kind of took over and absorbed all of them until only NVIDIA was left. Well, I think we have reached a fateful part in the story, Stephen, where we can go no further without talking about parallel computing. Yes. Difficult technical topic, which you brilliantly actually in the book, I think kind of make clear for the readers without PhDs in microelectronics.

But it's so important, isn't it? Because it allows Jensen to kind of find this technical moat as well that gives him blue water. So why don't you tell us what parallel computing is, and why it's an important part of the story? Sure. So the AI revolution is a two-part revolution. It's a revolution in software, and we call that software neural networks. And this is software that's designed to simulate human neurons, like the brain. It's designed to simulate the brain.

It is equally, and just as importantly, a revolution in hardware. A revolution in how the microchip looks, works, and acts. Completely redesigned almost from first principles. And this is parallel computing or accelerated computing.

So, historically, when computer scientists wanted something to speed up, they just packed more and more components on the microchip. Every year, Intel would double the number of transistors on the chip. This is known as Moore's Law. It wasn't really a law, but he called it that. What Jensen and his team saw coming was that Moore's Law was going to hit the physical limits of our universe.

they were going to get the components so small, like one atom in width, that they would start to be compromised and lose their ability to conduct electricity. And at that point, Moore's law would come to a stop. So Jensen and his team said, we have to find a way around this. And the way around it is something called parallel computing. With Intel's CPU, we bring one problem, one math problem, one arithmetic problem to the computer at a time.

With parallel computing, we take the problem and we divide it into hundreds of smaller problems and then execute them all at the same time. So metaphorically, a way to think about this is if I have a delivery van and I'm just throwing things in the back of the van and then the van takes it around the city, that's the CPU. That's normal computing. Parallel computing is I have a fleet of motorcyclists and each motorcyclist is going to have a specific package that they're going to deliver to the city as fast as possible.

Now, the motorcycle approach is much, much faster. The packages, they don't have to wait in line. They don't have to get on the van. It's much faster. They're much more mobile. But it's much harder to manage a fleet of motorcycles than it is to manage a delivery van. And this has actually been the reason nobody adopted parallel computing for a long time. It's just too hard to do. They just instead waited for Intel to catch up. They waited for the CPU to catch up to what they were doing.

So if I have a choice as a programmer, I can use the classic Intel approach and just wait for Intel to come out with a faster chip. Or if I want to do the parallel approach, I have to go rewrite one million lines of computer code. So nobody was doing that, right? But in fact, every parallel computing company that ever started before NVIDIA failed. They all went bankrupt without exception.

But what Jensen saw was that physics would force people to rewrite those million lines of code. They have to do it. Moore's law was going to collapse. There was no choice. Now, this is all a little technical, but the best way to think about it is that the brain of the computer was rewired in a fundamentally different way. It just looked and acted different. And it was NVIDIA that made that happen.

I think it is also worth just spending a moment dwelling on the sheer physical ridiculousness of the chips, where the widths of the transistors are measured in atoms. I can't think of anything else that we build as human beings that's quite like a silicon microchip.

I mean, I compare Jensen to Daedalus, the designer of the labyrinth in Greek mythology, because these things are labyrinths. When Jensen started his career, if you could blow up the microchip to the size of, let's say, a tennis court, and then you built a maze made out of strands of string, it would fill up that entire tennis court, made out of strands of human hair, actually, fine human hair. That's kind of like what designing the microchip was like, if you blew it up to scale.

Today, you would have to blow up that microchip to basically the size of Great Britain, okay? Or the size of at least like, you know, the British Isles, right? And then cover it with strands of human hair, the whole thing, the entire country, to resemble how many transistors are on a single chip. Not only that, they've actually brought the chips vertical. So now they actually stack transistors on top of each other. So it's almost like a three-dimensional labyrinth.

And this is only possible really because so much of the design of the microchip has been automated at that fine scale. It's so hard to conceptualize, actually. You know, when you and I probably are using the computer, quote unquote, what we're really thinking about is the keyboard and the monitor. And then somewhere way down in the guts of the machine, there's some kind of devices or electronics doing something and we don't understand it. For Jensen, it's exactly the opposite. He thinks from the motherboard up.

He starts with a circuit, with a single component, and then projects forward into other systems that it controls, like the graphics and the monitor and the keyboard. So he's approaching computing from a completely different perspective.

And he approaches business decision-making from that perspective as well, doesn't he? So one thing that really came through so clearly was his ability to reason from first principles to then decide something which might be sweepingly counterintuitive to a current market and then bet absolutely everything on that.

Exactly. He's very analytical, actually. Jensen is not driven by emotion at all. He's almost a pure – I mean, well, that's not true. But when it comes to decision-making, he's almost a pure engineer. As opposed to HR. Yeah, Jensen doesn't know what to do with his emotions, I will tell you that. He's an engineer, and he's like, why do I have these? But in terms of decision-making, he's extremely analytical, but he's also very farsighted. And as a result, he actually seems insane. Yeah.

because he's seeing so far ahead, farther than anyone else can see. And he does counterintuitive stuff, but the more you think about it, it has to be that way, or he wouldn't be a good CEO, right? If anybody could see what Jensen could see, then the market gets competed down to zero. You know, profits are competed down to zero because everyone's seeing the same thing. And I think he lived through this with 3D graphics, which a lot of people saw the same things he did.

He realized he had to see further and pursue markets that no one else was pursuing. And so analytically, what he decided in the mid-2000s was that he had to start taking these circuits that were used for 3D graphics and repurpose them for scientific uses. Now, this was not advised by the business community. Actually, his business community, sorry. His investors hated this idea, and he almost lost his job.

Basically, what he was trying to do is take these low-budget gaming cards and turn them into a supercomputer. And it's like, who is this for? The scientific market for this kind of thing is very small. It's really only for scientists who can't afford time on a conventional supercomputer. So scientists whose work is marginal, whose research is marginalized. Mad scientists, right? It's for mad scientists. It's

It's for scientists pursuing unpopular research programs that can't get funding. Well, that's a very small market, as you might imagine. But Jensen believed that one of these guys has to have some kind of brilliant insight. And this turned out to be true. That turned out to be neural networks. That turned out to be AI.

The neural network paradigm we live in today, those guys were absolutely mad scientists. They were working on the fringe. What they did was not popular. People did not believe it would work even within AI. This was not the leading paradigm. Most people were using other approaches. Neural networks were seen as like a museum piece. They didn't think anyone could ever get them to work.

And it was only really this platform that Jensen built to enable mad scientists that made this possible. Bill Dally, the former chair of the Stanford University Computer Science Department, told me, without Jensen, we'd be 10 years behind. Because nobody else was going to build this kind of system. There was no one out there building anything that looked anything like this.

And it was hard. I mean, it took a long time to get customers and investors hated it, right? So you had to really piss off both his customers and his investors to get this platform to market. So he was searching for this killer app. He was building scientific tooling for mad scientists to try and find it. And...

And then suddenly an NVIDIA card falls into the hands of Jeffrey Hinton, Sutskever, and Krzyzewski, doesn't it? Three, I think they would have considered the mad scientist then, now some of the world's most famous computer scientists. They were mad scientists back then. Jeffrey Hinton couldn't, 15 years ago, Jeffrey Hinton could not get a $10,000 research grant. He couldn't get it. The first system they built cost $600. That's what they had. And they used their housing stipend to build it.

I mean, it was absolutely on the fringe. He was at the University of Toronto. It's a great school, but it's not a center for computer science in any way. It is now, but it wasn't then. It is now because of Hinton. But before that, this was the Hinterlands. They were out there building systems that nobody wanted. There were zero customers whatsoever. They couldn't get published in journals. And I will say NVIDIA did not see this coming.

So when they built their supercomputing platform, NVIDIA made a long list of potential applications. This included things like quantum physics, medical imaging, oil prospecting, that kind of thing. AI was not on that list. AI was regarded as a career graveyard.

And if you went to work in AI, you held some kind of remote academic post and went to conferences your whole life and were unlikely to ever interact with Silicon Valley in any way. In 2010, funding for AI from Silicon Valley, from venture capitalists, was closer to zero than any other number. It was basically effectively zero. And it wasn't until Hinton and his team found NVIDIA's platform that the AI revolution started.

What's funny about that is when Hinton and his team found it, they contacted NVIDIA immediately saying, oh my God, you have to send us a free card. You have to kind of take advantage of what's going on here. And it was so remote, it was so mad that even NVIDIA didn't see it at first. They didn't respond to Hinton. They wouldn't respond to his emails at first. It was only a couple of years later that they really started to understand what was going on.

But when Jensen caught wind of it, he immediately pivoted NVIDIA away from graphics toward AI. I mean, over a weekend, he sent out an email to his company saying, we are now an AI company. And they started building an AI computer, an AI dedicated kind of hardware. This was the first time anything like this had ever been built. Today, AI hardware is a trillion dollar market, and it's effectively almost all NVIDIA.

I mean, what's so kind of striking and amazing about the story is the way in which you've got these two overlooked technologies, both on the margins, both considered to be kind of outlier, fringe kind of interests who needed each other to be revealed in the way that they did. Yeah, no, it's crazy. Imagine not taking one cast-off technology, but two of them. And they only work together when you put them together. Parallel computing had always failed.

It had always failed the test of the marketplace. There were like 20 dead parallel computing companies.

AI had always, always failed. There was not a single AI company. It just was seen as a way to burn money, basically. Nobody wanted anything to do with either of these technologies, neither the software nor the hardware side. They were absolutely seen as just, you know, Jensen said that the road to success for us was littered with dead bodies, right? We had to walk past all these failed companies, all these dead bodies to do what we wanted to do.

But for Jensen, actually, he liked it because he said to himself, you know what? I know I'm the only one who's going down this road. There won't be any competition when I get to the end here. And it turned out to be right.

You chose to hit play on this podcast today. Smart choice. Progressive loves to help people make smart choices. That's why they offer a tool called AutoQuote Explorer that allows you to compare your progressive car insurance quote with rates from other companies. So you save time on the research and can enjoy savings when you choose the best rate for you. Give it a try after this episode at Progressive.com. Progressive Casualty Insurance Company and Affiliates. Not available in all states or situations. Prices vary based on how you buy.

This message is brought to you by Abercrombie & Fitch. I've been ready for summer for a while, and now it's finally time for summer outfits. With the trip coming up, the A&F Vacation Shop has me covered. Abercrombie really knows how to do a lightweight outfit. Their tees, sweater polos, and linen blend shorts never miss. I wear Abercrombie denim year-round. Their shorts are no different and have the comfort I need for summer.

You can Venmo this or you can Venmo that. You can Venmo this or you can Venmo that.

The Venmo MasterCard is issued by the Bancorp Bank N.A. Pursuant to license by MasterCard International Incorporated. Card may be used everywhere MasterCard is accepted. Venmo purchase restrictions apply.

You know that feeling when someone shows up for you just when you need it most? That's what Uber is all about. Not just a ride or dinner at your door. It's how Uber helps you show up for the moments that matter. Because showing up can turn a tough day around or make a good one even better. Whatever it is, big or small, Uber is on the way. So you can be on yours. Uber, on our way.

This episode is brought to you by Amazon Prime. From streaming to shopping, Prime helps you get more out of your passions. So whether you're a fan of true crime or prefer a nail-biting novel from time to time, with services like Prime Video, Amazon Music, and fast free delivery, Prime makes it easy to get more out of whatever you're into or getting into. Visit amazon.com slash prime to learn more.

And so you have this joining, this fateful joining of parallel computing and neural networks. And AlexNet, and then it's a straight line, isn't it, Stephen? All the way through to ChatGPT, GPT-4.0, and everything else that people would now be familiar with. Yeah. So basically, the way to think about this is AlexNet is kind of, this is the first neural net that ran on the NVIDIA system. We can think of that as like the Wright brothers' first airplane.

The Wright Brothers' first airplane, I think it flew for like 11 seconds. You know, it wasn't in itself like a great piece of technology, except it showed what was possible. And if you think of NVIDIA in this analogy, then NVIDIA was building a propeller with an engine attached to it for 10 years before the first airplane launched. And they were just looking like, what are we going to do with this propeller? Who's going to use this? I don't know. And then the Wright Brothers come along. Then things started to progress very, very rapidly.

So another technology came along called a transformer, and this was sort of like the jet engine. And this happened within like six or seven years of the invention of AlexNet. So the speed at which neural nets has evolved over the last 10 years is just incredible. And what people have started to figure out is that the only thing that matters in making a neural net faster is not fancy code or insight or programming. It's just raw computing power.

And that's what NVIDIA provides. And that's what makes NVIDIA so wealthy. They're just there, you know, making faster and faster and faster computers. So the way to think about this, I think, remember, the neural net is patterned after the biological brain.

The first brain that we ever saw in the kind of fossil record belongs to a trilobite, and it's about the size of a grain of rice. And it's at the back of two eye stalks that lead down to this little grain of rice, and its only job is visual processing or visual recognition. That's where our brains come from in the evolutionary record. That's the first one.

that we've ever been able to find. And in 500 million years, they've gotten increasingly more sophisticated and complex until we have these things, which can do quite a lot. So what NVIDIA is trying to do is take that 500 million years of evolution and speed it up into about 500 minutes, right? They're going to take the germinal seed of intelligence and grow it as fast as possible. Hyperspeed evolution.

This is kind of what NVIDIA's effect on neural networks is. They start with these kind of raw grids of simulated neurons, arranged almost at random, arranged at random essentially. And then they train them to do different tasks. So with AlexNet, what they did was they showed it 10 million images over the course of about a week.

And very slowly, the neural net learned to process and identify these images just through repeatedly being shown them until the point where it had like an 80 or 90% success rate at the end of a couple of weeks. And this is evolution, right? Just like our ancestors lived in some kind of marine ecosystem where they were seeing, you know, kind of like 3D images and slowly got better and better at both identifying food and avoiding danger.

And so this is exactly how neural networks are trained. It's exactly how they work. And the only difference is that they just do it way faster than we do. And so this means that by the time the AI revolution is fully upon us, NVIDIA and Jensen are the only people really supplying the chips. They're the only ones supplying it. Yeah. Yeah. Yeah. Well, it's because they were there first. It's because they were 10 years ahead. And it's also because Jensen is paranoid.

Right. I mean, he's just absolutely driven, as I said, by fear. He's terrified that someone's going to catch up to him. And so that intensity, that drive of negative emotion, you know, I was talking to him at Caltech and he's sitting there kind of looking grumpy. He's got his arms folded. He's got his expensive shoes pressed against the cobblestone. And I'm like, well, tell me about your day. What did you do today? He's like today. Oh, OK. Like I woke up at three thirty a.m. today and my dog was sleeping in between my legs.

And I said to myself, do I really want to get up and go to work at 3.30 a.m.? This is a guy with $100 billion running the biggest company in the world. He's like, no, not really. He's like, is there any immediate threat that's going to destroy NVIDIA's competitive position today? No, not really. We're in a good place. And then he started to think, well, what about tomorrow, though? What about Intel? What about Huawei? What about the next Jensen Huang, who's looking at me the way I used to look at my competitors? And by 4 a.m., he's up and working.

And so he uses this kind of like motivating tactic of beating himself up of fear and paranoia to go work 14, 15 hours a day. He's almost never not working and creating these incredible products. It's funny. NVIDIA's had a tough year, actually, since the book came out with the tariffs, with Trump, with increasing competition from other vendors and with China. And I talked to someone at NVIDIA. They're like, actually, you know what?

Jensen seems a little more relaxed this year than he did when he was kind of the king of Wall Street last year. And Jensen told me this himself. He used to work the rush hour at a restaurant, Denny's Restaurant, which is like a chain of kind of diners and greasy spoon diners in America. And he was like, you know, I think when rush hour started at the restaurant, when the dinner rush started, I think my heart rate actually went down.

I work best under pressure. I feel confident and calm when the pressure is on. And I get nervous when the pressure is off. And I think this is what motivates Jensen. They told me this year, Jensen has been a better leader than he was last year with all of these problems, with NVIDIA's stock price going down 40%. He's kinder, he's more compassionate, and he sees further. He actually likes it when the pressure is dialed up in this way.

What's it like to interview him, Stephen? Because you've been subject to the wrath of Jensen, haven't you? Which is a kind of something that he has long used as a kind of management tactic. It's hard to interview Jensen. One of the hardest interviews I've ever had. One of his executives compared Jensen interacting with Jensen to sticking your finger in the electrical socket. And that is exactly what it's like. It's just like, whoa, this guy is so intense.

And he's a little hard to read, certainly. He's moody and his moods will shift over the course of the interview. Jensen's, in the sense that he's driven by negative emotions, is also driven by a manic desire to entertain. And so he's paranoid that if he's doing a public speaking event or an interview that the audience might get bored.

I understand this. This is actually what drives me. So we can all relate to the fear of a bored audience. No, totally. I completely get this. You put me on stage. I'll come out in a clown suit if I have to. Like, I'm terrified the audience is going to be bored.

And Jensen has the same thing. So I clocked this about him pretty early. And so when you're interviewing Jensen, you have to be a little careful because he will go for the most entertaining soundbite that's available. Even if it's not strictly true, he's just trying to generate some kind of spark or heat on stage and on TV. But he doesn't like to talk about himself. You know, I think inside of Jensen is like David Lynch.

This will sound weird, but you'll see where I'm going. David Lynch was once asked if he ever went to therapy. And he said he considered it, but then he went and talked to the therapist. He's like, could this affect my creative process in some way? And the therapist was like, yeah, I'm not going to lie to you, it might. And David Lynch was like, I'll never do therapy, ever. And I think with Jensen, it's maybe kind of the same thing. He's one of the most successful inventors. And alongside Thomas Edison, he's like the most successful electrical engineer of all time.

One could argue. And Gordon Moore and a couple others, right? And if that comes from this kind of farrago of negative emotions, this maelstrom of anxiety and grief and paranoia, but that works, then he's not going to change it. He's analytical. He's an engineer. What he's doing, his mode of operating has brought him to the pinnacle of business, the pinnacle of technology. And so he doesn't like questions that probe his internal emotional state.

He doesn't like questions about motivation or about kind of what he's thinking. It made him very uncomfortable to talk about this stuff. And so that was the hardest part of the interviews because I did go there. Not that I was trying to psychoanalyze Jensen or be his therapist or anything. I mean, I think he's doing fine. I don't think he needs therapy. It seems like it's working for him. But I think he just really resisted that. I asked a lot of probing questions and he did not like it. He ran away one time. I mean, he just didn't.

Didn't like questions that got to question, you know, areas of like motivation or emotion or feeling. I've encountered this before with other engineering types, other analytical types who see emotions as kind of useless, vestigial evolutionary appendages that they train themselves to either ignore or repurpose into high functioning accomplishment.

He also didn't like questions to do with AI safety, did he? No, no. And that seemed like that was the area of greatest intellectual friction between you and Jensen, where you clearly carry deep concerns, Stephen, about the future of AI and the dangers it might be created. And you alongside Jensen,

the three most cited computer scientists in history also sharing those concerns. But with Jensen, you found someone who seemed to be absolutely uninterested in the idea that AI might create risk. And they're just incapable really of seeing that as something which is a plausible thing to stop and pause and consider.

I really press this point in the book because, uh, the thing was, I was initially terrified of chat GPT. I mean, I had nightmares. I was like, what kind of world am I going to have a job? Am I ever going to be able to write again? Uh,

What kind of world are my kids going to grow up in? Like, what's up with this? Like, we've opened Pandora's box or whatever. You know, that's a bad metaphor. But we've, the genie, whatever. I don't know. You know what I'm saying? Like, we've sparked this super intelligence revolution. It's starting now and we don't know where it goes. And, you know, I was aware of arguments from like philosophers in the effective altruism community about this.

And I didn't want to give those a ton of ballast because frankly, a lot of the people just don't understand the technology very well and don't know what they're talking about.

It was only when I started talking to Jeffrey Hinton, who built this stuff, he absolutely invented it. Okay. He invented it. Joshua Bengio, they invented it, right? They won the Turing Prize for inventing this stuff. They know it cold and they saw the potential for it long before anyone else. They saw the future of this technology long, long, long before anyone else. And they were so terrified that I decided, okay, this is a real thing.

I have to talk about this in this book. I can't ignore this and just write a business story. And in particular, I have to ask Jensen about it. Now, I should say that it's not just Jensen. So three guys won the Turing Prize, really, for inventing neural networks. Joshua Bengio, Jeffrey Hinton, and Jan LeCun. And in fact, these are three of the most exciting computer scientists alive. Now, LeCun thinks that Bengio and Hinton are being ridiculous. And these guys are like best friends.

But Lacuna is completely split with them. And he's like, you know, Benji is being nuts. There's no risks to these systems, not in the way that they think. These systems are safe.

They're highly productive, and they're going to open up a new era of prosperity for human beings. Great numbers of diseases will be solved. This is all good. And worrying about the potential for AI is like worrying about the Industrial Revolution or worrying about agriculture. In the end, you will want to live in a world where AI systems are allowed to pursue the maximal capabilities that we can give them for the same reason that you enjoy the fruits of the Industrial and Agricultural Revolutions.

And Jensen shares this mindset. He thinks we're living through a new industrial revolution. The computers that he builds, he calls them AI factories, where data goes in and intelligence comes out. And his point of view is that more intelligence can only be good. It can only help solve the problems that we face.

And, you know, he was like, we have this conversation every time a new technology comes along. You know, you had the Luddites with the looms and probably in the agricultural era, there was some kind of hunters who didn't like it or whatever. But he's like, every time it just makes everything better. And so this conversation is stupid. And when I showed him a clip of Arthur C. Clarke talking about the future, he started screaming at me about this. Now, I had pressed him on this point several times. So maybe he was just sick of this question. But I felt it was important. I felt that we had to ask this.

For Benji and for Hinton, there's several worries about AI, but what if it sees humans as a risk? What if it just is somehow even inadvertently given some desire or need to survive or replicate itself? That could be very, very challenging for humanity. I think we're already approaching eras where these fears are not just concrete fears.

There was a famous thought experiment from Nick Bostrom saying it was the paperclip maximizer experiment where what if an AI was given the command or the prompt to go build as many paperclips as possible? And then it ran away and converted the entire universe into paperclips and tuning all of the atoms in human bodies and destroyed everything to create a universe of paperclips. Is that possible? It's a little abstract.

With the rise of AI agency that's coming in the next few years, someone is going to take an AI agent and they're going to say to it, hey, make me as much money as possible. Don't worry about the rules. Don't worry about the safeguards. Just go out there and make me money. And how that AI reacts and what it does, I don't think anyone has an answer for right now.

Well, I don't want to end on too chilling a note, Stephen, but the reason of one of the kind of ending lines in the book is I think he's worth reflecting on as a final thought connected to everything you just said. And it's about the executives were more afraid of Jensen yelling at them than wiping out the human race.

Not a line that you can ignore. And I guess just as a final question then, like, what do you think Jensen's role is going to be now, you know, when it comes to navigating IR risk, and maybe when it comes to navigating economic risk as well? Because you've got in Jensen, someone that's obviously unbelievably influential and convincing, as well as someone that's very powerful. And I suppose, like, kind of what role do you see him having in shaping the kind of near future for us?

So there's kind of two roles there. First is Jensen has embarked on essentially a campaign of private jet diplomacy since the Trump administration took office. He's in Mar-a-Lago, then he's in Beijing, then he's in Taipei. And he's kind of trying to manage NVIDIA's vast network of suppliers, customers, and then Trump's erratic pronouncements about tariffs. It's a really tough position to be in. But as I said, he seems to thrive in this kind of difficult environment. So maybe he's doing okay. Maybe he loves it. Actually, I'm not sure.

All of this has happened since the book came out. So NVIDIA has become a political football because they source everything from Taiwan and they sell a lot of equipment to China. And especially when they sell equipment to China, Washington, D.C. gets nervous. And so Trump is trying to sort of block them from doing this. On top of this, I think there's the longer question of where are these systems going or taking us? For Jensen, at least, the immediate answer is robotics. So he's trying to be at the center of a new robotics revolution.

He's working with academics and they kind of did this survey of what do people most want from robots? And they asked everybody a question. How much would you benefit if a robot did this for you? The number one answer was wash dishes.

So if we get a robot to wash dishes for us, that's the most important thing you can do. The final answer, the thing nobody wants a robot to do, is open presents. So if a robot can open presents, that doesn't benefit me. I want to open my own presents. I'm sorry. We have dishwashers. They want a robot that actually holds the dishes and washes them in the sink. Is that right? Precisely. Precisely. And so NVIDIA is now trying to build this. Okay. I feel like we need a whole other hour on that one. The problem is this, okay?

You know, they learn from reinforcement learning, the neural networks. They have to evolve to learn to wash dishes. And it's very costly, right? You're going to break 10 million dishes on the way to getting there, right? And it's going to be a huge mess and you're going to have to clean it up. So what Jetson wants to do is build a high fidelity physics simulator, basically a digital gymnasium where you replicate the dishwashing environment, the sink,

in code. And then you train the robot in this high fidelity physics simulator that you've got things like, you know, surface tension, soap, water, fragile dishes, and it can break 10 billion dishes in there. And who cares? You don't have to clean anything up. Then when you've trained the robot's brain to the highest capacity to wash dishes in the simulator, you download it into a real world body and deploy it on a physical actual kitchen sink.

Jensen thinks this is about to happen. And just as he kind of bet on CUDA on the supercomputing platform before, now he's betting on this robotics platform. This is his big bet. This is where he's pushed a lot of his chips lately. So this is what he sees coming. A world in which most drudgery

in which most human tasks are replaced by robots, and he's going to train the brains of the robots. Well, Stephen, thank you on that shock final revelation that the dishwashers are not enough and the actual robotic... But you will still get to open presents in the future. That will be the last human activity. There will be robots everywhere doing every job, and you and I will give each other presents. The robots will wrap the presents, but we will get to open them.

Well, that was Stephen Witt, everyone, author of The Thinking Machine, which is available now online or a bookshop near you. Stephen, thank you so much. That was brilliantly, brilliantly fascinating. And thank you, everyone, also for joining us. Of course, you've been listening to Intelligence Squared. I've been Carmilla. Thanks for joining us. Thanks so much. Thanks for listening to Intelligence Squared. This episode was produced by Leila Ismail with production and editing by Mark Roberts.