Deep Dive
Shownotes Transcript
This message comes from Progressive Insurance and the Name Your Price tool. It helps you find car insurance options in your budget. Try it today at Progressive.com. Progressive Casualty Insurance Company and Affiliates. Price and coverage match limited by state law. Not available in all states. A quick note before we start today's show. You may have heard that President Trump has issued an executive order seeking to block all federal funding to NPR. This is the latest in a series of threats to media organizations across the country.
The executive order is an affront to the First Amendment rights of public media organizations. It's also an affront to the First Amendment rights of the American people. NPR remains committed to serving the public. That's you. We lay out the facts and bring you stories that spark your curiosity that you won't find anywhere else. This is a pivotal moment. It's more important than ever that every supporter who can contribute comes together to pitch in as much as they are able.
Visit donate.npr.org now to give. And if you already support us via NPR Plus or another means, thank you. Your support means so much to us, now more than ever. You help make NPR shows freely available to everyone. We are proud to do this work for you and with you. You're listening to Shortwave from NPR.
Hey, short wavers. Regina Barber here with my co-host, Emily Kwong, with the second half of a mini-series she reported on the environmental footprint of AI. Hey, Em. Hi, Gina. So today I am bringing you a story of a personal crisis. It's very relatable. Go on. Okay. So in 2018, COVID-19 hit the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S. and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the U.S and the
computer scientist Sasha Luciani took a new job, AI researcher for Morgan Stanley. She was excited to learn something new in the field of AI, but she couldn't shake this worry.
I essentially was getting more and more climate anxiety. I was really feeling this profound disconnect between my job and my values and the things that I cared about. And so essentially I was like, oh, I should quit my job and go plant trees. I should, you know, I should do something that's really making a difference in the world. And then my partner was like, well, you have a PhD in AI. Maybe you can use that. So Sasha quit her job.
And she joined this growing movement to make AI more sustainable. Yeah, you were saying that like AI innovation was causing this like surge in energy and water use like to cool data centers. And the construction of those data centers was only going to increase. Yes. Some think exponentially. Yeah.
Gina, by 2028, Lawrence Berkeley National Laboratory forecasts that data centers could consume as much as 12% of the nation's electricity. That's 580 terawatt hours. Okay, can you give me like a different way to kind of think about how much that actually is? It's like the amount of energy that Canada consumed two years ago. Okay, so U.S. data centers alone could someday use a Canada-sized amount of energy? They could. Wow.
So Sasha is on a quest to find AI models that are smaller and use less energy. She is now the climate lead at Hugging Face, which is an online community for AI developers to share models and data sets. And a model is just like an AI program that's trained to take in data and like output data. Yes. So virtual assistants such as ChatGPT, Microsoft Copilot, Google Gemini, they are all powered by what's known as large language models. And
And Sasha, as she made quite plain in her 2023 TED Talk, is not a fan. In recent years, we've seen AI models balloon in size because the current trend in AI is bigger is better. But please don't get me started on why that's the case. Wait, so I actually do want her to get started. Like, why are these like big players all using these huge models? I'm glad you asked. Thank you. Because today on the show...
We're going to talk about why bigger isn't always better when it comes to generative AI. In part two of our series, we'll talk about how this big, sprawling industry is looking to shrink its environmental footprint with everything from small models, clean energy, and a back-to-the-future way of keeping data centers cool. I'm Emily Kwong. And I'm Regina Barber. You're listening to Shortwave, the science podcast from NPR. Don't worry, you won't be lost if you haven't heard part one.
This message comes from Thrive Market. The food industry is a multi-billion dollar industry, but not everything on the shelf is made with your health in mind.
At Thrive Market, they go beyond the standards, curating the highest quality products for you and your family while focusing on organic first and restricting more than 1,000 harmful ingredients. All shipped to your door. Shop at a grocery store that actually cares for your health at thrivemarket.com slash podcast for 30% off your first order plus a $60 free gift.
This message comes from Amica Insurance. As Amica says, empathy is our best policy. Whether you're seeking auto, home, or life coverage, they'll work with you to choose the policy that best serves you and your family. Discover how Amica can help protect what matters most to you today. Go to amica.com and get a quote today.
Okay, Em, you've been talking with like four of the biggest tech companies, Google, Meta, Microsoft, and Amazon, which I should say are like all financial supporters of NPR. It's true. Amazon also pays to distribute some of NPR's content. Right. And these four companies like all have ambitious goals for hitting net zero carbon emissions, most by 2030, Amazon by 2040. How are they going to get there?
There are three paths, as far as I can tell. But before we talk about small AI models, you know, what Sasha's describing, let's talk about two solutions to make large language model computing more green.
And that is more efficient data centers and nuclear power. What do you want to start with, Gina? I'm a physicist. Nuclear, obviously. Of course, of course. Nuclear, because Amazon Meta and Alphabet, which runs Google, made a big announcement in March, as reported by Straight Arrow News. Three of the world's largest tech companies are promising to help triple global nuclear power supply by 2050. They're going to build new nuclear power plants. And along with Microsoft,
purchase nuclear energy. And Microsoft plans to get its nuclear energy by reviving a plant in Pennsylvania. Yeah, our colleague Jeff Brumfield, he came on the show in December to talk about how Microsoft purchased Three Mile Island, like the site of a partial nuclear meltdown in 1979. Yes, only one of the reactors melted down, by the way. The whole site was shut down in 2019, and now Microsoft wants to bring it back. Okay, so are AI companies turning into energy companies? Yes.
They are turning into energy movers and shakers. Okay. For sure. But Jeff sees a discrepancy in this, you know, between the AI people and the nuclear energy people. Yeah. Silicon Valley loves to go fast and break things. The nuclear industry has to move very, very, very slowly because nothing can ever break.
Nuclear is also extremely expensive. Yes. And while solar and wind energy combined with batteries is quicker to build and more inexpensive than nuclear or gas power plants, it still takes time. I mean, like, do we need to move that quickly to grow AI?
It depends on who you ask. Kevin Miller, who runs global infrastructure at Amazon Web Services, says yes. I think you have to look at the world around us and say, we're moving towards a more digital economy overall. And that is ultimately kind of the biggest driver for the need for data centers and cloud computing. But Sasha Luciani, the computer scientist who we met earlier...
feels this rush for AI is coming from industry, not from consumers. It's unfair to say that users want more because users aren't given the choice. Yeah, I mean, like I hear Sasha here because like I'm a big fan of like AI's benefits. It's totally changed science and medicine and business and banking, all these things that affect our lives. But it does feel like opting out of AI is like becoming more and more difficult.
Absolutely. Yes. And until nuclear power catches up with AI's energy demand, data centers will, for the foreseeable future, continue to use fossil fuel sources. Yeah. So the question becomes, you know, is there a way to make data centers themselves more efficient? And the tech generation
Giants are trying through better hardware, better chips, and this really captured my attention, more efficient cooling systems. So that's solution number two. I love a tech solution to a tech problem. What are some of these strategies? Well, one method that's become quite popular is to design a data center to bring in cool air from outside the facility. No chilling required.
So they just like pull in this cold air. Yeah, this is what's known as a free air cooling system. And then there's a design paradigm that's getting a bit of buzz. Folks in the industry call it liquid cooling. Okay, and this is a different kind of liquid cooling evaporation we talked about in the first episode. Yes, this does not use water.
Liquid cooling uses a special synthetic fluid that runs through the hottest parts of the server to take the heat away. Or whole servers are immersed in this cool liquid bath. Okay. So the idea of like running coolant through like a car engine. The very same. You can think of this like coolant, but for computers. Okay.
Benjamin Lee, who studies computer architecture at the University of Pennsylvania, said this is just a much more efficient way to cool off a hot computer. Because now you're just cooling the surface of whatever the cold plate is covering rather than just blowing air through the entire machine.
So I wanted to talk to someone who's trying to bring liquid cooling to the market, and I found this company called Isotope. David Craig is their recently retired CEO. I definitely come from the point of view that, you know, we literally have just one planet and...
I cannot understand why anybody would want to do anything other than care for it. David says the older way of cooling data centers, that daisy chain of moving heat with air and water, is just completely consumptive. Yeah. And while he couldn't tell me which tech companies have struck agreements with Isotope... So there are a number of confidentiality issues
clauses that sit around kind of customers. Isotope has announced public partnerships with Hewlett Packard and Intel. And Ashley Settle, a spokesperson at Meta, told me that Meta anticipates some of its liquid cooling enabled data centers will be up and running by 2026. Wow. Okay. But because I'm a numbers person, like how much energy is being saved by liquid cooling versus like air or water cooling?
Depends on the data center. You can say that the very best liquid cooling system uses about 40% less energy than a traditional air cooling system. And it uses no water. And then what we're doing is we're capturing that heat in a closed water loop. So it's a bit like a domestic central heating system at that point.
And in huge parts of the planet, particularly the north, we can return that heat and do useful things with it in much more intelligent ways. David is talking about something called district heating. And that's where the heat from a data center, any data center, doesn't have to be liquid cooled, is then diverted to a local neighborhood.
And that is starting to happen at some data centers in Europe. Google has a data center in Finland that is providing heat to 2,000 people. That's so cool. I think I've actually read about this. Like, I think it's called Hamana data center. That's the one. Yeah. Now, Hamana does not use liquid cooling, but it is kind of a poster child for a green data center. Hamana runs on 97% renewable energy.
and pumps in seawater from the Bay of Finland to keep cool. Wow, that's really cool. Literally. Mm-hmm. Okay. So Hamana is just like one of these data centers out of thousands, right? Yes, and this is the challenge. Most data centers are not situated by bodies of water in northern Europe. Right. So I want to talk about a third and final innovation.
And it's the one that the tech companies I spoke to were kind of quiet about. Oh, okay. But the one that scientists and engineers outside the industry could not stop mentioning, and that is smaller AI models. I mean, of course. Right? One's good enough to complete a lot of the tasks we care about, but in a much less energy-intensive way. So a third and final solution to AI's climate problem is just to use less AI. One...
Right. That's the chatbot out of a company in China, and it is claiming to use less energy. Yes, I did reach out to DeepSeek for comment. I didn't hear back. But here's the thing, Gina. Large language models like ChatGPT are often trained with really large datasets. And that's because they're so large that they can't be used in a large space.
DeepSeq, on the other hand, appears to have been trained with fewer chips and consists of smaller models that run fewer parameters. Benjamin Lee at UPenn says this is called a mixture of experts. The idea behind a mixture of experts is you don't need a single huge model with a trillion parameters to answer every possible question under the sun. Rather, you would like to have a collection of experts.
smaller models, and then you just sort of route the request to the right expert. And because each expert is so much smaller, it's going to cost less energy to invoke. But DeepSeek hears the thing about it, it's still a big general-purpose model. And Sasha Luccioni at Hugging Face wants to walk away from large models entirely. Nowadays,
More and more, I think companies especially are like, for our intents and purposes, we want to do X, like whatever, summarize PDFs. But you don't need a general purpose model for that. You can use a model that's task specific and a lot smaller and a lot cheaper. Basically, Sasha wants to see companies develop small language models. Models that have far fewer parameters and are trained for a specific task. So like, are tech companies experimenting with this? No.
A few. Okay, here's what I found. Last year, Meta announced a smaller quantized version of some of their models and Microsoft announced a family of small models called Phi 3. But honestly, in all of my conversations and emails with big tech, it's clear to me that large language models are here to stay. So if that's the case, Sasha has one last idea.
an industry-wide score for AI models. Okay. So like Energy Star was created to like rank the energy efficiencies of appliances. You see that little star on a lot of those appliances. Yes. Something like that for AI models. But at least according to Sasha, tech companies are not embracing a rating system. There's like
Such a blanket ban on any kind of transparency because it could either like make you look bad, open you up for whatever legal action or just kind of give people a sneak peek behind the curtain. And as a science reporter for NPR, my question was just, do we really need all of this computing power when we don't know how much it's costing us environmentally and when it could imperil our climate goals?
And David Craig, the recently retired CEO of Isotope, he chuckled when I said this. And he's like, Emily, you know, human nature is against us. We are always that kid who does touch the very hot ring on the cooker when our mum said don't. We are always the people who touch the wet paint.
We're not good at learning until bad things happen to us. The truth is with data, you know, this stuff has just grown up in the background. People just haven't known about it. I mean, yeah, I mean, I certainly had no idea about all of this while I was growing up. All this was happening in the background. But you can't, like, disinvent the Internet, right? Nor should we. Where would we get our cat videos? In the mail. There's something I think that we as consumers can think about.
The AI revolution is fairly new. Google CEO Sundar Pichai compared it to the discovery of electricity. Except, unlike the people during the Industrial Revolution, we know that this has a climate cost. Wow. Yep. And there's still time to adjust how and how much we use AI. Em, thank you so much for bringing us this reporting. Thanks, Gina. If you liked this episode, you can check out part one of our series on AI and the environment. It's already out.
This episode was produced by Hannah Chin, edited by showrunner Rebecca Ramirez, and fact-checked by Tyler Jones. Robert Rodriguez was the audio engineer. Special thanks to Brent Bachman, Johannes Durge, and our incredible standards team. And special thanks to TED Conferences, LLC. Special thanks also to Julia Simon on NPR's Climate Desk.
Beth Donovan is our senior director, and Colin Campbell is our senior vice president of podcasting strategy. I'm Emily Kwong. And I'm Regina Barber. Thanks for listening to Shortwave, the science podcast from NPR.
On this week's Wildcard podcast, Weird Al Yankovic says there's something funny about being professionally weird. I think people realize, you know, Weird Al is almost ironic because I'm like one of the more normal people in showbiz, I think. I'm Rachel Martin. Weird Al is on Wildcard, the show where cards control the conversation.
On the Indicator from Planet Money podcast, we're here to help you make sense of the economic news from Trump's tariffs. It's called in game theory a trigger strategy, or sometimes called grim trigger, which sort of has a cowboy-esque ring to it. To what exactly a sovereign wealth fund is. For insight every weekday, listen to NPR's The Indicator from Planet Money.
Fall in love with new music every Friday at All Songs Considered. That's NPR's music recommendation podcast. Fridays are where we spend our whole show sharing all the greatest new releases of the week. Make the hunt for new music a part of your life again. Tap into New Music Friday from All Songs Considered, available wherever you get your podcasts.