Why AI Compute Consumption Isn't Slowing Down
20:51 Share

Thank you.

Welcome back to the AI Daily Brief Headlines Edition, all the daily AI news you need in around five minutes. Quick note, because I know that you are all going to be wondering about the coverage of OpenAI's GPT 4.5, which is very clearly launching today, given that at 10.30 a.m. Eastern Time, OpenAI tweeted live stream in 4.5 hours. Can't imagine what that means.

That will, of course, presumably be our main episode for tomorrow. Alas, we are recording this show before that comes out. And our main topic in the headlines today is that Amazon has unveiled the long-awaited AI Alexa. Agents were everywhere in this demo, with Amazon depicting the new Alexa, booking concert tickets, making restaurant reservations, and texting a babysitter.

Alexa's agents will be capable of interacting with first and third party apps, with Uber, Grubhub, TripAdvisor, and Ticketmaster all highlighted during the presentation. A suggested use case was asking Alexa whether anyone in the house had walked the dog recently, with the agents capable of checking security camera footage to figure it out. The live demo was focused on a typical conversation. It showed off updated voice capabilities, allowing a much more natural and free-flowing conversation than the old Alexa was capable of.

Now, I should say here, I have frequently called out that I don't love the personal assistant application of agents as the thing that companies focus on. I do think it might be a little bit different with a home system like Alexa. Talking to Alexa to order food makes more sense to me, given that it's already installed in your house and is an integrated part of your system than some other general purpose agentic app that you would sign up for in the future.

Still, ultimately this was all just demo. And the question will of course be whether Amazon can actually deliver a consistent experience once the rollout begins next month. You might remember that AI Alexa has had a troubled development cycle, facing multiple delays over the past couple of years. As recently as a month ago, it was reported that Amazon was unsure Alexa would be ready for the market. Last October, Bloomberg reported that the team was finding AI Alexa prone to lengthy droning answers.

and had trouble with basic functionality like turning on lights in a smart home. The demos we did get were promising, but most features weren't shown live and the press weren't able to test the device. So ultimately we'll have to wait and test it out for ourselves. One interesting part of the announcement is that rather than running a purpose-built model, we'll actually be model agnostic. The system can draw upon Amazon's Nova family of models, as well as Anthropix's Qlod 3.7 depending on which is best suited to the task.

Now, in case it wasn't clear, this new AI Alexa is a big bet for Amazon and is very important. Despite shipping hundreds of millions of Alexa-enabled devices, that division has actually lost over $25 billion since 2017.

The original thesis had been that consumers would order more from Amazon with Alexa facilitating the sales, but that never really played out. The AI upgrade was viewed as a product that could drive subscription revenue directly. Amazon is starting out with Alexa Plus as a $20 per month subscription, but will offer it for free to Amazon Prime subscribers.

Importantly, the upgrade will be compatible with almost every Alexa device. And so what we've got on our hands here, friends, is that this will be the first big test of household agents. Will we see a breakthrough of futuristic voice-activated smart homes or an embarrassing failure that shows that we just aren't ready for primetime yet?

Gavin Purcell of the AI for Humans show wrote, It took forever, but Amazon has finally shown up in consumer AI. Can't wait to try this. Alexa is the voice device in homes. We literally have five of them, but really only have been useful for alarms and reminders. If this works, Amazon immediately becomes way more AI relevant.

Investor and builder Yohei writes,

I don't have much to add to all this, other than I think that it is very smart to make this free for Prime members. I think that it's going to be harder than Amazon probably anticipates to get people to sign up for a new subscription for Alexa Plus, but I do think as a value add for a Prime subscription, it's incredibly valuable.

This will also get a lot of usage right away, and also the fact that it doesn't require upgrading devices significantly increases its chances of success in my estimation. Basically, people are just going to have a totally different threshold for how good it has to be if they can do it with the devices that they already have installed and it's bundled in with the subscription they already pay for. I think people will be a lot more forgiving and a lot more experimental and a lot more of a partner to help Amazon figure out where to actually drive value here.

Next up today, Perplexity continues its breakneck pace, announcing a $50 million investment fund for seed and pre-seed startups. Now, the majority of the capital will come from outside investors, although Perplexity is using some of its recent growth capital to anchor the fund.

On the one hand, some people are a little surprised to see a startup as young as Perplexity getting into this sort of game. But at the same time, I think Perplexity sees itself as competing with some of the biggest giants out there. It wants to be able to invest in its own ecosystem. It wants to keep developers close. And my guess is that they see this as a way to help that strategy.

Lastly today, an interesting one from Anthropic. Claude 3.7 Sonnet might have been kind of cheap to train. After sharing his testing of Anthropic's new flagship model, Professor Ethan Mollick was contacted by the team. They told him, quote,

This seems to demonstrate how much training costs have collapsed over just a few years. Sam Altman has said that the GPT-4 training run in 2023 cost upwards of $80 million, while Google's Gemini Ultra training in 2024 cost around $190 million.

Of course, getting these numbers out there is pretty important right now for US labs given the hype around DeepSeq. That company claims that DeepSeq V3 was trained using just $6 million worth of compute. And Anthropic CEO Dario Amadei had previously made the point that an apples-to-apples comparison shows US labs are capable of similarly frugal training on leading models. Last month, for example, he claimed that Claude 3.5 Sonnet had also been trained for a few tens of millions.

Still, Amadei does expect that future training runs on next-generation clusters could see costs in the billions, but also deliver improvements to justify the price tag.

Anyways, friends, interesting things happening out there in the world of AI. That is going to do it for today's AI Daily Brief Headlines Edition. Next up, the main episode. Today's episode is brought to you by Vanta. Trust isn't just earned, it's demanded. Whether you're a startup founder navigating your first audit or a seasoned security professional scaling your GRC program, proving your commitment to security has never been more critical or more complex. That's where Vanta comes in.

Businesses use Vanta to establish trust by automating compliance needs across over 35 frameworks like SOC 2 and ISO 27001. Centralized security workflows complete questionnaires up to 5x faster and proactively manage vendor risk. Vanta can help you start or scale up your security program by connecting you with auditors and experts to conduct your audit and set up your security program quickly. Plus, with automation and AI throughout the platform, Vanta gives you time back so you can focus on building your company.

Join over 9,000 global companies like Atlassian, Quora, and Factory who use Vantage to manage risk and prove security in real time.

For a limited time, this audience gets $1,000 off Vanta at vanta.com slash nlw. That's v-a-n-t-a dot com slash nlw for $1,000 off. If there is one thing that's clear about AI in 2025, it's that the agents are coming. Vertical agents by industry horizontal agent platforms.

Agents per function. If you are running a large enterprise, you will be experimenting with agents next year. And given how new this is, all of us are going to be back in pilot mode.

That's why Superintelligent is offering a new product for the beginning of this year. It's an agent readiness and opportunity audit. Over the course of a couple quick weeks, we dig in with your team to understand what type of agents make sense for you to test, what type of infrastructure support you need to be ready, and to ultimately come away with a set of actionable recommendations that get you prepared to figure out how agents can transform your business.

If you are interested in the agent readiness and opportunity audit, reach out directly to me, nlw at bsuper.ai. Put the word agent in the subject line so I know what you're talking about. And let's have you be a leader in the most dynamic part of the AI market. Hey, listeners, want to supercharge your business with AI?

In our fast-paced world, having a solid AI plan can make all the difference. Enabling organizations to create new value, grow, and stay ahead of the competition is what it's all about. KPMG is here to help you create an AI strategy that really works. Don't wait, now's the time to get ahead. Check out real stories from KPMG of how AI is driving success with its clients at kpmg.us slash AI. Again, that's www.kpmg.us slash AI.

Now, back to the show. Welcome back to the AI Daily Brief. If you are a long-time AI Daily Brief listener, you'll know that for as long as there has been a show, there has been an undercurrent on Wall Street of looking for some crack in the foundation of this AI shift, which has been at many points over the last two and a half years, completely propping up markets.

The latest chink in that armor and thing to get people all chattering is the notion that Microsoft might be shifting its strategy away from incredible compute consumption, which suggests maybe to some that the bubble is finally popping. Now, the specific proximate catalyst for this conversation was that last Friday, stock analyst at TD Cowen released a research note claiming that Microsoft was pulling back from their data center buildout.

Where they were getting this information is so-called channel checks, basically inquiries with sources in the supply chain. Those sources claim that Microsoft has canceled U.S. leases totaling a, quote, couple of hundred megawatts across at least two data center operations. In addition, the company has pulled back on converting statements of qualifications into new leases and is reallocating a, quote, considerable portion of their international spend back to the U.S. The TD Cowan analysts argue that these actions, quote, point to a potential oversupply position for Microsoft.

Now to level set, what we are doing here, both us on this show but also the market more broadly, is interpreting reports of behavior that could have several explanations. And of course, which explanation you choose to believe is likely to say as much as your priors and your positioning in the market as it is about what's actually going on. One popular interpretation has to do with the idea that this is a shift in demand expectations from OpenAI.

The logic of this for these folks goes that in 2023 and the first half of 2024, Microsoft had been the most active lessee of data centers, snapping up as much capacity as they could to service OpenAI's demand growth. Microsoft is no longer in that role, leading some to believe that medium-term demand has softened since last year.

Zach Vaupin on Blue Sky summed up the one interpretation here, writing, "...the canary just died. I think we're still probably many months to a year away from a total AI crash, but data centers are the only profitable aspect of AI in both the short and long term. And if Microsoft has lost faith in their ability to profit from them, that marks the beginning of the end."

Now, you can probably tell from my tone that I am not among those who has this interpretation, which is not to say, as I have made clear before, that I think it's inappropriate for Wall Street to reprice their expectations around AI and to better calibrate the risk of overspending on infrastructure build-out. But of course, this being the world that we live in, the conversation is never that nuanced. Instead, the conversation is bubble or not. And Wall Street has been looking for the AI infrastructure bubble to pop for a very long time now. You

You might remember last summer when things got boring and Goldman Sachs put out the report, Gen AI, too much spend, too little benefit. Of course, at the core of that was a question of the profitability of AI CapEx. Sequoia, for their part, reiterated this narrative with their blog post by David Kahn called AI's $600 billion question. Then there were news of delays for NVIDIA's Blackwell GPUs, the deep seek moment, each causing a stumble for AI stocks. In other words, market participants are actively looking for a reason to short big tech.

And it's understandable. Another thing that I've said frequently that I still continue to believe is that for basically the entire rate hiking cycle, the AI narrative was the one countervailing force. It was the thing competing with Jerome Powell getting up in front of the press and hiking rates every time the FOMC had a meeting. Subsequent to the end of the hiking cycle, it's felt to be very much like the market has been looking for an excuse to reprice AI stocks, but the sheer utter profitability of companies like NVIDIA just hasn't allowed them to.

But let's hold aside for a moment the Wall Street and market aspect of this and just talk about whether there's evidence that Microsoft's strategy specifically is changing. We do have a number of public statements over the past few months that suggest that Microsoft is mindful about getting out over their skis on the AI build-out. In October, the Information reported that OpenAI leadership didn't think Microsoft was doing enough to supply data centers. The potential falling out between the companies did eventually seem to be smoothed over. However, Microsoft also released OpenAI from their exclusive deal.

OpenAI is of course now sourcing their own compute, partnering with SoftBank and Oracle to build the project Stargate Data Centers.

At the same time, Microsoft CEO Satya Nadella has reinforced that his company is going to spend $80 billion on AI data centers in 2025. Then again, on the other side, earlier in the AI CapEx cycle, Microsoft had acknowledged that they were lagging behind. During earnings calls for the last two quarters, the company has forecast slowing growth for their cloud division. While data center capacity was a constraint on growth, CFO Amy Hood said in January that they expect capacity constraints to lift by the end of the fiscal year in June.

This context all feeds into statements made by Satya Nadella during last week's appearance on the Dwarkesh podcast. He said, "...at some point supply and demand have to map. That's why I'm tracking both sides of it. You can go off the rails completely when you're hyping yourself up with supply side versus really understanding how to translate that into real value to customers." Elsewhere in the podcast, he commented that supply is guaranteed to be overbuilt and that he's happy to be a leaser. Nadella even gave a time frame, expecting that a supply glut will drive compute prices down from 2027 onwards.

Many breathless analysts took these statements as Nadella calling for the AI infrastructure bubble to burst, whereas to me, it looked like a fairly sober read of the situation, designed to better explain how Microsoft is thinking about all of this. Now when it comes to this specific TD Cowan note, Microsoft has denied it or at least pushed back on the implications. During an investor conference in Sydney on Monday, the company reiterated their $80 billion CapEx commitment and denied any change to their data center strategy.

A spokesperson told the press, Thanks to the significant investments we have made up to this point, we're well positioned to meet our current and increasing customer demand. Last year alone, we added more capacity than any prior year in history. While we may strategically pace or adjust our infrastructure in some areas, we will continue to grow strongly in all regions. This allows us to invest and allocate resources to growth areas for our future.

Further reporting on Monday also revealed that canceled leases were in Wisconsin and Georgia, suggesting that there could be a regional element to the decision-making. And with all of this, the other side of the analysis started to come out as well. AI entrepreneur Shepel M suggested that analysts are getting ahead of themselves. He posted, I'm seeing lots of people with little experience in data centers coming on Microsoft. Here's my analysis.

First, let's keep this in proportion. I have the numbers in front of me, and 200 megawatts of data center capacity represents about a 2.5% change in Microsoft's current self-buildout development pipeline. To say nothing of their wholesale co-location and built-to-suit sites. Second, hyperscalers regularly take call options on data center schemes and then don't trigger the options. It costs them pennies to do so and gives them great strategic flexibility. Third, with the AI rush now a couple of years in, it makes sense that the hyperscalers are fine-tuning their requirements. You shouldn't make a long-term forecast based on short-term ups and downs.

Is AI here to stay and will it demand a lot more power? Yes. Is it likely that from time to time there will be less demand for AI capacity? Also yes. Fourth, I'm not surprised Microsoft would be reallocating IT capacity back to the US, with big tax breaks on the horizon and a government that is looking to reward increased domestic expenditure.

In fact, going even farther, there's an element of this story that is an extremely selective reading of what's going on. Last week, Microsoft also filed an application to expand their footprint in San Antonio, Texas, adding two data centers for $350 million apiece. Fired Up Wealth believes this could simply be analysts pushing the market around, commenting, "...Wall Street wants to tear down the AI thesis temporarily. It's the simplest way to pull the entire market down."

Semiconductor analyst Fabricated Knowledge had flagged these lease cancellations to his subscribers back in December and posted, "'It's all so tiring. We had the Blackwell delays echo in media like four times longer than it needed to be, and I think we will have that in the Microsoft Data Center pullback. Guys, it already happened and pretty broadly known. Somehow the broker notes just got out now?' Everyone closer to this was like, "'What are the notes talking about?' It's pretty clear it's a quarter-old news revived again.'"

So ultimately what you have here is potentially old news with Microsoft denying the implications that analysts are trying to write, the people who are squawking the most loudly about it being already short AI, and ultimately everything cascading forward at an ever-increasing rate.

Indeed, one might ask, are there any other stories which would suggest shifting fortunes or strategy among big tech when it comes to this data center build-out? Well, let's look over at Meta. The company is apparently in talks to construct a new data center campus, which could cost upwards of $200 billion. According to the information, Meta executives have reached out to data center developers on the project. They're reportedly considering sites in Louisiana, Wyoming, or Texas, and senior leaders have visited potential sites this month.

If this comes to fruition, it would be a massive ramp-up in CapEx for Meta, who are planning to spend $65 billion on infrastructure this year. A Meta spokesperson denied the reporting, stating that their CapEx plans have already been disclosed and anything beyond that is, quote, pure speculation. And yet, it appears in many ways that AI companies are sorting into two groups. Those who are spending big on data center expansion, and those who are not only spending big, but taking it a step further and planning multi-year megaprojects. If this plan comes to fruition, Meta will join OpenAI in the latter group.

Interestingly, this week, Apple also announced $500 billion in U.S. investment over the next four years, including a new AI server manufacturing facility in Houston. It's a little unclear how much of the $500 billion is new spending as opposed to money already committed. For example, development costs for Apple TV shows were included to ensure the announcement was as large as possible.

And then, of course, we have to talk about NVIDIA earnings. And the TLDR here is, man, if the AI infrastructure build-out is slowing, NVIDIA isn't seeing it. At least not yet. During yesterday's earnings, the company reported better-than-expected earnings for Q4. Their forecasts also exceeded expectations. The only real knock was that forecasts are less eye-popping, with NVIDIA no longer outperforming the wildest of Wall Street expectations. CEO Jensen Huang said, "...we will grow strongly in 2025. Demand for Blackwell is amazing."

Supply chain issues with the company's new chip have been cleared up and full-scale production is ramping. Jensen added, We have a fairly good line of sight on the amount of capital investments in data centers. We know going forward the vast majority of software will be based on machine learning. We have forecasts and plans for our top partners. The startups are still quite vibrant. Each one of them needs a fair amount of computing infrastructure.

Now, recently, one of the big concerns had been that DeepSeq's cheap training cost, or reported cheap training cost, we should say, implies a reduction in demand for chips. Huang dismissed the idea, stating, "...future reasoning models can consume much more compute."

And indeed, finally the idea is starting to be recognized that these reasoning models require much more resources and are likely a demand driver regardless of how cheap they are to train. This is hard to overstate. Compute costs for a reasoning model come at the point of inference when it's actually being used, not just in the training. The idea that everyone is hung up on the demand for compute going down because training costs are going down totally misses the way that this is actually rolling out to the world.

Huang also made the obvious but somehow still under-recognized point that we have an entire additional wave of AI coming that not only includes agentic AI for enterprises, but physical or embodied AI for robotics. I think investor Nick Carter summed it up best, NVIDIA beats as it becomes clear that reasoning models absolutely inhale compute.

So friends, like I said at the beginning, I think it's completely reasonable for Wall Street to be constantly re-evaluating how it thinks about how to price current AI revenue, a potential mismatch between how much is being spent and how much can be made. All of these are reasonable considerations. What I think gets silly is the way that these stories get amplified in media, and we whipsaw back and forth between AI hype talk and AI bubble talk.

Anyone denying at this point that this is a structural shift with radically transformative effects on the economy is just totally missing the point. However, that is going to do it for today's AI Daily Brief. Appreciate you listening as always. And until next time, peace.