AWS CEO Matt Garman Lays Out Amazon’s AI Plans
46:29 Share

This is an iHeart Podcast. Discover how one of China's largest financial services companies serves 240 million customers with AI-powered solutions. Hear from Ping An's co-CEO and chief scientist. We're a goldmine of data integrated to provide tailored solutions to our customers. Public domain tax 3.2 trillion tokens, 7.5 billion images. Tune in to our technology-powered growth podcast.

Find it at group.pingan.com, Spotify, or Apple Podcasts.

Developers like you are building the future, but you need the right tools to move fast and go further, right? That's where Microsoft comes in. With tools like GitHub Copilot, VS Code, and Azure AI Foundry, you have everything you need to push the limits and bring your ideas to life faster. And with security, compliance, and responsible AI built in, you can focus on what matters most, building the next big thing. Learn more at developer.microsoft.com.

Bloomberg Audio Studios. Podcasts. Radio. News. From the heart of where innovation, money, and power collide. In Silicon Valley and beyond. This is Bloomberg Technology with Caroline Hyde and Ed Ludlow. ♪

Live from San Francisco, this is Bloomberg Technology. Coming up, U.S. President Trump accuses China of violating an agreement with the U.S. to ease tariffs, escalating tensions between the two countries. Plus, sales of Apple's iPhone are set to take a big hit from Trump's new tariff policy. We get into the numbers and we speak with the CEO of AWS as the leading hyperscaler runs faster on Apple.

What a week it has been in the world of technology. I'm looking at the NASDAQ 100. It is a mega cap and technology heavy index. And over the course of the week, we're up a percentage point. We're actually lower in Friday's session. But this is a rebound from last week where earnings and that tariff story have been the driver of the markets and the headlines. There are two names in particular that we have been so focused on. Those names are NVIDIA. Of course, we had earnings 24 hours ago.

38 hours ago, 48 hours ago, and then we spoke to Jensen Huang. It's giving some of the gains that it has seen back, but there is a big question still about China and cross-border trade of the lead-edge technology. And then Apple again, softer, four-tenths of 1%. Just 24 hours ago, Nvidia was the world's most valuable company.

who'd regained some status as a $3 trillion cub, we're under pressure. Now, here's the top story. China's planning to allocate $70 billion of capital in a new effort to fast-track new infrastructure projects and shore up its own economy from ongoing U.S. tariffs. That's according to sources. Meanwhile, President Trump reigniting those tariff war concerns, posting on True Social that China has, quote, totally violated its agreement with U.S. or us.

offering no details of what agreements China could have violated. Let's get more from Bloomberg's Mike Sheppard. Do we know what the president is talking about here?

Well, we haven't gotten a clear signal from him directly about what had set him off this morning. But we have seen, Ed, over the past few weeks since that sit-down in Geneva between Treasury Secretary Scott Besson and his Chinese counterparts, sign that that little agreement, that that detente that they had forged there in Switzerland was starting to come apart.

the Chinese government had objected to some export controls that the US has increasingly been applying to China, keeping advanced technology from getting in there. And of course, this has been a sore spot for Beijing for years with limits on exports of semiconductors, but they have been escalating. And in more recent weeks, we even saw the US go after makers of chip design software, telling them essentially, you can't sell to China without violating our export restrictions.

So that is something that already Beijing was seeing as a violation of the spirit of those conversations. We had heard that a week ago. But now today we hear from the top US trade negotiator, Jameson Greer, talking about rare earths and critical minerals.

And during that discussion in Geneva, the two sides had agreed that Beijing might lift some of the barriers that it had placed in retaliation for U.S. export curbs on exports to the U.S. of critical minerals. So there is a bit of tit-for-tat that we are seeing here, and it may be bubbling over in the form of Donald Trump's truth social feed here today, Ed.

You know, I'm reading the news on the Bloomberg terminal and I go back to what Michael Kratios, who runs the White House Office of Science and Technology, told us. The policy from this administration is promote and protect. Protect is the tariffs bit, right? Promote is on-shoring and stimulus for the tech industry. China is doing the same thing, right? It's trying to drum up some money, $70 billion we're reporting, for its own industries in China. What do we know about that?

Well, that's a good question. I'm glad you brought that up because it's unclear whether the administration here is responding to that report of additional Chinese stimulus toward artificial intelligence and other high-tech projects that they have planned as a way to

spark their economy and make sure that they have some domestic demand to satisfy the industry it is trying to cultivate. But nonetheless, it would not sit well with the U.S. administration here because they are trying to bring some of that manufacturing to American shores, to try to build up the American tech manufacturing base. And we've seen so much of it, especially in semiconductors, over decades move to Asia.

And Donald Trump has been applying pressure to none other than Tim Cook, the Apple CEO, to try to get the iPhone production moved here. Of course, in our conversations with Mark Gurman and so many others, we know that that is a bit of a pipe dream. But nonetheless, it is a priority for the administration. And so you do see, as you said from your interview with Michael Kratios, this desire to promote American tech here at home, but also protect it against the challenge it sees coming from China.

Bloomberg's Mike Shepard, do not go far. I have a sense more news is coming on those topics. Meanwhile, sales of Apple's iPhone and those of its rivals are set to take a significant blow from President Trump's tariffs. IDC research showing smartphone growth of just 0.6% post-tariffs.

For more, Nabila Popal, IDC Senior Research Director, joins us. I always track your data, not just the backward-looking data, but extrapolating out and looking forward. You've kind of now got the post-tariff scenario. Just explain what you published overnight and how this impacts, in particular, Apple. Hi, thank you. It's nice to be here, as always.

Yeah, so exactly. And whenever we publish our forecast, we always like to show what was the scenario before, right? So in our latest forecast, which we just published, we're showing a flat growth. And it's really important to see how that's changed, right? Because 24...

was the year of, we had like 23, basically we've been in a few years of decline and then 24, exactly, thank you for showing that up, was a big year of growth. And then 25 was supposed to continue this growth, but we were supposed to have about

2%, 2.3%, you know, continued recovery. But since February, when our last forecast was published, we've been going through such tumulus times, right? There's been a whirlwind of uncertainty. You've seen that word thrown around many times. We've had to pull down the forecast because of, obviously, high uncertainty, so much tariffs, supply chain issues.

turmoil, and there's a lot of just soft demand across the board in large global markets. I'm trying to understand policy and how you factor policy into that model, right? So the situation was that smartphones were given a tariff exemption. That was mid-April. Then President Donald Trump, just this month, May, said that for smartphones,

made outside of America, there should be a, well, let's be honest, not just smartphones, the iPhone made outside of America, there should be a levy of 25%. How have you factored that in?

So, you know, that came out right actually after we had finalized our forecast. So we, you know, but we had taken into consideration, right, because we knew that even when the exemptions were announced or whatever, wherever the tariffs stood, that it was all temporary, that things could always, you know, change. So we had to draw a line in the sand at some point. But

There was, you know, we did bake in that there is, things could always change. So what we're saying to our clients or anyone that, you know, essentially asks us this question is that there is a huge downside risk to the U.S. forecast. And the tariffs, the 25% essentially brings, you know, would impose tariffs

the U.S. market, right? It would impact increased prices. It would change the demand, right, to the U.S. market. Right now we're expecting the U.S. market to grow about 1.9 percent, which is actually U.S. and China are what are holding up even that flat 0.6 percent growth that we're seeing globally. But it was pulled down. Both those large markets were

were previously expected to grow higher. And if this 25% were to go into effect, the US market would potentially even go into decline. We should say that this data from you and the news flow, it doesn't just apply to the iPhone, right? It also applies to other smartphone makers like on Android OS as well. But this has been our focus, which is Apple.

and the pressure that Tim Cook has been under from this administration, the supply chain is a question mark. But right now, do you envisage any impact to pricing of iPhone or any giant shift to where that company manufactures that handset?

So I'll address the second part first, right? Even when President Trump announced this potential 25% for anywhere, he even said that it's not just Apple, it would be to any brand, that as long as phones coming into the U.S. were not made in the U.S., they would face potentially 25% or anything, wherever they land, right? We do think that despite wherever the tariffs fall,

happen to land eventually. If they do, regardless of that, we think smartphone OEMs will continue to diversify outside of China and Vietnam and India. We said this in our press release, will continue to be the two primary global hubs for smartphone manufacturing for reasons that everyone on your show that has come on to say, right, because of the supply chain ecosystem that has already been established there, a

prior to all this tariffs madness that began earlier this year, right? It's really hard. I think anyone that says smartphone manufacturing, I mean, I understand the ideology behind bringing manufacturing to the U.S., and this question has also been asked. It's a dream, but I don't see any reality where that is possible. So it's just...

So that's the reason why we do feel that those two regions are going to continue to be the focus of OEM's diversification plans. IDC Senior Research Director Nabila Popad, it's great to have you on the show. Thank you so much. Now coming up, UiPath raised its outlook, but competition in agentic workforce solutions, it's intensifying. We'll talk to CEO Daniel Dines about what that means for his company. That's next. This is Bloomberg Technology.

Automation software company UiPath raised its full year sales outlook yesterday when it released its earnings. The company also stressed that its public sector clients renewed contracts. Let's get more on this with UiPath CEO Daniel Dines. Daniel, it's great to have you back here on Bloomberg Technology. A point of discussion for most of the year so far has been the impact of Doge.

on companies just like yours that have government, federal level contracts. I'm trying to read between the lines in the transcript of the earnings call. Was there any impact to you, positive or negative, from that Doge activity? Thank you for having me. Look, we had a great event in DC a month ago and we talked with a lot of agencies. There is a renewed interest in agentic automation as a space

And we basically renewed our contract. We have a great deal with the Air Force. They are coming with a great initiative called Airmen, which is aiming at a very transformative of their business processes.

So all in one, I think it's a more positive environment in DC, but still there is a lot of moving pieces in transition there. So from our perspective and our guidance, it's not a big difference. Many point to net new ARR additions and then being low. What then accounts for that?

Well, we continue to execute well. We just finished a big transition. We had an entire year of changes in the company. We accommodate everything to go really big in a genetic automation, basically. What is the growth picture? Our analysts write that growth indicators are mixed for UiPath. Would you agree with that analysis of it being mixed? And if you do see growth, where is it coming from?

Well, I think that agentic automation is a tremendous opportunity and I think everybody agrees that this is going to be very transformative on the business landscape. We believe this is going to be a bigger opportunity than RPA and we are really well positioned to capture a significant share of it. So, all in all, I think this is really our biggest opportunity for growth.

I'm looking at the cost of inference right now, particularly in the context of companies just like yours trickling down. A lot of companies are talking about the momentum that agentic AI is giving them. What's your point of difference? What is it that you're able to offer that others are not?

I think a lot of agents that we are seeing today are conversational agents. But customers would like agents that are deployed in the context of their existing business processes. And how can you trust the agents? It's the main point of the conversations. And we offer technology to orchestrate between customers.

RPA bots and agents and humans that really enhance the reliability of agents and the trust of customers. Right now, what is the biggest cost consideration for you? I talked about inference costs coming down. Even so, investors in particular love to see a commitment to access to compute and also just some like long term commitment that the investment is worthwhile.

Yeah, I would not say that in our business the interest cost is significant. The real cost comes with implementing of the agents. Still, the capabilities required to create really good agents that mimic the capabilities of humans are kind of rare. And we are working a lot to build technology that

help to democratize the ability to build agents. To me, this is going to be the real cost of deploying agentic automation at scale. Daniel Dimes, UiPath CEO, thank you so much for joining us here on Bloomberg Technology. Now, coming up, Microsoft's push to get corporate customers on board with Copilot seems to be taking off. We've got the details on its AI sales coming up next. This is Bloomberg Technology. ♪

Discover how one of China's largest financial services companies serves 240 million customers with AI-powered solutions. Hear from Ping An's co-CEO and chief scientist. We're a goldmine of data integrated to provide tailored solutions to our customers. Public domain tax, 3.2 trillion tokens, 7.5 billion images. Tune in to our technology-powered growth podcast.

Find it at group.pingan.com, Spotify, or Apple Podcasts. The world is built on code. From the apps we use every day to the systems powering industries, developers like you are the architects of tomorrow. But let's be real. The road to innovation can get a little tricky. You need the right tools to move fast, but you also need a community to help you go further.

That's where Microsoft comes in. Microsoft has the tools to help you move at lightning speed, like GitHub Copilot, VS Code, and a ton of AI resources to keep you on the cutting edge. But here's the best part. You can build with confidence, knowing that Microsoft's security and compliance are already taken care of. No more worrying about vulnerabilities or threats while you focus on your craft.

And with Azure AI Foundry, you can build your way. The future is yours to build, no strings attached. From ready-to-code tools to full flexibility, it's all in one place. The future's in your hands. So learn more at developer.microsoft.com.

A growing push for sovereign AI has led to a wave of data center announcements, including one this week from Bell Canada. It plans to invest hundreds of millions of dollars to build AI data centers across the country. And it has named chip startup Grok as the project's exclusive inference partner. Grok CEO Jonathan Ross spoke with Caroline Hyde about sovereign AI, but also US chip curbs on China.

From Grok's beginning, we have done no business in China. And this is not a geopolitical thing. This is just a shrewd business thing. We saw all of these very large tech companies go into China and lose over and over again. And there's just a thumb on the scale for any Western company.

I hope that someday China allows real competition because real competition makes people stronger and I want to compete against those Chinese companies. But until they allow real competition, we can't do that. So we've blocked all Chinese companies from having access to our API. We will not sell chips in China because they're just going to reverse engineer them and try and build them themselves and they're not going to compete fairly.

As a result, when we do these deals, we're not allowing Chinese companies to run on them. And we do build, operate, transfer. In these sovereign cases, we're doing build, operate. So we're the ones operating that. And that actually gives commerce and others a lot of comfort that Chinese companies aren't going to be getting access. But does the restrictions on, say, NVIDIA selling H20s, as Jensen would say, mean that China just gets better at doing it themselves and Huawei will be able to compete with Grok in the future?

Potentially, however, the reality is if you enable people now, then they're going to be able to use that advantage to keep growing quicker. We're going to be using AI to design our chips. We're going to be using AI to design our software. So why would you give that advantage? I think it's different. Going back to information age technology, it's about replicating and distributing.

Compute is about doing something specific. If you don't have the compute, you can't do it. Whereas information knowledge sort of diffuses and you'll catch up, with generative age technology, you can pull ahead.

You talk about wanting competition and to be competitive in China, for example. What about the competition with NVIDIA? How do you stand compared to AMD and the other offerings here in the United States? We actually think we're one of the best things that's ever happened to NVIDIA shareholders. So NVIDIA sells a product that is a premium product. It's a luxury good. It's for training the models. When you train a model, you're willing to spend more money because you get to amortize that across an enormous number of users.

If NVIDIA continues focusing on inference, which they have to do right now because there isn't enough inference capacity, they're going to have to bring their margins down. They're going to have to sell their chips for a lot less. Remember, our cost is much less than NVIDIA's cost. On top of that, higher volume, lower margin. And so the more inference compute we deploy,

The more demand there is for training compute, the more demand for training compute, the more GPUs NVIDIA sells. And the one thing that's really important here, NVIDIA wants to do between 3 and 5 million GPUs for AI this year. But that's based on what's called HBM, high bandwidth memory. It's this very scarce component. They can make as many GPU chips as they want. They can't make as much of the memories they want. Those chips are going to be built and sold. And the only question is, what are they going to be sold to do?

When we come in and we sell inference chips, then what we're doing is we're allowing those GPUs to be sold into training, which is a higher margin business. NVIDIA is going to make more profit thanks to us. All right, we have some breaking news crossing the Bloomberg terminal. UTEL SAT, satellite-based internet providers, in talks to raise 1.5 billion euros.

This is interesting because in Europe, the continent's hoping that UTEL sat will be able to displace and compete against the likes of Elon Musk, Starlink in Europe. But that's the red headline. UTEL sat in talks to raise 1.5 billion euros with a doubling of the stake from French entities. We'll get more on that story as we get more on it.

All right, let's get back to what's happening in the world of AI. Microsoft is focused in driving adoption of its AI tool, Copilot. And at a town hall this week, it touted its progress. Who broke the story?

Bloomberg's Brodie Ford. I think this is really interesting, right, because it's quick on the heels of this deep dive we did in Businessweek about Copilot, how it happened, how it launched, and what Microsoft's hopes are for it. What you reported are names across various industries, big corporate names that are doing deals.

Right. Yeah, the background here is that Microsoft, Salesforce companies like them. They have these AI tools, right? I mean, in the consumer space, ChatGPT and tools like it have ramped incredibly. In the enterprise, it hasn't been quite as easy of an adoption path.

And so with tools like Copilot, Microsoft's AI assistance suite, we don't have great intel over how many folks are really using it, how many folks are really paying for it. What we learned from the meeting yesterday, which sources were telling us about, is that

You know, there are dozens of customers who are having over 100,000 paid users. That's a pretty significant amount. I mean, of course, you know, there's some discounting in that, but this is a sign that adoption is ramping, maybe more than investors had been appreciating.

As part of your reporting, we know that Microsoft CEO Satya Nadella is very closely tracking all kinds of data, including who is using Copilot and how they're using it. Why is that important? Right. I mean, what's really important is that you don't have shelfware, that you don't buy 100,000 licenses, but, you know, 50 people use it in finance, right? I mean, this is the classic problem with software where maybe you can sell a lot of things, but you have to really make sure they're using it.

Bloomberg's Brodie Ford, another excellent piece of reporting. Thank you very much. Okay, coming up, Patrick McGoldrick from J.P. Morgan Private Capital joins us to talk about its approach to AI investment opportunities. That conversation's next. This is Bloomberg Technology.

Welcome back to Bloomberg Technology. I'm Ed Ludlow in San Francisco. It's kind of like the last day of technology earnings season, and we're seeing movers in the market on some numbers that got posted last night. Those of Dell and Marvell. In Dell's case, it was all about the profit outlook. It topped estimates. There is clearly a backslide

for AI server demand, but shares are a little bit softer. Marvell, a slightly different story. They kind of have like strong sales targets. They've won some business and they were pretty clear with the market, but investors still tepid on that chip name.

stock down 5%. That's having an impact at the index level as well. Let's go from public markets now to private markets. JPMorgan Private Capital's Growth Equity Partners. It's a $1 billion fund focusing on Series B right through to pre-IPO startups. Managing Partner Patrick McGoldrick joins us. The focus right now, the AI opportunity. I find this so interesting and it's part of today's VC Spotlight.

In my world, when I break a story about an AI startup or a new round, it's so interesting to look at the cap table. Traditional venture capital, traditional, but also people like you coming in more in the private growth equity space. Would you just talk a bit about that business model and what you try to do?

Sure, and thanks for having me on today, Ed. Appreciate the opportunity. So the way we position our strategy is to operate like a traditional venture and growth equity investor. So we start investing at the earlier stages of a company's formation, Series B through pre-IPO on the technology and consumer side, all the way through that last inflection point before going public. I think for us, we seek to leverage the whole firm to deliver value to companies. And in a world where capital is just that,

but value add in changing the inflection curve of a company's growth, de-risking, we think JPMorgan is well suited to be that partner of choice. So interesting. You're talking about leveraging the full scale of JPMorgan, right, as your pitch. The traditional VC would say, don't just take our check, take our operators, our knowledge. Many of us are entrepreneurs or startup founders. Your pitch is we're JPMorgan.

Yes, I think in the simplest of essence, of course, I think we're privileged to be part of an organization that spends $18 billion a year in technology spend. We have 63,000 technologists, and I think what is really important to recognize is we will work with companies across our firm as early as the seed stage. So we're on the hunt for exceptional technologies. Now, as investors, of course, first and foremost, we assess product market fit, the strength of the management team, but then where can we pair in

those relationships covering 90% of the Fortune 500 or 45,000 private companies. So we have the privilege of those operators that other VC firms have, but I think the global footprint of a firm like JPMorgan. What are you seeing in private markets right now?

It's a complicated question. It depends on the topic, but let's start with where we see the investment opportunity set. I think for us, we're spending a lot of time in areas like artificial intelligence and cybersecurity, both incredibly resilient from a tech spend perspective. There was a recent

review of chief information officers and chief technology officers budgeting priorities and i think for different reasons you're seeing increased commitments to those areas in ai i think it's a world of now having gone from experimentation to actual deployment over 65 percent of companies are now deploying ai in their full production suite those companies are now automating tasks they're creating efficiency and making a more delightful consumer and enterprise experience

In cyber, unfortunately, you could have a version of me that's virtually produced, my cadence and flexion, a little bit of my positioning, and you'd have to be able to protect against that. So that defensive posture is critical. It's why even in tougher macro environments, the defensiveness of the spend there is very clear. And so you'll see that reflected in some of our portfolio companies.

Some of your portfolio companies, we just showed AlphaSense, of course, the maker of Cursor, such a hot property right now. Tell me what's going on with that, but also Databricks. Databricks recently has been relying on tenders for employee liquidity rather than a primary raise. How have you navigated those two mechanisms in these two hot names in the world of AI?

Well, let's start with AlphaSense because I think, you know, at that company, think of it really as a market research and intelligence platform that in 2008 had the idea of bringing publicly available data in the world of markets, also accessing the research reports that are put out by Wall Street firms

digesting that and giving research analysts, corporate development arms, as well as portfolio managers the right solutions at their fingertips to make informed decisions. With the advent of AI, that's gotten even more acute and clear as to how they deliver value, and they recently acquired a company called Tegas, which provides private market perspective, so customer interviews that get transcribed. That marriage of data is critical to making better decisions.

On Databricks, you bring up an interesting point. I think the evolution of the private markets are such that companies are staying private longer. In order to provide liquidity, they look to access this tender solutions and letting employees seek some liquidity.

For us, we want to be active in names like Databricks where it's critical in the world of infrastructure for AI, growing well in excess of the public markets with an exceptional management team. And so we're thrilled to be involved in both of those and think they're critical in this new advent of AI.

Patrick, we just have 30 seconds, but you see some kind of exit coming in either of those names? Well, I wouldn't want to speculate on that too much. I think the management teams are better suited to do that. What I would say about the exit environment more broadly is we've gone from a faucet that's completely shut off to a trickle where you have companies like Inge Health, Service Titan, of course, CoreWeave that have used the public markets, which is important. Hopefully, the back half of the year produces more of that, but they are

companies that could effectively go public by virtue of their performance, again, the management team and the market size. Patrick, open up that tap. Next time, come back on and tell us when you have an exit. Patrick McGoldrick, managing partner, J.P. Morgan, private capital. Thank you very much. Coming up.

We speak with AWS CEO Matt Garman in an exclusive conversation. One year in the role, so much has happened from the sort of hyperscale perspective right through to the models that AWS is hosting largely through Bedrock.

We've also got some breaking news. Let's take a look at DJT. Hitting session higher, raising its decline. Just confirmation, the company's closed about $2.4 billion in a Bitcoin treasury deal. Something we brought you a little bit of earlier this week. We'll be right back. Don't go far. This is Bloomberg Technology. Remember when a single technology glitch could bring an entire workday to a standstill? I'm Mark Banfield, Chief Commercial Officer at TeamViewer. Today, most technical issues are recurring.

If you know the patterns, these issues can be remediated before they impact your business. One major UK retailer discovered a single point of sale outage cost them around $1 million in lost sales during a single lunchtime. Now, using TeamViewer's digital workplace platform, the same company is able to identify and fix those issues before they even happen. But proactive troubleshooting isn't just about the incremental improvements.

is about fundamentally reimagining how work happens for everyone. And the companies that move first get competitive advantages in terms of efficiency, productivity, and innovation. To find out more, visit bloomberg.com forward slash teamviewer.

The world is built on code. From the apps we use every day to the systems powering industries, developers like you are the architects of tomorrow. But let's be real. The road to innovation can get a little tricky. You need the right tools to move fast, but you also need a community to help you go further. That's where Microsoft comes in. Microsoft has the tools to help you move at lightning speed, like GitHub Copilot, VS Code, and a ton of AI resources to keep you on the cutting edge.

But here's the best part. You can build with confidence, knowing that Microsoft security and compliance are already taken care of. No more worrying about vulnerabilities or threats while you focus on your craft.

And with Azure AI Foundry, you can build your way. The future is yours to build, no strings attached. From ready-to-code tools to full flexibility, it's all in one place. The future's in your hands. So learn more at developer.microsoft.com.

Welcome to our Bloomberg Radio and television audiences worldwide. We go right now to a conversation with Matt Garman, AWS CEO. Matt, it's good to catch up. It has been basically one year that you've been in the role as AWS CEO. As a place to start, what has been the biggest achievement in that time for AWS?

Yeah, thanks for having me on. It's nice to be here again. Yeah, it's been a fantastic year of innovation. It's really been incredible. And as I look out there, one of the things that I've been most excited about is how fast our customers are innovating and adopting many of the new technologies that we have.

And as you think about customers that are on this cloud migration journey, many of them have been doing that for over the last several years. But this year in particular, we've really seen an explosion of AI technologies, of agentic technologies. And increasingly, we're seeing more and more customers move their entire estates into the cloud and AWS. So it's been really fun to see. It's been an incredible pace of technology. And it's been a really fun first year.

The moment that investors kind of sat up and paid attention was when Amazon said that its AI business was at a multi-billion dollar run rate in terms of sales. What we don't understand as well is what proportion of that is AWS infrastructure?

Yeah, that is AWS, right? And so the key is that's a mix of customers running their own models. Some of that is on Amazon Bedrock, which is our own hosted models where we have first-party models like Amazon Nova, as well as many of the third-party models like Anthropix models. And some of those are applications, things like Amazon Q, which helps people do automated software development.

as well as a host of other capabilities. And so there's a mix of that. And I think part of the most interesting thing about being at a multi-billion dollar run rate is we're at the very earliest stages of how AI is going to completely transform every single customer out there. And we talk to customers and we look at where the technology landscape is, and we firmly believe that every single business, every single industry, and really every single job is going to be fundamentally transformed by AI.

And I think we're starting to see the early start, the stages of that. But again, we're just at the very earliest stages of I think what's gonna be possible. And so that multi-billion dollar business that we have today is really just the start. - Can you give me a generative AI revenue number?

For the world or for AWS? For you guys, for AWS. Maybe Amazon as a whole. Yeah, like I said, we are in multiple billions of dollars and that's for customers using AWS. We also use lots of generative AI inside of Amazon for a wide range of things. We use it

to optimize our fulfillment centers. We use it when you go to the retail site to summarize reviews or to help customers find products in a faster and more interesting way. We use AI in Alexa, in our new Alexa Plus offering, where we conversationally talk to customers through the Alexa interface and help them accomplish things through voice that they were never able to do before. So every single aspect of what Amazon does leverages AI.

AI and our customers are exactly the same. Customers are looking to AWS to completely change, whether it's their contact centers through something like Amazon Connect, where it shows AI capabilities so that you don't have to go program it, all the way down to our custom chips or NVIDIA processors or anything where customers at the metal are building their own models. We have the whole range of people that are building AI on top of AWS, as well as Amazon themselves.

We always credit AWS as being number one hyperscaler. But just what you said there about what the client's using in the silicon level through to capacity, it would really help if you could proportionately tell me what percentage of workloads are being run for training and which proportion of workloads are being run for inference. Sure.

Yeah, and that changes over time. I think, look, as we progress over time, more and more of the AI workloads are being inference. I'd say in the early stages of AI, in generative AI, a lot of the usage was dominated by training as people were building these very large models with small amounts of usage. Now, the models are getting bigger and bigger, but the usage is exploding at a rapid rate. And so I expect that over the fullness of time,

80%, 90%, the vast majority of usage is going to be in inference out there. And really, and just for all those out there, inference, it really is how AI is embedded in the applications that everybody uses. And so as we think about our customers building, there's a small number of people who are going to be building these models.

but everyone out there is gonna use inference as a core building block in everything they do. And every application is gonna have inference and already is starting to see inference built in to every application. And we think about it as just the new building block. It's just like compute, it's just like storage, it's just like a database. Inference is a core building block. And so as you talk to people who are building new applications,

They don't think about it as AI is over here and my application is over here. They really think about AI is embedded in the experience. And so it's increasingly, I think it's going to be difficult for people to say what part of your revenue is going to be driven by AI. It's just part of the application that you're building. And it's going to be a core part of that experience. And it's going to deliver lots of benefits from efficiency, from capabilities, and from user experience for all sorts of applications and industries.

But present day, it's fair to say majority is still training? No, I think that at this point, definitely more usage is inference than training. We want to welcome our radio and television audiences around the world. We're speaking to AWS CEO, Matt Garman, who officially next week celebrates one year in that role leading AWS. A new metric...

that has been discussed, particularly this earnings season, we discussed it with Nvidia CEO Jensen Wang this week, is token growth and tokenization. Has AWS got a metric to share on that front?

I don't have any metrics to share on that front, but I think it's one of the measures that we can look at is the numbers of tokens that are being served out there, but it's not the only one. And I increasingly think that people are going to be thinking about these things differently. Tokens are a particularly interesting thing to look at when you're thinking about text generation, but not all things are created equal. I think particularly as you think about AI reasoning models, the input and output tokens don't necessarily...

talk about the work that's being done. And increasingly, you're seeing models that can do work for a really long period of time before they output tokens. And so you're having these models that can sometimes think for hours at a time, right? They might ask these things to go and actually do research on your behalf. They can go out to the internet, they can pull information back, they can synthesize, they can redo things. If you think about coding and QDeveloper, we're

We're seeing lots of coding where it goes and actually reasons and does iterations and iterations and improves on itself, looks at what it's done, and then eventually outputs the end result. And so at some point, kind of the final output token is not really the best measure of how much work is being done. If you think about images, if you think about videos, there's a lot of content that's being created.

and a lot of thought that's being done. And so tokens are one aspect of it, and that's an interesting measure, but I don't think it's the only measure to look at, although they are rapidly increasing.

Project Rainier, massive custom server design project. What is the operational status and latest on Project Rainier? Yeah, so we're incredibly excited about it. So Project Rainier is a collaboration that we have with our partners at Anthropic to build the largest compute cluster that they'll use to train their next generation of their cloud models.

And Anthropic has the very best models out there today. Cloud 4 just launched, I think it was last week. And it's been getting incredible adoption out there from our customer base.

Anthropic is going to be training their next version of their model on top of Tranium 2, which is Amazon's custom-built accelerator processors, purpose-built for AI workloads. And we're building one of the largest clusters ever released. It's an enormous cluster, more than five times the size of the cluster compared to the last one that they trained on, which, again, is the world's leading model. So we're super excited about that.

We're landing Tranium 2 servers now and they're already in operation and Anthropic is already using parts of that cluster. And so super excited about that and the performance that we're seeing out of Tranium 2 continues to be very impressive and really pushes the envelope, I think, on what's possible both from an absolute performance basis as well as a cost performance and scale basis. I think some of those are equally going to be really important as we move forward in this world.

Because today, much of the feedback you get is that AI is still too expensive. The costs are coming down pretty aggressively and it's still too expensive. And so we think there's a number of things that need to happen there. Innovation on the silicon level is one of those things that needs to help bring the cost down.

as well as innovation on the software side and algorithmic side, so that you have to use less compute per unit of inference or training. So all of those are important to bring that cost down to make it more and more possible for ADI to be used in all of the places that we think that it will be over time.

Matt, on Wednesday, NVIDIA CEO Jensen Wong summarized inference demand for me. I just wanted to play you that soundbite. Sure. Well, we've got a whole bunch of engines firing right now. The biggest one, of course, is the reasoning AI inference. The demand is just off the charts. You see the popularity of all these AI services now.

Your pitch for Tranium 2, and as you know, I've kind of taken apart the server design and looked at it, is the efficiency and cost efficiency relative to NVIDIA tech. Are you seeing that same demand Jensen outlined for Tranium 2 outside of the relationship with Anthropic?

Yeah, look, we're seeing it across a number of different places, but it's not really Tranium 2 versus NVIDIA. And I think that's not really the right way to think about it. I think there's plenty of room. The opportunity in this space is massive. It's not one versus the other. We think that there's plenty of room for both of these. And Jensen and I speak about this all the time, that NVIDIA is an incredibly fantastic platform. They've built a really strong platform that's useful and is the leading platform for many, many applications out there.

And so we are incredible design partners with them. We make sure that we have the latest NVIDIA technology for everyone and we continue to push the envelope on what's possible with all of the latest NVIDIA capabilities. And we think there's room for Tranium and other technologies as well and we're really excited about that. And so we have many of the leading AI labs are incredibly excited about using Tranium 2 and really leaning into the benefits that you get there. But

For a long time, these things are going to be living in concert together. And I think there's plenty of room and customers want choice. At the end of the day, customers don't want to be forced into using one platform or the other. They'd love to have choice. And our job at AWS is to give customers as much choice as possible.

What is general availability of NVIDIA GB200 for AWS? And have you, I guess, launched Grace Blackwell-backed instances yet? Yes. Yep. So we've launched our, we call them P6 instances. And so those are available in AWS today. And customers are using them and liking them. And the performance is fantastic. So those are available today. We're continuing to ramp capacity today.

We work very closely with the NVIDIA team to aggressively ramp capacity and demand is strong for those P6 instances. But customers are able to go and test those out today. And like I said, we're ramping capacity incredibly fast all around the world and in our various different regions. Matt, what is your attitude to Claude Anthropix model being available elsewhere on Azure Foundry, for example?

Great. I mean, that's okay too. I think many of our customers make their applications available in different places. And we understand that various different customers want to use capabilities in different areas and different clouds. Our job is to make AWS, and this is what we do, is to make AWS the best place to run every type of workload. And that includes anthropic cloud models,

But it includes a wide range of things. And frankly, that's why we see big customers migrating over to AWS. Take somebody like a Mondelez, who's really gone all in with AWS and moved some of their workloads to there. One of the reasons is that they see that we have capabilities, sometimes using AI, by the way, in order to really help them optimize their costs

and have the most available, most secure platform. In Mondly's case, they're taking many of their legacy Windows platforms and transforming them into Linux applications and saving all of that licensing cost. But we have many customers who are doing that, and so our job is to make AWS by far the most technically capable

a platform that has the most and widest set of services. And that's what we do. But I'm perfectly happy for other people to use. It's great that Claude's making their services available elsewhere. And we see the vast majority of that usage happening in AWS, though. Will we see open AI models on AWS this year? Well, just like we encourage all of our partners to be able to be available elsewhere, I'd love for others to take that same tack.

Let's end it with this, a question from the audience, actually, which is where you're going to grow data center capacity around the world. I got a lot of questions from Latin America and Europe in particular, where Jensen flies to next week.

Great. So in Latin America, we're continuing to expand our capacity pretty aggressively. Actually, earlier this year, we launched our Mexico region, which has been really well received by customers, and we've announced a new region in Chile. And we already have, and for many years, have had a region in Brazil, which is quite popular and has many of the largest financial institutions in South America running there. So across Central and South America, we are continuing to rapidly expand our

In Europe we're expanding as well. We have many regions already in Europe. One of the things I'm most excited about actually is at the end of this year, we're going to be launching the European Sovereign Cloud, which is a unique capability that no one has, which is completely designed for critical EU-focused sovereign workloads. And we think given some of the concerns that folks have around data sovereignty, particularly for government workloads as well as regulated workloads,

We think that's going to be an incredibly popular opportunity for everybody. Matt Garman, AWS CEO, thank you very much. Thank you for having me.

Let's get other headlines in Talking Tech. First up, TikTok shop is cutting several hundred jobs in Indonesia in its latest round of cuts, this after taking over the operations of local rival Tokopedia last year. Sources say the cuts are mainly in e-commerce teams and more cuts are set to happen as soon as July. Plus, Reform UK leader Nigel Farage announced plans to introduce a crypto assets and digital finance bill if his party wins the next general election, aiming to launch what he calls

a crypto revolution in the UK. The legislation would include a cut in capital gains tax on crypto investments to 10%, the creation of a Bitcoin digital reserve at the BOE, and provisions that would make it illegal to restrict services for people who want to pay with crypto. And Stripe has held early discussions with banks about the potential use of stablecoins. This comes as the payment firm debuted a number of stablecoin-related products in recent months. But does it?

for this edition of Bloomberg Technology. What a week it's been. Don't forget, check out the podcast. You can find it on the Terminal as well as online on Apple, on Spotify and on iHeart. From San Francisco, this is Bloomberg Technology.

Developers like you are building the future, but you need the right tools to move fast and go further, right? That's where Microsoft comes in. With tools like GitHub Copilot, VS Code, and Azure AI Foundry, you have everything you need to push the limits and bring your ideas to life faster. And with security, compliance, and responsible AI built in, you can focus on what matters most, building the next big thing. Learn more at developer.microsoft.com.

Let's talk.

This is an iHeart Podcast.