Google Just Embedded an AI Chatbot into Search — But It Might Not Be Enough | Behind the Numbers
31:31 Share

Are your brand campaigns as effective as they could be? If you're only getting insights when the campaign is over, then the answer is no. To make better campaign decisions, you need real-time measurement. You need Lucid Measurement by Sint. Discover the power of real-time brand lift measurement at sint.com slash insights. That's C-I-N-T dot com slash insights.

Hey gang, it's Monday, June 2nd. Gajo, Evelyn and listeners, welcome to Behind the Numbers, an eMarketer video podcast made possible by Synth. I'm Marcus and joining me for today's conversation, we have two of our senior analysts. One of them covers AI and everything tech for us. Living in New York, it's Gajo Sevilla. Hi Marcus. Hi Evelyn. Happy to be here with you guys. Hello fella. We're also joined by one covering digital advertising and media based in Virginia.

Evelyn Mitchell-Wolfe. Hi, Marcus. Hi, Gadjo. Hello, everybody. Hello. Today's fact. Pluto is about the same size as Russia. Who knows if this is true? So both of them are about 11 million square miles.

Like Pluto's surface area? What do you mean? If you stretch Pluto out, like if Pluto was like a tennis ball and you cut it and then stretched it out, it would be the similar surface area to Russia. So it's smaller than the moon? I don't know. I don't know. Just thinking.

Is it smaller than the moon? Let's figure this out. Is Pluto smaller than the moon? That's why you really come to Behind the Numbers. We answer the real questions. Yes, it is. It's smaller than the moon. Not by much. By a bit. Yeah, Pluto's about two-thirds smaller.

Okay. Anyway. So it's tiny. Yeah. Tiny little guy. It's also a lot cold, like a lot colder. Average temperatures of around negative 400 degrees Fahrenheit. About negative 230 for our European friends. So cold. Like Russia. Anyway, it's not that cold. Today's real topic. How AI is changing search.

"An AI chatbot is to be embedded in Google search," writes Lily Jamili of the BBC. Catherine Blunt of The Wall Street Journal was explaining that Google has started rolling out on its search page AI mode, which answers search queries in a chatbot-style conversation without the standard list of blue links. Jay Peters of The Verge was saying Google presented an example of someone using this

to do things in Nashville over the weekend with friends who like food, music and exploring off the beaten path. AI mode sprung into action, creating Google curated lists of restaurants good for foodies, recommending places with a chill bar atmosphere with live music and highlighting places off the beaten path and suggesting sites featuring good things to do in Nashville. It even created a custom map recommending places to go. Evelyn, what's your biggest takeaway from Google embedding an AI chatbot into search?

Yeah, this is on the scale of like one to 10 in terms of big deal. I'd say this is like a six or a seven at least. So it's short term. I don't think it's really going to do a ton because it is for now, at least it's still only accessible via like a separate tab under the search bar. So users have to click into it. They have to seek out that experience in order to

to have the experience. And any additional friction is a barrier to adoption. So AI mode being embedded in the legacy Google workflow at all is a big deal. And I think eventually we will see pretty strong adoption because of the proximity to

the search habits that consumers have built over decades. But I also, I think the bigger deal here is that it is, Google is sort of completing its menu of, of Gen AI powered search options. So they have AI overviews, which require nothing extra from consumers to receive them. Then there's AI, AI mode, which is still very accessible. And it is that conversational search interface. And then beyond that is Gemini, which is the, the, you know, chat bot,

that shares a name with the LLM that powers all three. Right.

So lots of different options for consumers, whether they, you know, if they actively want to seek out those experiences or not, which I think is a smart strategy for Google as there are sort of varying levels of desire for those products among consumers. Yeah. Gaja, your biggest takeaway? So I see this as Google hedging its bets. It's evolving search by including an AI mode,

but it's not replacing search completely. And so I think by putting it alongside traditional search results and, you know, like what Evelyn said, the way people are used to searching, it's kind of using it as a client capture strategy because, sure, if you want a chatbot, you don't have to fire up another app or go to another web page. You're still within Google's wheelhouse. Now,

Usage will depend, of course, on the quality and how good it is compared to what the competitors are offering. But the fact that it's a click away, I think, will kind of yield a lot of people to at least experiment with it. And they'll have to do so by choice. Yeah. 1.5 billion people, they said, now regularly engaging with AI overviews. And they said most users are now entering longer and more complex queries. But...

They also thought that AI overviews was going to trigger more searches, right? And more clicks to other sites. That doesn't seem to be the case. Bright Edge was noting that click-through rates from Google's search results are down 30% over the past year. And they attributed to the drop to folks becoming increasingly satisfied with AI overviews. Evelyn, you very validly pointed out that

Before AI overviews debuted, a lot of the Google queries were zero-click searches anyway. So there is that part of it. But what do you make of how this is changing people's behavior in terms of searching for things? So...

There's actually, I mean, Google has said a lot of things and not provided a lot of data to back up their statements. So that's one thing that's worth mentioning because there are some third party sources specifically in the SEO area that have search engine optimization for anyone that is not super familiar with this space. Yeah.

And those sources have not found any evidence that consumers have meaningfully altered their search behavior, including the average length of queries. So they're, you know,

I think until we see some real data from Google, I would take most of what they say with a grain of salt because they're trying to shape this narrative. We saw just a few weeks ago, Alphabet stock dropped pretty drastically because

somebody at Apple, Eddie Q, I can't remember his exact title, but testified in the search antitrust remedy trial that Google search volume had dropped for the first time on Safari for the first time in like 22 years or something. So that kind of, I guess,

I don't think it's necessarily new news that there are, because there are different places that consumers can go to satisfy their, their search use case, whatever it may be. Then of course, some people are going to use those, those options and there will probably be a reduction of search queries on Google. But in terms of,

How people are asking questions. I mean, we don't actually have a ton of insight into that. And I think for the most part, because they're

proportion of consumers that are using these tools with any sort of regularity is so small, the vast majority of search behaviors aren't going to change if they're still using Google. And, you know, AI overviews are still relatively rare. I think they are served on a

just under 15% of Google queries in total. And that sort of exposure varies from topic to topic. But if consumers aren't actively seeking out that kind of Gen AI search experience, and they are only just sort of

existing and doing their regular habits and sometimes they're getting hit with an AIO, why would something meaningfully change aside from then maybe they won't click through when that AIO is served if they would have otherwise? So it'll take more time for the vast majority of consumers that aren't sort of

predisposed to use these tools and test use cases, see if maybe ChatGPT might be better for this particular thing, if AI mode might be better for this kind of thing. Not everyone is going to want to take the time to figure out what best serves them for a particular question, so they're just going to default to Google. And for now, there's still not a ton of reason for them to change their behaviors.

They are. Yeah, they are looking, people are looking in a lot of different places. I mean, Google gets a lot of the credit for being the place where people search for, you know, 90% of things or whatever. But when you, we had some research from April of this year and we were asking consumers what platforms they've used for search in the past.

12 months. And yes, Google came out on top. 93% of people said that. But then Amazon was in second place, distant second, admittedly, 56%. But then all with about kind of 20 to 50% range, Amazon, YouTube, Walmart, Facebook, Instagram, TikTok, ChatGPT, Yahoo, Target. So people are looking in a lot of different places. And it seems, Gaja, that Google knows that and they're trying to fight off the gains made by these other platforms in terms of search. They...

90% share of search is what they had for a long time. That's down a bit from 93% from a few years ago. And StatCounter was saying that actually in the last six months, Google's share of search has been down below the 90% mark for six months, which is the first time that's happened in 10 years. So it seems like Google does know that there are...

alternatives, how much do you think this development, AI mode, how much can that fight off some of its competitors? I think one thing that sticks out to me is this stat from Pearl saying 57.8% of US adults still prefer Google search over AI tools. That's assuming that they know about these AI tools. 60%. I mean, that tells you it's

It's all about habit. The value proposition would have to be pretty evident and I guess special and not cost more than Google search for at least for consumers. Yeah.

When you're looking at these standalone AI search services, there's always kind of a subscription or some limit to how much you can use the service, right? So I think those are roadblocks. Will it change? I think sure it will. These tools are getting faster. They're smarter. They're starting to buy proper data training sources from publishers.

But I think it's going to be a slow burn before we see any dramatic move away from Google, at least for most users. They are still so far ahead of everyone else. During the year ending in March, Google had 136 billion monthly visits, which would be 34 times more than ChatGPT, according to OneLittleWeb.com.

Yeah, they have. They certainly I mean, Google has a traffic advantage. And you mentioned the survey that we ran in March, too, that had Google far and away out on top that that 90 percent penetration, roughly like near or above 90 percent penetration holds across demographics. And that's something that is not true for Google.

Specifically, we're talking about Gen AI first kind of platforms like ChatGPT. There are pockets of consumers that use them and TikTok too, right? That's very much a Gen Z kind of destination when it comes to search and really for entertainment as well. But Google...

consistently across demographics, regardless of age, gender, ethnicity, race, income, employment status, employment sector, it is at or near that 90% penetration benchmark. So that is also a huge, huge deal for Google. And I think distribution actually, rather than just pure competition is one of the things that keeps Google on top.

Yeah, just to add, they do have the power of default, right? I mean, on Android, they're the default search. On iOS, their biggest competitor, they're the default search because of that.

huge uh deal that they had that they've had for years with apple so yeah that's that's something yeah that's something that's going to be a hurdle to overcome right yeah i mean it seems like the market believes in google its stock price is double what it was when chat gpt first came out um we mentioned a bunch of their competitors uh in terms of searching for things already one of them

Gaja is one you were writing about. Anthropic launching a web search API for Clawed, which enables, you say, enables real-time online research with cited sources. You say this marks a major shift away from traditional keyword-based search.

to AI driven discovery per venture beat. Explain to us what you mean by this and why you think that Anthropix Clawed API might reshape search. I want to pretext this by saying that this is still a subscription service that's available to Clawed subscribers.

And to access the API, you know, they have to pay $10 per 1,000 searches. So this is not a consumer sort of product. I see this as an enterprise play. This intermediates Google from search, though, right? Because you can now basically take this API, build it within your internal parameters, tie it into your business's internal data, and...

and search various sources that you could even choose what to cover and what to avoid. So it essentially gives the power of search to the developers, right? And that, again, doesn't require SEO. It doesn't require keyword algorithms. Yeah. Sorry, go on. Finish. Sorry. So it's completely kind of,

devoid from those trappings of traditional search as we know it. So speaking about traditional search, when me and you were talking, I spoke to you last week to talk a bit about this before the episode. And we were talking about this idea that, um,

that websites, you know, the places people land to look for things might start to change. And I read this piece from Patrick Coffey in the Journal who was saying websites will soon serve primarily as data sources for AI platforms, not destinations for humans. This was a prediction from MailChimp. What do you talk to the listeners a bit about that concept, that our relationship with websites changing? Yeah, I mean, if this search API is as good as Anthropic is saying that it is,

It can crawl websites without necessarily needing to click through anything. So that would be in conflict with advertising models. So basically, the information will be accessible. Not only that, it's an AI agent, so it could actually validate the information as it's crawling. There are all these multiple functions that can happen in real time.

It's not just searching. It's giving you the best results based on what your requirements are. Yeah. And being a real-time tool, it could give you the latest news or a stock update on a specific company. What does that mean for the content providers, right? They need to now find...

other viable means to monetize their content, right? Yeah. So, I mean, circling back to Google for a second, Evelyn, one of my questions with the AI chatbot and how it's disrupting searches is how...

much this will affect the advertising dollars. As Corey Johnson, chief market strategist for Epistrophe Capital Research was saying, Google is getting more efficient at answering questions, but less efficient at generating clicks. And clicks is how they get paid. So what's your thought on how advertisers and publishers to a large extent fit into all of this? Yeah, it's a good question. I mean, for advertisers, I think, you know, the biggest

question mark right now is where are the ad opportunities if consumers are going to be to have their online experiences mediated to a large extent by these Gen AI search tools, by chat bots, then they and they cannot be reached as reliably on publishers web pages then

There need to be ad opportunities on those mediated platforms as well. And we're starting to get there. You know, AI overviews already have ads. Advertisers don't have a ton of control over what those formats look like or when they're triggered. It's sort of just part and parcel of Google search campaigns.

And there's not a lot of visibility into performance either because AIO ads are not broken out separately right now in campaign reporting. So it's kind of a black box of an opportunity here. And also, I mean, there are some other Gen AI search interfaces where there are ad opportunities. Copilot, Microsoft's Copilot has some. Amazon's Rufus has some. Perplexity has sponsored some.

like follow-up questions. So not necessarily a typical ad format there. Um, but there's not a lot of, of influence that brands have over what those look like and what those interactions will be, nor does it make sense for them to write because every conversation will be unique that they're the ad. If there's going to be an ad in the, in that flow, it needs to be

suited to whatever the conversation was on top of, you know, we already have personalization of ads to whatever we know about the individual that is browsing the web or whatever it is. So there are a lot of question marks right now for advertisers. In the immediate future, though, it doesn't, it's not that it doesn't mean much. It's just that there's not a ton that advertisers can do right now to change the way they approach search other than

trying to pay attention to what's happening from the platforms and pay attention to consumers because consumer behavior is not going to keep pace with the way that platforms are approaching this opportunity, right? We have AI mode rolling out. It's not something that consumers are asking for. It's something Google is pushing on consumers, at least for the majority of consumers. So it's definitely worth paying attention to. And then on the publisher side, of course, this is an existential threat to their business models, for better or worse.

Ads have funded the internet as we have known it. So I think in the short term, you know, publishers are already starting to feel that

significant revenue implications from the way things are going. And in the long term, I think this is going to be a problem for everybody, for advertisers, for publishers, for the platforms, for consumers, where if publishers aren't able to sustain themselves through this monetization upheaval that's happening, there will be fewer reputable publishers to fuel the models. The answers like

the answer quality, the result quality that these chatbots are able to put together based on the information that's out there on the internet, that quality will degrade. Consumers will get mad because they'll notice. And then there will be less monetizable interactions for the platforms. It's all... I mean, the...

You can see where the system, like the cracks are in this system. And I think a lot of these platforms, advertisers, everybody are sort of, they're focused on simultaneously the future prospects of what could happen in these kinds of conversations, the exciting ways, the exciting new touch points that could open up for advertisers to talk to their prospective customers and their just general customer base.

But then there's also this future where AI could destabilize the way that the ad industry works.

like the foundation on which the ad industry sits and could, I mean, yeah, there are opportunities in the future, but if the way the internet works breaks and consumers just can't find the information they're looking for, then they're not going to be there to be served an ad in the first place. So there's a lot of existential threats, I think, for the entire ecosystem, not just for publishers as well. Yeah, I think existential is the right word there because...

Well, according to Reuters, 45% of digital publishers reported a decline in referral traffic from search just in Q1 2025. That's palpable. So that is really something I don't think anyone was prepared to face at large. But here we are. I want to talk a bit about that. Just a quick note to conclude on the...

ads for the AI mode. Robbie Stein, Google's VP of products for search, was saying that they're considering ways to include ads within AI mode results. So stay tuned for that. But I do want to close the episode out by talking about

publishers and it's kind of shifting power within the ecosystem. There was a great piece by Matteo Wong who writes for The Atlantic saying the AI search war has begun and tech companies might not be the winners. Interestingly, he explains that initially many Gen AI products were effectively built on stolen information before a wave of content licensing deals came out or came about, I should say, right around

the time the New York Times sued for open AI, sorry, sued, yeah, open AI for copy infringement. AI sued,

The search company Perplexity has admitted that they need publishers to keep creating great journalism. Loaded up with facts because you can't answer questions or train models well if you don't have high quality, accurate, human-made source material, he says. So do publishers actually have more power than it seems? Mr. Wong notes that publishers do have some ability to limit AI search engines reading the websites,

but they can also refuse to sign or renegotiate deals. So Gajo, what's your take on Mr. Wong's article noting that tech companies might not be the winners of the AI search war after all? Yeah, I mean, they're facing a very limited number of sources for training data. And what happens there is, yeah, they can borrow or copy or steal this information, but

We've seen how content companies and publishers have pushed back. They're gatekeeping their data. They're blocking crawlers. And in some case, they're going into litigation to protect their intellectual property. That just goes to show you that

It is really a valuable and I think finite resource given that the whole media industry is undergoing tremendous flux. A lot of publications and outlets are closing down or scaling down. So AI is going to continue to demand quality data because otherwise it's just going to start copying and kind of just giving pale versions of reality.

results, right? And if you train your AI on that, it'll become evident pretty quickly that the quality is going to suffer. So I do agree with that point. Yeah, Evelyn, it's interesting this relationship, this evolving relationship between the AI search

platforms companies and publishers because i'm curious to know what this dynamic is going to look like i was thinking this through and wondering whether it's going to look more like the deals made between amazon and coles where coles like will accept amazon returns um or whether it's going to be more like kind of nfl and tv network deals and what i mean by that is coles needs amazon

it seems. So wondering whether like the New York Times will need open AI or as Nicholas Thompson, CEO of Atlantic recently posited, he was saying the Atlantic's contract

with open AI will expire after two years and is designed to create more leverage when there's another moment of negotiation. So will it be kind of Kohl's Amazon where one really needs the other like that? Or will it be kind of like the NFL where every single time the contract, the TV rights are up for renewal, they're in a much more powerful position to demand more money from the TV and streaming players? Yeah, I mean, I think it's a good question because historically publishers have needed Google.

And I'm going to focus on Google here because they're still, by and large, the single most important sort of portal to the internet that consumers use and the single most important

of referral traffic for publishers at this stage. If we have to say that there is a single most important one. But historically, publishers have needed to allow Google's crawler to index their websites in order to be surfaced effectively to the consumers they're trying to reach. And Google has always held all the power. I do think that...

There will be a point of reckoning where Google will realize that it needs publishers to provide the product that it says it wants to provide to consumers. However, I think at this stage, Google has not been making the actions of Google necessary.

make it pretty clear that Google still views itself as in the position of power over publishers. And so I don't, in the very near term, I don't see Google making a ton of concessions to publishers. You know, there will be

some content licensing deals here and there. And perhaps when those deals expire, there will be publishers may still have plenty of leverage. Um, but they could also not right. Google could decide that, um,

you know, that having that content licensing relationship, if they are not forced to license content by regulators in the future, they may decide that it didn't really actually provide any benefit to Google directly in terms of revenue. And therefore it doesn't... So this New York Times OpenAI case will be pivotal then? Yes, huge. I think if New York Times wins its copyright litigation against OpenAI,

open AI, this whole dynamic could kind of crack wide open. And not a lot of publishers have the resources that the New York Times has in order to fight those kinds of cases in court. So that's huge for the rest of the publishing industry as well. If there's a ruling that stands in their favor, of course, then there would have to be. I still think that the tech platforms are

are going to take the W in the short term. Because for as long as we're in this sort of limbo where publishers are still at a disadvantage and there's not regulation, there's sort of a regulatory void where there's not a ton of powerful, the law can't come down on these tech companies. Every day, they're making a ton of money off of these search engines

products. Google brings in, on an average day, over $200 million just in search ad revenues alone. Yeah. I sent with this from the piece from Mr. Wong. He says that the search war is an attempt to change how people navigate the internet, the system through which the contemporary world organizes and disseminates knowledge.

But the underlying terrain has not changed. Knowledge, no matter its organization, remains the sum of writing, art, and thinking from humanity, not from a bot.

That's all true unless people start to value bot-made content more and more. We'll see. Evelyn has just been writing some research on this. It's hot off the press. It's called Gen AI Search Advertising Trends 2025. Consumer adoption lags platform advancements. Link is in the show notes. Or you can head to emarketer.com if you're a pro subscriber. That's all we have time for today. Thank you so much to my guests. Thank you to Gaja.

Thanks, everybody. And thank you to Evelyn. Thanks, Marcus. Thanks, y'all. And thanks to the whole editing crew. And thanks to everyone for listening in to Behind the Numbers, an eMarketer video podcast made possible by Synth. Make sure you subscribe and follow. If you can, leave a rating and review. The mood takes you. Sarah will be here Wednesday taking a quick pulse check of the retail world.