Greylock Presents: The Intelligent Marketer
44:13 Share

Hey, everyone. Welcome to The Intelligent Marketer. I'm Mike DeBow, partner at Greylock. And I'm Rishabh Jain, co-founder and CEO of Fermat. This is a show where we talk about the intersection of AI marketing and commerce, what's changing, what's working, and where things are headed. So we're going to bring in guests and share what we're seeing in the field and try to make sense of how marketers could actually use this stuff.

Last week, we brought in Eric Sufert, a good friend and favorite of Rishabh and myself, to talk about the next iteration of his Everything is an Ad Network thesis. We also covered the future of marketing and ads and LLMs, something that we'll continue to explore with future guests. So thanks for hanging out with us. Let's get started. All right, Eric, thank you for joining us. You have the honor of being the first guest on this new series that Rishabh and I are doing around kind of AI and marketing.

And, you know, for those who don't know Eric Sufert, he's probably the who I believe to be the most insightful analyst, writer, thinker on all things advertising and really kind of marketing related. He writes Mobile Dev Memo, which is one of the best sources of content out there on all things advertising and specifically kind of mobile advertising. But much more than that nowadays.

and invests out of Heracles Capital and is an advisor to many successful companies on their ad strategies as well. We've had Eric on the podcast two years ago to discuss kind of the future of marketing in the post-privacy era, the post-ATT era. And I felt like a good place to start would be, you know, going back to that. So when we spoke two years ago, we talked about

how everything was changing now that ATT is a thing and we're kind of squarely in this post-privacy area. We discussed a few areas where we thought the ad stack would be transformed. So maybe we could look back at those and see where we were right or wrong and kind of what marketers should be planning for going forward. I think one of the first areas we discussed was measurement. And I think one of the biggest things

challenges that marketers were facing at that time was this direct attribution model that we had all been trained to had kind of broke. And so marketers were scrambling to test incremental lifts and build more probabilistic methods, maybe just jumping right into it, like via measurement or otherwise. I would love for you to just comment on if you think about kind of the era of post-privacy marketing over the last couple of years, what's been most notable and what kind of themes come top of mind to you?

Yeah, well, first of all, thank you for having me. It's an honor to join you, to be the guest on the first episode. Appreciate it. I think that measurement was the thing that changed most meaningfully as a result of ATT. And what's really interesting now is we're experiencing what I've called the privacy shuffle, right? So we had this period where the privacy parameters around using consumer data for targeting and for measurement, those parameters kind of became much more restrictive.

And I think there was generally a sense that like we need to prepare for the worst, right? So like ATT was pretty bad, but it could have gotten worse. And my sense was like the kind of prevailing wisdom was that it will continue to get worse. Like this is just going to be like a regular drumbeat of restrictions. And my sense is now that is not the case.

The prevailing wisdom isn't there yet, but I think it will be in six months that actually we've seen the worst. There may be actually even some capitulation, and there has been actually with Google, you know, kind of doing a 180 on cookie deprecation. I think they're probably not going to deprecate the GAID now. So we have seen a little bit of stepping back from the edge of the cliff. But measurement did get overhauled kind of utterly, right? Like fundamentally as a result of ATT. And I think what people are realizing is

is that, yes, that was catalyzed by AT&T and other privacy restrictions, but it would have been a good idea even absent AT&T and absent those privacy restrictions, and everyone's kind of glad that they did it. And that actually stays with us. Even if we do see kind of a privacy shuffle and there's less of a priority on privacy and that we're probably at the end of any sort of big, radical platform privacy changes,

Everyone realizes that, yes, making the change to probabilistic measurement was beneficial and it was productive, and they probably should have done it sooner than they did.

Yeah, so I think this is an interesting thread to stand. And I think rather than kind of us geeking out on how to execute lift testing the best or building MMMs, I think what's interesting is what are the second order effects? Like when you actually have a good sense of lift and kind of look at measurement probabilistically versus just kind of the attributing all credit to some last touch or some direct attribution model, how do marketers actually make changes accordingly? Who are some of the winners and losers of that game, Eric? And maybe how do you see that continuing?

I think everyone's a winner. I think everyone's a winner. So this is not zero-sum in any way, right? It's better measurement. And because it's better measurement, advertisers get to invest more. And if they get to invest more, they probably get to invest more everywhere, right? Like why did Facebook put so much money into Robin? Why did Google put so much money into Meridian? They ostensibly, they would be the benefactors of last-click attribution, right? But why were they so invested in getting these tools into people's hands? Because they know that if everyone's deploying money more wisely, they benefit. Even if it's not, even if they don't benefit the most, they still do,

Right. And so like, I think there are no losers if measurement gets better broadly. I mean, other than like kind of fraudy sources of traffic. But I think like the legitimate sources of traffic that provide incremental value, like they will win. They will win from this. And it's just gradations of winning. I wrote this article a while back called Flying Blind. Right. And it was actually the title of a keynote that I was giving for a while. And the idea here is like,

you know, I had this idea in the keynote of like Wittgenstein's ruler for ads, right? So like that's a concept from Nassim Taleb invoking Wittgenstein. But he was saying, if you're using a ruler to measure a table, you're also using the table to measure the ruler. And I think what a lot of people were getting wrong about

about deterministic attribution was like, well, there's always just sort of like a little bit more sophistication I can add to it and it'll be pseudo deterministic. I can always get there. There's just a little bit more voodoo I can throw into the mix and I'll be able to get to pseudo deterministic. And like what that tells me about them is they have no idea what they're doing.

Right. That's Wittgenstein's ruler. If you tell me like, no, we came up with this very convoluted, you know, complicated system for achieving pseudo deterministic measurement. It doesn't tell me that they're really good at measurement. It tells me they have no idea what they're doing. Right. It tells me more about them than it does about their measurement apparatus. And so I think like when you let go and you you sort of appreciate all the capabilities that are opened up to you, if you fly blind, you've got podcast advertising, you've got TV advertising, you've got CTV advertising, probably the most explosive, exciting growth, vertical and digital ads right now.

You've got out of home. You've got everything can be measured if you shift away from deterministic.

Where do you think we are on that shift? Out of curiosity, what are you seeing in terms of the change management expectations that people are seeing? Which segments are adapting faster versus slower? I'm pretty sure not everybody's there. No. But it is clear that we moved really fast. And so I'm wondering where you think we are on that adoption cycle. What is the set of things that need to happen for us to get further up that adoption cycle or accelerate that adoption cycle?

We're still, I think, in the early adopter phase, but I think what ends up kind of dragging everyone along is just the platform tools getting better. And by that, I mean, Advantage Plus and PMAX. Totally. Like you get Advantage Plus and PMAX. You need this kind of marketing measurement for just Advantage Plus. Like when you're throwing money into Advantage Plus, you're not just buying newsfeed, you're buying everything.

The metrics are obfuscated, right? You need this kind of measurement just for Advantage Plus, even if you're only working on meta. Only working on meta's properties, you need probabilistic measurement for Advantage Plus, right? And so I think what ends up happening is A, like the platform provided tools get better, but B, it's everyone recognizes, look, at 50% of my budget,

Or 80% of my budget on meta, which is maybe 40% of my budget overall for e-commerce hire, is going into Advantage Plus. I need this kind of measurement. I just can't rely on last click. It just won't work. But I think we're still in the early adopter phase. The other thing is like, look, the point I've been making for a long time, everything is an ad network, right? Well, that's retail media. That's CTV. You can't measure that with a term. You just can't. It's impossible.

So I think this transitions into one other trend around kind of post-ATT, which is leveraging your own zero part of your first party data, you know, to drive better targeting in a world where you actually can use, where there are less kind of signals that you can actually target upon. What are you observing there? So obviously retail media is one thing, but, you know, what are some of the smartest teams doing on that front right now? And maybe what, again, maybe jump into second order effects, like which new networks kind of most benefit from this?

At the frontier is actually doing signal engineering. It's building the best possible, the highest density value signals. It's making sure that what you get back to the platform is so valuable

that it sort of like disproportionately drives campaign optimization. The thing is like, I think people look at a Cappy and it's like, it's a machine gun, right? It's like you're just shooting everything over. Would you rather be sending 50 signals per user or would you rather send one really good high density, high value signal? And I think you'd rather send the high value signal. And then if, cause it's also like a function of the quality of the inputs, right?

impacts the quality of the outputs, right? So like if I'm sending like valid, you know, sort of like highly credible, meaning like the sort of like predictive power of that signal towards making a conversion as high. And so if I'm able to provide better inputs to the platform, they're able to do better predictions.

So like it just sort of, it sort of impacts the benefit compounds throughout the sort of like the model. So that would be like at the very cutting edge. And then I think sort of like slightly back from like the very bleeding edge of the frontier is just implementing CAPI, right? And make sure that you're able to do that, right? And so like that's actually more,

challenging than it sort of seems. Like just implementing a CAPI is no big deal, but implementing multiple CAPIs is fraught, right? Because I mean, you have actual like data leakage concerns, right? And so then you're talking about maybe working with like a CDP or some sort of like intermediary, which takes work, right? And it's engineering work. So it's like, you know, real work. It's not marketing work. Right?

But like, so that is, that's more like organizational challenge. But, you know, nonetheless, it's very valuable. And then I think, you know, the other piece of this, like that I'm really excited about when I think about like the applications of AI, and I wrote about this today on LinkedIn, it was like a shower thought kind of thing. But like, if you think about the applications that are the most exciting, it's actually like the down funnel stuff.

It's actually doing like personalized economy stuff. And that is a user acquisition function, right? That is a marketing function. Like if I get the monetization right, if I get the economy planning right, all of that flows through to user acquisition. Say more on that. What is personalization economy? How can you translate that?

So like the canonical hypothetical would be just a perfectly personalized price point or a bundle, right? And like, what's really exciting now is that's more possible today than it was a month ago with Apple being found to be in contempt of this injunction order. So let's say I can push someone to my website, right?

And I have a very good idea of like their willingness to pay, their propensity to pay and the value that they get out of like multiple. Let me use a game just as an example, because games are like the sort of easiest toy examples because everything is made up, right? Everything in a game economy is made up. There's no marginal cost of production for a virtual in-app good. I can make anything I want be an in-app good. It's just the best toy example. So let's say that I've got a very good idea that a user has a high propensity to pay and they get a lot of value out of the sort of virtual in-app goods.

I get them to click out to the website. I have total sort of agency over my website. I don't need to register bundles. I don't need to manage price points. I have a total freedom to make anything I want and offer it up as a product. So now I just offer this person a $50 bundle of a sword, a shield, 50 gems, some gold coins, and eight lives, right? Now the next person comes around, it's just a sword.

The next person comes around and it's 10 coins, but they're heavily discounted. I can personalize not just the price point, but I can personalize the product at the price point that I think has the highest probability of converting. There is a lot of power in that. And that is possible as a result of A, Apple having to give up the commission that's charged on the off-platform payments. But B, just this application of AI, right? That's all just optimization. Like if I, you know, that's a neural network power in that.

Right. What is the optimal thing to show this person to get the highest potential possible conversion rate? That is super exciting. Yeah, it's a very important point that of all the things that marketers have been trained to test and optimize over the years, creative messaging, content, website formatting, et cetera, like pricing and bundling has kind of been one of the last frontiers. And as investors, we're seeing more interesting

innovation here as well. And it's cool to hear you comment on that. And one more point is, what was one of the first things that Spotify did when Apple was forced to adhere to the rules of the injunction? They offered bundles. They offered, you can now buy audiobooks. You can buy above and beyond the sort of amount of free time that you get with the subscription. You can buy audiobooks, add-ons. Exactly. That kind of bundling is so economically powerful. No, I was actually literally going to say the same thing, is that actually it turns out that

bundle offer merchandising is actually the hardest thing is actually the thing that actually drives the highest ROI in most marketing funnels. It's just ridiculously painful.

to implement well. I agree with Eric completely, actually, is that AI for the first time might make it not ridiculously painful. Because if you were to talk to a merchandiser, whether it's for virtual goods or physical goods or whatever it may be, and tell them to do this, they would just start laughing. It would just be like a ridiculous proposition to ask someone to do that. And then in the physical good world, it then has the added complexity of you then have to tell your 3PL to do that. So yeah, I think...

You chose the perfect example with all virtual goods with this Apple injunction. Yeah, I'll be curious to see like who starts building those systems because somebody is going to and it's going to be in like the next year, basically.

Yeah, I think, well, okay, but let me toss out my hypothesis here. But I've been so knee deep in this stuff. This has kind of been dominating my kind of thinking for the last like week. But there's a paper, Bundling and Competition on the Internet. It's Bacos and Bjorn Jolfsson. So Bjorn Jolfsson is like one of the thought leaders in like digital products pricing. So like pricing for goods that have zero marginal cost of production. He and Hal Varian, like they kind of established that, a lot of that thinking there. But okay, so now Rishabh.

Humor me. What if Advantage Plus had landing page optimization? What if I had some kind of, there was some kind of API, right? Now they were serving the ad. They got the click. They registered the click.

And then they were optimizing my landing page on my behalf. Like that's feasible from a technical perspective. That's not infeasible. That's for sure going to happen. So not only is that going to happen, the inverse is going to happen too, which is like people are going to observe the shopping pattern and then generate the creative. Right. So the system has to talk to each other in both directions. For sure, that's going to happen.

There's only two questions, like everything in platform life, which is who builds it, how does it get built, and how do the signals go back and forth with the platform and the actual checkout? That's it. Those are the only questions. Because ultimately, when the transaction happens of a certain type of bundle, the platform has to be able to read that information. Today, I can tell you that Meta does not know how to read it. So...

What we're doing is we're actually saying, hey, platform, you need to learn how to take signals that are not identity-based signals. Like we are now talking about SKU signals. And I've had many conversations with the Facebook API team. The only thing they know how to consume is identity-based information. And that's a huge miss, basically. And so they're going to have to be able to get more effective at consuming new signals. But that is for sure going to happen.

Or they need to acquire FirmApp. Now Mike's going to have to walk into his partner meeting and say, like, guys, we have risk of this company that's going to get acquired. Sorry, Mike. Speaking of signals and different kind of signals upon which marketers could target on, I think we should shift gears and talk about marketing on the LLMs. And this has been a very hot topic both in Ventureland and with brands right now. I think very...

Very quickly, LLMs are where the, when I say LLMs, kind of like chat-based apps launched by the larger model providers are kind of where the customer discovery journey begins. And I think we should talk about the implications and kind of opportunities that fall from this. And so I want to talk about a few different themes here, but maybe just to start, a lot of brands are now scrambling to figure out how they optimize their presence for LLMs.

I know, Eric, you're more of a paid guy and this is kind of broaching into like SEO kind of land. But I'm curious for your view on this and kind of what what you're observing there on brands trying to go and influence these results. Maybe just talk about that for a minute. I feel like Eric is about to go onto the mountaintop and scream, everything is an ad network. But I'll let. Well, we're going to get to that. We're going to get to that.

I think these are inevitably ad networks. But before that, in today's world, before these platforms actually have perplexity as an ad offering, but like early right now, kind of before then, like what are you seeing here?

Yeah, you stole my thunder, Bishop. It's not going to have as much impact now. It's an interesting idea. I mean, the feedback loop is pretty slow there, right? I mean, if you're talking about pushing out content that will ultimately be trained on by an LLM, I mean, the kind of turnaround time on that is probably like months. Weirdly, I get a lot of, well, not a lot, but I get a decent amount of traffic from ChatGPT, even though I've got ChatGPT excluded in my robots.txt file.

Someone actually sent me a screenshot. They were talking to ChatGPT and they were asking about my view on something. And ChatGPT was referencing my articles behind the paywall. And so the person asked, are you a subscriber? And then ChatGPT, it was just the most weaselly response. Well, I'm not actually a subscriber, but I can infer what his opinions might be from the headlines or something like that. It's like...

Okay, they clearly found a way around the paywall or something. But I think it's an interesting approach. I mean, I think where I think there's probably the most value right now is just trying to find a way to get your content surfaced in AI overviews in Google search.

Like that's just kind of the existing approach of like SEO and just adjusting it for this new paradigm of Google search. I mean, getting a result in ChatGPT is great, but I think AI overviews just drives a lot of traffic. And we know that because we know that publishers are screaming about all the traffic they're losing.

Right. And so the companies that have sort of cozied up to Google and they've done data licensing agreements are getting that traffic to the beneficiaries of that. Right. And so I think generally, if you're talking about the open web, you're probably talking about. I mean, I hate to say this because people get upset. Right. But it's it's a dying paradigm. Right. You probably have to just do whatever you have to do to persist. Right. And what that is now is just making sure that you get surface in AI overviews.

Yeah. So maybe let's get to the fun topic then on everything is an ad network. And you've been right on that many times over the years. In the last week, OpenAI hired Fiji, who hailed from Meta and that Instacart and kind of built out in-house ad products there. I don't want to speculate on that, but it feels, you know, likely that over time they will work on something. And this is kind of just the inevitable, like once you've aggregated that kind of impression volume, like it's the inevitable path for these networks. And I think

the semantic context that you're sitting on is just such a rich kind of data mine for advertisers to go work on. It does feel inevitable for all of these. And so maybe how do you think these platforms are going to navigate launching their ad engines? And for marketers who want to stay ahead of that, how should they be thinking about it right now?

I think they would be remiss if, first of all, they'd be remiss if they didn't. And it is inevitable. Like there's no chance that OpenAI doesn't have a scaled ad platform in four years, right? The hiring of Fiji was just like a little bit too on the nose, even. Like you can't, I mean, I was going to write about that and I'm like, why? It's just so obvious. It's like saying the sky is blue.

Right. I mean, of course, that's what they hired her to do. That's not interesting. There's no insight there. But I think they would also be remiss in trying to like trying to engineer something that they feel like represents a new paradigm when the reality is advertising has existed for thousands of years and it's best when done in a legacy manner.

Right. And by that, I mean, you've got display, you've got it tailored to whatever conversion data you have on that person and you're able to measure outcomes. And that doesn't require anything fancy. That doesn't require, first of all, I don't, I think it's a horrible mistake to even raise the perception that you might be curating the content to

to serve an ad and not to serve the best possible answer, right? That will kill engagement, right? That just poisons the well. That will make people think that they're getting advertised to and they're not getting the best possible information. The best possible way to implement this is an ad alongside the best content. An ad alongside the content that most fulfills that user's needs, that will perform the best.

Yeah, totally. So there was this one thing that came up. I was at like a brand dinner three weeks ago, just bridging a little bit from our SEO conversation to this ads conversation. Mike, you might find this interesting too. But the question was asked, like, hey, how are people trying to like make sure their content is showing up in the AI overviews and things like that. And I want to say seven of 10, roughly 7%, 70% of the room literally said, we'll just wait for the ads product to get released.

I think brands, when confronted with this question of do we want to play the SEO game in AI search, they're just like, the total investment that I have to put into the SEO game relative to an ads network, I would rather wait for the ads product to come out and then invest the capital into the ads network versus into SEO. I think that that's actually a hard question for marketers, which is,

Like marketers get graded more on what they can actually show has an impact to the return than in something that is harder to show the impact. Right. And so I wonder how you think about that as it relates to where would you choose to invest right now? Would you choose to invest against trying to show up in SEO? Would you say like, hey, let's just do our standard SEO. That way we know what we're investing into and then just wait for the ad product because you know it's inevitable. Right.

Like, how would you decide how to navigate that in the organization? I mean, I'm kind of like a performance absolutist, right? So like, I've never really been a fan of like SEO or ASO, which is like the sort of apps equivalent because you don't control it, right? You're at the you're sort of like subject to the whims of any sort of algorithm change. I mean, your fortunes can change overnight as a result of that. I'd rather just be in total control, right? And so for me, it's that's advertising.

You know, with SEO, you're exactly right. I mean, like you can show, you can kind of use difference in difference techniques or you can do sort of like lift studies or geo holdouts or whatever to show the effects of like SEO, but they're like sort of like there's a long time horizon that you have to kind of consider. And how do you measure the cost side? I guess it's just the cost of the salaries. It's just, to me, it's just, I'd rather have a dial that I can turn to grow my

my revenue essentially with paid ads and just have a model that, you know, just sort of measures that with as much precision as possible. Right. So like my sense is like, that's probably the right approach. Also, you just don't know. I mean, like chat GPT might have some sort of change in policy around ingesting content, right. They may have a change in policy around a citing the source of content, right. Like AI overviews in the same way, like,

And it's also the I overuse. I mean, you're talking about data licensing deals that like, how do you supersede, right? And the most commercialized queries are probably going to be the most valuable to sort of use a data licensing agreement as leverage to inject your content into, right? So like all of these things are not in your control. I'd rather just build an end to end sort of growth engine oriented around ads.

Eric, I want to go back to your point on the format. And I very much agree that the biggest mistake these engines can make is just going and compromising user trust by influencing results in such a way that are, I guess, like unnatural or overly paid.

I feel like we're kind of waiting for our Instagram ad moment here where people kind of realized like, hey, these ads are actually enhancing my experience when they're surfacing products that are very relevant to me. The ads are reasonably engaging as long as you keep ad load kind of within reasonable range. I would argue that it actually enhanced the user experience.

A lot of the thinking that I have seen to date here just assumes that these are going to be basic text-based search ads that are going to look very similar to kind of what we've all seen in Google over the years. I kind of feel like there's a new native unit to be created that we haven't yet seen yet. And I thought that would be interesting to get your take on and kind of hear what ideas you have. Like if you were to be in charge of ads at, you know, one of these large players and you were tasked with creating the right unit,

That would be a new AI native unit. Like, what would that look like to you? Here's my freebie to you, OpenAI. I'm going to give this to you. You can have it. So there's a couple, right? So there's one is be full screen, non-skippable interstitial for every X queries on the free tier. So you do five queries. You got to watch a full screen, non-skippable interstitial. That's the YouTube app, right? Same. It exists, right? That's not new. The second could be just real time display insertions, right? So these would be similar to like Google Shopping ads.

You just have those in between blocks of texts as they render. The other is I could just imagine a tower unit, right? A tower unit on the side of the results, right? Like why not? And they could be totally divorced. So the thing is, the less related those things are, the more trust you have in the text, right? Because think about it. Like if I'm seeing an ad for, you know, I'm

I'm going to Hawaii tomorrow, Hawaiian Airlines. And the text is about what is Eric Super talking about when he says Wittgenstein's ruler? Then I don't doubt those results are sort of delivered to me independent of the sort of behaviorally targeted ad. But the other thing here is I think if you kind of think out like a couple of years from now, like I don't think that the primary interface is going to be tech.

You can imagine you actually are talking to an avatar who's persistent and consistent, or maybe you have a cast of characters, and this is all video. It's all delivered to you like a YouTube video, but it's all rendered in real time. Now, if you have that, then you just have a standard YouTube interstitial. You have a standard YouTube cutaway. Again, that doesn't dilute the perception of the content. This is a breakaway ad. I'm going to pause the content to show you an ad. That's how ads work best because you need to get people

the concentration on the ad. Rishabh, what do you think? I think that that's exactly right. Mostly because also the behaviors for a lot of these chat interfaces where you're going back and forth is a lot of questions that are unrelated to commercial intent. And so you actually want to design for a system that assumes that a lot of the engagement has no commercial intent. So it's like if I'm doing a bunch of research about

hey, what are the most interesting topics to talk about for AI and ads for an upcoming podcast? It's like, what are you going to show me, an ad for a mic? You know what I mean? That's not the best way to monetize me. The best way to monetize me is a bunch of other commercial information that you have about me next to that. And so also because of the usage pattern, trying to have it be inside of the content is actually just not the best monetization play. Purely economically, it would be the not best monetization play.

Yeah. I mean, I kind of agree with Eric. I think on the screen, you'll have like a tower on the side and then on the phone. I don't know exactly what it looks like, but the takeover is like by far the best idea that I've heard so far.

Yeah. So what if we go down the tail and there's a couple of companies we've come across that are trying to build essentially a new two-sided ad network between AI apps and advertisers. And I think if you remove kind of the big handful of players who are going to be building their own kind of in-house rails.

There's presumably a much longer tail of companies that are never going to be able to charge premium subscriptions that will have to be ad supported from day one that are going to be perhaps chat based to start with, but haven't yet figured out the right unit. Like if you were to go be designing an SSP for them,

What would that look like? Forget SSP. If you were to go design kind of in-house ads and like the rails for them and ideally a two-sided network for them, what would that look like? Do you believe such a network should exist and that it's not owned by one of the existing kind of players?

Yeah. I mean, the thing is, subscription is a suboptimal way to monetize pretty much anything, especially something with as much engagement as a chatbot, right? Like, okay, so Meta stopped reporting ARPU by Jio, but the last time they reported it was 70 bucks in the US. That's a quarter, right? So like, what's Chachi BT charging for the subscription, right? I mean, there's far more ARPU in ads if you can target them really well, like Meta can, right? So my sense is like, just

everybody should be looking into ads. It reduces consumer surplus, right? Like it produces a lot of consumer surplus, but it generates the most money. Actually, I should say it produces optimal unit economics is what I meant. And so like, I think everybody should be looking into ads if they can generate that kind of engagement at that kind of DAU, right? In terms of, you know, building like a sort of intermediary

I don't think it needs to look too radically different from what we know today. I mean, you need a DSP and SSP in an exchange, right? The companies that can build this stuff themselves and build like self-serve and, you know, have like an ads manager, they're probably going to get the best economics and they're probably going to, you know, sort of build the best and most like functional systems. But I mean, if you just build like the programmatic infrastructure

infrastructure for that. I think it functions pretty well. And I don't know that you need a whole lot more than what you have today, unless the formats are just radically different, right? But again, I just don't think it is a good idea to do like a sales pitch in an LLM result. So like I would advocate against that anyway. Does that mean that Trade Desk and Applovin and other networks start to serve that publisher space or does a new company get built?

I don't see why fundamentally you need a new company. I think, yeah, Trade Desk could probably intermediate that pretty well or like an app loving for different context, you know, the sort of content specificity. But I don't know that it gets any, like, especially if you're talking about a tower banner, that's Trade Desk could do that right now. There's nothing you need if that's serving on a website. And if it's serving in an ad, I mean, there's plenty of video ad networks that could do the full screen interstitial takeover.

Totally. I guess the only question would be like, do they actually have the system to take in the data to run the algorithm based on the content of the conversation that you're having in the actual publisher? Right? Like, that's actually the only delta is like, today, the signals that those algorithms take in are like the problem with today's ad network is like 100% of them are based on identity based data signals. And then they have some sort of taxonomy against that. But very few are based on conversational signals. That ultimately is sort of what

At least like when Mike and I have talked about it separately, it's like, I don't know whether the current systems can actually support that data signal.

Well, okay, so there's a couple things, right? Like, one is, if you consider that to be just, like, a contextual signal, right? Like, okay, so that's a piece of context. There's ways, just semantic analysis, that you could do that, right? Like, there's ways that you could just parse the conversation and send some key, right, that maps to, like, some kind of sentiment or whatever, a topic. But the thing is, like, you know, like, Applovin, for instance, I mean, I think they're principally serving, especially in games, they're principally serving against context. It's principally serving against the taxonomy of the games and the game subcategories, right? So I think...

They have ways of doing that. But my point, I guess, is like, I don't think it's a good idea. I don't think it's necessarily the best idea to take the topic or the subject or the general tone or whatever of the conversation and use that to target ads. Totally. But you don't think there is a format of that data that you would want to store? No.

That's, I guess, the question is, do these systems have the ability, just like you're saying, hey, taking a data piece associated with the person, now all of a sudden that data piece is like much richer because it's actually a behavior interaction back and forth that you use at a different time or from a different place. Do you use that type of data or do you not use that type of data?

Not only can you not use that kind of data, you wouldn't want to. You wouldn't want to give it away. You see these experiments that people post on X or whatever. They're like, hey, go to OpenAI and say, based upon what you know about me, show me a picture. And for me, it's the guy. Exactly. The result for that is the same for everybody. It's just someone looking at eight screens. But like,

There was a really interesting one and it was like, based upon what you know about me, what are the five top flaws that I could fix in the next month that would improve my output? And it actually was really insightful. And the thing is, OpenAI knows a lot about me.

I mean, I bet you it could map what it knows to exactly the kind of demographic features that people advertise against. It probably knows my age. It probably could guess my net worth or whatever. It might guess my disposable income. It could guess that I have kids. It knows all those things. And so knowing that, then what would you want to do? Would you want to pass that along to some intermediary? No. First of all, you probably can't. You wouldn't be able to package that up in a decipherable way and pass it along. But even if you could, you wouldn't want to. You'd want to keep that. That's gold.

That is worth its weight in gold, that information. And so you would build your own advertising system on top of that. Now, it's the companies that don't get that kind of engagement, that can't harness that kind of data, that probably would then need to rely on like a third party system. Yeah, I think part of your point that I'm hearing is actually you could derive identity based signals from this conversational data, too. And so maybe actually feeding it into the existing DSPs is not as problematic as kind of we thought.

I guess last topic on this, and I want to jump over and talk about agencies for a bit. So Shopify, you saw their integration with ChatGPT a couple of weeks back now. It makes sense that if we are all kind of starting our commerce discovery journey on these engines, it makes sense to be able to complete it there too. What do you think are the next couple of steps there? Like if you play that out, there's probably a conversation to be had on like, what's the role of the website in general? And so this is both for Rishabh and Eric. I feel like both you guys will have interesting thoughts on this.

How are you thinking about this? I think that generally speaking, I'll just allude back to this dinner again. It was like actually a very insightful dinner from a couple of weeks ago. They said that traffic that comes from one of these answer engines is higher converting than any other form of traffic.

The sort of obvious insight from that is that at a minimum, the total amount of research that you can do and therefore the total amount that you know before you get redirected is now so much higher that the role of the site has fundamentally changed to one of a transaction versus one of information. And so it kind of doesn't matter whether the transaction actually happens inside of ChatGPT or not to some extent, because ultimately if the person is redirected

already made up their mind on what they want to purchase inside of the answer engine then that is actually the most important thing so that's like at least my two cents is i think how we think about whatever are these other properties outside of the actual answer engine they need to serve what is the new reality which is people have a ton of information based on their interactions inside of the answer engine and so that's my principal point of view i'm curious what eric thinks by

By the way, if that's the case, do you think Shopify still deserves to own the cart? I'm going to let Eric answer so I don't get into too much trouble. Yeah, no, I agree with Rishabh. I think it's like by the time the users clicked on the link, they've been sold. So whether the transaction happens right there or it happens, you know, eight seconds later because they had to click through, like they've been sold on that thing. I don't know that like fundamentally alters the sort of like calculus for like the website or not. I think, and look, I mean, I just hate to be negative about this, but like it,

I do just think that this sort of open web paradigm has probably already come to an end, right? And you're kind of, what you're seeing now is just like everybody that couldn't walled Gardenify themselves. Like look at the New York Times. It's a perfect example, right? I mean, like if you look at their revenue breakdown, like a significant proportion of their revenue comes from games now. And it's all based on the subscription, right? They did a perfect job of transitioning away. And then when the revenue does come from ads, it's from their own proprietary ad tech, right?

They built a really scaled subscription business with like a number of different value propositions. Everything that's left on the open web is just the stuff that got picked over and couldn't do that. It couldn't make that transition because they don't offer enough value to actually get people to even maybe even just sign up. Right. And to get past even a free paywall or like an authentication gate. Right. Then what's the value of that?

It's like the dollars have shifted over to the high value properties and whatever's left is really just kind of like a, it's just bargain hunting, right? It's like trying to find gems. And so I think AI just accelerates a lot of that because you're not going to want to have your content just freely accessible in that way because it'll get ingested and not credited. And so you're going to see, I think, a lot more people just erect these, even if it is just a free authentication gate, but probably a lot more people shifting into paid models because it worked really well.

And particularly if you're like a niche subject, like advertising is never really going to be that scalable for you. So you'd rather just do, you know, the Substack thing. My sense is like, you know, we're sort of like past the open web era already. What exists is sort of in a state of like perpetual decline. And like, I think one of the things that people are sort of like mistakenly celebrating is this idea that like,

disrupting the sort of like Google monopoly around its ad tech business is actually going to hasten the demise. No, it's going to accelerate it. If Google has no skin in the game, they're going to shift all their resources into AI overviews. They have no reason to support the open web as such, right? That to me is just,

It's not a cause for celebration. It's actually like there was probably one stable piece of the foundation holding the open web up, and it was the ability to monetize it through Google's properties. And if they lose interest in that, like I just see no path forward.

Yeah. Mike, I mean, what do you tell your friends who are still like leading growth at retailers or at brands? Would you recommend investing in the website? Yeah, I mean, it's part of what makes growth fun is that like the set of challenges changes every few months. And I think growth people tend to be very good at designing an experiment roadmap to kind of validate the next thing. But I think it could sometimes be maybe short to medium term oriented as a result. And so frankly, I think

the stream of optimizing website has often hit diminishing returns by now anyway, that I don't know that people are thinking long-term enough and like, "Hey, is this thing just going to go away altogether?"

I think there's probably more excitement on what's to come than there is like worry on kind of what's starting to degrade. And I think more of my growth friends are spending more time thinking about the exact topics we're talking about today. So how do I go and increase discoverability in these new channels where all the impressions are coming from? How do I go take advantage of the fact that there is some alpha back in being a savvy media buyer now that maybe it wasn't a handful of years ago, but I'm having fun doing just like meta paid marketing still.

How do I go and kind of think about MCP? And we might be a little bit early to this, but in a world where we have all these different agents that are transacting with my brand, making myself transactable to them. So I think those are more of the conversations right now. And I'm not skirting the question, Rishabh. I just hear I hear the website come up like less. And that's kind of the reality of it.

Totally, totally. I'm going to just like use one of the things you said to touch on this last topic, which I'm really curious because Eric is, if nothing, the best documentation of meta in history. So there's this thing that Mark is now starting to say over and over again, which is like, you tell us your goal and we'll just do everything. And we already do it better than most humans, right? And so there's this question of like, do we just end up in a world of agents or, and

And what does the role of the agency become when that is actually the way that the platforms behave and especially meta, right? And so I think it started three earnings calls ago. If I'm getting, Eric can probably correct me if I'm getting this wrong. It started three earnings calls ago where he said for the first time, you're just going to tell us your goal. And then

And recently, I feel like he's been on like multiple podcasts where he's repeating the same message over and over again. And so what does that mean in terms of what is the actual role of the agency in that world where you can just tell Meta what your goal is? Or and or what is the role of the media buyer where you can tell Meta what you want? And that's basically it.

Yeah. So I think like the role of the media buyers is already changing fundamentally. I mean, like that already is kind of the modus operandi, right? If you're, you know, overseeing scale budget on meta, you're trying to lean in to the greatest possible extent to ASC if you're e-com or Advantage Plus for apps, if you're selling app installs or buying app installs. And so like, I think that has already kind of fundamentally changed. And I think like

That's better. It's objectively better. Like people complain about it. No, that's exactly what everyone wants, right? I don't think people, people don't like acknowledge the sort of like profundity of that shift in strategy, right? Like that was brilliant and it was perfectly timed. It was a result in large part of ATT. So it was kind of forced, but I mean, it demonstrably worked really well. I mean, look at the stock price, look at the revenue, right? But the other thing is like, look at the outcome. It's facilitating commerce to a sort of like a larger magnitude than it was before, right? This is what people want.

So it's not like, oh, you're killing jobs. No, this is what everyone has been clamoring for. And so what does the role of the media buyer become? Well, it becomes managing that, right? But also it becomes thinking through like strategically of how all these things work in concert because UA or whatever you want to call it, advertising, media buying, that doesn't exist in isolation from the product, from how the user engages with the product. And

And like the sort of like the tighter those two things are when you consider that entire like end to end user journey, like the better it's all going to perform. Right. Like is all flows through. These are not you can't analyze these things in isolation. You can't build for them in isolation. You can't optimize them in isolation. These are one in the same. And so I think, you know, that media buyer becomes more of a product analyst. They become more of an economist.

They become more of someone who thinks about like, how can we reduce that consumer surplus? Right. And so that is the real value out of a lot of this stuff. It's not cost reduction. It's not, oh, I get 10 creatives now for, you know, 50 bucks instead of one. It's not, oh, you know, my, I'm creating a lot of assets for free. No, it's, I'm actually improving outcomes for my consumers, for my customers. And they're spending more money as a result because they're getting more value out of the product. And then I can spend more money on user acquisition. Like all of these things fit into a cycle. And

And sort of like the more unified that is, the better it is for everyone. It's not zero song.

Maybe I'll approach this topic from a different angle, which is that of the agency. And I think, you know, we're all seeing all these marketing workflows from creative generation to media buying and measurement. Like they're all becoming agentized where sophisticated agents could handle 90% of the work that mid junior level marketers used to do. Does this replace the role of agencies? Does it help agencies operate in a much better margin? Like, how do you think about the role of agencies in this world? Yeah.

It's interesting, right? So like I invested in a company called Gigi and like they're building like the kind of AI control center, the AI nerve center for agencies. And I was kind of like, well, you know, the sort of like the thing you hear about agencies is like they have a lot of really young people that are working like very long hours for not very much money.

And so like how, how much you saving by shifting that over to like this kind of AI system. And the point was like, there's so much domain knowledge just sort of trapped inside agencies that can be unlocked within, within AI platform. And then sort of like brought to bear on, on clients that like, yeah, there's some cost savings there, but there's actually just like more value delivery.

delivery. Right. And so like, okay, that actually makes a lot of sense because if you think about like a lot of what you're paying for as an agency, I think a lot of it's just like FaceTime that's not really value add. I always come back to this example, right? So like I was giving a talk at this event and it was like a brand event. And like, I don't normally give talks at those. It's like kind of different

crowd, you know, different circle. And so I was talking to some people and I was like, so, you know, is it true? Like I hear all these stories all the time that you can brought to like the super bowl or you're getting brought to the Beyonce concert or you get the box at the, at the basketball game. And they're like, yeah,

That's why we love our job. That's why we love doing what we do. It's because like we basically get, you know, once a month or maybe twice a month, we're invited to some event and we get to sit in the box. We get to sit in the floor seats. And then someone was like, you know, at my agency, we throw a big Christmas party every year and it's sponsored by the SaaS tool providers. Like the Christmas party is paid for by these SaaS tool providers. And like they compete with each other to try to be like the main sponsor.

But one year, like there was one company that they really wanted it, but like they just they were too late or something. And so they said, look, we'll throw you a bone. You can sponsor the brunch that will serve in the office the next day.

And I was like, so you do business with these people because they sponsor your brunch. You're like hungover post Christmas party. But like, that's a reason to buy their product. And like, yeah, because like they got tacos, they had mimosas. And I was just like, it was like that scene in the big short where Steve Carell's character Mark Baum was like, why are they confessing? And then his, you know, his junior was like, I'm not confessing. They're bragging. Oh, my God.

That's the funniest thing. Speaking of which, Furmach has launched their ad. Yeah, I was going to say, you don't like tacos enough, Eric. We got to get you some nice tacos. That's one of the funniest stories I've heard. It's especially funny because I feel like in today's world of cutting through the noise, you know, this whole like, what is the only, what is the way to get attention? It's like, it's one of the few ways to get attention is in person, especially in SaaS. So.

Eric, I know we're up against time here. This has been a great conversation. As always, we got through like half of what we wanted to, but maybe we could, you know, sponsor your holiday party next time and get you to come back for another one. Thank you so much for doing this. We will chat again soon. Yeah. Nice to see you both. Thank you so much for having me.