Bucky Moore: The Next Decade of AI Infrastructure
48:17 Share

Hey, everyone. Welcome to Generative Now. I am Michael Mignano, a partner at Lightspeed. I'm really excited about this week's conversation because I got to sit down with someone I've recently had the privilege of working alongside here at Lightspeed, and that is Bucky Moore.

Bucky is one of the sharpest thinkers in enterprise software and AI. From foundational AI to infrastructure to cybersecurity, he's got the gift for spotting spiky founders early and helping them scale big ideas into world-changing companies. We'll dig into what's on his radar, how his investing philosophy has evolved, and what he thinks the next decade of AI innovation will look like. Let's get into it.

Hey, Bucky. Hey, Mike. How's it going? Good. Thanks for doing this. Anytime. Anything for my partner. Yeah, it's been great to have you at Lightspeed, obviously. Well, how long has it been? Is it a month, two? What is it? About a month and a half. So first day was last week of April in the office the following week. And yeah, time flies. Nice.

I knew of you before you joined, but and so I was very excited to get to know you directly and not just know of you from the Internet. I'm not I'm not like an infrastructure investor per se. Like I tend to do more consumer, some enterprise, et cetera. But you from afar, like even when you're a client or like I always thought of you as like.

the infra guy, like one of the, one of the best like infra VCs. And it's just such a, it's a world that like is so foreign to me. And so I don't know, like, I'd love to hear like sort of how you got into it and sort of how you thought about sort of traditional infrastructure investing in a pre AI world. I started my career in venture about 11 years ago. And before I got into venture, I was pulled into a role at Cisco on their corporate development team.

where the job was to basically look out for startups that were compelling and relevant to Cisco's future ambitions and

either invest in them, acquire them or partner with them. So that's where I met founders for the first time. And I just think by virtue of that vantage point, I kind of was born a little bit lower down the stack given that was my first foray into the industry. And so I think the time I spent there really just has trained me to look at things from the bottom up in that sense, rather than the top down. And what I found at Cisco is that it's just unbelievable how much impact these companies that are being built in that fashion can have in terms of like their surface area across every industry and companies of different sizes. And I

I think I got enamored by that. And so combination of the experience and kind of clicking for me is just how important these companies were. I've kind of dedicated my career to it. So I'd say that's one thing. And then I think the other thing is I'd say it really has been reinforced just watching some of these mega public companies get built that are effectively infrastructure companies. That if you look at the NASDAQ and the top 10 most valuable companies by their trading multiples, we're talking about Datadog. We're talking about Snowflake. We're talking about CrowdStrike. These are infrastructure companies. And so they...

they take longer to build. They have tons of technical risks that you have to mitigate early, which is capital intensive and uncertain. But I think once you do and you really figure out a seam in the market where that technology can apply uniquely,

These companies can compound over many decades in ways that I think like generic software companies have proven harder to do if all but the most exceptional cases like ServiceNow and Salesforce. That's what I think is interesting about infrastructure is it's just so fundamental to every business and every industry on earth, especially as more and more businesses are moving online and digitizing that just the scale of these opportunities is massive. Did you have this point of view when you got to Cisco? And like, I guess what led you there and maybe what were some of the

the deals that you were at least, you know, doing from an M&A side or investing. Like when I think of Cisco, this might be really dumb and overly simple, but like, I think of like routers and switches and things like that. Like,

When you get to Cisco and you show up and it's early in your career, did you already, did you already know the types of infrastructure companies and deals you'd be working on? Let's talk about how I got to Cisco first and then I'll answer that question. So I actually started my undergraduate, like I graduated from college. I went and actually worked as an analyst at a private equity firm and I absolutely hated it for a couple of reasons. Like one was private equity is really a game of value extraction. And I was more keen on being a part of value creation, which is sort of what drifted my mind, kind of looking at technology and seeing this as where all the value is being created. So yeah,

So when I decided I had to get out of that industry and kind of move into the Bay Area to be a part of this technology industry we now all live in, that was sort of like what prompted my search that led to landing at Cisco. And the way it actually happened was I sort of networked my way into someone who was like in a completely unrelated role to corporate development, who told me that they were working on an integration for an acquisition that they had just done with the corporate development team. And so they connected me with that team. And I immediately realized this was

where I could kind of apply my experience, but also like get kind of a crash course in just the world that we live in today. So I went in very naive. I didn't know the difference between a router and a switch. I knew very little about Cisco's business other than they did a bunch of IT infrastructure stuff. And out the other side, I met product managers, sales engineers,

I learned the difference between a router and a switch. And then to kind of answer your question, that is very much where Cisco started. But those industries have commoditized pretty rapidly. And so what they've been doing for the past couple of decades is really trying to like move more and more into software businesses. And so you see acquisitions like Splunk, you see acquisitions like Meraki, they're kind of forays into that.

So I'd say the story of Cisco today is they have this very large, slow growing legacy kind of hardware and systems business. And then they have a moderately to high growing suite of software businesses that kind of are the future of the company. So.

So when it comes to kind of infrastructure in that sense, I think this was a really, really interesting experience for me where I got, I kind of got to see old world, new world kind of colliding at the same time at Cisco and Cisco kind of living with this legacy set of, of incentives and assumptions, and also trying to kind of find their way into this new world of software. And boy, do I remember some crazy conversations being at, I mean, we used to debate whether SAS was going to be a thing or it was going to be virtual desktops. Like we used to debate whether cloud infrastructure was for tests and dev and anything more than that. And, and,

what i've learned from all those experiences is like ludism is inescapable for a company like that number one and number two it just pays to run towards the new whether you're an investor or not just run towards the new thing and the future happens fast usually faster than any of us as humans can comprehend and i think if you run towards the new you're just on that right side of history so that was like a really key takeaway for me at cisco and

In terms of one of the first projects I worked on on that CorpDev team, it was a project codenamed North Shore. North Shore stood for this company called Nicira Networks, which was basically the first company that was taking a lot of the networking IP that Cisco had and kind of moving it entirely into software and going from needing switches with custom ASICs to using literally like x86 Intel servers to do a lot of the work. The founder of that company was a guy called Martin Casado, who is now a partner at Andreessen Horowitz and a good friend and peer.

And it was a very, very interesting first deal to be involved in because he never had any interest in selling the company to Cisco. And in fact, Cisco had this view that they could really just buy the company and essentially put it in a quarter and let it die.

He was well aware of that and had no interest in that future. And so what happened was we, let's just say we didn't pay the highest price. VMware did. And he built an incredible business that is now this product called VMware NSX inside of VMware that has really changed the networking industry in a bunch of different ways and really sustained VMware as a business as well. And so Martin's an amazing guy and a great investor too. But, you know, that was a really, I'd say, formative experience for me in seeing just kind of that Luddism on display in a very strategically consequential moment.

and how it's just so inescapable. And ultimately, had we run towards the new as fast as we could have, instead of getting cute, I think there, let's just say the networking industry would have played out very differently over the past couple of decades. Right. And so you, it sounds like you're, you're sort of,

implying that Cisco, at least for a period of time, and this is common, right? And while all incumbents, this is the innovator's dilemma, sort of like clinging to the old in which they obviously had like a very, very dominant position, sort of refusing to accept that like their hardware would actually become digitized and software-based. And these incentives run deep, right? You have sales reps who are making money selling those products. You have-

Leaders of those businesses whose empires are built on those products being the most important thing and commanding the most budget. So I think inside of every large company you look, you see these incentives and it may not be as clean as like a hardware software divide like it is in this case, but you see that. And I think that's why we're all very lucky that we get to spend time with the startups who have a chance to kind of

move a lot faster and not be encumbered by these incentives the way big companies are. And so one of the things I think that this time at Cisco also really taught me was that infrastructure tends to kind of move in lockstep with technology cycles, right? And so when we go back to having personal computing, like that was obviously something that fundamentally changed the way infrastructure was built in the sense that you had all these computers that you needed to network together so people get their work

done. Then we had the internet and that obviously changed the way the computers could be connected from like local area networks to like an entirely, you know, globally distributed mesh. And then we have things like mobile and cloud, right? Most recently, and most recently AI that have just effectively changed the course of how infrastructure gets built. So typically what you see in infrastructure is that a new pattern comes along. That new pattern is often enabled by some new technical innovation. That new pattern unlocks new workloads and those new workloads need new infrastructure.

And so I think that's typically the way that we approach this is we kind of go from that top-down view of like, what's happening in the world that's changing? What are the new workloads that emerge from that? And then what are the new infrastructure requirements of those workloads? And what you find is that, at least if you look back in history, like every...

every decade or so there is a, there is a new change and a new, a new sea change that drives that new workload. And those new workloads have new infrastructure. And I think right now it goes without saying what that is in the form of AI. But I think if you look back five or 10 years ago, like what we were talking about was how do we, how do we make the world cloud native, so to speak. Right. And so I think that's the really interesting thing about infrastructure for me as an investor is that as the world changes, so does it.

And therefore, you're kind of having to reinvent yourself and your mindset as an investor and certainly entrepreneur every decade or so, right? Like I remember meeting so many founders who were kind of coming out of the era where the way you built big infrastructure companies was you built something that was used inside of like an enterprise data center. Those folks had to kind of reinvent their mindsets, reinvent their

their, their technical priors and really kind of like take a current view of the world that was like cloud native in shape. And I think the same thing now is happening with all these folks who have cloud and distributed systems expertise. I mean, they're, they're out there building a lot of these important infrastructure companies in the era of AI, but they've really had to evolve themselves. And so I think that's a really interesting kind of aspect, certainly as an investor or a builder is that you do things change really fast. And, and, uh, and when things change, you have to change with it. So this theme of like, you know, old meets the new run towards the new, I, I,

I feel like, you know, my guess is that that's going to be a repeated sort of theme in this conversation. You know, Cisco, you leave Cisco. You went right to Kleiner from Cisco. Is that right? No. So I actually started my career adventure at Battery Ventures. Oh, right. OK. Yes. Right. And I ended up at Kleiner Perkins in January of 2018. So at Battery and maybe the earlier days of Kleiner, you know, we've sort of moved from the old of switches and routers into the new of kind of cloud solutions.

while you're sort of getting started in venture, like talk to us about that time. Like what are the types of deals and the companies being built and how does that sort of inspire and sort of like teach you the business of venture? I think when I was at Cisco, the main pocket of innovation was around essentially allowing enterprises that had data centers to modernize those data centers with more modern storage networking and compute primitives. And so Cisco was obviously on the networking side of that.

Companies like NetApp and Nimble Storage, many of whom were actually light speed companies, including Nimble Storage, that were sort of reinventing the storage side of it. And then on the compute side of it, you obviously had like sort of a commodity hardware set in the form of Intel's x86. And then you had kind of all this software from VMware that was sort of used on top of it to make sure that all that stuff worked together.

So that was kind of the world that we lived in. And the interesting time about when I was at Cisco, which was 2011 to 2014, was like the world was starting to change really rapidly and that those companies were a lot less relevant than before because what was happening is that Amazon Web Services had come out and Amazon Web Services had not just convinced the world that it was a place where you could try and try building new things, but it was a place where you should actually move all your applications. And the reason that it was a place where you should move all your applications in the eyes of enterprises was they were able to essentially have this level of agility and flexibility that they never had before.

Those same drivers of agility and flexibility, which was the idea of just being able to spin up compute and infrastructure resources on demand, were new primitives that actually allowed for entirely new infrastructure companies to be built. So like a good example of this would be Snowflake, right? So Snowflake is effectively going after Oracle, who dominated the data warehousing business in tandem with Teradata for decades. But when the cloud came around, there was just an entirely new architecture that you could pursue. And Snowflake, I think, really illustrates what that looks like in the case of analytical databases.

And I think what happened was like at the, at least at the time, I just never seen a move in the market so fast towards the new products. Right. I mean, this was a market that was so deeply entrenched by, by Oracle. I don't think anyone prior had really thought of it as something that you could disrupt, but Snowflake effectively brought a product to market that was like 10 X cheaper, 10 X faster. And, and with that, um, the entire market moved in its direction. Right. So I think one of the lessons that, that seeing kind of that on-premise to cloud transition taught me was just, again, this comment of like the future happens fast. Right. And I think, um,

What you start to see is that like every subsequent wave, it seems like the future happens even faster. And I think we saw that between like, say, you know, the on-premise era and the cloud era. I think we saw that in terms of like the PC or the mobile era. And I think we're actually seeing it now to a large degree in terms of the cloud to the AI era, meaning people are just moving faster towards the benefits of these technologies because I think in each subsequent wave, it kind of stacks like layers on a cake on top of the others.

And just unlocks like economic transformation that the world has not seen before. And I think Snowflake did a good job of that in the context of data warehousing. And we're actually seeing that happen in a lot of cases with these frontier models that everyone's building around it and certainly the infrastructure that surrounds them. What were some of the other primitives in sort of the pre-AI space?

of infrastructure. Cloud, obviously, we've talked about a little bit here. Like, what were some of the other primitives? Well, I mean, here's the thing is we talked about Snowflake as an analytical database, but there's, you know, the other side of databases is transactional databases. So this is like the market, again, that was once dominated by Oracle. Okay. And now what you see is that companies like AWS and Google as hyperscalers have basically delivered managed database services that have now really started to dominate the market, right? Yeah.

So that flavor of database, I think, is a really interesting one to look at right now because in a sense, you can kind of, as an infrastructure investor, you can kind of think of LLMs as databases in the sense that you query them and you get information out. In the old world, you'd query a relational database and you'd have to be really thoughtful about what data's in there, how it's laid out, how fast you can get it out, and the types of queries that you write

And so I think one like really, really interesting kind of paradigm shift that I'm kind of feeling as an investor right now is like, if we think about these LLMs as databases in the same way, the databases are ubiquitous in software and they come in all different shapes and sizes. I think we're seeing basically a new way to retrieve information. Right. And while they're a bit more stochastic by comparison, I think you, uh,

you start to really think about like, okay, so like if these things are going to be as ubiquitous as databases and developers are going to be building with them in all kinds of new ways, it's not hard to see how the flexibility they have and just how much more robust they are by comparison because you don't even have to put data in them necessarily. Just how like, why this is such a massive opportunity, right? And so that's kind of like one thing I find is you start to see these things kind of rhyme in terms of at least how developers think about them. And I think as an infrastructure investor, it is very informative to ask yourself like, okay, so like what's in the developer's toolbox at any given point in time?

And I think for a long time, you had Postgres, you had things like Kafka, which is Confluence business. You have things like ClickHouse nowadays, where obviously Lightspeed is an investor. And I think what's changing is LLMs are now in the toolbox. And that's a really, really exciting place to be as a piece of technology and a company providing that technology. And one could argue that the game and infrastructure is really like finding your way into the developer's toolbox at that level. Okay, so let's talk about AI. So I guess during your time at Klum,

is probably when AI really took hold, you know, the chat GPT moment. We talked about LLMs and we're going to definitely talk more about that. I want to talk about like this stochastic comment you mentioned, but what are some of the other now AI first primitives that you've seen come out of this wave that you maybe,

could not have anticipated or maybe don't rhyme necessarily with some of the ones we saw in sort of the pre-AI era. Before I hit that head on, I would acknowledge that I think the, I talked about how in infrastructure, you really do have to transition as an investor really quickly into these new paradigms and make sure that you're steeped in them and really understand and accept their potential. I would say doing that within the context of AI was probably like one of the most challenging endeavors that I've gone through as an investor. And the reason for that is like sort of twofold.

One is I think things just happen so much faster than I've ever seen before, meaning like it just felt like a switch was flipped and suddenly like this was all the developers were spending time thinking about. And two, they seem to carry these really, really big philosophical questions, especially early in the lifecycle of this technology, although we could still say we're early, but let's say very early, like back in early 2023, let's say after ChatGPT had become kind of a household name and people understood where this was going.

there were these big questions around like, will there be one model to rule them all? Or will there be many small models? Will, will companies train their own models or will they use frontier models? Will the models be open or closed? There's so many big questions like that. And I think given the rate of change of the industry, like the answers to those questions were like extremely opaque for a pretty long time. And one could argue like the answers are still quite, quite undetermined. I would say though, like that was a really challenging dynamic for me to get ahold of as an investor, because candidly, like in the 10 or so years I've been doing this at the time, I was, uh,

I was pretty comfortable with like a base set of assumptions where I could go do my research, I could form conclusions, and then I could act on my investment interests within the bounds of those conclusions. It was really hard to get to that place for me with AI. So I feel like I was actually quite slow to lean in and probably missed some great opportunities as a result. But yeah,

I just wanted to acknowledge that because I look back on it and think of like, what could I have done differently? And I thought it was a very interesting, but hard challenge. So I would acknowledge that. And then what I would say in terms of primitives today is like, yeah, it's very clear now, like the market is speaking loudly in the form of like, there is some fundamental infrastructure you need to be an AI native company. And so let's talk about that, right? Like, I think at the core level, you have these models and the question of like, whether you're consuming these models in the form of like the proprietary solutions from the Xs or open AIs or anthropics of the world.

Or you're consuming open source models, right? That's question one. And without getting into kind of the reason for why companies arrive at the conclusion of open or closed, what you start to see is that as companies are moving more to closed open models, which you're seeing pretty broadly for cost, performance, and flexibility reasons. They're moving towards open. I think that if you look at the inference calls that the average AI native app is making today, you're starting to see a percentage of those calls go towards open source models.

It's usually lower stakes stuff where cost matters, where convenience matters, where control matters. But I think that's happening and it's pretty hard to ignore, at least from where I sit. And so I think what's interesting about the move towards more open models is you start to see that there's all this infrastructure you kind of need to own to run those models. Right. And so that's where these businesses that are building inference platforms are starting to really explode. And you're also starting to see a lot of a lot of work around the data infrastructure side of things where it's like if I have open models.

That means I have access to their weights. That means I can fine tune them in ways I can't with the closed models. So there's some really interesting work happening around just like helping customers connect their personal data, their proprietary data to these open models so that they can fine tune and post train with that, right? So that's, I think, a big area. It's just like the sort of compute and bringing data to the open model side of things that I've been excited about. So that's one. Before we move to the next one, can I ask you a question about that one? So moving from closed to open, I guess that, based on what you're saying, what I'm hearing is like, it's gonna create a lot of,

investable opportunities in terms of the infra needed to sort of support these open source models. Whereas on the closed side, like an open AI or an anthropic, like they're going to roll all that stuff themselves or in partnership, right, with a Microsoft or an Amazon or something. And the developer is not going to have to worry about that. Now they're going to pay a premium for it.

but they're not going to have to worry about it. And so moving to open source though, now, all of a sudden you've got these components you need to think about as developer. And therefore it's, it's an opportunity from an investment standpoint. Is that, is that what you mean? I think that's right. And what I would say though, on the, on the cost point is that I wouldn't actually say it's like evident yet that cost is the only reason you'd move away from closed models, because as we saw recently, uh, the price of opening eyes for, Oh, just dropped dramatically. Right. And, uh,

So given token prices are kind of decreasing by like multiple orders of magnitude every year, I think you will be able to use very, very cheap and performant closed models. So then the question is like, why would you need open? And at least what I'm seeing out there is that, yes, there are use cases where it can be cheaper to use an open model, but sometimes you want to use a smaller model because you get better latency properties. Or sometimes you simply don't want the data to leave the environment that you're building in and you want to have full control over the model in a way that only open source can give you. So

So I think those are some of the bigger drivers that I see. But again, this field is changing so fast. What's really exciting is that I think the open source models are becoming a lot more performant.

And as they become more performant, I think the argument for reaping the benefits of open source as developers have in databases and middleware and all these other categories becomes a lot clearer and clean. So I think it's pretty obvious that open source models are here to stay. And I think it's even more obvious that the rate at which they're being used versus proprietary models is ticking upward. But I think it's still really early. You know, another thing that I've been thinking about a lot the past couple of days, not weeks, not months.

is, you know, in Apple's announcement at WWDC the other day, they revealed that now they're going to have a platform in iOS whereby, you know, developers can take advantage of on-device models that Apple provides, totally free inference on-device. How does that throw a wrench

Or how will, you know, solutions like that throw a wrench into the dynamic you just laid out? Yeah, so I think the on-device stuff is super interesting. And at Lightspeed, we have a company called Cartesia that's doing some really amazing work there. The way I'd put it is that today there are still,

a lot of use cases where the quality of an on-device model that you can serve is just not good enough in terms of its like performance and the level of intelligence that you can deliver to your users to make it worth skipping the call back to some bigger model that's on some server in someone's cloud, right? So for example, in CodeGen, you're still seeing a lot of use of like large frontier models rather than local models for that reason, right? Yeah.

But on the other hand, if you think about like, say, you know, every mobile device in the world, whether it's an Apple device or a Samsung device or what have you, having some kind of like AI assistant that has like really robust audio and speech capabilities that you can kind of communicate with in that way. I can think of all kinds of reasons why in the near term, we're going to see on-device model servos use cases where latency is really, really key and performance is good enough with the scale and quality of model you can put on device models.

So I think it's an exciting paradigm. I think it's going to likely be its own kind of like, I don't want to say niche, but let's just say like, you know, kind of discrete market. And I think that the frontier labs could care less about that market right now, as far as I can tell, because their whole thing is about building these bigger models that can deliver a maximum amount of intelligence and kind of deliver on those frontier like properties. That's a little bit diametrically opposed to like, how do we shrink this thing down and kind of constrain it to the environment that say a phone or a laptop represents?

So my point is to say there that I think it's a really interesting pocket of white space for companies to actually go and play in an area where I don't anticipate the frontier labs being as aggressive. That could change. But as far as like how they seem to have, how it seems to align with their commercial objectives, it's a lot less clear to me that the frontier labs are going to want to play that game because they're just so focused on pushing the state of the art and scale and performance. Yeah, that makes sense. Okay. So we have open source, closed source models and local models.

I think you were going to get to another primitive that I cut you off of. Yeah, I mean, I think there's this really exciting new scaling paradigm in the form of test time compute and reinforcement learning that is emerging. And I think what you're starting to see when you go and talk to folks at the Frontier Labs is they'll tell you that they're quote unquote infrastructure constrained.

And they don't mean that they don't have enough GPUs when they say that. What they mean is that there is a set of infrastructure that one has to build, maintain, and scale out in order to do reinforcement learning at really large scale on these tasks like programming or math or perhaps even more qualitative tasks that have less clear reward models to them that you have to build environments to actually simulate so that you can do the reinforcement learning on. And so an example of this would be, suppose you wanted to train an agent that does a

kind of security. So you want to train an agent to behave like a hacker, right? Well, in order to train that agent to understand how to hack, you have to show it what it means to hack and you have to help it find vulnerabilities and exploit those vulnerabilities. And to do that, you have to spin up a bunch of servers and you have to, you know, put vulnerabilities on those servers and you have to create an environment that represents sort of the real world use case that you hope this agent can thrive in.

And so the ball of wax there in terms of creating these arbitrary environments where agents can actually learn in a practical and scalable way is very hard. And so one area that we're starting to see a lot of innovation in is, well, what does it mean to actually create these environments? And what does it mean to maintain these environments? And what does it mean to ensure that the results you're getting from the agent playing and learning in these environments are good results?

And so I think this whole area of effectively simulation environments for reinforcement learning is very fertile ground in an area that I think we're excited about as a firm and certainly is an area that a lot of really smart people in the industry are spending time building it. So I'd say that's a big area where I think if you believe that RL is sort of the next scaling paradigm, say post pre-training and post test time compute.

what you're going to start to see very quickly is that the infrastructure around doing RL and making it easy and easily accessible for enterprises is going to be really, really critical. So that's an area that I think is really exciting. And while we're on the topic of agents, I think it goes without saying, if we just look at even this most recent Y Combinator batch, the number of companies that are effectively making a bet that there will be agents all around us and those agents will need infrastructure and tools and all kinds of primitives to do their job in the way that we do as humans is pretty clear. And so, you know, everything from like

What browser does the agent use when it needs to use the web? How does it connect to tools on your computer and the cloud? How do you bring data from third-party applications into the agent? These are all, I think, really needy problems that I suspect we'll see companies get built in and the winners be very successful and valuable as companies in. So that's another area that I'm excited about is just this world where agents are doing more and more of the work, not just how do you teach them to do the work, as I was talking about earlier, but then how do you give them the tools and the infrastructure they need to actually carry it out?

Right. So that's another really exciting area that I think a lot about. And I think you're seeing a lot of investment dollars flow into and a lot of good entrepreneurial activity. What would you say some of the less obvious infra opportunities around agents are right now? Everyone's talking about MCP, right?

and, you know, connect connecting models to, you know, infinite number of, you know, sources of data or like what are some of the less obvious things that excite you in terms of agents? I think until it became clear to me how important this was, like these building these simulation environments was certainly like a non-obvious one that seems very, very fundamental. So that's one. And I think if you're inside of one of the big frontier labs, it's a very obvious one. But outside of it, you just have a little bit less exposure to the nitty gritty of like how you actually go about solving these problems. So that was one.

Another one that I would point to is, I think what you're going to start to see is that these vertical software companies that effectively are AI native in shape, as well as those that weren't necessarily born in an AI native era, but are kind of bringing AI into their product,

A lot of what they have to do is interact with these really like old, often on-premise pieces of software for their customers. And so if they want to build agentic flows, the question of how you connect those agents to like an on-premise EHR in a healthcare sense, or maybe a logistics management software that runs in a server in an office of like a trucking warehouse. Like these are actually really important problems. If you think about enabling these agents to make their full impact in more esoteric industries where automation is super, super high value and

So I've also been really interested in that, which is like, there's just all these interaction points that have existed for a really long time that were not built to accommodate agents. How do you enable them to do that in a way that kind of aligns with like, say, if you're, you know, a company like Samsara, they're customers who are all trucking companies.

as an example, right? And so I think there's kind of this long tail of industries and businesses that have very, very old technology that they want to build automations over. And I think how you teach agents to actually engage with that technology is a really important and interesting problem. Yeah, I guess like if you're, if you're anthropic,

it's very obvious that you add a, you add a button to connect with the Google drive or the email or the calendar, but it's less obvious for how the trucking data gets into, gets into the model. Right. Yeah. And I think if you're anthropic, you're focused on the, you know, sort of the most popular applications, right? Like the Salesforce is in the Slack and we should talk about Slack too, given the recent news about the data sharing stuff. I think that's, that's really interesting. Right. And so for those listening to this, that haven't seen the news, um,

And Salesforce very recently said they're going to start to be a little bit more restrictive with how companies can access the data inside of Slack. And the reason they did this was in the context of how people are using Slack data to build their AI systems, it's very, very valuable data. And Salesforce has ambitions to build some of that stuff themselves. And so they've decided that they want to give themselves an inherent advantage in terms of being able to wield that data in ways that vendors who might be building competitive products to what they have on their roadmap cannot. Right.

I think this is really interesting because it seems like there's a precedent that's going to be set by this. And I think it could go one of two ways. Like the first is that customers, you know, speak up and say, Hey, this is, this isn't cool. Like I, I want to make sure I have the ability to bring the best technology to bear on top of this data. And if I can't do that, that's a big problem for me. And then the other would be, and this is like sort of a Rorschach test for, for Salesforce's leverage with their customers. Like Salesforce says no, and this is how it's going to be. And, and,

I worry that if the latter plays out, which I'll caveat, I don't think is very likely because I think in these situations, what's best for the customer tends to prevail, which is a good thing. But if it does play out, you can imagine Atlassian, you can imagine ServiceNow and other core systems of record to very quickly start to say, okay, we're going to do that too.

And what would happen, I think, if that really played out that way, which I don't want to happen, is it would just be a very, very challenging environment for a lot of these AI application companies that rely on that data that their customers store in those systems to innovate the way that they want to. And so I'm watching this very closely. I'm curious what you think of it.

Yeah, I find it fascinating. I mean, I've been following this trend on more of like the consumer side for a while. Obviously, you know, lots of big publishers aren't thrilled that their their content is being, you know, scraped via rag. And there are services out there that are helping them monetize it. And obviously, we've seen things like Reddit, you know, saying, you know, locking their data and doing these one off deals. It feels like this slack.

back Salesforce thing is like the first big shoe to drop more on the enterprise side. And it's different because these are businesses and companies where sort of the exchange of data is commonplace via APIs and things like that. And now it's like, nope, actually the walls are coming up. Yeah. Right. And so, yeah, I'm like super, super interested to see how this plays out. It also, for me, what it highlights is

If you're a business that actually doesn't have your own data and you're like just you sort of run off of other people's data, like you're probably in a pretty uncomfortable spot right now because you're starting to wonder if like the data that you rely on might go away tomorrow. So, yeah, I don't know. I've been thinking about it as well nonstop since the news came out. Yeah, the Reddit analogy is interesting because I guess my knee jerk reaction to that is, well, like if I'm a customer of Salesforce, like the data I store in Salesforce is mine. Right. And I think it's contractually important.

I don't know the extent to which that's true, but that's my understanding. Whereas I think if it's Reddit, it's a much more global corpus. It's like user generated, but maybe not owned by the user. Whereas I think the messages in Slack are owned by, say, Lightspeed in our case. And so it'll be really interesting to see just how that works.

place things out. I mean, I don't know Reddit's terms of service, but, you know, I built a large user generated content platform, you know, many, many millions or however many hundreds or billions of hours of audio content. And I can tell you that the, you know, our terms were the users owned it.

You know, our platform, we have the license to distribute it and, you know, to monetize it on behalf of the user. But the user owns typically the user owns user generated content. I don't know what's the cases with like a slack. I mean, on one hand, yeah, it's my I wrote the words, but it's sort of wrapped in a format that is.

own unique proprietary format. So I don't know. It's actually a very, very interesting question. And Slack's just one example, right? Yeah. Yeah. I don't know. I think it's, I think it's fascinating. We're going to, it's, we're,

we're about to see something we're about to see sort of like a new era, I think of the internet and especially, especially like enterprise software, I think. Totally agree. Talk to us about compute, right? Like you did the, you did the together deal when you're a Kleiner. There are a number of companies in the space that have gone after this opportunity. And I'm not just, I'm not talking about like Nvidia. I'm talking like the startups that are, you know, reselling and, you know, provisioning and things like that. Like

Talk to us about that space and how it's evolved over the past two years since ChatGPT came out and where it's going. So AI compute is a very, very interesting space to me on multiple levels. Like the first level that I think is interesting is just how misunderstood it is, right? Like I think there's this common term thrown around by the media to refer to these companies as GPU resellers and resellers.

Well, that is technically true that they take delivery of GPUs that they either own or lease, and then they run software on them inside of data centers and then deliver that as a service to customers. I feel like if I just replaced GPUs with x86 servers, I could have described AWS, right? And so like, what's the fundamental difference between these companies that are known as GPU resellers by folks like the information and AWS?

AWS has had multiple decades now to build higher level APIs and services on top of that core offering that is servers being resold by them. Right. And so I think what's happening here is like these companies are sort of misunderstood and like not being treated like the cloud providers that they effectively are.

Because they are just so early in their life cycle that a lot of the innovation that got AWS to where it is today is very much underway. So that's point one of why I think it's interesting, is I think these companies are misunderstood in that in the limit, the winning companies will look more like AWS or GCP than they will, say, a colo provider, which I think is sort of the pejorative way of putting reseller.

So that's kind of point one. Point two is that I think there's like two really important modalities to talk about on compute, right? The first is training and the second is inference. So let's talk about training first, right? So training is typically skewing towards, I mean, obviously companies that are training models, which tend to be either frontier labs or very, very well-capitalized companies that are in say modalities that are non-overlapping with what the frontier labs are focused on, say like audio, for example, right?

And so in that space, typically customers are shopping for a large number of chips, like on the order of thousands, sometimes many thousands, number one. And number two, because of that, the number one heuristic that they're using to make the decision is like, how do I get the best possible price? The second would probably be, can I ensure that the chips that I'm bringing online with this partner, are they reliable, right? As in when they go down, are they going to come back up? And can I get to the SLAs that I need to ensure that when I run my trading jobs, I can run those trading jobs without excessive failure, right?

So I call that out because I think given it's such a price-driven market, the opportunity to actually differentiate with software today is relatively limited because if you're, say, SSI or you're OpenAI or any one of these other frontier labs that are buying tens of thousands of chips, you really just are trying to think about how to minimize CapEx exposure without necessarily having to compromise on the reliability of your access to those chips. So that's kind of modality one, right? And that's, I think, again...

Probably a little bit more commodity than not today, if you look across the landscape from the hyperscalers to new clouds like a CoreWeave or a Together AI, but still a very fast growing market and where the lion's share of the dollars are. Is there a startup opportunity, though, if it's generally been commoditized at this point? I think the opportunity for startups is as that market becomes more software driven.

I think that's where the opportunity really arises for startups to thrive, right? So if you talk about Together AI, for example, there is still a fairly meaningful layer of software that exists on the training side of things in the form of these custom CUDA kernels. So what I mean by custom CUDA kernels is for an NVIDIA GPU, it has this programming language on top of it that you use to program the GPU called CUDA, C-U-D-A. And when I refer to a kernel, what I mean is writing a custom software path of how the model actually interacts with that CUDA code and the GPU itself.

to make it faster and to get higher utilization out of the GPU because GPUs are very expensive. Getting high utilization out of them is very, very lucrative for anyone using GPUs, right? So a company like Together AI, what they've really done is they've said, hey, we're going to write a lot of these custom kernels, these custom pathways down to the GPU for companies that want to train models.

just like Anthropic and OpenAI have, right? Because a lot of the talent that actually understands how to do that, they're inside of the Frontier Labs. And the promise of Together is we've brought a bunch of those people together from Stanford and other key places and said, hey, we're going to deliver that platform engineering service to everyone.

So that's sort of like the layer of software that exists in training. Whereas by comparison on the inference side, the second workload, one, it's growing very fast, right? Because intuitively more companies are saying, hey, I've trained models or I figured out how to use other models and I want to put them in production and point them at my customers and my software. So that market's growing very, very fast. And what's interesting about it is one, because it's more online, meaning like you're embedding this inference endpoint in your product.

You have kind of more concerns like you would in running any production software, which is like, how does it scale out? How do I ensure that it stays on if it fails? How do I monitor performance? How do I ensure that the accuracy of the model as I'm doing tricks to make the output tokens go faster isn't being compromised? So there are a lot of these kind of really interesting concerns that kind of fall into place where what you start to see is that if you're an open air anthropic, you've got a team of folks that work on the serving team, which is effectively the inference infrastructure team.

that are making sure all this stuff works in a way that delivers a great customer experience. And so the opportunity for these next-gen players on the inference side, by comparison to training, is much meatier in terms of software, I would argue. And so what you're starting to see is that companies like Together AI or Fireworks or Base10 are starting to emerge as leaders in this space and grow very, very fast because they've solved a lot of those operational pain points that exist when you have a custom model or an open source model that you want to serve in your application. And how much do the labs...

want to own of that workflow, the inference opportunity, be it open or closed, right? So like a Mistral or an OpenAI or an Anthropic. If you're in the business of serving customer open source models, you want to own all this, right? Because this is where the value accrues at the infrastructure layer, I would argue, right?

Now, if you are selling a closed model a la OpenAI or Anthropic, for example, you have to solve all these problems yourself because your customers aren't looking for an inference runtime. They're looking for an endpoint that they can call and get answers back, right? They want to query your LLM and they want to get answers back. They don't want to have to worry about auto-scaling. It's your job to make it work and stay on and be performant. And that's sort of what they pay you the big bucks for. And so I think if you're out of front of your lab, you don't want your customers to think about any of this. You want to own it all.

And you want to own it all so that you can deliver the best possible customer experience. It's magic. It's behind the scenes. But if you're not a frontier lab and you're one of these companies that's in the inference business, what you really want is you want to enable other companies to deliver that experience to their customers without having the same inference and serving expertise that OpenAI or Anthropic or DeepMind has. Yeah, makes sense. Makes sense. We talked about LLMs, but let's just talk about models and sort of new architectures in general. You know, obviously the things we're talking about, they're all...

transformer-based. As an infrastructure or as an infra investor, how much do you think about the potential for new architectures? I'm sure you've heard of some of these labs that are taking completely new approaches or attempting to take completely new approaches to the model layer. I mean, is that something you're thinking about, you're looking for, you're focused on? So I think in the research community, the reason why alternative architectures are very interesting is one, the frontier labs are

betting so big on transformers and really mostly innovating around how you scale transformers and how you curate new data into it and more data into it, as well as compute is what I mean by scaling that there is a bit of white space right now where it's like, well, what if one of these alternative architectures really, really turns out to be the way forward? Or what if blending one of these alternative architectures with transformers turns out to be the unlock that the industry hasn't seen before? And

And so I think what you're starting to see is that this is actually very fertile ground for startups. One, because the frontier labs aren't focused on it, like I said. And two, I think you're a bit less compute bound, meaning you don't necessarily have to kind of keep up with the big boys, so to speak, and buy 50,000 or 100,000 H200s or what have you to prove that an alternative architecture actually works in scales. So I think it's exciting in the sense that like it's a bit less CapEx intensive than say going and training frontier transformers in that sense.

But on the other hand, I think it's still very early to say that any one of these alternative architectures is, uh, is likely to reach some level of ubiquity or standardization the way a transformer has. Right. And so an example of this would be like state space models, which a company called Cartesia is very known for, for taking forward. And, uh,

I think what they found is that it unlocks some really, really interesting properties when it comes to use cases where you have long sequence lakes on the input side. So you want to pass the model a lot of context and you care a lot about latency, right? So that's where these states-based models seem to be quite special. So what they've done is they've actually said, hey, like with that in mind, let's verticalize and let's actually build the best audio models and the best platform for developing audio agents and voice agents in the market. And so I think one instantiation of what alternative architectures could lead to is companies that find a way to verticalize in that sense, right?

But then I think the other would be, Hey, maybe, maybe one of these alternative architectures, if not SSMs, um, start to, to pay off in like a nonlinear way. And it makes sense to actually go and raise a bunch of money to scale them to the level that we're seeing LLMs being pushed to today. And I think that is, uh, an unanswered, but very interesting question as an infrastructure investor that you're thinking about. And, uh,

The other thing I'd say is you're actually starting to see techniques like diffusion, which has become very popular on the image and video side, be applied to LLMs as well. So we're starting to see kind of cross-pollination of these techniques across different modalities in ways that I think, you know, frankly, just lead to more exciting range of outcomes. And I think it's good for everyone in the industry and namely the users of this technology to see that these experiments are being run because I

I think the guarantee is that in the end, developers and consumers of AI are going to get the best possible product. So you mentioned battery, obviously Kleiner you were at, now you're at Lightspeed. You've seen, you've seen, you know, you've seen venture over the past, you know, chunk of years.

And you've seen it evolve. And now, obviously, you're at Lightspeed, which is a large platform. Like, give us your take on the state of venture, especially as it relates to AI. Who are the winners and losers? What's the what's the best place to be from a venture standpoint? What's not a great place to be? And what's your outlook for the current sort of batch of funds, I would say?

So I think there's a lot of like conventional wisdoms that are thrown around in the context of this conversation that I actually believe are true, hence why they're conventional wisdoms. Like one would be companies are staying private longer and so therefore they're going to raise more capital in the private markets and more of the returns are going to be generated as private companies. That's one. I think a second is that the scale of the opportunities in AI is larger than we've ever seen before. And so these companies are going to digest a lot more money, I think, from investors and require a lot more investor capital than companies previously.

And I think as a consequence of that, you have more capital going into these companies and larger end outcomes. I think you could see a world in which venture returns actually at an industry level really start to look a lot better even than they have been in the past. Right. And so that I think is like pretty conventionally true. And I believe it deeply, which is like, I think we will see trillion dollar companies that go public as like multi-trillion dollar companies for the first time. And

I think what that kind of comes back to is this view that large platforms like Lightspeed arguably have an advantage because they have a chip stack, so to speak, to actually be able to capitalize these companies in ways that I think they need and others can't. And I think what's really interesting and sort of what drew me to Lightspeed was I feel like being on the right side of history right now is being at one of these large platforms who can kind of meet these really ambitious needs.

AI founders and their companies where they are and kind of be their partner of record all the way through, which I think has a lot of benefits. I would also say that I think there's increasingly clear room for specialists, meaning folks that specialize on a given stage or whether it's like focusing entirely on formation stage AI companies or helping build vertical AI companies or the like.

It does seem like specialization and being more boutique is another avenue of really standing out in this industry that I think is becoming clear. And the reason for that is because now the founders have more options than they've ever had before.

their ability to optimize like what they're, what they're really looking for when they're building their company, especially the early stages is like unparalleled. And so if I'm a founder starting a company today at formation stage, like I have my choice between raising seed capital from a large platform, that's going to be able to invest hundreds of millions of dollars to billions in my company over time and go all the way with me and have all the reach and resources of, of, of, of a global platform like Lightspeed. But I also can think about, Hey, like what if I want to go and raise seed funds from, from someone who all they do all day is build formation stage AI companies. And I think

That's a really good thing for entrepreneurs that can be overwhelming at times. But I think what it says to me is that you're going to see this just like really extreme bifurcation in the industry. And this is not the first time that you've heard the term bifurcation thrown around in this context. The reason I bring it back up is not to, not to say that it's novel, but more to say that I think it's, it just seems like more of an inevitability to me than ever before, given these constraints that are going on in the industry. And so, so I believe in bifurcation. I believe that scale and specialization are the only two paths to, to really differentiating in this industry at this point.

And then the other thing I'd say is, I just think because of how online our industry is now, it's increasingly becoming about who the partner is at the firm you're working with. And so typically what you see is that, sure, firms have brands, and I think that's never going to go away. But I think individuals and their brands and what they're known for and how they're regarded by entrepreneurs does seem to carry a lot of weight in the industry. And I think that's a very good thing. And what I mean to say by it being a good thing is that

I think now it's impossible for GPs to hide behind their firm's reputations. Like they have to be good investors. They have to be good partners and they have to really have strong reputations by doing good work and being good human beings with high integrity to the founders that they work with. And so I think when I think about like governing the industry and like good behavior of investors, I think that's a very good dynamic and one that I'm seeing play out. Yeah.

So you mentioned there will be trillion dollar companies, multi-trillion dollar companies. Like what does that company look like? Well, I think the first candidate we could point to would be like whoever gets to win the AGI race is pretty obviously a trillion dollar company. And I think you could argue that whoever delivers AI in the most novel and sort of dominant way to consumers is a trillion dollar company. And today it looks a lot like OpenAI that's kind of has that pole position. One could imagine the company that owns Cogen and builds the most performant and widely used frontier model for

building agents for software engineering is a trillion dollar company. It's not hard for me to see that. And today one might guess that's Anthropic, for example, right? And so I think these foundation model labs that are able to find a seam in the market on the enterprise or consumer side and build like market leading positions in those segments are trillion dollar companies, in my humble opinion. I think if you can also look at some of these like

really vertically integrated companies going after industries like space or defense as like, I mean, just the GDP of these industries is just so massive that it's just hard for me to see how a company that doesn't come in and vertically integrated and completely upends the existing industry structure doesn't have a chance to scale into that type of outcome. And obviously SpaceX seems well on its way, right? So that, so that, uh,

Those are a few examples. And I look, I think the big question that I initiated, we should always be asking ourselves is like, is there something systemic about the future that suggests that there will be more of these companies? Or are we just at some unique point in time where we're going to see five or six of them and then we're going to go back to the way things were, which was, you know, we used to get really excited about $20 billion companies. And now I think $20 billion is sort of a means to that, to that, to that higher end in a lot of cases.

So that's what I don't know. And I'm humble about the fact that I don't know the answer to that. But if I were to, if I were to take one side of it, I would say that there's definitely something systemic about the confluence of AI, about the confluence of having things like ubiquitous space travel and some of these other like fundamental capabilities that seem a little sci-fi at our fingertips. And the reason I get excited about that is like, yes, we see this set of companies being built right now that looks like they're going to be trillion dollar companies.

But I think what's more interesting is like once these companies are household names and ubiquitous and essentially act as platforms for entrepreneurs to come in and build new companies on them. I think the power of these technologies is just so unparalleled that I can imagine, for example, that we're going to see AI agent companies that are worth a trillion dollars because they're not just automating existing labor spend, but they're creating like entirely new economies around the work that they can do that humans weren't capable of. Bucky, thank you so much. I feel way smarter about the state of infra.

and AI and venture. I'm sure the listeners and the viewers do as well. Really appreciate you doing this. Anytime. Thanks, partner. Thank you for listening to Generative Now. If you like this episode, please rate and review the show. And of course, subscribe. It really does help. And if you want to learn more, follow Lightspeed at Lightspeed VP on X, YouTube or LinkedIn. Generative Now is produced by Lightspeed in partnership with Pod People. I am Michael McNano, and we'll be back next week. See you then.