From Rules to Reasoning Engines // George Mathew // #296
01:05:26 Share

Hello there, I'm George Matthew. I'm a managing director at Inside Partners and I take my coffee as a hot or cold vanilla latte with at least 1% milk. Welcome to round two of this conversation that I'm having with George Matthew. This is the MLOps Community Podcast and I'm your host, Demetrios.

In the first edition, when I spoke with George so many years ago, we reminisced about what he did before he became one of the most predominant VCs in the AI and ML industry. And in this edition, we talk all about what he's been seeing on the market and his theories for the next couple of years. Let's dive right in.

It's been two years since we chatted. I remember when we chatted last, I was in Greece enjoying the sun and I was totally oblivious to what was about to happen, which I think we talked even before the chat GPT moment ate up the internet. Now, what else has changed though for those last two years? What have you been focusing on and thinking about?

Oh my God, like nothing happened in the last two years, right? It was so quiet and such peaceful moments and everything related to data, AI, ML, to meet you as well. Wonderful to see you again. It has been too long. Thank you again for having me back on the MLOps podcast. It's a pleasure what you've been able to do with this community for

the last several years. Oh, thanks, man. Coming back to a podcast a second time around, particularly after, you know, having some perspectives even before everything got crazy. Yeah. When we last talked. So yeah, this will be fun. Yeah, I remember distinctly that you talked about, I was asking, how do you look at companies and what are things that get you excited in companies? And there was something you said, and I'm paraphrasing here, but it was along the lines of, you're looking for people

companies that are going to really be able to build generational organizations or something like that. Generational outcomes, yeah. I think it's generational outcomes. Which was the first time that I had thought in such long terms. And I was just like, wow, how do you even...

Wow. That blew my mind. And so maybe we can talk a bit about generational outcomes and what you think are the next wave of generational outcomes. Sure. Yeah, happy to do that. And I think it's a timely moment for me to have that conversation because I am entering my fifth year as a venture capitalist and certainly had quite a long career as a builder and

that has been able to scale up companies, particularly in the data AI ML space prior to, of course, being a venture capitalist at Insight for four years and going on my fifth year. I mentioned that because Insight just celebrated our 30th anniversary and we just raised our 13th fund, which is a $12.5 billion fund 13. And, and,

If you look at the arc of software for 30 years, and Insight has been very focused as investors in software from 1995 to now, I graduated college in 1995. And so for me to almost think about what I did in my career as a builder for as long as I was and now being a VC, it is

a dramatic shift to go from the moment in time where we started to see the introduction of client-server in 1995 and the advent of the web, the emergence of mobile at scale, the cloud infrastructure that ended up defining a good decade plus of software. And now we're again at this precipice and we're well past it in terms of how AI systems are being built as we speak. So

I can't be more excited by, you know, not only what we've been through for almost 30 plus years, if not longer, in terms of how software has become the most important substrate for how humanity continues to express itself and driving productivity around the world, but also what we can do for the next 30, 40, 50 years. And it's very clear to me that everything that

particularly in software from this moment on, is an AI-driven feature. And what does it look like? Because there's some things that I know we were talking about where it's like AI systems are taking more than...

And it's really thinking about how do we go beyond the screen? And I think probably the first step is going beyond the chat box and then we can go beyond the screen maybe. And so how are you looking at that and what are some things that you're seeing that excite you? Yeah, I want to set that up as well in terms of the march towards this current generation of AI systems that are being built.

It goes without saying, we should really highlight that we couldn't be building these modern AI systems without the emergence of the modern data stack itself, right? Because it's very clear

now that you need high quality data to be able to build your AI systems at scale. And certainly that proved itself out with the transformer architecture that really got to the GPT-4 models, the O1 models, certainly the things that Anthropic has released and what many of the foundation model builders have been able to capably ascribe in terms of both knowledge and reasoning. And that

embedded into, or let's say more precisely, imbued into the models themselves. And so you see the march from the modern data stack to now taking those models and defining the machine learning operations surrounding how you build and scale a model. And so more recently around LLMs and the operations of LLM ops.

And now the agent-based systems that are now possible, right? It's not just about what you can do in the foundation models themselves, but how do you orchestrate those effectively so that you have a agentic approach to being able to have multiple reasoning systems, multiple knowledge systems converge together to have a new AI system being delivered for an existing or a new problem. And so,

That's where I'm starting to get pretty excited right now because I realize that most of software up to this point has been based on

and building highly capable rules engines. But over time, rules get very brittle because you have to continue to encode what your business processes are into those rules. But what if you didn't? What if you had almost a reasoning system that can continuously reason what's happening and that reasoning could be human-like, perhaps even superhuman-like in terms of the reasoning that you're coming out of an AI system today?

And that's exciting, right? Because it starts to change a lot of where you can see the value proposition and generally how software itself is consumed, right? Because at some point up to this moment, you had to buy your software. You had to enable someone from a services standpoint to get that software working inside the enterprise. And you had some people that were operating internally.

the software as the personas that cared about that particular piece of software. If you use the CRM software, then salespeople, sales managers, sales operations cared about that software. If you were an ERP software provider, then you had a number of folks that were in finance and operations that cared about that software. Now, this future that we're

barreling towards incredibly quickly is one where the AI systems are encompassing the software value proposition with the services in terms of enabling that software in the enterprise and the first advent of digital workers, synthetics, like things that we haven't seen before where there's a full

autonomous experience of how that software is now functioning inside the enterprise without necessarily as much of the cost involved with services in terms of putting that software into work. And certainly now, even synthetic SDRs, synthetic BDRs, synthetic developers, synthetic

Process automation folks that can now, you know, instead of manually processing that information, you have a full reasoning system that's processing. That's very similar to what the investment we made with Crew AI was in terms of just being able to automate all the, what would be historically automated.

manual robotic processing with very little smart exception handling. And now you have a high reasoning function involved with how you do a lot of your back office automation and processing. So that arc of like software now transforming all the way to the emergence of these AI first systems and these AI first systems are encompassing software innovation.

and the services and labor, that's something we haven't really fully seen yet. It's just starting to emerge in some of these key markets. And I'm pretty excited with what that opportunity looks like. Let's talk for a second about a few different papers. I know that you pulled up what you really liked about Dario's paper. And then there's these different

almost views of where we could go with the future. And it's almost like one is hugging machines and the other is being strangled by machines in a way. Yeah, that's funny. Let's set the context, right? So I think it starts certainly with Leopold Aschenbrenner's paper in June of 2024, which is situational awareness, right? He was outlining the next decade ahead in terms of

what was necessary to build this future. And we'll talk about both papers in a moment. But what I think I saw in that paper was a very mechanistic view of the future. What would it take to scale out these clusters? What would it be necessary to go from AGI to superintelligence? And what the timeframe was to do that? What does it mean to build a trillion dollar cluster itself? Like,

How much we have to lock the labs down, which is really funny that we talk about locking labs and security of AGI, particularly the deep seek situation. That's even six months or whatever, seven months. It's like a profoundly different way. At least the rest of the world has reacted to it than at least the paper came about it. What does alignment look like? What does it mean to humans?

drive democracy in a free world while all this is happening. It's just a brilliantly thoughtful paper. I think it was also just a little heavy-handed in terms of how it thought about the timing of AGI, like what would happen. I thought one of the most interesting things was, which whether and when it happened or not didn't matter, that was really exciting, was this idea that if you could build a superintelligence, it could automate

the research output that's necessary to create all the possible futures that you would want as humanity. And I think that was my, yes, that does make sense. No matter when it happens, that is an incredible view of unlocking cognitive labor, unlocking additional

opportunities for science and technology, unlocking robotics, unlocking the military edge, and more broadly, just GDP. But it does reflect a very mechanistic view of all this. If you do these things, these outcomes will emerge. And it was pretty almost brutalistic in terms of how I thought about it.

wildly important paper though. But then Dario comes out with to your point, Demetrius, like Machines of Loving Grace. And it talks about, you know, not a completely different future, but it talks about like

why these machines can actually help us as humans. Like, how do we enable a better humanity because the machines are helping us get there? It actually reminded me of the, I don't know if you came across the science fiction series called the Culture series, right? Culture is

It's the systems, the AI systems in particular, are just like infused into how we live, right? And it's like a very post-economic society that there's an infinite amount of resources and how the machines have helped us progress beyond our current constraints as humans. And what does humanity look like in the future? It's a beautiful view of what we think about the coexistence of the convergence, the singularity of humans and machines together.

And both are, are definitely possible. But I do find something beautiful about Daria's thinking because we have this, this tremendous angst about the future of AI and its impact on humanity. And, and,

I think there is a lot to be worried about. I have noted out about that, particularly if we are in a stage where we're building systems that are not aligned to human values, certainly at scale. But there is a possibility of reaching a future where

humanity itself progresses in an exponential way for the benefit of humanity because our AI systems are here to enable that to happen. And I think I would highly recommend anyone who hasn't read the Machines of Labyrinth Grace to spend some time with that paper in particular. Both papers, but certainly that one if you want to believe in a more positive future for AI will transform humanity. And one thing I wonder about as we

think about that shift is it going to be a layer on top of legacy or is it going to be built from the ground up and the reason i ask is because the i was with some friends two weeks ago and they were talking about how they tried to create a jira agent and it worked great in their hackathon and then when they plugged it into their real jira instance it fell flat on its face because

Humans in JIRA are doing the least amount possible to get across what they need to get across. And another human can see it and understand it because they have all the context around it. But the agent was seeing it and it had no idea what was going on. And so I asked, would you then go and rebuild it with some kind of a graph knowledge base around it so that it could have the context? And he said something funny. He said, no, I would go and...

and I would just think about it differently. Like instead of trying to shoehorn an agent into Jira, I would just think like, how can I make a better Jira? Yeah, I think both things are going to happen, right? I think there's going to be a number of situations where you are going to be able to take your existing systems

put that into an agentic workflow where you are combining it with some of the reasoning and knowledge that's imbued into a foundation model that might be fine-tuned or you might have a retrieval augmented generated view of just giving contextualization for how

that model is going to behave with that data that's provided from that existing system. And you can get some tremendous benefits for how a compound AI system emerges there. And then to make sure it's your point, like there's going to be like a number of key new systems that are going to emerge that you're just not going to want to take that existing process. You're just going to come up with something completely new and

that you don't even need to wrap your complexity of the existing function towards and just build something radically different from what you've done before. And that's an interesting idea. Is there a different way of thinking about JIRA and AI?

ticketing and software development that we've been very used to a process that could be radically reimagined. All right. Yeah. And I think it's a fair point. There could be quite a few of those things that we're just used to doing in a certain way. And we no longer have to use AI to automate what we're doing, but just completely create a new way of doing it.

Yeah, for me, I feel like every time I see an update from the Cursor folks, I think, man, they're going to eat up everything that a developer does. And so it feels like there's a world, a future where everything that a developer has to touch is in some way, shape or form available.

connected with, it doesn't necessarily have to be cursor because cursor may not perform if we see another UI UX change or shift as we saw with, with co-pilot, like losing the lead. And so you think about it that way. If something is aware of what you're doing and what you're trying to do, then wouldn't it be able to update tickets for you too? That's right. Yeah.

Yeah, go further downstream in terms of what the

that user, their personas and what they're doing versus just the task at hand, right? Yeah, this is where I look. I've shifted my thinking on this quite a bit even this past year because I think about a year and a half ago, a lot of my thought process was that most of what this AI-based emergence in software is going to look like is like a bunch of, call it co-pilots, right? You have work and this thing is going to just help you do that work better and that's...

generally the mindset that a lot of folks have had on co-pilots. Now I've really embraced the idea that you can see full autonomous autopilots and those full autonomous autopilots don't have to even do the work the same way you've been doing it as a human. Wait, click into that a little bit more because...

It makes me think about how when they let two chatbots loose, they created their own language. Yeah, that's the beauty of some of the probabilistic capabilities of the models that we can now see these emergent behaviors that we were...

not expecting just because of the constraints of like how we think about it as humans but these probabilistic models can interface with each other in ways that we probably haven't even thought of and so I do think there is this value in we worry about confabulations and hallucinations but I actually think there is some significant value in the systems that are emerging here that perpetuate

the probabilistic systems that are emerging here to have different conversations with each other than the ones that they would typically have with existing business processes and existing people that are interfacing with them.

So I think there is a lot to be said there. Now, look, there's a lot more to go because it does feel like the vector of reasoning, there's certainly quite a bit of advancements that have been made with the emergence of one model. Certainly we can talk a lot about where DeepSeek is, but you can sense that

The reasoning vector has been really kicked off this past year, even as the knowledge vector, the accumulation of knowledge, we're probably slowing down because there's less and less data that is pristine to consume from the public internet. And therefore, we're not using private data sources, et cetera, to be able to train this next generation of models that are coming to market.

Yeah, there's that vector. Do you feel like there's other vectors that we are going to pull on to? Because I look at the idea just how humans now are okay with waiting 20 minutes for an answer. And when ChatGPT first came out, that was unthinkable.

We wouldn't wait around for 20 minutes, but now I use deep research all the time and I'm like, yeah, just go and do it, whatever. And so that's another vector of the patience of a human. And then you have the actual like tech vectors that you're looking at. Like you're saying there's the reasoning or maybe there is some kind of, there's other things that you've been thinking about that we could augment. Keep in mind, a lot of ways, the reasoning vector encompasses some of the

things that you just mentioned, right? Because if you think about the notion of the chat itself, reasoning inside the chat is instantaneous, right? It's synchronous. It's immediate. You want that answer right away. But you can also think of, as you advance into more complicated, longer form thinking, the short and long form thinking, and so you get into some of the longer form thinking, then the reasoning can take longer.

right? You can think about it and come back with an answer that doesn't have to be synchronous to the chat. It can be quite asynchronous to go to your point, do the research and come back. And we're patient about research being done as long as their thinking is higher quality at that point with a longer form response time. So I do think that it's been proven now that they're just like humans have

short form and long form thinking, there's no reason why our AI systems shouldn't also have more immediate short form as well as longer form reasoning and thinking. It's funny to think about the idea of hallucinations as a feature and allowing those to happen in a way or almost like you're giving the space for models to go and get creative and

and they can do that over here maybe in a little sandbox and then they can come back with some wild ideas and you have the almost like the judge that is going to say does this work or not and then reason with that idea and might be total trash might not work at all but in a way like giving it

be directive to think outside the box and almost like hallucinate on this is a funny way of looking at it. Yeah. And when you think about the type of work we're either augmenting or fully embracing as an autonomous system,

You can see vast quantities of creative work where you do want, you know, a highly, call it, diverse output that you can iterate through and just have the models, you know, to a lesser or greater degree, genuinely hallucinate and come back with output that you wouldn't have thought of and leverage that quite effectively, particularly in many of these creative domains, right? You do want that constraint to be relaxed, to get more creative,

more possible creative things out of the model that you might be working with. And then in other cases, if you're doing a trade settlement process or a straight-through process in a back office, that's the last thing you need. You just need a very tightly wound model that doesn't go off its guardrails. It has...

The human like reasoning to handle and understand like when an exception occurs, like how to process that, but then everything else, it literally does what it needs to complete that process. And I think that's where I'm really excited, honestly, with where

the recent developments around like something like deep seek is, is really opened everyone's eyes. There's a lot of things that we can talk about or with deep seek, which happy to do it. But one of the things that really caught my eye was like, you can take a very large model, uh,

and drive model distillation to a point and using some really compelling techniques to just be able to hyper optimize the efficiency of how a model is not only trained, but also inference using reinforcement learning, transfer learning techniques.

you can build very interesting domain-specific models that really stay on the rails, do exactly what they need to go do, have high-quality human-like reasoning involved in it, and doesn't go off and think about 14th century British history. So I think that is a tremendously valuable element of what we are going to likely see, that it's not just this idea that you need...

A, go from a trillion parameter GPT-4 style of large language model to whatever, 10 or 20 trillion parameter model or maybe even bigger. There's a lot of questions about how that's going to emerge. And we'll see. I'm sure the researchers are working through that. But there's something to be said that these

elegant, compact, domain-specific models. And I said this years ago. I just never saw a manifestation of it like we see with DeepSeek are going to be just as important, if not more important, for how the proliferation of AI systems function in not only the enterprise, but probably most of humanity. Yeah. Let's keep pulling on the thread of DeepSeek right now because it is so fascinating. It's captivated the world. And one thing that I...

am wondering about is your thoughts on the other model providers and what that's done to their positions. It's got to raise a bunch of questions. Like I'm not inside of any one of the model providers and just to state it out loud, Insight never invested it to any of the foundation model builders, which not because we didn't love what the foundation model builders were doing. We didn't have deep faith in what was

coming out of the model builders, factories and research shops, who were profound believers in a lot of ways. One was just the valuations never felt right at any moment in time.

And hindsight P20, there was probably a few that we should have been in right at certain valuations. Like I'm pretty sure like the $1 billion valuation round at Anthropic was a good one. And we had a chance to be in that one. So you think about these anti-portfolios that you build over time. That's another one in my anti-portfolio for sure. But we just had a very strong belief that the data...

was the most important substrate. And I'll talk about some of the sort of thematic investments that we've made, particularly the idea that the data was the most important substrate for building these AI systems. And we also just knew that with how much was continuing to popularize in terms of open source models, what we're seeing in Huggy Face, like there was always a moment where the best model in the world

from a knowledge and or reasoning standpoint, continuously had, call it the near equivalent, emerge within six to nine months afterwards in the open source community. And DeepSeq is probably one of the best examples of that, at least for the reasoning vector. Like this is equivalent to what the O1 models, if not as good or better than what the O1 models can do, and done with hypothetically a lot less resources. So if you're a

big model builder, you got some work cut out for you at this moment, right? Because you got to show this vastly incremental traversal of either the reasoning vector or the knowledge vector, which again, the model builders are really thinking about as they're training these next foundation models, frontier models that they've been working on. But when you do that, you're spending hundreds of billions of dollars across the network, the

GPUs, the power required, the resources required to make it happen. Up to this point, every time a significant model evolution has occurred, it does seem there is a more elegant, efficient way to catch up. And certainly that's what should worry any of the model builders, right? Particularly in the large scale research shops that are building these frontier foundation models.

Like when do they actually have a true moat that is sustainable for more than six to nine months? And that, that should worry everybody. I heard a rumor that even one of the more efficient model builders that were open sourcing like Meta and the Lava models, like,

I think there was a moment where Zuckerberg was probably yelling at his team about, okay, how is it possible that you're asking for Y amount of dollars to go build this next generation of Lama 4 and beyond models? And this is out here doing it at a fraction of the cost with some very clever methodologies and distillation techniques. Yeah.

So, yeah, everyone's got to be thinking about this, right? Everyone who has spent hundreds of billions of dollars up to this point have to be thinking, probably important, right? Important like the, I used to talk about this, as you needed the space program to build the aerospace industry. And yes, we do need to put the fundamental dollars into the space program. But now we're at a point where it does feel like

we should be able to build the aerospace industry. And so at least in the analogy, what I'd look at is like the AI research versus the industry that's forming around AI systems. So when we're doing that, should we be spending the dollars that we had to spend on the research to get there and how much more is really necessary? So there's got to be a lot of questions going on and the big research about this exact topic. That's a great analogy, bringing the aerospace industry to life.

the 2025 basically and making sure that we're building on top of what has already been established in a way and now it's just time to take off and so there there is another thing that is interesting when it comes to these models being built in my eyes that is it's a bit of a crapshoot and you don't really know what you're going to come out with after you spend all that money

And it could be good. It could just be mediocre. And I think we've seen a lot of models that come out. They don't really make a splash because they're it's cool. Yeah, there's another foundational model that's out and almost like this base model that you can then go and fine tune. But you quickly see them fade away into history because they're not

that good. Yeah. Compared to what's out there. Yeah. Yeah. Now I think the long tail of models littered on Huggy Face is hopefully as much an indicator of some incredible things clearly in the leaderboard, but also there is quite a bit of a long tail there. And I think it just shows that there is a

There is definitely room to continue to create these beyond expected outcomes like we just saw with DeepSeek. But to get there, there's a lot of failure that you need to iterate through. And sometimes that's happening inside of a specific research shop and their work. Sometimes it's happening across research shops. And certainly it shows in the long tail of what we see on Hugging Face today. Yeah.

The interesting thing, too, from my side, I guess not as interesting, but I don't know if you saw that Mistral dropped a new model.

a few days ago. And for me, it was like bad timing, man. Why would you try it? It's like just a complete, yeah, tough choice to have to put a new model out there this week. Yeah. But Mistral was that in the past. Like Mistral had their moment of they put out an open source model. It was all the rage. Everyone was talking about it. And I think they got a little confused that, oh, we could do it again.

and we'll put it out. But it wasn't, again, going back to the quality of the model, it's not this groundbreaking thing. It doesn't have this whole narrative around it. It's not capturing and captivating everyone's eyes. And in my eyes, it's like you do have the mistrals of the world that are in a bit of a tough spot right now because of how they're trying to do things. And somebody sent me an article that said that mistral's trying to go public.

And it was like, out of all of the model builders, I was not expecting Mish Draw to be the first one that goes public. I'm going to be honest. It wasn't on my dingo card. Yeah. Yeah. The race is continuing across the existing model providers. And yeah, I would agree that, you know, you'd expect...

a little bit more scale, right? For the few that would end up going public, you certainly would expect a bit more scale in terms of where they are in their respective journeys. We tend to see a little bit of this, uh,

bit from the view of weights and biases being a an investment that my first investment fact that i made at insight and being the experiment tracking parameters tuning version control de facto solution it turns out that open ai anthropic cohere mistrel meta nvidia

are all customers using weights and biases for building high quality models. And because of that, you get a pretty interesting sensibility, even just in terms of the amount of users there are in these respective organizations and how much leverage they're getting from a tool like WMB to converge and build their models. It is fascinating to see that, you know, there's definitely like this, call it core of five or 10 research shops that are doing the

the work that's leading the market. But then there's these moments where everything gets disrupted or reinterpreted when something like DeepSeat comes along and then you have to think about what just happened. But it just seems like there's still great work coming out of OpenAI and Anthropic and Cohere and Mischel and Meta and NVIDIA for sure. Yeah. Yeah. And you need these moments to raise the bar and keep people on their toes. And also it

It changes the landscape. It makes things a little bit more, oh, cool, I can go use DeepSeek now. I'm going to think twice about a different model if I can host it on my own. But it's not like you just grab the model and then you're good and you can be all right with it. I think the other piece is,

We just had this thread in the community that was fascinating to me because there was someone asking about the cost differences that people have seen between using SageMaker and not using SageMaker. And somebody chimed in on the thread and said, yeah, we saved five grand a month from our

switching from SageMaker AI to just regular, I think they were using, what is it, ECS? EC2, yeah. EC2, that was it. EC2 instances. Oh, so they didn't have a, call it, managed SageMaker experience. They just rolled their own tools into AWS Empra. And they saw that, wow, okay, now we're saving five grand a month. But what you don't see, and they were very confident

transparent about this in the thread was how much is the cost of having engineers that know how to do this what are you really trying to optimize for here how big is your team how mature is your team all of that stuff comes into play and it's that classic build versus buy decision

And so you can't just think, all right, I'm going to just grab DeepSeek now and I'm good. I don't have to worry about anything. Because as another friend told me, he said, whenever possible at my job as a ML platform engineer, I am trying to outsource the ML platform to one of the big model providers because I don't want to have to deal with that. It's such a headache. Yeah. No, I mean, it certainly makes sense, right? Because if you're one of those people

ML teams, your job is to introduce the highest quality AI system that incorporates that model, that brings in the data that's necessary to train a domain-specific model for your needs, and get that into a production workload that is actually benefiting that organization. And yeah, the more you can get

platforms and tools that help you support that. It just, it makes sense. If anything, that was, for lack of a better expression, the picks and shovels thesis that we had at Insight for everything in this space. We just definitely leaned in on most of the picks and shovels in this area. Yeah. And he's even in a regulated space and he was saying, unless you have

data issues and regulations just outsource it picks and shovels it and get rid of it and then you can spend your time on higher priority issues and so it's fascinating to think about that the the other yeah by the way I've come across certain enterprises where I've seen oh yeah we fill in with a lot of pride like

We built very custom bespoke ML ops pipelines. And I was like, oh, you poor thing. Like, why would you put all that energy into that when you could? And some of the decisions were made a certain time. There's like a belief that you couldn't take everything from what's available in the market and sell.

coerce it and harness it together to make it make sense. But yeah, I got to imagine like that's not going to be a great use of resources, particularly in an organization. And you're always having the conversations every quarter, I imagine, or every year you're having a conversation like, do we continue to support this or do we try and migrate to something newer? Because in a way, when you choose your stack, right,

It's a snapshot in time of what the best capabilities are at that moment in time. And if you're really going all in on it, I remember we had a guy from Pinterest on here and a lot of the stuff that they are using for their ads platform was chosen in 2018 or 2017. And there's been so much advancement and they have to decide, do we...

you manually, maybe you take a piece off here and then upgrade it here, but maybe it doesn't fit. And so you can't add that new shiny thing, which can be good. It can be bad. You don't always necessarily need the new shiny things, but, but really thinking about like how, if you decide to go down that route, you're deciding to take what is out there best right now and

and have that snapshot be what you're using. And then forego all the future innovation that could come. A little bit, huh? From the things that are outside your four walls. Yeah. That's the thing you got to be careful about these build versus buy decisions. And this is not just in your AS and in just general. If you're going to lock down something now, you're going to get likely something that is very highly specific to your needs. Yeah.

but then eventually just going to be this external innovation that's going to keep moving ahead.

One of the things that I've been fascinated to see, by the way, since we did talk about build and buy, is also this wave in AI-based software systems or AI systems that are empowering and enabling this next-generation software, to be more precise, is how much more you can now do custom software development better. And the reason I just mentioned this is that before, you had to almost rely on your package software provider to do the things that

you needed them to do in the product and you're waiting for those releases to happen and then you were implementing that package software. But we're not a world where you can build taking some of the componentry that we're talking about and the componentry, the data, the pipelines, the

Building blocks can continue to evolve, but then how you harness that can be very much highly customized software experiences that are, for all intents and purposes, custom software with a number of really upgradable underpinnings, like the lifecycle management of it becomes much more straightforward. So I do think there is a moment here that you can just deliver things.

highly specified custom software, but the advancement of all the underlying componentry to do it continues to have its own upgradability lifecycle. That's not where you have to build bespoke stuff. It's actually in the layer, the abstraction layer above, where you can continue to iterate that faster because you have these building blocks you're working from. And it is really fascinating to see how much

you can get done today from a custom software development than ever before. Is this because you're thinking about it as I can prompt my software or my software, I'm not interacting it always,

Or I'm not interacting with it always through a GUI. It's also just through language. There's that, just the way that the UI falls out in this experience. But it's also just when you have to now take your existing software packages and almost bolt on the AI capabilities versus natively building it up from the bottom up and then natively having a very...

enterprise specific functional experience with your software that you're not waiting for someone to develop a feature that you need versus you're capable of building those higher level features yourself. I think you're getting much more specific curated software built for your business.

Right. And that, that, and by the way, like there's moments you don't want to do that, right? When everything's settled and there isn't a lot of innovation that's going on and you can just take something off the shelf and it's as good or better than anything else that you can build. But we're at a moment where you can build stuff better than you can take anything off the shelf. And so that's why I think this would be a compelling moment for custom software development in the middle of all this.

Yeah, and it is looking at that abstraction layer and saying that we're going to benefit from things as it continuously moves underneath us, but we can have what is needed above for the users. And yeah, it is interesting to think about because it does feel like it is more Lego blocks now. And since it is so new and there isn't really much settled yet,

And every day you get something new, you are more inclined to build something

quicker or build on your own. And I imagine, I'm just thinking back to the Pinterest story and how I bet in those days they went out and they were like, look, there's not really anything out there that's good enough. We can create the abstraction layer we need, and then we'll have our custom stuff underneath that is getting us to where we want to go. But I

inevitably, like you said, it changes. There's that right now we're in that phase of it's easier to build for this type of stuff and create our own custom software. And then you probably will get over the hump and you'll start to realize, all right, now there's offerings that are pretty nifty and we're going to want to go with those instead. Yep. And there's another nuance to add in since you mentioned Pinterest. When you look at

Some of the more compelling open source that has actually emerged this past decade, it actually started as custom functional work that was needed inside a civic organization. Like, oh, other people might actually need it. Airflow came from Airbnb. Yeah. Right? Kafka came from LinkedIn. Yeah.

And so you look at these like now modern OSS components, they actually emerge from a very domain specific need that happened to be in a very specific domain and then generalized into the rest of the world by just releasing it as open source. So that there is some interesting benefit to even building these things in a way that you're getting what you need out of it through that custom environment.

And then when you realize there is some things that other people will also need that is reusable and there's a benefit to that open sourcing, we certainly saw that with the number of open source capabilities, particularly this last decade. Yeah. What a great example. Speaking of open source, yeah.

There's a little open source project called Spark. Oh, that one. Is that still around? Is that still doing anything out there? People are trying to kill it. Yeah. But it's still around. Yeah. I got a fun story about that, by the way. About Spark? Yeah. I was at Spark Summit 2015 and...

My team at Alteryx, including myself, worked on shipping Spark R and our co-developers on it

was a very clever VP of engineering at Databricks named Ali Ghatzi and his team. No way. And yeah, that's how we got SparkR into the market. No way. Yeah. So you were working with him back in the day and now you're working with him again because I saw recently you posted you were part of their

massive Series J, I think. Series J, yeah. One would not have imagined that Databricks back then would have arrived at its Series J, but here we are a decade later. Yeah. How is it even possible that a Series J exists? That's one thing. And then why now? What is it about you and you've

obviously been in the Databricks slash Spark ecosystem for a decade plus. Yeah, yeah.

Why all of a sudden do you want to become a shareholder? Yeah. Sometimes it is okay to be late to a party, right? Yeah. In this case, it's a little bit more methodical than just being late to the party. We were investors about two rounds ago with a much smaller position. And we were continuously excited by how much the evolution from Spark to Databricks to really...

what Databricks stands for today as a unified data management platform using the lake house as the construct for all the pre-processing, all the data prep at petabyte scale, and then this little

capability that they've introduced called data warehousing inside of Databricks, which is now the data warehouse product is a $600 million business for Databricks, which is incredible, right? You think about it. As you sum all of these things up, the Mosaic acquisition, the in-house built

capability around data warehousing, the Unity catalog itself, and how it becomes this unifying layer for the enterprise. The tabular acquisition, and now owning the Iceberg format. And that's just like the product strategy. We're not even talking about how much the go-to-market and industry strategies have really evolved from even its early inception.

It just feels is a generational company. I don't think I'm saying anything out of turn by saying that out loud. And it does feel like there's many more miles to go. And so that was our thought process there. The opportunity came up to

be part of a very large $10 billion Series J, which is, I think, the largest venture capital deal ever. Our friends at Thrive and Josh Kirshner led it. We co-led it with a pretty sizable check that was really helping

Databricks continue its journey. And for them, it was just making sure that there was tremendous amount of just liquidity for everyone who had been on this journey for a long period of time to consider their needs as they continue to grow and scale the company all the way to the next levels and stages of growth. Yeah, we couldn't have been more excited. And it certainly was a, was a big investment for Insight. It was a

very large round for everyone that was involved. And it does appear like there's many more miles for Databricks to go in the market. And we're just delighted to be a part of that journey. And I guess one would say it's never too late to come to a company like that and just help them along their journey. Yeah. You say like generational, and it really does feel like that. And I instantly thought,

Do you feel like there are any other generational companies in this space as you look around in the data AI space? Because for me, it's like Databricks is just performing top notch right now. But I don't know. Maybe you're looking at other companies and thinking, yeah, they're generational companies too. Or maybe it's if some things go right,

There's going to be another one of those. Yeah. Look, there should be, by definition, the next generation of companies that emerge. And so they're coming. And I just want to be clear about that. But it does feel like there's something special at this moment. Certainly with Databricks, you know, we shouldn't not mention what Snowflake has been able to accomplish as a company in that same cohort and period.

If you look at Palantir as another great example of a company that has in a very different way, build something generational in nature. Yeah. It's, it is exciting to see those folks who started 10, 15 years ago, build the companies that are now the sort of the hemiots, the foundations of how at least the data IML space continues to progress. Yeah.

I think there is a clear view of what's next, right? Without stretching too much, you can certainly say that the foundation model builders and the growth and scale that they've respectively had have been quite impressive. The enablers of that growth and scale, like a company like Scale.ai, for instance, is pretty impressive as well. And just being able to initially provide a lot of the necessary resources

efforts to be able to enrich data that's necessary to build a model and now providing things like RLHF for the model providers themselves as they're coming to market. So it does feel like there is something surrounding these current model builders and the capabilities underlying the model builders that have gotten pretty big.

it is a little trickier, right? Because when you look at something like Databricks, it hugely is benefited by DeepSeq, right? And the reason I mentioned that is if you think about what DeepSeq is a manifestation of is, and I think everyone's been talking about this past week, it's like this idea of Jevons paradox, right? As the cost of a resource comes down,

you consume that resource more and it takes up all the capacity that's available to you at that moment. So I think that should be the case for everything related to the GPUs, the compute infrastructure that we're talking about over time. But we have a moment here where no matter what happens, you just need high quality data to be able to build this next generation of models, whether they are hyper efficient models,

domain specific ones that you can see emerge from like the likes of deep seek and its cousins that will literally come out within the next weeks and months.

as well as some of the largest foundation models in the world. So we feel pretty good that we're sitting in a good place with the Databricks investment. It feels like it's going to be a bit of a roller coaster for a little while for the foundation models themselves. And look, the next generation of companies are coming about and getting to scale. I mentioned the Weights and Biases example, right? There's 900,000 machine learning practitioners that are now standardized on Weights and Biases.

There's a lot more miles to go for a company like Weights and Prizies now. So I do think it comes in these waves, right? And so the wave of like full scale and maturity is out there and we see them as public and private companies now. This next wave of particularly around the foundation models and the support infrastructure around those foundation models have come to fruition as we speak. And then you're now starting to see this next, you know, much smaller companies

incredible founders starting to build that will get to the next level of scale in the five, 10 year timeframe. Yeah. That is one thing that you have a nice position of being in is the

almost like round agnostic ability that I know you don't do super early pre-seed, but you still see really early companies and you get to see and participate in the later companies too. We've been very specific at Insight about this. So we've even said this out loud. We don't think that stage is a strategy, right? We don't believe that this idea as an investor stages a strategy. We should be able to find

the generational growth opportunities at whatever stage that they might exist. And we tend to be a little bit more focused on that middle stage

stage of growth and certainly have done quite a bit of work, particularly in our teams that do a lot of the scaled out work in terms of buyouts and some of the larger checks that we've written. But that's not a strategy. We look for growth everywhere. And one of the wonderful things I had a chance to do working with my partners for the last four years is really build out our early stage, right? Where we are now very...

capable at not only the series H check, but occasionally even the C checks, right? Because we see these opportunities and we see that profound growth happening with a founder that just really has a maniacal view of what they want to deliver and build in the future and thematically want to be part of those conversations. So that certainly has been part of our strategy. So work with founders whenever and wherever that growth is existing, independent of stage. How

Are you thinking about when it comes to the way that we are interacting with computers right now, we have very compartmentalized applications, but it almost feels like some of the stuff that you're insinuating at, it would be better to just have the one AI application that can know and be with us, right? And you mentioned it about the sci-fi series, how...

It's called culture. Yeah, the culture tiers, yeah. And in order to enable something like that, we need this... I think Microsoft is trying to do it right now with Recall, where it's like this always-on AI that can come and it can help you and be there with you when you need it and then just be in the background when you don't. And for me, the way that I'm trying to wrap my head around it is, yeah...

it's much better if you have all of the context to give to the language model and then it can reason about it and it can understand. When we go back to that Jira example, if it doesn't have the context, then it gets lost. And so in striving to get to that point where you want,

To give all of the context, I think what you end up doing is just saying, come with me everywhere, be with me, like my shadow in a way. And so have you thought about that and how that could potentially play out in the future? I have a little bit. Yeah, I have. And by the way, interestingly, that description of a almost ever present in how to say it, like just like completely in the moment.

available AI system that's doing things that it needs you to, you need it to do. It doesn't have to be single and monolithic either when you think about it, right? There could be a view that it's like just,

hundreds of agents that are just continuing to marshal out to do many things and then converge into your needs. So I don't know if I necessarily see this as a singular thing that just is always constantly doing the bidding of what you want it or other folks would want it to do. I actually see a lot more, at least in the near term, this emergence of many

massive amounts of personal enterprise and almost hybrid agents that are continuously doing little things, medium-sized things, big things on our behalf. While we're talking here, wouldn't it have been nice for an agent to go figure out how to book that trip that I need to make to the

book the ticket. That's the kind of things that we're starting to see with these automations that are embedded into the foundation models, but also just in generalized sort of agentic systems that are coming about. So I do think that we're going to end up in a world, at least in the near future, that it's about these really well-meaning agents that are doing smaller things, tasks that are

rote repeatable that should not have to be done by any other human on our behalf. And I think that should be how at least the near-term future is. And then we'll see where the more science fiction-y future of a complete system that's always with you, which is part of your being. It seems like it's a little further out, but certainly that is a possibility that could emerge when we're all said and done. Yeah. And I've thought about it as the

what you're talking about there is like where is it going to live is it going to be in on the web and or is it going to be something that you're starting to see more is these ways that the products are being created is directly in slack like devon yeah for example and then you just deal with these agents in slack and i think it would be yeah

Pretty funny if all of a sudden Slack becomes your control center too, and you just send out agents from Slack to go do different things that you needed to do. Yeah, it's actually, I was searching for the word because it was escaping me, but it's the nature of how AI systems are going to be ambient or not.

I think a lot about that, the ambience of our AI systems of now and the future and how could they be so embedded, so ambient, so just an extension of how our minds work that we think about it and things get done. That does feel like...

not a science fiction-y future. I think actually that we're not that far away from getting to more of these sort of ambient AI systems that are doing important work at home and in the enterprise and everywhere in between. Yeah, because how many times, I don't know if it's happened to you, you might have more willpower and focus than me, but

I have something in my mind that needs to get done. And then I pick up my phone with the intention to do that thing in the moment. And some kind of a notification distracts me. Of course. And then 30 minutes later, I'm like, oh, what was I? What did I pick up my phone for? I can't even remember anymore. And that task that I wanted to do is in the ether. And maybe it comes back to me. Maybe it doesn't.

But what if you had an ambient agent that the moment that task came up, fires off and goes and does that task in a highly reasoning, long form way, 20, 30 minutes later, or even longer if needed? Yeah, I'm just excited for a Siri that actually works. And then I can give tasks to because right now the Siri that I use, it's very hit or miss. And so you do see the

where, yeah, typing into Slack chat is great, but I also have a million notifications on Slack, so it would just be cool if I could talk to the phone directly or talk to the computer directly. It does seem like a more obvious opportunity for Apple intelligence, right? Just to be that conversational layer, but it doesn't seem like it's a workshop. We'll see. Something's going on there. I don't know what it is, but something, yeah.

♪ I'm such a bitch, monster ♪