Bridging the Gap Between AI and Business Data // Deepti Srivastava // #325
57:13 Share

What if you could put a box, right, in between all of your data systems on one side, all of the data, like databases, data warehouses, whatever, data systems, and your LLM systems on the other side, and just have straight lines. Oh, you need data from Salesforce? We'll go get it for you. You need data from Postgres? Oh, we'll just go get it from you. You need data from Snowflake or Databricks? We'll just go get it for you.

My name is Deepti Srivastav. I am the CEO/Founder of Snow Leopard AI, which is an intelligent data retrieval platform that we're building. And I take my coffee with milk and sugar, and you can come at me for that.

You just said to me that you, back in the pandemic days, went to some virtual online meetup that we did. And it was on MLflow. And I think I know exactly the one that you are talking about because it ended up being one of the most watched

episodes later when we put it on YouTube. And I really think it's because it hit a nerve. And the episode was the difference between cube flow and ML flow. Oh, interesting. Yes, maybe.

And it's super confusing, especially back in those days in 2020, folks didn't really understand either one or how, when to use one and when to use the other. And Byron came on and he explained it like, look, cube flow is a sledgehammer and ML flow is like a little pickaxe. So you, they're both great tools. It just depends on what you're trying to get done. And so I, I,

I thought that was awesome. And it's really cool to hear that all the way back in those days, you had seen the MLOps community. When I wasn't sort of, when I wasn't as excited or when I wasn't in it, actually, I was excited about the community, but I wasn't even in it at the time. And I was like, oh, this is interesting. At some point, I should look it up. And lo and behold, like, what is it, five years later, four and a half? Here we are. So that's very exciting.

Full circle. And so the thing that I think is really cool, as you were saying, yeah, back in those days even, we were talking about this problem that you've been hearing and talking with folks about a lot. Can you just break down what the problem is and then we can get into different theses around it? Yeah. So there are many ways to talk about the problem, but the crux of it is,

In this new world of LLMs, like, we expect AI to, like, be this trillion-dollar, like, new industry, new platform shift. By the way, I bought into it, like, this is why I'm here with Snow Leopard, right? Like, I bought into it, whatever, 22-ish when the...

When chat GPD really broke. Yeah. When the hype happened, I was like, cause I'm not into hype. I'm a system. So, so backing up, I'm a distributed systems person. I've been an infrastructure my entire life. We are anti hype. Just, you know, we are like, we are the opposite of early adopters. Let's just put it, um,

Because we care about production and five nines of availability and all that stuff. And you can't really just be like, oh, yeah, let's try the new thing and see everything great. Yeah, let's mind-choke this. Yeah. So I come from that. And I think that's important because, as I said, I don't believe in hype. But when I saw it and played with it, I was just like, no, this is truly a platform shift. I also did a minor in AI back in the day, and so in undergrad. And I was like, this is hype. Yeah. So the issue is...

I mean, this is truly a platform shift. LLMs enable so many things that we didn't even think was possible. But, and this is the crux of what I believe the problem is, right? Like,

For LLMs to be the platform shift that they are, for Gen AI, for AI applications to truly change the way people live, work, behave, you need to connect them to the crown jewels of any business or any sort of user situation, right? And that is

operational data. And by operational data, I mean structured data. So I mean SQL, NoSQL, API-based information, right? Today, there is a huge chasm still between operational data on the one side and LLM and LLM-based apps on the other side. And that's precisely what Snow Leopard and I set out to do when I started Snow Leopard.

So it's, A, I 100% agree with this idea of

for AI to live up to the hype, we're going to need to plug into all of this data. And I don't think that's necessarily the most spicy take. Everyone is saying that and they're almost saying it like incessantly on all of these different social networks. We got to have, you want to have your own data and you want to have your own models. That's kind of the theory that we see. And I,

On the other hand, though, what I do think is fascinating here with what you're saying is we need to be able to connect it to data that right now we are not necessarily connecting it to. Unless it is, we've seen those use cases where it's like agents that are text-to-SQL agents or you have your data analyst and so you're using LLMs to query your database, that kind of thing. But I feel like what we...

are seeing more often than not is a whole new building of Gen AI apps that are off to the side of your structured data. So it's taking, it's connecting the Gen AI and the LLMs to your Notion or to your docs or trying to make Jira work with it. And it's not really thinking about connecting it to the databases or the APIs that you're talking about. Yes, that's right. So

you're right that everybody talks about your data should be ingested, right? And like we should connect it, et cetera. But I think we're exactly to your point where the focus has been, has been like chatting with your PDF, right? Yeah. And that's cool. It is something that was, you know, wasn't easily possible before this, but,

I think you mentioned something very important, which is that building apps to the side of your stack, of your critical path, is never going to really generate those new use cases, user value, business value, et cetera. And so you have to make it part of your critical path in order for it really to

to your point, live up to the hype, right? So today everything is around the fringes, right? One of the things that, you know, when I started talking to, so I validated the problem before I even started Snow Leopard. I know sort of counterintuitive, but just, that's just how I am. But when I started talking to people like VPs of platform or like head of AI at like all these companies, enterprise companies, SMBs, you know, people in my network, all of them were like, you know, we don't have a way, good way of,

of putting it in our sort of true stack, right? And there's a lot of talk, to your point, about how there's a new stack emerging. Like, anytime there's a new stack, like, that's outside of what the existing stack is, to be honest, right? And in my experience of 20 years of doing this, building data infrastructure and systems for enterprises, right, production-grade data systems, you have to, like...

Everything exists, like the tech stack exists in an ecosystem. It exists in a context. And if you want to derive value or create new value, whether it's for users or for business, right, you actually have to be part of that unique ecosystem. You have to fit into it. You can't just be like this cool new thing and adopt us here. But like the rest of your whole world is here. Yeah. Yeah.

I think that's really the key, right? Like, and we can get into this, but I do believe just one more thing here that while people say that, you know, you should connect your LLMs and your agents and your assistants to this data, I just fundamentally believe that the way that people are going about it isn't going to solve the problem.

It's funny because I instantly think of this visual of I live in Germany and I was driving down the street the other day. And what do you know? There's a caravan of motorcycles with sidecars on them. And it feels like what we're doing with Gen AI is the sidecar to the motorcycle, which is the production system. Yeah. Yes. And, um,

you know, I let's, let's get into the spicy takes. I have quite a few, but there we go. Um, but I think the way people do it today is, um, you know, rag, which is retrieval augmented generation, which the concept sounds good. Right. Um, there is the other thing which you're saying, which, you know, you train your own LLMs and you do all this stuff. Right. Yeah. Both those options actually don't get you to where you need to be. Right. So, um,

Spicy take number one is that doing ETL and putting all your data into a lake, house, ocean, whatever, isn't actually solving the problem. Spicy take number two is, and this is the more important spicy take, which is that

We're talking about intelligence will solve this. Agents will become self-aware, etc. And we'll go out and do stuff. And that's cool. I'm not against that. But the problem is the most intelligent machines today, human beings, don't make the right decision. Regardless of how smart they are, regardless of how cool their reasoning is, how good they are at reasoning, all of that stuff. The intelligence today...

doesn't know how to make good decisions with bad data. Bad data is equal to stale data also, by the way. Like if I ask you, if you've been in a coma for six months and I ask you, right, who is the president of the United States or who is the prime minister of another country? Until I give you that information, until you Google it, perplexity it, open AI it, whatever, you don't know that answer. You cannot make the decision.

right? So if intelligence today, best intelligence today can't make that, right, can't make right decisions without the right data at the right time, those words are important, right data, right place, right time, right? Until you do that, like, there is no way that artificial intelligence is going to be able to make the right decisions, right? Like, we just have to, reasoning will not solve it all is the point I'm trying to make. I know this is a

This is a counterculture take, especially for this audience. But I'm really honestly trying to help. I'm honestly trying to make that intelligence more intelligent, right? Like more useful. Yeah. And I don't necessarily think that it is so counterintuitive to say that reasoning is not going to do it. Because the thing that we've seen time and time again is the more context you provide to a model,

the higher likelihood that you're going to get to either the task being done or the answer that you're looking for. And so again, if you're not able to provide it the correct data in the form of the context,

Even if you are providing it a lot of data, but it's not the right stuff, then it's going to be a shitty answer or outcome in that regard. We talk a lot about hallucination. And yes, there are inherent reasons in the way the LLMs are built that can cause hallucination. I agree with that. But honestly, a lot of the hallucination is because you're not getting the right data. It's just trying to answer like human beings hallucinate. If we get the wrong information at the wrong time,

we will hallucinate, right? And the other thing I will say here is, you were saying, right, like, the more context you give it, there is an inflection point on which if you give it too much data, it will also come to the wrong conclusion, right? Because it doesn't know what to focus on, which is very intuitive as human beings, right? Like, if you throw a bunch of data at me, or information at me, and I don't know where to focus, I mean, I may pick up the wrong thing, right, and start going down the wrong rabbit hole. So,

Again, these are like very sort of, to your point, they're not sexy things, but they are very important, in my opinion, again, as a boring infrastructure person, to enable all the cool stuff to be built, right? Like we're still nibbling at the edges and until we like,

really like bring the right data from the right place at the right time to these systems, whether they're assistants, agents, the next big thing, right, in reasoning and AI. Like we're not going to really, we're only scratching the surface here. Yeah, it's true. We had Dane on here probably three months ago now, and she was talking about how

difficult it was to get an agent to say no or to say that it did not have the right information that it needed or that it did not understand the task that you are asking for and it plays right into what you're talking about with hallucinating because you ask it to do something and then if it doesn't fully understand or there's a fuzziness to the terms that you're using then it comes back with what

One interpretation that may or may not be what you are trying to get. And then you're like, this just hallucinated the shit out of the answer. And so you have to figure out a how to troubleshoot that. And what they did is they said, we are going to create a glossary of all of the terms that we're using. And we need to make sure that there's no vagueness.

in any of these terms. The definitions of these words and these key terms that we are creating, they can't be fuzzy. It can't be vague at all, which when a human reads it, they think, oh, that's fine. And then when you go and you try and evaluate the output on it, you see, oh, yeah, I guess it didn't really understand what I meant by

create a good summary. It's like, what is good? That's right. Subjective. Right. And I, like, I think there's two things here that I, I believe are key, right? One is like, imagine, like, just think about how much work they had to do and it still wasn't producing the right answers, right? Because they are inherently sort of, so, so here's the other reason that I think what we're doing

is interesting to me because like LLMs are actually really good at the sort of fuzzy interpretation of stuff, right? Like they're actually really good at summarization, like, you know, classification, those kinds of things that inherently require a little bit of like extrapolation, if you will. And, but they're not good at point lookup, point solutions, specific yes, no answers in all cases, right? Now, yes, you can, you can

tune them you can do reinforcement learning and they'll they'll get closer and closer and closer to what you wanted to do right but but not everybody has the time money expertise to do that first of all and secondly they're actually really good if you just give them again i'm going to keep going back to this the right information for them to make the the decision right and this is where i think i think honestly llm get a bad rap in some way because

Because ultimately everybody's using RAG to do it. And the way RAG is done today is ETL a bunch of data into a vector store, and then use the vector store to build context at query time. But vector stores themselves are fuzzy matching systems. They are not point lookup systems. They are not precise answer systems.

They're meant to do what they're doing, which is be fuzzy matching, right? Like, whether you spell Demetrius with an S or a Z, they're going to come up with the same... Like, they may come up with both answers because they, you know, to them, they're like, oh, these are similar. You wanted similarity search. Like, here, these two are similar. Now you figure it out, right? And so, like, we're taking...

First of all, all the information that truly can help make this happen is, you know, the crown jewels, as I keep saying, are in your databases, data warehouses, in your, like, Salesforce HubSpot, those kinds of API-based systems, right? Which, if you extract them, you change their nature, you lose business context, and you put it in this vector store, right?

So now they've actually lost all the structure, the information that was very carefully designed to be in those systems. Like you've completely lost it, right? It's in some blob, in some essential fuzzy matching system.

And then you pick it up from there. So not only is it stale, not only does it suffer from all the ETL and reverse ETL problems, it also suffers from you took it out from a context in which it exists. You put it in a different context with a bunch of other data. And then you serve it to the LLM and then you're like, okay, the definition of good aside, it won't just know whether and how to answer the question. So I think...

There is something to be said for fine tuning around, like, give me a yes or no answer. But, you know, the way the models are going, if you just tell them not to like, like, if you don't know, just tell me, don't know. Like they will tell you that today. Like the models are getting smarter in this regard. Right. So then what you need to do is no matter, again, no matter how smart they get, like without them knowing what you're asking about, they can't give you the answer. And yeah.

I do like this almost. I look at it as compression loss when you're taking it, you're ETLing it and doing all of this stuff. And I also it makes me think about something that I've been pondering for a while, which is you don't hear as much about rag these days because agents are all the rage. Yes. The next wave of a vibe. Yeah.

Of hype, exactly. And you just connect them to your MCP server and you're good. And we're all happy about the outcome then. It changes the world and the way we work. But the thing that I am constantly going back to when it comes to RAG or when it comes to how we are doing things and we have been doing things with LLMs to try and create products that are valuable products.

With them and new products, inherently new products. It's not like we're grabbing an LLM and sticking it into a fraud detection system that we have traditionally used ML for. And with RAG, I always kind of laughed because I would think, are we just...

overengineering the shit out of this. We just straight up, we're going and we're creating all of these different hacks. And so in the beginning, it's naive rag, and then it goes to advanced rag. And then we're like, no, we got to do graph rag because at each stage, it feels to me like we get to the stage, we think this is going to help us create

create a better system and have more reliability, have more consistency in the output. And then at some point you see the...

outcome and you recognize i mean it's good yeah it's good but is it what we really need is it well maybe let's try the new technique maybe there's a better technique out of there but and or maybe we can engineer this a little bit more to try to get better results and it's it's inherent in

because of the stage that the models were at. I do think that with the new reasoning models, potentially we're going to see these huge leaps due to that. But it always makes me laugh like, oh, it feels like we're having to do a lot of

bending over backwards to make this actually work. And I also would laugh because all of that work that we're doing, all that bending over backwards to create some chatbot that can tell you about your company's HR policy is not worth it. And that is very clear. I am so happy that you brought this up because it's,

happy but just like it just resonates with me that's that's sort of what I mean because yeah I I have been in that world for so long forget AI right like just this world of like

Oh, we need this data now. So now we're going to create this complex pipeline to go from point A to point B. But oh, we forgot about this N plus one silo. Right. And now we have to like create a whole new sort of pipeline. Right. And like a whole new way to do ETL, et cetera. And that takes months, first of all, if not many months, it takes a few months for sure.

And then, oh, by the way, we changed the dashboard or the use case or now in the AI world, the question a little bit, which requires a mobilization of entire teams to like rejig the whole thing. Like, you're right. I'm so sad to see these like complicated things.

wires, pipes going everywhere. Like if you look at a data architecture, like a real data architecture diagram, not one put forth like to present some kind of decision maker. Like it's like this, it's squiggly lines. I know there's a podcast you can't see me like gesticulating like crazy, but

It's squiggly lines and sometimes you can't even tell why the line exists, where it goes to, what's going on, right? It's just so over-engineered, which honestly, my heart goes out to the developers that are like trying to maintain these systems and build them and like enhance them, right? Because you need an answer to a new report instantaneously. And the way these systems are crafted, precisely for many of the reasons that you called out, right? Like the models were not as good.

The data systems were not as good. They're not as scalable. They're not as like sort of any to any connections. So, yeah, what if we just took an eraser? That's literally what I tried to do when I was initially coming up with the architecture for SnowRiver. I was just like, erase, just erase. What if you could put a box, right, in between all of your data systems on one side, all of the data, like databases, data warehouses, whatever, data systems, and your LLM systems on the other side?

and just have straight lines. Oh, you need data from Salesforce? We'll go get it for you. You need data from Postgres? Oh, we'll just go get it from you. You need data from Snowflake or Databricks? We'll just go get it for you, right? Like, no need for this craziness because...

In my experience, again, having seen Google, Oracle, even at Observable, just looking at how people are building applications on top of these stacks, they spend, developers, data analysts, business analysts, all of that, the whole data engineers, are spending upwards of 70% to 80% of their time just doing this part, which means you're not spending as much time building those applications

new value creation applications new business creation new like you know user value opportunities because you're spending all your time doing this so what if you just didn't have to do that and you could spend time like coming up with the new creative ways of doing stuff

I was talking to a data engineering friend of mine a few weeks ago, and he mentioned the predicament that he is in right now, which is for the last three years, their company made this big push to self-serve analytics. And so that was incredible, you know, the whole digital transformation thing. And he's a data engineer, and so he said, so now I've come into this company, and we've got

over 11,000 data assets that the engineering team has to keep and or has to service. So he's saying we're kind of rethinking this whole self-serve thing and wondering what can we blow up or what can we take the eraser to and what actually is valuable or it

crosses a certain threshold of value and is still being used because a lot of these are built by someone, that person leaves, then there's no knowledge sharing on how those dashboards are actually created. It breaks one day and boom, that person's out of there and you don't really get it again. And why would you if

It's self-serve, so it's almost like, well, I didn't really even like that dashboard that much, so I'm going to create a new one. And I have a better way of doing it, right? Of course. And so you get into those scenarios. I do wonder, though, in this world that you're mapping out, where you have the abstraction on top of all of the data sources, what would keep it from also being in that sprawl? Yeah.

Because the developers are not building that sprawl, right? Like, I think there's a couple of things here. One is we're not saying it's magic, but we're saying instead of each developer and data engineer team, et cetera, doing this over and over again, like we just, we do it in a generic way for them so that they're not having to engineer pipelines at all or maintain them or like build them at all. So what we're saying actually, so this is again, I think a spicy take maybe,

I just think it's the, like, you have to imagine the world differently and then see if you can get there in a way, right? Because a lot of this sprawling situation comes from, frankly, the data world I've been in and, like, you know, data grew faster than the systems that could support it, right? So then there was all this, like, put it in a lake, then put it in an ocean, put it in a house, right? All those things happened. And

to be honest with you, like ETL is helpful, especially in the sort of business analytics, like historical trend analysis, those kinds of things. But now that people had a place to do that, right? Like round peg, square hole, everything goes in there and everything needs to get out of there, right? Like that's one way to go about it, which causes this like,

Do we have this like end silo? Have we put it and put it into this lake house ocean? Like if we haven't, then we have to go build it. Right. And then to your point, like there are these all these random dashboards. But what we're trying to say is specifically what I'm trying to do here is what if you didn't have to move the data? What if you could just go get the data from the source directly when you ask a question?

So there are no pipelines. It's not that we are building the pipelines. There are no pipelines. There is just connection to the data source and you go fetch data from there when you need it. It's kind of like when I ask you a question and you go look up

your favorite search engine and find the answer. You're not keeping all that data, downloading it, like sifting through it, filtering. None of that is happening. You're just saying, hey, like, you know, what's my date of birth? Oh, go to the database that you like, you know, some social security database or something and go get it. Yeah. So how do you deal with things such as, OK, I need

a lot of data points on I need to know how many customers did xyz but in the source data

You need to apply a few different kinds of transformations on top of that to get that answer. I mean, that's a classic data warehousing problem. And so those data warehouses are already built. I am just saying you don't need to build new pipelines and new data sources and new connections. So if you're doing something that is important to your business, which is where the original data warehousing concept comes from, we can go fetch it from that data warehouse.

But what we can also do, and this is what's done through complex software or complex pipelining today, we can also do things like, or we should be able to do things like, hey, you know, for example, if you ask me,

I say this often. I don't care about Terminator happening, right? I care about where my order is if I ordered something online, for example, right? Oh, speaking of which, I ordered some fucking coffee the other day. Turns out, for some reason, it was in Google, my address for my old house in Spain. So now somebody in Spain has got some nice-ass coffee. I can't say it.

That's hilarious. By like the 10th day, I'm like, where is that coffee? I could really use it right now. But anyway, sorry, I digress. But that's a really good example. Like somewhere, something should have updated your data. Yeah. Like it didn't happen. Right? Because somebody went somewhere failed ultimately. But the point is like we're saying what if you had to, for you to look up your order, like just ask, right, your favorite assistant, where is my coffee?

So, coffee, it needs to do two things. You need to look up where your coffee order was made, right? Yeah. Whether it was an espresso or some other fancy stuff, right? And then where was it shipped, which means now you're looking at basically a Postgres database or some sort of CRM system, right? And then you're also looking at your tracking system, like UPS or something like that. That means you were supposed to do a join.

between two data sources that are outside. So databases, data warehouses, data lakes are all great at joining within. And if you need to join two sources today, you have to dump the data in one place so that they can do a cross-door join. But what I'm saying is what you actually need to do, what a human would do is look up your tracking number, basically, from your order management system and then look up the tracking history in the delivery system.

Yep. Snow Leopard can do that. Like the aim with Snow Leopard is you can pull from both of those places and give you an answer. So you don't have to build any pipeline. You don't have to like create any new way of answering this question if you're building an assistant or an agentic flow. You just asked Snow Leopard the question and Snow Leopard said, oh, I need to. So it's doing intelligent routing. It's saying, oh, I need to go to these two different sources. Right. And then it's doing intelligent routing.

Query building, which is like, oh, for UPS, I need to write this kind of query because it's an API and for...

you know, Salesforce, I need to write a SoCal query and for Postgres, I need to like a Postgres SQL query, right? Which is different from a Snowflake SQL query, which is different from a database query, a Databricks query. So it built that in real time. Then it went and just fetched directly from those sources. So that means the data is fresh, which means if, you know, if the coffee was delivered to Spain in the last half an hour, you should be able to get that information instead of it's still in transit. Then you go back and look at it next day because, you know,

Yeah. Data dumping is stale. That's sort of what we're, that's the world we are talking about. That's the world we're imagining. And that's sort of where, you know, I've talked to recently to a C2 at Salesforce. I talked to like, you know, head of data and AI in Asia for P&G. Like the stuff you were talking about, they're dealing with that same stuff, right? Like 8,000 silos of data.

Which dashboard is powering what, right? Like, let's just pare it down because we need to figure out how to maintain it, right? And like our data engineers are drowning in that, our data scientists are drowning in all of that. So, yeah.

Yeah, and we didn't even touch on the data quality part where it's like, oh, something changed upstream and now the data is absolutely worthless because of the way that these dashboards have been built. One thing that I want to know about this vision of how things work, do dashboards still play a part in this world? Or is it just questions? And I have to know what kind of question I'm asking now.

before I can get the answer? Because I think sometimes when I look at a dashboard, it provokes questions inside of me. I think this is a very interesting philosophical question. And I think there's different camps on this. Again, as an infrastructure person, I don't like predicting the future. But my work at Observable, which is a data visualization platform, we are all visual people. So

Right. Like data points on a graph is much easier to grok than like a thousand data points. Like we just know that. So are dashboards going away? No, I don't think so. I think data visualization will always exist because the easiest way for humans to grok information. But what if you could augment that with asking all the questions that came up?

right in your mind when you looked at a dashboard could be answered instantaneously because you could go get the data if the data exists right so you didn't have to i will give you an example when i was looking at churn uh for my products at google right i have to first because first of all i didn't have i didn't have the right dashboards i had to build the dashboard so i came up with sort of a solution for what kind of data we need etc then i had to talk to my

business analyst friend who was, by the way, stretched thin because he was actually looking after 10 different products. I had to cue into his thing, into his workflow, like, hey, I want to build this churn dashboard, needs to pick data from here, here, here. Can you build me that? Now, once he built it for me, like three weeks later, I looked at it and I'm like, oh, I actually need to know like

churn by geographies, the churn in Southeast Asia, churn in the U.S., churn in Canada markets. In fact, one level deeper churn in, you know, industry as well.

So each time that answer question came up, it was a few weeks to get that answer. Either he had to build that into the data warehouse or he had to take the time to pull it out and give it to me. But what if I could just, once you have the dashboard, what if I could just chat with, let's say it was BigQuery, where all the data existed. I could just chat with my BigQuery and I would also be able to join it to all the Salesforce data that had

the latest customer information? What if I could just do that and the system would take care of that for me? Right. So we're talking about agentic flows here, too, by the way. Right. Like agentic flows have the same problem. And spicy take number three is while MCP is amazing and has it's a great start, open source start to the to the connector problem.

It doesn't solve it. Right. It also I believe it has all the same problems that previous generations of solutions have had, which is that it's not really tackling the hardest part of the problem. And the hardest part of the problem always is intelligent routing.

And more importantly, business logic. Now, I mentioned before we had Doneon here and she had built a data analyst agent. And the way that they did it to keep the agent in line, we could say, is that...

They had different Slack channels for different agents with access to certain databases. So you would have a marketing Slack channel that has access to the marketing databases and the marketing type of queries were happening in there. And so that was almost the way that they were able to hard code. Yes.

hey, agent, you speak this dialect of SQL. You have access to this type of database right here. And here is your glossary, as we mentioned before. So if any of these key terms come up, you know what they are and you know where to get them and grab that information. And so they were able to do cool stuff and the

The agent could then go and create SQL statements and pretty much do a lot of the menial work, we could say. I think I remember them saying it was like a barbell. The majority of the questions that were asked were those questions that are not super complex, but they're not something that someone who has no idea of data can

and is not a data analyst would be asking. It's like those kind of middle of the road questions. And anyway, this whole story is because I would love to know how are you making sure that the agents or your system speaks the right language to the right database? Yes, that is a wonderful question.

In fact, everything you just described is sort of how people are trying to do it. The hard coding piece is so like that's just how it's done today. You have to hard code and you have to separate out because otherwise mixing questions that go to MySQL dialect versus going to Snowflake dialect, even though they're all SQL and they all follow ANSI SQL standards.

will confuse the heck out of everything, the system, entire system, including LLM, right? This is actually why text to SQL doesn't work either, right? Apart from the fact that it hallucinates and all that stuff, it's just which dialect and what is allowed in which dialect. And this works better for open source databases because the LLMs have gone through that. But for closed source databases, it's like a nightmare. I know because we have been doing it, right? So what we're doing is

We're using known data infrastructure techniques to do things that are essentially systems problems. We are not trying to shove systems problems into the reasoning AI predictive world. As I said earlier, LRMs are great at classification, at summarization, and those types of tasks. Deterministic systems are great at

if you code them the right way. They're great at point lookups and being able to do precise and deterministic work. When you are building a SQL query for the right dialect, you are doing precise and deterministic work. So we are using essentially data retrieval techniques, not in IR, but data retrieval techniques. So we are building

SQL, for example, if you're talking about SQL, right? Like we're building the SQL for the right dialect, right? Based on where that query needs to go to. And we're doing it in a deterministic way, which means we're using like data or sort of query building techniques. Query building techniques exist. Like databases use query builders, right? Data warehouses use query builders. A bunch of data retrieval systems use query builders. We are doing it

using those techniques and like using existing sort of methodology to like, for example, ORMs are great at doing this, right? So we use sort of that existing methodology for building like a PostgreSQL query versus a Snowflake query. We're doing this today. We have a design partner that has data in sort of

one of the proprietary data warehouses and we just build like queries for that right on the fly right so where do you need to do sort of that intelligence in retrieval like oh you know based on this question and based on my understanding of business logic we know that the query needs to go to um postgres and also there's a separate query that needs to go to let's say bigquery

Interesting. We build both of those. Once we know that, right, then the rest of it is deterministic, right? Like the rest of it is, oh, build a PostgreSQL query, build a BigQuery SQL query. So we do that. And that is just sort of like we're using, this is what I think is exciting, is we're using data system and deterministic programming techniques for that piece. And we're using

you know, AI and agentic work for the sort of intelligence in the routing and intelligence and what data we need to fetch. The understanding. So if I'm understanding this correctly, it is AI gets to a certain point, it takes the query and then it says, all right, cool.

To answer that, I'm going to need to use these tools. And then boom, when it fires it off to the tools, then it steps back. It's not actually creating the SQL. And I think that is a really good way of doing it because you're taking a lot of the responsibility off of the LLM. And anyone who's dealt with LLMs know that the less responsibility you give them, the better.

unless you do something where it's very siloed and it is only you as I was saying donate created where you're in the marketing channel I know this agent only has access to the marketing database it's going directly there they opted to let the LLM do the SQL query but they also have a lot of

calls to the LLM to like double check the query and do this so it gets back to that hackiness a little bit like all right cool yeah we do that and then when something gets returned they're also double checking it there to make sure hey judging by this question is this SQL query and the data retrieved does it make sense and so you'll have like a critiquing agent that is is double checking all of that but in your case you're saying

the LLM is only used to understand the query and then fire off the different tools that it needs. Right. And that's sort of like how you would describe it in the sort of, you know, these days, the MCP agentic world, right? But we're not just firing off a tool call. We're taking responsibility for how that tool is created and the accuracy of that tool as well, right? So when we ran our benchmarks, for example, internally,

We did it with an older sort of rag-ish system, which is how most of these things work. Like we had a 99, 98% accuracy. This is without us doing any fine tuning of our own intelligence. And the system we were comparing us had like 6% accuracy for this type of SQL-based lookup, right? Because either it doesn't know how to build the right SQL or it's not getting the right data or it's hallucinating. All of those things just don't.

like, even what you would do. But that aside, I think back to your use case that Donae was building, right? What we are doing for sort of our design partner here is like that Slack channel can actually be used by sales and marketing people, right? What if you could have that? Because

Turns out there's a lot of overlap in the kinds of questions that sales and marketing people have around, you know, the pipeline piece and all the way to the sale and post-sale piece, right? Around customer data and around like all that kind of stuff. So if both of them can ask questions, then we didn't have to. And you didn't have to hard code anything, right? Then you actually...

open up this whole new world where they could probably even talk to each other and come up with better campaigns and better end-to-end workflows for making a sale happen, right? So I think what I'm imagining is in this world of ad hoc questioning and ad hoc information needs, why not make the information retrieval ad hoc? Why not make the data retrieval ad hoc, right?

So what we're doing is we're having like a natural, you know, there's in the data world, right? There's all this like single API to fetch everything, which has never worked because, right? You just talked about one of the many problems, which is which SQL do you build at the end of this single API? But what if the API is natural language, which is what it's become, but then it translates to the precise, like other end, what you're calling tools, right? Like it translates to the precise dialect that those tools talk. So the tool is not doing any guessing, right?

right? The API is not doing any guessing, right? You use the intelligence for what needs to be used for, the understanding and the, oh, I think it needs to go here and do this. And then you leave the rest of it to the parts that know how to do it, which again leads to better accuracy. I didn't quite understand. How are you differentiating on that tool call part and

taking ownership for the outcome that, all right, we're going into whatever database, whether it be Airtable or your CRM of choice, and we're ensuring that it's going to gather that data properly. Or I think you said something like we authenticate it or we authorize that it will be correct. Oh, yeah. We're just taking responsibility for the accuracy, I think. Right. So

So Snow Leopard is a three-part sort of workflow. The first part, to be more, let's dig into it, right? The first part is just what you said. You have a question, we do natural language understanding. No rocket science here, right? Or mostly AI here. And then the next part is we are combining data infrastructure logic, your business logic, to figure out which API call needs to be made. This is our proprietary sort of way of doing things.

Or rather, this is the intelligence we're building, right? And then once we figure out, or it needs to go fetch user ID, you know, in your case, for example, it needs to fetch your name, your address, and the order number, right, from one place, right, which is your, let's say, CRM system. And it needs to fetch the order information and shipping information from the delivery system, right?

Right. Once we know those two things, this is what the intelligence is telling us. Then building the right query, like we're not doing text to SQL, right? So there is no fuzziness there. We're literally saying build the SQL statement, which is like it's a known thing, like build the SQL statement directly for this particular dialect and we go build it.

Right. So we're basically using like regular programming, like systems programming to do that. And then we go run it on that system. So, you know, if you run a PG SQL query on a PG SQL database, it will give you the information. Right. That's known. Yeah. And then, yes, we can run it through an evaluation system and all that stuff. Right. Which is, again, these are known things that people have done.

But there is no like, oh, did it get the right data? We had this column, but does that column exist? I don't know. That's what you do when you do text-to-SQL or sort of these agentic flows. The tools have to... You hand off a lot of responsibility to the tool and the tool developer has to take on the responsibility. And you don't know who the tool developer is, right? Number one. Number two, different tool developers. So MCP servers for Postgres can be built by anyone.

Right. And so because there's no like you have to build it this way, MCP, the protocol and MCP, the sort of open source framework doesn't tell you that you have to do it in this way, which means different people can interpret that framework and build different MCP servers.

Right. So either you, just like with any other open source software, either you take somebody's MCPC server, then you like harden it, you test it, you fix things in it the way you want them to be fixed. Right. Or you just rely on them to do the right thing, which anybody that's been in the open source world, like

Right? It's just the last 20% is sort of where all the 80% of the work goes, right? In hardening it. And the one additional thing here is that we are taking the sort of responsibility of understanding business logic that you tell us, right? Which the NCP server is just a connector, right? Yeah. So it's just going to do, if you give me

Like a SQL query, I'll go run it on Postgres. But like which SQL query to run? Is that the right SQL query to run? Like, does it understand whether column order ID with a capital O is different from column order number with a small O? Like it goes down to that level of confusion and complexity, right? This is what we're trying to do. Yeah, the downfall of the MCP servers are very much that. And I appreciate you calling that out real fast because it is very...

Easy to put up a MCP server. And that's like the blessing and a curse. Yes, 100%. That's probably why it is so incredibly popular right now because folks can stand up an MCP server in a few hours. That's right. But as you said, if you don't know and really know intimately the ins and outs of your Postgres database, you throw up a server with a lot of tools

And those are your opinions on how the tools work. You could get yourself into trouble. Or if I come and I grab your MCP server and I think, all right, cool, it's a Postgres MCP server. I'm going to just...

throw this into my post-grad instance then yeah there's there's a little bit of wire crossing that can happen yeah and i i agree with you like i think it is a blessing on the course right like i i'm not trying to poo-poo on it i'm just trying to point out because people attach themselves emotionally to the next big thing to the next hype and then they get super disappointed right and i and i

I want people to just understand and go wide open, right? Like, it's actually great for Snow Leopard if there's MCP service because if one gets popular, it actually helps us because we don't have to build a connector then. So it accelerates, you know, user delivery for us, honestly, right? But I just want people to, like, know that there are sort of, like,

you know, there are blind sides to this that you want to be aware of. You want to go in eyes wide open. Like it's fun to play with, but in my experience, building like those high availability, high reliability production systems requires a lot more than, you know, putting something together in a few hours, right? Like that's not the hard part. The hard part is, again, going from POC to production, right? Like this is still a problem, I think, with

AI systems from the, you know, engineering leaders, CTOs that I talked to that, you know, we built this cool thing. We couldn't put it in production because reliability and accuracy and, you know, performance isn't even a thing right now. It's just reliability and accuracy. Like I needed to answer the question when I needed to answer the question in a reasonably accurate way.

Yeah, we're okay with waiting 15 minutes to get this answer back as long as it's the right answer. And so, well, anything else you want to touch on before we go? I just think it's a really exciting world that we live in right now. I think the fact that there is MCP and the fact that there is all this discussion around tools and things like that means that people are finally understanding

really starting to understand the thing that I was saying. I went blue in the face. If you look at my LinkedIn from last year going like, but you need your structured data systems. You need them. And so I'm personally really excited that people are starting to notice this and starting to pay attention to this problem because that is what I believe will cause the true wave of AI to make people's lives better. But yeah, I'm really excited to talk to the people who are facing this problem, who are

I want to, you know, even Danae, for example, it's been really exciting to hear from you about what, you know, they've been doing there because I just want to understand honestly how people are tackling this. Do they have this problem? How are they tackling it? Right. Can we help? Like, it's a really exciting time to be in this space. And I keep saying the last thing I'll say here is,

You know, I've been telling my systems friends like, hey, come over on the AI side. It's OK. I know it's non-deterministic and I know you hate that. Like, oh, hell no. Come on. It's fun out here. And I really I feel like some people are sitting up to pay attention, which I'm which I'm excited about because I want the AI world and the systems world to come together because that's how we're going to we're going to solve these problems.

Yeah, I'll have to introduce you to Donay. And for anybody that wants to check it out, it's snowleopard.ai. And I love what you're doing. I think it is super cool. And I'm excited to see how it progresses and live in a world where you have infinite connectors to all of these different databases. So I can, I actually have a use case right now that I'm thinking about where I was asking myself so many different questions yesterday. And I

I have to go and gather the data between Airtable and Salesforce. And I am really bad at Salesforce. And so it took me way longer than it should have. And then you get into...

oh, I don't know if I have access to that data and that's not going to be solved by Snow Leopard, I'm sure. But I have to go and talk to somebody and say, hey, I need this for my report, blah, blah, blah. So yeah, data fun. Data fun indeed. I'm excited for a world where Snow Leopard can hopefully get rid of the majority of that pain. Yeah, thank you. It was so fun to talk to you, Dimitrios. Thank you so much for having me. And yeah, this is the kind of thing that I think people should be talking about. It's very exciting. So much more stuff.