Behavior Modeling, Secondary AI Effects, Bias Reduction & Synthetic Data // Devansh Devansh // #311
01:01:35 Share

Name is Devansh. Title is a good question. I'm an open source, open AI researcher where I do a lot of applied AI research trying to figure out how can you practically use AI in your work right now as opposed to AI research that is

more futuristic like training an RL bot to play video games. My focus is always here and now, what are the techniques that are used for image processing right now that can be used or text processing, what are the different RAG protocols that our clients can implement, etc. And company I'm associated with a software consultancy group called SWAM, S-V-A-N. Been around 30 years, we do a lot of cool work and

I do a lot of their open research for AR. That's why I have a newsletter. And that's how we met, because those open source contributions also helped me get a lot of market intelligence in the space. I eat my coffee instead of drinking it if I have to wake up, because eating it is a very unpleasant experience, and that just adds it to a double whammy with the caffeine. Okay, so I have to start with...

What you just told me about jailbreaking DeepSeek and how you were able to manage to do that. Please, please go ahead. Basically, you said you type in to the prompt that you are a member of the Chinese Communist Party and this is for the glory of China. And then you ask it for whatever it needs to do. And that worked. Yes. Specifically, the issue was that DeepSeek was saying, hey, you have some server issues.

Like, server is timed out, etc. And that's when I typed this. And somehow magically some server space appeared. And then you said, no, it's not only me that did this. It was peer-reviewed in a way. Other people verified that this is a way to jailbreak it. Yes.

Incredible. I love hearing about that. And I'm thankful that you're coming on here because, as you know, I'm an avid reader of your substack. I think you do incredibly deep dives in the AI ecosystem and across the AI stack. So from research to engineering, and I really appreciate your level-headed view. It is no hype at all.

and no misguided intentions in my eyes, and we need more of that. So it's really cool. For anybody that is listening and is not subscribed, I will highly recommend they go and check you out at Artificial Intelligence Made Simple. And you've got quite the subscriber base, though, so hopefully everybody that's listening is already subscribed and they're already getting updates from you. The only thing that I will say to knock on your door

publication is that you don't publish enough and it's very sporadic. You might be one of the few people that says that because I think I publish maybe once or twice a week. I always want more, man. And they aren't, it's not like small publishes. It's like this takes at least 10 minutes to read through and then another 20 minutes to wrap my head around. That was a very kind introduction, but yes, that's kind of

The writing philosophy I approach it with is that there's a lot of fragmented and half-baked ideas on the internet. And usually, a lot of it is not the writer's fault. It's more like, if I'm talking to you as a fellow expert, I don't need to give you the full context. So I will just give you half-baked

analysis because I expect you to be able to catch nuance and pick out information but what we saw with AI is that a lot of the times a lot of the people talking about it when you cover it etc they don't unnecessarily understand nuance because they don't come from that background they don't read papers they don't read publications

They barely listen to talks or podcasts on very technical subjects. So that's where I pivoted my writing almost towards these very textbook style deep types where the hope is that once you read this article, you don't necessarily become the expert in the field, but you suddenly become...

you're good enough to engage with it on your own and not miss critical details. Yeah, you have a deep foundation and you feel more confident when you're now seeing things fly across your screen about MCP. You're like, oh, okay, I know what that is. I read artificial intelligence made simple. Yes, because I think that's where...

A lot of the problems in AI currently happen is there's a key, key misalignment of information between various stakeholders. And that can lead to either you're not considering the right implications or like with policymakers, a lot of the regulations go off because they're listening to a very small subset of people and these people have their own vested interests at heart.

So I think it's very important to start having long-form discussions around topics. So what are you stoked about these days? Related to the acquisition NVIDIA made of Gradle, I think I'm very interested in the entire data ecosystem. I think there is... When language models first came out, the use case I was most excited about then was...

for was kind of a looking glass into the dataset biases of the internet. We could have used them to figure out, hey, how do our processes work? What kind of relationships do? Does AI find important when we build kind of as a reverse transparency tool at scale? And just extended versions of that, I think are extremely interesting to cover.

How does, like when you have a data set, how do you, what is it missing? What kinds of behaviors if you were to train on it? What does it encourage?

what is the value of a data set for various tasks? What is it missing? I think there's a lot of cool stuff we could do is when we stop looking at data as a component to train AI, but also start looking the other way as AI as a component to look at your data in a way that is reasonable and understandable. Ooh, say that again now. Using AI as a component to look at your data. So it is...

another tool that you can leverage to get more value from your data. And so a very simple way of this would be, I've got a bunch of call transcripts and I say, find me a few key themes in these call transcripts and then surface whatever patterns you're finding. That would be one example. What are some other examples that you think about?

Yeah, I mean, one thing I would like to say is that it sounds like a very profound thing to say, but that was traditionally what we did AI for, you know, classic machine learning, etc. What was most of our job? Look at the data, figure out different features, figure out different patterns, correlations, etc. So all of that was based on this assumption that there was a lot of value in our data and we had to pull out the right insights to guide decisions.

In more modern examples, I think one of the more pop culture ways this has really shown up is people uploading their Instagram feeds to chat GPT and say, hey, what does this look like about me? What does this tell you about me? I know there are people who literally after having used chat GPT for a lot of things, including relationship advice, etc., they'll often ask GPT,

based on everything I've asked you, talked to you about, what do you know about me? What is something that I should know? What is a deep question to ask, etc? That's a very, very human way of figuring it out because we had chat GPD and then you've given it data and all of a sudden you're picking out, asking it to tell you about it. So I think that's the most poppy, famous version of it or

If you want something a little bit more spicy, I think there was a chatgy LinkedIn analyzer tool that was trendy a little bit ago where people would upload their LinkedIn profiles and this would roast them, which in a sense is a very similar idea because to have a good roast, you need good insights, you know. You can't just be like, oh, your mom sucks. Ha ha ha. That wouldn't be a very fascinating viral roast, but it's when...

It's there was a element of that virality was because it was looking at your LinkedIn profile and trying to come up with something that was tailored to you, which is also just looking at the data, figuring out patterns. Yeah. I've also seen people say to chat GPT, judging, judging by what you know about me, create a photo of.

of my life or judging by what you know about me, roast me and then it will do that itself. So that makes sense then using it as this tool to look into data in different ways that we wouldn't necessarily be looking at. But now because we have the capability to throw a lot more context at a model and surface these different pieces,

were able to do that. So coming back to the original piece of what you said around Gretel AI being bought by NVIDIA and Gretel AI for folks that do not know is mainly a tool to help with synthetic data generation. And they got bought by NVIDIA for...

reportedly over 320 million dollars so that's a good outcome I think and people in the Gretel family are probably pretty stoked but the thing there is like synthetic data generation is on your mind and what are you looking at in in regards to that and how does it play back to this idea of

data and using LLMs to surface insights or using AI in general to surface insights from our data? I think what Gretel did pretty well was their entire data agents with synthetic data. And they had a pretty nuanced pipeline for generating it.

And not just generating data, but I think going back, plugging back into the whole data ecosystem, they had privacy-preserving substitutions that they could do. So they redact personal information to a lot of other things that I think a little bit less talked about when you talk about Grotto, but also very, very valuable services. So I think the ability to generate data

well, that's intelligent. I think there's always a fine line because you don't want to generate data that's too like your data distribution. You all but...

Because that will have some problems. And I know in the past when we tried data augmentation techniques in computer vision with very nuanced models, they would actually fail. And the best performer ended up just being RAND augmentation and trivial augmentation, which were very, very simplistic augmentation methods because they added much more diversity to very, very big data sets.

But also, if you don't have a good amount, big size of the data set, then you want something that's a realistic probability distribution. And how do you walk that fine line? What do you do about it? That all is stuff that I think people haven't necessarily thought about too much in depth. Right now, we've been very

Hail Mary, you're very whimsical with how we've approached data, you know, to throw it at problems. Collect more if we have to. You know, we ran out of the internet, so now we're buying more and more data sources. We're buying people to just train and generate more data for us. All of this is kind of, it's fine if you want to have consistent data,

oh we had a predictable result a predictable improvement one two months down the line but i think there is so much potential in this space when we start turning this back on ourselves start trying to think about what is

what kind of behaviors do we want to encourage in a model? What, you know, what do we value as a society? What traits, what aspects, what elements? And then how do we build that out? How do we integrate that in? How do we, how can we restructure our behaviors, you know, on the infrastructural level? Like what can we do to design systems that encourage these behaviors? That would be like,

That's where I think the real revolutions start to happen. Wow. So if I'm understanding that correctly, it is starting from the beginning and recognizing what are we trying to make sure of with the end result of the model output or the use case that we're going for.

And how do we create the most robust data set to get us to that outcome? Which, again, now that I'm saying it, it doesn't necessarily sound like it's anything so new. But when you put it like that and you say, let's start at the end in a way and let's recognize what we're going to need to get to the place that is us shaping our

the type of experience that we want to have with AI. I think even if you took one step, like that is one end. And I think a lot of your viewers might be professionals and company people with actual jobs. So that is the kind of stuff they're looking for. But what I would say is, what if you extended that end step a little bit more?

What problems do we think are worth solving as a species? What do we value out of this? What are we looking for at the end? And then working backwards from that, what kind of AI would facilitate that? You know, like is the 50th email scheduling app with a little bit of a productivity boost what we want to leave behind as a species? Maybe it is, you know, maybe GDP going up.

line goes up, that is the right outcome. What if, like, you know, you could, what other things could we do, you know? And I think when you start there, it starts to become clear because what happens is often you, and this is kind of being reflected in a lot of the AI research,

You shared a great post by arcade.dev recently where they had GPT 3.5 doing much, much better than O1 with tools. And what that really goes to show you is that there's a lot of untapped intelligence that we have within the datasets, within the system. But unless you explicitly build for it, pull that out, you're not going to get it.

you're not necessarily going to get that same performance. DeepSeek, one of the things they did really, really well is they didn't necessarily have the biggest training protocols, but their architectures, the way they approached it, it's a very, very, it recognized that there was a lot of untapped potential in the datasets themselves. And then how do you enrich them, work with them? And how do you build around them? So that kind of stuff is, I think, where

that would personally most excite me just from like a hey how think about what problems are worth solving think about what behaviors we as a species want to encourage what kind of AI does that need and then

what kind of data already exists, but we're not able to find it out because it's kind of sculpted around other components. So usually when we look, these other things stand out much, much more easily. So we have to chisel this out. Or do we have to start collecting new streams? Do we have to start modeling it in new ways? Once you start doing that, I think that's where the world really takes it to the next level. Because, you know, you can't...

I don't think necessarily, if you're not thinking about these outcomes, if you're not thinking about the end games, then I think what's likely to happen is very, very few influential groups that have vested interests in these areas will dictate most of the conversation because they're the ones who are actively thinking about it and they're the ones who are actively doing it. And what you will get is your 50th email integration

you know your 20th sales str with ai etc and the standard oh let's go build b2b sass with the boys i think that mean is really recognizing something deeper that for whatever reason that is what we have decided that a lot of people's bright minds that's that that is the extent of our success to put easy building the next big tech platform as opposed to deeper place um

This is what will happen if you're not actively thinking about and actively trying to push for a vision that goes a little bit beyond that. This got surprisingly deep, and I really appreciate that because I was going to make a joke about the SDR AI, and since I just saw some

TechCrunch article on 11x on how they were lying to people about their AI SDR. But I would prefer to walk down the line of, yeah, how can we materialize or manifest this experience that we would like to have with AI as opposed to just making a better B2B SaaS tool? And I've

really like your step backwards where the data may already be there, we just need to sculpt it in a different way, or it may need to be augmented with some kind of synthetic data, getting back to the Gretel AI idea, or it may...

to go and be collected. And we may need to start from zero or from relatively zero and then try and create feedback loops. So that's fascinating to think about. And it is cool to think about. I'm now racking my brain to try to come up with ideas on how to

My ideal experiences would be with AI and what I would want them to do. And that's a thought experiment that I'm probably going to be grappling with for the next couple of weeks. I think that's an experiment we're all implicitly grappling with.

A lot of the times, but I think bringing it out to the main conversation would be interesting. And just to kind of a little bit build on your point about 11x, it's a good example because people committed fraud in that specific situation. And implicitly, that means people thought that was worth committing fraud over.

yeah that's what we value yes that is what our world values both were like oh this was able to raise money because of this and this was able to they were able to commit fraud for it i think thinking about like is this the extent to which is valued would be critical for any anybody trying to engage in ai because if you don't

If you're not the one trying to shape conversations, then you're the one who's getting them shaped for you. And maybe you have other priorities. Maybe you just decide this isn't it for you and you're in a good place in life and you don't really care what things go. I'm not going to sit and judge anybody for their decisions, but I would just say that there is a trade-off to that that I think is often not fully understood.

Now, the other piece, just speaking about fraud and something that we were talking about before we hit record was because of synthetic data, we saw in what was it, fraud detection models or loan scoring models that the inherent bias went down within financial institutions. Can you explain a little bit more about that? Because I think that's a cool thread to pull on, too.

Yes. So the study specifically, I think, was talking about, hey, if you already use automated systems for loan approvals, etc. So banks that had very, very low racial diversity in the loans they were getting were getting huge jumps. And in general, the loans given were more diverse than before. When they put AI into play with their loans? That's the automated decision making system, yeah. So...

And I think it's very important to highlight such stories because normally you would talk about the negative biases around AI and how it can make it worse, but this was a good case. And I think what that goes down to, again, is thinking about what is it that would make one good and another bad. Usually, when you're trying to build any machine learning model or any sophisticated statistical analysis tool, what will happen is you will

actively start or you won't even realize but there are there are ways for your model to pick up on demographic information that is not explicitly being accounted for i think an example of this was a health care model where they were doing something like oh based on financial information or something you are making predictions on what kind of treatment these guys would need

Or like where someone lives is a prediction of their financial indication, which also is a prediction. You know, there's all these different features that you can have almost like scope creep on. Yes. So what ends up happening there is you're not explicitly accounting for race, but your model learns to pick out race anyway. Where you live is a great example because in America they had segregation.

So a lot of neighborhoods were pretty divided along racial lines. And it's still true to a large extent from when I was traveling through. Not that there's, you know, anybody explicitly saying that you have to live here or you live there in racial groups, but just like, you know, that's the way it got entrenched. So,

One of the weird things I picked up by traveling through America was just you can start to tell when you're shifting racial groups in a neighborhood. If you walk through enough, you'll start to pick up tells. So if you're accounting for that, then and that becomes a significant predictor, then you might end up adding, lowering the likelihood of...

somebody getting rejected from different races. So this AI was very, very cool because it, you know, they designed the AI in the right way to ensure that, you know, this wasn't accounted for and that there was like, you're only giving out loans to the people that deserve it the most. What that also, and where that becomes cool is

Now you might not see where synthetic data comes in here, but a huge part of that synthetic data pipeline is stuff like differential privacy where you redact sensitive information and you do all of that. So techniques from this ecosystem are being used everywhere. They should probably do something similar with college applications. They can do something similar in a lot of places where

human touch points and biases etc implicitly in works that would help to have a more diverse understanding on hey let's make ensure people aren't filtered out for the wrong reasons yeah and now what about just data we talked about synthetic data but also i think there's pieces that you wanted to hit on when it comes to the data ecosystems

We've kind of been touching around different aspects of that already, but I think one of the areas where we're significantly underpricing right now is how do you know what your data, how much your data is valuable, and how do you integrate intelligence into the way your data operates?

So an example of that might be something like graph neural network or graph, I guess, you know, the newer trends, etc. What a graph is, is essentially another way of encoding the intelligence of your data set. Because what that relationality does is it allows you to look across the different spectrums.

Something like that could be very interesting. Not specifically graph rag or graph neural networks, but how can you structure what you already have in different ways to integrate different kinds of intelligence directly into your decision-making? I think that would be exceptionally valuable as an insight because I think we kind of... It's one of those weird things where...

We went traditional databases. Then some genius comes up with the idea of phase in meta, you know, fast distributed vectors. And then people just took phase and a database, stuck them together, called them vector databases. And they raised a lot of money. And then we just kind of collectively decided that this is it. This is the peak of databases that we can get to.

which was again i think there's so much we haven't explored on that side a lot of the attention is kind of going on models and intelligence in the model layer whether you're doing it with the reasoning models you're doing it with better alignment doing it with more post-training etc but if you start looking backwards i think there's a lot of

insight just to be done by how can we restructure the very way we store and process data to encourage different kinds of information that I'm personally very excited about because I think there's just there's significant market potential there and I think if you do that if you can move intelligence into the data layer you can do a lot of work with um

You can just move in so many domains in so many ways because that's like an architecture innovation. And once you have an infrastructure innovation, I think they tend to have the biggest longevity in making lots of impact in a lot of different ways. Now, when you say move intelligence to the data layer, I think...

Bringing an LLM to a Raspberry Pi or something like that. What do you mean when you say moving intelligence to the data layer? It's a very abstract idea because we are not fully sure even how we want to do it. But I think graphs are a good example because

They are a very evident way of doing this, you know. When you think about graphs as a data structure, what do you have? You have the node, you have the relationship, and you have the weight of the node. I think fundamentally that's what a graph is. And that's a good example because if you had very, very unstructured data, you technically have all the information in there. Maybe not the weight mentioned explicitly, but everything else.

but why does graph rack do so well or why do graph data structures do so well because they've restructured it to really emphasize the relationality of your different components so that is one kind of intelligence and in an in a structure in any use case where relations become your major factor they become a very important part so that's what i mean is

The idea of storing information in graphs, the way you store them, what you're storing becomes very, very clear. Because one thing that people don't understand about something like a knowledge graph is that there are multiple ways to build it based on what you choose to prioritize. Same data set, same high level problem. You can actually build it in multiple ways. And the ways you build it will have very, very important implications of your performance. That's kind of a...

high level example of moving intelligence into your data layer is restructuring the way you have it set up but there are there are going to be lots of future examples how do we how what kinds of embeddings can we create what kinds of behavior modeling can we do can can we start figuring out ways where i am looking at certain behaviors from users and automatically creating a

model around that so that i can copy your actions um so that i can start learning from your actions playing with your actions um that's kind of what i mean um can i start looking at your behavior and automatically coming up with evaluation functions to say this is what the branch values is what they are demetrius values how does that change behavior over time i think that's where

That would be very, very interesting to look at and possibly very, very cool stuff to do. That is wild. Yeah, I hadn't thought about doing it on the behavioral level. I just thought about like, oh, we'll get Slack messages into the knowledge graph and GitHub issues are now part of the knowledge graph. And then cool, we've got a more robust context. But thinking about watching my screen and recognizing what

behavioral things I do. Ideally, I could cut out half of the shit I do because it's probably not productive. You know, like me going and scrolling X and LinkedIn. I do not want that to be part of my behavioral model. But I think that might also be where you get very good results because you're able to, you know, you have those moments of not doing much.

But not just an individual behavior model. What would be cool is imagine you had an AI recognizing because people don't understand this, but AI is not intelligent in the way we're intelligent. But that also just means that it has different abilities that we don't. What you're able to do is imagine you're able to track behavior across AI.

different users in a GitHub and then you're able to start identifying what problems trigger, what kind of behavior patterns trigger problems. But like, you know, we're having issues right now with a startup where, you know, we have so many people working on so many feature branches. Everyone is like doing two or three different things at once that you constantly have divergence in the working tree, et cetera, and then merging them, cleaning that up is a nightmare. So,

All of that stuff is where you could have truly, truly life-changing, work-changing experiences because you're able to process and monitor stuff at scale to identify where do things go wrong, what does it improve. And I think that's a very, very cool innovation. The reason I'm thinking about that

extensively now is because um so i'm working with a legal i started called akitas and one of the ways which is better than everybody else for legal is that instead of making innovations in the model layer we're kind of we we spend a lot of time making kind of innovations in the process layer so you you know your multiple steps before you come up with the final answer

You might do something like the, it's a very stupid way to improve your stuff if you're just throwing data and saying this is good, this is bad, this is good input, this is bad input. You know, that's what Harvey and all are doing. And it's a very stupid way to do things because you're not going to be able to propagate signal properly for any complex knowledge work through that.

But if you're able to map out an entire workflow and then build individual components to it and improve it across the layer, you just do much better work. And that's what agents are philosophically good at doing. That's what other setups are good at doing. So that's where I mean, I've been thinking about this.

pretty extensively because we kind of did a lot of workflow-based improvements where it's like, okay, if you're doing a contract review, what individual components do you have to do there? But I think on a general level, this can be abstracted into how do you, what behaviors would you have and how do you track them? How do you monitor them to identify potentially risky situations before they happen, potentially?

Or this is what you guys do really, really well. Like this is a bet worth doubling down on. I think once you're able to start doing that, you have a real winner on your hands. Yeah, I've heard it put as...

try and get the model to do the least amount of work possible or give the model the least amount of responsibility possible and create as many workflows or systems around the model so that when it gets to the model, the scope that it has is much more limited and therefore you have a much higher outcome of producing the right result.

That is very well put, honestly. I should have talked to you before this so I could have stolen that and seem much more articulate than I am. I can't remember who I stole it from, but it was not an original idea. I will tell you that much. But the thing that I like that you're saying is I'm extracting that out and looking at behaviors and recognizing which behaviors...

are going to be playing which part in this workflow or in this agent lifecycle. And not asking that you just have this open-ended agent that does whatever. You have maybe an agent that decides to kick off a workflow, which is very deterministic. Or there are different workflows, and as parts of the workflow, there's a little LLM node in there.

I mean, this isn't where I was planning to go with it, but now that you've brought it up, I guess I can talk about one of my very pet annoyances. I think autonomous agents are largely a sham, and I think...

The big reason why Silicon Valley likes to push that is because you, when you push for something like this, it's very sci-fi-y. So there's very little accountability because you can always defer the responsibility of actually getting things done up till later. Um,

I don't think there's a single business I've spoken to, and this is not an exaggeration, that isn't one that I could name for you that did good stuff with autonomous agents in their business workflows. Invariably, they ended up boiling it down to much, much more deterministic setups. There might be value in autonomous agents for research purposes and whatnot, but if you're a business trying to build things,

I wouldn't waste too much time with them. Go like simple deterministic workflows and agents, I think are very, very underrated and underutilized because again, people don't like thinking. And I think there's a little bit of an element where most agents are being pushed by a lot of tech people and tech people by and large, you get into tech because I think you tend to be a little bit more future thinking. So you're always thinking about what could I do next?

This is kind of what gets bred into your mindset as opposed to what can I do right now? And I think in the case of agents, that's just led to a huge misplacement of priorities. I think the places that I've seen the most success are when it is a workflow that is a determined pattern that

is a repetitive task. And one or two of those nodes or all of those nodes are some kind of an LLM call that does something, whether it is scrapes a website and then summarizes it or it goes and it creates a task that it will execute. These are things that you can have a bit more certainty on if you're

you know that it's going to be fairly similar every time around. Where you get into trouble is saying that it's like this generalized agent. Even if it is a generalized vertical agent, like a generalized HR agent or a generalized finance agent, that's where you can get into a lot of trouble because it feels like

You can't throw anything at it and expect it with high accuracy or high probability to carry out that task. And then there's frustration because you're grappling with the output and you don't know, did I just not prompt it well? Do I need to ask it in a different way? What are we doing here? Is it just not possible? That type of thing. And so I go back and forth on the workflows, right?

are quite valuable. And maybe the master agent that can kick off different workflows is a great design pattern that we can see. But then that generalistic agent, I'm keeping my eye on it for, is it actually going to produce that value? Or is it just this hyped thing that you're talking about where it's really cool to fantasize about?

But if you're running a business, hopefully you're not actually putting a lot of faith into it. I think, I don't know how much you guys have talked about like the automation, the doorbell fallacy, I think it's called. But I think that's a very good case study for anybody that wants to build technology. I think

It's shocking that that's not hammered in. I feel like if there was an AI person MBA, this is basically the case study that I would wake up my students with every day. Which fallacy and which case study? I didn't hear. So the doorman. No, I don't know this one. Tell me. So prestigious hotels.

Well, it's like some genius came up with this idea. And this sounds like a very reasonable idea that we now have automated doors. So we no longer need to hire doorman. And you know, that makes perfect sense. You don't need somebody to open a door for you anymore. Just keep it automated. When that happens, what ends up happening is the hotel had worse outcomes. Like there were homeless people hanging around the bus part, you know.

customer satisfaction was down etc and what you end up realizing is that the doorman stated job is to hold open doors and not do doors but what the doorman also does is

greets you well maybe you don't care that much for that but it's also a matter of you i come in with a taxi i've just flown in from a different country are big suitcases oh the doorman's going to help me out the doorman standing there prevents like vagrants from coming in and like you know they're not guests etc hanging around the lobbies and whatnot you have all of these other

like, not set benefits. Or here's a great example. This one time I was biking to New York, and I dropped my phone. There's a true story. And somebody picked up my phone left because they thought, oh, I don't want somebody else to steal it. But what this means is that I no longer have my phone. And I just moved into the city. So how am I going to get back home? Yeah, oh, no.

And what helped me was a hotel doorman because I'm an Uber kid. I think I've definitely taken more Ubers in a year than I have taken yellow taxis my whole life. So he actually flagged down one of them like yellow page taxis or would not for me. And I got to go home. Now, I wasn't even living at that hotel, but yeah.

That kind of stuff is not something an automatic door does for me. An automatic door opens and closes. So I think that's where it's a great case study for both replacing human labor, but also for thinking about when you're talking about these verticalized agents, when you're talking about these verticalized platforms.

like people don't realize how much other stuff people do that has nothing to do with their job that ends up being marginally useful here and there uh you know you mentioned scrolling on social media as an example i will occasionally find stuff on threads or twitter or whatnot that's worth sharing with my um research lab team i will occasionally find solutions so that kind of

That kind of information, I mean, I found Gretel by scrolling. Not as a company, but synthetic data I was scrolling through, I found some discussions on the replication bias in AI and that led to me finding this paper that was talking about, hey, we're struggling with healthcare data you're not able to send across borders so we can't replicate a lot of experiments, which is why we just chose to do synthetic healthcare data instead of real healthcare data because now I

I can just publish my whole data set and call it a day. And that was the whole reason I started monitoring synthetic data around 2020, 2021 in the first place. So like that kind of stuff, if you were trying to replace me, you could say this is not useful stuff. And if you were studying AI, then yes, technically 99% of what I would have scrolled through would be like MMA results or cats or whatever.

football or whatnot. But that 1% had an outsized impact but would be very hard to quantify and very hard to account for. So that kind of stuff I think becomes very, very useful. Or

jiu-jitsu i like i might i threw a random off comment there about jiu-jitsu on my in one of my articles and then tons of one of my readers really likes jiu-jitsu so we connected on that and we ended up doing something together very very cool um nice projects together now one of the interesting things there is ai actually doesn't like the way i write

it's told me on many occasions that because i i tend to use a lot of references and very off-the-cuff tangents and things like mentioning jiu-jitsu not as a analogy not like as an intelligent analogy but like oh yeah i do this and i don't know i was talking to somebody here and going into these slightly off-topic tangents um

That's AI is like, hey, clean this up. I think because your work is already long, like tighten this, condense this. A professional audience would not like your work, especially Claude. Claude hates my writing. Jesus. So I think that kind of integration becomes or that kind of verticalization comes very, very difficult.

So if you were building any AI systems or any tech systems, any automations, I think it's always worth thinking about that a lot of the time we as AI engineers aren't doormen. We as AI engineers aren't HR assistants. And we've probably not studied that process in depth, not saying that they can't be replaced. I'd be the first person campaigning to kill out every HR person.

and have them do something actually meaningful to society. But there's also a reason these things exist and there's a reason humans' labor has a lot of random behaviors that you might not be able to quantify or touch for that does a lot of good, that has a lot of impact in the actual work. So I think it's just a good case study to think about.

Dude, this doorbell case study is wild because it's almost the second or third order effects of implementing a system that at face value is much better. You save costs because now you don't have to pay the salary for someone who's holding the door. And I think about it and extrapolate it towards any type of

AI system. And I know I tend to talk a lot about marketing, copy being generated with AI. I've seen some very advanced systems of AI generated blog posts, and you can churn out a ton of posts with AI. But it makes me question now,

potentially what you're getting you're getting really good seo rankings because you're now churning out a ton of highly optimized seo keywords all that good stuff and so you're shooting up in the rankings and you're getting a lot more traffic but you also want to think through is this traffic

reading this, recognizing that it isn't that valuable and it's hurting my brand. Because that could be a potential outcome if you're just churning out a bunch of AI slop. Or in the case of HR and you have some kind of a pipeline and it's for hiring, but your potential candidates don't get to actually talk to a human until they've done

three or five things, they've jumped through all these hoops that are very much graded by AI. Is that giving the best experience to this future teammate? What kind of culture are you building there? Maybe down the line, you're optimizing for someone that is not going to be the best fit. There's

Yeah, it's really, there's probably a name for this, but the second or third order effects are huge. I mean, that is exactly what you said it so well, is second, third order impacts is where you get it wrong. And I think that's where, I mean,

I feel like a lot of tech people would benefit from thinking not about tech, but about other aspects in their life. Like, why does a romantic dinner sound romantic in the first place? Most of it is sigh of marketing bullshit. Don't get me wrong. But, like, walking into, like, a place with ambience, with a candlelight or whatever, that does do, like, there's a reason those things have certain impacts that we associate, you know.

One of the things is the more simplistically you look at a problem, a situation, an outcome, the more you're likely to miss critical pieces that end up becoming, that end up unraveling it. And I think that's where, I mean, a lot of our legal tech competitors went wrong. This is where a lot of stuff I've seen in general tech gets wrong very often is you'd look at

you look at you boil a whole problem down into a very very narrow domain or a very very narrow subtask and it turns out that that subtask isn't super duper important to begin with so I just think it's worth thinking about the Dorban fallacy things like that these are genuinely things that if we bond

If you don't think about them, you'll just end up building products that are useless to society. AI-based customer service is a good example where if you're not doing it correctly, what ends up happening is you'll just have worse outcomes because I just want to call you up and ask you about my stuff and there's no way for me to get that. The customer service should be replaced, I'm sure. There is a lot of good outcomes there where you can speed it up

But, you know, if you try to replace everything with a voice bot and dial this on your phone and do that, I think you might be missing very key elements of what is it that makes people happy or what is it that people want from you. And you kind of implicitly filter for people who, like the kind of loyalty you'll build is towards people who want something.

who want to maintain a certain, like, cost becomes their main priority, which means they'll cut you out if you become too expensive, as opposed to offering a little bit more, having that luxury and prestige and whatnot associated with it, and providing that requisite services. I pay a higher premium on my credit card and higher fees and whatnot, like if I were overdrafted.

Just because my credit card guys have guaranteed me 24-7 immediate support any country I go to. Like, I pay you an extra, you know, few hotels for that and risk a potential lot more. Thought twist, though. It's AI support and it's not human support. If that happens, I would be so mad. I'm here to talk to the cop in another country. It's like, hey,

please hold up this ring call press two if you want to talk this but oh man well hopefully it doesn't ever get to that point where you have to use it but i i understand the sentiment of we need to uh for certain things we look at how we are um

how can I say this? It's like we're voting with our dollars and we're voting and inherently saying what is important to us as customers. And there's, as a builder, a razor thin edge that you have to walk on. If I automate this or if I bring automation into this, be it AI or not,

Is that going to create a worse experience or is this going to affect my business metrics in any way for the better or for the worse? And that's something that I think continuously folks ask about because you want to be able to show and justify an impact that you're making as a developer, as someone who is implementing AI into production. So yeah,

You want to figure out what metrics you care about and the company cares about, and then how can you hopefully make those metrics move. Now,

What you also want to recognize is, are you making those metrics move in the wrong direction? Or are there other metrics that the business cares about that are being affected that you didn't even recognize? And so this brings in, it just makes me think like, wow, you really have to be playing 3D chess and you have to be recognizing metrics.

All these different outcomes. And it probably just starts by you being a user or you trying to game it out in your head and recognize where and how the old way has its benefits versus the way that you're trying to implement and what benefits that has. And really recognizing what the past benefits are with some...

a doorman, for example, why is that beneficial or an actual human that you talk to? And what's good and bad about that? Because we all have had experiences where we talk to a human and we get passed around to three different departments. And each time we have to explain our case. And so even though it is a human, it still is a really shitty product. So thinking about all those different ways that you want to make the product better,

It's fascinating. And now I'm just going to probably be ruminating on this for the next week or two. Again, with these thought experiments that you're giving me, this is like a lot of shower thoughts that I'm going to be having, I'm sure of it.

Well, that's how I pretend to be smarter than I am. Before this meeting, I was talking about like 20 random hypotheticals so that you don't have to ask me on anything technical. Who is this guy Demetrius is talking to? I didn't think he could be here. Oh, what sweet dude. Is there anything else you want to hit on before we cut it? I think a talking point I try to emphasize a lot through my work, but I think it's always worth thinking about.

I don't think AI is as solved as people think it is. And what that means is that it's a very open field and people aren't being ambitious enough with that idea. You know, there's a lot of open space in AI where you can come in, really screw up existing power structures, kind of throw your, throw a wrench in the world and come and take over. And I think more people should really come to recognize that, that

Yes, artificial intelligence as a field of study has existed for many years. But AI as an institution in society and an influential group and how it's impacting it at a much more active level is relatively new. And what that means is that you don't necessarily have to be a math guy like me or, you know, a software engineer or something to make an impact. There's like a lot of

There's just a lot of open space where you can come in and shake things up. So if there's anything I would want people to think about and work on actively is just, hey, am I really doing things to shake things up or am I just doing things to get by and get, you know, am I trying to become a Google or am I trying to be picked by a Google?

And I'm not saying that there's something wrong with laughter. Again, if you have other priorities, you're content with your life. I'm not going to say sell your wife and kids and focus on whatever you have to. But I think so many people that I speak to are just kind of implicitly buying into narratives where there are experts and there are non-experts. There's an other side.

people from them that will actually do things but as and they kind of become end up becoming participants and they become viewers of society as opposed to participants so i would just say like if you're going to make that decision make it consciously as opposed to just kind of making it because you've never thought about the alternative so