"Software should be soft" - Meter Command with Anil Varanasi
58:22 Share

When young people ask me, what's enterprise software? I tell them it's important. The last 10 years of software of us startups learning how to run better startups is really all about efficiency gains. You're in a fugue state with a newborn and the creativity is oozing off of you. And you talked about like a mantra of like software should be soft, but like clearly you had something else in mind. Welcome back everybody. It's Fraser. I'm Nabil. And we have a guest today, Neil from Meter.

Hey, Nabil Frey here. Thank you so much for having me. Good to have you here. This came from reaching out to Anil after I saw a tweet almost like a month ago now, right? A month ago as of tomorrow.

- Yeah, METR is the company, but the thing that really like lit us up was METR Command. And the basic description is it takes questions about your network infrastructure. And then like, in addition to answering them, it builds a real time visualization and dashboard to augment that experience. It's like part gen AI, part malleable software, part clawed artifacts, command K magic, canvas design, I'm sure we'll cover all. It just was one of these moments where it felt like you had peaked in the future of software.

And it wasn't widely distributed yet. And all of this out of a network infrastructure company. I had a bunch of friends bugging me about it. Like, what is this? This is crazy. And we just wanted to talk to you a little bit today about how this came to be. For sure. For sure. I'm really excited. Maybe you can introduce what the product is. And tell us where it came from.

Like Nabil was mentioning, we're networking with a company, build routing, switching, wireless, hardware, operating systems, firmware, all that good stuff. Just the sexy stuff. Yeah. Everything in the world runs on packets, and hopefully those packets are running on meter hardware and software. Historically, how you interacted with infrastructure was CLI. The good parts of CLI were that

You can get information really fast. Bad parts are you have to learn the shibboleths of every CLI over and over again.

Over the last 10, 15 years, starting with this company called Meraki, they pushed infrastructure into having like dashboards. And it was interesting because they were a networking company that did that, but it kind of proliferated to every infrastructure saying, whoa, if networking can have dashboards, so should we and storage and compute and servers and other things. Dashboards are really good because you don't really have to learn anything. You can just kind of point and click and get up and running really fast.

But the down parts are they're kind of slow if you know what you're doing. And then especially if you have feature requests,

It might take three months or six months because you put in a feature request and some product manager will pick it up, get it to a designer, engineer and testing. It couldn't be that we are all destined to be in this Dante's version of just making dashboards over and over again, like for every company, like what's next after this? So that's where like a lot of the explorations started from and a lot of our thinking came from. Where command a lot of it came from, we actually had the idea maybe a year and a half, two years ago,

is can we take the best of CLI and best of dashboards and really think about where software is going after that? You introduced the product with what I think is basically a manifesto, arguing that software originated for these various purposes and then it became as rigid as hardware in many respects. And

It's fascinating to me that this would come from your type of company. How did you give the team permission to explore this? The original works of software, whether it's like HyperCard or VisiCalc or Engelbart or whatever you want to think about and Brett Victor's work and Nabil, you mentioned Malleable Software, Jeffrey Litt's work and you can switch and all of this. There's a lot to be thought about as what is software. And there's still a bunch of folks that are thinking about that, right? Yeah.

In general, there's a few things about software that I think we all tend to forget. One, it's an artisanal handmade product. Like it's one of those things that, you know, it's just literally a handmade product. We haven't been doing it for that long. We've only been doing it for about 40, 50 years. So it's not a craft that has, you know, storied hundreds of years of kind of history and knowledge and other things.

In general, what we try to do is in our industry, particularly in networking and infrastructure, software isn't that great. It's kind of stuck 10, 15, 20 years ago. And interestingly, METO is a mix of folks that come from really hard infrastructure networking companies, which is about 50% of the company.

And then 50% of the company deliberately do not come from that. They just come from regularly building products. So I think it's less of permission in how we build software at Meteor. It's more of those collisions of those ideas, which is, oh, we're building this hardware. It's so challenging to actually build software, right? In our case, when

When we build a piece of hardware, we have to think about the firmware, the operating system, the distributed systems on top of it, the APIs. Then we get permission to actually build software on top that end users can use. So in that process, there's actually a lot of back and forth and collision between different teams that I think is a little bit unique compared to like pure software companies. I mean, you word it that way, but.

But that's also an intentional choice, right? You're saying that's endemic to the product. But if that was true, then we'd see the kind of crazy, amazing UI experiments and new user interface designs happening at every infrastructure company. The way that, say, video games, as a good example, do create that kind of font of exploration. Arguably, the first malleable software examples are things like

League of Legends and World of Warcraft, where really competitive players are building custom dashboards and custom keyboard commands and all the rest of these things to augment their experience. Whereas most infrastructure companies are not structured that way, right? What's the phrasing like people focus on the flowers, not the soil. And I'd love to know why you structured the company this way. Like, you're obviously sitting inside of an org. There's some long term product roadmap and linear somewhere or whatever. And like, and like, you're like, yeah, I

yeah, but also I want to mess around with this interface. Like, how does that happen? For sure. You're right. It has been deliberate, right? And like at the same time that we have folks that know how to do operating systems, our first kind of product designer was one of the first designers to strike. He was there nine or 10 years. It's not that many opportunities where you have those types of ideas and like operating systems collision in the same place. So it has been very deliberate. As far as kind of taking the roadmap and saying you want to try stuff,

I do think when you're presented with new evidence, you should change your mind. And the new evidence was that models have gotten sufficiently good to do a lot of things, like a lot of things. And interesting, actually, this is a question for both of you. We thought command was possible a year and a half ago, and somebody else would do something like that.

We had to go build a bunch of real-time data pipelines that we did not have, that our team was working on, all the way from every single piece of our hardware to be able to send real-time data. Because our version of command is, you know, it can get information, it can take action, it can write software, but that should all be rooted in real-time data. But I'm curious for both of you, why do you think there haven't been explorations? Because like the

The sufficient capabilities of models today, roughly, you know, GPT-4, cloud every year, year and a half ago. Why aren't we seeing more? You guys are both at the edges of this and see arguably every company that gets built. I'm curious what you both think.

Well, I think that that's where Nabil's question was trying to come from. But Nabil, I'm going to disagree with you. I don't think he went far enough. And Neil, to answer your question, I think it's just we haven't seen people explore creatively broadly enough on what these models are able to do.

You would assume that some product that's focused on things that are like much more creative historically would have the permission to explore in this way. And I don't think they have been like, right. That's my answer to you is I think we've seen a lot of predictable things come down product roadmaps and you have delivered something that feels different. And so like Nabil's question is like culturally different.

What have you done over the past year that has allowed this creative thing to not just like take hold, but to, I'm sure there was an idea that then had to be fragilely brought into existence. Can I kind of take a stab at that? I mean, I'll try. Yeah, please. I'd love to hear. Yeah. I think it's probably two reasons.

And similarly to you, I agree. I think actually even me and Fraser have been talking about for over a year, like, why are we seeing more of this iteration and why are we seeing more of this experimentation? And you see it in things like hackathons and meetups, but it's not really presenting itself inside of startups and real software. I think it's kind of two things, and both of them are cultural, which is why we're kind of backing up the cultural answer. The first is that I think we are over 10 years into...

I would call it the build something people want phase of software, where if you just back up 10 years, you get post-mobile explosion and you get a raft of blogs that are all trying to give you product development frameworks, OKRs, how to build a startup faster, how not to make mistakes. And it's all ostensibly arbitrage, right? The last 10 years of software of us startups learning how to run better startups is really all about efficiency gains, right?

Because we haven't had an explosion of new stuff, right? We haven't had a big platform shift. We haven't had a big technology shift. And so it really came down to, can I run my playbook faster than you can run your playbook on go-to-market and sales and virality and all of these other things? And I don't think we really think about this as a crank we've been turning as a startup ecosystem for a decade. Right.

And then it's not surprising that obviously like this huge, amazing thing happens a couple of years ago with GPT and with AI and everybody can kind of see it and can kind of get super excited about it and maybe do a weekend project about it. But we still live in the water that we're kind of swimming around in and it can be really hard to then translate that to, no, actually it's disruptive. It's disruptive, not just at the, now I can make better queries into my database, but

and instead of my SQL queries can be natural language level of innovation or my little chat bot prompt can do rag search on my docs kind of innovation. But like, no, it's actually going to innovate all the way up into the stack and maybe that's going to be inefficient to start with. Maybe that is going to take a little wandering in the desert to start with to try and figure out how this is going to manifest itself.

The good news is, like, last year, I was relatively despondent on this problem. And I made way fewer investments last year than I thought I was going to, given what was going on in the world. And it does feel like we are coming around. Is there a vector of things that you're seeing now that's got you more excited, like, compared to last year? Or has it been more broad than that?

I can only speak personally. For me, it's more broad than that. I think it's more like everybody realized that if you love AI, I can sell you AI to Main Street and to Wall Street, and I can get my revenue pop for a quarter and feel really good. And so you had a whole bunch of companies pop revenue for a quarter because they chased after the shiny penny. And then, of course, none of those things get retained.

And so they all talk to their founder friends about how hard it's been and how they can't grow revenue anymore after they raise their big fat round. And so the next wave of founders or those founders themselves are digging a little bit deeper and asking slightly more fundamental questions. And I think that's honestly like a good thing. Like deep, deep and slow is the cheat code in a world of fast and surfy flashiness. Yeah.

I couldn't agree more. I think rapid iteration internally, but good kind of polished things externally. There's a couple of things you mentioned, right? What is the parable of the diver going all the way deep, meeting like a legendary fish there and saying, you know, what do you think about water? All this fish is like, what do you mean water? Like, I'm here. Like, what's water? Right. And like...

Related to that, I think one of the things I do disagree with or maybe orthogonally think about is the thing of build something people want. I do think we need a little bit of reversal into build something you want a bit more, especially when new tools are available rather than what people want. We can talk about OKRs separately, but I just entirely disagree with OKRs fully. Preach!

Yeah. Hunter Walker wrote this great post. I don't even know if he remembers from over a decade ago. I think it was right after he left Google or something, which is like, you know, Google has the luxury to do OKRs. You do not. Yeah. Look, you don't. Like, they're a money printing cash machine unlike we've ever seen in the history of kind of business. No.

It's possible that their problems aren't your problems. Yeah, nothing applies to you, what they do, right? I think for us particularly, what it started, this phrasing that some people really like and some people hate that Sunil, my brother, and I had stuck in our heads the last couple of years is what Fraser was mentioning at the top, which is we thought software should be soft and easy.

That was kind of the impetus and where it started for us is just like, can we make software soft again and really give it malleable or, you know, whatever you want to call it. We had our first daughter earlier this year in January. And right around that time is when we were finishing a lot of our API work and real-time data work.

that enable something like this. So I was, you know, not sleeping as much. You're not sleeping anyway. You got an eight month old. Yes, exactly. You know, she's like two weeks old and I'm like, what does this mean? I can't sleep anymore. I'm kind of questioning everything. And then I get updates from theme, like, hey, the thing that we've been doing the last year and two years, it's kind of coming to completion. I'm like, oh,

finally we can go do the thing we wanted to do a year and a half ago or something like that. So originally how a lot of command came from is just doodles and sketches and prototypes for me as I was not sleeping as much. And then when I went back after taking leave for a little bit, I kind of just got some help from one or two folks to just prototype something together and then just presented it to the company. And literally I just recorded a loom one day and just dropped it in the company Slack.

unbeknownst to anyone saying, hey, this could be meter command. We are a very skeptical bunch at meter. And I think that happens to be true at most infrastructure companies and particularly at meter. We're a little bit older than the average startup crowd. Buzzwords are like,

Anti-meter and everything we stand for. And so, you know, I had to tread carefully there, right? So I just recorded a loom, dropped it there. And even given all of our crowd and the high standards, I think my colleagues have, the reaction was actually really good. And people were like, can I just use it?

And that generally is not the reaction we get out of stuff. Most times it's like, well, this one little thing sucks here. What about this? What about that? What's the user experience here? How are you doing this? But first reactions were, can I use it? Or this looks really cool. And that gave me even more solace than anything. And then my brother, who tends to have a really high bar for anything,

It's a miracle we've released any products working in the last 20 years together. He's like, oh yeah, this is good. I'm like, wow, since I've been born, I've not heard this from you. So like, this is good to go try it out. But that's kind of how it started, which is thinking like software is soft and actually just prototyping it and showing it. Because with a lot of the stuff that I've noticed with other founders I know or other product folks is,

is it's a bit harder to explain when you're building stuff on models because there's no reference point. There's nothing you can say like, it's like this or it's like that. So I particularly felt it's much better to show rather than tell, even if it was like a...

you know, janky prototype, if you will. That's kind of how it started for us. I think we need to do two things here. So clearly you didn't set an OKR for the company and then go for like Q2 planning and come back with like, let's work backwards from moving that metric 20%, which is our goal. And here's this like idea of what we're going to do. You're in a fugue state with the newborn and the creativity is oozing off of you. And you talked about like a mantra of like software should be soft.

But clearly you had something else in mind. You were trying to solve your own problem or some sort of problem for the organization. And maybe while telling us what that was, what was the germ of the idea that you were trying to get out? Maybe even talk a little bit about what command does so that people get a sense of what it is. We have security appliances, switches, wireless access points,

power distribution units, all designed and built by meter and then operating systems and firmware and APIs on top. Then normally we have a beautiful dashboard that I think the team has done a really good job on. The problem we're trying to solve of this is actually is that because of how good building certain type of software has gotten, we tend to either build software these days for the novice or the expert.

You're kind of told, who's your persona? Pick that persona. Build it for that. Ignore others. And all these, you know, kind of aphorisms that go over and over again. One of the things I was noticing is we started to have a real barbell of users, which is some of the largest organizations in the world.

With actually IT teams larger than Meter's entire company use Meter. And then some of the smallest companies that might be startups that you all are funding or others that are growing really fast who don't really have IT teams, they just have product people, they're using Meter. Both ends of the barbell. Right?

And we see this also in like retail and hospitality that are even further from technology where they don't even have technology teams. So kind of the original problem we're trying to solve for is it's going to be impossible for us to build for both. Can we use models to be able to solve that? So that's one problem.

Second problem is in infrastructure, a lot of it comes from huge amounts of logs, right? That might be logs from the hardware, that might be logs from the clients that are connected to the networks, anything like that. And a job for the user and for us is to be able to build things so that you can actually understand what's going on. That's problem number two.

And then problem number three is if you want to take action, it could be slow. You have to learn what you want to do and you have to kind of figure out the exact terms. If there is an industry that is littered with acronyms, come to networking. So what command essentially does is sits on top of our APIs. It understands our hardware, our architecture, our APIs, and also how we design software and how we write software.

Kind of those building blocks for command. And then it enables essentially three problems that we wanted to solve and nothing beyond that. First is being able to ask questions. And this is where a lot of the UI exploration started, which is our customers might have hundreds of thousands of pieces of hardware that they get from meter. We all use Slack a lot and email and other things. So one of the things we started doing is like, can we just hit at and actually talk to a location?

or talk to any piece of hardware anywhere and personifying hardware a little bit of just like, I just want to talk to this piece of hardware because I just care about it right now for any particular reason. And that's in the demo where you're like at 570 York Street and you're like, that's our address. I just want to query a location. Yeah, that's right. Yeah, that's our office network that we were showing in that demo. I just want to query a location.

And then the first problem is just like, we just want users to be able to ask questions. And that can be anything. That can be anything from like, what's happening with this particular piece of hardware or the location or one particular client or what's happening with security, what's happening with throughput, bandwidth, use, anything you can kind of think of to be able to ask questions.

Second is kind of just taking action. You want to do complicated security firewall rules because you're going through new compliance or some new large deal you're working on or in large companies case, they're constantly adding new things. Can you just take action really quickly that might otherwise take five to 10 clicks or two, three different pages? So,

What a user would type in a ticket, can they just put into command and just have it be done? Instead of having to like put in a ticket that somebody looks at it and then somebody does something. Can they just type that there, right? This is what I'm trying to do, do this.

Get me the outcome. So that was problem number two. And the problem number three command solves, which is we should talk a bit deeper on like the whole interface, but can I actually take it software the way I want? What we've noticed is most enterprise software ends up being about compliance or reporting. Right. I need to give reports to somebody else about this thing that I bought. Right.

That's kind of it. That's enterprise software. When young people ask me, what's enterprise software? I tell them it's reporting. So what we wanted to actually be able to do is instead of everybody having to like figure out how to get the reports they want or being able to understand their infrastructure or create dashboards the way they want.

software, especially models, have gotten particularly good. And if you do a really good job of actually training and keeping the model small, you can asymptotically get to 100% accuracy. And we saw that. And it turned out to be an incredible new architecture that one of my colleagues, Natan, ended up just... Honestly, it was three months of seven days a week, 15 hours a day from him, just all the time. And so...

Can we just like have software be built? So command is exactly that. You can just go in, talk to any piece of hardware, ask any questions, take any action and have software written for you entirely on the fly, just like you had your own engineer for you. You were asking me about earlier where you're like, why hasn't this happened yet? Right. And the answer is, is really short term versus long term thinking.

And I think if I'm going to raise my next round, I need to get my revenue growth right. AI looks like it might be able to help that. I got to chase that. But I also got to solve that in three months or six months in order to hit my OKRs or to raise my round. I want Shmooby Shmooby Ventures to see the quick rapid growth rate and mark me up. Yeah.

because I'm terrified and or the reverse side, which is like, again, I just dropped into a seed round or YC it's three months. Please show me my week over week growth rate. And yeah, how can I have a I solve my near term problems? I can see now why you reacted so positively earlier to my comment. Slow and long term is a huge advantage in a world of fast and surface. If you think backwards,

then the conversation about AI doesn't start from the UI layer. The conversation about AI starts from the data and infrastructure layer. And I don't just mean that as you as a company, I just mean that like for everybody. It's the core question is, what data am I collecting as a company? And so if I'm you, I am collecting a bunch of data

about network configurations and what people are doing to fix problems and all of those things. If I'm Replit, then I'm collecting a whole bunch of data, not just about the code that an AI agent is generating, but also like what does a human do to fix those problems? And I have like an incredible amount of logs, ostensibly of human behavior doing these behaviors.

And then the kind of core model question, if I really care about AI long-term and how it might transform my business, first starts from a, how do I structure my APIs and my data in order to capture those insights so that both a human and a computer, aka a model, over time can access that data, take actions, do those things. And then as you kind of referenced, know what

what format to ask for help in so that eventually as the model is doing more and more things automatically and to be able to ask for help, right? Yeah, for sure. You have to sufficiently assume that at some exponential rate, the models would get better. And you can say like the rate of growth might slow down and we can all agree on that or disagree, it doesn't really matter. But I think we can all agree that it will continue to get better for sufficient horizon that we can all see. If that is the case,

What is it that you can build and only you can build that can give a lot of value to your customers? That feels so important in this story. The realization from the person running the company that this is a major inflection moment and then the

I don't know, the permission to be super ambitious with what you're going to do. And I can't help but think that a big part of that also comes from the fact that you're clearly playing around with these technologies. They're on the fringes and trying to understand them. Yeah, I try to read every paper that's coming out. Okay.

Right, right, right. Like you are not, you are not like two steps removed. Well, this is the way to hack it, right? You ping all your friends at OpenAI, Anthropic, everywhere and say everything that you've read, which...

which ones do you care about the most? And in a given week, they all mentioned the same thing. Then you should just understand that one entirely deeply. You need Reddit for AI papers where only a small selection of friends of a certain caliber are allowed to vote up. And actually, surprisingly, even though Google put out the Transformer paper and kind of, you know, pun intended, transformed, they still are doing some of the best work as far as research is concerned. And like some of the best work is still being put out by them.

Maybe others are doing better work, possibly, but it is not out in the open. Whereas in Google's case, a lot of it is still out in the open that I've been a huge beneficiary of because of what they're doing. Yeah, they're still one of the houses where the culture is to release...

We're not going to sidebar into a little bit of a worry about how all the paper culture is going closed and what that means for the future of innovation, all the rest of that stuff. Maybe another podcast at another time. Maybe this is something Fraser should be answering for us. What's happening, Fraser?

I have no idea. I'll tell you a meme that I like. Slash I can't say. I'm going to a meme. I'm sure you two saw, but it was the DeepMind researcher who is frustrated when they agree to publish his research because he realizes that it's no longer important or it's like insignificant. It's like hilarious and a damning comment on where we are with this piece of science. So...

I'm assuming you were like prototyping and hacking and the proof of concept that you built was on either like cloud or reopen AI API. At what point do you like decide that you want to train your own model? That's a non-trivial undertaking.

I started for Command specifically realizing we just need the right talent. And never in my mind it meant like dozens of people, if you will. It meant just a couple people. And I started asking everybody I knew. And luckily there's a bunch of people involved in Meter that are running the best companies in the space and building models that are large for the world. I ended up meeting maybe over the course of a few months people

roughly somewhere between 40 to 60 people. And the unique combination I was looking for is folks that understood actually how models work and building it. When I see things like post-training, it's like the anti thing I'm looking for, which is like post-training is, you know, another rough equivalent of prompt engineering, if you will. And somebody that actually been a little bit deeper, but somebody that actually cares about building products.

A lot of the issues I had with folks that I met were they were very interested in these problems from an academic sense and from a research sense. But turning them into products essentially means you have to cover all the edge cases.

It's not all the times that it's going to work. It's actually all the times that people are going to break it. I ended up meeting someone I was mentioning earlier, my colleague, Nitin, who's actually not from this world at all. He was at one of the best biology labs at MIT. I'm going to butcher this woman's name who's a legend, but Regina Barzile. She's like this legend in biology and MIT world. And he came from that, but he was particularly interested in building products.

And I luckily happened to have met him at the right time. I kind of had it so that he literally sits next to me at the office. Talking about like culture, like literally sits next to me so that I can at least through osmosis learn everything he knows as quickly as possible. So,

It started from there. And then, you know, I kind of dropped him into this prototype code base that we had. And then just he went at it. I'm not exaggerating here. There were many weeks that, you know, we were literally up to 3 a.m. all the time.

We've gone through this arc. I'm a founder. Let's assume I'm the founder listening to this or a VP of product or VP of eng. I'm now thinking about how people use my software and I'm thinking about all the data and bits and user behavior that's going on inside of it. I'm thinking about how that needs to be. Oh, we're not even recording all this stuff. We're recording in stupid logs. I'm not getting half the stuff even captured. What behaviors do I even capture? How do I even do model building? And I will back up to, I literally had this conversation yesterday with a CEO who, you

without saying what the company was, they actually have some AI engineers. He's like, I think I need to reset the team or something's wrong or maybe I need a new head because

Because what I feel like I have is a bunch of AI researchers and then a set of people that are telling me and giving me advice that I don't need researchers anymore or model builders. What I really need is just like good AI prompt engineering people who sit at the product layer and just like hire those people, let them get to work. Most of this stuff can happen at the prompt level. And what I feel like you're pushing back on is engineering.

It's neither of those two things, right? You're talking about somebody who is not a core AI researcher, aka you're not competing with the X million dollar bonus person who's going to DeepMind or OpenAI and trying to hire that person away. Well, this person could have, but luckily they're a meter. But they're not 10 years into DeepMind research and being poached by OpenAI or Claude, right? Correct.

But they're also a model builder. I'm trying to get into founder mode. You're now talking to three other founders about how they could try to execute this idea. They want to build new sets of interfaces. They want to build the next stage of malleable software. They know that that's not just some prompt engineered thing. It has to actually go down to having real-time access to data. It has to go down to custom models. It feels like a lot. How do I hire and staff against this? There's a couple of things you mentioned there that I have like a...

cultural anti-reaction to. I love it. One is actually, I don't know if people should even be saying AI to the point that we don't anywhere. It's true. You've only used the word model in this whole conversation, right? You will find it anywhere. Because the thing is this, right? I am now sufficiently old where technologies come all the time that are there.

And it roughly feels like telling people that I use Postgres or MySQL or ClickHouse or something like that. I don't think users care. I really don't. And I have yet to meet a normal person, not one of our crazies here in the Valley, that's like, oh man, what database are you using? This old thing of like, when artists get together, they talk about turpentine. Well, actually turns out the majority of people actually don't care about turpentines. They just...

They just want to see the art. So I think like I just have this reaction to like, if you're taking an existing company, existing product, not a new one, I think new one, you can do whatever you want. I haven't started a new company in a long time to remember what that's like, I think. But I think existing company, I don't even think you introduce AI to your company. I think you have to talk about what are the problems you're trying to solve? And this tool can help solve it.

Because I think if you come in blazing with like saying AI, one, I just think it's wrong personally. But two, it doesn't seem strategic. That's how you introduce things in a company. Here's one trite example. Let's say tomorrow, some CEO or founder or CTO goes to the company who has been using Postgres for the last five or 10 years. And then suddenly he's like, we're switching to MySQL tomorrow.

A lot of people would just quit. A lot of engineers. Just like, one, why are we doing it? And two, it's not like a sufficiently interesting thing to do or it's possibly detrimental to the business. So...

I have this like anti-reaction to even saying AI at all, anywhere. I think the people that are building it get to say it. And that to only a few labs or few companies, everybody else, I don't know. It feels crazy to me to say. Then second, I also disagree with how quickly we all pick up on truisms a lot.

which is we all end up saying something and somehow it becomes like Beetlejuice and becomes like real, right? Which is everybody kind of just says is like, you don't need models. You don't need anything. Just use an API. Just prompt it. And somehow people end up believing that. And the thing that you actually get with large models is creativity, creativity of answers, not precision creativity.

And I think it's actually one of the greatest things that came from next token prediction or anything you kind of, however you want to phrase it, is that you can actually get creativity out of machines that were only deterministic in the past. But precisely because of that, you actually cannot have creativity in enterprise products. What you want is accuracy and speed.

Like if you're creative and data, you're giving back to your users. Well, they're not going to be your users for a long time. So the truisms, everybody kind of just repeats, you know, I don't go out when I do. These are the things that I just generally entirely disagree with. And the third thing, probably what's going on is you were kind of alluding to this a little bit. I do think you have to own the entire thing, especially if you're not playing short-term games.

If you believe in what you're doing and you think you can solve something that's unique to your customers and your users in a long enough time horizon, you kind of have to own the whole thing from the architecture to the data to how you're going to label it, how are people using it. Basically, if you go look at what does OpenAI do, for example, they painstakingly do all of these things to get models for what they want.

I just don't think other people are that different and other companies are that different. You just have to own the thing, like actually own the thing. I mostly agree with you. Some of the more interesting conversations that I had internally there was when I would kindly point out that the generalizability of the model was something that we cared deeply about, but many of our customers found to be a bug rather than a feature, right? Yes, it can translate French to German in the middle of, well, I'm doing SQL. But I do think that like,

If you care deeply about the product that you're delivering and it is using this technology in a very meaningful way to enable that in time, I think a lot of groups are going to have your point of view, right? Like we have learned. Look, if I interject. Of course. I disagree with you entirely there. Do it. Let's hear it. I actually think it's the inverse of,

which is the reason to own it today is because the ecosystem is not mature enough. Over time, you have the luxury of not owning it because all the things around it will come that you need to not having to do it. Today, that's not the case. And it won't be for five or 10 years. You guys might remember when EC2 came out or RDS came out.

It was a pain to use EC2 to get what you want out of it. And all these different companies existed as tools on top for you to be able to help to get there. Or a lot of companies ended up building it internally. A lot. Talk about rappers, if you will. Those were the rappers 10 years ago, right? So the thing I disagree with you is, I actually don't think over time people will come to my view. I hope people come to my view now. And over time, they don't have to, if that makes sense.

You're talking about the arc from innovation to commoditization. I remember in the early days of Zynga, we were there and Farmville's out. It's scaling like crazy. We are literally the largest customer on Amazon AWS. And we were breaking their infrastructure constantly because of how quickly we were growing. And so a bunch of folks that were actually pinging the AWS guys and having them fix bugs, and we were the only ones bumping into them. And so you end up with a whole bunch of

custom software that didn't exist out in the world and only existed inside of Zynga for a while that is now like everywhere and is either built inside of AWS or there's other separate companies you can get that will do that as like middleware companies and so on and so forth. You're just saying like, listen, if you're ahead of the curve slightly, it naturally means you're going to end up with more bespoke things. And then over time, portions of those things stop becoming strategic and

Because other people can build them, they've realized it, and then you can actually, if they're not deeply, deeply strategic, then sure, it's an API call, it's a commodity, it costs, you know, 0.001 cents per whatever, and it doesn't matter anymore. That doesn't mean it wasn't important to do in the meantime. Yeah, I just think today none of that exists anymore.

Even if you go talk to labs, I'm sure you know your colleagues, even at Okneon, they're just blistering releasing stuff, right? They know that there's a bunch of things to be built that's scaffolding and tooling and infrastructure. There's just not enough time. And if you're a company using all these things, I think the main thing I'm positing is not that you should go pre-train hundreds of billions of parameters and do the thing yourself, but more at least the scaffolding on top

And everything that it takes with your data, you have to go do. So for example, we're talking about creativity and you were mentioning, Fraser, about the generalizability is a bug, not a feature. One of the things we were doing in our architecture is before an answer goes out in command, there's what we call a denier, which is even after going through the entire architecture, there should be a mechanism that actually denies the answer because either it wasn't a good answer

So, for example, when you ask Command, who is LeBron James? The only answer is it's not about networking. I don't know. That's the only answer. You just answered your own question about why we haven't seen

more companies do the work that you've been doing, which is quite simply because it's hard and it's not just prompt engineering, right? Like you can see where the future is going to be in software, but in a way you're doing multiple different things in order to get this to fruition from internal custom model building, but you're building a denier of the prompts, which is...

I've seen several CCH companies that are kind of doing that as an outsource, but they're super early. Really? Incredible. I had no idea. Lots of tiny model builders, but they're all pretty early. Like you're going to get to how do I build a malleable software canvas on the right-hand side so that after I type in my command, it turns into a little UI framework that then I can swap into this canvas and move around.

There are also companies like TL Draw and others that are helping as API infrastructure layers in order to solve that problems, but they're early. And so I'm sure for many founders, they look at this stuff, they prototype it for a day or a couple of days,

And then they're like, yeah, but actually it's kind of kludgy and it's kind of broken and it's only 80% right. And it gives me bad responses and it just seems like a lot of work. And meanwhile, I got to hit next month's. And so they're just next month's revenue or next month's growth target. And so what they say is like, it just feels too early.

Like I tried it. It's kind of nice to prototype in. I think a lot of people feel this. Like it's kind of cool to prototype in. It feels kind of magical, but the path to take it from nice demo to actually working is a lot of bespoke stuff.

And, you know, if I'm a founder, I'm like, yeah, why don't I just wait? Still feels too early. Why don't I just wait two years? But isn't that the exact opposite way of building companies? Like you guys tell me, like you want to be ahead of two years ahead of anybody else, right? I think if you wait two years, isn't it possible that other people can also do it? Oh, we are very against the OKRification and short-termism of startups. We're on board with that. Yeah.

I'm being silent not because I feel like I just got smacked down. I've been silent because when somebody smart who's thought very deeply about a problem forcefully puts forward a worldview that's misaligned with yours, you should stop and figure out where the disagreements are. And maybe it's not interesting in this conversation, but I still feel like I'm right and I can't figure out where I disagree with you. We should argue offline for sure. I'll come see you.

I have a ton of questions for you in general, but I guess coming back to some of the questions you guys were asking on command itself too, I felt the same way about the chat interface, right? Everybody that was building the interface itself, it was all about text. And, you know, it's like more beautiful text or more marked down text or better font text. But at the end of the day, it was just text. And that also surprised us a lot. Yeah.

But we had this all done before artifacts came out, like way before. And since then, we've been kind of perfecting it and making it good. And so the thing that also surprises is why is nobody just doing inline software? We're all visual creatures. That's why software exists as a whole. It's not memos that we give to our customers. It's software that we give. So why isn't there, along with tech, software that just comes inline? And we started doing that.

Then when looking at the interface, we felt, oh, everything in the chat style interface is very ephemeral. It felt like I had no relationship with the thing and I did it and it was gone. And what you actually want, and if you talk to product folks, and Zynga, you guys know this, Nabil, you want people to have like favorite spots, bookmarks, and you want people to have all these things because they fall.

form sort of a connection. Yes, it's good because you want engagement and all this stuff and metrics look good. But actually what the signal is, whatever the software ended up doing, it's some sort of signal that it was a tiny bit valuable to the user, just a tiny bit. And so we were thinking about is like, why are all these interfaces ephemeral? Like I'm doing all these things and models are doing cool stuff, but it's kind of just gone in the vapor. So we were doing inline software. And then I was really thinking about

Can we just like pick it up and move it to say, I care about this and say, like, this is the thing I want.

And that's where a lot of the canvassing came from. And once we picked it up and moved it there, I was heavily influenced the last five, 10 years, especially Victor's work and what they've been doing at Dynamic Land and other things, which is the objects that you care about should be on a canvas that's malleable and kind of like Play-Doh in your own hands.

So then we said, okay, if I'm able to put stuff there, can my colleagues put stuff there too? And what it actually ended up just hearkening back to is when I studied networking, it was in this dark room of the engineering building nobody else went into. But as you were doing CLI commands and other things, somebody else would peer over your shoulder and say, have you tried this? Or what about this? Or something like that. And we wanted the canvas to be entirely multiplayer.

And I'm a huge fan of what Figma has been able to do from both the usability and inventing a new thing on the web to the performance of it. And just like the sheer nature of it. But also very surprised that that is not a normal pattern that's used in other software. Even if you put the models aside, I think even all of this you should own right now because these components really don't exist outside to go bring. Yeah, I think we can be frustrated about...

why we don't see more of this. But it's just, people start with the chat interface because it's the place where we started. And it's not surprising to me it's where we started. These transferable models output text. And so, I mean, Fraser knows this a lot better than we do about how this comes to fruition. But for me, it's like, it's what you talked about at the beginning. You start with command line interfaces,

which means you start from, if you just really back up, you're starting from MS-DOS. You're starting by typing in into a little command line on an ASCII character sheet. Before we get to graphical interfaces, you start with icons, right? It's just a couple of little icons at the top of Microsoft Word.

or WordPerfector back then. Now I can click five buttons and that's the interface. And then we're like, well, why isn't the whole thing a bunch of buttons? And why don't I have lots of choice and I can't move things around? And that's where we get to dashboards. I just feel like in AI, sorry to use a word you don't want to use, but I think in AI, what we're getting is exactly those same things

patterns of history playing out slowly over time. You start from command line, you start from text, you move into, I have a couple of buttons I can press. I have a couple of actions I can take, which is what you get in something like a Notion-like interface right now, right? I right click and I can hit shorten text or like improve text. And you're moving towards a situation where you are building more malleable software over time. Credit to you,

to give, I think, maybe a bunch of other founders and product builders a little window into what that might look like in the future. Yeah, I think there's a lot to be done there. We also just try to extrapolate a little bit, which is if we extrapolated five or ten years, is it possible that most software is just generated on the fly for you? I think that's true. But...

Then I started thinking about is, let's imagine all state-of-the-art model development stops today, which is something happens in the world. We no longer build any new models after this. And what we have currently today is what we're stuck with.

I still think all of those things are possible. That's what we have today. Even if we froze everything in time, I think what the labs have done and what these companies have done is just remarkable on the progress they've made. And this actually comes back to is when you go talk to folks that are actually building this stuff, and Frasier, you can correct me if I'm wrong, a lot of use cases surprise those people too, because you actually don't know everything a model can do.

Right? Like, just really surprising. People did what with it? They're very upfront with that this past week with the 01 release, where they're like, hey, this feels like GPT-2, where it's going to point to a new paradigm. We're not sure what's going to come. It's going to be surprising and great. Let's go for a trip.

Yeah. I know why we haven't seen it elsewhere and we're seeing it in meter. Like it's, it's you, this has been a great conversation in the sense that you're, you're rare. You're a long-term view oriented founder who is, you know, really immersed in the technology that's changing things right now to the depths that you are and your product motivated and product minded and like have good product taste. And so I think that those are all, all rare. Yeah.

Some PM in some division who's trying to work on a roadmap can't do this. Think about all the things that you just said that you just had to tackle to make this possible and your long-term orientation. Yeah, I think what's been really fortunate is that the underneath core product is so interesting for stuff like this. It just delves itself for the possibilities. That's the other thing, which is

You can only do with what you've been given. And I think like we were given really interesting hardware, operating systems and data that we control and, and,

infrastructure that's critical to people. One user told me a couple of years ago, which is if meter goes down, I can roughly calculate how much money I lose. Well, that always woke me up. And because there's like critical infrastructure in these warehouses and schools and labs and retail and offices and all this stuff. I think we have the building blocks that maybe if I were building, not to pick on HR software, but if I'm building HR software, I probably don't have

this data and building blocks underneath? Oh, I'm sure we could workshop it. I actually, I'm sure if we spend another 20 minutes here, which we won't, we could workshop what user actions and behavior you could capture, model build around, build action against, and know when to ask for help to use your framework from before. Because I think the truth is that there's like founder market fit, which we talk about a lot. And then there's founder product fit. And I don't have a great

headline saying for this, but there's also a founder being fit for the times. And where we're in right now is a situation where, as we said, kind of said earlier, like this is not a time for running, uh,

Your arbitrage playbook, right? This is a time for thinking long-term, how transformative can this be for this business? And then working your way backwards to what we can get done now, right? It's kudos to you for taking that time and energy. I think it's a great place to stop. Last question for both of you. If you could assemble the best people today, each of you, and have them go work on one thing, and that can be any layer of the stack,

where would you point them to? Let's imagine you have the best five or 10 people in the world possible at your disposal. They're only doing your bidding. What do you wish both of you point them at? I'll go first. Mine's so uninteresting in this moment, but it's the honest truth. Our applications will write themselves soon. I very much believe in the future that you've talked about a couple of times here that we will have software that is

just in time for our specific need. And I think we are still dramatically underestimating the impact on our lives when that's true. And I think that you have the very best people in the world working on a problem that's that ambitious with that much leverage. That's the thing that you should do, even if it's everybody's working toward that direction. I think my way of answering this question is that

One of the reasons I'm like super excited to do the job that we're doing right now is that

Much like you had to rebuild all of the economy for a post-industrialized era at a certain part of our history, we're going to go through literally all of the industries and do this over the next decade. And that just feels like a decade of work worth helping founders through. So the fact that it's not one thing is why, frankly, for me, I'm not founding a company right now. I actually like this work across all of things. But I'm not going to dodge your answer. No.

If I were to pick one thing, the headline would be AI and how you learn in education. But I think it's more nuanced than that. I was having a conversation with my son and he said something very insightful that I'm just going to repeat, which was like, all of learning is basically coding and

writing, art, or math on the micro level. So the macro might be I major in philosophy, but the micro is I'm writing. And the macro might be I'm majoring in economics, but the micro is mostly math and a little bit of writing.

And so as he's thinking about navigating what he wants to learn about, he's thinking about how the micro matches to the macro. And he's like, well, what I like out of coding, writing, art, and math is mostly coding. And so if somebody is going to ask me which one of those things I have to do for eight hours a day, then I will do coding. And so now that means I will go into data science or go into blah, blah, blah, or whatever, the things of which I'm being asked to do this so that I like what I'm doing every day.

But especially in a world where we have a core transformer, which can move one thing to another thing, and we have models that do that, it would be really interesting to build an educational paradigm where the macro and the micro are decoupled.

What would it feel like for me to learn about philosophy through the act of art or math versus writing? What would it feel like to learn about computer science, but at least a micro, at least initially, is, you know, through writing?

And it's very, very possible. It's probably at the edge of what models capabilities are. But it feels like the way you could decouple being a curious human about the world around you and grabbing at things from the, you know, I happen to be 12 years old and I happen to be better at art than math right now in this moment. And that shouldn't block me from learning about certain types of things in the world.

Really cool. Thank you both for your time. That's a good answer. That's a subset of my answer, though. My software is going to go and write that. The question goes back to you. You're not allowed to turn this back to me like, oh, I would go build the version of meter that I'm building now. What would you do? I would rewrite the operating system entirely from the ground up.

And I actually don't think enough people are ambitious about this. My comment at the end of Nabil's was flippant, but my comment here is real. Aren't you and I saying the same thing? We are. Very much so. Very much so.

Look, if you come across one, let us know. I'd first recruit them, but sure. And if I come across one, I'll let you know. Thank you so much, Anil. I think it's a great place to call it. Thank you for spending a little time today with us. Thank you so much. Thank you both. Really appreciate it.