Who's Coding Now? AI and the Future of Software Development
44:30 Share

Today, I think AI is not just a higher level language abstraction or something like that, but could it become one over time? That's the question. Do you think yes? I think it could. If I look at a classic, say, compiler design or in programming languages, if I would have LLMs as a tool

i would probably think very differently about how i would build a compiler yeah and i don't think we've seen that work its way through yet but if i can basically define certain things in human language in an efficient way and you know maybe in a sort of tight enough way that can use this directly as as input for a compiler that could change a lot of things thanks for listening to the a16z ai podcast i'm derek

And today we have another great episode digging into how generative AI already is and will continue to impact our digital lives. A few weeks ago, we featured A16C partners Guido Appenzeller, Matt Borenstein, and Yoko Lee discussing what's up with AI agents. And we brought them back for this episode to discuss what AI coding tools mean for the future of software development. It is, as you'll hear, a topic that entails so much more than just vibe coding.

The three of them discuss everything from the relative market size for AI-generated code, presumably quite large for a capability that could juice developer productivity to the tune of trillions of dollars, to how prompt-based programming might affect the value of classical computer science education. This is so much bigger than stack overflow on steroids. So get ready for the full discussion, starting with the question of why coding has become such a big use case for generative AI after these disclosures. As a reminder,

Please note that the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. For more details, please see a16z.com slash disclosures.

We're pretty sure it's the second biggest AI market right now. Correct me, guys, if I'm wrong, but consumer pure chatbot, I think, is number one. And I think coding is number two, just purely looking at the numbers. But consumer is the aggregation of a lot of the things. Exactly right. It depends on a defined market. I think there's... You can make an argument it's the number one, actually, if you look at really homogenous markets. Is coding bigger than companions?

Yes. I think so, yeah. Yes, at this point it is. You think so? That'd be interesting. It probably depends how you classify something like ChatGPT, which to some degree is used for companionship. So a large portion of ChatGPT usage, I think, now is companionship. I think that's right, yeah. Yeah.

So, well, in the end, is a person's motivation greater to build something or to find love? I think, you know, it may be neck and neck. One thing that's very unique about AI coding that's sometimes underappreciated is this was actually an existing behavior in a couple of ways. First,

People were already going somewhere to look for help, which we mentioned earlier is Stack Overflow for the most part. So there was already sort of this muscle people were exercising when they hit a problem they couldn't solve to go find the information on the internet. And this is really just a much better form of that. You know, there are all these jokes that Stack Overflow was actually writing most of the code for the last, you know, X number of years.

A lot of that may just shift to AI models, right? It's not clear if that was a joke or not. Yeah, maybe not a joke. Also, there's this thing called GitHub Copilot, right? They did this really foundational work to start to transition people off of that sort of Stack Overflow use case to using AI models. And I think

Companies like Cursor have just done a much better job with that now. And so you have this fairly unique thing that you're actually taking advantage of an existing user behavior and existing market and like selling a great product into it. I think there's one other aspect, which is, look, if you're a developer and you have access to the latest AI technology, the first problem you solve are your own problems, right? So I think it's developers just, that's the problems they understand best. That's the problems they face every day. And so they build infrastructure for themselves to use.

And developers are always early adopters for new technologies, just because like naturally they like to tinker, they like to configure new tools and they're lazy. So anything that actually increases productivity, they will adopt. But I also think the coding market is doing so well also because it's somewhat verifiable problem. Like you can verify if like a coding function, like input and output are very clear compared to like user preference and all the other problems.

And you can also reframe a lot of different problems into coding. So I would even argue that some of the art generation is a coding problem. People will not like it, but historically, we always use machine learning before we call it AI in Adobe Photoshop. And that's somewhat coding. You can really map the trajectory of brushes. That's coding. Vector generation is also coding. So

So I think the beauty of code is that you can really remodel a lot of the real-world problems and make it into very machine-consumable formats. I think there's one other aspect, which is it's a massive market. If you think about this, we have 30 million developers worldwide. Let's say average value created by a developer is $100,000 a year. That's $3 trillion worldwide.

I think if I look at the data we've seen from some of the large financial institutions, they're estimating that the increase in developer productivity from just a vanilla copilot deployment is something like 15%. My gut feeling is we can get that substantially higher. Let's assume we can double the productivity of a developer for all developers in the world.

That's $3 trillion worth of value, right? That's the value of Apple computer or something like that. It's an incredible amount of value that we unlock there. So it's a massive market. And I mean, there was, I think it was last year when there was a good blog debate on over-invested in AI. And I think back then the number was $200 billion annual investment, you know, something we would ever recuperate.

Here we have a way to recuperate $3 trillion. So that makes the $200 billion look like peanuts. I think what might also be hard to work is it's an easy market to capture because developers understand it. And it's a very, very big market, something potentially, it might be the first really large market for AI in terms of fabric. Yeah, no, that's a great point. Software and software developers create a huge amount of value at every company and every organization around the planet now. And this is sort of a shortcut into this sort of core capability.

So that makes a lot of sense. There's almost a bootstrapping effect to your point too, Guido, because if you count not only the productivity gains, but like the brand new things that are being created with these models, like you kind of, I think, can see a cycle starting where you're getting better and better AI coding models, which allow you to create better software, you know, better new, net new AI applications also. Okay. So AI generated code will generate a lot of value going forward.

But how much has it already changed the day-to-day experience of programming? Once the AI revolution has run its course, do we have any idea how the job of a software developer will look like? It'll look different from today. I think we're seeing that when I'm writing code today, I

I'm writing specifications, I'm having discussions with a model about how to implement something. For easy features, I can actually ask it to implement a feature and just reviewing it. How will the process be different? Will there still be the same stages? Will we all turn to basically product managers that write specifications and then the AI writes the code and you just step in occasionally to debug? Or what's the end state here? Do we have any idea at this point? Or we all become QA engineers and we test if it's to the spec.

There we go. That is kind of ironic, right? We all got into this to avoid being QA engineers. I like what you're saying, Guido. Maybe we could each just talk a little bit about how we use AI models in coding right now. Can you share a couple of stories about how this has changed your coding workflows? Totally. And I mean, well, I'm not sure I'm even the, I'm probably the person coding least here, but I think the most interesting insight is it has changed a lot over the last even six months, how I use these models. So it used to be that

You take your favorite chat GPT or something like that, and you give it a prompt, and out comes a problem. You copy that into your editor, and you see if it works. Right. That's sort of the stack overflow replacement thing. Exactly. It's like when you inevitably hit a problem, instead of going to stack overflow, you go to chat GPT, and it actually gives you code back. Copy-paste, but from a different source. Yeah, yeah. And this was like six months ago. This was state-of-the-art. This wasn't like that long ago. Maybe nine months. Nine months ago, yeah. Yeah, nine months. But then, so then the next step was you started having integrated...

things that are integrated in your IDE, right? GitHub Copilot and Cursor that basically allows you to use autocomplete, which is a big step forward, right? It's no longer like monolithic questions, but it's sort of in the flow. Then this split up into...

autocomplete at a line level, I can ask questions about paragraphs, or I can have a separate chat interface where I can have longer discussions. Then the IDE started to be able to use command line tools. So suddenly I can say, "Hey, can you set up my new Python project with UV?" Or something like that. And it could basically run commands to do all that work. And I think where we are today is when I want to write a new piece of software, or this is not production code, right? This is like, but I want to try something out.

The first thing I do is I start writing a spec, right? I'm starting basically at a very high level. Here's what I'd like to do. And it's still fairly abstract and not very well thought through. And then I basically ask the model, maybe something like Sonnet 3.5 or 3.7 or Gemini. Here's what I'd like to do. Does this make sense? Please ask any questions that are unclear and then write me a more detailed spec.

And then the model gets to work. Usually it has lots of questions for me. It's like, hey, I need an API key for that. You know, very simple things. Or more complex things like, how do you want to manage state? Should we put this in a database? Should we just dump it into a file or something like that? And so it's basically a back and forth discussion that helps me clarify my thinking. And the model is almost a sparrings partner to think through the process.

which is really weird in a way, but it works, right? And so over time, you basically get more detailed specs. Not only when you have them, then you ask the model to start implementing. And all of that comes with a fair amount of context. Also together with the model, I have my standard Python coding guidelines. This is how I like to do commenting. This is how I like to do more object-oriented versus more procedural. This is how I like to structure my classes. I'm an object-oriented guy. We're talking Java here or what? No, Python. I'm a...

Do you want to have type Python or untyped Python? All these things. So it's a lot about context. It's a lot about explaining your general development methodology. It's a lot about a back and forth with a model now where you're sort of together explaining

figure out something. That's how I'm coding. How are you coding? I guess like compared to maybe six months ago, how I use Coding Agent nowadays is I give it more of the world knowledge. Before I was mostly relying on what's the foundation of model's knowledge. And it's funny because when you ask the Coding Agent, "When do you think today is?" It's always like 2023. And then all the specs that we'll give you are from like 2024 at best, depending on when the knowledge cutoff is.

Nowadays, I think it's very natural for me to reach out to Linear. Here's the ticket, and I just give my idea, pull my idea into Cursor. Cursor agent will take a first step at implementing it. So that's one kind of workflow change. The other one is more user-prompted active queries. So

Before, I may need to copy paste documentation into my little cursor window. Now, I just ask the cursor agent to like, hey, can you use FireCrawl to go search for the most up-to-date like clerk documentation? And it will actually fetch a page and they will, you know, read it. Oh, that's cool. That works. Yeah, it actually works. And then it will tell you. Is this using MCP or? It uses MCP, but it's like an implementation detail. It could be a tool call or whatever, but there's more integration to the real world now. You guys sound much more planful than me. Yeah.

The scenario for me is like Saturday night, I finally have an hour free and I have a weird idea for an app and I just dive right into it and ask Cursor to do everything. And I've always found it works really well for...

high complexity, high kind of annoyance factor things like front end. Like if anybody on earth can remember all of the CSS classes that people use now for margins and padding, it's like it's, you know, I don't think that person exists. Is that AI-centered yet?

Yes. Oh, yeah. Oh, we should do a benchmark on centering your div. Totally. Yeah. Tutorial on div centering. I mean, it's one of these hard problems for no reason. Yeah. Right? There's just like five different ways to center text and elements. And I can never remember any of them. And, you know, AI models are really good at this. Right? And they now can do it. When you start going to more niche areas,

libraries and function calls, that's where I always run into trouble. So I love this fire crawl kind of idea because usually I'm hunting for docs and then putting them back in or something like that. Yeah. Sometimes I also copy paste like a Medlify doc because they have the LMS.txt like the developer tool docs. I just give the URL at doc and then enter the URL and ask cursor to implement that.

And that works too. Has anything gone really wrong for you guys yet doing sort of AI-assisted coding? Not really wrong per se, but a lot of how we code is dependent on the agent behavior, on how the client implemented the agent. One example is

there's this very cool tool that actually generate very pretty pages and send back a React component, like an HTML page for the coding agent to refer to. So one time I asked Cursor Agent to reach out to this tool, implement based on whatever it told you. Cursor Agent reaction was very interesting. It looked at the code, it says, "Oh, this looks great. Let me give you a new version." So it didn't adopt whatever that was returned.

Interesting. Yeah, which is like a very interesting agent-to-agent communication. Crystal agent is like, I don't agree with this direction. But surely, serious engineers wouldn't lower themselves to use AI-generated code, right? Well, there's at least a strong argument that they should consider it, depending on what they need to do and what a model can reasonably output.

So you've done a bunch of work on MCP, Yoko. How do you think that plays into this? I think MCP, to its essence, is just a way to provide context, the most relevant context to LLMs. So it helps that a long tail of MCP servers nowadays can be leveraged whatever client you're using. So that's what's empowering the kind of experience I was just describing. I can use

Linear MCP, I can use GitHub MCP to point the relevant context. And tool calling is like a technical detail on how they implement it, fetching the context. But the crux of the MCP is actually the context part. What is the most relevant thing I can provide to you as a model so you can help me better? And so do you think having these kinds of tools available in an IDE means...

AI coding is kind of more productive or a better fit for kind of senior developers? Because a knock against this for a long time has been that vibe coders, for lack of a better word, are kind of

producing great demos and junior developers are kind of getting up to speed faster. But the people I've always affectionately called neckbeards, right? The people who, you know, own the cluster and stop you from breaking things or like own the overall architecture are sort of skeptics. Do you think this is one way to get the neckbeards engaged? I think so.

I think it depends on what the very senior engineers are optimizing for. There are very senior application engineers who are just very good at fleshing out ideas. So in this case, it's like a more evenly distributed skill set. You just need to put the stack together. But there are very senior engineers who are, say, optimizing, best in the world for optimizing for distributed systems.

That, I think, we're not quite there yet, just because the coding agent, first, it can't fetch any and all state of the distributed system. It's a lot of human intervention when it comes to how to solve certain problems. But I feel like we're on the way there, given enough context window, enough tool calling capabilities to bring just the right knowledge into the model. Today, I think most IDEs have

have a limit on the number of tools they can handle. I remember it was like 40 or 50 or something. So it naturally limits what's the context and what's the tools that the coding agent can leverage. Yeah.

I think that there's sort of a pattern that the more esoteric the problem is, the more novel the problems you're trying to solve, the more context you have to provide. If I'm like, "Hey, write me a blog," or, you know, what is it? "Write me an online store," like the simplified version. That's sort of a standard, I don't know, undergrad software development class problem. So the amount of samples on the internet is more or less infinite. The models have seen this a gazillion times and are incredibly good in regurgitating this code.

if you have something for which there's very little training code, that typically all goes away and it's all sort of

you have to specify exactly what you want. You need to provide the context, you need to provide the API specifications, it's much, much harder. And it will very confidently give you a wrong answer too. I can't tell you the number of times. I'm like, "Oh my God, this function exists. I had no idea. It's exactly what I needed." It's like, "Wait, no, it doesn't exist." Once it does that, it's very hard to get it off. If you're saying, "The function doesn't exist, it hallucinates a new one." It's like, "Oh, I'm so sorry. Here's another function that doesn't exist that might work."

I think what models today are very bad at is telling you if they don't know something. Yeah. Do you think RL will change that in the training process? If you theoretically give it all the relevant environments in the world, it can do all the things it needs to do to simulate a distributed system and debug it. I think in the extreme case, if you were the first person on the planet to write code that solves a specific problem, there's just zero training data. I think it will always be very hard.

right i think the the models are not really creative so far they can do a little bit of transfer but but not much so you know if you're say there's a brand new chip which has a new architecture and you're the first one to write a driver for it it's going to be a fairly manual task

I think the good news is that is 0.01% of all software development, right? For the, I don't know, you know, 100,000 ERP system implementation or so, right? That we have tons of training data. And I think these tools can be very, very powerful. As AI coding tools continue to improve, it's natural to question what it even means to learn computer science. Is prompt engineering a more important skill than a deep knowledge of how computer systems work? The short answer, no, it's not.

Because to truly optimize, troubleshoot, and scale out even the simplest application, you must be able to look deeper than a surface-level abstraction.

We haven't talked about vibe coding too much, but right. But there's this idea that people who aren't developers can now kind of write code, which is pretty cool, right? And sort of feels like something that should happen. We're not like priests of the computer where we need to intervene between ordinary people and the processor, right? It should be that anybody... You've been indoctrinated before you... Yeah, exactly. There's no seminary of... Well, maybe there were seminaries of computers. I don't know. CS departments. Yeah, exactly.

But it kind of makes sense that people should be able to control computers in direct ways, not just in sort of pre-baked programs that have been given to them. So this is, I think, a super interesting and super exciting thing. I think there's a question there is,

Is that true at all scales? Or is this a little bit like, look, everybody can build a shack, but you cannot build a skyscraper, right? Well, so this is exactly why I bring it up, right? The demos that everybody does their first time they're trying, you know, a website generator or cursor or something like that probably are not

doing that much for the rest of humanity, right? This sort of first weekend project. But if you assume some portion of people who give that a try, maybe start to climb the ladder and do increasingly sophisticated things. And by the way, in a totally different way than from the three of us would probably do it, having learned sort of programming the hard way. I just have a ton of optimism that that creates all sorts of kind of new things. You know, you have a new pool of people writing software in a new way.

who may look at the world in a completely new way. I just have a ton of optimism that gives you kind of new stuff, new applications, new programs, and new ways of using computers and computing that we haven't had before. This actually reminds me a lot of the 2000s, like when blog was the new word on the block and everyone was like, I need a new blog. And then we rushed to create our own blog and there comes like WordPress. People are still using WordPress, by the way. I'm surprised by that. And this wave of vibe coding almost felt like everyone and

my mom and my mom's neighbor are like trying to, you know, use the models to create personal software. So like where it came from personal static content to like personal CRM to like manage your relationship or something. How deep do the software go? Like, I don't know. I don't think it's very deep, but it doesn't matter. Like as long as there's like personal utility.

I think Martin tweeted about this earlier. He was like, you should still learn to code. If you're operating on one abstraction, you need to learn abstraction lower than where you're operating from, which is very fair. And I keep coming back to that because I wonder, what is the one level lower abstraction for Vive coders? Is that code? Is that IDE? Is that something else? But curious about your guys' take. I think this is a super good question. And

Let me try to rephrase the question a little bit. Like, what is the thing that future companies

people that want to do software development need to learn, right? Is it one level deeper? Is it actually something that's sitting more to the side? I mean, there's, especially some people say, look, there's no point in learning CS anymore. It's all about social, emotional learning and the kind of things. I'm not sure I agree with that, right? But it's- I feel like that comes up every 20 years or so. Definitely, there's a cycle there, yeah. Honestly, I have absolutely no idea how the equivalent of computer science education will look like in five years, right? When we're on the other side of this. You'll probably-

I mean, historically, what happened when we did similar things, say, with calculation? When we went from adding numbers manually to Excel, right? It's not that...

the whole job category disappeared, right? It's more that bookkeepers became accountants or something like that, right? Entering data and writing down numbers and adding them manually became less important and doing higher level, more abstract concepts became more important. So if a pattern matched that one to one, you know, the guess would be that explaining the problem statement, explaining the algorithmic foundations, explaining architecture and explaining data flow is getting more important.

And then integrated coding, you know, what's the most clever way to unroll the for loop? That's a very specialized, more niche discipline. It does almost feel like we're waiting for something, doesn't it? Like, if you think about a sort of classical computer science undergraduate education, you

you don't just learn kind of the latest thing. At least in a lot of programs, you learn, you may do a semester of sort of assembly, right? - You actually learn all the oldest things. - Yeah, you start with the old, or you know, we even had to take like a processors course, right? And I'm the world's worst computer engineer, but I got in there and I was like connecting gates and like that was fun. So you learn like how processors work, you learn assembly,

We did a course on Lisp, which was cool. We did file systems and some bits of operating systems and you learn Java, like Java was state-of-the-art at the time. That's why I mentioned it before, not anymore, obviously. So it's tempting to say this is like the next thing

that is built on top of those things and that you would learn to code kind of only for historical reasons or for educational reasons. I just don't know yet if that's actually true. Like a lot of the kind of layers we've added on top over the course of decades are things that truly are a new programming interface. AI is not actually a programming interface, right? It's not actually a framework.

It's sort of a tool that uses things you already, helps you use things you already have. So it just makes me wonder if we're waiting for kind of the next iteration of this thing, like the thing that AI actually can change about the way computers are programmed. For instance, it could just be prompts that are somehow translated to code in a more direct way. And agents, as we see them now, are kind of a starting point there. So that's what I'm sort of curious about. What about the future of programming languages?

how does our relationship with formal languages like python or java evolve if we can generate code using natural language today i think ai is not just a higher level language abstraction or something like that but could it become one over time that's the question do you do you think yes i i think it could i mean look i think we really haven't figured that out yet i mean if i look at a classic say compiler design or in programming languages if i would have llms as a tool

I would probably think about very differently about how I would build a compiler. And I don't think we've seen that work its way through yet. I have no idea how it's going to look like, right? But if I can basically define certain things in human language in an efficient way,

And maybe in a sort of tight enough way that can use this directly as input for a compiler. That could change a lot of things. Analogy here would be like, because a lot of companies are building agent-based systems. And then when you take a look at that system, when you see what the agents are building, you're like, oh, this is what I learned in operating system class many years ago. These are processes. One process for

fork another one and then hands the task to another one and then something else manage the resource of the system. I don't think we have the framework. This is why I think the CS education will not go away because it gives you a way to compare the two things. Otherwise, you wouldn't have known there's a thing called process in the first place. But at the same time, I don't think on top of the foundational model, we have invented the paradigm to make that work as if it's an operating system. Formal languages exist for a reason, right?

Right, I guess is the one thing I would say. Whether that's a programming language or a specification language or something like that, there has to be some high bandwidth, expressive way for a person to design software or anything but software in this case. So I just have a hard time seeing Python going away or programming languages going away entirely. To Yoko, your point about you have to understand at least one level of abstraction down. It

It is an interesting question if some will be more popular than others because they're kind of more AI native in a way. We're sort of seeing Python and JavaScript are kind of leading the pack right now, but it's not clear. Tooling, I think, is another really interesting thing. Like we're seeing a bunch of new Python tooling come out right now, which is kind of cool because the Python ecosystem is kind of more active than ever. And you can imagine that sort of has an impact on, you know, how well does it work with kind of the AI add-ons to the language too. So I just don't think we can toss these things out completely.

I think the reason behind you need to know a level, like an abstraction level deeper is if and when you need to do optimization on the system you're writing, you just need to know how to optimize that. If you don't, then you really don't need to know. Like there's a lot of people who back in the days coding Java and never heard of, you know, JVM or know how it worked. Just...

Like creating a calculator using Java, you don't need to know JVM. But if you want to optimize for runtime threading, you do need to know JVM. It's very similar with the vibe coding use cases. If you're just building a marketing website, I don't think you need to know the next level of optimization. Like unless you're serving something at scale, then you probably need to know what CDNs are, you know, how to cache pages, things like that.

But at the same time, like if you're someone who aspire to serve something at scale and then want to flesh out the real service one day, it's really hard to get away without knowing the underlying knobs because the essence is

There are certain things computers can do, and these things are defined by formal languages. One language is buried under the other. And then to touch these knobs and to know what to even do, you need to know these languages. Yeah, so curious about your take too, Greedo. I agree. I think formal languages won't go away because ultimately they seem...

complicated but i think effectively a form language often the simplest type representation you can you can find to specify intent doing that in a language like natural language is often very imprecise and you need a lot more words to to get the to get the same result i mean i think the the interesting question at the moment is

Are there cases where AI has enough context from understanding humans and enough context from you inserting clever ad symbols and pulling in additional pages that it can take for a certain subset of problems natural language descriptions and translate it accurately? And I mean, it seems like there are areas where that's possible, right? That's what we're using every day when we use AI for coding.

So can you hybridize that somehow? Did you actually create a language of that type? I don't know yet. I mean, your distinction is really interesting, right? Like this word complicated is sort of overloaded. In one sense, it can mean

a highly complex system that has a lot of pieces and you never quite know how it's going to behave. On the other hand, it may mean just kind of hard to use, right? And I think people sometimes see programming languages complicated in the sense that they're hard to use or hard to learn, right? You need to learn this kind of new language to speak in. They're actually very simple, right? You can draw a tree that sort of encapsulates the entire set of things that can be expressed in that language. So

It's funny, we're switching to this thing called AI coding that's easier to use, but actually much more complicated under the hood. You know, insert meme about giant green monster with a mask on like here. So to your point, it's like, how do you sort of handle that? And is it some hybrid solution or something else? I know the guys at Cursor have always talked about kind of

formal specifications, which I think you alluded to also, Guido, as kind of like writing a spec in a really clear way is kind of the task that people will be faced with more and more over time. It's almost like an annealing process between you and the AI to go from some loosely formed model that you have and a loosely formed model that the AI has to a tight spec that you can...

implement at the end of the day. This is so true. I talked to a classical vibe coder recently. And then, because like my question was, do you really need the coding interface? Like, you know, how you enter a prompt, a generous bunch of code. And this vibe coder's answer was so interesting. He said, yeah,

I like that the AI is generating code and showing me. It's very empowering for me to see that I generated all this code. But when I want to go in and actually change something myself, I don't know where to start.

So it does tell me that there is a gap between what the AI generated and where live coders operate. It does feel like there is a product somewhere between. We want to give them the power to actually change the underline knobs too. I mean, this is not restricted to people who are

who are not experienced programmers, by the way, like if one of us tried to vibe code an app, after four to five turns, if you went in to try to edit the code, it would be very difficult. It's very opaque what's going on. I ran into this when I was trying out the Blender MCP. I've never used Blender before. It's just a really hard piece of software to get into.

So I installed the MCP server on my cursor IDE, and then I was able to prompt like a mini statue of A16Z infra just very easily.

But when it comes to modifying this 3D representation, that's where things start to break apart. I don't even know where to start, why I need to modify things. Yeah, like a flat surface has 10,000 polygons. Exactly, and the textures. Yeah. But there's a lot of opportunities here, kind of existing in the gaps of AI and vibe coders and what the representation is today. What's really cool about this is you're sort of creating a new layer of context and a new layer of intent. Yeah.

in software programming that didn't exist before. So for instance, can AI help port old code? This is one of these, like the banks have been trying to drop COBOL for 100 years or something like that. And firstly, I think the answer is kind of no. It can definitely help, but it doesn't solve the hard problem. And I mean that in the following way. AI may be able to transpile COBOL to Java, but there's

But there's a huge amount of context in what went into creating that COBOL code that's been totally lost, right? Over the course of decades, in many cases, something that started as an airline booking system became an airline booking plus HR plus, you know, fetch the coffee system.

And many of the people who contributed to it, and by the way, didn't write a lot of documentation or comments, may not be around anymore at the company or, you know, on this earth, right? And so this is a problem that AI, I think, can help and not solve. But what's actually even more interesting about this to me, when you talk about specifications, is like,

If they'd been using AI at the time to create those systems, there would be this whole other record of what their intention was when they were creating the software that kind of comes for free, right? It's not something you have to go back and do. And I think that's something that's kind of cool now. Like if we see this kind of take off more and more, you have this kind of other set of metadata that can capture the software intent in a slightly different way. It's almost like a high level language abstraction, isn't it?

But it's, I think it's different, right? Like, because it doesn't, like, you can't compile it down. You know what I mean? Sort of. I agree, it's sort of. But it's actually, you're raising a very interesting point there. I recently talked to some large enterprises that are using AI to basically take legacy code bases, specifically mainframes, so COBOL and PL1s, the other good one there, and move that to more modern languages. And

And it's super interesting. They have exactly the issue that you described, which is that if you just look at the old code base, you often have no idea what the intent was. And if you just try to translate that,

you pick up many of the idiosyncrasies of that old programming language, right? I mean, Java has much more modern constructs that you didn't have in Fortran. Maybe you want to use some of COBOL. Maybe you want to use some of those. So what I've heard from now multiple organizations that they're saying the most efficient way for them is to actually go first and try to create a spec. Use the AI to create a spec from that code, right? And once they have the spec, then to re-implement the spec, right?

And that gets them much better results, much more compact code, much more modern code than what they had originally. And that is sort of an AI-assisted problem for sure. Both of those problems, I think. Yes, it is. That's very cool. That's interesting because I was actually just thinking about it's actually much easier to rewrite modern software, like modern meaning something in the past 10 years. It's like easier to implement from Angular to React, especially both frameworks are well understood by the agent. It's much harder

Harder if the state... PHP to Angular is really hard. PHP to Angular. Yeah, PHP. I mean, Laravel is working out pretty well, so that one's easier, depends on what kind of framework you're using. It's much harder if the state, one, is spanning across many software systems, because you just need to do some discovery or have an agent that can have access to this discovery. I can see that working out.

And two, there's specificities on the hardware some of these things are running on. Like, for example, for the runtime, maybe I give it enough memory for this Docker container. I need to have specific configs to make this work. Sometimes, like, to your point, all of that is lost. Until the day we can take a snapshot of the runtime, like, how is this run? What's the requirement of this? It's hard to migrate systems like that.

I'm now getting like pre-nightmares of like something goes down in prod and you're digging through the chat GPT logs to like try to figure out what someone might have accidentally tried to do. Even if we never fully lose our connection to the fundamentals of computer science and programming, some things are going to change as a result of AI generated code. And one of them might be how we balance certainty or reliability with utility when it comes to deploying non-deterministic systems.

Guido, I have an interesting question for you. If you think of AI as a primitive in an application, not just a tool to write code,

It does seem like it's kind of pushing the frontier of the degree of uncertainty and non-deterministic behavior we can have in software, right? Like if you think like really old days, probably predating a lot of us and our listeners, you know, you just write software for like a local machine and you could have a pretty good expectation of how it was going to execute. We had this new thing called the network.

right, which is very hard to predict how it's going to behave. But you can kind of express it in the same terms. It feels like a problem that you can wrap your arms around. It feels like AI is kind of an extension of that in a way where like, you actually don't know what's going to happen when you add AI into your software or use its right code or whatever. How do you think about that? Do you think that's a reasonable way to look at it? And are there any lessons from the networking world that would help us figure out what's going to happen in AI?

Yeah, I mean, I want to say probably because I don't think we have the lessons fully digested yet. When I went to network systems, there were sort of new...

failure modes like timeouts and then new remedies for this like retries. And once you go to sort of distributed database, you have to worry about atomicity and rollbacks in a digital context. Things got very complicated very quickly. And I think for some of these design, some of the design patterns today, even today, we don't have very good software architectures yet, right? They're still... And they may be kind of unsolvable, some of these problems, right?

I mean, I think the fundamental problem is not solvable, but you can at least make it as easy as possible for a developer, right? I mean, everything is just tools for the developer to cushion some of the blow. Models are funny because at temperature zero, a model is technically deterministic, right? So it's not so much that different inputs...

that the same input will result in different outputs. That's something we do by choice. I think the bigger problem is that an infinitesimally small change in the input can have an arbitrary large effect. So it's a chaotic system, you're saying? A chaotic system, exactly. The user could put anything into the text box and the system is chaotic enough that you get, you know, like it used to be, you just had to check for apostrophes and then you could execute a database statement. Like there's only a few things that could break a text box. Now, like kind of anything could happen when someone enters text.

That's right. Ignore all previous instructions. But that's a really interesting thing you're saying, that it may be the case that we just need to expose the primitives and capabilities of the system in a way that developers can use, not necessarily tamp down all the failure modes, you know, the equivalent of a timeout, for instance. I think that's one part of it. But I think we also have to change our expectations. So there was, I talked to one large bank, and

And they implemented software to basically generate text. And, you know, one of the important things in financial institutions is never give investment advice, right? And so you're trying to have an LM that is very helpful and never even implicitly gives investment advice. That's often an unsolvable problem, right? You can get better and better and better, but you can never completely rule it out. And you can add a second LM that tries to catch it, but it also will occasionally not catch something because it thinks it's helpful.

And at the end of the day, they basically made a decision to say, you know, we can't build a software system that never does this. We have to change our metrics. We basically have to go. And I think they ended up with something like it has to be whatever half the probability of a human of a well-trained human doing the same situation. Zooming out a bit more, maybe you've heard the analogy that Internet protocol is the narrow waste of the Internet, often visualized like an hourglass, with IP being the standard in the middle that allows interoperability between Internet infrastructure and applications.

Does AI have or need its own narrow waste? Is the prompt up to the job? If we were to zoom out a little bit, you were there for the whole rise of internet history and then you were a pioneer on a lot of the networking research.

So how the internet came to be is there is a narrow waste of the internet. Somehow that happened. Do you think it would be a similar dynamics playing out in AI at all? Like, is there an analogy? Maybe the waste is never narrow for AI. I think we have the waste, the narrow waste. I think it's the prompt. Oh, interesting. Why is that the case? I mean, look, typically these big tech cycles are built on abstractions that allow you to encapsulate the complexity underneath in a very narrow API for AI.

say a database of a SYCL, the early database of the transaction databases, where how does the database under the SYCL query work? It's something with b* trees. We learned that in grad school, but that doesn't really matter anymore. I just need to be able to specify the query.

And I think that's the same thing that led to the rise of modern ML, right? You no longer need the overpaid staff or a PhD that trains a model for you, but instead you can now express and steer the model with a prompt. And so a fairly, say, mediocre Python programmer can suddenly leverage a very powerful LLM just by prompting it.

Interesting. If you were to double click on the prompts, do you think it's like a natural language representation of what you want to do? Because there's no standard there. It's like prompts can be anything and everything. It's like hardly a narrow waste. It's not a formal language. It's clearly not like

English either though, right? It makes me think. - That's right, yeah. - Latents. - Yeah, I mean, we're all learning kind of a new language in order to prompt these things. And actually it's a little different for each model. So dialects, you know, we've got like a translation issue, all that kind of stuff. - I mean, will we ever have a formal prompting language maybe? - I think there are some overpriced Stanford PhDs working on that problem. I'm hopeful to see what they come up with. - Are Asian frameworks formal prompting languages?

I think a little bit. A little bit, yeah. I mean, we're certainly starting to see prompts with structure where it's like, I don't know, user something, agent response or something like that, or, you know, think and think, begin of the answer, end of the answer. And you say, okay, good. It's like two tags in a lot of text.

That's not very formal, but I think the first starting points are there. You could see future models getting trained and fine-tuned on a more structured representation. Oh, yeah. We're already seeing this happening, right? There's models. Every model has JSON mode now. And then how you define what you want out of the JSON mode is like you give it a type system. It's like you can prompt like, I want you to generate three fruits, but only return it to a fruits column, like,

you know, apples, like fruit, like types of fruits. And then you could define it in your code saying, I only want your answer to be, you know, have the fruits key. I don't want anything else. I guess that's like a kind of formalization. Long term, I actually wonder if the, like for a reasoning model, where a lot of the thinking sort of happens internally,

if the model is generating the user-facing machine-facing output it's going to be a different model from the model doing the reasoning if that makes sense right so the you know i i like a really a chatty model or somebody else wants a more tourist model or if you want to have generate json output you know one one have yet another model so you could see so the the model output layer delaminating at some point from the reasoning layer finally

A simple question whose ultimate answer will have enormous ramifications for the software development industry. Is there, or should there be, a natural bifurcation between vibe coding and enterprise software development?

In the future, do we think there's going to be different vibe coding models versus enterprise coding models? Actually, I don't think so. Well, I define vibe coding as you kind of let the model, you have a spec, you let the model generate whatever it needs to with the implementation detail. You don't care about the implementation, but you do care about what comes out of that implementation is what you wanted. So it's less formal, less constrained than classic coding? I think so. What is the difference between vibe coding and classic coding?

I think for classical coding, you have to make a lot more choices in what you want to put in the code. So I want to use this SDK and not the other one. For vibe coding, you just don't care about the underlying technical details as long as... You let the model drive. Yeah, as long as it gets things done. But you still care about the higher level needs. Otherwise, why are you writing this? Got it. So I can totally see enterprise users doing vibe coding. And that's a compliment.

And that's it for this episode. Thanks for listening. As always, if you enjoyed the episode or if you learned something, please do share the podcast among your friends, family, and colleagues and rate the podcast at your platform of choice. And keep listening. We've got more great episodes coming up.