If 10% of the World Were Developers: GitHub’s Mario Rodriguez
28:47 Share

Today, we're airing an episode produced by our friends at the Modern CTO Podcast, who were kind enough to have me on recently as a guest. We talked about the rise of generative AI, what it means to be successful with technology, and some considerations for leaders to think about as they shepherd technology implementation efforts. Find the Modern CTO Podcast on Apple Podcast, Spotify, or wherever you get your podcast.

Stable products typically require extensive testing to make sure they're reliable for customers. But products based on inherently changing tools, like generative AI, fundamentally change that. Find out how on today's episode. I'm Mario Rodriguez from GitHub, and you're listening to Me, Myself, and AI. Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI.

I'm Sam Ransbotham, Professor of Analytics at Boston College. I'm also the AI and Business Strategy Guest Editor at MIT Sloan Management Review.

And I'm Sherwin Korubande, senior partner with BCG and one of the leaders of our AI business. Together, MIT SMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

Hello everyone. Today, Sam and I are excited to be speaking with Mario Rodriguez, Senior Vice President of Product at GitHub. Mario, welcome to the show. Hello, how are we doing?

We're doing great. I think most of our listeners know what GitHub is, but can you describe it briefly for us? Yeah, absolutely. So we usually describe GitHub as the home of all developers. It is the place where you store your code and other assets that are necessary to develop and ship software. It's also a place where you come to collaborate and remix ideas with the open source community.

I have two daughters. One of them is six years old. And sometimes the way that I describe what daddy does is my daughter has a book bag and she puts a bunch of things necessary in there. You know, her lunch, her snack and her folder. She has a purple folder where all the homework goes and things like that. And that is the space for her to accomplish the stuff that she needs to do at school.

And think about GitHub being that for developers. It's the place where you come and put everything in order to develop and ship software. And we're lucky to have an amazing community on top of it.

And my role is I am the SVP of product at GitHub and I oversee both our product and design teams, including GitHub Copilot. I joined GitHub as part of the Microsoft acquisition in 2018. In fact, I've been working in DevTools for 20 plus years.

I consider myself super lucky in the sense that I love what I do for a living. I sometimes have to pinch myself on that. I think about it.

Humanity already created the wheel, created fire. Everything that we're going to be doing going forward to advance it is going to be through software. And giving these software developers amazing tools to continue to push humanity forward is a lucky place to be at.

And I just love that mission and that drive. So considering myself lucky in that. Right now I'm overseeing our product and design teams and making sure that we continue to innovate and we continue to meet the needs of our customers.

I couldn't agree more with what you said about the importance of enabling developers to collaborate more, particularly as everything is moving more and more towards that. Tell us more about what the product group does.

Product management, maybe this is a little bit controversial out there, but it's a very new discipline, in my opinion. It's not something super big. Like we know how to do agriculture as an example. Developers have been doing stuff for a very long time too, and in one way or another. But product management as a discipline is, I think it's probably new. I think of products...

role overall is to take something like an outcome that you want to achieve and then figure out how to get the product to be able to accomplish that outcome. And it's magical to do that. People think it's a set of features and you then get into this thing called feature factories, if you have heard that term. And I think that's

That's not right. Like the product discipline itself is more art than science many times. You're not there just to run stand-ups or you're not there just to talk to customers either. There's this beauty of the product discipline that is all about taking what does a customer want to accomplish or what does the business want to accomplish, but for them are super valid, right? And then figuring out how to get there.

at least from my end, I love to do it incrementally. It's very hard to be right all the time. And for me, what I always tell the product team is, how can we actually...

figured out what is the next incremental thing that we could do to validate a set of assumptions to try to get us there. And that incrementality doesn't have to show up to their customer right away. We do this thing at GitHub called Staff Ship. So we ship it first to ourselves, but that is a point where we could validate the idea. And that is a point where we could then figure out did this work or this didn't, and what's the next ref of the idea to accomplish that outcome.

I usually say a product team is really there to take these ideas, marry with outcomes, and they fill the middle of that equation. And they do it iteratively, not just like get it right the first time, right? Which is, I think, what you're talking about as the mental shift in life. It's not a big bang. And as the more we get comfortable, and if you look at some of the greatest products have been introduced and evolved over many, many iterations.

A hundred percent. And you build on top of each other too. Like you might try this, by the way, the same thing, in my opinion, happens in medicine and in other industries as well that do it very well. It's,

The biggest breakthroughs have happened, sometimes by chance, by just testing a bunch of things, right? Thomas Edison and the light bulb, right? It took a lot of iterations to get there. And you would think we would have learned from now that that is the way to accomplish the biggest breakthroughs is just through that iteration. But you do get, at times, mistakes.

a set of beliefs that product management is all about managing a project and things like that. And the reality is it's just about iteration and finding the next set of things that would accomplish the outcome that you want, 100%. So in that middle ground, you mentioned earlier Copilot. Where does Copilot fit in that middle ground? Actually, tell people what Copilot is to start with. And then how does that fit in the middle ground?

GitHub Copilot is our AI assistant, and we're kind of a little bit proud about this. We were the first Copilot out there. And if you were internally within GitHub at the time that we started this, it was a big memo at one moment done by a set of researchers at GitHub. We have this division called GitHub Next.

And they oversee Horizon 2, Horizon 3 set of investments for GitHub. And we had just started to play with some of the models, some of these new LLM models coming from OpenAI. And the paper was about that is, can we create this AI coding assistant that can really help you and supercharge you overall?

So GitHub Copilot is the brand name for that. And we've been very successful with, again, going back into that iteration, iterating towards amazing value to developers.

We have stats like 55% improvement in productivity. And this one I'm really proud of. We try to ask a lot of our customers to really measure developer happiness overall. We know that if developers are unhappy in an organization, not a lot of things are happening, right?

You want to keep them happy. Talent out there is scarce, so you definitely want to keep your developers happy. We've been very successful in both the adoption side and the value side. But even more importantly, at least for me at times, is that developer happiness and increasing that across the world as well. I wanted to ask you about that. We've seen a variety of folks

get far more effective and efficient in writing and debugging and even thinking through like you know how to structure code and often things that won't even come across a human mind because of how we have sort of this maybe the herd thinking or whatever and sometimes you know the code is much simpler and much cleaner when you are in the co-pilot mode yet the

What I've seen and heard somewhat anecdotally is also like folks are adopting it and they're loving it. And then there are folks who just don't even go near it. And they're like, it's not for us. Why do you think that is? Is it fear? I'm going to lose my job and this thing is going to take over. I mean, why do you think there is maybe a little bit of a bifurcation of reactions to this? Yeah, I love the question. Sorry.

Maybe let's try to explore in three different angles. Number one, going back into also something that Sam had asked, how do we think about Copilot and the value that Copilot does? And how do we even do the product GitHub Copilot? I would say...

AI products and developing these AI products is completely different than any other products or features that we do. That has been one of the biggest shifts I think me, my team, and even GitHub has had to go through.

which is if you think about just a normal feature that we do, we understand, okay, this is kind of what the outcome that we want another feature. Here's some code. Well, we could write unit tests for that code, right? We could test it. We kind of found it's very deterministic. We know if something is working or not. We know if there's a bug or not. We know how to fix that many times that bug. We could see even the error that we make while developing the feature, right?

LLMs are not deterministic. They're not. And you cannot just expect them to be deterministic. So how do you end up providing value within a non-deterministic set of inputs that are at your disposal there? So

One of the biggest things that I would say to the community at large, if you're going to start developing features against, I would say AI features or features against LMS is you have to rethink how you're thinking about quality. You have to rethink how you're approaching the feature. We have all these evaluations and offline evaluations that we do for GitHub Copilot. And that's how we determine the quality of what is giving you in something like code completion.

So going back into the adoption side, we know that many times Copilu is going to be generating around 30%, 40%, 50 plus percent of your code, right? Like the acceptance that you have, depending on how many lines and stuff like that is in the 30 plus percent for many of these developers.

And we could see it in our offline evaluation as well that where Copilot does really, really well and where Copilot needs to continue to improve, meaning we probably need to pass it a lot more context for it to generate the right piece of code for you because Copilot

Code and code development has a lot to do with the context that you're dealing with. Are you in a legacy code base or not? Are you in COBOL or in UNC++ or in JavaScript or TypeScript? So it's a lot of context that needs to happen for the quality of that code to be high and for you to accept it. But we have this offline evaluation that we utilize for it. So a lot of the adoption today is coming because it's providing you value.

We strongly believe that the human has to be at the center at all times. But you have this copilot that is very patient, right? Like it doesn't get upset if you ask it, you know, how does it actually work to do X, Y, and Z within this code base? Or I forgot how to write this register. Can you just please tell me what's the best way of doing that? Tell me again, even though you told me eight times. Don't judge me. It is incredible.

incredibly patient and very empathetic, you know, and especially if you ask it, yeah, if you ask it with please and things like that, believe it or not, it does actually matter. And it does respond at times. So you're like, did it get spotted just by me asking please? But it does actually respond very well to empathy for sure.

So I do think it creates this productivity polarity out there. And that's why the adoption is high. It's because the people that use it end up being significantly more productive and you see the difference. You see the difference in their output. You see the difference at times in the cut quality. And then you see the difference as well in

their happiness because they don't have to write 10,000 lines of tests when Copilot can help them with that. And then there's this magical moments where you're like, just through a comment. And by the way, this is the thing that won me over back in the day. I think it was early 2021 when I first saw it.

Because we had IntelliSense before. We had a bunch of other technologies that could complete code for you, by the way. But we never had the ability to start something with a comment to describe what the next code should be.

I wonder if the IntelliSense example has anchored people. If I think about IntelliSense, it was about filling in, well, you started typing in for, and it would open parentheses for you, and you'd do that sort of stuff. But the Copilot is at a different level. And when I think about that, you mentioned a 30% code produced, 30% or 40%.

I'm guessing it's really not the interesting code. You know, I think there's probably a distinction here that that 30%. Is that really the case, though? I think it could be quite interesting code. It could be quite interesting code as well. Again, because of a comment that you could make, it could get you started. So you're right. It's not going to write the entirety, although...

with more of these agents that we're developing, more and more of that interesting code can be developed. But just with a human at the center, whatever they describe, and you have heard this term prompt engineering and all of those things, actually prompting Copilot can help it generate, and prompting Copilot the right way, can help it generate very interesting code. And we see this, we have a Slack channel internally where people post what Copilot did for them at times.

and you see some things in there that you're like, wow, it did that? Like, that is amazing. I guess I was thinking about, like, as we approach a task, the easy ones are going to be first and the later ones are going to be harder. A hundred percent. And then the latest one is going to be hard. Also, maybe let's try to explore this angle or this question too. I would love to get your opinions on this. So,

Imagine if the prompt to all of us is, what does the world look like where 10% of the population can code? Just think about that. Because sometimes I get out of bed asking myself these questions. It's like,

What needs to completely change for 10% of the population to be able to be called a developer? Because today you don't have a lot of people throughout the entirety of the world with a computer as an example, especially a very advanced graphics and CPU and all of those type of things. A lot of people have phones, but not a lot of people have computers in their house.

And so I think that's very interesting. Like, what needs to change? And for me, I believe, like, the definition of a software developer will start changing going forward. And the interesting part of it is, like, probably all of us know how to cook or all of us know how to bike or all of us know how to swim. But look, I am not opening a Michelin star restaurant tomorrow.

But the rest of humanity, what if a text box is all they needed to be able to accomplish something that creates software and something that they could then drive value on? Like, you would consider that, in my opinion, a software developer. If I could tell it very quickly in my phone,

Hey, I am thinking of talking to my daughter about these things. Can you give me the last three X, Y, and Z articles and then just create a little program that we could play as a game? Do you envision Copilot being able to help you with that in the future? Sure.

I'd be curious what that 10 percent is now. What percentage is there now? But one thing that's interesting if I combine what you just said with GitHub in general, you said a lot about, "Oh, I can make this thing do what I want to do, highly personalized."

But then GitHub is also about sharing that, which I think is a very interesting dynamic that would play out. If you could both create something personally and then widely distribute it for others to improve and iterate on, that seems even maybe even bigger than the 10 percent because you get the multiplicative of the 10 percent and then everyone else improving it.

A hundred percent. You know, the beauties of communities is that you can network effects on those communities, right? This is how languages work too. Like you can network effects because someone else knows your language and then you communicate more broadly. So the beauty of community is that, and that is what unlocks, in my opinion, creativity as well. And it unlocks innovation. Imagine then if this little program that you

compiler created for you, you could also share it and say, Hey, this is what I created. And you could remix it and you keep remixing and remixing, you know, artists do this all the time, by the way, right? Like if Picasso started not what you know today as cubism, like it started completely different. It started actually in very realistic, painting very realistic pictures and then kind of evolved from there and evolved from there until it found its own style.

And that's beautiful. So I think tools like Copilot are going to help you do that. And it's going to be unbelievable what our children are going to experience through the advance of this technology. And look, we also need to get significantly better in developing AI tools as well. We're in nascent mode right now. One thing that I really liked is

You mentioned the idea of sparking someone's interest. Maybe they see something and then that clicks for them and then they decide to go deeper. But this increased exposure to lots of things gives us more opportunity for things to click. I don't want my dean to listen to this, but I mean, I think that's a lot of what we do in the classroom is not so much teaching, but exposing to lots of ideas. And then some people, you know, this will click with one person, this will click with another.

I really liked your analogy there with coding, that maybe there's some part of it that clicks and sparks some interest, or maybe there's some thing that pushes someone to go forward. But I think it's also true, it makes sense too, that not everybody is going to be that superstar or wants to be that superstar. Mario, I wanted to ask you about your founding a charter school. Ooh, yes. We haven't talked about that. Tell us about that.

I was born in Cuba and I lived there until I was around 14 years old. I came to the United States

And I didn't speak any word of English. And in fact, I went directly into high school. So I highly recommend to go directly into high school without speaking the language. But we made it happen. And the reason I'm saying that is when I was in Cuba, my mother was a teacher at a university. And I spent a lot of time with her in kind of helping her grade tests and kind of just seeing what an educator does best.

And to tell you the truth, I was fascinated by it. My father also was a physics teacher at one moment. Then he eventually became an electrical engineer. And when I got to the United States, I was like, you know what, I'm probably going to study electrical engineering and also do something, maybe get a PhD. So eventually, if I want to go back to school, I could do that.

And all of a sudden, I find myself, you know, I was working at Microsoft and then eventually GitHub, and I was in North Carolina. And I noticed that the rural parts and certain parts of North Carolina, it was really hard to attract great teachers there and maintain that and continually to push education forward, especially in things like computers or programming and things like that. And we got together and

And we say, what if we open a STEM school with a slightly different take on it?

And we allowed the students to, you know how Google at times I think has 10% time or 20% time or something like that. What if they actually, some of these students can own their own curriculum at times and they could learn at their own pace on this 20% time as well. And what if we don't have so many walls between the classrooms and we could sometimes unite classrooms together and have them really share, to me talking about community, really share.

So we opened a charter school just to explore those ideas and to be able to provide better education in places where it's really hard to do that and sustain it at a high level. So think about iterative correctness. I think about the school as a product and you have to continuously try new things and try to see how it evolves. And we were so successful.

We just did a middle school to start. We were so successful with that that we are opening a high school now. We did that last year. I love education. I think it's a great way to give back overall. And then the more we could do then again in technology and arming these students with both sides, the art side and the science side, then the better we'll be. That's wonderful. That's a great story.

Let me transition here to, we have a segment where we ask you some short rapid fire questions. So just give us a quick answer here. What do you think the biggest opportunity is for AI right now? I think the biggest opportunity is productivity right now. And at least in software development, it will be democratizing software development overall. Yeah, that's interesting. You mentioned the productivity versus the democratization, which are very different twists on that. What's the biggest misconception people have right now with artificial intelligence? What are people getting wrong?

That it's going to take jobs away, I think is the biggest misconception in that it's going to actually then bring humanity backwards instead of forward. Yeah. What's the first career you wanted?

So my first career, I would say I was always an entrepreneur. So the first thing that I did to make money was actually selling, you know, selling things so you could do kites overall. So I sold these kite kits and that was the first thing that I ended up doing. So when is there too much AI? When the human is not at the center. Nice. What's one thing you wish AI could do right now that it can't do?

transpile code in any language that I want and do that with high quality. If that could happen, that would be amazing because then I could put something that I wrote in TypeScript and try to see, okay, how would that then, or in Python as an example, how would then that look like in these other languages? I always find fascinating different programming languages, what's good about them or not. You know, Lisp as an example, right? Very hard to read at times because all the open brackets, but

That's what I would say. That's interesting. So I think one of the things we've covered today is just the wide gamut of possibilities here. One of the things you've kind of opened my eyes to is this idea of what happens when even a percentage of people who can code in the world changes. What could that do? I think that's really interesting for us to think about, both positive and negative, that that has some big implications. Thanks for taking the time to talk with us today. We've enjoyed it. Thank you. Pleasure to be here.

Thanks for tuning in today. Next time, Sam and I meet Patrick Hebron. Please join us. Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders. And if you join us, you can chat with show creators and hosts, ask your own questions, share your insights,

and gain access to valuable resources about AI implementation from MIT SMR and BCG, you can access it by visiting mitsmr.com forward slash AI for Leaders. We'll put that link in the show notes and we hope to see you there.