This Crazy Wave We’re Riding: Walmart’s Prakhar Mehrotra and the Ups and Downs of AI
22:57 Share

Today we're airing an episode produced by our friends at the Modern CTO Podcast, who were kind enough to have me on recently as a guest. We talked about the rise of generative AI, what it means to be successful with technology, and some considerations for leaders to think about as they shepherd technology implementation efforts. Find the Modern CTO Podcast on Apple Podcast, Spotify, or wherever you get your podcasts. When you work in an established, successful company, current managers already know a ton.

Still, AI solutions can offer insights to even experienced managers if you can get the humans and the AI to work together. In this episode, Prakash Mehrotra describes some moments where human and AI efforts came together for Walmart. And even more fun, he describes the hard work that it took to make those moments happen. Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI.

I'm Sam Ransbotham, Professor of Information Systems at Boston College. I'm also the guest editor for the AI and Business Strategy Big Idea program at MIT Sloan Management Review.

And I'm Shervin Kodabande, senior partner with BCG, and I co-lead BCG's AI practice in North America. And together, BCG and MIT SMR have been researching AI for four years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and deploy and scale AI capabilities and really transform the way organizations operate.

Shervin, I'm looking forward to kicking off our series with today's episode. Thanks, Sam. Me too. Our guest today is Prakhar Mehrotra, Vice President of Machine Learning at Walmart. He's joining us from Sunnyvale, California. Prakhar, thank you so much for speaking with us today. Could you introduce yourself and share a bit about what you do?

Hi, I'm Prakhar Mehrotra. I'm the Vice President of Machine Learning at Walmart US. My responsibilities include building algorithms that will power the decision-making of our merchants into the core areas like assortment, pricing, inventory management, financial planning, all aspects of merchandising. I lead a team of 80 people. They are data scientists, a full-stack team from data scientists to data analysts, data engineers. That's my role at Walmart.

We're particularly interested in you because you're a top expert in artificial intelligence. Can you tell us how Walmart is using artificial intelligence to improve their business? Walmart wants to use AI to serve our consumers better.

And so my role is to make that happen. And my expertise that I gained from Uber and Twitter and my graduate studies helped me achieve that dream. And the secret sauce that I realized was that AI will be successful in companies if we partner with business closely and take business stakeholders along the journey. It's not just about algorithms, it's about business. Because the eventual goal of AI is to

improved the business. I'm responsible for all the machine algorithmic developments for core areas of merchandising, which include how do you price something, how do you select the right assortment, replenishment strategies, forecasting and planning. So all the core aspects of merchandising is what we are trying to use machine learning and AI towards. So Prakara, how did you get started on your path in AI? What are some of the more challenging aspects of implementing AI in your work now?

So I started my career at Twitter, where I was a data scientist. I picked up all the fundamentals of scaling and engineering at Twitter. At Uber gave me a

massive break where it was like a juggernaut, right? Like it's like rolling. Like what disruption is it? Uber taught me. And then when I joined Walmart, I had learned something about AI. Like I knew how to, I had got some experience about AI management algorithms. I'd picked up on the fundamentals of AI. And so the most challenging part about at least the work at Walmart on the store side is

there are no labels in the data, there are no tags. When customer shops in our store, all we record is or all we'd have information about is that the transaction was made.

Unlike social media or unlike an app where you, in a Netflix type of, you know, a recommendation type of environment where you know, you can track the history of a consumer and you can learn from it. That environment is not present in the store side. We don't know what items customers are picking up and when they're making these choices. So the job of algorithms is actually a lot harder as they have to infer all this information.

as opposed to directly learn from the data, right? And so inference became a big part about Walmart and then translating that inference into actionable insight, that's something we can make a forward-looking decision on. And so that was the key challenge at Walmart. How does it feel when you're going from a world where everything is highly quantified to things where everything is abstract but you're still asked to make a decision? The honest answer is you actually feel AI is a bubble. Yeah.

Right. The power of the promise of AI that we have, that it can solve anything and then I can deploy, write an algorithm and I'll get a very quick answer tomorrow. That starts to get challenged, right? Because when you're doing something like inference or when you're trying to find, you're trying to identify these hidden patterns that are not pretty obvious.

There are multiple challenges. There are challenges for a scientist to learn them from the available data that we have. And then also, how will you be able to explain it to the end user that why the algorithm is inferring what it is trying to infer? I think my education or my graduate studies at Caltech really helped me think through, it taught us how to think of a problem.

I still remember my candidacy where my advisor literally asked me a question that had nothing to do with my PhD and wanted me to defend it. And this was pretty routine thing in aeronautics at Caltech.

And so that type of training helped me to move areas. So like when I decided to change fields and venture out into Silicon Valley, social media, Twitter, that was a completely different ballgame, right? Like on the way I met people who were on day one not judging me, but they were more like, you know what, we'll invest in you and we'll teach you. And I was able to connect the dots between science and business.

Prakhar, you talked about the scientist's job being more challenging in an environment where data is not tagged and inferences have to be made. What are your observations around the attributes of a good scientist in business versus a good scientist in academia or in a research lab?

Yeah, that's a good question. At the end of the day, if a scientist has decided to spend time in industry and work in partner with and work in companies like Walmart, like oil companies or like all this, I would say non-traditional software companies where the core business is different. The key element becomes that you should be able to explain what you are doing. You should take the business user on the journey. Data science, we usually say that we should, data scientists should be able to tell a good story.

But the story has many parts, right? Like when I first started working at Walmart, I actually spent first three months of my time in stores just trying to understand the terminology. What does VPI mean? What does core metrics are like? How do we do things? And that basically got me credibility with the leadership. That all sounds very easy. I think we can wrap from here. Yeah.

One thing I also came to realize is that when you're taking big bets or moonshots in enterprise setting, usually there is a very optimism on day one. But you have to deliver something quickly to retain that trust. And so there's a delicate balance. So the transition was very challenging for me where in my previous roles, everybody was believing in data science. And yeah, we have to go do this. And it was more about execution and writing those codes and faster. I mean,

move fast, break things, right? That's a mantra in Silicon Valley here. It was kind of slightly different in my current role at Walmart where I have to act as a thought partner and show them the North Star, show them what is possible and also tell them the risks. So balance between overselling and showing what is possible. That was also a big challenge for me as a leader. What's the parallel difficult challenge at Walmart that's not execution?

explaining what is possible and deciding on the big bets that this is the power of AI. Because I think it's not only unique to Walmart, it's unique to any industry like healthcare or wherever you have this human expertise, where human has expertise because of our way we think or way we are wired and way we can deal with uncertainty or unforeseen situations, right? And where the cost of doing a mistake

is very heavy. There's a penalization cost. Like in some sense, I was able to find this parallel with aerospace, where when your rocket is launched, it has to now land on Mars. There's no coming back. And so while most of the data science training that I had in my previous roles or jobs was, you can do millions of experiments.

That's not the paradigm here. And your algorithm is not the only decision maker. A good example might be how we decide assortment, how we think about what items should go into. So that involves financial planning, negotiation with the suppliers, costing, how do I price it? That's not just algorithmic decisions. On the other hand, I see the point here too, that unless some of these sort of mistakes are being made,

there's a danger of slipping back into the execution. And then it slips back into a pure execution, pure refinement mode versus I think what I might call more of an exchange mode where you're exchanging experiences. So execution versus exchange. It's about the journey. It's not about just end execution. It's about the journey you take, right? It's about, and the journey involves exchange of ideas. Like you can't execute it if you have not taken people along.

And I think there's also a difference between, it's a journey from BI to AI, right? Like business intelligence, like that's how it would call, right? So you have to take a journey from business intelligence to artificial intelligence. Prakhar, you were a finalist for the Edelman Award. Yes. For those of you who don't know, the Edelman Award recognizes examples of outstanding operations research in practice, which is a big deal in the OR world.

How does that feel? How did your team feel? How did you feel? It was a proud moment. It was a proud achievement because not only it was like a breakthrough for me because I'm coming from aerospace and like our community recognizing the work that I'm doing. It was also a proud moment for me and Walmart that look, the work that we are doing is recognized by a broader community. So I feel very humbled and like, it's like, yes, you are doing something right, right? Because you're not right. My PhD, my thesis is not in this

And so when you're running something for a Fortune company and the broader company, it was like a stamp of assurance that yes, you got it right. A moonshot might be possible. For me, one reason why I chose Walmart as a place to work in was because like,

A dollar or 10 cents price savings might not mean much for at least most of the people in Silicon Valley or wherever I was thought of, but that 10 cents can mean a world to a consumer. And so that basically gave me a meaning to it. Like, you know what, it's about finding the 10 cent savings or 20 cent savings. And you do that across many items that we carry in our stores and across our merchandising network. And those 10 cents add up and make a $10 and make $20.

I like the framing of, you know, you can check my math on this, but I'm pretty sure if you save 10 cents 10 times, you're going to have a dollar. And, you know, that you keep doing that over and over. And I like the way you framed it from not taking the dollar, you're saving the dollar out of the process. And I think that's where your team has a lot of potential. Tell us a bit about your role at Walmart and like how much of that is science and technical stuff.

management, how much of that is team and stakeholder management, how much of that is evangelism and inspiration and whatever else? I spent probably 30% of my time

in management, which involves upward management, trying to set the expectation with the company, what is possible, what is not, acting as a thought partner to the leaderships, both in the technology side and in the business side. Another 10 to 20 percent, because I am a firm believer that if you are an AI leader and you're leading a team and you're in management, you can't just be a people manager. What's the funnest part of your job?

Funnest part of my job is... Besides talking to us. Of course, stuff like this. I get to opportunity at Walmart. I think the most funnest part is when you see somebody who is not a believer in AI starts believing in AI. So that happiness that you get when you see people start to believe in something, the passion that you share is amazing. And second is like,

I'm just making Fortune One Company a better place. Like Walmart is an essential part of our life. It's part of our, like, I mean, during these difficult times in COVID, like we have to keep our stores open, right? We have to do it. It has a role in the society. And so you keep that running. You play a teeny tiny part in that. And so that is, it gives a meaning for me to come every day to office. And then support from, from

from the leadership, right? Like those are the best parts of my job. And then you build a team. You have a team of very smart people spread across who share the passion with me. And you see them rising in their career, working with young people, right? And then looking up to this crazy wave that we are riding in AI, right? Like on one side, everything is possible. Then you come to a job and you're like, no, it's not possible, right? These ups and downs that you see, it's a rollercoaster ride leading an AI team.

and it works extremely. - Where do I apply?

But like Walmart was never on my radar also to join because it was like, why Walmart, right? Like when you have Googles and Facebooks next to you where you live. And then when you realize, like what I realized was that like we had to tell the story. Somebody had to make this connection between the awesomeness of retail and how it connects to daily life, to the power of machine learning. And so I spent a lot of time there, the rest of my time there. Yeah.

And whenever I'm not at work, I'm at home playing with the daughter and then figuring out life. Well, we don't want to keep Prakhar from his family any longer. Thanks for taking the time to talk with us, Prakhar. Thank you so much.

So Sam, let's recap what we heard from Prakar. He made a lot of good points. Yeah, a lot of great points. I certainly enjoyed the conversation with him a lot. There was a lot of passion, but there's also a lot of also understanding of sort of deep practicalities of what it takes to actually transform a company at scale, a company like Walmart, because, you know, he's talking about certain processes where, you know,

You cannot be dogmatic about it and say, well, this is what the engine says, and therefore you should do that. And some of these things are inventory management or on-shelf assortment or store operations and store labor and things like that, where he talked about this notion of exchange and bringing the business owners along

for the ride and during the ride and sort of designing the solution with them in mind so that by the time it's done, they're not surprised. And they've been not only involved, but instrumental in its design and build and incubation and implementation. That's really, really critical. The tough part, too, is that these things aren't going to be perfect. I think I really heard that in his discussion, that he knew that they won't be perfect on day one. And

When they're not perfect, he's going to lose some credibility. And how do they build that trust? And how do they build that credibility on an ongoing basis?

Yeah. And that's a very good point too. And I'm reading between the lines of what he said, but an admission of sort of vulnerability and willingness for the AI engine and for his teams to learn from those experts and setting the right expectations that just because we've built a piece of technology, it's not supposed to be a hundred percent perfect. That is actually not how any learning system works. Yeah.

Yeah, I like your word vulnerability there because it came across. I mean, he's clearly smart. He clearly knows what he's doing, but he's still willing to learn and listen to what other people said and recognize that his algorithms weren't going to be perfect right off the bat. That was a humility that came through. And I think actually that is a secret sauce of...

Somebody like him, whether it's at Walmart or another person like him in a different company, being successful in that role is the willingness to listen, the willingness to partner, the ability to admit vulnerability and a desire to learn and that passion that he has that, look, when this thing works, it's fantastic.

And when it doesn't work, I've already sort of set your expectations that it will not always work perfectly, but every day will get better than the day before. And I think that humility and that willingness is a real characteristics of folks that are in these roles increasingly, because like there could be some organ reject that you bring an expert in AI from a different industry, different field, like Silicon Valley, like Uber into a

traditionally a brick and mortar company because there is a belief that, hey, we've already done it. It's the right way. You guys have to just listen to me. And we know that won't work. And so the sort of EQ that comes along with a role like that is really, really super crucial. And he really demonstrated that too. One of the things that was interesting about Pekar was how much it aligned with what we found in our research this year.

We found that only 10% of organizations are getting significant financial benefits from artificial intelligence. And Prakhar really shows why that's so hard. Most of the things he talked about weren't technical. You could see him almost wistful for the days of perfectly labeled data. But that wasn't the problems that he was facing. Yeah, the problems were a lot more organizational issues.

change management, bringing users along. That's all the human aspect and not so much the tech aspect. And, you know, I think part of his formula for the 10% is going in upfront with the admission that AI is not perfect and AI has a ton to learn.

from the process, from the experts, from the organization. And sometimes it will be right. And when it's right, it will be accretive to the judgment of those people. And sometimes it will be wrong. And when it's wrong, it has the ability to learn. And so I think actually going in with a mindset that AI is perfect is sure a recipe for disaster, right? And any good AI practitioner knows that that's not the case. Exactly. Exactly.

Sherfin and I are really excited about our next episode with Slavic Kiraner from Humana. Please join us. Thanks for listening to Me, Myself, and AI. If you're enjoying the show, take a minute to write us a review. If you send us a screenshot, we'll send you a collection of MIT SMR's best articles on artificial intelligence, free for a limited time. Send your review screenshot to smrfeedback at mit.edu.