Investing in the Last Mile: PayPal’s Khatereh Khodavirdi
29:24 Share

Today we're airing an episode produced by our friends at the Modern CTO Podcast, who were kind enough to have me on recently as a guest. We talked about the rise of generative AI, what it means to be successful with technology, and some considerations for leaders to think about as they shepherd technology implementation efforts. Find the Modern CTO Podcast on Apple Podcast, Spotify, or wherever you get your podcast. What do Lego have to do with how PayPal thinks about AI?

Find out on today's episode. I'm Khater Khudaverdi from PayPal, and you're listening to Me, Myself, and AI. Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and business strategy guest editor at MIT Sloan Management Review.

And I'm Sherwin Kodabande, senior partner with BCG, and I co-lead BCG's AI practice in North America. Together, MIT SMR and BCG have been researching and publishing on AI for six years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

Today, Shervin and I are talking with Katera Kodavirade. She's the Senior Director of Data Science at PayPal. Katera, thanks for joining us. Welcome. Thanks for having me. Great to be here.

You build and oversee giant data science teams for the many different entities within PayPal. A lot of people know PayPal, but probably some people don't realize the extent of all the PayPal activities. So maybe let's start there. Tell us a bit about what PayPal does and what all these different sub-entities do as well. How are they connected?

Yes, I would say majority of the people know PayPal through PayPal checkout, which is a core products that we have. And the company has started from that product, but we have a wealth of different products, especially on the consumer and merchant side. We have whole suits of product to provide people the capability on the merchant side to run their business from invoicing, having the offline capability for the financial services, and then the online transaction.

And on the consumer side, starting from peer-to-peer payment to financial services to different types of credits and bonds.

And buy now, pay later capability and saving accounts. I can go on and on. But on the consumer side, we have a wealth of different products. And over time, we also acquired many other companies to help us accelerate and also be incremental in terms of the value chain that we are creating for the consumer and merchant. Such as on the consumer side, a couple of years ago, we acquired a company which name is HubSpot.

which is actually help you to find the best deal on the internet when you're shopping online.

I started at PayPal on the consumer side. So I was helping the small business group, helping accelerate and solve the problems for our customers through data and data science capability. And then over time, I kind of supported all the merchant side of our equation, all the enterprise merchant, channel partner, our relationship with Shopify, WooCommerce, Magento, that programmatically bringing a merchant for us.

And then a couple of months ago, I actually switched completely on the consumer side of the house. So I'm new on the consumer side, but super excited because you can leverage a lot of AI and data science capability to solve a lot of interesting problem in this domain.

That's great. So obviously, PayPal is a big multifaceted company with different businesses and subdivisions. And you talked about those. But since you're in the consumer group, let's talk about that. Share with us a bit how AI is being used in the consumer business to drive the themes that you want with your consumers and what use cases is AI being used for?

We can actually bring all the different accounts together so we have a point of view around the user interaction with us. For example, if Sherwin has a relationship through Honey with us, but at the same time he's using peer-to-peer or the checkout product or our credit capability, how can we make sure that

We can actually look at share in relationship with PayPal through one lens. So that is, I would say, one of the fundamental problems we are trying to solve as a company. But the piece I'm personally super excited about is around the fact of understanding the customer journey with us and how we can leverage personalization and AI.

to actually solve customer use cases. That what are the jobs that are coming to PayPal to get done? And how can we show them the personalized and relevant messages to help it? So think about it internally. We always said, hey, there are a bunch of happy paths

with PayPal and they're a bunch of sad pads with PayPal. How can we migrate people from one happy pad to a happier pad with us? And how can we avoid the sad pad with the customer? And I truly believe that this is basically an AI capability that we need to develop for the customers. But if you build that, it will unlock humongous amounts of value for our consumer and for us as a company.

Let's talk about some of those paths. You said what kind of jobs are people trying to get done? So let's talk about that. Tell us how personalization could be helpful there.

Think about it this way. Imagine Sherveen is one of the people that is using PayPal checkout to shop through different merchants throughout the internet as well. As you can imagine, once we acquired Honey, we also have like a whole belt of actually the coupons that are happening out there. And over time, if you get a better sense around

what type of categories or what type of merchant Shervin is interested, we can actually show Shervin right deals at the right time. For example, it's back to school time. And we know that historically at this time, you shop actually in this type of category. Currently, Target or Walmart or some of the top merchants for back to school are running these deals.

And then we remind you and show you relevance and personalized deal to actually drive activity with us. Or you might use Venmo on a weekly or bi-weekly basis to actually send somebody who cleaned your house the money. Hey, we can remind you that, hey, Shervin, it looks like you are doing this. And through just one click touch, you go and do that. So basically we become partners.

part of Sherwin's life and understand what type of activity Sherwin is trying to do and make it much easier for him. The other part of it that we are all super excited about is tracking. Like when you shop online to different retailer, one part of it is that, hey, you want to track that order and see where your order is and where are you getting it. We also want to make it much easier that you can actually get the notification

and see, hey, where your order is. And you can look at your whole commerce activity and financial activity in one place. I would say for me also, the other part is that, yes, I'm in a data function and we build model and I look at the outcome of our model. But also the more important part for me is also, I always call it, it's like qualitative and quantitative, right? Even like I would say, hey, I want a sample of

Some of the customers in this group, I want to understand their journey, the activity they had with us, or, you know, actually attend the user research studies that we are doing with customers. So firsthand, I actually hear the challenges that the problem are trying to solve. Because for me, the most important part is that I just cannot go and in an isolation, build the model and solve all the problem. It's really like doing the qualitative and quantitative aspect and learn, you know, from each other and improve it over time.

That's very helpful. I have to imagine that being more of a tech company than, let's say, a financial services company, that some of these challenges are actually a lot easier for you to deal with than it would be, let's say, for a gigantic global bank that's trying to personalize across products and business lines and all that. Share with us some of the challenges. What's difficult about what you need to do?

I think one of the biggest challenges we have is that some of the other companies that we acquired over time, each of them are using different data stacks. So basically like migrating all of them to one data lake and having one kind of technology to use it across the board and being able to have like building that conversation.

common data layer platform around end-to-end understanding all the touch point and all the data points we have with the customer. So having that common data technology platform is one of the common challenges we have internally. That's huge, I think, for everybody. I remember my first experience with this. Back in my past life, I used to work at the United Nations and I was working with databases and

And I looked down and we had just literally dozens of databases we were supporting. And, you know, I asked, well, which is the standard? And they said, well, this is the standard. And I said, well, what are all the rest of them? Well, those were the standards then. And so you're in that same situation when you're acquiring companies where you're pulling together these lots of technology stacks that you don't want to just kind of rip them out and start over. How do you manage that process? How do you get those in a cohesive data science process?

Yeah. So I would say, Sam, you brought also a very good question as well, that it's not only the technology problem aspect of it. I call it the data governance aspect of it as well. What are the definitions and how different people define and look at different things differently and how you can define that common language across the board internally within the company as well. So that is why...

we are trying internally to develop a best practices that, Hey, every new company that we have here are there like a step one to end that we are going through to incorporate them as part of the rest of the data assets we have for the company. But you can imagine it's not an

easy exercise. It's a humongous task. But that data governance aspect of it is also very important because when you are bringing different components together, you need to take a step back and look at the definition from different lenses as well and see if those definitions still are relevant in the new construct or not. Let's talk about teams a little bit. You're talking about a series of challenges with

with personalization, with other use cases, and then the capabilities that are required to get that done from data coming together and identity resolution and many, many other things. Tell us about the team. Clearly, they have to have very strong technical capabilities, but what else do they need to have to work in that kind of an environment?

I would say that this area is one of the areas that it's so multidisciplinary and you can imagine, hey, the different type of problems you want to solve needs actually different type of skill sets and these type of talent. So I always said that in data science, AI, or overall data fields, one of the things that are really important is diversity of talent. And by diversity of talent, I don't only mean diversity of gender or background, which is very important, but diversity of

taught leadership, diversity of problem solving, diversity of technical skill set, because on the different stage of the problem, basically you need to practice different type of muscle in order to get the desired outcome. So for example, one of the areas that I really feel like it's very underappreciated in the data world is actually business acumen, like people who can actually tackle the problem

through a very structured framework and be able to synthesize the recommendation and the so what to the business. Because the worst thing that can happen is that you look at your data team and you feel like, hey, they are building a bunch of black box for the rest of the organization. And you're not investing in the last mile that people actually translating the

the what and the why and the so what to the business group, to the product group, and to the rest of the organization. So you would not get the adoption that you are hoping from the capabilities that you are building. So the way I'm looking at it is that we are actually also building a product organization who support the personalization and AI because

Like any other product development cycle, you basically have a product strategy behind this that you're not tactically building the AI model to solve a specific use cases. We actually take a step back, understanding our consumer persona, what are the jobs, again, they are coming to us to get done and build the product roadmap and product vision around this and tackle this problem in a cross-functional fashion instead of just internally within the data group.

Is it harder to get that last mile with AI type projects? Is that something that people have a harder time understanding? Is it something that is harder for people to relate to?

It might be a little bit harder because of just the scale of the problems that you are trying to solve here, because people cannot relate to it when you're talking about like millions of customers. So for me, is that how I can actually break down the problem and solve a smaller use cases through AI to create that adoption and championship in the organization that it will help me to solve the biggest problem.

It's a humongous task because I'm not only talking about the product touch points that we have, I'm talking about every touch point that the customers have at us, either through customer service, through risk, through all the different functions within the company, who

rallying all of those cross-functional function around solving this problem would be much harder versus my approach is that I will first start with solving it within the product organization, understanding all the touch points with all the products, understanding how we can understand the product paths

and personalizing that component. And then you can add an additional layer, like bring the risk component with each of the product component, then you add the customer service. So I would think about it more of a Legoland that at the end of the day, we will have a like AI Legoland for PayPal. But right now the way I'm attacking to solve this problem is to build infrastructure

each of the individual Lego pieces with the hope that I can orchestrate and build the Legoland and it won't become like a bunch of separate Legos that are not orchestrated to solve the common vision. I love the Lego analogy because my kids are totally into Legos and we have probably like 900,000 different pieces of Legos going on at any given time. And if you just look at it in an isolation, you think, okay, this is all we're doing.

But of course, you got to start there and then the pieces come together. So then my question to you is, as with Legos, when I see my 12-year-old or 8-year-old building stuff and I'm looking at it in isolation, I might not have a full sense of the vision that the whole thing is going to be. So I might say, oh, this is nothing. Or like, what are you building? It's like a small piece. Didn't you do something like this before? And then they bring out the box with like 8,000 pieces that's going to look like this.

And then I go, aha, how are you doing the big aha here at PayPal so people don't lose sight of the big vision and don't get myopic about the little things that takes to get there?

You can imagine, hey, there is no shortage of individual use cases, that there are many individual AI or whatever you name it capability within the organization. But when you take a step back, you do not have that guiding principle to see how they can help you to actually build that Legoland. And actually, I want to tackle this problem in a reverse order that

First, I want to take a step back and see, hey, what is the blue sky will look like in terms of the AI capability and personalization for us? So building that product strategy and vision around it, and then try to solve backward and then break it down into smaller pieces of the component that Legos and be very prescriptive around what are the key problem each of these Lego will try to solve and

and why we are building each piece of the Lego and what would be the so what to the organization, then you can actually build something that because if you show that whole vision to everybody, probably it might be a little bit too much for some of the people to absorb it. So

It might really slow down your progress in the organization versus when you show the bigger vision, you have something to rally the whole organization around. But at the same time, you can break it down into more tangible component so you can start making progress while you're actually keeping that energy and enthusiasm around the organization to the North Star that you have.

The breaking down actually is quite critical. To build on the Lego analogy, usually you get these 5,000, 6,000 pieces of Legos and they come in like 20, 50, 30 boxes or little bags. And so you first do this and then you do that, but then you still have the whole thing. Well, I got one two days ago that had 3,000 pieces in 20 different bags, but all the bags are unlabeled.

So you don't know what goes with what. And so then what we have is like 3,000 pieces and we're trying to build a piece. And the analogy I'm trying to draw here is as you're building these little capabilities that then come together and get stitched together to support your bigger vision,

How do you make sure that these pieces are actually connecting rather than the organizations looking to find what piece goes where or how does this connect or do I have eight of these instead of five of those? How do you avoid that? And, you know, that's, by the way, something that goes on in a lot of other organizations where there's like silos of folks building things and they don't all come together. How do you approach that?

You brought up a very good question that, Sharon, basically, hey, I cannot go to one step from that vision and product strategy for the AI and personalization to really tactical component of the puzzle that you have. But once you build actually that mental model around the common themes of the problem you are trying to solve, then I would say the other reality also is that I cannot solve all this problem by myself or not that easily.

handful number of people can solve this problem in the organization, you need to create that culture and you need to rally your organization around this. That makes sense because

I like the Lego analogy, but the world isn't as simple as Legos. You don't have that perfectly labeled bag that you know will fit together in the end. You've got to bring those people along to pull that in. One of the things that we did early in the pandemic was sort our giant things of Legos. And when you have those Lego that are not in the little bags, they're practically impossible to find the right piece to pull it back together.

Servant reminds me of when we were talking to Artie Zaccami at H&M, and he was talking about working with individual pieces. On the wheel, right? Yeah, with the wheel. His analogy there was tightening each lug nut a little bit as you go around. And there's some logic here, too, that what's different, too, about Lego is that you could...

solve one bag and move to the next bag. But in reality, you got a lot of people working on lots of different bags who are moving at different paces. No, totally, Sam. And I would say that is like why it is very important that

In my view, in order to make progress on such a complex topic, it's kind of a top-down and bottom-up approach. And you just need to have that check-in on a regular basis. The top-down, I would call it your blue sky strategy around where you want to be with AI strategy. And the bottom-up is just mainly like that tactical different pieces of the Lego strategy.

that somehow they exist in the organization or there are different components that different teams are building, how you can actually bring the two components together. And you're absolutely right that the speed of the development and making progress for some part are more difficult than the others as well. So you also coordinate the different components together. So at the same time, you are making progress. You can rally the organization around that, but also be realistic that some other parts are more complex and it will take more time.

And the one thing that I think companies have that Lego pieces don't is they still have P&Ls and targets and numbers. And so I think that's probably why it's such a nuanced approach, as you're saying, KK, that it's like you got to figure out where the most practical path is for your organization, given all the players and all the stakeholders and all the pieces. And maybe for RD, it was...

a bit one log not at a time to get the whole thing going. And maybe here it is like, no, we got to get personalization perfect before we move on to risk or pricing. And I think that's the nuances of different organizations a little bit.

Yeah, and I was just showing you brought up a really good point as well that, hey, at the end of the day, all the organizations are very, you know, value focused, both for the, you know, customers and also for the company as its own and the shareholder as well. And for me, one of the biggest risks that people can do is just looking at the output and the outcome of the P&L versus P&L.

For such a like big projects like this, like personalization, the reality is that, hey, it will definitely take some time for you to actually get the true benefit of this from the output standpoint of your P&L. But what are the leading indicator and the KPI you can have to actually keep the team and the organization accountable to make progress so you make sure that you are moving in the right direction?

while it will take you more time to see the whole benefit as an output in your P&L. Because the reality is that the worst thing that can happen is that all of us know that this is absolutely the right thing to do for the company. But because there is no magic, there is always like,

good work consistently over a long period of time, there is no magic that you see there like output overnight. So how can you keep the team accountable, the working team to make progress, but through the leading indicator that you know that eventually it will get you to the outcome? Yeah, for sure, right? As you're saying, the worst possible thing would be to...

set a big outcome goal for the vision, but for the wrong time. KK, this has been quite insightful. I think, Sam, we should move to a rapid fire question segment. So this is a segment we do, KK, where we ask you a bunch of questions in rapid fire style, and you please tell us what comes to your mind first. Sounds good. What has been your proudest AI moment?

Proudest AI moment goes back to grad school when I was at Carnegie Mellon before this field was this much in demand. And as part of actually my graduate research studies, I was building capability and platform for energy management, so smart energy management for residential buildings. So that was the most proud AI moment of my life. Cool. What's your favorite activity that involves no technology?

playing tennis because it really helps me to focus on the moment. What was the first career you wanted when you were like, what do you want to be when you grew up? Probably the first career I wanted to become a professor or a pilot. I don't exactly remember which one came first because at our family, like education is like a big piece. And my mom actually had an educational career, but probably either a professor or a pilot.

What worries you about AI? I would say they're having a lot of conversation about responsible AI and bias. And I chatted about this earlier that it's qualitative and quantitative as well. I think it would be a huge mistake to assume that AI can solve basically all the problem without having the right checks and balances in place. What is your greatest wish for AI in the future?

I would say there are a lot of challenges that the humanities are facing from climate change or a bunch of other stuff. So I really hope that more and more people actually play a role of using AI to solve those problems. A lot of my colleagues actually started investing more of their time and energy. And I really hope that eventually in my career, I also can play a role as well. Great. Thank you very much. So KK, I think that obviously the

The Lego analogy is going to be interesting for people. But I also like what you were saying about things like governance that I think are fundamentally important for this. And the idea that you'd come on and mention some of that importance of things like governance, getting you to that scale that you need. I think that's something that maybe is more widespread or universal. Thanks for taking the time to talk with us today. We really appreciate it. Thanks for joining us. Thank you guys so much for having me.

Thanks for listening. Join us next time when we talk with Fiona Tan, Chief Technology Officer at Wayfair. Please join us. Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights,

and gain access to valuable resources about AI implementation from MIT SMR and BCG, you can access it by visiting mitsmr.com forward slash AI for Leaders. We'll put that link in the show notes and we hope to see you there.