AI in Your Living Room: Peloton’s Sanjay Nichani
21:57 Share

Today, we're airing an episode produced by our friends at the Modern CTO Podcast, who were kind enough to have me on recently as a guest. We talked about the rise of generative AI, what it means to be successful with technology, and some considerations for leaders to think about as they shepherd technology implementation efforts. Find the Modern CTO Podcast on Apple Podcast, Spotify, or wherever you get your podcast.

Many people have already invited artificial intelligence into their homes with voice assistants like Siri and Alexa. But how can we individually benefit from computer vision? Today we talk with Sanjay Nachani, Vice President of AI and Computer Vision at Peloton Interactive about a new product that incorporates AI for fitness coaching. Welcome to Me, Myself and AI, a podcast on artificial intelligence in business. Each episode, we introduce you to someone innovating with AI,

I'm Sam Ransbotham, professor of information systems at Boston College. I'm also the guest editor for the AI and business strategy Big Idea program at MIT Sloan Management Review.

And I'm Sherven Kodabande, senior partner with BCG, and I co-lead BCG's AI practice in North America. And together, MIT SMR and BCG have been researching AI for five years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities across the organization and really transform the way organizations operate.

Today, Shervin and I are talking with Sanjay Natchani, Vice President, Artificial Intelligence and Computer Vision at Peloton Interactive. Sanjay, thanks for joining us. Welcome. Yeah, thank you. Thank you for having me. Can you tell us about your current role at Peloton? I lead the AI and Computer Vision team at Peloton. Peloton's mission is to use technology and design to connect the world to fitness.

empowering people to be the best version of themselves anywhere and anytime. Most people recognize this via our bikes and treads that we sell. We have world-class instructors that teach some really awesome content related to cardio and strength and yoga and meditation. And this gets streamed not only to your bike's treads, but also onto your digital apps, whether it's iPhone or Android, and also is available on TV as an app.

you're here to make people happy and healthy and change their lives. Tell us a bit about your education and background. I know you studied at Babson, which is down the street from us at MIT and Cambridge, actually. Take us on your path to get to Peloton and what got you interested in artificial intelligence?

I would say that my career is sort of divided into four phases. The first one was more around using computer vision for manufacturing and factory automation. And then my second phase was more around security and access control. So that's where a lot of 3D computer vision applications are used for like finding people in revolving doors and retail stores and so on. And then I did a little bit of a stint in identity verification and document forensics.

So now I'm in my fourth stint working in the fitness space, which I'm really excited about. But all this experience I've had in computer vision over the years, just bringing into the space is exciting. I think we all are, you know, obviously, as you said, very familiar with Peloton and the bike. What other things are going on that we may not be aware of? What kinds of uses of artificial intelligence are you using there that maybe we can't see?

Pertan Lite, this was something that we recently announced. This is our first strength product. And it's also the first that uses AI technology that actually runs on a physical device in the form of a platform. We're quite excited about it. It basically connects to any TV to transform that TV into sort of an interactive personal training studio.

But our instructors lead a wide range of fun classes, but quite intense, that actually use dumbbells and body weights. And so where we bring computer vision technology and AI there is we have something called the movement tracker, which allows you to track members, allows you to recognize that activity, so that as you follow along the instructors to make sure that they're actually completing these moves as you go through the class.

And this real-time feedback and metric-driven accountability is very appealing to our members because now they have a goal to work to, especially when you don't have a coach at home. And that's a device that they put in the room. Exactly, yeah.

It connects to your TV. The other really nice thing about the device is that it has what is called Smart Frame technology. It basically gives you the freedom to go around the room and it automatically pans and zooms where you are. And then you can see yourself on TV. So you're reflected on TV so that you can see your phone. Really excited about this product because a lot of the technologies all the way from finding people and figuring out what activity they're doing,

All that is driven by computer vision. - So as I'm listening to you about the setup here, my mind goes to just the amount of real-time data that's coming from many, many thousands of people at the same time. Tell us a bit about how you're processing all that and how much of that is really real-time versus pre-packaged or just how, yeah.

Yeah, definitely. Let me talk a little bit about the production aspects itself. This device over here is completely self-contained. There is really nothing that's going out of the device into the cloud as far as image data is concerned or any other type of data that is concerned. It's mostly the content that's coming through, that's streaming in, that is displayed to the user for them to follow. It makes it a very secure system. One of the big things for AI is keeping them secure and private, and so we respect that.

From a training perspective, you have to bootstrap your AI. And that's where you need the data. As we're aware, we need a fair amount of data that fuels the AI. This is where we have spent a fair amount of time sourcing data, annotating data. We'll talk a little bit probably more about some of the other aspects about having diversity of this data that's very vital for you to bootstrap your system. And once you have the data and label the data, then you build your AI systems from that.

There's a separation of what happens during training and what happens during production. During training, there's a bootstrapping process that we use. Basically, the feedback to the person is, are you following along to that particular exercise or not? And you can imagine how powerful that is. So for example, if he gave you a feedback saying that, hey, last week you did X number of moves, but now you did X plus Y number of moves.

That's very motivating to the user. So you might be good with bicep curls, but not as good with planks or pushups, right? We provide that feedback so you can work on it. Or you could also look at another dimension. For example, you could say, hey, you worked on these muscle groups. How about focusing on some other muscle groups? And how about taking these classes to focus on the other muscle groups? So really having that feedback come back to the user and really guiding the user in their fitness journey, basically. So that really is a purpose of guide.

That seems pretty fascinating because you talked about, you know, recommending classes, but some of the appeal here might be that class doesn't have to be packaged anymore. It could be

Well, Sam needs, you know, he's a slacker on planks, so he needs lots of ab work or core work, whereas he's awesome at doing push-ups or something because of my massive build. Fortunately, this is on audio, so no one can verify that that's not true. But I can see then that you could actually somehow generate these classes maybe in real time, so they're not necessarily have to be packaged. They could be adaptive. Is that some of the goal or some of the, is that too thinking too far ahead?

Yes, absolutely. To your point that we do know exactly what the class plans are. What we're working on is recommend classes that might be more appropriate or personalized to a person's fitness journey. Absolutely. That's kind of interesting because when I hear about education in general, we're kind of shifting from talking about a fitness class, more almost talking about more of an education product here that it's adapting to what you need. And so much of what I hear talks about

individualized training. But the production part you're mentioning and the packaging and the overall scope is still important too.

Absolutely. Yeah. And to me, it's a combination of things. Education is great because you're providing insights and metrics that help you improve your performance. I think one of the big things about guide is the accountability. You have nobody looking over your shoulder, just you have a machine looking at you and it holds you accountable and it brings out the competitiveness in you, right? It's that accountability that's important. I

I think there are other things that we are striving for in terms of making the experience gamified in the sense that you want your whole workout routine to be fun and engaging, right? You don't want to keep looking at your watch and say, "30 minutes done yet." And that is what I think is fantastic about just all Peloton products, but particularly even this guide product is that we really strive for making it a fun process in addition to all the other advantages I mentioned.

Sanjay, this is a great example of AI enabling an experience in a different setting, in your privacy of your own home. And it's a great example of AI creating something that is not possible to do without it. What are some other uses of AI for Peloton as a company?

There are already AI initiatives that are going on and we are making them better. Another area is voice. The convenience of using voice for hands-free operation, right? Especially for a product like Guide, where if you're holding dumbbells or if you're on the floor, you're prone and trying to do exercises, you can't hold a remote. So we have an AI team that's focused on voice. It is also going to be making its debut in the Guide. So there's a fair amount of AI in that too, other than computer vision.

Sanjay, also, this is a big deal too. I mean, you guys are putting like a product with artificial intelligence in lots of people's home as a consumer product. You know, there's just not a lot of that going on. Yeah. Sanjay, one question I wanted to ask you is if you help sort of peel back the onion for our audience in terms of what it actually takes to design and scale a solution like you were talking about, either with guide or with voice, how

That goes beyond the technical aspects and the algorithms and the technical aspects of the product. Where my mind is going is just the user experience itself. Talk a little bit about the process of the product design itself and how you bring that aspect into it.

Let me start off answering the question with what really kicks off things at Peloton. One of the real core values of Peloton is put members first. We are obsessive about customer experience and everything is centered around that. We listen to our members, get feedback from our members,

And a lot of product work really starts with that. We have a cross-functional team that looks into many of those things, but it really starts from the member experience and how do we make our members happy and healthy. And there's a fair amount of work that's done also by user research teams, maybe building prototypes and putting in front of users, sometimes experts. In this particular case, we've interviewed coaches.

What I really like about AI or generally machine learning development is that it's fundamentally iterative and it sort of intersects with the whole agile philosophy of software development. You basically say

all right now we have a hypothesis right now so we build a prototype and the way ml works is what you need to do is first deploy a minimal system see where your errors are and that decides oh do you need more data do you need to improve your models is the problem the data quality of the data that you already had then when you actually put out that really doesn't give the intended benefit i feel that ml forces you to be agile that's how things get started it's more of an iterative process

That's really something that we as an organization and all the people who are developing AI products have to realize that it is something that gets better over time. As people use it, it gets more and more accurate. And that's fundamentally because you are always looking at errors, looking at feedback, and that drives the whole process of continuous improvement.

But something that seems different about this to me, because you're talking about the culture within Peloton that may understand this need to iterate and improve and be agile. But when you're talking about delivering this to a product to consumers, I feel like that they might have a different expectation of how... Well, first, I'm not even sure if they're going to know that there's artificial intelligence machine learning involved in the product. Maybe they will or maybe they won't. You could comment on that. But how do you get that culture about...

iteratively improving and don't expect it to be perfect? I think consumers expect things to be perfect initially. Yeah, that's a great point, right? I wasn't trying to say that we should be deploying stuff that's not perfect or close to perfect. The point I was making was that, for example, like when we are going to be launching the guide, we have gone through a lot of trials, field trials, and we're really trying to identify what are the operating conditions for guide, right? What makes it perfect?

There are a lot of things, some things that we cannot absolutely compromise on. Safety, reliability, making sure it works for...

not anywhere in any time, but for everyone, right? Those are areas you can improve over time. Those are the areas you can improve over time. So it's the question of, we start with our operating parameters we are very confident of. And that's what good companies do is figuring out, okay, it is a space that's big enough to provide value to the customer. But at the same time, we have to be absolutely certain that it worked us very, very well in that, right? Or near perfection as you're talking about. And then you kind of expand from there.

maybe adding more features or perhaps like being able to handle more occlusion of body parts, things like that.

I wanted to maybe switch gears a bit and talk about talent. You guys are doing really cool stuff with AI at the core and the heart of the product and the customer experience. What are your thoughts on the kind of talent you and companies that are aspiring to do similar things need? And there's a talent war out there. And what do you think it takes for the right talent to join and want to stay?

So I'll answer this question in two parts. I have strong opinions about it. The first one is more around the talent itself. I feel like that is one of the biggest challenges. But it's not only just the challenge of the scarcity of talent, but also the type of talent, right? I feel like there is a lot of research and researchers in AI, a lot of work being done in AI, but the focus is more on competitions, on papers.

topics such as architectures or optimization techniques, but there's clearly not enough people focused on practical aspects of deployment. I just talked about what it takes to build an AI product, right? And to me, that's where I think that having people focus more on

deployment and on production is very, very vital, right? And so this requires people all the way from how do I source the right type of data? Do I have the right data quality? How do I mix it with synthetic data? How do I build data pipelines and how do I version it? All that becomes important. And also finding people with experience around just deployment of these models. Finding people in that area is where I feel like there's a bigger scarcity is

That's something that Sherva and I come back to a lot. So much emphasis is on production of these algorithms. And what we frame as consumption is really where a lot of the bottleneck is. Coming back to what really keeps talent at Peloton, I think it's the mission. I think that Peloton's mission is just to

empower people to be the best versions of themselves and have them feel good about themselves, be happy, be healthy. It is a very noble mission that every time that I ask people, why do you want to join Peloton? And that's the first thing that comes out is that I have a bike. I know someone who has a bike and, you know, how it's changed the life. And

You have to be in line with that mission. That to me is the primary driver. There are other things like from a culture perspective, the way we operate as teams. One more thing that actually is fairly important is the impact that you can make. All people working on Guide are going to be making impact on millions of members. And this is just one of the products, right? So I think that having that sort of impact also drives people a lot. I would say those are the three reasons really.

Great. One of the things you described was a very edge-oriented approach to ML, that it's within the box, it's within the home, the data doesn't leave. But some of the things you're describing now seem like that they would benefit from aggregation. It'd really help collectively if we understood better how to get fit or how to improve our health. Health information is something that we tend to keep private and we tend to want it to be private, but I can't help but wonder

Would we in aggregate benefit if we were a bit more open with that data? What are your thoughts on that? I think that's trust, right? From an industry standpoint, AI needs to get there. Once you get to that point, maybe it's possible. We've all seen what has happened with face recognition systems and other examples, right? So definitely there are trade-offs. There is a move more towards trying to anonymize it in some way. Can you achieve both objectives?

It's great talking with you. I think most people are familiar with the physical Peloton bike, but this is a product that you're talking about putting artificial intelligence in real time in people's lives. And there's just not a lot of examples of that going on that people are used to. We really enjoyed talking with you. Thank you. Yeah, thank you so much. Yeah, thank you. Really appreciate you having me. Thanks for listening. Tune in next time when we talk with Katya Walsh, Levi Strauss & Company's Chief Global Strategy and AI Officer.

Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders. And if you join us, you can chat with show creators and hosts, ask your own questions, share your insights,

and gain access to valuable resources about AI implementation from MIT SMR and BCG, you can access it by visiting mitsmr.com forward slash AI for Leaders. We'll put that link in the show notes, and we hope to see you there.