Keeping Humans in the (Feedback) Loop: Orangetheory Fitness’s Ameen Kazerouni
24:05 Share

Today, we're airing an episode produced by our friends at the Modern CTO Podcast, who were kind enough to have me on recently as a guest. We talked about the rise of generative AI, what it means to be successful with technology, and some considerations for leaders to think about as they shepherd technology implementation efforts. Find the Modern CTO Podcast on Apple Podcast, Spotify, or wherever you get your podcast.

Real-time data collection means organizations can make many more informed choices based on metrics. But when do they still need humans? Find out on today's episode. I'm Amin Khazrani from Orange Theory Fitness, and you're listening to Me, Myself, and AI. Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College.

I'm also the AI and business strategy guest editor at MIT Sloan Management Review.

And I'm Sherwin Kodabande, senior partner with BCG, and I co-lead BCG's AI practice in North America. Together, MIT SMR and BCG have been researching and publishing on AI for six years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

Today, Shervin and I are excited to be joined by Amin Kazarouni, Chief Data and Analytics Officer for Orange Theory Fitness. Amin, thanks for joining us. Welcome. Thank you, Sam. It's great to be here. Excited for the conversation. Currently, you lead the data and analytics function at Orange Theory Fitness. Maybe tell us a little bit about the organization.

Absolutely. Orange Theory Fitness is a heart rate-based total body group workout. It combines science, great coaching, technology, and it's designed to provide what we like to think of as a more vibrant life. The workouts develop to motivate each individual member to achieve their desired results. And, you know, if you're starting on your wellness journey or you're

a seasoned fitness enthusiast. Each OTF workout creates a community of shared experience, but also uses heart rate-based training to allow you to experience the workout in a way that's most comfortable for you. And that's honestly where my role comes in. There's a tremendous amount of second-by-second telemetry data from the fitness equipment, from the heart rate monitors that allow us to create

the most curated, personalized kind of boutique fitness experience in the world. That's Orange Theory. All right. What do you do with all this? You've collected all this data. You've got it, this telemetry. You've got heart rate information, I assume, since you're heart rate based. How does the process work? Take us through the steps. Most...

Orange Theory members in the studio will be wearing what we call a OT-Beat heart rate monitor, which is a proprietary piece of wearable technology. There's two purposes to that. One is it gives you a real-life feedback loop as to how you're performing, what intensity level you're outputting in the studio. But it also allows the coach to see the intensity level that you're outputting in the studio and help effectively provide a kind of personalized fitness experience

training experience in a studio, in a group setting. My role is focused on kind of unlocking that telemetry data and helping personalize the experience even more. A really cool example is that we recently launched a personalized max heart rate algorithm and members now experience a much more curated experience in the studio. And that's allowing us to

use proprietary algorithms to determine what the max heart rate, which is a physiological term for the maximum output that your heart can beat at, is for an individual member.

And percentages of that max heart rate tell you which heart rate zone you're training in. So anaerobic training versus aerobic training have different physiological impacts. Time spent in different intensity zones have been proven to have varying effects on longevity and health in general. And being able to personalize that per member

we're able to make this experience even more curated in the studio, while most places that leverage max heart rate will rely on a generic age-based kind of equation. And you know that, as you can imagine, every 30-year-old or 40-year-old doesn't have the same heart. So things like that are an example of how we curate the experience for our members using this data.

That is a super cool example, right? Not the average for your age and gender. And then it goes by bands of 10 anyway, right? Yeah, exactly. Like as if all 40 to 50 year old males are like exactly have the same ability. So that's really super cool. Yeah, yeah. It makes the experience safer. It makes you more aware of what you're doing, what your capability is. And you see that cardiorespiratory fitness climb over your time with the program.

And I like what's sort of inherent in what you're saying, the rapid feedback, right? Within a few seconds, you get feedback. But also in a broader symbolic sense, what you've been proposing is more and more experimentation in general as you build AI algorithms. So it's not just the algo, but it's also experimentation because you get feedback. The philosophy of how to build AI

AI models which relies on feedback, you're also translating that into a real use case based on rapid feedback. There's some poetry there. I didn't think of it that way, but absolutely there's poetry there. And I think that's one of the beauties of artificial intelligence algorithms is that they're so reliant on things they've seen before and constant feedback loops to get smarter and

That's exactly what the Orange Theory experience is indexed on, is as the members are getting stronger, we're getting smarter and making sure that we move with them. Tell us more a bit about your data science development philosophy and how you balance experimentation with more methodical, get the algorithm to be perfect before you launch it. Just give us a sense of the trade-offs and how you and your team think about that.

I love that question. And it's almost like I'll take it even a step outside of developing algorithms. I'm a firm believer that companies have

started indexing so heavily on collecting as much data as is humanly possible, as much as their compute and their storage costs and their boards and their investors will allow them to, companies collect data. And I think that what happens is once you have the data, the expectation is let's jump to machine learning, let's jump to AI. And I would argue that those, how many debates have you been in on what's AI and what's machine learning and what do the words mean?

My philosophy is to go and find those mundane, repetitive tasks and automate them first with your data where possible. Go and find intuitive, gut-based decisions that your stakeholders and verticals are uncomfortable making based off of intuition and would love to make off of data and make that data democratized, clean, and available online.

And after that, you start the whole machine learning journey. And I think when it comes to machine learning and AI and developing an algorithm in particular,

It really depends on the context and the domain in which you're working, whether you're focusing on that precision, whether you're focusing on that recall. And it really depends on what the implications of the prediction are. But generally speaking, I always err on the side of if it's safe enough, experiment and learn rather than relying on training and validation sets to chase perfection. It's kind of my rule of thumb philosophy.

There's something also interesting, and I know you've shifted slightly into thinking about how your organization uses data, but maybe going back to how you're using it within the studios, how do you incorporate data from outside? You know, let's say that I've been training and working and improving within the studio, but little do you know that I've hurt my foot or I've pulled a leg muscle. How does that sort of outside information come in?

I like that question a lot, Sam, because I've got two answers there. One is the coach is the hero at Orange Theory. And the way we think about it is there's 20 to 30 people in a class, but the

You don't want to think of it as group fitness as much as you want to think of it as, probably at least I think of it as subsidized personal training. So you've got a pretty good personal relationship with that coach. And there's no algorithm that I'm going to build that's going to be better than you telling your coach, hey, I hurt my foot last night. What do you think I should do? And we've got alternative low impact cardio equipment in the studio so you don't

get on a treadmill and start trying to run at eight miles an hour. In fact, one of the things the coach says before every class starts is if you've got any orthopedic issues, please let me know and so on and so forth. So I think that one thing that I've learned about data over the course of my career is that data is valuable and it's really good to make decisions with. But if there's a human in the loop that can scalably provide the answer instead, then

It's likely going to be hard to beat that expert with an algorithm. So don't get in the way of the expert. The second part to that answer is, let's say you had a bunch of caffeine or you ran a marathon the previous day or you...

Didn't sleep very well. All those external factors affect your cardiac output when you're in the studio. So that real-time feedback loop allows you and the coach to modulate yourself. One thing that we encourage is if you're not feeling it, take a green day, quote unquote. Green being the zones are broken up into colors, orange and red being the anaerobic zones.

zones, zone four and five, green being your zone three, blue being your zone two. So we recommend don't shoot for those 12 to 20 minutes in the orange and red zone. Look at your heart rate zones, listen to that real-time feedback loop, and take a green day if you need to. So

A bit of a cop-out answer there, but we've got the coach, we've got the member, and we've got a real-time feedback loop. And when there's an intuitive answer like that right in front of you, I don't think you should interfere with an algorithm is kind of the thought process there. I don't think that's a cop-out answer at all because historically you had a...

Okay, you could have a personal trainer or you could be in a group setting. I think I personally identified this because you could either teach someone one-on-one as a tutor or you could teach someone in a giant classroom. And I'm just excited about applications like this that are letting us pull together the best of both of those worlds. I'm hungry for it in education, but I can see making a lot of progress on that. And sure, I've talked with Peloton as well. They're starting to think about how you can

get to an individual level experience at scale. And that seems like what you're really trying to do. Absolutely. I couldn't have put it better myself. I think that's exactly it. I like that a lot. This is like all about rapid feedback, which is a cornerstone of building expertise in any system, right? Whether it's chess or whether it's running or whether it's machine learning, the more you combine experimentation and rapid feedback and the human in the loop, right?

And apply that across different industries, the more opportunity and values you're going to unleash for personalized everything, not just fitness or shopping for pants, but also education and everything. You're onto something here.

Absolutely. And I prior life, I was in retail. And when you think about the role of machine learning and deep learning in retail, it's that you're trying to recreate the in-person shopping experience on a website. Like how do we curate this? How do we create a personal shopper? How do we create that boutique experience? How do we predict the size correctly? How do we use AR so you can see what a shoe looks like on your foot? We're always trying to close that gap.

and arrive back at what the real thing would be like personalized but at scale. And I think that's the secret sauce there is that a rapid feedback loop, algorithmic support with large volumes of data, but then also not trying to circumnavigate around the expert, the human in the loop. Exactly. And I think that's really critical, Sam, because I remember when we did the first

few years of the AI report, like back in 2018, 2019. So long ago. There was still a lot of... I mean, that's only three years ago, but a lot of folks still today are thinking of AI as that which replaces human and that which must automate. And the more we think that narrowly, the more...

outcome we're leaving on the table, the more successful workforce and humans we're disenfranchising. And also, the more opportunities we're leaving unaddressed because we think, well, there's no way I can fully replace a human here, so I'm not going to do it. And there's so many of these things where it's not just AI versus human, but it's AI and human. Yeah.

You know, I think that there is a conflation of companies where the product is AI. Like when you think of AI, you think of Tesla and there's different industries all of them play in. But there's a very core part of the product that's a standalone piece of intellectual property that's heavily rooted in artificial intelligence and AI.

That's not how a majority of the world is going to use artificial intelligence. And I think that's one of the kind of key differences in what you were just talking about, Shervin, is that people try and use AI the way those companies use AI and see them as the gold standard. But our product's not AI. Our product is a curated, science-backed, coach-inspired fitness experience that's just merely augmented in parts by AI.

Yeah, this is something that Shervin and I are pretty excited about. I don't want to foreshadow too much, but we're thinking about these many uses of AI beyond just this kind of poster use of AI that you're talking about. I mean, yes, we all are attracted to the Boston Dynamics robots that look very cool, but there's a lot going on that isn't that level. And there's so much value in those. And I think you're starting to capture some of that. I wanted to segue from this question

We talked quite a lot about the importance of human in the loop, experimentation, being hypothesis driven, all these things you said. Maybe tell us a bit around the operating model and the ways of working in a company that is not a AI product company, but is a company like you with a strong mission. What does it take to take a use case and bring it to life?

I think it honestly comes down to three things. Is having the data. You can't really get around not having the data. Investment. I think a big mistake companies make is not investing in data engineers early, thinking that you can just sprinkle AI like some kind of magic powder on raw data sets and it's going to produce something. I think data engineers are a critical commodity that

You want to invest in early so your data is at a point where you can actually use it. So that investment's really important. Wherever it's coming from, there needs to be a serious decision made to invest in your data practice if you're going to really try and build one. And then finally, it's buy-in. When the product's not AI, you're convincing domain experts, in our case, fitness experts that have been doing this for a long time, that are formally educated in these fields of study, that are

always going to know more about the product than you do, that an algorithm is going to help them and make their job easier. And I think that that relationship can be a beautiful partnership or it can be an extremely antagonistic one.

One of the things that I've kind of strived towards in my role at Orange Theory is to have a strong partnership with our template design team, our workout design team, because at the end of the day, they're the protectors and designers of that product. And we're, again, just a tool that's supporting them. Their buy-in is very critical because their understanding of the algorithms is what then makes it to coaches in learning and development material. At the end of the day, we've got a

thousands of coaches across 24 countries explaining that max heart rate algorithm that I mentioned. And they're not explaining it like a piece of mathematics. They're explaining it like a piece of exercise fitness, like a piece of exercise physiology. And that requires that buy-in. Like the AI, the data team and the fitness team need to be in lockstep. Otherwise it's destined to fail.

And what does that mean in terms of the talent and the team, the technical team that you oversee and you hire and recruit?

They're difficult to find, is what it means. I think there's already a scarcity of talent in this space. I think in a mission-driven, purpose-driven company like Orange Theory, you'd think it's harder to find, but it's actually easier in the sense that if you find someone that's aligned with the mission, it's almost exciting to them that there's an opportunity to apply that skill set on what they considered an

outside the job fashion, but we've also focused on our

data organization being a separate entity. So we've got my role, the chief data and analytics officer, running a data organization, reporting into our CEO. And we've got our chief digital and technology officer running a separate digital and technology organization. And what's really powerful about that is that we're able to riff off of each other and have one team provide building blocks to the other team and vice versa. And what you'd imagine

creates a interesting working experience actually drives a lot of velocity and drives a really cool partnership that's very exciting to be a part of as well. So I think it's all about that, partnerships and buy-ins and collaboration across teams.

Very well said. I mean, we have a special segment here where we ask you five rapid fire style questions. Just give us the first thing that comes to your mind. The key thing is like intuitive, whatever comes to your mind and shortens to read answers. So ready for that? Let's do it. All right. What's your proudest AI moment? When we solved it using a linear regression. Love it. All right.

What worries you about AI? A lack of consultation with domain experts. Well said. Your favorite activity that involves no technology? Hiking. It's a question? Yeah, it's very simple. I was going to say Orange Theory, but there's too much technology in there. The first career you wanted, like what you wanted to be when you grew up? Environmental biologist. Your greatest wish for AI in the future?

More access. Great. Actually, I got a follow-up. Access for who? Who needs access?

I think it would be great if some of the simpler portions of AI that unlocked decisioning off of data that companies have collected was easier to tap into without the financial and human capital that you require to invest as an organization. I think the general efficiency of the world will just go up, you know? Yeah. More open source kind of stuff on that. Yeah. Yeah. Yeah.

I mean, this has been exceedingly insightful and a lot of fun. Thank you for making time for us. Yeah, thanks for coming. Thanks for having me. This was a lot of fun. I truly enjoyed it. On our next episode, we'll speak with Katera Kodeverde, Senior Director of Data Science and Analytics at PayPal, where she oversees data teams at Venmo and Honey. Please join us.

Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders. And if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and learn more about AI.

and gain access to valuable resources about AI implementation from MIT SMR and BCG, you can access it by visiting mitsmr.com forward slash AI for Leaders. We'll put that link in the show notes and we hope to see you there.