Episode 289: OKRs in Product Development with Jeff Gothelf
34:38 Share

Hello, ladies and gentlemen, and welcome to the UI Breakfast podcast. I'm your host, Jane Portman, and today our awesome guest is Jeff Gotthelf, a coach, trainer, speaker, and author of five books. And we're going to talk about OKRs today, OKRs in product development.

This episode is brought to you by Wix Studio, the new web platform for agencies and enterprises. The magic of Wix Studio is its advanced design capabilities, which makes website creation efficient and intuitive. Here are a few things you can do. Work in sync with your team on one canvas, reuse templates, widgets, and sections across sites, and create a website.

create a client kit for seamless handovers, and leverage best-in-class SEO defaults across all your Wix sites. Step into Wix Studio to see more at wix.com/studio. Hey, Jeff. Hello, Jane. It's nice to see you again. It's good to be back. Welcome back. Indeed, we had a wonderful conversation in 2019.

And you laid out all the product methodologies for us. But today we have something more specific. Give us a recap of your background story for those who didn't listen to the original. And also, how did you evolve from focusing on design methodologies towards the numbers, the business, and that side of things? Sure. So...

I was a designer, a UX designer for the first part of my career and then into design leadership. I built and led some teams, moved into product management and then product leadership. About 12 years ago, I launched my own consulting company with the help of my co-author, Josh Seiden, my longtime business partner, Josh Seiden, and some other folks. And we ran that business for four years.

And so a bit of an entrepreneurial business. And then when that ended in late 2015, I became a solopreneur, a solo practitioner, teaching the material in the books that I've written and consulting and speaking and working with companies, usually large and mid-sized companies, on building a better integration of design, product management, and engineering, helping them with their organizational agility and product management, and these days with objectives and key results.

I've written five books over the last 12 years.

or so. Started with Lean UX, co-written with Josh Seiden. Sense and Respond, also co-written with Josh Seiden. I wrote a book called Lean versus Agile versus Design Thinking. A bit of a left turn book called Forever Employable, which was about how to build the kind of business that I have. And then most recently, again, with Josh Seiden, I like to work with Josh. We wrote a book called Who Does What by How Much? A Practical Guide to Customer-Centric OKRs, which is...

the latest application of our ideas sort of at a different level of the organization. I love to see an evolution, how authors go from boring titles, you never got boring, from like grand sounding titles like Lean UX to entertaining edgy titles, like who does what by how much. That's pretty interesting to see how you're thinking involved. Well, I mean, look, it's never an easy process to name a book.

and there are an infinite number of options. We ran some tests. We tried a few different things. Ultimately, and we'll talk about this, I'm sure, during the conversation, the way that we teach OKRs comes down to that key question, who does what by how much? And so it just made sense for it to be the title of the book because that's sort of the embodiment of our strong opinion, our thesis in that book.

Give us a story what happened and how is that going? You were, I think, starting with your own publishing house when we last talked. So how's that been going? Well, it was super interesting. We ran a business called Sense and Respond Press for four years. And over the course of four years, we published 20 books. And we learned a lot about the publishing business.

And one of the things that we learned was that it's not a profitable business, at least not at the level that we were operating at. And that there was a lot of work that we needed to do to make our authors successful. And it was becoming too much, honestly, on top of everything else that we were doing. So after four years, we shut down the publishing business. We've maintained the

The brand and the organization, the company exists, Sense and Respond Press, and we publish our own books on there now. And then we're using that same brand and expanding it into a spinoff called Sense and Respond Learning to scale our training business around the world. And so we're building training partnerships with folks in different countries who want to teach our materials in their language, in their country. And those experiments are running now. We've got six or seven of those live right now.

Well, that's a fascinating model. Congrats. It's like franchise, but a little better.

Yeah, exactly. And franchising was exactly the term that we were using for a while until a lawyer advised me that franchising is very complicated. So just stop calling it that. But that's roughly what it is. By the way, speaking of publishing, we had a very cool interview here with Lou Rosenfeld of Rosenfeld Media a few episodes back. So we're just going to link to that in the show notes in case someone is in the mood for more book talk. But today we're going to talk about who does what and by how much. Congrats on the launch. Thank you.

You take a pretty interesting opinion stance for OKRs. OKRs is a popular idea. How is your angle on OKR special? Throughout our career, everything that we've talked about, whether it's Lean UX or Sense and Respond or trying to reconcile Lean and Agile and Design Thinking, we have focused on a theme of customer centricity.

Understand the customer, understand the problem that you're solving for them, make sure that you're solving a real problem for a real human in a meaningful way. And in theory, success will follow. And so we've carried that customer-centric theme into the OKR conversation, to the goal-setting conversation. Now, the leading book, the most popular book on the topic of objectives and key results is John Doerr's Measure What Matters.

That book came out in 2016 and was a very high level case study heavy book of here's how Google used OKRs and became massively successful. Here is how Bono uses OKRs. And without a lot of detail around sort of implementation or scale,

And without a lot of rigor around what makes for a good objective, what makes for a good key result. There's a variety of different types of examples in there. And Dora doesn't take a strong stance on kind of what's good and what's bad. We do. And fundamentally, the thing that we care about the most is that your objective is a

qualitative future state that you would like to create for your customer. And your key results are changes in the behavior of the humans who you're building products and services for. And so the measure of success is determined by answering the question, who does what by how much, right? And what you'll notice that's missing from that conversation is any kind of output.

features, products, apps, systems, campaigns, policies, programs. None of that stuff is in there, which is often how most teams get measured and how their work gets assigned to them. Build this thing, right? Deploy this campaign, launch this application. We're saying that the measure of success is a change, a positive change in the behavior of the people that you serve.

And you have to go out into the world and figure out whether you need to ship this feature or that feature, design it this way or that way, change your policies versus change something that you're actually building. There's a variety of different ways to influence human behavior. And it's up to you to figure out the best way to do that.

My lightweight personal understanding of OKRs was that the results could be a combination of outcomes and outputs because then you can have perfectly successful teams output, but your hypothesis was wrong and the outcome is not as intended. So what's your angle to solving this equation disparity between what you do and what you're aiming to achieve?

Look, I think the question that needs to be asked over and over and over and over again in every company, every time somebody ships something is, so what? Okay, you built it. Cool. Is anybody looking for it? Did anybody find it?

Did they use it? Did they use it more than once? Are they using it with the frequency that you expect them to? Are a lot of people using it? Are they telling their friends? Are they giving you money for it? Do they subscribe to it and continue to give you money for it? Right? These are all the behavior changes that we need to look for that tell us that we've actually delivered something of value. And I think that that's,

where the overwhelming majority of organizations struggle with OKRs and really just treat it as a rebranding exercise of their goal-setting framework. Everyone's doing OKRs, so we'll do OKRs too, but we won't change anything, right? The measure of success will still be delivering the thing. And delivering the thing is, to your point, it's meaningless if nobody actually got any value from the thing that you delivered.

You just named about five plus different ways, different metrics that can help you measure, for example, effectiveness of the feature that was built. So is there any guidance, any preference on what metric we should be using for those OKRs then?

Yes. Again, so the interesting thing about OKRs is that they are the specific things that people do in the system that you're building. You think about sort of two levels, right? At the bottom most level, you've got output, the stuff that you make.

Whatever it is that you make. Again, features, apps, systems, deliverables, whatever it is that you make for somebody else to consume. And at the top end, you've got revenue or money or profit or whatever it is. Everything that happens in between those two things, that's where your OKRs sit. They are measures of human behavior that affect business results. And so if you think about this from a design perspective,

you can outline the customer journey and every step in that customer journey can be a key result, right? So user lands on homepage. Okay, great. Then what happens? Then the user creates an account. Okay. Then what happens? Then the user searches for a product. Okay. Right. And so on and so forth and so forth. At some point we get to, we make money, right? But all those steps in the customer journey, in the user journey are candidates for good key results, right?

We'd love some examples here. And I mean, both OKRs themselves and what's the frequency we should be setting them and changing them, et cetera, et cetera. Sure. So look, I mean, there's a variety of ways to think about this. For example, you can say, you know, our objective is to become the easiest way for people to shop for furniture on their mobile devices. You know, let's just

geofence it in North America by the end of next year. That's a good objective, right? We'd like to be the easiest way for folks to shop,

for furniture on their mobile devices in North America by the end of next year. It's qualitative. It's pretty ambitious too. It's ambitious, but it's aspirational. It's inspirational. It's got a deadline. It's qualitative, right? We're just saying we'd like people to be able to shop this, to do this. Great. How do we know that that's happening? Now, traditionally, in all likelihood, in the companies where people work, people listening work, the measures of success would be things like launch the mobile app,

right? Launch the augmented reality feature, launch, you know, buy now, pay later, things like that, right? And maybe you do need those things and maybe you don't, and maybe you know how to implement those things successfully. And maybe you, you won't implement those things successfully, but

The measure of success of becoming the easiest way to shop for furniture on a mobile device in North America by the end of next year are things like how many people are signing up for the app? How many people are searching for furniture? How many people are adding furniture to cart? What percentage of people are completing the purchase? How many people are coming back and buying again within the next

six months or 12 months? How many folks are referring a friend, right? All of these types of behaviors, those are your key results. Those are your measures of success. How you get people to do that is the variable, right? So the output becomes the variable. So things like we're going to, we're going to launch a marketing campaign so that people download the app. Great. Did people download the app? No, they didn't. Okay, good. We have to try something else.

Okay. Or we're going to launch an incentive to get people to refer the app to a friend. Okay, good. What's the incentive? We're going to do a 10% on your order if you refer a friend. Okay, great. Did that work? Did people refer a friend? Yeah. 30% of new orders refer to friend. Okay. That worked.

So the definition of whether something actually worked, the definition of whether you built something, you chose the right thing and you implemented it well, you designed it well, you developed it well, is whether or not people change their behavior. That's the key here. The way it's often shown, you have the objective and one, two, three key results. In your case, the objective is the one you named and the key results would be

strive for 10% activation of the referral program and another metric in another project and one more. So it's not describing a single activity, but like a combination of different things you're doing. This is the interesting thing about OKRs are supposed to help you focus on the most important things right now. And so let's assume that we've got an existing system, right? An existing furniture buying system today. And we can see what people are doing. And what we've discovered is that

People have no problem finding us. They have no problem searching and finding stuff they like. They have no problem adding to cart, but then there's a huge drop-off between adding to cart and completing a purchase. Okay. So maybe our objective for the next three, six, nine months becomes something like to make it dead simple for people to buy furniture online by the end of the year.

or to complete a purchase, whatever it is, right? Cool. How do we know? Well, we want to see the number of abandoned shopping carts decrease by 90%. We want to see the amount of time it takes somebody to complete a purchase process, which today takes them five to seven minutes. We want to reduce that by 50%, right? Those are the types of things that we're looking for to tell us that we've made the checkout process better, right? So we're focusing specifically on that part of the system rather than the whole thing.

And so OKRs really do help us focus with that. And that helps us determine which of those behaviors we're going to focus on now. Because like I mentioned before, I was starting to kind of outline a customer journey. That customer journey is just one path and it could be multiple steps and there's multiple permutations of that path. And so we need to decide what's most important for us right now, strategically as an organization, and focus on those behaviors. I know you have

case studies in the book? Could you shed some light on a couple that you think are the most representative for our listeners? So there's a few interesting ones. And we really, really strived to find case studies beyond tech.

Because OKR is, you know, a relatively easy or should be relatively easy year in tech because our products and services are inherently measurable. So we pick things like there's a supermarket and restaurant supply company in Germany called Metro. They're huge.

200,000 employees. It's a massive company. And they provide, again, supplies for supermarkets. So groceries, it's a wholesaler and supplies for restaurants. That's what they do. Now the restaurant, the supermarket business is one thing, but the restaurant supply business is really interesting because what they need to do is they need to operate their warehouse and delivery functions properly.

on a high number of variables, right? So what's in demand right now, what's fresh, what's available. Is there a chain of restaurants that's having a promotion for a specific type of food? And so they've got to figure out, for example, how to reduce the time it takes them to pick and pack the orders that the restaurants bring in every day. So they've got warehouse behavior changes, right? So the behavior changes, these are internal

OKRs, right? The objective is to create the most efficient restaurant supply warehouse in Germany, right? Something like that. And the measure is reduce the amount of time it takes to pick and pack an order by whatever percent it is, right? 75%. Make sure that 80% of orders are delivered successfully and reduce waste, right? They've got a real issue with waste where they have to buy their food

and then sell it out, right? So they want to reduce the amount of food that gets thrown out by a certain percentage as well, right? And none of that says lay out the warehouse like this, hire these types of folks, put in this type of warehouse process. They keep experimenting with different designs, different processes, different incentives, get folks to behave differently.

There are some amazing stories in the business books, for example, that airline that decided to focus exclusively on optimizing takeoff time so that they always take off on time and that suddenly magically transformed their business. Looks superb in retrospect. In reality, it's not like we know what's going to really make such a wonderful North Star metric. What are some of the recommendations that you make in the book for this?

The reality is, look, like to optimize for on-time takeoff is a strategic one. That's a company strategy. And when OKRs are anchored in a clear and well-understood company strategy, we can make decisions around that. So for example, right, let's say that whatever airline this is,

has strategically decided that their brand differentiation moving forward for the foreseeable future is on-time departures. Okay, great, right? What drives on-time departures?

Well, it's getting people to show up at the airport on time. It's getting people through the check-in process quickly. It's optimizing security, if at all possible. It's improving the amount of time it takes to load an airplane, both with people and with baggage. And again, those are all key results, right? Those are all measures of human behavior. And so then the organization has to decide, you know what, like we can get

We can get people through check-in pretty quickly. We're pretty good at that. We have no control over security. And so where we as a team are going to focus today is we're going to focus on designing the best possible way for people to board an airplane efficiently and quickly, which if you've ever flown, you know that there's a thousand different ways that airlines do this. And there's a reason.

for that because there's no one right answer, right? There are a variety of different answers that work for your particular airline. Maybe, right, there's some airlines, like especially the low cost airlines like Ryanair and Southwest and the US, et cetera, that have standardized on one airplane, right? It's one, like they don't have a variety of different airplanes. They have one airplane, they have one configuration. And that was a decision that helps them optimize, you know, the loading of plane, which then drives the strategic goal of taking off on time.

And so the advice here to kind of bring this, to make this real is the most successful thing that you can do to have OKRs that are meaningful and focused is to understand your company strategy. And if you don't understand, ask, what do we care about strategically? Because again, like when you map out these customer journeys, there are going to be dozens of different customer behaviors you can optimize.

To determine which one you want to work, which ones you want to work on, you need some kind of focus and strategy helps with that. Interesting, because when I was asking this question, takeoff time sounded like a potential key result. Yeah. You know, let's say objective being improve customer satisfaction and takeoff time,

is just one of the ways that that can be measured. Yeah. Right. But in your case, you're using it as an overarching strategic focus, which then leads to other OKRs. So think about this. So that's really great clarification. I'm glad you brought that up. Right. So objective is, you know, be the most on-time takeoff airline in the world, right, by the end of the year. Okay. Key results, 90% of flights take off on time.

Okay. Who does what by how much? Right. Like that's the question. It could be like, let's break that down. I think it's a high level key results. That's a good one. Right. What are the leading indicators of that? Because we've talked about customer behavior, like actual passenger behavior. Let's be specific. Right. We've talked about what passengers could potentially be doing differently. What can the flight crew be doing differently?

What can the pilots be doing differently? What can the baggage loaders be doing differently? And so I think all of those, you can measure behaviors, right? So baggage loading takes 50% less time, right? Flight crews can clean a plane in under three minutes. Pilots do safety checks 20% faster. So that's where this becomes really interesting, right? Because if you keep asking that question, who does what by how much, the who is

starts to reveal the various personas in your value chain, right? So if you're a designer and you're thinking about all the personas that you could potentially impact, in the airline industry, you've got pilots, you've got flight crew, you've got mechanics, baggage loaders, right? The airport staff themselves, you've got passengers, security staff, et cetera. And so really thinking through all those folks and deciding who you're going to focus on first.

Because we're not going to boil the ocean and try to fix it all at once. But that's a really, really good question because it starts to reveal the complexities of this and understanding sort of where we might want to design a better experience. And you also clarified an important aspect because to an amateur eye, it seems that behavior change means the customer behavior change. But obviously, it's also organizational behavior change that can be a key result. Well,

One of my favorite examples is, and we use it in the book and I use it in my teaching. And sadly, they didn't give us a case study, but the good news is they published their OKRs publicly. So they're easy to access. There's a healthcare organization called the Cleveland Clinic. Cleveland Clinic is a high-end healthcare organization. It's a hospital. It's a global organization, but as you might guess, the headquarters is in Cleveland.

Kind of a no-brainer there. And they publish their OKRs publicly every year. And one of the examples that we use, the CEO's OKRs, is they have an objective that says, best place to receive care anywhere, right? Which has, it's an externally facing objective, but the key results are measures of staff behavior.

They talk about the absolute number of serious safety events that happen. They want to reduce the absolute number. They want to reduce the frequency in which serious safety events take place. The behavior change there is staff behavior change, right? Which is really interesting. The other one I like that they have is they have best place to work in healthcare.

That's a really interesting one as well. That is also an internally facing objective. And the key results are, you know, percentage of staff that would recommend this as a place to work. And this other thing that they called regrettable turnover.

regrettable turnover is the number, the percentage of people who quit every year voluntarily. Voluntary turn. Yeah. Voluntary turn, basically. Like why would anyone leave here? And so that, that is an internally facing objective and key result. And it's up to the organization to create the kind of place that people want to work and want to recommend to their friends and don't want to leave as well. And so your OKRs can be customer facing, but they can also be organizationally facing as well.

Let's talk about our niche product development, design, software products, our line of work. Sure. What's the role of the design department here? What's the role of product managers and how do they all work together in this framework? Yeah. So if you've read any of my stuff, my God, there's a lot of stuff to read. So you got a lot of choice. I've been writing for a decade and a half books and blog posts, but if you've read any of them,

There's a couple of themes that come up over and over and over and over again. One is customer centricity, but the other one is cross-functional collaboration. Talk about that from the beginning, from the Lean UX days, we've talked about cross-functional collaboration. Cross-functional collaboration means that product design and engineering, at the very least, the leadership of those three disciplines are collaborating on a regular basis on a project

at the same time, while working on the same thing at the same time. That's an ideal state, and I recognize that's not reality for a lot of folks or a lot of teams. That said, when you're dealing with objectives and key results, specifically, designers need to participate in that conversation.

So that cross-functional collaboration needs to take place. Designers are close to the customer, they're close to the user. They can bring in that point of view about what we might focus on, what's currently not working well, what we might be able to influence, and that helps to determine

what key results, what behavior changes we might work towards. Now, the product manager may ultimately make the decision, right? That's usually their job about which key results we'll work towards, but designers need to be a part of that conversation. They need to care about that conversation. So if you're not getting invited to those conversations today, ask, ask to join, even if it's just to listen and then eventually be able to participate, right? Because the engineers are likely going to be there and they're going to talk a lot about feasibility,

And honestly, feasibility is important when we've decided on a particular direction. But since our measure of success is not, did we deploy the thing? It's, did we change human behavior? We need people in the room who are advocates for the user and design plays a crucial role in that. I strongly believe you need to be in that conversation.

I'm curious, in your training educational business, when you're working with other teams, how do you collect feedback and just overall gauge how your method is applied in reality? Because honestly, as a software founder, reality is very intricate and very not perfect. So how do you make sure the method syncs with what people are actually doing in real life? The nice thing about writing books is

is that you can paint an ideal picture. Exactly. Yes. Here's how it needs to look. And then you go out into the real world and it's messy and it's never as clean and clear and crisp as it is in the book.

The reality is that the data that you're going to collect is rarely going to be definitive, right? The decisions you'll have to make are going to be based on mixed results or incomplete data or who knows what. And so I think the most important things that you can do to start to make this a reality is to start to define success at the outset of an initiative and

with some understanding of who the customer is. And if we choose the right thing and we design and deliver it in a compelling way, what do we expect those people to be doing differently than they're doing today? That's an easy conversation in the sense that it's an easy question to ask. It may take a while to resolve the answers to that, but that's really why we call this book, Who Does What by How Much? If you can, at the beginning of every initiative, if you can just kind of say, hey, when it comes to defining success, right?

Who does what by how much, roughly speaking, right? Who are we building this for again? Who's our target audience? Let's be specific. Who's the target persona? What are they doing today, right? Well, today they're doing X, Y, and Z. They're really struggling with Z. Okay. And when we deliver this thing and we do a great job with it, what do we expect to see? We expect to see them getting through the process 50% faster. Okay, good.

At the very least, and if you do nothing else with this, if we just start there, at the very least, after you ship your thing, whatever the thing is, you can now go back and say, hey guys, remember at the beginning of this initiative, we said that we're going to try to get our folks through the process 50% faster. How's that going? Well, I think maybe they're getting 10, 12% faster, but we didn't get to 50%. Well, why?

Well, we don't know. Well, let's go talk to them and let's do research and let's do prototypes and let's do design work to go figure out

how we can make this better, what the next iteration is, right? And so there's a lot of power in starting to have a conversation around human behavior. First of all, design becomes explicitly in demand, right? Because the measure of success starts to move away from building a thing to changing behavior. And so we have to go out, we have to do research, we have to talk to customers, we have to build prototypes and do early designs and show things to people and figure out kind of what stands the greatest chance of success.

And as we begin to collect evidence, right? So again, it doesn't have to be perfect, but as evidence starts to come in and says, you know what, this idea is working, this idea isn't working, that evidence allows us to change course. And changing course based on evidence is being agile. That's agility comes from collecting evidence and changing course. And so to me, like if you can just start with the very basic question, just as who does what by how much at the beginning of an initiative,

And again, just wait till the end at this point. We're not trying to change everybody all at once. We're just saying, look, did it do it? If it didn't, let's go figure out why and try again. That's the whole point of software development today is iterative. Let's use that power to make things better continuously. Thank you for sharing an insight on this. As the ultimate piece of advice, if there is a 20% SaaS team who wants to start doing OKRs tomorrow,

what's their one, two, three steps they should be taking? So number one, figure out what your strategy is. Ask around. If you don't know what it is, see if you can at least take a guess. Strategically, what does the organization care about for the next six to 12 months? I think that's really important. You can do OKRs without strategy, but there's a lot of alignment issues that tend to happen in that situation. So number one, understand the strategy. Number two, within that strategic space,

decide what problem you're going to solve first. There's going to be a lot of opportunity within that strategic space, right? What's the most important problem you need to solve first? It doesn't mean you're going to ignore the rest, but you can only focus on one thing at a time. So what's the most egregious thing we need to deal with immediately? Okay, great. If we solve that thing, who does what by how much, right? What will people be doing? Which people

And what will they be doing differently when we solve the thing? So at that point, what you've done is you've picked a strategic direction. You've got a clearly defined problem space, which creates guardrails and constraints, right? So what's in scope and what's out of scope. And then you've set success criteria in terms of human behavior. And then from there,

The rest of these sort of lean startup, lean UX, et cetera, design thinking processes begin to happen naturally because what will happen is, okay, well, we want people to reduce abandoned shopping carts by 90%. Okay, great. How are we going to do that? I have an idea and you have an idea and that person over there has an idea and who's right? We don't know.

We've got hypotheses. So now we've got to build some experiments. So all that stuff starts to flow in theory with those initial three questions. What's the strategy? What's the problem? And if we solve the problem, who does what by how much? What was the behavior change we expect to see?

If our listeners want to learn more, if they want to read the book, where can they find it and where can they find you and your other books and writings? So the Who Does What by How Much is on Amazon. If you go to Amazon and you search for my name or Who Does What by How Much, you'll find it there. And it's in digital, it's in audio and it's in print. And that's a great place to do it. And if you do read it, I would very much appreciate a review. Reviews go a long way.

Everything else about me, you can connect with me on LinkedIn. I'm very active on there these days. Or you can go to my website, jeffgotthealth.com. And my blog is there and all the other info is there as well, including what I do for a living. Awesome. Thank you so much for your time. And I hope your work is influential and more organizations work with better wikiares. I hope so too, Jane. It's a pleasure to see you. And thanks so much for having me on the show again. Have a wonderful rest of your week. Thank you.