'Would you recommend this show to a friend?'
46:52 Share

Support for this podcast comes from On Air Fest. WBUR is a media partner of On Air Fest, the festival for sound and storytelling happening February 19th through 21st in Brooklyn. This year's lineup features SNL's James Austin Johnson and a sale of Death, Sex, and Money and over 200 other creators.

Will, talk to me. ABC Tuesday. They took his daughter. She's coming home alive. Will Tread, the series critics are calling powerful must-see TV, continues to thrill. Shouldn't we strategize before we go in there? If we screw up this case, a cop killer walks free. With a riveting conclusion to a two-part season premiere. Get down! Will Tread, all new Tuesday on ABC and stream on Hulu.

This is On Point. I'm Debra Becker, in for Meghna Chakrabarty. Today, we want to talk about those seemingly ubiquitous customer surveys. Doesn't it feel as if we're flooded with them? The grocery store, the doctor, the pharmacy, the rideshare, the online retailer. Why are so many companies asking, how did we do? How likely are you to recommend us to a friend? So, we surveyed some On Point listeners.

Asking me to fill out a survey gets you one star. I've grown so weary of all of these surveys and customer satisfaction pushed in front of me, whether it be by text or email or on my receipt. It's always linking to something else and more questions.

Rate your service on a 1 through 10. Fast checkout, was it friendly associates, was it we got the right stuff in stock? We're always told to tell the customers to rate us a 10. Most people don't fill them out. I think we, on average, had maybe a 1% return rate on that. I do occasionally fill out the customer surveys. Normally it's when I have something to complain about. All it takes is one bad survey to ding your score for that week.

It basically ranks your servers from best to worst. If you're getting a 10 or a 9, great. Essentially anything at like 7 or below is a 0. I don't believe my information is going to actually improve the product or service.

They probably gave us a low rating because we didn't have the eggs that they wanted. That's something we can't control. If there's no eggs in the warehouse for us to get to the store, there's nothing we can do.

Those folks called us from Virginia, Wisconsin, California, Indiana, Washington, Georgia, Colorado, and New Jersey. So where are these surveys coming from? Do they work? And what is happening with all that survey data? Joining us is Fred Reichelt. He's a fellow at Bain and founder of Bain's Loyalty Practice. Fred, welcome to On Point.

Nice to be here, Deborah. So I understand that we should be thanking you, blaming you maybe, for the question. How likely are you to recommend us to a friend? You came up with that survey question for businesses. Is that right? I did. I invented it. You called them ubiquitous. I would call them annoying. But in all honesty, they started out 20 years ago or more

very effectively because I saw a company, Enterprise Rent-A-Car, who had grown to be the largest in the world by treating customers better than their competitors. They've been a great success story. And one of the things they used that I admired was a very short survey, two questions. And they didn't send it to billions of customers. They got just enough customers at each branch to have a reasonable response that's trustworthy.

And they didn't design it as a big market research project for corporate to learn more about their customers. They designed it for each individual branch to learn how to get better. It was genius. And I just took that

and used it as a model to build something I called Net Promoter Score, which is a simple way almost any business can measure, of all the lives I touch, how many are enriched, how many are diminished. And that's a pretty worthwhile goal. And basically, Net Promoter Score means, will you promote to someone else? Will you recommend? One question, how likely would you recommend this to a friend, zero through 10? Some people use a five-star system, same idea. And

And, you know, when you get a 10, at least in the old days when people could be honest without fearing that their service provider would be fired or dinged in their bonus, a 10 meant that you've made my life better. I'd recommend you to a friend, which is the highest standard you can have. But how do you know that? I'm wondering where you get the data or where maybe Enterprise got the data. How do you know that when someone is saying they would recommend that that is accurately reflective of

of the quality and service of the product that they're being asked about. They could be talking about something completely different. How do you zero down on exactly what the customer is talking about when they might be a promoter of that business? Well, Enterprise actually didn't use likelihood to recommend as their question. But I wanted to test. This was the young Fred, economist, math and statistics major. I

tested which question best predicted future customer behaviors that would have indicated that their life was enriched. They expanded their purchases. They made referrals to friends. And we tested how satisfied are you? Would you recommend to a friend? Does this company deserve your loyalty? Do you intend to return all the classics?

But I knew people's time is precious. I just wanted one question as the scoring question. And it turned out likelihood to recommend was the best predictor of those real behaviors. And that's why we picked that one question. And you think it is reflective then of really customer satisfaction, which means what for a business? Well, the highest standard is

is that your life has been so enriched by an experience that you want to share that with a loved one, thus the recommendation. So a recommendation is an act of love. I want my friend, my colleague to have this wonderful experience. I trust this brand, this company is going to do it for them. That's pretty awesome. It also turns out to be that's what drives

company success. But since companies don't measure it very well, they don't see it. They think accounting measures success, which actually takes them down the wrong path more often than not. But also there could be misinterpretations of some of the answers, right? Of course. Think about Amazon with their rating system.

They want you to give a star rating, but is it about the delivery experience or the shopping experience or the product itself or the service that did or didn't get applied to that product? It gets fuzzy.

I'll take my car dealer who is just always begging for 10s because somehow I know that they must fire the bottom 10% of salesmen based on how many 10s they get. Only a 10 is a passing grade. How many times have I heard that?

The reason I'm not happy with my car is the battery in the key fob went bad after three weeks of this brand new car and left me stranded in a parking lot. It had nothing to do with the salesman, but he's still afraid that I'll give a low score that will affect his future.

And why do you think it is that we are so inundated, annoyed, as you might say, with these surveys at this particular time? Why are so many companies deciding that now is the time to do them? What does it do for the businesses involved?

Well, Aristotle understood this hundreds and hundreds of years ago, that there is a tragedy of the commons. When there's a free good that everyone gets to use or abuse, eventually it gets abused. Whether it's the common, the village green where they graze the cows or the forest that gets cut down by if everyone has rights or fishing.

It's a common phenomenon. And that's what's happened with these surveys. Customer goodwill has been abused by people misusing surveys.

And we're now at the stage where it needs a little bit of reinvention and rethinking. But why? What was the point of using these surveys to begin with? Aristotle didn't have the Internet. Let's talk about it. Well, I hope they read my book. I made a pretty strong case that companies who do the best job of earning the loyalty of their customers are the ones that grow profitably, make their employees proud, and their investors rich.

There's a really strong economic argument that underlies this idea.

And I think leaders intuitively understand that. You don't want to lose customers at a rapid rate and have to refill that leaky bucket. That's terrible. And the notion that, gosh, a referred customer is a higher quality customer. They stay longer. I didn't have the acquisition cost. More referrals makes everyone rich. So there's common sense underlying it as well as my more bane rigor analysis. But I think the reason it's falling apart

is because it's being misused. People are linking scores to employee bonuses, and therefore the employee doesn't really care whether you were happy or not. They just care about your score. Right, right. And so then how do they need to be restructured? What would be your main recommendation in terms of restructuring them? Do less of them or not link them to employee compensation? Or what do we do? I would do way less. Well, for me,

One of the interviews that you had played a little bit earlier on the show said only 1% of people respond to these surveys. That's about right. So 99% of survey requests are being ignored.

Now, that has a cost. Aristotle knew this cost. The customer is saying, this is an inappropriate waste of my time. It's a communication that's wasting my energy. It's bad. It weakens the brand.

But 1% are responding. Now, are these 1% typical of your customer base so you can actually learn from them reliably? I don't think so. The other, you know, why, you ask, what do I do? Yes, far fewer surveys. They're obviously failing.

And what do you replace it with? I need to know whether I'm enriching a customer's life. Am I earning their loyalty? Will they come back for more and bring their friends? That's the core engine that drives prosperous business growth. I think in today's digital world, it's simple. And this is a case I make in Winning on Purpose, my last book. Just track

Repeat purchase and referrals. Back for more and bring your friends. Those are signals that are in your own accounting system. Well, at least back for more is in your accounting system, but referrals can be tracked today. And use that as the signal of whether you made the customer happy or not. Then if you need surveys or outreach with conversations and so forth, great.

But much of the learning can be done just with analyzing data that already exists in your system. But I still don't understand, and we've only got about a minute before we take a break here, but why with a 1% response rate, so many businesses have decided that this is the vehicle that they want to measure their customers by? They're desperate to hold someone accountable.

for delighting customers. The CEOs know if they don't make customers happy, their company won't grow. So they've delegated down into the silos of the organization

this survey methodology that they think will hold those people accountable to turning customers into happy customers, loyal customers. Okay. We're talking about customer surveys. Fred Reichelt, who's a Bain Fellow and founder of Bain's Loyalty Practice, is with us. We'll have more in just a moment. I'm Deborah Becker. This is On Point.

Support for On Point comes from Indeed. You just realized that your business needed to hire someone yesterday. How can you find amazing candidates fast? Easy, just use Indeed. There's no need to wait. You can speed up your hiring with Indeed.

and On Point listeners will get a $75 sponsored job credit to get your jobs more visibility at Indeed.com slash On Point. Just go to Indeed.com slash On Point right now and support the show by saying you heard about Indeed on this podcast. Indeed.com slash On Point. Terms and conditions apply. Hiring? Indeed is all you need.

At Radiolab, we love nothing more than nerding out about science, neuroscience, chemistry. But, but, we do also like to get into other kinds of stories. Stories about policing, or politics, country music, hockey, sex.

Of bugs. Regardless of whether we're looking at science or not science, we bring a rigorous curiosity to get you the answers. And hopefully make you see the world anew. Radiolab, adventures on the edge of what we think we know. Wherever you get your podcasts.

This is On Point. I'm Debra Becker. Today we're talking about customer experience or customer service surveys, why it seems like we're getting so many of them and where they're coming from. We're joined by Fred Reichelt. He's a fellow at Bain and also with us right now is Brad Anderson. He's president of products and engineering at Qualtrics, which is one of the largest survey data companies. Brad, welcome to On Point. On Point.

Hey, thank you. I appreciate you being here. We want to note that Boston University, which owns the license for WBUR, where On Point is produced, is a customer of Qualtrics. BUR is editorially independent. And Brad, I just want to...

We want to get a look at sort of how big this survey industry is. And I've read that Qualtrics has analyzed billions, with a B, billions of survey responses. And your business continues to increase even more than you've projected. So I'm wondering, can you explain, you know, how many surveys Qualtrics is currently analyzing and businesses you're working for? How big is this industry?

Yeah, you know, so in 2024, we had 1.2 billion surveys come back to the platform that we analyzed. That actually grew 15% year over year. At the beginning of the year, to your point, I had projected that surveys would grow between 4% and 5%, so it grew much, much faster than I thought that it would.

And so, you know, survey fatigue is a real thing. And so getting it right in some of the points that Fred made is just critical. And so generally then to see this kind of growth, why? Why? It's the same question that I asked Fred. Why are we seeing so many businesses doing surveys? I can understand that you want to know whether your customers are satisfied. But is it because it's less expensive and easier to do this and we have tools to do this online? It takes, you know,

minutes to do an email or send a text, or is there something else happening that you really need to try to hold on to customers and understand where they're at and perhaps improve their experiences and all of those things? Why are we seeing so much of this now?

I think organizations, as they see so much of their business going digital, are struggling to understand how do they build that connection with people. When everything goes digital and if you've just got some kind of a flat experience, it isn't personalized, it's very hard to build that relationship. And the brands that build relationships with their users are the ones who get the points that Fred was making, repeat, furls, all those kinds of things. And so really what I, as I'm talking to the organizations who I would classify as the leaders in

in really listening and understanding and most importantly, acting on customer feedback. It really is about focused on that personalization and building a relationship with people. Mm-hmm.

And so when you design a survey for a particular business, I'm assuming that you go in and work with folks to design the questions. I mean, what are the rules for a general rules for what you would say is an effective way to reach out to customers to make that connection and get the company what it wants in terms of customer data, but also make sure that the customer responds to the questions? Yeah.

Yeah, a couple of things come to mind. First of all, you want to be brief. You don't want to ask too many questions. We have, you know, more than 100 IO psychologists on staff here that we can have work with organizations as they're developing the questions in a way that is, you know, we know that they'll be answered. And so there's a couple of things that I would say. First of all, be brief.

be crystal clear and don't flood your people, you know, your customers with emails. It's interesting, you know, as you're talking the previous section, I was just pulling up some data here while I was listening in our leading customers, the customers that I would say are really doing the best and getting this feedback are actually getting 13, 14, 15% response rates on their surveys a lot more than 1%. But I think one of the most important things here is, is there are so many other ways to listen in to what customers and employees are saying and

that are not surveys. And that is, go ahead. No, no, go ahead. Okay. So what I'm seeing is, you know, let me, let me give you an example. Take a, you know, a hotel or an airline and,

In a given year, they may receive five or six million surveys back. But as we pull down from them all the reviews that are being posted on all the social sites, that's another 10 million. As we pull in their chats and the calls that they're having with customers, we analyze all that. That's another 20 million interactions. And so what I see is surveys are definitely growing. But what is growing much, much faster is organizations listening to what customers are already saying about

about their experiences without having to ask that. And that actually is growing four times what surveys is. In 2024, the number of chats, reviews, calls that came through our system was 2.2 billion, and that grew 60% year over year. So it's basically monitoring customers' online activity to get a fuller picture of exactly what the customer experience might have been. That's right. Okay.

But I wonder, you know, when you said, you know, minimize time, designing questions, a hundred psychologists on staff, that number really I found really astounding. But I guess, you know, is there sort of a general framework? So when I get an email, say, from someone that I've purchased something from, I tend to get a few repeat emails. They give me a deadline by which to respond. I mean, what's the general framework there in terms of how many emails? Yeah.

might be okay and not annoying? Do you give a deadline? You know, do you compensate people for responding to a survey with a gift card or, you know, 10% off their next purchase? I mean, what are some of the general things that you've found that perhaps give you that higher response rate than what Fred had mentioned initially? Yeah.

The biggest thing that we see is the proximity of the time when you ask for the feedback relative to the experience is actually the biggest driver of one, response rate, and two, quality of the response. For example, if someone has an experience and they get a link, whether it be an email or a text to take a survey, and it's two days, three days, five days later, they've already moved on.

But if you can actually collect that feedback in the moment, we'll talk about that in a minute about how AI is changing all of this. But if you can collect that feedback in the moment or immediately thereafter, you do get much better response rates and higher quality data. So do it fast. Don't do it repeatedly. Exactly.

That's right. And so we have rules within our platform that a brand can go in and say, hey, if a customer has received a survey from us in the last 30, 60, 90 days, don't send them another one. And so we have all of these things that we have built in to help organizations govern that. But I'll tell you where this is heading to is, you know, what we have been innovating on is how does generative AI just completely transform a survey?

If you think about historically, a survey is a flat, non-personalized, you know, just a set of questions. It's interesting. The average survey has 14 questions in it. And in that average survey, five of them are open-ended. It just tells what you think, not multiple choice. So what we have been doing now is applying generative AI to the survey.

And so, for example, taking, you know, what Fred and team built, if someone comes back and says, how likely would you be to recommend us? And they say a seven. The first question every organization has is, well, what could we have done better to get a nine or a 10?

And so our AI could detect that and then automatically say, well, what could we have done better? And then as that feedback comes in with generative AI, we're actually able to have an interview with the customer, which gets much better data. Okay, we're going to talk about AI in just a moment. Let's just talk about what typically is being done right now with these surveys and why they are so important.

so ubiquitous. And so let's talk about in terms of what companies do with the data. Let's get to that because you may be able to collect all kinds of data and generative AI may make it so much easier, but the company's got to do something once it has that information about its customers. And so typically then where do you come in and advising businesses on what they do with customer data?

There's two things that I think really make the difference here. The first one is culture. You talked earlier about reward systems and how people are rewarded. The reward system does have a profound impact on how any organization works.

And so getting the reward system right to where you just don't have, for example, salespeople at a car dealership trying to game the system, but really having a culture that says, we are curious, we are inquisitive, we want to understand what we can do better. And that, for the companies that are really doing this well, starts at the top. When I say starts at the top, you have CEOs who are not saying, we have to have an NPS of X.

They're saying, we want to deliver an incredible, unique, differentiated experience. How do we learn what that is? That's the first thing. And then the second thing is you have to take action. You know, there was one of the comments from one of the callers that said, I don't believe anything's going to happen from this. Right. I think that's probably why a lot of people don't respond. They don't feel that it's worth their time. That's right. And so if they don't see action and that action can be personalized, that's

then the likelihood of them filling out another survey for you is going to go down. Can you give me an example of a company that demonstrated that it took action because of customer feedback? I mean, I'm just not thinking of anything off the top of my head. Is there one that you know of that said, our customers said that this wasn't working for them, and so they took action?

Yeah, I'll give you a couple of examples. Let's start with Hilton Hotels, which is a global brand, 8,500 locations. And what they did is instead of asking questions all the time, they started listening in to the chats that were coming in through the application. They were listening in to what was being posted up on social websites. And what they found is for families, the number of towels in the bathroom

as a profound impact on the satisfaction of the state. And their comment back to me was, we never would have known to ask the question, do you need more towels in a survey? And so this ambient listening coupled with surveys really gives an organization the ability to understand in a much more broader way, in a much more comprehensive way, the things that they can be doing to delight their customers.

That's one example. I'll give you another example, which was Delta Airlines has talked about what they are doing.

it's all about understanding and personalizing the experience. And so, you know, now when you go onto a Delta flight, the in-seat console actually is personalized. You can log into that with your SkyMiles number. It understands who you are.

It understands what your preferences are. Back in the system, you know, we've collected things like does the individual, if there is an issue with the flight, do they like a red eye? Do they want a direct flight? Do they want a hotel? So we have all those kinds of preferences that we have collected. Now when Delta Airlines takes action, like, you know, there's been a flight canceled, they can take all the preference data that we have collected and then personalize the action that they take in the moment to fix that situation.

Brad, as a journalist, when you say things like collected preferences that I might have as a consumer or listening in to chats and aggregating data for a business, it makes me nervous. What kind of consent do you need from a customer to be able to aggregate this data and collect preferences on someone?

Yeah. So, you know, when you call a call center, one of the things that you'll hear almost every time you call a call center is that little recording that says, this data may be used to improve the product, improve the quality of the service. That is what our customers put in front of things like calls. For things that are posted on social sites, that's public data. So we can pull that back and don't have to ask for approval. You know, the individual has given their approval when they post it.

And then, of course, as we think about our survey data, you know, there's what we do is we give the individual the ability to identify what things they want to share with the company that's using us.

And so, you know, if you're asking a set of questions about, you know, what are your preferences here, we're not trying to discover this. What we are doing is taking what the customer tells us they want the company to know about them, and then that's what we store. So we only store in the profiles that we create things that the customer has said, this is what I want you to know about me. So legally you're covered. Oh, absolutely. Mm-hmm.

And it's honestly less about the legal piece. It's less about being legally covered.

It's about what is doing right and doing things in the right way to where customers actually have more confidence, more trust with their organizations that they're getting services from. So that really is the focus we have. You know, Fred Reichelt is still with us, and I want to bring him back into the conversation here. And Fred, I'm assuming you've been listening to much of what Brad has said, and he said a few things that were different from what you said, particularly in terms of the response rate.

to customer surveys. He says it's, you know, more like 15%. You said it was about 1%. And also just the size of evaluating 1.2 billion survey responses and the enormous growth of companies like Brad's, like Qualtrics. I mean, what are your thoughts, Fred, when you hear some of the things that we've been talking about?

Well, I have, let me say, I greatly admire companies like Qualtrics. My daughter, Jenny, works at Qualtrics, so I know it well. So there's a disclaimer. Don't give them your only daughter if you don't think highly of them. We love Jenny. And yes, there are ranges of response rates. I think the best companies can find ways to increase response rates.

But I'm still, you know, I might say, hey, if 85% of your survey requests are still being ignored, that can't be considered a wonderful winning process. There is something fundamentally wrong there. So what do I see in the great, I'll go back to Enterprise Rent-A-Car, where I learned this in the first place.

two of the core concepts, three concepts. One, local learning. This isn't for Big Brother at headquarters. This is for the local team. Two, two questions. Not going to waste anybody's time. And three, which I didn't mention, if you get anything other than a high score, if you get a detractor score,

there is a requirement that someone from that branch close the loop, calls the customer, apologizes, tries to fix the problem, and then figures out a way systematically to fix it for any other future customer. So when I look at a company and say, do you have a good net promoter system, a good feedback system? I ask them, what percentage of detractors have a closed loop?

within 24 or 48 hours. And most companies just throw up their hands and say, "Oh, I could never do that. I don't have enough people." And that's saying, "Oh, so your time is valuable, but the customer's time, and especially a customer has told you they didn't get what they paid for, we just ignore that. What a screaming communication that I don't really love my customers."

And Brad, what do you tell the businesses you work with about that, about sort of following up with the customer? We only have a few seconds here before we have to take a break. Yes. You know, Fred and I are actually aligned on the things that he is saying. You know, you've got to do this right. So be brief, be clear and close the loop. And so the other thing that I would say here is, you know, NPS is a system. It's a culture system.

And so the organizations that really are using NPS to the fullest extent have infused it through the culture. And I think Fred's measurement of how many times are you closing the action is the absolute best measure. Okay. We're talking about customer surveys and why we seem to be getting so many of them and also what happens with that data. We'll be back in a moment. I'm Debra Becker. This is On Point. On Point.

Support for AI coverage in On Point comes from MathWorks, creator of MATLAB and Simulink software for technical computing and model-based design. MathWorks, accelerating the pace of discovery in engineering and science. Learn more at mathworks.com.

and from Olin College of Engineering, committed to introducing students to the ethical implications of artificial intelligence in engineering through classes like AI and Society, olin.edu.

This episode is brought to you by Amazon. Sometimes the most painful part of getting sick is the getting better part. Waiting on hold for an appointment, sitting in crowded waiting rooms, standing in line at the pharmacy, that's painful. Amazon One Medical and Amazon Pharmacy remove those painful parts of getting better with things like 24-7 virtual visits and prescriptions delivered to your door. Thanks to Amazon Pharmacy and Amazon One Medical, healthcare just got less painful.

This is On Point. I'm Debra Becker. Today we're talking about customer surveys. At some jobs, customer survey results can determine how much money people make. Jaysa Wright in Littleton, Colorado, spent four years as a server at a chain restaurant. She says her restaurant had tablets on each table where customers could pay their bill and fill out a survey. All of this data from the surveys is put into a little poster in the back of the kitchen.

available for all staff to see. It basically ranks your servers from best to worst. This could be kind of fun if you wanted to see how you were doing compared to your other co-workers, but it could sometimes be stressful if you had a bad serving day.

Or sometimes management would use those rankings to either give or take away shifts based off performance. So basically how customers answered a survey could affect how many shifts she worked. And at car dealerships, where many salespeople work on commission, high customer service scores can mean more money.

Clayton Mantell is a sales manager at Ramsey Auto Group in New Jersey. He says an even slightly lower score could mean missing out on tens of thousands of dollars in bonuses, depending on how many cars someone has sold. If you're getting a 10 or a 9, great. But if you're getting an 8 or lower, you're basically screwed. Essentially anything at like 7 or below is a 0. You'd think, well, just tell your customer. But that's also a survey metric that typically...

would lead to you not being paid if you don't have like 90% of your customers saying you didn't influence the survey in a specific manner. I routinely see guys get beat out of money when customer comments come in with it. Great, best salesman ever, stuff like that. And Mantella's concerned. He says customers might also take off survey points for things that aren't related to him, like maybe the dealership's vending machine is out of a particular item or they're just having a bad day.

People buy cars, having walked out the door in the morning, getting told, you know, their wife's cheating on them and she's out and she's taking the dog, you know, and they walk in there and like, you got to be the guy to talk to. And we want people to have 10 out of 10 experience, obviously, with our dealership.

To be rated on a scale that's like a warped funhouse mirror is just crazy. Mantell says there are industry stories of car salespeople, quote-unquote, forgetting to send surveys to customers who they think might leave bad reviews or even getting their co-workers to send in reviews instead. I believe Hyundai back in the day used to do a phone interview survey.

And the dealer could just basically put in like their buddy's phone number or a pay phone or whatever and just tell them great news, stuff like that. I'm sure that happens. You know, it doesn't happen a lot, but with so much money on the line and the goalpost being so high, like it's going to happen. That's Clayton Mandel, general sales manager at Ramsey Auto Group in New Jersey. We're talking about customer surveys this hour. And with us is Clayton.

Fred Reichelt, he's a Bain Fellow. Also, Brad Anderson from Qualtrics is with us. And I wonder, Fred, could I get your reaction first to this idea of compensating employees based on survey results and the inherent conflict in that? What can you do to get around that to make sure that folks are delivering good service, but also make sure that your survey results aren't skewed here?

This is a vital, vital issue. And if you read any one of the books I've written that defines the net promoter system and guides people toward best practices, you will read these words. Do not link survey scores to employee bonuses. Only bad things will happen. And it might take a year or two, but eventually employees feel like the job is to get a high score no matter what it takes because they get desperate.

It's a horrible idea, and I know companies do this with good intention. They want to show that they're serious about this, and they take scores just as seriously as they do accounting results. But really, surveys are soft. 1% response rate, sampling error, all of the biases and cheating and gaming from the ideas you just reviewed.

Don't use surveys in the wrong way. That's part of why we have 1% response rates. But if customers need to feel that what they're telling a particular business is meaningful and they need to see action, you know, it would almost seem like maybe it is a tool to help. Maybe you don't directly link bonuses or shifts worked ahead.

at a restaurant to it, but it could help guide someone if a particular employee might need support in some area to deliver better customer service, couldn't it?

One of my sons worked at Apple in the retail and the stores for at least a decade. And Apple has used Net Promoter the right way. They do not link people's compensation to it. They do not put the rank ordering of employees that week on their Net Promoter score on the locker room door. That's humiliating. And it makes the employee fixate

on the score when in fact the score for the employee should be a signal. They just want to learn when a customer is not happy, what did they do that they can change? They celebrate the 10s, they think of them as standing ovations, so they get all the energy of an employee enriching someone's life and the team hears these stories.

And they keep the detractor. They close the loop. The same thing I said that any good company does. Well, Apple has some one of the store managers reach out to those customers and try to fix it. And then they talk with employees one-on-one if there's individual feedback that's appropriate. So done right, it's a great system. It's done right very, very infrequently these days. Mm-hmm.

So Brad Anderson is with us from Qualtrics, one of the largest survey data companies. Brad, what do you tell folks about tying survey results to compensation or something beneficial to employees? Yeah, I've always believed, and I certainly learned this, you know, as I was on the front row of the transition from Balmer to Satya at Microsoft and just saw how that culture changed.

The reward system and the things that you measure and reward are really important. And so to Fred's point, if you're going to use something like the NPS system, it has to be to drive a culture of continuous improvement. It's not punitive, but it has to be something to where the culture of the entire organization is curious and you're digging in, what could we have done better? Now, this is a place, I think the example that was given by the car dealership is really an interesting one to spend a second on.

You know, in the past, someone would get a survey and that survey would say, you know, would you like to recommend us? And let's say they say a seven. Well, then now using generative AI, our solution can say, well, what could we have done better?

And the solution could be, could, you know, the, the answer that comes back to the customer could have been like, well, the food was terrible, you know, and so they could, well, give us more information about that. And that's what we would find out that it's the vending machine. And so by using AI, you can now pick apart and understand what are the things that could be used to help coach the, the car salesman or the agent. And what are the things that are structural and our meta issues that need to be solved at, you know, a dealership or at a brand level, right?

So that's where the power of AI and really understanding the context and the specifics about what caused a great experience or what caused a poor experience, and then having that culture of curiosity and continuous improvement. So don't necessarily do away with bonuses based on surveys, but make sure that the survey is accurately reflecting what the employee's performance was here before you do so.

Is that what you're saying? Yeah. Well, that's definitely part of it. You know, I do believe that if you reward great customer service, it can drive the right behaviors. But, you know, to the points that were being made before, it can become embarrassing. It can become dismissive. And so you have to do it right. Mm-hmm.

What about compensating consumers for doing surveys? What do we know about that? Does that help improve response rates or responses and get more stars if you give somebody a discount or a gift card or something along those lines, or does it skew the results? What do you say about that? I don't see a lot of organizations who are giving rewards to customers to fill out surveys. Where I see rewards being given is when they're doing research.

So research could be of prospective customers or, you know, what's the next service or product that we should build? And I do see rewards there because then, you know, generally speaking with those kinds of feedback and those kind of surveys, they're much more time consuming. And so I don't believe that there is a requirement to reward that's going to –

increase their response rates. In fact, what I've actually seen is there are organizations that will look for those kinds of reward and then try to game the system. And so we've had to build AI that is able to understand, is this a human? Is this not a human? Is this a bot? To make sure that organizations are not just paying some kind of a crime organization in another country to fill out a survey. Hmm. Hmm.

So let's go to generative AI because you've mentioned it a couple of times and it is really fascinating because obviously it has this ability to aggregate a lot of data which you're dealing with 1.2 billion survey responses I think you said at the start of our conversation.

So tell me, what are some of the most interesting things that you're working on right now, perhaps the most, we'll pick one, that you think is really going to help improve this sort of customer service interaction with businesses? Far and away, the thing that I am seeing is the most impactful, you know, of the dozen plus AI capabilities that we've built into the product over the last year is what we call conversational feedback.

And this transforms a survey from a flat, impersonal experience to an experience that is personalized and adapts while the survey is being given. And what that allows an organization to do is, first of all, get much better data. And then second, really have a personalized experience for the individual filling out the survey. So let me give you just two data points that are two examples you'll find interesting here.

So what we have a bunch of measurements that we do as we get these open-ended comments coming back to us. And so, for example, with these conversational feedback surveys, what we're able to do is take a look at the first answer, how many syllables in it. We have some lexicon measurements that we use to understand the quality of the response. And then we can compare that with the second response that comes in when we prompt for more information.

And so what we see, for example, is seven times the number of words or syllables that come in on the second response. From a lexicon perspective, the quality of the second response is much higher.

And then third, the most important things is organizations who are seeing this are actually seeing double the number of key things that they can go structurally fix in order to drive improvement in the customer experience. And so those are very, very specific measurements. And it's interesting, with a lot of our customers who have used these capabilities, we've asked them to allow us to put a question at the end of the survey so we can understand how the consumer feeling out the survey felt.

And resoundingly, when it gets personalized and it has this adaptive conversation, the feedback coming from the individuals filling out the survey, the satisfaction of the survey is high. So I think that that's a good sign.

Second example I'll give you, Deborah, that I thought was fascinating is we had a healthcare organization a couple of months ago who was using these adaptive conversational feedback capabilities. And there was a survey that was being given to an older individual who had had a poor experience. Well, our AI detected the poor experience, showed empathy in the questions that were being asked and the follow-up questions.

So much of the impact of this was that the individual actually commented, I appreciate you showing me empathy. I love working with an organization like this. And so these conversational AI capabilities can really improve the relationship, which I think is what every organization is trying to do. But basically, it is really the AI-generated conversation. The computer is having this conversation. A human being is not. Right.

That is right. And so, you know, let's talk about, you know, getting high quality data. With surveys, you know, there's a level of data that you can get. And if you compare that, for example, with a five-minute personal interview that would be done by a human, of course, you're going to get a lot more data in the interview because the human will adjust to ask more questions to get down to, okay, what makes this actionable? Right.

Well, we automate all that through conversational life, through generative AI. And so, yes, it is the generative AI having the conversation, but we can, for example, train the AI to have the tone of the company, to understand what rewards can be given. And you think about where this will head. You can see where, you know, as an organization is conducting a survey and there's a poor experience, the system will automatically be able to close the loop and

and say, oh, we apologize about this. Hey, here are some points. Here is a gift certificate to come back on us. And you can have all that be controlled based upon the feedback that is being given. And to the point that Fred made earlier, the best measurement

of are you doing the right things? Are you closing the loop? And so you can see where this conversational AI is going to take us to, to where we can close the loop in the moment when that feedback is being given. But if personal customer experience is so important and a customer feeling like they are important,

It would seem that we're not at a point yet where perhaps with your response from the patient who went to the hospital and felt that the AI conversation was empathetic, perhaps I'm not right here, but it would seem that we're not at a point yet where we could use AI to be authentic enough to create a personal experience for a customer. But you think we're almost there or we are there? What would you say, Brad? Yeah.

Yeah, we're definitely on the path there. There's a lot of conversation in the industry right now about agentic AI, and that really is the next frontier.

And what agentic AI is, is autonomous agents that have been trained to understand the customer experience, that have been trained to understand the organization, their brand, their tone. And so as I think out the next two to three years as these agentic solutions come out, you'll see a lot more of exactly what you were saying of where you will have agents who will be able to take action on behalf of customers.

And so as I fast forward two or three years from now, you'll start seeing things rather than an individual coming to a website or in an app and then navigating that to accomplish something. They'll be able to come up to that app or website and just describe what it is that they want or what they're trying to accomplish. And then you'll have a set of autonomous agentic AI that's operating in the background, these agents, to take that action.

And our belief is you want to infuse all those agents with an understanding of the human experience, which is where our investments have been for the last five years. You know, yes, we continue to improve surveys, but all the investments, the majority of the investments has been in how do we build the AI to understand the human experience to then allow organizations in these systems like an NPS system to

to be able to understand what the lived experience of the customer and the employee was and then take action. Okay, 30 seconds left. I know, Fred, there's a lot there, but we're going to give you the last word here. We have come a long way from developing the question. Would you recommend us to a friend, it seems? You know, Brad started out saying that Qualtrics analyzed 1.2 billion responses at a 1% response rate. I don't know, you're off.

And I don't know what market share Qualtrics has of the market. It's very substantial. But it means there's hundreds of billions, maybe a trillion responses or requests for survey. There's a problem out there. The tragedy of the commons is upon us. I think we need to simplify. Yes, AI can help, but my simplification is even simpler.

The real goal is to treat a customer so well that they come back for more and especially they refer their friends. Don't ask for a survey. Thank you, Fred. Ask them if they refer. Thanks, Fred. I'm Debra Becker. This is On Point.