Question Your Customers
18:23 Share

Welcome to Rework, a podcast by 37signals about the better way to work and run your business.

I'm your host, Kimberly Rhodes. And if you are a Basecamp customer, you may have seen a link to a survey at the top of your account. We recently reached out to a few of our customers to get some feedback on the product. And we don't do that often. So I thought it'd be a good time for us to talk about it, the reasoning behind it, and what exactly we learned to do that. I'm joined by Jason Fried and David Heinemeier-Hansen, co-founders of 37signals. Jason, I'm going to start with you.

have we reached out to customers often to ask them questions i saw it

hop up on Basecamp and I was like, oh, this is new. Yeah. We don't do it very often. Over the past couple of months, I've done it twice. I typically do them when I'm curious about something and there's something, one thing I'm curious about. And what I found is that like, that is mentally what I'm curious about. Hey, I'm curious about this. So let me ask the one thing. And we tend to get really high response rates because it's one question. And the question is always a question with a big text area. And it's like,

fill it in. It's not multiple choice. It's like, whatever you want to say, say, and if you don't want to say it, you don't have to say it. And we get somewhere, I think the last one we did, we got about 1600 responses. Oh, wow. And it was so full of incredibly useful, rich stuff by just asking one question that's open-ended that there's no way I could have mined for that information any other way. In fact, I would have gotten less useful information had I asked more questions and been more specific about the kinds of answers I wanted.

So just as if you're going to ask a friend a question, you don't give them multiple choice options. You just ask a question and then people share. And so typically what I'll do is I might give an example of how you might respond and

if I'm looking for a kind of a directional type of answer, like, for example, the last question we asked was about like, how'd you land on Basecamp? Like, how'd you get here? And I said, like, for example, maybe you start with email, and then you went to Slack, and then you went to something else, and then you went to something else, and you went to Asana, and then you got Basecamp. That

That could be an example path for how you landed here. And so most people will take that format, essentially that template and kind of fill it in for themselves. Other people will tell like a long story. Other people will just say something else. So you can kind of point someone into a relative direction of like what you kind of want, but you still want to let them do it themselves. And I have no problem answering questions like that. What I do have a problem with, I bought a car recently and I got a survey from the brand, I think it was, or the dealership, I think it's the brand. And

And I'm like, okay, I'll dive in. So I hit the thing and you get like a few questions like, okay, I can answer those. And it's like next. And it's like another few questions. And you're like, you kind of go through maybe a page or two and you're like, oh, I get this. This can be like 45 questions and they're all in a zero to 10 scale. And I don't even know what it means to be a seven or an eight.

Like, how was your dealership experience? Seven, eight? I don't even know. And the other thing is, of course, when you buy a car, they always tell you, hey, they nudge you, hey, hey, I need tens. Give me some tens. We need tens. It's like, the whole thing is such a scam. But most of all, you're asking me so many questions that in the end, I'm actually not going to finish the survey. I'm not going to submit these answers. You're going to get nothing from me.

You would have been better off asking me one thing or three things or whatever, or telling me this is the only page of questions you're going to get. And okay, there's six of them. I can do that.

But this never-ending survey, and this comes from marketing departments and product development departments who feel like they just need so much. And they're maybe only going to get one chance, so they ask everything. You have as many chances as you want if you ask decent questions occasionally. Just take your time, get good answers from people, and get great response rates. And I find it to be a very, very wonderful way to do this.

this. So was there an impetus for the survey, like a reason why you wanted to know this specific question at this specific time? Or is it just like general curiosity? Yeah, David, actually, and I were sitting down for lunch with someone just kind of riffing on some stuff. And I just got curious about we're talking about marketing and whatnot, like, how do people get to Basecamp? And I just had this thought about, well, let's I don't know, let's find out. Like, I'm like, I think I know.

But let's find out. Let's find out. And I'm sort of curious to see what products people have gone through. Because it turns out, it's very interesting. Some people discover Basecamp initially and they just stick with it. Other people start with Basecamp, they leave because they think they need more. It's amazing. You watch their path through these other prominent tools we've all heard of, and they come back to Basecamp. It turns out 40 some odd percent, I think it was 41%,

of people had Basecamp in their thing twice. So they started with something else. They tried Basecamp, they left and they came back. And the things they went through were Monday, Asana, ClickUp, Trello, Notion, Slack, some Atlassian products, like all the usual suspects. And you could see what they did. They left...

I need more. You can imagine like we need more power. We need more this thing that we think we need. Yeah, go ahead. Go out there and check it out. And they check it out and they go, yeah, we either didn't need it or it's way too complicated or no one's using the damn thing at all. Or like, wow, this is not what I thought it was going to be. And they come back and they circle back. It was just very interesting to see that. The one previous to that I asked, and we asked this every few years, which is what's changed for the better since you switched to Basecamp? And it's not about Basecamp.

It's about however you want to answer the question. But it turns out it's mostly about what people end up answering is like workflows are better. People are communicating more clearly. People are writing things up in ways that are better. There are fewer meetings, all these things. Basecamp doesn't have a fewer meetings feature, but using Basecamp leads to people having fewer meetings. And so they share that kind of information. It's very, very interesting. So I like the open-ended questions without a distinct answer that I'm fishing for.

I just kind of want to know, which is, I think, a much better way to do it. What's so amazing about these open-ended questions is that the customers who answer them will give you the greatest gift of all. They will give you the language by which they think about these problems. That is such a goldmine to everything from how to name a feature to how to market that feature because they're presenting the problems like they think about them.

I remember when we did one of these open-ended inquiries a while back, and these sayings would go over and over again, like things falling through the cracks. This was then a term we would use in some of the positional material we have on, well, Basecamp is a place where things don't fall through the cracks, or when things fall through the cracks, you need Basecamp. The richness of that unprocessed language, that's where all the gold isd.

And it's just so tragic to me that marketers in particular are willing to trade that because it's manual. You have to dig through it. You can't just tabulate it. You can't just get this junk specificity. You can't just get like, oh, well, the customer satisfaction was 7.25%.

three this quarter. That is a 11.2% increase over last quarter. What the fuck does that even mean? Like, how does that connect to any human in any way that anyone would respond to? But this is the problem of incentives sometimes, right? You're incentivized for like a quarter after quarter, you have to show some growth on some KPIs, some metrics, right?

It's difficult to show that growth and, oh, I have a deeper understanding about how our customers think about our problems or how our products are received. But that's where all the nutrients are. Like this ultra-processed form junk stats that comes out of these multiple choice, these zero to 10, these one to five bars. There's just so little in them. There's so little nourishment in them. There's so little you can use them for other than just putting them into some

quarterly report that no one's going to read anyway, and it's not going to have any bearing on whether you can attract more customers in the future or not. It's all just circumstantial, hyper-extracted bullshit. If you instead dive down into these human narratives, I'll

A lot of them, actually, if you go, I think it's on Basecamp.com, we have these 1,000 testimonials that came from one of these questions. I think, Jason, wasn't it the, what things got better? Yeah. You can read through that and you can see entire story arcs. You can imagine the entire customer. You can get closer to the customer in that sense. And it's just, it's so valuable when you're willing to actually engage with the material, not just treat it as stats.

Yeah, by the way, that page is at Basecamp.com slash customers, which has close to a thousand quotes, simply answering the open-ended question. And to David's point, which I thought was great about this point about the language and the nourishment. Like if you look at that page, Basecamp.com slash customers,

And you scan, you know, I'm not expecting anyone to read the whole thing. And the whole point of it being so long is that it's just supposed to be overwhelming. Actually, that's the idea. But we've highlighted some things. Okay, so you look at that, that data set, right? And then I could have had another data set, which is like how many fewer meetings you have.

per week. And it could have said three or four or five, whatever, right? This could have been a choice, right? There's no language. There's no energy or emotion. So I could have shown the data set that we have with Basecamp.com slash customers and other data set, which is numbers, which is answers, which is pure like statistical data. There's no richness in that statistic. I can make a chart. I can make a graph. I can put it in Excel.

I can ask questions about it roughly in trends and percentages. And do more or fewer people are avoiding more meetings than the last time we asked this? I can get at all that stuff. I would never take that, though. I'd always take the open-ended, use your own words and tell me what you think. Is there more to read? Yes. Can I analyze it? No, not really. I can read it and absorb it and feel it.

Yeah, it can be analyzed. I'm sure if you want to look for keywords, how many times is this word mentioned and all the things, whatever, you can do all that too. But it's actually readable. It's ingestible. It's something you can feel. And that is where the value is. I'll extend this metaphor one step further. What this is doing is it's training your gut instincts.

When you read through all these customer accounts about how they found your product, why the product is valuable, where it's going, you're teaching that supercomputer, as we've talked about, your gut instincts to be in touch with what customers actually want. Can you directly map that to how this feature should go or some growth targets? No, you cannot.

But you will be wiser and you will be more in tune with the customer center you have when you have to make those decisions. And you won't be able to trace it back necessarily to one of these thousand anecdotes. But I guarantee you, you will be wiser and more informed and therefore better able to make snap decisions.

gut instinct decisions when you are making new features. What matters? What doesn't matter? So much of this is in the subtlety. So much of this is not something you can tabulate or even A-B test. Someone actually just asked Jason and I today, do you do a lot of A-B testing? And I thought about the fact that I don't think we've done an A-B test inside the product for probably 10 plus years.

And the last time I remember I was doing an A-B test inside the product, I ended up with more questions than answers. So what does it mean that 41% uses it in this way? I don't know any of your motivations. I don't know what you're trying to accomplish. I don't know where you're trying to go. I don't know what to do with this information.

versus when you get the stories about what people are trying to do. You get the motivation behind what they're trying to accomplish. You can go like, oh, okay, I see what you're trying to do here. We could actually fix that. We could dive a little deeper and we could make some tweaks and I can satisfy that. I can't find that out of 41.97%.

I think the other thing that's interesting about these open-ended questions or just having one single open-ended question is the ability to follow up. So Brian, our head of product, was looking through some of these surveys results and sent me a couple. They were like, hey, Kimberly, is customers asking about

videos to understand Basecamp. Like, well, we can help you with that. Or someone had a question about using Basecamp for event management. Like, we can help you with that. But it wasn't a question of like one through five or tell me what your problem is. It was open-ended, but it gave us an opportunity to follow up with people who we didn't even know that they had this question. It just gave them a place to put it.

Something I did on the what's changed for the better question. There was technically, I think, two questions on that, which is like, what's changed for the better? And then the other one was like, anything else you want to tell us? Which is not really a question. I mean, it is. It's a question technically, but it's really like a catch-all. Like just leave a space for people to say something and they will. Thousand people said something else.

Some of them was just wonderful to read. Basecamp changed our business. I can't believe it. Thank you so much. Other things are like, we've been using Basecamp for a number of years, like 12 years, 13 years, amazing stuff like that. Other people just share stuff about their company. You don't even know what people are going to share, which is awesome. It's more space for people just to say something.

because a lot of people have something on their mind. Like you can ask 30 questions in a survey and get less information than asking two questions. And so I feel like we get way more information out of asking two questions or one question than we would if we asked 30 questions.

Because the information we get is what's truly on their mind. And that's actually the more valuable stuff, not the stuff that we want to squeeze out of them. I keep going back to this car example, because it's like, I don't actually have an opinion of how the staff treated me. It's not how I think about that experience. So now I got to make something up and I don't, I don't know. And like, they were nice people, I guess, five. I don't, I don't know. Like,

And then you're like just giving them information that's not even accurate in a sense. I just feel like the best thing you can do is ask fewer questions and have them more open-ended and really try to get to one if you can.

What I love about this example with the car survey, and I've seen those too, both from cars and other experiences, is that you always know when you allow something to be quantified, it's going to be gamed. And this is exactly what happens with those car surveys. The salesperson knows that there is a metric that they have to hit, and it becomes a very circumspect

I have to extract a five out of five out of you. Otherwise, like my job is in danger. One of the great examples of this is

On Uber, if a driver has, I believe it's less than 4.7 as their average score, it basically means they're total shit. The scale doesn't go from one to five. It goes from 4.7 to five. That's where all of it is, right? And you just go like, there's such a poverty in that realm. And you can't really gamify an anecdote, right?

That's the other thing. It just has built in protection against that. What's the salesperson going to say? Say something nice about me. Okay. But you know what? It's a lot harder to feign that if you didn't really mean it or that you could be subtle about it in a way where the person reading it can understand what was going on. So I think the open-ended question is simply more open.

a more resilient form of surveying. It is more resilient to gamification, to junk statification, to aggregate averages that compresses all the nutrients into this ultra-processed bullshit. There's just... The stuff is here. Now...

Whether you're able to take that and turn it into either product improvements or otherwise, okay, that's up to you. But at least the opportunity is there. The soil is rich. Another way to think about this is the reason why these things have numbers is so they can be processed correctly.

and they can be turned into charts and put in a presentation. They can be presented to the board. If I was the board or whatever, I'd rather take 10 minutes, give me 100 responses, and let's read them together. That is way more valuable than seeing some chart. But the chart is easy to disseminate.

It doesn't ask anything of anybody else. And so that's what you end up with. You end up with, and it also, of course, it can be measured over time. You could say like customer satisfaction is up 18% this quarter. What does that even mean? I'm sure many of the questions were on this car survey. Like I wasn't even thinking about those things. That wasn't how my experience was reflected. So yet somehow like I would have given it a number. It actually is meaningless. It doesn't mean anything. Um,

But now it's in the data. And so the data represents a lot of nothing, actually. So let me ask you this. We know what type of questions you like to ask and how many questions you like to ask. How often do you like to ask your customers for their feedback? Is this something you do regularly with all the products? Kind of tell me when this comes up for you guys. Well...

First off, we hear from 500 plus people a day via customer service. So there's always stuff coming in and going out. These kind of pointed questions are once or twice a year, usually is how I would do it. Maybe three times, maybe often two or three times.

I can't remember the last time I asked something three times in a year, probably, I don't know, once or twice. So I'd say one or two questions. We've done them only for Basecamp, have not done them for Hay. We actually should. Thank you for bringing this up because I wasn't even thinking about it. But I'm going to do one for Hay as well. What's cool about Hay is that we could actually just like just hit reply or something and like just respond that way. Maybe, maybe not. I'm not sure. That might be kind of a messy thing. But yeah. Yeah.

That's it though. Okay. Well, thank you for that. We're going to wrap it up. Rework is a production of 37 Signals. You can find show notes and transcripts on our website, 37signals.com slash podcast. Full video episodes are on YouTube and X. And if you have a question for Jason or David about a better way to work and run your business, leave us a voicemail at 708-628-7850. You can also text that number or leave us an email at rework at 37signals.com.