Episode 291: Shippable User Research with Alice Lee
35:44 Share

Hello, ladies and gentlemen, and welcome to the UI Breakfast podcast. I'm your host, Jane Portman. And today, our awesome guest is Alice Lee, Strategy Director at Working Co. And we're going to talk about shippable user research.

This episode is brought to you by Wix Studio, the new web platform for agencies and enterprises. The magic of Wix Studio is its advanced design capabilities, which makes website creation efficient and intuitive. Here are a few things you can do. Work in sync with your team on one canvas. Reuse templates, widgets, and sections across sites.

Hey Alice!

Hi, thanks so much for having me. We're super excited to learn from you today. But before we dive in, tell us a bit more about yourself and what your background story is. Sure. So my name is Alice. I live in Brooklyn. I'm a strategy director at Working Co., all of which is stuff you already mentioned. I have been a strategist for my entire career. I started out actually working in advertising as a brand strategist.

And I bopped around a little bit doing that from BBDO to other agencies in the Omnicom network, doing some tours of duty around the different creative shops in New York. And then ultimately I segued from brand strategy into digital. This was a time when, you know, if you weren't in digital, what were you even doing? And I wanted to be, you know, at that cutting edge of what was happening and, you know,

So I was a digital strategy generalist at a certain point, digital marketing strategy, digital creative strategy, and digital product strategy. And that's when I discovered that this is what I really love and what I'm passionate about. And so I've been specializing in digital product strategy now for the last stretch of my career. And that's how I ended up at Work & Co because I wanted to be with the best. And I've been with this agency now for the past five years.

So Working Co. is a giant approaching 400 people shop agency that's serving leading world brands. Can you talk a bit more about what they do and what you do as a strategy director?

Yeah, definitely. So Work & Co. is a design and technology company. We focus only on launching digital user experiences that help transform our clients' brands, push the industry forward and explore everything that technology can do for people. So what does that actually mean? If you boil down from all the buzzwords, we make technology.

really great apps, websites, and digital experiences. So nowadays that may also include AI experiences, spatial computing, like VR, AR, but things that will ultimately be used by people in the digital space.

And yeah, we've been around since 2013 with this one really focused mission. Do digital products and do them as best as we can and work with the best people in the industry. And I think we're proud to say that we've really stayed true to that 10 plus years on. And we've recently joined forces with Accenture Song to help, you know, keep propelling our work forward. So we're really excited with where we are and where we're going.

So the reason why you have shippable as a label to your research method is that your company really takes pride into shipping things. Tell us more and how that does sometimes compromise the culture of, you know, researching and investing time into that. Yeah, definitely. This is something that we say is our most proud metric or the metric that we are the proudest of is how many things we've shipped.

You know, we have such talented ideators and thinkers and makers here, but everything we do is with the intent of actually putting it out into the world is something that people will use. We ship. And that's kind of our driving mindset.

We don't do things with the intent for it to, you know, be a deck or like be a really cool presentation. We want to make products that see the light of day. And so I think it can feel and sometimes seems kind of hypothetically like that could be intention with the idea of doing research. I think especially these days with how quickly research

Emerging technologies like AI are developing, for example, as if people aren't already aware of that. I say this random example of AI, but it can become a race, you know, especially as we think about innovation and emerging technology. There is this imperative, sometimes like all consuming desire to be first.

to get to market fast and to ship as quickly as you can. And oftentimes it can seem like research is maybe just inefficiency that can be cut along the way or a step that maybe we can skip or just something that takes too long, costs too much money. And why not just put something out there and see how it goes and iterate from there, you know, build, measure, learn. And, you know, that is a useful framework for certain contexts and for certain needs and

But the stakes are higher these days and there's kind of more need than ever to be thoughtful and to be intentional about what goes out into the world and impacts people who use them. And so that's why I think it's so important to maintain that spirit of shipability. You know, we're doing things that matter, that we want people to use, that we want to see the light of day, but we need to do it in a thought through way.

And that's where the research comes in. And I don't think it has to be either or. I really think you can, you know, have your cake and eat it too. You can ship stuff and you can do research on it as well.

So when you have a client project and you don't have two months to throw in and dedicate to user research, I imagine that that's plain English description. What is the alternative, your method, shippable user research? What's the definition? What does it entail? Yeah, that's a great question. I think there are three main things that I would say characterize the ethos of shippable research. One is that design.

Design is research and research is design. You know, the design process would only be so much if it didn't involve design thinking and formal practices of learning. And what does that actually mean? I think that comes to the second thing too about collaboration and about the researcher not being the person who owns the act of learning. I want everybody to be involved in doing the research, doing the learning.

identifying what it is that we need to learn and what knowledge gaps we need to fill. And it becomes a team responsibility of how we're going to fill those gaps. And so to the question you asked before about, so what does that really look like in practice if we don't have two months to spare? It means that at the beginning,

We might ask ourselves this question I mentioned, what are the learning objectives? What do we need to know in order to get this product out into the world? And we'll do a quick and dirty round of user interviews. My preferred method, my favorite personal method of getting insights quickly.

We'll do user interviews. We generate, you know, insights, hypotheses, and then we start exploring those hypotheses through the act of designing. You know, design can be a way of learning and thinking, you know, are these hypotheses real? Are these solutions going to work?

Are there maybe new questions that we have by trying to design against these hypotheses? And then you iterate from there. We might say, okay, so we were able to maybe rip out five interviews in a day and then start designing. We have some more questions. Can we maybe do a few more? Do we want to administer a survey? Do we want to...

Just keep iterating until we have a concept and then test a concept with people versus just doing an interview. And so it's about, again, being very purposeful about what questions you're trying to answer and having a really firm grasp on learning objectives, recognizing that new questions will arise, you know, as time goes on and as you work your way through the process.

And just being able to break down a big kind of research initiative into rounds of progressive learning that are building on what came before and helping you iterate on the design process that's unfolding in parallel and making it everybody's responsibility, as I said.

Everyone can have an insight. And oftentimes the designers will come to insights that I wouldn't have thought about as someone who's not a design practitioner myself. The technologists might have insights that speak to a blind spot for me as someone who's not specialized in technology. And so enrolling the whole team along the way and building that sense of shared ownership just makes for...

Kind of more flexible and resilient insights, I find, you know, ones that will survive long term and not just be something that a UX researcher strategist came up with eight months ago and wrote in a brief and now somebody has to read them and buy into them and act on them.

You literally started answering my next question, which was you repeated a few times that, you know, design is research and technologists can also bring in some insights, but you're not a designer, not a technologist. You're the one responsible for strategy. So

If you go beyond the term collaboration as a vague thing, how do you orchestrate all of this process between the parties and managing insights and doing all this in a rapid fashion? Yeah, the most basic and straightforward answer is that you have to talk to people. And I think my job can sometimes be seen as a professional talker.

where you just do have to build the bridges early on. You know, I think in a kind of traditional waterfall process where there might be like a research and strategy phase, and then it goes into a design phase, and then the designers hand off to the technologist. In that kind of more linear process, I might never meet the people who will develop and build

digital product experience that I came up with the strategy for. And that's different than, you know, how we operate our projects more agile style here. But I do think it also does create an onus and a responsibility I take on gladly to be the one who builds the bridge to the technology team or to the design team at the very beginning. And we talk about, okay,

okay, so what is it that we need to know? And I'm thinking from more of like a business and a product strategy lens, like we need to know

how to differentiate, you know, what are other experiences that people use? The designers have a whole different set of questions that might never occur to me. And I'm not even going to try and make them up now. And the same with the technology team. So it's about just continuously talking to them and finding out what they need to know, what ideas they're coming up with, how they're reacting to what we're hearing, and just continuing to keep that channel of communication open. I feel really bad for omitting product managers from my previous question. I

Do you have product managers at Work & Co? How does this work? Yes, we do. This is a question we get a lot. I was curious about myself when I came to Work & Co because often product managers are the owners of the strategy. And so the way that we interplay as two different disciplines here at Work & Co is product managers are the kind of owners of delivery. They help

kind of facilitate the effective delivery of every single phase of the product and ultimately the delivery of the product itself and launch. And strategists are considered more specialists. We come in when there are kind of specific needs. Usually we'll say projects with high ambiguity.

are ones that require strategists to come in where maybe it's not clear, you know, what the, where the opportunity is for this product, like how it should be positioned, what its core value proposition should be, or maybe we don't know something about the target audiences or we need to define the audiences and just kind of more

Thinking and researching and definition has to be put into play. So that's when strategists kind of get called in. But there are many projects at Work & Co that don't have strategists on them at all. And our product managers are such strategic expert practitioners. And also, I would say that all of our teams, like our designers here are all highly strategic thinkers and our technologists are strategic thinkers. So lots of projects are just strategic projects.

by nature without the strategist being around, which is great. One of the things as a practitioner, as a product person myself, I find the hardest is

How to ask the right questions about what we don't know. I don't mean interviewing techniques. I mean exactly what we need to learn at this stage to make a good choice in the product or in marketing or elsewhere. Could you share some of your personal best findings on this? How do you find out what exactly we shall ask and who is the audience we shall ask this and things like that?

It's definitely, I feel you on that. It's an ongoing challenge. And with every new project kickoff, I find myself with the same, sometimes crippling fear of, I don't know what I don't know. And it's like doing it for the first time every time, which I think is good, by the way, having this like beginner's mindset. But what I found is it's a combination of things. I kind of have at this point, just based on my experience, a set of

areas or questions that I know must be answered for any product to be successful. Who is it for? Why do they need it? To do what in their lives? What is the value proposition? Who are we up against? And what does the business and brand have to offer by way of, you know, assets, advantages, perceptions that will help make this a success? Those are kind of

the 360 view of things that you need to know. But then obviously there are going to be much more specific questions that sit under that. Are there subgroups in this audience? You know, are there specific use cases that they will or won't use this for? Like, is this a product they will use alongside other products or is it a winner take all market? Those are things that get really into the weeds and you could kind of

make yourself go crazy, honestly, trying to list out all of those things upfront. So I try and anchor myself on those like high level questions that I'm like, this is what it takes to be successful as a product. That's probably what I was saying just now is like the highest level North star question that I just like try and

come back to as like a thought exercise of, you know, if I'm trying to come up with all of the right questions to ask or writing a research plan, it's like, what does it take for this product to get to market and be successful and start helping people? So that's one thing. And then the other thing is just being open to the fact that you're not gonna be able to come up with everything in advance.

and having flexibility, both in your own mindset and approach to this product process, like being open to the fact that you're probably going to learn things down the line that might fly in the face of your early hypotheses. You might discover new questions that maybe you thought were valuable before and kind of just being open personally to that. And ideally also having

a kind of flexible enough process or checkpoints along the way where you'll be able to act on that, where you're not like totally locked in to an approach or a strategy or a concept even that was defined ahead of time before you learn new things. So being able to iterate, you know, and having those checkpoints and the humility to say, we learned something new or we didn't realize this was a thing or even we were wrong about this and now we're going to adjust.

This sounds natural in a startup environment, but when you have like 20, 30, 50 very expensive people on the project, how do you keep this humble approach and be like, oh, we just spent two months in vain. Now we're going to iterate and pivot. How do you do this?

Yeah, your kind of, I think, implicit point, which is that it does kind of suck, is right. You know, it's not easy, certainly. And I would be remiss to kind of position us all as these like humble monk-like people who are

consistently like able to admit when we were wrong or able to pivot. But so yeah, I will say it is, it is certainly challenging and we do try and do two things. We try and craft the process in such a way, like our project timelines and our process in such a way that we built in these checkpoints along the way so that ideally you don't get too far into the project and

with, you know, operating on one hypothesis and then you find out like months and months later that you were wrong. We try and, you know, build in multiple, like I said, like rounds of research. We have ingoing research. We have validation testing, oftentimes multiple rounds of validation testing for design outputs and concepts and prototypes so that we're never going too long without some kind of like validation touchpoint.

So that's one thing. And the other thing is, I mean, this comes back to what you were asking before of just trying to take a beat at the beginning and be thoughtful about what the right questions are to ask.

I mean, I thought it was really cool that you even brought that up. It's a really valuable thought exercise to do is just to sit for a second as a practitioner and as a member of a team at the beginning of a project and ask yourself, you know, what are the questions here? And am I asking the right questions?

And even like, what are the most important questions, the kind of upstream questions that if we are able to answer those will help maybe de-risk things down the line. And I bring that up to say, like, sometimes it doesn't happen. I think it's very easy to be

on a really fast moving project, quick turnaround, or it might just seem very straightforward. We've done this kind of thing a million times. We've redesigned websites like this all the time. And you might not, you know, take a second at the beginning to examine where you might have knowledge gaps and what things might come up later that could bite you in the ass, quite frankly.

And so I think it's about doing that too. So it was a long answer, but to kind of put it back in order, take that moment at the beginning of any project, especially to your point when that might be expensive and higher stakes and for a client to be really thoughtful about what you need to know to A, make this product a success, B, de-risk it and de-risk the product development process. And then the second thing is

try and create those checkpoints along the way to just kind of check back in with the market, with users to validate your hypotheses before too much time goes by. Speaking of strategy, I'd really love to hear if your role as a strategy director implies that you take strategy seriously

decisions about the flow of the project or more like the flow of the project is determined somewhere else and you go into the product strategy? What does strategy mean in your case? Yeah, really, really good question. So it is more the latter, but a little bit of the former. So I

the scope of work and how the project flows and how the process flows is, you know, decided ahead of time between a group of people, you know, our clients, obviously our business development and growth team,

Our product managers who really, like I said, like are tasked with the delivery of the work and the product at the end. And strategists will obviously kind of advise on that and have a point of view, especially with regard to like how the strategy work and the research is done. But then really like our main responsibility is just like being in the work and doing the product strategy. That's also one of the things Working Co. is really big on and that I think is really great is the emphasis on

letting practitioners be practitioners and focus on their craft and ideally spend the majority of their time and their day and their working week just like doing the work and doing the products and the product thinking versus, you know, having to figure out like project management, for example, like we leave that to our great product managers or having to do, you know, bureaucratic things or paperwork or things that are kind of outside of our core craft.

It's the perfect time to segue into your awesome case study about GoFundMe. And that's one of those stories where the team just arrived with a perfectly shaped question that you took quite a while to answer. So tell us more about this project.

Yeah. So GoFundMe, I am comfortable saying is one of my favorite clients at Work & Co. And I'm sure most people are familiar with them, but just to say, you know, they are the world's biggest online crowdfunding platform where anyone can create a campaign and raise money. And we had the privilege of working with them for a couple of years across multiple projects, across a lot of different things. And

It was a really great example of shippable research in action. So I can give a few examples and like key learnings from our work with them. One is

They came to us with a kind of different brief, one of the more unusual briefs than I've ever had because the question was so big and it wasn't tied necessarily to a specific or prescriptive product ask. It wasn't like redesign the homepage or like redesign the fundraiser page, which they call the campaign page. It was what would make more people ask for help?

What stops people from asking for help online? And that's a really big question. Obviously, the ultimate goal was to

to get more people to use GoFundMe to raise money online and ask for help from strangers. But it could be attacked in many different ways. There are lots of different product solutions that you could come up with, whether it is redesigning the homepage or redesigning the campaign page or coming up with all kinds of other pages or product surfaces. But they wanted us to really think deeply about

that question first, which was really cool and also quite terrifying. We started as Most Working Co-Projects to all together, like every discipline starts on day one. It's not, you know, first there's research and then there's design and then there's technology. Like the designers and I all came together and

at the same time and they needed to start concepting and I needed to start giving them something to work off of. And it was really scary. It was just such an ambiguous space. And so what I thought was really cool that we did that was very kind of emblematic of Shippable Research was we broke it down. We made research more about learning and the mindset than just like a checkbox and

of something that we had to do up front. It wasn't like we're going to do this big round of research that answers this question once and for all, and then we'll figure out what the right design is, and the team will execute on it. We broke it out into what we call a progressive research program, where

where first there were exploratory user interviews to get into that really kind of meaty, qualitative human insight around people's mindsets around that, their behaviors and how those two things relate. We were able to come up with a bunch of hypotheses that informed a whole lot of different concepts and ideas.

And we're like, you know, what if we did redesign the homepage? What if we created a whole new page? What if we did this? What if we did that? We had loads and loads of concepts. And then we would just do another round of learning. We might have done two rounds of interviews, eventually did a survey to narrow down on concepts and start prioritizing.

We took our prioritized concepts, turned them into prototypes, and then concept tested those with users again. So going back to what I was saying of these kind of repeated checkpoints and these rounds of validation. Along the way, we're coming up with new and more and more specific questions that we wouldn't have known up front. Like, for example, as we started getting further down this funnel of learning,

and designing and iterating and narrowing in on concepts, we started having new questions about, you know, what is the role of geography and like locality? Do you feel more likely to give money to or ask for help from people who live near you? This is something we just couldn't have known up front.

And so that was one kind of aspect of the approach that I would call out of like shippable research in action was just, again, like breaking it down into iterative multiple activities that got progressively sharper and more specific and learning objectives. And the other thing I would say too, was the way that the designers were researching right alongside me. You know, I made sure to spend time with them before,

Each round of research was planned and executed to make sure we're kind of all on the same page about what we're trying to learn, that I am understanding what they're trying to learn. And they did research too. You know, they did desk research. We would all read books.

because it was such a kind of broad and interesting question. And we got really creative about reading books about psychology and memoirs and self-help books, and we would give each other book reports. And so the designers were also like conducting research in that way.

We had a kind of shared repository for our ongoing landscape analysis where I was thinking about competitors from a more strategic angle of kind of players and how they're positioning and what's making them successful. And the design team was contributing to it from their own lens of kind of visual strategies, where might we find inspiration, how our kind of look and feel contributing to different experiences and ways that we could learn from.

So that was the second big thing of it was collaborative. You know, I was not the owner of the research. And I think that's what makes research shippable at the end of the day is that everyone feels bought in and enrolled and can really kind of

rally behind what you're learning. That's how they see the value. That's how you make sure that research gets used by people and also sees the light of day. And it just makes the work better because everyone feels such ownership over it. So I would call that out.

And just kind of say that there were lots of other examples of work that we did with GoFundMe aside this kind of big research question. You know, we also worked on a Giving Tuesday activation that focused on getting people to raise money for charities or kind of on behalf of charities, not just for themselves. And we were able to kind of take these same principles and put them into action.

for a really different kind of project and a really different kind of ask that was a lot faster and more time bound because it needed to be in market by a certain time. And so the other thing that I would call, the reason I bring that example up with regard to shippable research and how that was a good case study of it was

the charity action page, which is what that thing was called, project, was a twistier and turnier one and a faster one than the bigger kind of how do we drive campaign starts? You know, why aren't more people asking for help one? And that was where the kind of third thing I want to call out or the third learning about shipable research really came into play, which is your process is probably going to break down. So you just really have to hold true to the objectives instead, where, you know, we had lots of

partners involved for the charity action page and you know stakeholders would change we had new stakeholders there are lots of voices in the room we had to kind of change plans a lot as we were trying to get into market really quickly with this and we just had to be ready to shift and iterate really quickly as the needs changed and the team changed and the market changed and our process was

we had to get really kind of creative with. But, you know, we tried not to let that phase us and just like held true to the objectives and the principles, even if we didn't get a chance to like do a full prototype test with our users. Instead, we would test one line of copy at a time, like really rapid, unmoderated tests. And so we, again, just got creative. So I think those would be the kind of big moments and examples that I would call out from the co-funding case.

You've had your...

fingers in dozens of client projects, I'm assuming in five years. What's your sentiment? What do you find more often? Is there a place for some revolutionary, oh my God, sparkle lighting moment that just changes everything and you're like green light all in? Or is it more like quiet evolutionary findings that just help improve what they have and, you know, go the traditional route? What's the statistics on this?

Yeah, my unscientific take would be it's definitely 95% the kind of quiet building up and five, maybe like three to five percent of the like glamorous light bulb moment where it's like, aha, I'm a genius. Like I've done it. I've cracked this puzzle. Like, God, I wish there were more, honestly, because it's so satisfying and kind of a romantic image. But I think

More often, it just is about the quiet and often unsexy work of changing the minds of the people around you. You know, getting your design team, like the people on my internal project team, as well as my clients to kind of creep along with me on the journey.

because it honestly is less risky. And also sometimes there just isn't the big killer insight, unfortunately. And it's about kind of like building up a really rich, full knowledge base and understanding versus having like the one thing that makes everything else make sense. And so, like I was saying, it is kind of a risky proposition, right? If you're

putting all your eggs in the one sparkling idea basket, and then your stakeholders just don't agree, like you're screwed. Or rather, there's a lot more work that has to be done and you're kind of in a tricky position versus like if they're bought in along the way, you know, they feel like they also like came to these conclusions themselves because you brought them along on the learning journey.

What I'm trying to say here is to encourage our listeners for doing their quiet product work is that many successful projects don't have this sexy light bulb part to them, right? They're just about doing this work right, not about discovering a plot twist. Yeah, exactly. Exactly. It can be done, you know, it's...

I think there's a lot of pressure that's associated with that like big idea concept where it's like, oh my God, we have to pour time or money into like finding the big idea. Or like, if I don't come up with a big idea, we're screwed. And like this project isn't thoughtful or like we don't have good insights.

And it really just doesn't have to be that way. You know, it can be everyone's job. As we've been saying, it can be done quickly, efficiently in a fit for purpose manner along the way. And honestly, it's better for it.

What are your favorite resources, maybe books, maybe blogs, when it comes to product strategy, research, anything that our readers, listeners could find helpful? I'll answer that in a couple of ways. I think this is like maybe the most sideways answer to your question, but like my favorite resource when it comes to research is just, as I said,

interviewing and like talking to people. That's my favorite technique. I also do often participate in other user studies. You know, I signed up to research panels and I'll be like someone who gets interviewed and someone who participates in focus groups because I

you know, it's a changing market. It's a changing field like everything else. And I like seeing how other people conduct their research and like how other people moderate. I've learned a lot about how to be a good moderator just by being moderated upon or being a moderator. Aside from that, I am such a huge follower of Erica Hall, who is the founder of Mule Design. And she is

wrote this book that really informed so much of my thinking and of my approach to this topic. And the book is called Just Enough Research. And it's really wonderful. And I so recommend that everybody read it, everybody buy a copy. And she also is just a really good

speaker and writer on LinkedIn. Honestly, I think she's worth following on LinkedIn because she often posts her ongoing thoughts and her insights. And she'll also post links to her latest writing and her talks, which I always love to listen to. I once participated in research where she interviewed me, which was really, really cool. And I was trying not to be a fangirl. I was like playing it really cool, but I learned a lot from her. So I can't

say enough about her work. So I would call that out.

I also think the UX Collective is a really great blog that covers not just research, but just design broadly and from and about many different disciplines, many different topics and industries, kind of trends. It was founded and is edited by my good friend and colleague Fabricio, who is one of the design partners at Work & Co. So giving him a shout out for that too. But I would put those as my biggest resources that I would recommend.

Awesome. And when it comes to your own writing, where can people find you online? Yeah. So I have...

A website that I will say is probably not the most rich well. It's really just kind of a landing place. But my website is Alice Lee, A-L-I-C-E-L-E-E dot X-Y-Z. But really the best place for my latest kind of updates and the best place to reach me is on LinkedIn. So you can find me on LinkedIn just by searching my name and work and co.

Awesome. Thank you so much, Alice, for being our guest today, for inspiring us and for sharing your hard-earned wisdom. It was such a pleasure. Thank you so much for having me. I really enjoyed it. Thank you, Alice, and have a wonderful rest of your week. Yes, you too. Bye.