Micro Utility With Gen AI: Shopify's Miqdad Jaffer
35:16 Share

Today, we're airing an episode produced by our friends at the Modern CTO Podcast, who were kind enough to have me on recently as a guest. We talked about the rise of generative AI, what it means to be successful with technology, and some considerations for leaders to think about as they shepherd technology implementation efforts. Find the Modern CTO Podcast on Apple Podcast, Spotify, or wherever you get your podcast.

How can generative AI tools help grow businesses while keeping users in control? Find out on today's episode. I'm Mikdal Jaffer from Shopify, and you're listening to Me, Myself, and AI. Welcome to Me, Myself, and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and business strategy guest editor at MIT Sloan Management Review.

And I'm Sherwin Korubande, senior partner with BCG and one of the leaders of our AI business. Together, MIT SMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.

Hey, everyone. Today, Shervin and I are pleased to be joined by McDad Jaffer, Director of Product at Shopify. McDad, thanks for taking the time to talk with us. Let's get started. Thanks for having me. Happy to be here and excited to talk a little bit about AI. Great. We all use Shopify, but we may not know it. Can you give us a brief overview of Shopify and tell us what a Director of Product actually does?

Sure. So Shopify is a platform for entrepreneurs to be able to set up their storefronts online. You may have seen Shopify whenever you're going to purchase an item from a retailer. You may have gone to the checkout and seen that purple shop pay button. And for many of us, that's probably where you've been exposed to Shopify. It's a way for an entrepreneur to be able to set up their storefront on the internet and for a way as you as a buyer to be able to

transact in the fastest, simplest way possible to be able to make sure that the item arrives at your door. In other cases, you may have seen Shopify on a platform like Shop.ai, where you may have searched for a specific event that you're putting together, a specific product that you're looking for, and tried to filter it all down across the many merchants on Shopify.

A merchant would go into Shopify and log in. There is a product section on the left as an example. They would go into the product section, they would click add a new product. And then within add a new product,

There is space for a title, a description, and some other fields. In terms of my role, my responsibilities are how do I think about AI and how can it fit into that realm of entrepreneurship? And what can we do to accelerate our merchants' success and progress as they build up their storefronts, sell their products, and grow as entrepreneurs? So how does that involve artificial intelligence? I heard a lot of e-commerce. I hear a lot of connecting, but how does that involve artificial intelligence?

Yeah, it's a great question. I think that when people are looking to start as an entrepreneur, there's a lot to it. So there are considerations of what's the right product? What's the right price? What's the right placement? What's the right promotion? How do I set up everything with my storefront? How do I maintain what good looks like as I grow and as I advance? What we've seen is that AI is an opportunity to make entrepreneurship accessible for everyone.

With every business unit, we wanted to utilize AI to create more efficiency from managing administrative tasks to supporting business operations. Administrative tasks can be as simple as creating a new product description, figuring out what the right subject line for an email could be. And then the business operations side of it could be just about everything from understanding the business, figuring out the analytics of it, what questions should I ask, what's the strategy that I should employ? And we view AI as a powerful assistant

that can weave itself in and out of that process. What you're saying is Shopify is...

Letting the entrepreneurs focus on the actual product and their brand and all of that kind of stuff, but then have Shopify be a platform for how they take that to market, how they want their brand to appear, and a lot of the backend operations. Obviously, AI could be quite prevalent across all of those. Was AI embedded from day one as it launched?

We've had AI in its most traditional sense embedded since early days. We did things along the lines of fraud detection, where a transaction comes in and it's not easy to tell whether this is fraudulent or non-fraudulent. We've done basic classification models to be able to determine whether something is fraudulent and then provide a score back to the merchant in a reasonable fashion. And that's been in there from the start. Then generative AI comes along and there's opportunities to be able to do a lot more and

And what we've seen is we've created a suite of products called Shopify Magic. And the idea behind this was how do we embed this directly into the workflows that our merchants are having to go through? And how do we make that easier to start, run, and grow your business? We did things like auto-write where we can...

Create a description for a product, create a blog, create a new page, help with the subject line of an email, create the body of an email and look at even ways for on the customer service side to be able to provide responses for a merchant as they interact with their customer. So those are some of the early stages, but we're trying to accelerate it even more these days.

A lot of companies are thinking about generative AI, and it seems like you've been doing it for quite some time. Can you comment a bit on when you started thinking about it, prepping for it, and I would assume to some extent redirecting or upskilling or reskilling some of your technical folks to be able to actually take these things to market?

I'll give you a little bit of the genesis of how we kind of started with the first thing and accelerated it even more. We have some general themes that come up every year in terms of things that we should be looking at. And one of the ones that's usually pretty prevalent on there is staying on the cutting edge of technology is critical. What we want to do is try to bring as much of the latest technology into the hands of our merchants so they have every advantage possible.

And we've always tried to do things with AI. And one of the more recent ones, even before generative AI, was product classification. So the idea of being able to determine the category of a product using its text and as well as its image to be able to figure out where this belongs. It helps in channels. It helps you basically market to a wider audience and get a reasonable taxonomy set. When ChatGPT came out, the question was, what can we do with this technology to put it in front of merchants?

And this was one where we wanted to lean in for a couple of reasons. One, it's important to get this in the hands of merchants as fast as possible, staying with the cutting edge of technology side. And the other part of it was we actually saw that there was utility for the first time. And that utility was important to get out there and to see whether our merchants would use it and what we could learn from it. So we started with a simple path of what's the thing that merchants are struggling with? And we saw them struggling with

product descriptions daily. There's billions of them across Shopify and we saw that as like a wide space to work in. We had all of the conditions around what would make a good product description. And then we felt, okay, let's get this in and see how people use it and see whether this is something that's palatable.

So we quickly spun up a team and we relied upon a couple different principles. Shopify generally builds off of a set of principles and allows the team to go fast. Some of the principles we worked off of was this has to be easy. This has to be integrated into their workflow. And the merchant need to be able to have final say as to what goes out there to represent their business.

It's a simple start. We said, okay, let's figure out what the rules of a good product description are. How long should it be? What's the tone it should be used in? What are the type of words that will be used and what makes it really good? We did the basic thing of prompting against that and then said, okay, well, what additional context should the merchant provide and what should we look for to be able to help them? And so we said, okay, well, they can probably list out a couple keywords about their product and that fits into the realm of knowledge they know about.

And the rest of it should be us. Like this should be as simple as like pressing a button. It should feel like magic. So apropos to that, we named it Shopify Magic and we wanted to stick with that. All a merchant has to do is they enter a couple of keywords, they pick a tone of voice and they press generate.

That was it. That was the first version. We tried to ship that real quick and see what would happen. And the feedback we got was, this is amazing. Can you do this everywhere? So it's like, okay, so there's some positivity here. So what can we do next? And so we added a special instructions field. And initially in our trials, we were doing things like add a quote from a famous celebrity that would be related to my product, just as like a meme for how to try it. And then...

Then we saw merchants using it in different ways. So we saw them using it for language. And this was an interesting one. We introduced the ability to support eight languages and we saw people writing translations with it. And that was expected, but it was interesting to see. And I think one of the most interesting ones we saw was there were merchants that didn't speak English and wanted to sell to an English audience and saw product descriptions as a way to bridge that gap.

But at the same time, it's one of those things where you put the product in the hands of the user and then the user decides what they're going to use it for. And it was remarkable to see that we opened up entrepreneurship for even a small window more of people than were there previously. And that's what kind of got the ball rolling on the AI at first. After that, it was every team wanted a piece, every team wanted to do something, and we accelerated it in every way we could. We've tried to like...

systematize things from a UX perspective. Whenever a merchant sees something, they know that this is what that's going to do. And we've also tried to look into building Sidekiq, which is the first of its kind AI-enabled assistant, purpose-built for commerce. And so that's kind of been the journey that we've gone into. We have dedicated teams that are working on specific features, and then we have the entire company that is looking at opportunities of where AI can accelerate commerce.

So when Sherman and I first heard about what you were doing, I think what kind of got us excited was how pervasive this is. The effect of something like Shopify integrating artificial intelligence into its tool is huge. And I think we were talking earlier, you talked about introducing this product in February. This is very early on in the generative days.

A lot of people are investigating generative, but this was an exciting story where you're using these tools in production very quickly. I'm sure that was a little bit painful. Maybe you could speak a little bit about any of the pain and difficulty you had. Maybe other people have learned from that, but also at how pervasive it is. Both of those are interesting ways that people can learn from what you've done.

Yeah, early days came with its challenges, but I think that that's kind of the fun of it. It was interesting because this was the latest technology. There was so much utility to it, and we just wanted to get that in the hands of our users. Along the way, we had to figure out things like, how do you do evaluation? Like, what is this prompting stuff? How do you systematize this and make this make sense? What's all this RAG stuff about like RAG being retrieval augmented generation? How do I bring in more data into this context of a prompt? And how do I make this make sense?

And the crazy part was every week, a month or a year's worth of other technology time passes. And all of a sudden, there's a new thing to try and a new thing to explore. And we didn't want to miss that either.

every time something new came out, we tried, we explored. And I think that the culture at Shopify is one of exploration and a one of crafters and builders. And what we tried to do is put this in the hands of people that could build and to try things. And we also did things internally to encourage the use of AI and,

We've built our own kind of internal ChatGPT instance equivalent. We have it for our internal wikis for search and discovery. And what we've tried to do is just make this into the fabric of the company.

It's normal. It's expected. And then all of the risks and challenges that come with it, hallucinatory included, are things that people are familiar with. And once we got them familiar with it, then we figured out ways to work around it and to embrace it in some cases, too. Yeah, I think that's a very important point you're making because as I reflect back on my own experience and BCG's experience in this field,

We're working with many companies and many of them have the same mindset and ambition as you've described and are putting solutions in production and are embracing it. And then there are some that are

I would say, quite reserved about this. And some of that is based on, I would say, some unfounded fears. I mean, the reality of it is this is new technology and it comes with a lot of risk and maybe a little bit more risks than traditional AI because of how pervasive it can become. But I think the mentality of...

for all of the risks to be figured out by other people then puts you at somewhat of a huge disadvantage because the only way, as you described, I mean, I love how you said it, the only way to actually get ahead of these things is to start experimenting with it and getting comfortable with its uncertainty

I'm sure at the board level, there were some risks and some concerns. And maybe just comment on what many people thought back then versus what the reality was and put it in perspective for some of our listeners so that they understand that these risks can be mitigated versus, you know, you just stay away forever.

Yeah, one of the first things I'll say is that I want to describe who an entrepreneur is to even put this into context. Every entrepreneur I have ever come across has the highest risk tolerance of anyone I've ever met. They are willing to drop everything in their life to start a business. And sometimes they don't know anything except that they want to start a business.

And these people are inherently risk tolerant. So our user group is very different than others. And obviously, you're going to get a spectrum of tolerances across that user base. But by nature, this is a group that is willing to dive in, take risks, and try things.

And then one of the things that we observed was that they were already using it. So we already had merchants that were using ChatGPT or whatever was out there to be able to prompt it, figure out alternatives and use it directly into their software. We did a brief round of research where we asked a number of merchants,

Is this something that you know about? How have you used it? What are you doing with it? And many of them were like, yeah, well, I mean, I just ask it everything. And so I'm just getting to figure it out. And we already saw that happening. So for us, it was, all right, well, we already have this risk tolerance merchant base. And the biggest concerns that we have are typically on two vectors. Vector one is risk.

a fear of losing control. So asking questions along the lines of what's it going to say? What do I do if it hallucinates? Is this going to be right for me? Is it going to speak to my buyer? So all of those initial questions and that fear of losing control was easily mitigated. So we said, okay, well, here's a quick fear. How do we mitigate this fear? And we mitigate a fear of losing control by putting the control back in the hands of the user. So CARP launch principle was the AI will never write without user intervention.

And so the merchant will always be able to see what the message is that's going to go out there. So all of the hallucination risk, all of the what's it going to do risk is gone because now it puts control right back in the hands of the user. The second risk is around whether it's factually accurate or not. And that's typically one that a lot of people come across. The way that we've tried to manage that is we've

The person that knows the most is the person selling the product. The thing that they know the most about is their product. And that's what they're trying to talk about. And they're trying to talk about their brand. And so that one didn't feel as risky for us either because it was going to be up to the merchant to decide how they were going to be represented in a factual way.

You talk about this in such matter-of-fact way, but when you think about the actual key word here is the user, which is actually, when you think about, Sam, like a lot of our work around human and AI versus just human alone or AI alone. Thanks for talking about that. That's very key. One of the things I really like, too, you keep using the word utility. A lot of the examples that Shervin and I talk about, we'll ask people for an example, and they'll bring out some

AI heavy application that they've implemented that crosses lots of parts of their organization and it's a big deal. And those are great. But the utility you're talking about seems like micro utility. And so what I mean by that is that, as you mentioned, people may have already been going out to some large language model and generating some text and then pasting it into Shopify.

Your incremental utility here was saving them some back and forth between those two tools, but also then incrementally making it work better within your tools and within your context. And I think that's exciting when you think about how all those tiny pieces multiply by the billions. And that's a different story than what Shervin and I see sometimes, which is we have a big system which has a big utility and there may be not so many users of it.

I want to come back to something that you mentioned earlier, and maybe I just picked up on your tone of it, but you're talking about fraud detection and these somewhat traditional uses of artificial intelligence. I mean, that feels like an oxymoron now, but you said, oh yeah, we're using it for fraud. We're using it for backend business optimization and these sorts of things. Oh yeah, yeah, yeah. We do all that stuff, but here's the really cool stuff. It seems really exciting that you're able to stay forward. What's next?

It's a great question. I think the hope for us is that we want AI to lower the barrier of entry for entrepreneurship.

Period. And I think the idea for us is let's look at everything that's on the space. So we did something relatively recently about image generation. And we saw this as, okay, well, we can lean in here too. There's different modes of operating that we can play with. So we released the Hugging Face pipeline. And the idea behind this was, let's put this out into the market. Let's see how people are going to use it. And let's do it in a way that's going to be innocuous. The idea behind it is you upload a product image and

And then you pick what background needs to be switched out on. And some of this, there's some fun stuff behind the scenes of like uplift, upscaling the image of figuring out the right mask for it, making sure that the reflections and shadows are going to be good. And then generating all sorts of backgrounds that make sense in the vein of commerce. And what we see from this is the ability to explore in a different way. What we've tried to do with AI and to your point about

the risk tolerance of our user base, it's not meant to be something that is just disruptive to their existing workflows. Many of them, especially established ones, have existing workflows where they do everything in their specific way and that's what works for them.

All we're doing is for anyone that needs that extra push or needs that extra piece, there's a button to click, a way to invoke this, and it is their choice to make on that. And Shervin, I like some of the stuff that you were talking about in terms of putting the user in place. And our adage has always been that

This is meant to be something that augments and isn't a replacement. So this is always going to be something that helps a user be the best version of themselves. And with image generation, we see the opportunity for them to holistically change their storefront if they want to, if they have a

Thanksgiving sale coming up and they want to be able to change all the backgrounds to be fall themed. It's a quick, easy move for them to do. Their product stays at the forefront. Their branding stays at the forefront. But they can merchandise just for the specific event. If it's Thanksgiving, if it's Christmas, if it's New Year's, whatever it might be, something that would require additional photo shoots might be a lot easier to achieve now than they were previously.

Exactly. Just remember like how complex and unnecessarily sort of process oriented something like this would have been, you know, a few years back where you have just agreeing on the background and the photo shoot and the look and feel and the impact on brand and like all of that. And now you're putting it all in the hands of the merchandise.

For us, it was really important that many people across Shopify were using this, asking the right questions, pushing the boundaries, and understanding some of the nomenclature around it. When I say hallucination, I can almost guarantee that 99% of Shopify knows what that means and how that happens and what that works. And that makes the conversation much different every time you go towards product development because...

It's not bringing it up and relitigating it every single time. It's people trying the solution around the normal problems that come up with this technology. Yeah. If I squint, I kind of see something developing over this market of artificial intelligence. You alluded to it with the ease of transfer learning. You're able to take these existing models and augment them slightly. And I see that as an improvement over the last decade from...

Oh, yeah, sure. You can download these tools and build them yourselves to, hey, no, you can take what we've done and what we've trained our GPUs and burned up our GPUs on and incrementally burn up a little GPU on your own, but then move forward. I think that's a dynamic that's happening in this market here. But let me go darker here and talk about another dynamic. Is there a

let's say a race towards mediocrity and I think that's a little charged way of saying it but if all of your users in Shopify are using the same tools to generate the same sort of product descriptions isn't that going to lead to a sort of vanilla feeling of all product descriptions how do we push back and get beyond that it's great that we get everybody to mediocre well how do we get

People distinctive. Where do we build off of that? How do we make that beyond mediocre? Yeah, the problem you're talking about is a local maxima, effectively, right? So everyone moves towards this standard, simplistic, homogenous, boring thing. What we've looked at is...

that all of the tools that we provide will get you started. The idea is not to be, this is the finish line. The finish line is the merchant. The finish line is their product. The finish line is their brand and being able to represent accordingly. For many cases, this is about like getting you something, which is better than nothing and getting you started in thinking. We had one merchant that really described it well for me. He said,

wanted to set up an outlet store for some of the overstock inventory that they had. And they had 3,000 SKUs that they wanted to put together, 3,000 different products that they wanted to put on this initial outlet store. And what he said to me was, yeah, typically what I would have done is I would have hired a team. We would have taken three months to get this thing spun up and we would have tried to put it together.

And he said, I spent the weekend by myself and I just used your generation feature. It got me started. I made a couple of changes. And in three days, I had an outlet store stood up and ready to go for all my products. And yeah.

I made the changes necessary. I don't like the style that you had. So I gave it some specific instructions and it got to what I wanted. And I think the idea is like flexibility within the framework. So provide a framework, let them get started, but give them all the flexibility in the world to do what is appropriate for their business and for them to be able to differentiate to their audience. And I think the key is every audience is different.

And nobody understands that better than the merchant themselves. The idea is capability and augmentation, not replacement. And you're right. There could be a point of local maxima for people, but we want to leave every escape hatch possible so that they can get out of that. One thing I wanted to stick back to, and you alluded to it, was the deterministic versus stochastic nature of this. That seems really difficult to work with in a production standpoint, that you're using tools that have stochastic outputs, but you'd like to have a

defined, tested, refined process. And those two things seem antithetical to me in many ways. How do you deal with that?

Part of it is there needs to be a risk tolerance built in. So there needs to be almost an expectation that the system will fail and that the user needs to have a way to be able to work through that failure. So product description is a great one. We actually see that it takes multiple generations before the user is happy with what they have in front of them.

So with that in mind, you have to allow for that second, third, fourth, fifth generation to take place with no effort whatsoever. And so that's an error case that we know is going to happen. So we plan for it. So this is regenerate button right there. It presents multiple options. In other cases, we will try to present as many options as we possibly can to make it easier. So subject lines, you get three options to choose from. You can change it, do whatever you want to do. Then the other part of this is that

We understand that technology is going to change and the way that people use it is going to change and the expertise level is going to change. So there are people that are going to try and break the system and there are people that are going to try and work around the system.

It's important that we plan for that scenario too. So it's every risk and every failure case has to be considered upfront. And then you almost do it in a fun way. And with the one that's typical is prompt hacking, where people are trying to figure out what the prompt was that was used for this.

In most of those cases, you do the best you can, but you expect that the prompt will get out there. So you write the prompt in a way that like when it gets out there, it's like, hey, you found me. That's great. Do you want to apply for a job or something? Like there's lots of different ways to like think about this. You're human in the loop solves a lot of this, too. Correct. Your whole design philosophy really addresses, I think, a lot of the backstops, a lot of the stochastic nature, too. Can you give us a bit of your background? How did you get to your role?

Yeah, I started out as an engineer. I think since I was like 12 years old, I was programming and trying to figure out how to make the computer do what I wanted it to do. And I took an initial role as an engineer and was introduced to product management in what was then meter data management. So it was like big data for utilities that just come out. I got to work in that space and then I moved into engineering.

Document classification, then ad tech, and then retail data platform management. And I've always been adjacent to data. And at Shopify, my journey had been build our infrastructure and data platforms, work on our machine learning platforms and figure out what needs to be built there. And I always had like one toe in the machine learning AI pool at all times.

Then I had teams that were focused on the analytics side of it and what merchants were going to experience and how they were going to understand their business. And the mandate was always like, how do I optimize my business with data? And we looked at opportunities with ML. We looked at opportunities with AI. And then when generative AI came out, it was just the next step along the way. So it's,

by luck, by accident, by design, who knows, but I've always been in data for the last like 15, 20 years. And this was the next step. Okay. Time for five questions. Madhav, do you know what that is? I do not. Okay. I mean, I know what five questions are. Good. So we're going to ask you five questions, rapid fire style. Just tell us the first thing that comes to your mind. Sure. What do you see as the biggest opportunity for AI right now?

I think the biggest opportunity is solving cold start for many people. What I mean when I say cold start is looking at a blank screen and not knowing what to do, whether it's how to navigate a system, write a thing or whatever it might be, just that unknown moment. I think that's where AI steps in and does the biggest. I love that. What is the biggest misconception about AI?

that it will take over. I think that there's a long way to go. It's an augmentation, not a replacement. And I think it's the same as when Photoshop came out. Designers got much better and much more creative. And I think this will raise the bar for everyone in terms of what they can do. Is there such a thing as too much AI? When is there too much?

I think it's too much when the user is out of the loop. I think the user needs to have a stronghold in how they use AI and how they interact with it. I think this is not about replacing, and I think this is a matter of augmenting and helping the user be better. What was the first career you wanted? I wanted to be a doctor, but I'm faint at the sight of blood, so that didn't work out so well. What is the one thing you wish AI could do that it cannot do right now?

I wish we could incorporate all of the modes of a user wanting to interact, whether it be a combination of voice, facial expressions, just to be able to get that semantic meaning behind the scenes that words can't do by themselves. Wonderful.

We really appreciate the idea that what you're doing is using gendered AI right now, just barely after it got started. I think a lot of people are wringing their hands and wondering how to use these technologies. And what you've shown is a great example of how people can get real utility from this tool right now. Thanks for taking the time to join us today. Thank you so much for having me, both.

Thanks for listening. On our next episode, Sam and I speak with Ellen Nielsen, Chief Data Officer at Chevron. Please join us. Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders. And if you join us, you can chat with show creators and hosts, ask your own questions, share your insights,

and gain access to valuable resources about AI implementation from MIT SMR and BCG, you can access it by visiting mitsmr.com forward slash AI for Leaders. We'll put that link in the show notes and we hope to see you there.