Sidekick is an AI Shopify expert
51:39 Share

Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, and accessible to all. If you like this show, you will love The Change Log. It's news on Mondays, deep technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts.

Thanks to our partners at Fly.io. Launch your AI apps in five minutes or less. Learn how at Fly.io.

what's up friends i'm here with kurt mackie co-founder and ceo of fly as you know we love fly that is the home of changelog.com but kurt i want to know how you explain fly to developers do you tell them a story first how do you do it i kind of change how i explain it based on almost like the generation of developer i'm talking to so like for me i built and shipped apps on heroku which if you've never used heroku is roughly like building and shipping an app on vercell today it's just it's 2024 instead of 2008 or whatever

And what frustrated me about doing that was I didn't, I got stuck. You can build and ship a Rails app with a Postgres on Heroku, the same way you can build and ship a Next.js app on Vercel. But as soon as you want to do something interesting, like as soon as you want to, at the time, I think one of the things I ran into is like, I wanted to add what used to be like kind of the basis for Elasticsearch. I want to do full text search in my applications.

you kind of hit this wall with something like Heroku where you can't really do that. I think lately we've seen it with like people wanting to add LLMs kind of inference stuff to their applications on Vercel or Heroku or Cloudflare or whoever these days they've started like releasing abstractions that sort of let you do this. But I can't just run the model I'd run locally on these black box platforms that are very specialized. For the people my age, it's always like, oh, Heroku was great, but I outgrew it.

And one of the things that I felt like I should be able to do when I was using Heroku was like run my app close to people in Tokyo for users that were in Tokyo. And that was never possible. For modern generation devs, it's a lot more Vercel based. It's a lot like Vercel is great right up until you hit one of their hard line boundaries and then you're kind of stuck. There's another one we've had someone within the company. I can't remember the name of this game, but

The tagline was like five minutes to start forever to master. That's sort of how we're pitching Fly is like, you can get an app going in five minutes, but there's so much depth to the platform that you're never gonna run out of things you can do with it. - So unlike AWS or Heroku or Vercel, which are all great platforms,

The cool thing we love here at ChangeLog most about Fly is that no matter what we want to do on the platform, we have primitives, we have abilities, and we as developers can charge our own mission on Fly. It is a no-limits platform built for developers, and we think you should try it out. Go to fly.io to learn more. Launch your app in five minutes. Too easy. Once again, fly.io.

you

Welcome to another edition of the Practical AI Podcast. This is Chris Benson. I am going solo today. Daniel's not able to join me, but we have two guests today. And I would like to introduce you to Mike Tamir, who is Distinguished Machine Learning Engineer and Head of Machine Learning at Shopify, as well as his colleague, Mike Collier, who is the Director of Product Management for Sidekick. Gentlemen, welcome to the show.

Thanks, Chris. Yeah, thanks for having us. Glad to have you on board. I know you guys are doing a lot of cool stuff in the AI space. And so thank you both for joining to cover the different aspects of it. For those who may be joining who probably have heard of Shopify, but may not be users or may not be intimately familiar. Can you guys talk a little bit before we dive into all the good AI goodness stuff?

Could you guys talk a little bit about what Shopify is as a company and kind of how you see the space that you're in? What need are you fulfilling? Just some of the general understanding of your business before we get into the AI stuff. Yeah, I can jump in and then maybe Mike can add in, sprinkle in some bits. Sure. Yeah, Shopify is one of those incredible companies that you might not be aware of, but you've probably used it, even if you aren't specifically aware of it.

So our mission is to create the retail operating system behind your favorite brands. So you can think about it. It's like there's the chain restaurants you go to, right? All around your hometown. But then there's that your favorite coffee shop that's run by like a local operator, right? And like, it's just, there's something better about that coffee, right? And so Shopify's goal on the internet is to enable those kinds of operators to have a successful business out there. And so we power many brands online. Some of the ones that are out there that are more famous are like Dropbox,

Drake, Patel, Gymshark, Heinz, just to name a few. And so like some of the world's biggest brands are on there, but also some of those entrepreneurs that are in your local area are too.

Can you describe like what sets Shopify apart from other, you know, like other processing? I mean, it has a very distinctive brand, I know. But can you share a little bit about what what makes it distinct from other things like credit card processors and, you know, cart processors? I know you guys kind of have your own distinctive way of doing things.

Yeah, I mean, it's a full soup to nuts kind of solution. So like we do e-commerce, so like we can help you build your site. We can help you create merchandising around the products that you offer. We also offer payment solutions. So you don't have to set up a separate credit card processor, but if you have one, you can bring it too. I think that's actually one of the most powerful things about the Shopify platform is that it's got a very extensive developer ecosystem. And so many of our merchants install apps from our partners to do specific things. So if you've got a specific platform

shipping provider, you can use their app for it. If you've got a specific email provider, you can use your app for it. For many of these categories, we also have solutions, but the ecosystem is definitely the richest part. It sounds like you also do lots of different market segments from some of the large brands that you just talked about down to, I know when I've come across Shopify in the past, it's been in the context of kind of smaller business and e-commerce and mid-sized business and things like that. So

It sounds like you guys hit quite an array of different customer segments with technology solutions. Yeah, I mean, maybe, Mike, do you want to chime in? I feel like you work on more of them than I do these days. If you think about what, you know, all the things that Matt described, you know, establishing a website, payment processing, that's the infrastructure for something you could provide for a large established brand all the way down to a smaller brand.

More and more, especially in recent years, what Shopify has been focusing on is not just providing you with the website, but also providing you with kind of the tools for growth. And this is where AI and the focus that we've had on moving to machine learning and AI has really made itself apparent. So, for instance, we have our shop app where a new merchant who has no track record of sales or history can join us.

that app. And when we search, if we understand the backend machine learning and understanding how to do the retrieval and the ranking well, we can reveal a new fresh merchant to a customer right away. Very cool. Piggyback on what Mike was saying. I think it's like a great example of how he worked in the AI component of it. But like one of the core values of Shopify is that we keep merchants on the cutting edge.

So like we view our mission to like understand technology at a really deep level and always be out there scouring the best and then figuring out how we can apply it to our merchants businesses. Right. And so like Mike was talking about, how can we help our merchants grow? Right. And so how can we apply these machine learning models or these AI techniques and really help these people grow their business? With you guys having that technology infrastructure that you've been supporting all these businesses with and everything, right.

At what point did you start thinking about the fact that there were these AI technologies that have been on the rise in recent years? What was the turning point for the company where you started seriously looking at AI as a supporting factor in the business model? What made you say, I see an opportunity to go help our customer get done the things that we're trying to do that we've been doing for years now?

what was that turning point? Kind of how did that come about and how did you start thinking about AI in the business? I mean, knowing the timeline, you know, I think that the turning point was what we might be the chicken and not the egg, so to speak. Um, and that, you know, Shopify maybe historically was not as invested in machine learning and AI, but, um,

by 2022 had become interested in that and has certainly massively redirected our forces and the work that Matt and I joined and have focused on over the last several years. And can you talk a little bit about that vision as you talk about, you know, you guys kind of coming into the company at that point and carrying that forward? How

How do you think about the mission, if you will, that you're doing? How do you contextualize it in terms of how you want to carry it forward and how you're going to serve your customers with that effort?

I'll give you my product manager-y answer to this. That sounds fine. It was interesting hearing Mike's science answer to this. I think it's about finding out what's out there in the world. So to be quite honest, like in that time period, like if you all recall, it feels like, I don't know, a million years ago, but ChatGPT didn't exist like three years ago, right? That's right. And we forget that that was a world, but it was. And so like, I think we got enamored, like the folks that work with Shopify with that technology just as much as everybody else did. I definitely remember there was a peak

chat GPT moment where my mom was telling me about how she got it to write it a poem. And so I think the whole world was just captivated by the fact that we had computers that could, you know, write us stories. And so I think I think

I think that's kind of the culture of Shopify, right? Like we're all tinkerers and we like to build and we're all there getting messages from our moms about how they can write stories that they couldn't write before. And they were like, naturally you're like, maybe, maybe there's a way to apply this to commerce. Right. I'm curious as you talk about that. I like to, I like the mom story because that literally holds true with me. I have a mom. She's long since retired, but she, she was a technologist. And so, yeah, we have those moments where she's like, well, I'm going to go try that thing out.

and do that. I'm kind of curious, as these new technologies were coming about and you guys are coming into the company and kind of carrying it forward, there were a lot of choices that you guys had to make, you know, in terms of like, there's obviously we talked about chat GPT, there's open source, there's a whole bunch of different approaches that

to how you're going to support your customers with different technologies and different ways of addressing. What was your thinking, both on the technical and on the product side, in terms of how you might do that? I would describe our approach as unembarrassed.

with how pragmatic we are on these issues, whether it's open source or one of the commodity providers for LLMs. We tend to gravitate to whatever works and we do keep multiple threads of experiments with all of these different options, technological options, open for solving every problem. I think that that pans out with...

What we currently have in production and active is a nice array of not just all of the foundation models that you might think, the commodity options that are out there, but also being pretty aggressive with how we use the open source versions that are out there. As we've talked about that, you bring up open source there, and we've talked about productized offerings such as ChatGPT through API and stuff.

Going into this, and before you got to the point now where there are a number of things that I know we're about to talk about, how did you, like from a strategic standpoint, how did you parse that? How did you say, we have this challenge and that we're a big, successful technology company. We're moving into the brave new world of AI. And you had to kind of like...

figure out what do you want to do with foundation models? Are you hosting your own? Are you going to go to APIs? How do you think about that? And not just from a technical standpoint, but from the perspective of serving your customers and stuff, how do you see the strengths

and the weaknesses of different perspectives there. If you could share a little bit about your insight as you had to analyze on behalf of your own business. Echoing what Mike said, it's like, I think it's early innings in this industry, right? And so like at the beginning of an industry, it's like there's a lot of change and it happens very, very quickly. And

And so I think it's hard to pick any one solution. I think what people thought at the beginning of this new wave of technology is that maybe fine tuning was the answer, right? And it's like, as model and context windows will be small forever and it will be expensive. And I think it's blown everybody's expectations out of the water at how quickly how things have gotten less expensive. I don't think anybody could have predicted that. And so I think...

It's just such a world of abundance, both in the operational costs, but also the other thing I would say that's unpredicted is that the number of solutions out there is just unparalleled. I don't know without the leaking of the Lama weights, I forget back in March of, I don't

I don't know that we'd have the open source community that we have today, but since that Cambrian explosion of open source models, it's been crazy, some of the innovation that's out there. And so I would like to think that we had a master plan. We're like, oh, yes, we saw exactly. We were going to use these commodity models and then move to the open source models. But the reality of it is always messier than the historical written version, I think. And so I think the short answer to all of that is that we try a bunch of stuff. And I think the people that win in this space are the people that try most things the fastest.

And so I'd say that's our overall strategy is we try a lot of different things. And then like Mike said, like we have a rigorous way of determining which is the best, but that frequently changes. Like one approach that worked three months ago, like I can't even count how many times on Sidekick we've replaced the core underpinnings of it because we found a new approach that's better.

I'm not at all surprised by that. And I appreciate that answer because I think that's a challenge that everyone out there, a lot of folks listening to this are facing as well as it is moving so fast. And what was expensive yesterday is plunging and costs as new things are on the rise and such.

And so I like your notion of experiment very fast, fail fast, I guess, in the process on your experiment so that you know what's working for you at least today until the next thing hits you tomorrow. Have there been any growing pains that come to mind that you can share that have been kind of like, oh, you know, where you plant your palm on your forehead and say, I wish I could have seen that coming?

So this is something that I think the entire community has been working out in real time. Like Matt said, you can try different things and see

which strategy works the best or which tactic works best or what combination of those tactics work best and change all the time. Because we do know that new models are going to keep being released, large commodity ones, smaller open source ones, large open source ones. And we're learning all the time in the research what sorts of tactics with whether it's fine tuning or using long prompts or combinations of domain adaptation, that's all going to be in flux. And we should believe that that's going to be in flux for a very long time.

And so if you have a mandate of, "Hey, we're open to anything and we'll use what works," we have to have a definition of what working looks like. And that means, and Matt's going to laugh because this is something that we've worked on quite a bit, you have to have your eval system dialed in. And that's the sort of work that we're moving into different formats for how you might eval, especially with unstructured text generation for figuring out this

This was a good answer. This was a bad answer. And being able to measure it in various ways. And there's all sorts of creative solutions depending on the context. But making sure that we have a way of measuring that versus saying two anecdotes is enough for me to think that all the swans are white. That's an important part of the process if we want to be pragmatic about our solutioning.

Yeah, we have like a, I guess a story on the team. We talk about it being the dark forest. And I think we're all grasping, we wish we had our GPS enabled phones of pinpointing us on the map of how you get out of the dark forest. But I think what we've had to settle for with eval is like a compass, if you will, like the metaphor here is that it's like, everybody wishes we knew absolutely like,

this is how good the system is, right? But like what we've kind of settled for instead that's more practical is like, well, we know A is better than B, right? And so if you just make enough decisions in aggregate where A is always better than B, you eventually find yourself out of the forest, right? But we wish we knew how long it would take. The question's always like, well, when are you going to find your way out? It's like, well, we don't know. We're just going to keep making the best next decision we can.

Okay, friends, I'm with a good friend of mine, Avtar Swithin from Timescale. They're positioning Postgres for everything from IoT, sensors, AI, dev tools, crypto, and finance apps. So Avtar, help me understand why Timescale feels Postgres is most well positioned to be the database for AI applications.

It's the most popular database according to the Stack Overflow Developer Survey. And Postgres, one of the distinguishing characteristics is that it's extensible. And so you can extend it for use cases beyond just relational and transactional data for use cases like time series and analytics. That's kind of where Timescale, the company, started, as well as now more recently, vector search and vector storage, which are super impactful for applications like RAD

recommendation systems, and even AI agents, which we're seeing more and more of those things today. Yeah, Postgres is super powerful. It's well-loved by developers. I feel like more devs, because they know it, it can enable more developers to become AI

AI developers, AI engineers, and build AI apps. From our side, we think Postgres is really the no-brainer choice. You don't have to manage a different database. You don't have to deal with data synchronization and data isolation because you have like three different systems and three different sources of truth. And one area where we've done work in

is around the performance and scalability. So we've built an extension called PG Vector Scale that enhances the performance and scalability of Postgres so that you can use it with confidence for large-scale AI applications like RAG and agents and such. And then also another area is, coming back to something that you said, enabling more and more developers to make the jump into building AI applications and become AI engineers using the expertise that they already have. And so that's where we built the PG AI extension that brings LLMs to Postgres to enable the

things like LLM reasoning on your Postgres data, as well as embedding creation. And for all those reasons, I think when you're building an AI application, you don't have to use something new. You can just use Postgres. Well, friends, learn how Timescale is making Postgres powerful. Over 3 million Timescale databases power IoT, sensors, AI, dev tools, crypto and finance applications, and they do it all on Postgres. Timescale uses Postgres for everything, and now you can too.

Learn more at timescale.com. Again, timescale.com.

So guys, I love the fact that you're, you're kind of your notion of eval and finding your way back out of the forest. You kind of with that in mind, you have a lot of different products that you work with. And, and as you're bringing these new technologies in and you're doing these evals and you're trying to find your way out of the black forest through that and managing across multiple areas there, you,

How does that look for you? You know, how do you unify different products so that you can effectively serve your customers with these technologies? And how do you make all those Legos come together in a usable way? There are certain genres of problems. And it seems like one or more strategies for eval will be appropriate for each genre. So let me give an example with Lego.

search, right? And evaluating search quality. There's been some good research that shows that LLMs tend to be better at rank ordering or labeling relevant, not relevant of a product to a query, you know, at a different, at a certain resolution, right? In fact, there's research that shows that the LLMs are better than a human at doing this.

And you might ask yourself, why is that? Why? You know, you're a human, Chris. You searched for white flower dress.

and you're going to click on one of those, right? One of those is going to be the right answer for that query for you. And then I might search for a white flower dress, and you search for one because you wanted a dress with a white flower. And I didn't want a dress with a white flower. I wanted a white dress with colorful flowers. And both of these are completely legitimate answers to that query. And what we're seeing here is it's actually just a sampling problem. If you think about it, there's a distribution of

appropriate products matching any query. And so every time we ask a human, we're getting a sample from that distribution. But if that distribution is very flat, then we're going to sample across a wide variety of different answers. And so what we've done with LLMs in this is, you know, you can think about this as an analogy. In the morning, I like fill in the blank, right?

And there's a lot of good answers to that. I like to exercise. I like coffee. I like breakfast. I like orange juice, whatever it is, right? There's a lot of ways of completing that sentence and they're all legitimate. It's just that it's a very flat distribution. And with language, what we do is we just overwhelm with sample after sample after sample after sample so that we can fill in that whole distribution. In the query product genre, it takes too much time and too much cost to

to fill in that distribution in mass that way until you get into implicit feedback.

So you have to find another solution. And this is one of the reasons why when you're using an LLM and replacing a typical Terker from filling in those answers, you get better results. Now, something important is something that Matt and I have seen in other contexts. You can't do this ungrounded. You can't just have the robots grade the robots and then hope for the best. You have to have different expert supervision to ground those.

those answers, whether it's in a search context, a personalization context, in more of a chat context, like the Sidekick product, you have to have that grounding. And once you inject that kind of like course correction, then you kind of get the best of both worlds.

Could you talk a little bit about the different products? You talked about query, you talked about personalization, but are there any others there that you say are kind of very prominent in your world that you're thinking about applying LLMs or other AI technologies to? So we've got several products that are AI enabled or magic enabled Shopify. So Sidekick is kind of the main one, which we'll talk more about. But to give you a general idea, it's a tool that helps merchants buy

find a way around Shopify, but also answer questions about their business. So you can think of it as like the co-founder they wish they had that's available 24 seven and isn't judgmental. So that's kind of like the sidekick idea. And then we've got a variety of other ones as well. So we have a lot of imagery. So it turns out shopping, like people want to see what the thing is before they purchase it. Not terribly surprising. And one of the things that merchants often want to be able to do before they've scaled up to a whole team that has a studio and a photographer and the rest of it,

is they want to enhance the pictures that they do have. Right. So like at the scale that they're at, this is where technology, again, bringing back like the best from the frontier and making it accessible to all of our merchants is exciting. So like there is technology that's out there now that you can essentially describe what you want to do to an image. You're like, Hey,

hey, my background's a little bit messy. Can you replace it with a studio background instead? Because we all know that the nice white studio background that looks like the object's floating in space, right? It's like, you can do that in real life. It's just really hard and expensive, and you have to know what you're doing. And there are very few people who know how to do that well. And so it turns out we've created models that can do it fairly well as well.

And so bringing that technology back, that's one of the products we do offer integrated into Shopify today is background generation. So merchants can import an image that they already have, replace the background with something that's more on brand. Like say they want to set their coffee to the background of like a jungle, right? They can place it on a table in front of a jungle if that's what they would prefer, or they could do it into the void of the white space. So lots of exciting opportunities with that. Another area that we've been investing in is that we have a product called Inbox, which

which allows our merchants to talk with the buyers that they have on their site. And if a buyer has a question like, hey, what's your return policy? Or like, where's my order at? They can interact with the merchants through inbox. And so one of the things that we're offering today is that we look at all the merchants policies and all the other things that they've given to us and then can help formulate answers for those common questions, right? It's like, well, what's your return policy? It's like, well,

we're pretty sure that this is the answer. And then we can suggest that to a merchant who then says, yep, that's right. Or if that's not right, they can adjust it to be correct and then send it. And so merchants love that because it saves them time for answering a lot of those repetitive questions. And then those ones that are a little bit harder, they can write them themselves just as they would before. And then the last one that I'm thinking about is like,

or again, going back to that product and merchandising kind of task is that oftentimes merchants are uploading a lot of these at the same time, right? And so like they don't always capture all the metadata in the first go. And so this is again, where we created models that actually can help with that. So like if you upload an image of a white flower dress, we have a model that can actually understand what that picture is and suggest that, Hey man,

"Hey, maybe this is a white flower dress and this should be categorized under dresses and like under cotton." And like maybe it can suggest, "Oh, it turns out if you upload multiple different colors, it's like, well, maybe you want to create product variants." And so that's some of the other technology that we're kind of working on today is like using the data that we have from merchants, enabling them to more expressively describe their products through our sites.

With Inbox, is Inbox part of Sidekick or is it adjacent or parallel to Sidekick in the way that you see it? No, it's a completely separate offering. So merchants have to choose to install. Again, going back to kind of the platform thing we talked about earlier, they choose to install the Inbox app, which Shopify builds. And then within the Inbox app, you can choose to use this behavior or not. You mentioned also the ecosystem a little while ago. And I'm curious how, as you have created these new AI-enabled products, how

How has the ecosystem been plugging into that? Is that something that's possible? What kind of interactions do they have together? - So today we have a very extensive API through GraphQL that we expose, like the data we just talked about, right? The categorization. So whatever the merchant decides, like we make a recommendation, they say, yes, that's correct. This dress is actually a dress.

Once they save that change, that information is then available through that product description API. I see. Matt covered a nice breadth of generative text, generative images, product understanding, all of which are kind of adjacent to or image generation is sort of a different algorithm under the hood. There's also a direction where we can think about something that I've... It's been a gift that keeps on giving for...

for all of my career is that the sorts of machine learning techniques that work with text often also work with commerce. And so this goes back to old-fashioned matrix factorization for recommender engines was also useful for understanding text.

RNNs, useful for looking at sequences or sentences before transformers took over. Good for text also. - Boy, you're taking me way back. We did many whole shows on RNNs and that seems like the stone age now. - Yeah, the stone age of the 20 teens, right? - That's right.

that also, you know, a lot of the techniques, you know, it was, you could always just peek at what you're doing in language and come up with a cool idea for e-commerce and vice versa. And so this has not stopped. I mean, transformers have kind of taken over everything, but, you know,

There are not quite transformer architectures, but heavy attention method transformer-like architectures that can look at sequences of behaviors of merchants, of buyers, the people that are shopping with our merchants. Those are sequences too and can be processed in an analogous way in order to understand what is next.

step on the journey for a merchant and how can we help them get to that journey? What are the likely ways we can simulate that? And that's been sort of one of our frontier cutting edge areas that we've been applying ML. Actually, Mike's answer reminded me, I want to add one more product that always slips my mind here. And it also blends with the ecosystem thing that we talked about. So as I'm sure you've talked with other folks on the show about, one of the exciting parts about LLMs is the ability to write code.

And we talked about the GraphQL API. And so one of the other exciting applications that we've done is for our developer ecosystem is enhancing our developer docs in the way that we now have an integrated tool that assists developers in writing code. You describe, you're like, "Hey, I'm looking to find the product category. Can you write me the query to do it?" And it will literally write you the GraphQL query. You can copy and paste that, put it right in your application.

I think we're still in the early days of figuring out how, like it's such a dramatic shift for engineering to like figure out how we apply these LLMs and it's exciting to see these new applications to like existing documentation sites and just unlock them and making it more straightforward to develop apps.

What's up friends? I love my 8sleep. Check them out, 8sleep.com. I've never slept better. And you know I love biohacking. I love sleep science. And this is all about sleep science mixed with AI to keep you at your best while you sleep. This technology is pushing the boundaries of what's possible in our bedrooms. Let me tell you about 8sleep and their cutting edge Pod 4 Ultra. So what exactly is the Pod? Imagine

a high-tech mattress cover that you can easily add to any bed. But this isn't just any cover, it's packed with sensors, heating and cooling elements, and it's all controlled by sophisticated AI algorithms. It's like having a sleep lab, a smart thermostat, and a personal sleep coach all rolled into one single device. And the pod uses a network of sensors to track a wide array of biometrics while you sleep.

It tracks sleep stages, heart rate variability, respiratory rate, temperature, and more. And the really cool part is this. It does all this without you having to wear any devices. The accuracy of this thing rivals what you would get in a professional sleep lab.

Now, let me tell you about my personal favorite thing. Autopilot recap. Every day, My 8 Sleep tells me what my autopilot did for me to help me sleep better at night. Here's what it said last night. Last night, autopilot made adjustments to boost your REM sleep by 62%.

62%. That means that it updated and changed my temperature to cool, to warm, and helped me fine-tune exactly where I wanted to be with precision temperature control to get to that maximum REM sleep. And sleep is the most important function we do every single day. As you can probably tell, I'm a massive fan of My 8 Sleep, and I think you should get one. So go to

8sleep.com slash changelog and right now they have an awesome deal for Black Friday going from November 11th through December 14th. The discount code changelog will get you up to $600 off the Pug 4 Ultra when you bundle it. Again, the code to use is changelog and that's from November 11th through December 14th. Once again, that's 8sleep.com slash changelog. I know you love it. I sleep on this thing every night and I

and I absolutely love it. It's a game changer and it's going to change your game. Once again, 8sleep.com slash changelog.

Going back to something that you had mentioned earlier in the conversation, you had talked about magic and magic enabled and stuff. Could you tell me a little bit about that? I may have misunderstood. Is that a product or a supporting technology that you guys are using?

I mean, it's kind of the way we refer to things. We should have done a better job explaining it. So all the things we just talked about, we consider part of the magic brand at Shopify. So it's like the product taxonomy stuff, the text generation, the sidekick, the image generation in the background. So those are all magic features that Shopify offers. Gotcha. So it's kind of the AI enabling brand thing.

that's around all these things. So as you, you know, we've kind of talked a little bit about Sidekick and we've gone through, there was another thing I always wanted to ask you about and that was how your current array of kind of AI enabled capabilities that we've been talking about

How are you thinking about that going forward? Where are you looking at? How are you, you know, are you going to add any more in there that are announceable yet or maybe at least allutable to? How are you thinking about kind of where you're at today versus kind of some of the things you might be doing in, you know, in the fairly near future and stuff? And we'll get into farther future a little bit later. Okay.

I can't answer today, unfortunately, other announcements that are coming, but I think we could talk about generalities of like what's interesting, right? Fair enough. That Mike talked about of like, you know, applying old techniques and new ways around commerce specific things. Like I think predictions and customization around that are interesting. I'd say like me personally, I think the thing that I get excited about is like the other modalities that are out there.

So going back to that reference from earlier, like chat GPT was cool. I feel like I had a second bout of chat GPT when the voice mode came out. I don't know if you've played with it, Chris, but it's like absolutely incredible. During the typical day, I have an ongoing. I probably this is really terrible that I would say this, but I probably talk to chat GPT more than I talk to my wife. Between us.

Thank goodness she doesn't want to hear me anymore than she already does. So she won't hear this on the show. So but yes, I have an ongoing conversation about a plethora of topics. So which beckons back to the fact that this is moving so fast. And you guys are as you guys are having to kind of match your customer needs with products that support with the technologies that are driving that forward.

What are some of the things that you're thinking about now for maybe as you go forward into the future? And more specifically, how are you thinking about handling the risks associated with changing technology right now? We've talked a little bit about constant experimentation and everything, but there's also a point where you kind of have to make

investments in different directions and trade-offs and stuff like that. Other than the experimentation of that to support as this increasing line of capabilities that you guys are offering, how are you thinking about that risk directions and stuff?

Do you think that commercial product offerings, for instance, one topic that comes up all the time on the show, do you think open source is going to overcome that and kind of take over since things are slowing a little bit on the frontier models in terms of the gains they're making and open source seems to be catching up faster? How are you guys thinking about problems like that as you're dealing with these business issues in your company?

I think it will likely be hybrid. So like, I know we talked to kind of like our strategy is like everything all the time. So I can't imagine a world where the commercial offerings completely take over. And I really, I don't know if I could imagine a world where open source entirely takes over either. And I think that's probably a good thing for the world. Like, I think that's what drives the innovation, right? It's like a competition between the two.

And there are some things that one is good at and the other is not. So like I can't imagine a world where we aren't using both at Shopify. Can you talk a little bit about kind of how you see the strengths and weaknesses, recognizing it may change tomorrow given how fast things are moving. But like when you look at that kind of what we might go with a commercial offering like ChatGPT or one of the other several biggest competitors,

versus the open source and probably the foundation infrastructure that you guys will have. How do you guys know where to go? Like, how do you know to go to chat GPT as an API versus using a foundation model you're storing in your infrastructure?

Well, I think it goes back to Mike's favorite point from earlier of evals. We've got to have our compass because without the compass, we're lost. So that's how we answer which one. But I think the other part of like you're asking, like which one is good at what at this point? And so I think in general, the way to frame it is like open source. The power is in the control. It's like you're guaranteed to run this exact model with this exact set of training data with this exact outcome. So it's very predictable. You have way more control over the training process and like the post-training process and like

you just get a lot more knobs right but also with great power comes great responsibility right like there's a cost to operating all those knobs and knowing what the correct values are for that so for problems that you have pretty fully defined and you know exactly how you want to do it it's like open source is great right the commercial models fewer knobs but um the thing that's great about that is like you know the defaults out of the box usually work pretty well right and so like

I think if you're looking at things that are early on in prototyping, like the commercial models work great, right? They can get you from zero to one real quick. And then when you get to that one, you realize they're like, oh, well, I want to get to 2.0. And like, sometimes it necessitates a need to that shift to an open source model to get that extra control out of it. There's kind of this question of what size of a problem are you trying to solve? If you're trying to solve a, you know, a central do everything, you know, the founder that you would, the co-founder that you wish you had,

we're going to need to pull out all the guns, right? And really put everything we can into making this leveraging as

as much power as we can, right? And the question is just what is the most powerful for this task at this time? There's another side of things where maybe we're trying to solve problems that aren't supersize problems, but they're more manageable problems, right? Or maybe we need to do it at scale. And of course, the commercial models are getting faster and faster and cheaper and cheaper.

But when you need to do something at scale, it might be worthwhile to distill a model from some patterns and then run it at scale. We have billions of products, if you look over our entire history. Doing that at scale for the product that Matt described, where we understand all the different attributes and the taxonomy and we normalize the description of those products, that's a true engineering feat that we need to work on. And that may not be a great idea to send that to

GPT-01, right?

Absolutely. One question, I'm guessing, Mike, this is coming to you. The last two, three years, we've been so focused on LLMs and generative AI capabilities. And I know in general, the industry is starting to kind of also kind of pull back and look at some of the other things, things we used to talk about, other technologies in the AI space we talked about a lot. It seems that some industries, things like reinforcement learning and CNNs and things like that,

Depending on the industry, I think in my own experience as I've talked to different people, some find utility in these other architectures with other purposes and some don't.

How are you guys? Are you really primarily focused on LLMs and generative, or do you have use cases where some of the other technologies that we haven't talked about as much lately but are still very much out there in industry, are they coming into play for you guys? Yeah, so this might be where we peel the technology. I need one more layer, right? At a base level, any neural net is a universal approximator, right? And so if we have enough data

there is a big enough neural net that will solve, just an MLP, right? A fully connected neural network. Sure.

And so the way I tend to think about it, whether it's a CNN or a heavy attention model or an RNN, whatever it is, all that's really in the business of doing is making it so that even though there is a number of neurons, it might be way too many neurons, right? And we might need way too much data in order to do that. And all of these are really just tactics for reducing the amount of data that we need in order to approximate the patterns that we want.

Right now, for sure, heavy attention models, whether it's traditional transformers or evolutions like you might see of the original 2017, attention is all you need transformer architecture to what you see in Lama. These are kind of like tweaks and they're still very multi-head attention focused. There are other techniques like the one I described for e-commerce that are making substantial changes, like removing the soft max out of multi-head attention, which is a, you know,

sort of like having the sigmoid as our activation function 10 years ago. It was just a mistake and a sociological mistake at that. So seeing small changes like that that maybe move us out of transformer architectures, I think that we're definitely in an era where that makes sense.

There's also kind of combinations of things, right? So you mentioned reinforcement learning, there's GNN architectures, and these are actually compatible with vision transformers for planning and reinforcement learning, you know, using transformers for your aggregation functions for a graph neural network.

And so it's not an either or, it's that now we have another tool for either in the former case, modeling the world so that we can do a good job at our Q learning and our policy learning, or do a good job in capturing the right information when we have a graph structure of how we've organized the different kinds of nodes, different kinds of, in our case, merchants and products and buyers.

That being said, there's another way of taking your question, which is, look at when are Transformers going to be done and are they gone already? That was on my mind as well, actually, because everyone's talking about, okay, what does a post-Transformer world look like? So, yes. I think I can say this, and I say this to my students pretty religiously. I'm going to make this claim with full understanding that you should never make a prediction that will be falsified in your lifetime. I'm pretty sure Transformers is not the last architecture out there.

It seemed for a decade that CNN was almost synonymous with vision since 2012, right? And now it's not. And if you would have asked me in the late 20-teens if I thought it was, I'd probably say the same thing.

It's a heavy thing. It's a heavy bet. CNN seemed to be the top of the hill and they seem to do such a good job with image classification. It's hard to imagine what will replace it, but probably it's not the last.

Chapter of the story. And I think that you could probably say the same thing for Transformers. So with the a little bit ironically, and you've sort of kind of covered this territory a little bit with that last answer, Mike, from each of you, we usually finish the show really wanting to get perspectives from from our guests on kind of what the future looks like.

And with each of you addressing kind of different areas, you probably have somewhat different answers based on your focus and stuff. And Mike, recognizing that you've already kind of touched a little bit on the future, but I'm actually, despite your comment about not making predictions that might prove falsified in your lifetime, I'm going to ask you both to kind of do that. If you're looking out, and I'll let you kind of decide on what timeframe you're

works for you. But maybe, you know, beyond the short term waxing poetic a little possibly and trying to say, you know, what would you, what do you think you're going to see? What, what do you want to see and how might your various jobs and how your company serves customers, you know, how do you see this fast moving their twists and turns all along the way that catches all by surprise?

How do you see that playing out from each of you? Matt, if you could lead off and then Mike, I'll come back to you for that. I think what's most exciting about this, I mean, when I grew up, I remember when we first got the internet and it was like the first ISP out there and there was like a dial-up modem and there was like a BBS. Like it was just...

That was like the first wave of technology to me. And like, that's how I got into this field. And then I feel like the mobile revolution caught me by surprise. Like I think at the moment, like I knew when the first iPhone came out, I was like, I need to have one of those. But what I didn't expect was like how much the world would change after that. And it feels like this time around, I wasn't a big believer in Web3. I was like, what is this Web3 business? But like,

I feel like this is again that same kind of shift. So I'm just going to ignore Web3. I think this is the real Web3. It's AI. And so how does this play out? I think what's different this time is that for the last, I don't know, 70 years that we've had computers, we as humans have had to conform to how computers work. At first we wrote assembly code. At first we wrote literally bits. Then we wrote assembly code. Then we're like, well, maybe we should have languages. And it's like, OK, so we're slowly crawling there.

And then the next revolution was like, oh, we should have point and click. And so these boxes. And so now we have a world where everybody spends, you know, eight hours a day clicking on little colored boxes and then typing in to other colored boxes like characters on a keyboard. And I think what's what's fascinating to me is that we've become we've shaped who we are to conform to how computers work today.

But I think this point in time, and I'm sorry, I'm giving it 10 years out for now. No, it's fine. All that's going to change. Like, I think that whole, like all these browsers that we click buttons on to like set settings, like all that is going to go away. I think it's going to be that we interact with like an agent or some amorphous entity that it's like,

And instead of, you know, listing all the steps of like first search for this, then click on this link, then do this thing. It's like, I'd be like, I would like to buy a box of toothpaste. And it's like, the agent's like, great. Do you want to buy one for that ships tomorrow? Or do you want one that's like cheaper, but ships next week? And you're like the cheaper one. And then it's like done, right? You didn't fill out a credit card form. You didn't click through 10 sites. You didn't do any of that. So like, I'm going to put my bet on that. It's like, I just think

the web will change again. Like, I think we've gotten so used to SAS and all these models and like, I'm just, I don't know how long it's going to take. That's why I'm like 10 years out. I don't know. It might be three years out, but it might also be 20, but I don't think we're going to be typing in boxes in 20 years from now. Good answer.

Mike, back to you. Yeah, I think I'm going to flank this from both directions. So I spent a little bit of time in the self-driving space years ago. And this was during the, I think, the maximum hype period for self-driving, where everyone I talked to said, oh, well, people won't even need to drive in five years. And this was more than five years ago. And on

On the one hand, just across the bay, Waymo is giving rides to people, right? Not at scale yet, but it is, right? For sure, a lot of people said my son would never need to get a license and he's

He's about to get his driver's permit, right? So just to draw an analogy, I think it's reasonable to expect what Matt is expecting of having kind of like a self-driving assistant that can do that. I have all the respect for Matt in the world because he was careful about how he's going to do the time, not five years or whatever. There's probably going to be a little bit of difficulty smoothing down the edges for that. And luckily, you know, a crash with...

with a self-driving assistant is all far less dangerous than a self-driving car. So it could be that we get imperfect models there. I'll take an extra five boxes of toothpaste. That seems like I'm careful. So that's one side of flanking it is, you know, we very well will, right? There will be a self-driving moment for these assistants and how long out is completely on the same page with Matt. That's, you know, we're going to hit some bumps before we actually get there.

One thing that I feel very confident of is that we are going to change the way we organize and access and utilize information. This is going to be a forcing function that we really haven't seen since early search days for the internet, which is also a way of just completely transforming how we organize and access information. And, you know, a lot of

people you will talk to will already say that they go to their favorite LLM first before they go to a search experience. And there are also a whole host of product and interface and questions like that about what's going to be the best way of doing this. But there's also, once again, piggybacking on something that Matt said, it is...

incredibly significant that we are now speaking in the same language quite literally when we want to access and refine the information we're looking for. And that's something that's really never happened before. Well said.

Well, gentlemen, thank you very much for coming on the show. It was really interesting. I learned a lot. And thanks for sharing your perspectives going forward. I hope you guys will come back as things evolve and you have more things that you want to share with the audience. Thanks for coming on. I'd be happy to. Thanks, Chris. Thanks, Chris.

All right, that is our show for this week. If you haven't checked out our ChangeLog newsletter, head to changelog.com slash news. There you'll find 29 reasons, yes, 29 reasons why you should subscribe.

I'll tell you reason number 17, you might actually start looking forward to Mondays. Sounds like somebody's got a case of the Mondays. 28 more reasons are waiting for you at changelog.com slash news. Thanks again to our partners at Fly.io, to Breakmaster Cylinder for the beats, and to you for listening. That is all for now, but we'll talk to you again next time.