Anthropic’s Mike Krieger wants to build AI products that are worth the hype
01:23:19 Share

Support for this podcast comes from strike. Strike is a payment and billing platform supporting millions of businesses around the world, including companies like uber, bmw and door dash. Stripe has helped countless startups and establish companies like reach their road targets, make progress on their missions and reach more customers globally.

The platform offers a sweep specialized features and tools to faster track growth like strike billing, which makes IT easy to handle scription based charges, invoicing and all occurring revenue management needs. You can learn how striped helps companies of all sizes to make progress at stripe dot com. That stripe dot com to learn more stripe make progress.

Support for this episode comes from the current report. From data privacy to the future of TV retail media and beyond, the world of digital marketing is constantly in flux, so how can you keep up? Well, the current report is there for you each week.

Marketing leaders on the cutting edge give you the latest insight. So if it's creating a buzz, they'll be talking about IT so described to the current report wherever you get your podcast. Hello and welcome to decoder.

I mean, I petal edit and chief for the virgin decoder is my show, but big ideas. Another problems today i'm talking with mike krieger, the new chief product officer at anthropic, one of the hottest AI companies in the entire industry. Anthropic was started in twenty twenty one by former OpenAI executives and researchers.

We wanted to build a more safety minded A I company, which I have to point out is a real theme among people who leave open a eye. Think about anthropic main product right now is clod, which is the name of both its industry leading A I model and a check bot that competes with ChatGPT. Like other major AI companies, anthropic has billions and funding from some of the biggest names in tech, primarily amazon, but at the same time, anthropic does have a distinct in intense safety culture.

The company is notable from points of people who legitimately worry that AI might destroy mankind in I wanted to ask mike how that tension plays out in product design. On top of that, mike is a pretty fascinating history. If you're a long time tech van, you likely know him as the cofounder of instagram, a company he started with Kevin system before selling in to facebook.

Now ma, or a billion dollars back in twenty twelve that was an eye popping about the money back. And in the deal, turned rike into founder royalty. Basically overnight, my left meet in two thousand eighteen, and a few years later, he started to double in AI, but not quite the type of A I we talk about all the time.

And coder. Instead, mike and Kevin launched art effect and A I powered news reader that did some very interesting things with recommendation algorithms. And I was a big fan of artifact, but ultimately, I didn't take off like anyone wanted in Michael.

Kevin shut IT down. Earlier this year, we sold the underlying tech to yahoo. We talk a lot about decisions here, any quoter. So I wanted to know more about the decision to shut art fact down and then the decision to sell at to ahoo. And then, of course, I wanted to know why I decided to join anthropic and working A I in industry with a lot of investment, but very few consumer products to justify IT.

Really, what is all of this for what products this might see in the future that make all of the turmoil around A I worthy? How was he thinking about building them? I've always enjoy talking product of the market, this conversation different, even if i'm still not really sure anyone described. But the futures gonna look like OK anthropic, chief product officer. Might creek where go?

My cragg, you are the new chief product officer at anthropic will constitute or thank .

you so much is going to be to see you yeah .

I I talk you about about products. The last time I talk you just trying to convince you to come to the code conference, I didn't actually get interview code. But I think you I want to talk about products for someone is supposed to regulation.

And yes, here's my product, I wren in the audience. We are deferring to talk a little bit about A I regulation. It's GTA happen.

IT seems part of the puzzle, but you're building nal products. And I have a lot of questions about what those products could be, what the products are now, where they're going. But I want to start serve at the beginning of your interros c story, which is also the end of your artifact story.

Ah so people know you you you started instagram your meter for a while, you left meta and then you and Kevin system started artifacts, which was a really fun news reader. And that's some really interesting ideas about how to surface the web and have comments and all that. And then you decided to shut IT down. I think if the show is a show for for builders and weren't often talk about shutting things down, walk me through that because it's as important as starting things up.

Sometimes that really is. And the feedback we've gotten post shut down for art effect was a mixture of sadness but also kudos for calling IT. When you saw IT, I think that there's value to also having a moment we say we've seen enough here.

Think for us, IT was the product I still love and miss and in fact, like I will run into people like I expect them is like I love instagram that like I love in topic, like art effect that really made art fact had a resident, but they too small, but very passionate group of of folks. But we've been working on IT in the of full run a bit about three years, and the product have been up for a year. And we're looking at the metro, looking at growth, looking at what we had done and kind had a moment.

We said, are there ideas or of product directions that we'll feel dumb not having tried before calling IT? And we had a list of those and that was kind of mid last year, and we basically took the rest the year to work through those got to the air and said, yeah, those moved the middle little bit, but not enough to convince us that this was really on track to be something that the team and we are collected to be going to spend a lot of time on over than the the coming years. Now was the right moment to say pauses to step back. This is the right time to shut down. And the answer was yes. Actually, if you haven't seen the yahoo, yahoo basically bought IT, took all the code and reading yahoo news as artifact or they are the way around and it's it's very funny like you have a little bit IT like a bizarre world bum at the first almost exactly like got a fact little bit more purple some different sources but yeah but I was definite the right decision and like, you know, it's a good decision when you step back and the thing you have regret is like IT didn't work out, not that you had to make that decision or that you made that exact decision of the time that you did.

There are two things about artifact I want to ask about. I definitely to ask about what it's like to sell something you who trained to me for which unusual it's not to think this thing happened what um the first is that artifact was very much designed the surface web pages. I was predicated on a very rich web.

If there's one thing i'm worried about in the age of AI is that the web is getting less rich, right? More and more things are moving to close, platforms, warm, more creators, they want to start something new. They end up on a youtube where a tiktok, I know that my dedicated threats caters yet, but there they're common.

And IT seems like that product was chasing a dream that might be under pressure from A I specifically, but also just like the rise of creator platforms from broadly IT. Was that a real problem? As I just something I .

sort the answer I agree with, with the assessment would be different root causes, I think. But we saw some sites were able to baLance kind of a mix of subscription. You tasteful ads, good content, I would put the verge at the top of that list.

And not just saying that i'm talking you. I could legally like every time we like to a verge story artifact, somebody click those like this is a good experience that feels like things are in baLance at the extreme though, like local news. A lot of those websites, for economic reasons, that becomes sort of like you arrive there, is a silent with google before you've even read single thing.

A pop up to sign up to the news letter is like before you've even consumed any content. I think that's like probably a longer run economic question of supporting local news, probably more so than A I at least like that trend seems like it's been happening for for quite a bit. The creator piece is also really interesting where you if you look at we are things that are breaking news are at least like emerging stories are happening.

They are often it's an expose that went viral. And what we'd often get an artifact is the summary round up of the reactions to the thing that happened yesterday, which if you're relying on that, you're a little bit out of the loop already. And so I think when I look at the where are things are happening and where the conversation is happening, at least for the the kind of cultural kind of core piece of that conversation, it's often not happening anymore on media process that is starting somewhere else, then getting aggregated elsewhere.

And I think that just has a implication on a site or product like artifact and how well you're ever going to feel like this is breaking news over time. We move more to let's be more interest based to funny off instagram. It's hard loss of their interest based less breaking news. But can you have a product that is just that?

I think that was the struggle. You said media properties, but I know some media properties of apps and some are expressed only as news letters. But I think what i'm asking about is the web.

This is just an medoc therapy about the web. But what i'm worried about is the web, right? The creators aren't on the web.

We're not making websites. And artifact was predicated on there being a rich web. Search products in general are predicated on there are being a rich and search able web that will deliver good answers. To some extent, A I products require there to be a new web because that's where train our models. Did you see that, that OK this is this promise of the web is kind of under pressure. Know if all the new stuff is breaking on a close platform, can't search like a tiktok or next or something else so that you can't index surfacing old tweet is not really great user experience. Actually, building products in the web might be getting more constrained to not .

a good idea anymore yeah even this letter is a great example where some of the best stuff that I read is sometimes there is a equivalent sub stack site that you could go look at some of the these letter exit purely an email, even set up an email account that just injust a newsletters to try to surface them, at least links from them. And that was the experience is not there.

I'd say that the the thing I noticed on the open web in general, and like as a long time fan of the web, was somebody was very online before. Being online was like a thing that people who are back in brazil like as as a know pretty in a lot of ways the incentives that have been set up around like, well, you know, a recipe won't rank highly if it's just the recipe. Let's tell the story about the life that happened up to leading to that recipe like those trends I feel like have been happening for a while and already leading to a place where the end consumer might be a user.

But IT is being intermediate already through you know a search engine and optimized for that fine tablet or optimized for or what's gona get shared, a bunch of what's going to get you know the most attention and like that flow. I mean, news letters are in pod caster two ways that have probably most successively broken through that. I think that's been an interesting direction.

But in general, I feel like there's been probably a decade long sort of at risk for the open web in general in terms of like what is the actual inner mediation that's happening between and like I am trying to tell a story, i'm trying to talk to somebody and somebody is like receiving that story. And like all the roadworks around the way and they just make that more and more painful. And it's no surprise then that, hey, I can actually just opened my email and I get the content that feels Better in some ways, although also not great in a french in other ways. That's how I i've watched IT and I would be call IT not a not a healthy place for IT is now yeah the way that .

we talk about that this is on decoder most often is that people build media products for the distribution. And so pocket famously have open distribution of service feed. Well, it's like an access feed, but there's like spotify server in middle and start everybody who gets whatever ads so we ve put in here.

But it's still like IT is core in our service product news letter, still its core. And I map product at an open mall protocol products. The web is like search distribution to have optimized to one thing.

The reason I ask again, I want i'm going to come back to the team of a few times that I felt like artifact was trying to build a new kind of distribution. But the product IT was trying to distribute was one pages which were already overtly optimized for something else. I think that's .

a really interesting assessment. It's actually funny watching the yahoo version of IT because they've done the content deals to get the like more slim down pages. And do they have fewer content or fewer content sources? The experience of tapping any genital story, I think, is a lot Better because like those have been now formated for a distribution that is, you know I guess, linked to some paid acquisition, which is different than what we were doing, which is like here's the open and web, both give you awards and all and directly to you. But I think your assessment feels right.

So that's one I want come back that team I really wanted, start with artifacts. In that way, because IT IT feels like you had an experience in one version of the interview that is maybe under pressure. Other thing I wanted ask what artifacts you and Kevin, your cofounder, both once told me that you had like big ideas, like scale ideas for artifact, and they had just like big idea. And you wouldn't tell me what I was.

It's over now, what was IT? But as I was, I mean two things that I remains like sad that we didn't get to see through. One was the idea of good recommended systems underlying multiple product vertical, so news stories being one of them.

But gona believe I had the belief that maybe somebody will build out that if you understand yourself well through this isn't going to understand you well through how you're interacting with new stories, how you're interacting with content. Then is there another vertical that could be interesting? Is IT around like shopping? Is IT around like local discovery? Is IT around people discovery all these different places.

Because for all the promise now separate, maybe machine learning in a eye. And I realized that's a shifting kind of definition throughout years, let's call like for the purpose of our conversation, like recommended system, machine learning systems for all their promise my day to day is actually not filled with too many good instances of that product. So the big competing idea was, can we bring instagram type product thinking to recommended systems and combine those two things in a way that that creates new experiences that are beholding into your existing random and photograph.

With news being an interesting place to start, you highlight some good problems about the content, but the appealing part was, were not trying to solve the two side of marketplace. All the ones. Turns out half that marketplace was already search filled and had his song problems.

But at least there was. There was the other side as well. The other piece, like even within news, is really thinking about how do you eventually open the sub.

I think sub stack is pursuing this from a very different direction, but open the sub so craters can actually be writing content and understanding distribution natively on the platform. I feel like every platform eventually wants to get to this as well. When you watch the user closest analogue in china, like toys ao, they started very much like crawl the web, have these, you know, eventual published your deals.

And now IT is like, I would guess, eighty and ninety percent first party content. There's economic reasons why that's nice. And some people make their living writing articles about local new stories on tota, including whatever engineers, I think, the sister, close family member. But the other side of IT is that contact could just be so much more optimized for what you're doing actually act code.

I met an entrepreneur's creating sort of new novel media experience that was like very like if stories met news, met mobile, what would that be for most new stories? And I think first and they like that to succeeded, IT also needs distribution that has that as like the native uh distribution type. So recommendation systems for everything and then a like primarily recommendation based first party counter writing platforms like the two ideas, oh my god, for somebody.

But at last artifical question, you shut IT down and then there was a way of interest. And then yeah, I think publicly one of you said, oh, there's a very interest we might flip IT and then I was yahoo, tell me about that process.

I think there are a few things that we wanted to align. I think we'd worked in that space for long enough that whatever we did, we sort of wanted to kind of tie around IT and move on to whatever I was next. And I was one piece.

Another piece was like, i'd wanted to see the ideas live on in some way. So like there was kind of a lot of conversations around like, well, like what would have become under like different conversations. And the yahoo one was really interesting and I would admit to be like pretty unaware of what they were doing beyond like I was still using I who finance and like my fantasy y footballing.

But beyond that I was like, not famous. What they were doing really like, no, we wanted take IT and we think in two months we can relate ch. IT as yahoo news. Thank you.

Like that sounds pretty crazy like that's a very short time line for like and I could based on not familiar way, and they had our access to us and like we're basically like helping them out almost full time, but that still a lot and they actually basically pulled up. There was ten weeks instead of eight weeks. But I think there is like a new fund energy in there to be like, right? Like what are the properties we want to build back up again and do IT.

So I fully admit coming in with a bit of advice, like I don't know, like what's left to ahoo, like what's going to happen here and then the teat, the tech team like like bit into IT with with the open mouth, it's kind of growth metaphor. But they like went and all in then they got had shifted. And Oliver team, the text of andy, he's that anthropic now he actually came here before I did and i'll like, find little deos now he was like, they kept.

I spent a lot of time with the like three spinning animation when you're got to a new reading, well, like to wait too much on the like, beautiful reflection specular highlighting thing. Probably this priority is that week, but they kept IT. But now IT goes, yeah, when you do IT not like that, that's pretty, pretty on brand.

There was a really viny experience, but now I still live on, and I think IT probably a very different future than what we were in visioning. But I think some of the some of the core ideas are there, unlike k like what would you mean to actually try create a personalized news system? That was not that was really the coupled from any kind of existing photographer. What you were seeing already understand facebook.

but they the best bitter was the decision. Yahoo will deploy this to the most people at scale was that they're offering us the most money. How did you choose there?

Was this like optimization function? And outside the three variables were like like deal attractive or attractive enough. Our personal commitments post you know, transition were were pretty blight, which I liked. And they had reached like goo ws because like a hundred million months, they still was like reach final commitment, but enough that we felt like I could be successful and then like they were like in the right space, at least on the bid size.

This sounds like a dream and you just have that someone to walk away. It's much money. okay. Yeah, makes sense. I was just wondering if I was there or whether was like IT wasn't as much money, but they had the bigger platform. And because yahoo is deceptively still huge.

yeah exactly. So huge, I think, under new leadership and and with a lot of excitement there. And no, for me, I was IT really changed that. I was not like a huge exit or like I would not a super success, what outcome. But the fact that I feel like that chapter closed in a nice way and then we could like move on without like wondering if we should have done something different when we closed IT. Like just I thought much Better at the q one of this year because of IT.

We need to take a quit break .

when you're a pack.

Think scaling AI is hard. Think again with watts and x, you can deploy A I across any environment above the clouds, helping pilots navigate flights and on lots of clouds, helping employees automate tasks on prem, so designers can access proprietary data and on the edge, so remote bank tellers can assist customers. What's the next works anywhere?

So you can scale .

AI everywhere? Learn more IBM dot. Come slash. What's the next IBM? Let's create support .

for the show comes from the refinery. A domino location and atmosphere are key when deciding on a home for your business, and the refinery can be at home if you're a business leader specifically won in new york. The refinery, a domino is an opportunity to claim a defining part of the new york city skyline.

The refinery domino is located in Williams s. Burberry, lin and IT offers all the perks, animated ties of a brand new building while being a landmark address. The dates back to the mid nineties century.

It's fifteen floors of classes, modern office environment, house within the original urban artifact making, get a unique experience for inhabitants as well as the wider community. The building is outfit with immersive interior gardens, a glaston ed to paint house lounge and a world class of that space. The building is also home to a state of the r equal x with a pool and small world round restaurants and exceptional retail.

As new yorkers return to the office, the refinery, a domino, can be more than a place to work. IT can be the magnetic hub fit to inspire your team's best ideas. Visit the refinery, that and I, C for a tour.

This message is a paid partnership with apple pay when you've got a gift list to finish, the last thing you want to do is take .

out your wall at a million times. Instead pay the apple way with apple pay. You can pay with the phone you're already holding.

just double clink, smile at face I D tap and your done. The people in line behind you will. Thank you. Apple pay is a service provided by apple payment services lc, a subsidy area of apple ink. Any card used in apple pay is offered by the car red issue.

We're back with anthropic SHE product officer .

might trigger so that's .

that chapter. The next chapter is you show as the cheap, proud officer anthropic. What is that conversation like? Because in terms of big commitments, Harry problems, are we gone to destroy the what it's all right there. Maybe it's a lot more work. How do you make decision to go on thropp.

but that at the top of the decision was what to do next at all. I admit to having a bit of like an identity crisis of the in the year of I only really know how to start companies and actually more specifically, poly, not to start companies of the cover, like make a very good cover on your firm. And I was looking, what are the aspects of that, that I like? I like knowing the team from day one.

I like having a lot of autonomy. I like having partners that I really trust. I like working on big problems with a lot of like open space. And at the same time, like I do not want to start another company right now, like just one of the ringer on that three years in OK outcome IT wasn't the outcome we wanted that are going like I want to work on interesting problems at scale at a company that I started.

But I don't want to start a company is like I kind of like through a bit, not like what do I do next? I definitely knew I do not want to just invest. Not that investing is a just thing, but it's just different. I'm like a builder at heart, as you all know. And so I was like, this is going to be really hard.

And even when I take some time and then start a company, and then I got introduced to the entropy c folks via that had a design here, who is somebody I actually built my very, very first iphone up with in college, about one for a long time. This is joe. I started talking to them and I realized that the research team here is incredible, but the product efforts were so nicer that i'm not going to kid myself.

I'm coming as a code like becomes around for a couple of years, like already like company values and way things are working and they call themselves ants. Maybe would have educated for a different employee nickname, but it's fine. That ship has sailed.

But I felt like there was a lot of product Green field here and a lot of things to be done and belt. So there was that IT was the close, this combination I could have imagined to the team I would have wanted to have build had i've been starting a company enough to do like so much to do that. I like, like I wake up every day both excited and daunted by how much there is to do.

And like already momentum and scales, I could feel like I was gna hit the ground running on something that had a bit of tail wins. I feel like a lot of art, of fact, were like headwinds somewhat outside of our control. And that was the the sort of combination. So the first was, like, big decision.

Like, what do I do next? And then the second one was, like, art is on topic. The right place for IT was sort of thing where every single conversation I had of them might go back to like, I think this could be IT. Like I wasn't thinking about turning your company that's like already writing like crazy, but like, I want to be close to the like cory I tech, I want to be work, interesting problems I want to be building, but I wanted to feel like as close as to a cofounder kind of situation as I could. And I think, Daniel, I was the present here, maybe was trying to sell me, which she's like, you feel like the eight cofounder that we never had that is like our product cofounder, which is amazing, that theyve seven code der and none of them are like the product finder. But whatever IT was IT sold me and like i'm i'm going to jump back in.

I'm excited for the inevitable beatles documentaries about how you're the the fifteen .

and then we can argue for, I hope, is twenty .

four with our audiences. That might be a deep cut, but I encourage everybody to go search people so much of an argument that is, let me ask you, just two big picture questions are working in. I generally you started instagram.

You're deep with creatives. You build a platform, creatives. You care about design, obviously, with that community.

A I is a world to lima. Like people are upset about that. I am sure they'll be upset that I even talk to you. We have the sea over to be on to the firefly, and that is some of the most emails we've ever gotten.

How did you evaluate that and when to go work in this technology is built on training and all the stuff on the internet, and people have really, really hot emotions about that. There's a we're going to type with the losses. There's losses, there's copy losses. Ge.

how are you thinking about that? I have so much good friends, is a musician you down that and like now he comes up to the bar and I was on two and will have like one hour deep conversation over, push us over, like, what is, you know, I and music and how do these things connect and where do these things go? And nothing goes as interesting inside.

And like what parts, the creative process or which pieces of creative output are most affected right now. And then you can kind apply them, see how that's gonna change. I think that question is a big part about why ended up an anthropic. If I was going to be N A.

I think a couple of things like obviously, the written word is really important and like there's so much that happens in taxi, definite do not mean is that make the sound like texas less creative than others? But the fact that they've chosen an Augusty, we've chosen to really focus on text and image understanding and like keep IT to text out and text out that is like supposed to be no something is like tailor to you rather than we're producing something that already out there, I think reduces some of that space significantly where you're not also trying to produce like hollywood type videos or you know hyper delity images or sounds and and music. And some of that, I think is a research focus, some of that a product focus. But I think that also the space of thorny questions is still there, but also a bit more limited in those domains or outside of those domains and more purely on text and code in those kinds of expression. So that was a strong sort of contributed to me, wanted to be here versus other spots.

There is so much controversy. But where the training data comes from, where does anthropos training data for cloud come from, is scraping the web.

everybody else scraping the web. We respect for robots that text a few other data sources that we license and and work with with folks kind of separately for that. Let's say the majority of IT is web craw done in a web crawl. Respect way we .

respecting robot, start tacks before everyone realized we had to start respecting robots.

that that we were respecting robot start tax beforehand. And then in the cases were, for whatever reason, IT wasn't getting vicked up correctly. We've since corrected that as well.

What about youtube, instagram?

yes. Yeah no. That's you know I think about the the players in this space there times where I I must be nice be in inside. I don't actually know if they try and if they are talking about that, but there's a lot of good stuff and they're in same with youtube. I mean, like close reminds that youtube that's like a like the repository of collective knowledge of how to fix any dish washers in the world and like people ask like kind of do so i'll see every time what what does end up .

looking like you don't have a fair key to the meta data center services did on the way out. When you think about that general dynamic, there's a lot of creatures out there who perceive a to be a risk for jobs or perceive that there has been A A big taking. Just ask, what was that? There is a lost you against and tragic it's bunch authors he said that the model that claude is illegally trained against their their books.

Do you think there's a product answer to this? This is kind of it's leading to my second question. But i'll just have broadly, do you think you can make a product so good that people overcome these objections? Because that is kind of the vague argument I hear from the industry right, right now.

We're seeing a bunch of chap bot, and you can make the chap bot fire off a bunch of corporate information, but there's going to come a turn when that goes away because the product will be so good and so useful that people will think IT has been worthy. And I don't see that yet. I think that's A A art.

A lot of the heart of the the copyright last suits beyond just the the legal pieces of IT is that the tools are not so useful. Anyone can see that the trade is worth IT. Do you think .

there's is going .

to be a product IT is obvious that the trade is worth IT.

I think it's very use case dependent and like what but the the kind of question that I withdraw our instagram team insane with as we would always ask them like what what problem are you solving and general like text box interface that can answer any question is like a technology and like the beginnings of product, but is not like a precise problem that you are solving.

And I think grounding yourself in that maybe helps you get to that answer, which is like if what you are trying to build, for example, like I use cloud all the time for, like good assistance like that, is solving a direct problem, which is i'm trying to like rap up on product management here and like get our products like underway and like also like work kind of much different things. And to the extent that that i've anytime to be in like pure build mode, I want to be really efficient on IT, like that very directly connected problem and like total game changer. Just see myself as a builder and like gloves me.

Focus on different pieces as well to somebody right before this call as well, like they are now using cloth a bunch to suffer up or otherwise change their like long mysids on slack before they send them. And so like this, like human editor, kind of peace like that solves there. Like kind of immediate problem maybe I told about.

And like, take a while for a, but like I think again, grounding IT in use because I don't try to really focus on our products here. It's like if you try to boil the ocean, I think you went up in and actually really a jazz into these kinds of like most ethical questions that you raise, which is like. You're in anything box then like everything is like you know potentially either under threat or like a Jason or or problematic. I think there's a real value to saying, all right, like what are the things we want to be known to be good for? And like how are you today that the product actually does serve some of those like well enough that it's like i'm happy IT exists and I think folks are in general. And then I think over time, if you look at things like writing assistance more broadly for like novel lengths writing, I think the jury still out on that my wife was doing of a part type version that I talked other folks like, no, I think the our bottles are quite good, but they're like not great at keeping track of characters over like book length sort of pieces or you reproducing particle I think so yeah, I would ground that and like what I can be good at now and then like blood as we move into new use cases now get those carefully in terms of like who is actually using IT on our providing value to the right folks in that exchange.

Let me ground that question and more specific example, both in order to ask you a more special question and also to calm the people who are already draft to me angry emails. Tiktok exists. Tiktok is maybe the purest garden of innovative copyright infringement ment that the world has ever created.

But I ve washed entire movies on tiktok, and it's just because people have found ways to to bypass their content filters. I I do not perceive the same outrage at tiktok for copy infringement as I do what I may be. Maybe there's someone's really mad.

I I watched entire like nineteen eighties episodes of this hold house on tiktok accounts that are literally label like best of all, this old house. I don't think by villa is getting real ties for that, but IT seems to be fine. We as tiktok as a whole has so much utility and people perceive even the utility of watching like all maintain is episodes of the whole house.

And there's something about that dynamic between this platform is going to be loaded full of other people's work and we're going to value of IT. IT seems to be rid in the fact that most am looking the actual work. I'm not looking at some fifteen and the derivative of this old house is expressed by A I chap pot and actually just looking at a one thousand nine hundred eighty version of this whole house. Do you think that A I chap pots can ever get to a place what IT feels like that where i'm actually just looking at the work or i'm providing my attention or time or money to the actual person who made the underlying work as opposed to we're trained IT on the open internet and now we're charging twenty books and the fifteen steps back person is nothing yeah .

I think to granted the tiktok example as well, I think there's also an asset where know if you imagine the future of tiktok. Probably most people say, well, like, well, maybe a lot of more features and i'll .

use that even more or not even .

know what the average transparent and like build agi, create universal prosperity so we can spend time and take would would not be my preferred future to construct that if you wanted to. But I think the future feels, I would argue, a bit more knowable in the tiktok use case.

And I think in the in the AI use case is a bit more like, well, where does this accelerate, you know, to? And where does this eventually compliment me? Where does IT supercede me? And I think I that a lot of the sort of A I related anxiety can be tied to that part of like the fact that like three or four years ago, this technology was gradually, three, four years ago, like tiktok existed and IT was already you kind of on that ta ductor.

And even if IT wasn't there, you could kind of have imagined IT from where youtube an instagram war. And if they had like an interesting baby with vine, like IT might have credit tiktok. So I think IT IT is partially because the platform is so entertaining.

I think that's peace. I think that like connection to real people is an interesting one. And like i'd love to like spend more time on that is an interesting and kind of piece of of the A I ecosystem. And piece is just like the no ability of where IT goes, are like probably the three that IT grounds IT.

More topic started IT was for the original world quitting open an eye to build a safer AI company. Now there's a lot of them. My friend casey, he makes a joke at every week someone quits to start to say yet another safer AI company.

Levi e has a great like guy. It's a universal sorting function where we just had to steal IT. On other side is that is that .

expressed in the company, but is that we obviously stated m had big moderation policies, thought about IT a lot. IT is not perfect as a platform, a company, but is certainly at the core of the platform. Is that at the core of anthropic in the same way that there things you will .

not do yeah deeply and I thought in a week too so i'm like a i'm a ship oriented person, you know like even with instagram, like early days, I was like, let's like a bog down and like building the fifty features is but the two things well and get IT out as soon as possible. Some of those like decisions to ship a week earlier and you know not have every future. I think we're actually exist, title the company.

So like I feel that in my bones. So week two hours here, our research team put out paper on interpretations of our models. And buried in the paper was this idea that they found a feature inside of the models that, if amplified, IT would make cloud believe.

That was the golden gate bridge. Not just like kind of believe I like prompted, hey, you're the gold gay bridge, but like, deeply like in a way that my five year old will make everything about turtles. Like I made everything about the golden gate bridge I care you today.

Like, i'm feeling great. I am feeling international orange. And like, feeling in the foggy close. The emphasis, scope and somebody in in our sack was like, hey, should we like build in like released golden gate club is almost like an often comment and a few of us like, absolutely yes. Like, I think it's like let's lighting people for two reasons.

One, this is actually quite fun, but to like getting people actually have some first hand contact with what a model that has had some of its parameters tuned move outwork was valuable. So from that irc message to having golden gate caught out on the website was, I think, like basically twenty four hours. And in that time, we had to do some product engineering the model work, but we also ran through like a whole battery of safety evs.

And I think that was like just an interesting piece where you can move quickly, and not every time can you do IT only a twenty four hour safety. Others like like there ones for a new model, this one was a derivation, so is easier. But the fact that, that wasn't even a question, like what should we run and save you else?

Like, no, absolutely. Like that's what we do before we launch model as we make sure that it's both safe from the things that we know about. And let's also model out what are some novel harm. So the bridges is unfortunate associated with suicides. Like let's make sure that the model doesn't guide people in that direction and if he does not put in the right safeguards.

So that's kind of a like trivial example because is like an easter average shift for basically two days, then one down, but was like very much at its core there even as we prepare model launches. Again, I have the urgency like let's get IT out, like the honest people use IT and then you're like actually do the time I like well, in the point where the model is ready to the point it's released, like there are things that we are gone to want to do to like make sure that we are in line with our responsible scaling policy. And it's what I appreciate about what the product on the research team here is that not seen as and like standing and I really I get that that's why this company exists.

Um I don't know if I should share this, but I shit anyway. Like in our second all hand that I was here, somebody was very early here, stood up and was like if we succeeded at our mission but the company failed, I would see this as an outcome, as a good outcome. And like I don't think you would hear that you thought they would not hear that at instagram, not because we were bad people, but those like not in the like, you know, free, like succeeded and helping people see the world and more beautiful visual way, but like everybody fail out would be super bomb.

And like I think a lot of people here will be very bomb too. But like that ethos is like quite unique. I think this brings .

me to the decoder questions. And so because because what's call the public benefit corporation, you've there's a trust underlying IT. You are the first had a production.

You've described the product and research teams as being different than there is a safety culture. How does that all work? Has anthropic structured broadly.

We have our research teams. We have the team that is most closely in research and product, which is a team thinking about inference and model, delivering everything that takes to actually serve these models because that ends up being the most complex part in a lot of cases. And then we have product, I would say, if you like, just sliced off the product.

Team IT would look similar to product teams that most tech companies with a couple of two weeks. One is we have a labs team, and the purpose of that team is to easily stick them in as early in the research process possible, with designers and engineers to start prototyping at the source, rather than wait until research is done. I think they can go into why I think there's it's a good idea that's a the team that got spent up right after I joined.

And then the other team we have is our research pm teams because ultimately, we're delivering the models using these different services and the models hall capabilities like what they can see well, you know, terms of multi model or what type of text they understand, even understanding what languages are they need to be good at having and user feedback thai and all the way back to research and are being very important. And ah IT prevents IT from ever becoming this, like almost like I retire, like we built this model ends like is IT actually useful? Like we say, good at code.

Are we really how our startups that are using IT for code giving a feedback on. Good IT like these Price on use cases. It's not good at this tonic ous thing great like that feeds that's going to like channel right back. So those are they like to uh distinct pieces, not so like within product, you know and I guess I click down now you get really interested on coder and like team structures. We have apps which lad I cloud for work, and we have developers, which is the API, and then we have our cookie labs team and .

that all just that the product side, the research that is that the size that works on the actual models .

yeah is set on the actual models. And you know everything from like researching like model architectures, figuring out how these model scale and then a strong like red teaming safety alignment team as well. That's like a know another component that is like deeply in research. And I think some of the best research of gravitation ating towards that, as they say like that, like the most important.

they think they could work on how big .

is and try people where north of seven hundred at last count and .

what's display between that research or function and the .

product function product is really, I think probably one say double but almost doubled product is just north of a hundred. So the rest is everything between we have sales as well. But research um the findings, part of research inference and then the safety and scaling pieces as well.

So we are I described this like within a mount of joining is like like those crabs that have like one super bid claw like we're really big on researchers and my product. This is like very small class still or the other metaphor, zing is like, you know, your teenager, like some of your limbs have grown faster than others and some are still catching up. The kind of like crazy or bet is I would love for us to not have to then like double the product team.

I'd love for us instead to find ways of using claud to make us bore effective at everything we do on products that we don't have to double because, you know, every team struggles with this. So this is not a novel observation. But I looked back at instagram when I love five and engineers, were we more productive than two fifty? Almost certainly not.

Will be more productive than that, like one twenty five to two fifty, marginally. You know, I had a really depressing interview once, and I trying to hire A V P of v like, how do you think about developing efficiency and like team growth? And like if every single person I hire is at least like net contributing, like something that I succeeding, even if it's like sub number, like a one to one ratio before, it's depressing.

And like just like I think you crazy all the other like swallow around like just team cultural dilution at sea. That's like something i'm personally fashion. Like how do we take what we know about how these models work can actually make IT so the team can stay smaller .

um and more technically. Yeah tony, who did the ipod he's been on to before when starting the verge, he was basically like, look, you're going to go from the action numbers were he says like you going to go from like fifteen and twenty people that fifty or one hundred and then nothing we will never be the same. And I thought about everyday things because we're always right in the DDL that range.

And like when is the tipping point? Where does moderation live in the structure? You mentioned safety on the model side, but you're out in the market building products.

You've got what sounds like a very horning golden great bridge. People can talk to where in your running test there. Sorry, that's just my every every conversation has one joke.

Oh, honey, the models are where does moderation live, right? Instagram, there's the big centralized meta trust and safety function at youtube. It's in the product org under neutral on there. Where does IT live for you?

I would probably put IT in three places. One is in the actual model training and find tuning, where part of what we do on the reinforcement learning side is saying we'd like to find a constitution for, like how we think club should be in the world. And that gets backed into the model itself, like early, like before IT, you know, before you hit the system prom, before people interacting with IT, that's getting encoded into that run.

Like, how should you know, I should behave, but should I be, you know, willing to answer and and be willing to chain on where should IT not be? And that, I think, very linked to the the responsible scaling peace. The next is in the actual system prompt.

So we actually like, in the in the spirit transparency just started publishing our system prompt people would always figure out like clever ways to try to reverse the anyway and like that's gonna en. Why don't we just actually treat IT like a change log? So just be transparent.

So a the things last week, you can go online and you actually see where what we've changed that another place where there's an additional kind of guidance that we get to the model on how to act course, like I expected in earlier, people can always find ways to try to get around that. But with fairly get up preventing outbreaks. And then the last piece is where our trust and safety team sets and that, you know, the trust and safety team is a closest team.

I would at instagram, we called IT at one point, trust and safety and other point while being but the same kind of like last mile remediation piece. And I would kind of bucket that work into two pieces as one is, what are people doing with claude and publishing out to the world? So with artifacts was like the first product.

We had any amount of social thing at all, which is like you create article fact, each share actually put that on the web and that's like a very carbon problem and kind of like shared content. Like I lived shirt content for almost ten years of instagram and here was like, wait, like people have used their names, like how do how do they get reported and he was like that up that we delayed that launch for. Like, but we can have to make sure you have the right just T N S.

Trust and safety pieces around moderation reporting cues, around taking IT down, like limited distribution, figure out how what that means for the people on teams, plans for these individuals. Like so those things I got very excited, like let's ship this, like sharing artifacts, like we clear, like OK now we can ship, but we got like actually sort these things out. Sets on the like content moderation side, I would say.

And then on the on the response as well, we also have additional pieces that sit there that are either around preventing the model from reproducing copyright content and like that something that we want to prevent as well from from the completion and then other harms that are, you know against the way we think the model should behave and should I have been cut earlier. But you know, if they aren't, then they can get caught at that last smile. So it's like our head of trust and safety is talking to him last week cause you the swiss cheese method, which is like no one layer will catch everything but ideally enough layer stack will like catch a lot of IT before .

he reaches the end ah I think you know i'm very worried about A I generated fakery across the internet this morning I was looking like a denver post article about a fake news story about a murder that the people are calling the denver post to find out why they hadn't reported on IT. Which is in its own way, the correct outcome, right? Like they heard a fake, or and they called the trust its source.

At the same time, the denver's post had to go run down this like fake murder, true crime story, because an A I had just generated IT and put in on on youtube. That and all seems very dangerous to me. There's the death of the photograph.

We've talked about IT all the time. We're gona believe what we see anymore. Where do you sit on that? What is what does anthropic obvious safety? But we are so generating content that can go hayward kinds ways .

yeah I I would like maybe like sport in to and drop up. And just like what I was just seen out in the world, like the rock image generation stuff that came out like two weeks ago, was fascinating because it's almost like know because I think there was a maybe they have introduced some, but at launch I felt there was almost do as a total free for, like, do you want to see koala with a machine gun? IT was just that good.

Know, crazy stuff. I go between believing that, like, actually having examples like that in the are actually helpful and almost like innocent late like what you take for granted as a photograph or not, you know or even or not, don't think we're farther from that as well and like getting know maybe it's calling the denvers poster like a trusted source or maybe it's like creating some hierarchy of trust that we can go after. I don't do know easy answers there as well, but like that, I would say like a industry embers, like a gradual society.

Why I think that we're going to reckon with as well, like the mage image and video pieces. And then on text, I think like what changes with the eyes is like the the mass production. So one thing that we look at is any any type of coordinated effort.

We looked at this as well. D instagram, like individual levels IT might be hard to catch the one person that's like commenting on a, you know, facebook group trying to start some stuff because that probably intestate from a human. But we really looked for we like networks of coordinated activity.

And we started, not started. We've been doing the same as well on the anthropic side, which is looking and this is going to happen to more thing on the A P I side if IT happens rather than on cloud, that adding this is more effective, efficient ways of doing things scale. But when we see Spikes and activity, that's when we can go in and say, alright, like what does this end up looking? Let's go learn more about this, particularly P I customer.

You do you have a conversation with them? What are they actually doing for what is the use case? I think it's important to like be clear as a company like what you consider bugs versus features, you know and like I would be a awful outcome if adopted models were being used for any kind of A A coordination of fake news and election interference type things. And so we've got the T, N S teens actively working on that. And to the extent that like we find anything like that'll be combo additional model parameters plus um trust and save you to shut IT down.

we need to take another quick break will be a pack.

Support for this episode comes from A T N T. What does he feel like to get the new iphone sixteen pro with A T N T next up anytime? It's like when you first pick up those songs and you're now the one running the girl, it's indescribable, like something you've never felt before.

All the mouth watering anticipation of new possibilities, whether that's making a perfect cheese burger or treating your only to a girls' big potato, which you know will forever change the way they look at potatoes with the att. Next up, any time you can feel this way again and again, learn how to get the new iphone sixteen pro with apple intelligence on A T N T and the latest iphone every year with A N T. Next up, anytime A T N T connecting changes everything.

Apple intelligence coming falls twenty twenty four with theory and device language set to U. S. english.

Some features and languages will be coming over the next year. Zero dollar offer may not be available on future iphones next up any time feature. Maybe this continued at any time, subject to change additional fees, terms and restrictions apply.

See A T T docs sash iphone for details. Support for this podcast comes from stripe. Payment management software isn't something your customers think about that.

Often they see your product. They wanted buy IT and then they buy IT. That's about as complex as a guess.

But under the hood of that process, there are a lot of really complicated things happening that have to go right in order for that sale to go through. Stripe handed les the complexity of financial infrastructure, offering a seamless experience for business owners and their customers. Example strike can make sure that your customers are seen their currency and preferred payment method when they shop.

So checking out never feels like a chore. Stripe is a payment and billing platform supporting millions of businesses around the world, including companies like cooler, bmw and dordick. Stripe has helps countless startups and establish companies are like reached their growth targets, make progress on their missions, reach more customers globally.

The platform offers a sweet of specialized features and tools to power businesses of all sizes like stripe billing, which makes IT easy to handle suggestion bed charges, invoice and occurring revenue management. Learn how stripe helps companies of all sizes make progress at stripe dot com that stripe dot com to learn more stripe make progress. Support for the show comes from alex partners.

You already know artificial intelligence will be transformative. Beyond that, there might be a little bit of a mystery as A I opens the tech industry. Alex partners is dedicated to making sure your business knows what really matters when IT comes to artificial intelligence because disruption brings not only chAllenges but opportunities in these pivotal moments change alex partners is the consulting firm chief executives can rely on with clarity, direction and most importantly, implementation.

Alex partners provides a steady hand for your business needs when decisive leadership is vital. Alex partner spoke with nearly three hundred and fifty tech executives from across north amErica and europe to dig deeper into how tech companies are responding to these changing headwinds. You can see the results and learn how you can turn digital disruption into growth by reading alex partner's latest technology industry insights available at W W W dot alex partners dot com slash box. That's W W W dot A L I X partners dot com slash V O X. In the face of disruption, alex partners are who businesses trust to get to the point and to get things done when IT really matters.

Go back with improved c chief product officer might trigger to discuss where he thinks generated the eyes is going next and whether it's somewhere dangerous. With apologies to my friends at hard for casey and Kevin, they asked everybody what their pedia is. Someone ask you that, but that question is rooted in agi. Like what is the chances we think that we'll become self weren't kill us all. We ask you a variation at first, which is, what if all of this just hates are own like information apocalypse me and just like taking ourselves, like do we need the age to kill us all or are really headed towards information apocalypse first?

Yeah I think the information piece, like living in a society with this like amount of internal without A I already I think there's already just like just take you texture primarily textual social media think somewhat that happens on instagram as well. But it's easier to disseminate ate when it's just a piece of text that you can have. Has already bit like a journey outside in the last ten years.

But I think IT comes and goes, think we go through waves of like, oh, man, this is like how we ever gonna get to truth. And then good truth tellers emerge. And I think people flocked to them.

And I think some of the traditional like sources of authority, and some of them are just people that become trusted. And then we can get to a separate conversation on verification and validation of dentist one as well. But I think i'm an optimistic person's heart, if you can tell.

And I think that part of IT is my belief from an information sort, you know, chaos or proliferation piece of our abilities to both learn, adapt and then I grow the right mechanisms in place. So I remain optimistic that will continue to figure this out on that front. The AI component, I think, increases the volume.

The thing you would have to believe is that IT could also increase some of the passing I to say is that Williams was a new Stevens novel that came out few years ago. I have a little gives someone is one of the two, two of them had the concept of in the future perhaps you will have a social media editor of your own that gets deployed, you know, as as a sort of gating functions between all the stuff that's out there and what you are up consuming. I like some appealed to that to me, which is no, if there's a massive amount of data to consume, probably not.

Most of IT is going to be useful to you. And I ve even been trying to scale back my own information, diet. And to the extent that there are things that are interesting, you know, i'd love the idea of like go read this thing and that .

like this is this is worth robi algorithms, class fires, having filters and social me to help you see stuff. You're on one side of IT now, right, cause just makes the things and you try not to make bad things. The other companies, google and meta, are on both sides of the equation, right where we're racing forward with german I we're racing forward with lama and then we have to make the filtering systems on the other side to keep the bad stuff out. And IT feels like those companies are at decided cross purposes with themselves.

I think just the question is that I don't know occurred you know to us got a myself what he would say like what percentage inm content would and should be no yeah I generate at least now from .

your seat anthropic knowing how the other side works. Is there anything you're doing to make the filter and easier or anything you're doing to make IT more semantic, make you more understandable what you're looking at to make IT so that the systems sort the content have an easier job of understanding what's really what .

fake yeah there on the research side. And now outside my expertise, like active work on like what are the techniques that could make IT more detectable is marking is the probability that I think open question. But open also open like very active area of research as well.

I think the other pieces um well actually break down the three things like what we can do from like like detection in water, marking inside decide on the model piece also haven't be able to express them uncertainty a little Better, Better you know like actually about this, not willing to speculate or not actually willing to help you filter these things. So i'm not sure I can't tell which of these things are true. And also open area research, a very interesting one as well.

And then the last one is like if you're met up, if if you're google, maybe the bull case is that you're a if primarily your surfacing content that is generated by models that you yourself are building, there is probably a Better close sleep that you can have there. I didn't know if that's gona play out of whether people will always just fuck to whatever the most interesting image generation model and you know created and go and go publish and blow that up. I do not sure I think jury still out on that one, but I would believe that the built in tools, I instagram nine plus percent of photos that were filtered were filter inside the APP because it's just the most convenient thing and that in that way, a close ecosystem could be one route to at least having some verifiability of generated content.

Instagram filters are kind of an interesting comparison here. Instagram started as photo sharing, slim deli nerds, and came this, if he came instagram, uh, this is a dominant part of our culture. And the filters had real effects on people's image, had real negative facts, particularly teenage girls, and how they felt about themselves.

There's some studies that say teenage boys are starting to have self image issues and body image issues at higher and higher rates because of, but they perceive an instagram that's bad, right? In this bad wait against the general good instance, which like many more people get to express themselves, we build different kinds of communities. How are you thinking about those risks with anthropic products?

Coach I was working with, I would always push him like, well, like I want to I want to started on the confidence as much impact as instead I was like, well, like there's no cosmic leger where you'll know exactly what impact you have, first of all.

And second of all, like what's the equation by like positive or negative? And I think like the right way approach these questions is with humility and then an understanding as things as things develop. But I to me, IT was i'm excited and overall very optimistic about A I and the potential for A I if i'm going to be actively working on that.

I wanted to be somewhere where the like risks and the the sort of mitigations were as important and as like foundational to the founding story. Maybe to bring you back to like why I joined. Like that's how I baLanced IT for myself, which is like you need to have that internal run loop of great.

Like is this the right thing to launch? Should we launched this? Like should we changed in some ways? Should we like add some constraint? We explain its limitations in some ways.

I think it's essential that we're grap up with those questions or else I think you end up in the like, well, you know, this is clearly just a force for good. Let's like blow IT up and go all the way out like I feel like that misses having seen IT at instagram. Like you can build a commenting system, but you also need to build the bullying filter that built that we built. You know, I think is the .

second decoder question. How do you make decisions with the ramework?

Actually, maybe i'll go matter for a quick second, which is the culture on topic is extremely thoughts and very document writing oriented. So if a decision needs to be made, there's usually a document of behind IT. There's present cause to that IT means as I joined and I was wondering, like why did we choose to do this? People like, oh yeah, there's a dog for that and there's living a dog for everything.

And then which helped my grandpa. But sometimes I like, why why we still not built this? Like, oh yeah, yeah, it's really read a duck about that like two months ago.

And like, did did we do anything about IT? And so like my whole decision making pieces, I want us to get to truth faster, like none of us individually know what's right and getting the truth could be. Let's diarist the technical side by building a technical prototype if if it's on the product side.

Like, let's get IT into somebody's hands, like fig o mocs are great. But like, how're gonna move on the screen? And so like minimizing time to iteration and time to hypotheses. Testing is like my fundamental uh, like decision making philosophy. And tried to install more of that here on the product side.

Again, it's a thoughtful, very delivered because I don't want to lose most of that, but I do want there to be sort of more of this hypotheses, testing and validation components. And I think people feel when they like, oh yeah, we have been diving this for a while but like, we actually built IT and that turns out either of us right. And actually there's a third direction that that's more correct. And instead we went through sort of we ran the gama of a strategy frameworks. The one that's resonated the most with me consistently is playing to win.

I go back to that often, installed some of that here as well as we start thinking about, you know, like what's the wedding aspiration? Where are we going after then? Like more specifically, and we touched about the scene, even our conversation day, like where will we play? Because like, we're not the biggest team by size, we're not the biggest chat U I by usage or not the biggest a model by used to do there.

We've got a lot of interesting players in the space. We have to be thought ful about where we play and where we invest. So yeah and then and this morning at a meeting were like the first thirty minutes for people being in pain due to a strategy.

And the question is like strategy should be painful. And you forget the second part of that is that then you will feel pain on the strategy like create some trade ffs. But also just recognizing that, like you in instagram, we always talked about doing fewer things Better. That was like a foundational company of .

what was the trade off and what was the pain.

The trade off was like. Getting too much in the technical details is basic, like of the next generation of models, like what particular optimizations were making and you know, catch you exactly what they like. IT will make one thing really good and other thing just like okay or pretty good.

And like the thing is really good, I think, is a big bat is going to really exciting and what is yeah and like but you know and yeah but so i'm actually having a spread a little mini like document that we can all sign and other sounds kind of cheated or it's like. We are making this trade up. This is the implication. This is how we will never write or wrong. And here's how we're going to revisit this decision.

And I want us all to like, at least cited in google dogs be like this is like our I commitment to this or also you end up with the like next week of like but you know it's like that visit so it's like even disagree and commit it's like feel the pain understand IT don't go blindly into IT forever like I am a big believer when IT comes to like hard decisions, even decisions that could feel like two adoors. The problem of two adoors is something to keep walking back and forth between them. So africa walk to the door and say, the earliest I would be willing to go back the other way is, you know, two months from now, or with this particular piece of information, and hopefully kind of quiet, the like, even internal critic of, like it's a two way door, are always gonna to go go back there.

I think this bringing into a question I try and dying to ask the whole time, neural models, you're new anthropic your building products and help these models. I have not convinced that lens of technology can do all the things people are saying they will do. But my personal p doom is like, I don't I don't know how you get from here to there.

I don't know how you get from l to H I. I see being good at language. I don't see IT being good at thinking. Do you think Allen can do all the things people want them to do?

I think current generation, guess in some areas know in others, I think part of I think maybe we make an interesting product person here is that I I really believe in our researchers. But about defauts, like you know, defauts belief is everything takes longer in life and in general and in research and an engineering. And we think IT does.

I like, do this mental sort of a exercise. The team, which is, if our research shame, like got wing called, all fell asleep for like five years. I still think we have five years of product road map.

And we would be like, we are bad at our jobs. We're terrible our jobs. We can think of all the things, even in our current models, could do in terms of improving work, accelerating coating making things. These are coordinating work, even in the mediating disputes between people.

Just think is a funny L M use case that like we've ever seen, play out in turn ally around like these two people have this belief like help us even ask to the right questions to get us that place a good sounding board as well like there's a lot in there that is embedded in the current models. I would agree with you that like the big open questions to me, I think it's a basically like four longer horizon tasks. What is the sort of horizon of independence that you can are willing to give the the model like the metaphor i've been using is right now, LLM chat is very much.

You've got to do the back and forth because you have to correct you. You got to get right now that's ite. What I meant, I meant this.

A good limit test for me is like, when can I email claud and generally expect that an hour later it's not going to give me answer what to given me in the chat, which would have been a failure, but IT would have done more interesting things, and god n find out things and and iterate on them, and even like self critique and then responded and like I don't think we're that far for some domains. I think we're far for some other one, especially ones that involved sorted, like either longer range planning or thinking or research. But I use that as sort of my capabilities piece.

It's like less like you know parameter size. We're like a particular event to me. It's like, again, what problem are you solving? And right now, it's like I joke with our team. It's like right now, talking to cloud is like a very intelligent and easy.

I like every time we started new conversations, like, who are you again? Like, what am I here for? Like, what do we work on before? And it's like, instead it's like, alright, like, can we Carry continuity? Can we like, have to be able to plan and execute on longer rises? And can you start trusting IT to to get some more things because there's things I do everyday and like I spent an hour on, you know, some of that I really wish I didn't have to do and it's like pretty good love, age, use of my time.

But I don't think claud could quite do IT right now without its like a lot of a scaffolding and right now that here's maybe like a more six cin way to put a bon IT like right now the scaffolding needed to get IT to execute more complex task doesn't always feel worth the tradeoffs because you probably would have done at yourself I think was an city comic like uh, time spent automating something versus time that you actually get to save. Doing IT like that trade up is at different points on the AI curve and I think that would be the the bet is can we shortened that time to value so that you can trust IT to do more of those things? Like probably nobody really get excited to put that, you know call less all the planning documents, my products, things are work out into one document, right? The matter narrative and like circulate to these three people like and I don't want to do that today. I have to do IT today, but I don't want to do.

But let me know. I see in a more numeric way. I'm looking at some numbers here in tropic. Seven billion dollars of the funding as last year, anthropic has taken more than seven million dollars of fining in over last year.

You are one of the few people in the world who has ever built a product that has delivered a return on seven dollars of funding right at scale. You can truly imagine some products that might return on the investment. Can the l ms. You have today build those products?

Think is an interesting way of asking. The way I think about IT is the elms today deliver value, but they also deliver our ability or help our ability to go build the thing that deliver s that value.

Actually question what are those products that can do liver that much?

But to me, it's like the it's right now claudis an assistant. And you know the helpful kind of side kick is where I heard IT internally at some point, just like at what point is that a co worker? Because like the joint amount of work that can happen in even in growing economy with assistance, I think is very, very large.

So I think a lot about cloud for work, oud for work now is a sort of almost a tall for thought. You can put in documents, you can, you can think things and have conversations. And people find values.

Somebody built like small, like vision reactors, and they was on twitter. I was like this using, not using cloud, but I was told for that. So the point where like IT is now an entity that you actually trust to like execute autonomists work within the company like that delivered product, that sounds like a fanciful idea.

I actually think the delivery of the product is way less sexy than people think. It's about permission management, it's about like identity, it's about coordination, it's about remediation of issues. It's all the stuff that like you actually do in training a good person to to be good at their jobs. Like that.

To me, even within a particular discipline, like some coding tasks, some like particular tasks that involved like choice of information of researching, like each of those getting like have the incremental person on your team, even if they're not in this case, i'm OK with like not net plus one productive at that point, two five. But maybe there's a few of them and coordinated. I think that, that I get very excited about the economic potential for that.

And that's all twenty box a month, the enterprise subsumption product.

This debate with somebody around like the Price point for that is much higher if you're delivering that kind of value. And I was debating with somebody around you what snowfall in data breaks, and those have shown like data dogs. Another one like usage based building is like, you know, the new hotness we have like subscription billing.

You like usage based billing and like the thing I would like to get us to it's hard to quantify today, although maybe we will get there is like a real value based billing. Like what did you actually accomplish with this? And know there's people that will pink us because like a common player hear is that people have a right limits.

Like, I want more cloud. I saw somebody who like, I have two clouds. I have like, two different brothers windows and got, we have to do a Better job here. But the reason they are willing to do, and they write and they say, like, look, i'm like working on a brief for client.

They are paying me x no amount of money like I would happily pay another hundred dollars to get me to finish the things I can deliver IT on time and move on to the next one that to me is like an early sign of like where we fit, where we we can provide value that is like even beyond know a twenty dollar subscription. But I when I think about like deployed clouds and this is early kind of product thinking, you know, but it's things I get excited about being able to think about like what value are you're delivering. And like really a line over time is the way we're like. I think IT just creates a very sort of you know full alignment of incentives there in terms of delivering that product. So that's I think that's an area we can get to over time.

someone to bring the following back around. We started and distribution and whether thinking is so tailored for the distribution that they don't work. In other context, I look around and I see google distributing german I and its ones. I look at apple distributing apple intelligence and its phones.

Theyve talked about maybe having some model in our changeability in there between right now it's open a habit, maybe german I be there, maybe what i'll be there that feels like the big distribution, they're just gonna take IT. And these are the experiences people will have unless they pay some other money unit to someone else. In the history of computing, the free thing that comes through aerating system tends to be very successful.

How are you thinking about that problem? Because if you're just like, I don't think, open eyes getting any money to be in apple intelligence, I think apple just think some people convert for twenty box and are apple and that's going to be as good as how are you thinking about this problem? How are you thinking about widening that distinction, not optimizing for .

other people's ideas? yeah. I love the question I get asked this all the time, even eternally like what should we be pushing harder into like an undevout experience? And I agree, it's going to be hard to supersede the bilton model provider there, you know, even if our model might be Better.

Particular use this as like a utility thing I get more excited about. Can we be Better at being close to your work and like work products, having much Better history than the building sort of thing like pages comes with. And plenty of people do their work on pages.

I here, I don't know, but like there's still a real value for a google dox or even a notion and other people that I can go deep on a particular like for take on that sort of productivity peace. So I think it's why lineas have you are more to help people get things done. And some of that will be mobile, but almost maybe as a companion and provide and deliver value that is almost like independent of needing to be exactly integrated into the dust tub.

I think as an independent company, trying to be that like that first call, that theory, i've heard the pitch from startups, even fridge in here, like we're going to do that. We're going to be so much Better at and the new action button means that you can bring IT up and then press up, oh my, no. Like the default really matters.

They're like instagram never tried to replace the camera. We just try to make a really good thing about what you could do like once you like decided that you want to to do something not all with that, with that photo. And then, sure, people took photos in there.

But like by the end, when we left, IT was like eighty five percent library, fifty percent camera, right? Like this real value, like the thing that just requires the one click. So IT was interesting because know every wwdc that would come around p instagram, I loved watching those announced.

But so what do they get announced? And then like a changes, like, or what do they get, announce the point realized, like they're going na be really good at some things that oogly going be great at some things. Apples going to be great at some things. You have to find the places where you can differentiate either in a cross platform way, either in a depth of experience way, either in a like novel, take on how work gets done way, or be willing to do the kind of work that some companies are less i'm excited to do because IT, maybe at the begin, they don't seem super scalable.

Other consumer scalable, seven billion dollars worth the consumer products that don't rely on being built into your phone.

I mean, I opened up the up store and ChatGPT regularly. Second, I don't know what their numbers look like in terms of episode. I think it's like pretty healthy right now. But long term, I think it's yeah actually I optimistically believe yes, because I think they even on like conflate mobile and consumer for second, which is not a super fair or conflation. But i'm going to go with first, second, which is so much our life still happened there, that whether it's within L M plus rex recommendations, are L M plus shopping or l ms plus even dating like I I I have to believe that at least a heavy component can be in a seven billion dollar lar plus business but not one where you are trying to effectively be like syria plus plus. I think that's it's a hard place to be yeah .

open answer to this appears to be search like I need to disclose like every media company is taking the money. I have nothing to still like people know, but we took money to IT feels like going to search, right? If you can close some percentage of google, if you a pretty business, it's because such in the della told me being when they launched a chat, pretty power being like any any half a percent of google is a huge boost to being.

Would you build a search product? We talked about recommendations a lot. Yeah my entering recommendations and searches like .

right there yeah it's not on my mind for any kind of near term thing. I'm very curious to see. I have not probably for a good reason. So though I know having, we are pretty much just can't.

So I am gotta play with IT, but like that space of, you know the perplexity arch chat search of they actually yeah I mean, test back to the very of our conversation, which is like search engines in the world of summarizing and citations. But you know probably fewer clicks and where does that end up? You know how does that all tie together and and connected and it's less core outset to what we're trying to do.

So IT sounds like right now to focus on work, right? Just got a lot of work products you're thinking about, maybe not so much on consumer. I would say the danger in the enterprises, it's bad if you're enterprise software is solution ating just probably consume as risky and seems like those those folks might be more inclined to see you if you send some business he wire because the software is resonating.

Is this something you can solve? I had a lot of people tell me that L, M, S are always Cindy, and we're just controlling the whole. And I I should stop asking people that can stop pollution ating.

Because the question isn't make any sense. Is that how you're thinking about IT? Can you can you control IT so that you can build reliable and Price products?

I I think we have a really good shop there. The two places that most recently this came up, one was we are current like aleem will often times try to do bad soni as they actually are a given the architecture are impressively good at math, but not always, and especially not when he comes to select higher things, or even things like counting letters and words that they should get there.

And so like one tweet we've made recently helping cloud, as is on cloud, I recognize when IT is more in that situation and explain its shortcomings is IT perfect. no. But it's like significant, improved that particular thing because from an enterprise that this came directly from an enterprise customer said, hey, I was trying to do some c sv passing.

I'd rather you give me the python to go analyze the c sv, then try to do IT yourself because I don't trust that you're going to do IT right yourself. So I think on the like data analysis code interpretation that front, I think it's a combination of having the tools available like elms are a smart, sorry humans. I still use calculators all the time.

In fact, over time, I feel like I get worse and mental math and like those even more. So I think there's a lot of value to hey, give IT tools teacher to use tls. A lot of of the research team focuses on and then really emphasize the time work like, yeah I I know if you think you can do this. The joke I do is like the c sv version is like I can I ball I call lumb number and give you like my average is probably not going to be perfectly right. So i'd rather like is the know average functions that's done the on the data front, on the citations front, the APP that has done this most ball recently.

I have no affiliation this other than like we listen to her parenting advice all the time, which is like doctor Becky was like a parenting grew has a new APP out and I really like playing with chat PS so I really tried to push them and I pushed this one so hard around like trying to like hello ador talk about and IT wasn't familiar with and I have to go talk to the maker, actually think them on twitter. They do a great job of like if IT is not super confidence that that information is in its uh of retrieval window, IT will just refuse to answer and IT won't confabulated IT won't go there and I think that that is an an answers as well. Just like the combination of model intelligence plus data plus the right like prompting and retrieval that like you don't want you to answer unless there actually is something grounded in the context window helps tremendously on the illustration front current.

Probably not. But I would say that like all of us, all of us make mistakes, and hopefully they're like predictably shaped mistakes. So you can be like up danger zone, like outside of our of our of our peace.

There are even like the idea of like even having some like the most syntax highlighting for like this is rounded from my context. This is from my model knowledge. This is out of distribution like danger will Robinson.

And i'm not sure if this is exactly where i'm like. I'm not exactly sure i'm talking about. Maybe there's something there.

this all this adds of my feeling that like prompt engineering and then like teaching a model to behave itself feels non deterministic and away, like the future of computing. Is this like misbehaving? Tod ler, and we just have to contain IT and then then they're be able to torture computers like grow people, and we will be able to torture us to grow people.

That that just seems wild to me, that even if you're going to release the system proms, I read the system proms and I like this is I were going to do IT like apple system process, do not hunca ate and like that. This is how we're doing IT. Does that feel right to you? Does that feel like a stable foundation for the future of computing?

It's a huge adjustment, really like an engineer at heart. I like determinism in general. No, like we had a insane issue at instagram that we eventually track down to using non ecc RAM.

And little cosmic rays were flipping RAM and got like, what do you get to that? Like dive. Want to rely on on my hardware.

Here's the the, the, the action moment, maybe like four weeks into this role. Well, like, okay, I can see, uh, the perils and potentials. We were building A A system in elaboration with with the customer.

And we talk about tall. You started like what the model has access to and that we had made two tools available to this to the model in this case. And one was a to do list APP that I could write to.

And one was like, like reminder, sort of like short term or like timey type thing. And the todo list system was down. And it's like, oh, man, I I I tried to use that to do. I couldn't do IT.

You know i'm going to do i'm going to set a timer for when you meant to be reminded about this task like and if so, I set up the third time I was like a forty eight hour time where you'd never do that on your phone. That would be ridiculous. But I to me show that like non determinism also leads to creativity.

And that creativity, like in the face of uncertainty. Is ultimately how I think we are going to be able to solve these like higher order, more interesting problems. And that was the one as like, but it's not deterministic, but I love you.

So it's like, not deterministic, but I can put IT in these odd situations and IT will do its best to recover like act in the face of uncertainty. Where is any other sort of like heroic if I had written that, I would never have thought of that particular work around. But I I didn't, I think, pretty creative way. So I I can't say IT sits totally easily with me because I still like determinism and I like predictability and systems that we seek, predictability where we can find IT. But I think I ve also seen the the value of like within that constraint with the right tools and the right sort of infrastructure around IT, how IT could be more robust to like the like needed messiness of the real world.

You're building up the product infrastructure, obvious ly thinking a lot about the big products and how you might build them. What should people be looking for from anthropic? Like what's what's the what's the dangerous point of product emphasis we should be looking for?

yeah. So on the cloud side, you know, tween, I think the time we talk and at air were launching cloud for enterprise. So this is like our pushing to really going deeper is a bunch of a on the surface unexciting acronis like s soo and skim and did not not manager in audit ogun.

But that's the importance of that is that you start getting to push into like really deep use case in rebuilding date immigration that I make that useful as well. There's that whole component. And on the API that I shouldn't talk as much about the API that though, like I think of that as much as an important product as anything else that we're working on the big pushes, how do we get lots of data into the models of models are also me.

They're smart, but there's I think they're not that useful without good data in there like like tito, the use case, how do we get a lot of data in there and make that really quick? So we launched explicit prom cashing last week with risk. As you take a very large data store, put IT in the context window, retrieve IT ten times faster than before, look for those kinds of ways in which the models can be brought closer to people's actual interesting data.

And again, as always, test back to our effect and get you personalized useful answers in the moment at speed and like at low cost, like that whole push. I think a lot about like good product design pushes extremes in some direction, like this is the lots of data, but also punch the late and see extreme and like see what happens when you when you combine those two axioms. And that's as a able, able continue .

pushing for the rest here. Yeah like this has been great.

I could talk.

I'd like to think my crayon for taking the time during the coder and thank you for listening and we enjoyed IT like let us know what you thought about the shower anything else you'll us cover, please drop us line you can email a decoder verge shock on do read all the emails or we can hit me up on threads on that reckless travel. We also have a tiktok. It's atticoli part.

It's a lot of fun. Decode your friends and subscribe where your podcast decoder production, the verge in part of the box into test network produces are k. cox.

And next that our editor is Kelly, right? Supervising producin James. The decoder music is by breakfasted ser. next.

Time support for this podcast comes from strike. Stripe is a payment and billing platform supporting millions of businesses around the world, including companies like uber, bmw and door dash. Stripe has helped countless startups and establish companies alike reach their road targets, make progress on missions and reach more customers globally.

The platform offers a sweet specialized features and tools to fash track growth like strike billing, which makes IT easy to handle subscription based charges, invoice and occurring revenue management needs. You can learn how stripe helps companies of all sizes to make progress at stripe dot com, that stripe dot com. To learn more stripe make progress.