The chatbot becomes the teacher
01:00:06 Share

Welcome to the verge cast the flagship podcast of infinite context windows. I'm my friend David pierce, and this is the first episode in our new mini series, all about A I and real life. We did a few episodes on the subject earlier this year, and IT continues to be a thing that we're talking and thinking a lot about for all the big heady talk of how A I will either change everything, or kill us all, or make nobody ever have to work again, or make us all have to work training robots.

What is any of this actually good for? Like right now that's not even trying to figure out for today's episode, I am talking with Stephen Johnson, who is a personal favorite author of mine. He's written fourteen books over the years, and he actually told me that he can name them all in order off the top of his head, which I believe, and also find very impressive.

And some of those books are books you've probably heard of, like where good ideas come from and how we got to know. But in addition to all of that, Stephen has also spent the last two years working at google on a project called notebook L M. No, book them if you've never heard of IT is an experimental thing out of a team called google labs.

IT started out as a thing called project tae in a couple of years ago. And the idea has always been to make an AI power tool, basically for making sense of your notes in the book. L, M, which they all call notebook.

So I just start calling IT. No book. You first upload a bunch of documents or links to websites, or P, D, F, or whatever else, and the tool builds sort of a corpus of stuff, the ideas.

You put a bunch of related things into a notebook in notebook. Then you can ask questions about those documents, or you can have notebook. Build you an automatic study guide or an FAQ of the information in those documents.

You can have IT find you stuff in those documents, all that kind of stuff. It's sort of a note taking tool, but mostly it's a research tool. Steven calls IT a tool for understanding things which I like i've been covering and using no book for a long time, but I wanted to have even on now because no book is kind of going legit.

I assumed, if i'm being completely honest to you, that IT was like a need experiment that would eventually die because everything dies at google or at best, IT would just be a tiny feature buried in a menu of google dox or something. But no book is growing and it's expanding and it's actually starting to do some really interesting new stuff. Recently, they launched a new feature called the audio overviews, which generates a podcast hosted by two chatbot based on whatever documents you upload.

IT is wild, actually. You should just hear this. So I made a notebook in notebook with a bunch of stuff from the ongoing U.

S. vers. Google ad tech trial. And here's just a few seconds of the podcast generated OK, and that's .

where heater bidding enters the picture. Herbie ding, yes, this was their attempt to kind of out eu google into the way is specially offer their ads space .

to multiple add exchanges at the same term. Look, I don't know if that's good or bad. I don't know if all of that information is even true, but I am fascinated by the idea of a tool that tries to make IT easy and automatic to learn almost anything in whatever style works for you.

And Steven is just fascinated by IT. So I figured we should talk about IT. All that is coming up in just a second. This is the forecast over IT back support for the verge cast comes from strike.

Strike is a payment and billing platform supporting millions of businesses around the world, including companies like uber, B, M, W and door dash. Stripe has countless startups and establish companies alike, reached their growth, make progress on their emissions and reach more customers globally. The platform offers a sweet of specialized features and tools to fash track growth like stripe billing, which makes IT easy to handle subscription based charges, invoices and all reoccurring revenue management needs.

You can learn how stripe helps companies of all sizes make progress at striped out com. That striped out com to learn more strike, make progress. Right, we're back.

Let's get into my conversation with Stephen Johnson. I figured the best place to start with the notebook alam story was just at the beginning. So that's where we started. How is IT that an author comes to be a google employee working on an I product yet?

It's a really interesting story, I think so well, I have spent most of my career writing books about kind of science and technology and history. I've always had this kind of side interest in using the technology to help me with the book writing process, with the research process, you tools for thought. That whole tradition has been a big influence on me and and i've always been kind of an early adopter of, you know, use this program called David and think or organizing all my notes and quotations from books that I read in the in the two thousand the verge cast.

We're going to have to do at some point .

to get ready for that. Yeah yeah so you know, I know wrote about IT and kind of blog posts, and I wrote a couple things for the times about this and IT shows up in my book where good ideas come from. I talk about using tools like this and was a big script er user and evAngels and all that. So so i've always been interested in the in the software side of writing and thinking and research and in the spring of twenty twenty two, you know, so six months before the ChatGPT moment, I wrote a very long peace from the times magazine about language models in general that was effectively just making the argument like, forget about a or any super intelligence for anything like these models have basically learned how to community in coherent language, and they understand what we're saying and the and whatever else like this is going to create all these new possibilities.

Yeah, that piece was very good. And I remember a bunch of the the response you got to that was from people being like, this is bonkers. He's out of his mind. There's no way any other surface going to be that big. And then boy, did you time that create.

well, I did except that I was painful, like I am very conflict diversity, if I can, the us. And there there was, I mean, a lot of people did like that piece. And I think where was fired by IT.

But there was definitely a lot of comments from people seem like, oh, he fell for the height. This stuff is just auto complete on steroids. And I can't believe he's so naive that he got excited about this thing. The piece addressed a lot of the objections and criticism took them very seriously.

But I was like, the one thing you can do is dismiss this technology like something fundamental is just happened and we're going to spend years figuring out like how we apply IT and how we deal with the upside in the potential downsides. And like, just take IT seriously. people. So you know, if I tied IT later, I wouldn't have maybe had as much of the push back, which was hard to I couldn't enjoy that .

piece going out.

Good peace. I like, well, appreciate that now. So so right up on that point, kind of a new division inside of google called google labs or had been a previous google labs s and this is kind of a new one was was getting spend up um and at the time I got him clap before was running IT and josh wood ward, who runs in now, was talking back and forth with clay and labs s had kind of eos of the division was basically we want to create a place where we can do faster and more nimble l product focused experiments with new technologies where you know it's not just kind of research um but it's not working within the existing mature products. And when something new comes along, we can experiment and you know build things very quickly.

So somewhere in between the like twenty percent time project and the like full blow new google product.

yeah yeah. Was just a little hole that didn't quite exist. And there were so many interesting new technologies coming out, particularly the language models, that that seemed like this was a time for a lot of experiments to bloom, right? And they had had this idea that maybe they could also have an ethos of cocreate where they bring in outsider.

So if their, if they're making a product that involves music, there should be a musician in the room from the beginning and it's not something that kind of they build. And then they showed to the musicians or they have you know ux interviews with musicians, there's actually someone there who represents that kind of profession in the room for the life of the product. And so so I was kind of the giddy pic for this approach and this so they reached out, john and clay had read some of my books and to read that cme article.

And so they reach out and said, hey, you know, you've been dreaming of this ideal software tool that helps you organized your thoughts and helps you write and helps you formulate connections and you brainstorm. Um we think we can now do IT with language models like this thing you've been chasing, literally, you know, i've been chasing this ince. I was in college in the late eighties when hypercard came out, know for that old first.

I've been after this for a long time, and they were like, look, I think if if you came to google come part time initially, and we have a small team and we can we can build something. And I thought that sounded like an amazing journey. Honestly, I thought will build a prototype.

It'll be fun and meet some interesting people, but nothing will come of IT. But IT was still you a fun ride to go on because of that, my passion for this and and then we and we built something that was originally called tail and project Taylor. But from the beginning, and very like day, one of the of the idea, there was always a sense that this was not just going to be an open ended conversation with the language model.

IT was always going to be about the model being grounded in the in the sources and the information that you gave IT and IT was really about respecting the original kind of human authored information um whether it's a book or your own notes or a scholarly paper or your silence for your class and and basically saying to the model, take this information which is personally relevant to me and is verifiable on somewhere you're actually trustworthy and base your answers and everything asked to do on that information and that was the seed of we had a version of that in August of twenty twenty two, ceded me there. There was a program called talk to a small corpus that was about a month underway when I got there. And then one of the first things we did, we put in my book, wonderland.

And we just like I had this experience of kind of chatting with the model a and having IT answer based on information in my book. And you know that was one of the moment a lot of possibility just open up. Well, I actually the weird moment I would say was a little bit later um in over a Christmas break um the internal version of what became barred was kind of released to some of us insight over Christmas break.

I was spending a lot of time with bard. My family had gone off schemes of where I don't. I was just like home alone and barred came out like, well, I just now have like eight hours a day to my my new friend bar.

So I would occasionally just start with just a little preambles to get its bearing ings. And I would say I would like to discuss Stephen Johnson s book, the ghost map. And this would just based on this kind of training data, this isn't what sort goni.

And so one day I do this, and right, I O we am. I would love to discuss that. That's a that's a fascinating book about a medical mysteries.

Y in the thousand nine hundred centuries for the impact of coloradan epidium logy on the history of landon, history of generally. And so i'm like, oh, well, thank you very much. I'm i'm actually the .

author of this .

at barn said, oh my gosh, I am so sorry. I can't believe I didn't recognize you, mr. Johnson and like you know and I was like, I know but there was no way you could have recognize right but um but he was just one of those robots like sitting alone in my study having this conversation with and algorithm that's apologizing for not recognizing me when it's a fan of my book and at one point I said i'm just so excited at the opportunity to get to work with people like yourself and I was just like, this is this is me so there are a lot of like uncanny moments like that.

But in a way, you know, there was part of that that I also recognized was an illusion, right? IT was trained on the way that people react when they meet. People are failed to recognize someone. And so IT responded in that way that obviously had no inner life IT did not IT was not actually like embarrassed by the fact he was meeting me, was just kind of play acting at that.

So to me, the stuff that really was mind blowing was just its ability and and this really kicked in um know for us when we switched to german I its ability to extract and and see patterns in large amounts of information. You know I have this no book where we have um something like you know almost a million words of transcripts from the NASA oral history project. So just interviews with like that you that NASA astronauts and flight directors and things like that and you can go into IT and say i'm interested in i'm working of documentary about um the early Apollo program and i'm interested in the emotional connections between the participants, particularly the astronauts.

So can you create a detailed guide of all the points in these transcripts where anything that seems interpersonal or emotional comes up? Give me a summary of that section um give me a direct quote from the section and of course include locations so I can click immediately, go back and read the passage in its original form like and it'll just do IT like it'll take a little bit at time because it's a complicated query. But like IT, IT can take something that would have taken forty hours to compile that document like how I mean, you know what it's like working with information like this.

Like IT, IT will generate a incredibly convincing and accurate first draft with ground citations to all the passages in about forty five seconds maybe. And so it's literally know a thousand times faster than that would have been before to do that kind of thing. And it's not pretending to be a person.

It's not pretending to have feelings about IT. It's just grammy, that very subtle. Kind of collection of information that is not anchored in any keyword, right? Know it's not looking for mentions of emotion. Like I just understands that these other things where they talk about their kids, that's an emotional moment and it's able to to call late that way when IT started being capable doing that. That was the point for me where I was like, oh, this is really a fundamental change and .

that was that happened with geri what he was what earlier this .

year yeah we switch watched to germany one point of in in december and and IT was great, but I was really one point five for. And the bigger context window, I am the only thing that happened to be, by the way, of a nice writer, dude. And now, like I hear, like a new million token model.

And it's like the most exciting cannot wait together, heads out IT. So when we got first access to that um that model I took the entire text of of my book and feral machine, which just came out a couple months ago, was in manuscrip for at that point. So this is really important.

None of the words from my book, we're in the training data for the model itself as all you had had never been published, I had never been discussed about in any coverage. So the fact is the book of history. So the facts are probably in some form in the models training, but the book itself, in the way that I presented the facts, was not. And the thing that had struck me about the early discussions of large context was that people were were using IT to do these kind of middle in a haste test, where they're like, oh, we gave IT, you know, the full text to mobile deck, but we added this one line that was different in was able to find the right and which was cool. And you know, you couldn't do that before.

But the point that that I was so obsessed with this, that once you have the full text of something like a book in context, IT means that the the model, no, I can can find the obscure things in the in the text, but IT understands the sequence of the text, and I can understand large kind of movements of like cause and a factor change over time in in a document which you can't get if you're just giving a isolated paragraphs and snmp of things. And so I put in front machine into this version in german. I and I basically asked inside a new book.

I was like, give me. I was like, i'm interested in the way that Johnson uses suspense in this book. I would like you to list four places where he deliberately withholds information from the reader in order to peak their interest. Described the passages where he does this quote from them and then explain the future later in the book that he's obliged referring to that doesn't arrive for another no fifty ages, whatever.

And IT just absolutely nailed that he put the first example give, gave was exactly what I would have picked, is the ultimate kind of former suspence, which is a illusion to a ticking, mysterious, ticking suitcase that happens in the in the profit. That doing the mystery behind IT doesn't get explained for another two hundred pages. And so like think about that.

Is that like a search query, like search is find examples where something isn't mentioned and isn't mentioned in a very provocative way, and then fill in the blanks of the thing two hundred pages later that is obliquely referring to like that. You know, again, the model is not understanding the book on some level, because understanding is a word that we associated with consciousness from sentence and with an interview of, like what IT means to read a book. But the model is doing the thing that human understanding does. Yes, you know, and that's that's an important distinction.

I think no, I think I think that's right. One thing I heard somebody say not that long ago is dead. The Better metaphor they liked than its understanding is just that I can see the whole thing at the same time.

Yeah, and I was thought that was really great. It's like if I I can see one page at the time, this thing can see all three hundred and that isn't it's not Better at knowing those things. It's just literally by being able to see IT all at once yeah the the number of things that suddenly become very basic because you can see them all together is very powerful.

And I just that makes IT both sort of simpler and cooler all of the same time, which I really like. But you you break up this tension that I think is fascinating with all A I stuff, which is there is a set of things that you are just sort of remarkable that they're possible, right? And it's you see this with every new model that comes out and every new product that comes out.

One of the first things everybody does is just try wild stuff and some of IT works and some of IT dos, some of its amazing and some of its dangerous and whatever. So there there's the the sort of novelty facto IT that I think so so rampant in everything A I right now. And then there's the question of what is any of this actually for? And I think one of the things that i've liked about notebook and sort of watching and develop time is IT feels like your sense of not just what this can do, but what is for has gotten much Better over time.

Uh, and I want to if part of that is like, does the novelty of moments like that start to wear off and you start to realize that, okay, that's cool. But no academic is actually going in here searching for what's missing from this. That's not like a thing most people need in their lives.

And he started to sort of wit back to like, okay, how do we bring that sort of enabling in technology, the things people actually do need or maybe is the the crazy is the point, and we've just never been able to do before. So now we're trying to discovered at all, all at once. So I guess that especially in those early days, really before german, I one point five come the later brain on fire, like what what is that process of figuring out, not just sort of what what can this thing do, but like what are we building this tool for?

Yeah, it's such a great question. So many different ways to get into that. I know. I think in the very early days, there was a sense that we were building something we knew was not gonna really work because the context wasn't big enough, the model wasn't sops cared enough, but you could see where things were going.

And so, so much of what we really focus on from the beginning is, like, what is the proper interface for this kind of thing? Like if you know it's not just about a text message thread. Like surely there are other kind of forms of you.

I like if you were going to build a product from the ground up knowing that he was can be built around language point. Like what would you look like? Let's start from scratch.

And that's a very fun open canvas to have. But IT also means you make a stuff to this really make sense. And IT doesn't work because the model isn't conduit yet. So so there was a lot of experiment with that. I think we had an advantage in the early days in that I was just trying to drive IT towards my very specific cause of, like, I want a thing that is read all of my work and all of the quotes from books that have influenced me and can be a second brain to help me like, remember those things and make connections, and like in so kind of author researcher mode and that we kind of build the first prototype with that. And then once we were able to kind of open IT up to more users, I think then then we were just constantly discovering all these amazing things and and a big addition.

This was rise of Martin, our product manager, who has just been, she's just incredible, like listener to user SHE set up this discord that was so sense, like of the very beginning of a discord, or just not a Normal thing to do and something. And we now have this amazing community and it's costing filled people being like, oh yeah, I saw this opportunity to use IT in this way. So like our favorite one that completely came out of the discord was dungeons and dragons players started using IT because I like, I have these big, you know, campaigns that i've designed, you know, i'm a dunder master, and I have, like, created this whole virtual fantasy world, and it's filled with all this information, and it's hard to keep track of.

But I can load these documents into new book alam, and then I can just like ask any kind of open any question, and you will be like, how many hit points do I need to kill this work or whatever is not A D and d player. And they were using in that. And people were writing fantasy, also like world building kind of fantasy novels, and they just needed to have a kind of story bible with all their characters in the back story, everything like that.

And I turned out that, you know, theyd never really had an interface to let them do that. And that was not something we were thinking about IT at all. So we've just been like once we've got passed that first little prototypes stage, we've just been really listening like intently to like where people are trying to push the tool and then just like making IT easier, making IT so they don't have to push quite as hard to use the tool that way. And that's yes, that's very real. Well.

it's fun. I mean, even thinking about you, you've written a couple of times for years about your your sort of endless I think you called the Spark file ah that is basically just like a mountains document of all the good and bad story ideas you have. I'm partha rasing.

But I think that's right.

All this, this buy that book now. Ideas, ideas. And I think like just listening. You describe the the story bible for the world, building stuff like that actually is like a perfect down the middle use case for this in a way I had never really thought about.

And so just now that IT is like, here's a bunch of stuff that I have decided one way another right here. Here are my inputs. Help me make things with that.

It's actually that kind of an amazing and very difficult otherwise use case because it's like, oh yeah, how many I I have to go and collect that piece from invention, that piece from formation? I think there's all kinds of like complicated things with how we think about the art on top of all of that. But that thing where is just like I want to tell you the rules and I want you to help me make games out of IT yeah feels awesome. Like I I am so much less conflicted about that and I am about so many things in A I that's so cool. I love that.

Yeah well, we were trying to do the things that less conflict. I said I was conflict diverse. So I just like child to steer towards the thing. But but yeah, it's a great point that kind of like take this massive unstructured data and turn IT into a set of kind of formats that helped do the job that i'm trying to do with some guidance for me. And and this was, by the way, this was another great place where, like, rise.

I really saw this before I did because I was thinking of IT is i'm going to write my book so i'm going to do all the like concentration here. I just need to be able to like surface the facts and make some connections. But IT turns out they're just all these places where you've got all your company documents and you want to create n FAQ for new employees like no one is gona win a nobel Price creating that document like and if you can get a first draft of that document in forty five seconds instead of in four hours, like that's a win, right? That that is good news. And so there there are all these different workflows that are out there where there's massive information needs to be kind of like filtered in some way and turned into something else.

All right, we ve got to take a break and then we will be back with more from Stephen Johnson. We will get back. Support for the show comes from crucible moments, a podcast from scope capital.

We've all had turning points in our lives where the decisions we make end up having lasting consequences. No one knows this Better than the founders of some of today's most influential in crucial moments. Let's listeners in on the maker break events that defined major companies like dropbox, youtube, Robinson od and more told by the founders themselves.

Tune into season two of crucial moments. Today you can listen a crucible moments stop com, or rarely listen podcasts. We're back.

One of the things that i've been tracking about notebook for a while is how IT approaches, things like allusions and locations because this isn't like a silly chatbot, you know, it's a research tool. It's not silly or acceptable or just a signal that it's early. If it's wrong, it's a problem. If it's wrong, it's useless, Frankly, if it's wrong. So I asked him how they're trying to solve that very real problem in A I. So before we get too far away from me, I do want to talk about the the accuracy and the citation bit IT because you sign yourself up for a pretty high bar on both of those things, uh, both by virtue of like the people who are going to use your product, Angeles, what the product is, right?

Like you really get to have the warning at the bottom of everything that says this thing, make some information up like its entire purpose is not to make information up ah how do you? What if you guys done? I know no book was an early experiment in in rag, which is a way of basically windowing down some of these systems but like what what if you guys are definitely at no book to try and solve that? And here is both on the underlying text side and on the user experience side.

Yeah when we tried um i've kept calling its force grounding. Think I think that's a Better .

name than rag. I don't know. Mean, we all know GPT now. So I just i'm given up. We're hoping the acronym.

So you know I think part of IT was the fact that we were doing IT from the beginning like that IT started with that IT wasn't like, go let's build a job model and and oh shoot, we need to be able to like, you know grounded and other documents like I was from the beginning. We were and so we just had a lot of time to like iterate and explore. The gi models are really good at at source grounding.

Just there's a lot of training sets. We contributed a bunch of them to, like just given this document, answer questions factually based on the information that document. So we inherited like a great tool that we did, very little good us to the germany team for building a model that that is is much more faithful to source material.

You give IT, but we build IT also. This is this is one of these things very like it's it's underline model plus the U I. Is has always been like our montreal from the beginning.

And you know both things are required. And so one of the key things that we've had pretty much from the beginning is that you can always read your sources in the APP. And then with the release in june, we we switched over so that you now have in line citations tions to everything. So anything the model says has a little link. You can read the original passage if you ever over that shows you, you know the and you can click on IT and you can go read the source in the APP.

Do you think that's enough?

We've talked about. so. So one of the things you can do right now, actually with no book that we want to actually turn into a proper future, but you can do right now is you can update, load a bunch of, you know kind of source material, factual source material, and then you can upload the article you're writing for, and you can say fact checked article based on these sources and suggest improvements.

And that's clever.

That's a good idea. And yeah, it's amazing. And I will go through and be like, well, this is cracked. This is financially wrong. You will see anyone have blinks everything. And so to me, if I felt that the model were just luCindy wildly in its responses, that I do not believe that just providing citations and the ability to kind of fact check manually and go back and see the original passage would be enough.

But um we have we've been sitting their banging away like quality reviews like constantly for the last year and a half, like we have a poll sheet and sheet and sheet of of sample questions and sample documents. And we can just see that accuracy rate going up you dramatically, particularly with with these latest models. And so right now, I feel like we're we're in a pretty good place.

I rather I honestly, I rarely see no book and just wildly fluctuate something. I mean, one thing that is important, people may not know this. I take this for going to because they're never in this product.

If you loaded in a bunch of sources about the history asa, and then ask your question about Taylor swift in general, new poem will say, i'm sorry, your sources don't discuss the swift so I can answer this question. And obviously the model knows a lot about tayle slip is probably a pretty big fan of tailor's lip. But it's but it's been specifically you instructed to not answer questions that are outside the source material and I mean, too a fault.

I think sometimes you know one would like to bring about knowledge, you know and trust, but verify with that. But we have heard on the side of like stick to the facts in these documents. And so with the increase in accuracy and with the U I observations, there's more things we can do. We can make that fact. You can feature um double check kind of as as something, but I feel pretty good about where we are and trips the quality side of that where we are kind of state of the art is upload many, many documents, ask a complex question that involves like multiple kind of variables drawn across like multiple documents, get a deep, long answer with locations. Follow those citations to read in the original text like I think that no book and you know kind of does that flow as well as anybody right now.

I feel like you just described like like a personal wikipedia. And I mean, that is a compliment like the the thing that wikipedia is best, that is just being like a starting point to go learn about something on the internet, right? Like you open the week P D page, you go click on all the references of the bottom and you go read those references and you're you're often running. And I feel like what you just described is like I can shortcut that with any process of any single thing that I want to learn more about. I just dumped in and like what's going on here ah and I will just be a here some stuff.

I have so many things to say that you're going to go up to give you. okay. So one thing is i'm in the process of thinking about what i'm going to right the next book on.

And so I created that this is just kind of second nature workplace for me now. But uh, I wouldn't have curled to me a year ago. So I created a new nobo called the next book.

And whenever there's an idea that kind of come to my mind or the art, wow. And but it's focused on this project of like what the next book. And so the other day, I know late night, I have this kind of thing I do late because I no life.

My children have all got to college, and I sit around and I, so I was like, I wonder, you know, has ever been a good book written about the anti nuclear movements, anti nuclear power movement of the sixties and seventies? So that's an interesting case where we like stop the technology partially in its tracks and maybe made a mistake. And how do we interpret that? And so I just went and grab like two wikipedia pages.

In the past, I would have, like you started by reading through all the wikipedia ge. But I Brown in this, a new book, and I was like, i'm even Johnson. I'm a thing about writing a book in the mode of my other book. I can from the machine and goes map potentially about the anti new year powers.

Take a look at these, like what do you think there? Are there any interesting story lines that there would be good starting places? Like what what would be good there? And IT just does not like IT, you know, go through and it's like, well, you can focus on this period.

This figures is kind of interesting, whatever. Then I went in red, all i'm going to read the material. But as as a first gLance, like inroads, the material is amazing for that kind of exploration. The experience of navigating through a book, through a conversational interface is really interesting.

And it's one of these things were like, you know, until now, if you wanted to explore the ideas in an authors' work through a conversation, you could only do that by finding the author in person or finding A A scholar tuder who's an expert in the authors ideas. And like that was IT. That was not there is no other way to do IT. But now you can load, you know, a book and and you can start with like i'm interested in this, tell me about that and then slowly read the book in a nonlinear way through a conversational interface, kind of dipping in and out of the kind of original passages, which is terrible if it's a novel or terrible if it's straightness your kind of history book, like I sometimes, right? But if it's an advice book or a book of ideas, there are lots of books that I think could be explored in that way that just, you know, weren't possible before.

Does that feel like cheating to do that? Like for you as an author, you have you have worked through these books. You have done the hard work.

You have state up late at night. I'm sure you've like woking up in the middle of the book idea. Does sitting down and asking a computer what your netbook should be feel like cheating?

I think if I were literally like, hey, what should my next book be one I don't think I would. I mean, right now, in a way, it's easy to answer this question right now because IT wouldn't be good enough like IT wouldn't generate like tell me what my book should be and now start writing IT like I just wouldn't able to do IT. Okay.

current trend lines continue minute.

Come back to me in two years, we will see. But what I have to do is like i'm not using in that way.

I'm using IT like these very like kind of targeted queries like I was I actually was using the NASA notebook because I was interested in like know, maybe there's something to be done about the Apollo thirteen fire and so then I was kind of like, okay, look, I need to know what I need to read because it's a million pages, a million words of transcripts. I do actually don't need to read the whole thing. What I want to read the sections that are relevant to their public thirteen fire.

And so I just wanted nobody and to tell me what I should read. You and IT was like, you should start here, you should be there and I can click immediate in there and and read to that. And so me i'm using IT as a way of accelerating the process of discovery that would just have been painful to do before, but i'm making just as many unclean serendipity discoveries along the way.

I mean, I think the criticism is like, well, if the model serves IT up to you so quickly, even if you end up writing the book, you've missed the surprising thing that would have never found. You would have only found IT by reading through IT, an incredibly Linda way. And I think that that is just not true, like I is constantly surfacing things that I had thought of and making connections that I had thought of.

And so it's helping me understand the material more deeply where IT gets complicated. And you know, i've talked about this before. The way I think about IT is is a tool that helps you understand things.

If you are genuinely interested in understanding things and having that understanding being in your brain, it's it's an a huge net positive hunter. Was that like if if you go into IT with good intentions, IT is a, you know, an amazing tool for that. IT helps you have richer, more complex ideas under same material. Better if you are not interested understanding things, but rather interested in creating the illusion that you understand things and just want to bluff your way through life without ever actually understanding anything, but creating outputs and make look like you understand things eventually will help you do that as well.

And the question is, the question is, like, how often is that useful as a strategy in the world? And and to me, like if you go into work and you're like i've got this great hack, I never any of the emails from my boss, I just like put that into the model and then I output anything. Eventually your boss will like have a conversation with you, say, Stephen, you don't understand anything, you have not learnt anything, and you'll get fired, right? IT just doesn't.

It's not a good long time strategy. The one place where it's tRicky is school, where there is potentially a strange incentive to bluff your way through something. We think that you know we're seeing a lot of adoption of new book element eighteen plus currently.

So it's not usable by high scholars. But um we're seeing a lot college and hired a adoption of IT. It's probably our biggest community so far. We are very excited about that nescafe.

But we are also like you know, we done a lot of things to to you know ensure that you know it's hard to use IT to bypass understanding like it's it's constant steering you back towards the regal text. You're always kind of being pointed back towards that source materials always there in the U I with you. So and they're more things we can do on that front. But that's the line that I think is like know if if you go into this with good intentions, I don't feel concerned about using the tool in this way.

okay. I buy that. And he feels like part of the line there exists in kind of what you let people make with what is coming out of IT.

And I I wonder that we've talked a lot about kind of the the inputs and the processing in the understanding and its its not you're sort of one you know published the sensor on amazon as a self published book with your name on IT away from IT being this being a very different conversation. But my sense is you're deliberately or not, not in investing a turn in the like red meal research paper from these six documents kind of use case. yeah. If you look.

look at the new book guides, like one of them is called steady guide, right? IT crazes, a study guide IT doesn't replace your work that helps you actually like no, a set of review questions that creates a little multiple choice as IT creates, like esa suggest questions that crazed a glossary of key terms. So you know, that's what we're trying to do inside of the product.

We're steering everybody towards that kind of like help me understand mode with the idea that you know, the people have different ways of understanding and there are different ways the people have like information sticks with them in three different forms. And you know, our job is to kind of do that and that's IT turns out, is writing about this a little bit this week that you know translation and summarizing and something that the models have always quite well like from the beginning, like kind of like the first deep learning breakthrough that really made a difference to consumers was really like translate. You will translate another set that was basically like set of tokens that are in this language and turn IT and set a tokens in this language. And you they are really good at at summarizing in all of its different forms. And so we've kind of like embrace that because embrace the things that the models do well is generally alright.

We have to take one more break and then we will be back with the rest of my conversation with Steven Johnson, all right back. Support for this show comes from the aclu. The aclu knows exactly what threats a second Donald trump term presents, and they are ready with the battle tested playback.

The acu took legal action against the first trumpet administration four hundred and thirty four times, and they will do IT again to protect immigrants rights, defend reproductive freedom, safeguard free speech and fight for all of our fundamental rights and freedoms. Join the aclu today to help stop the extreme project twenty twenty five agenda. Learn more at aclu dot org.

Back, okay, we have waited long enough to talk about audio overviews. Those wild A I generated podcast snow has been working on. So that is what I asked even about next.

So and speaking of that, actually that this is a good time to talk about audio overviews in the sense of different ways that people learn. Uh, would never in a million years have guessed this was going to be the next big future to come out of no book. M, so rewind the history of tell me where to from.

Yes, it's a really interesting story. Let's just describe IT briefly. IT has been all over the twitter sphere in other places. I still called twitter sorry.

OK.

since this is just A A few days ago that had launched. So basically, it's an the new book guide kind of panel instead of taking your sources that you've uploaded IT and turning them into A A brief fin dog attacks based grief ing docker and FAQ IT turns them into a ten minute long podcast conversation between two A I hosts who discuss the material, tell stories, share interesting facts, banter in a playful way with remarkable human like intonation and and all stuff.

And so it's basically the ideas like some people like to learn through reading a brief ing dog, some people like to learn and remember Better when they hear information, convey to an interesting, stimulating, engaging conversation between two people. And you couldn't create that artificially before. Now you can. So we've added IT to the sweet of of those .

that feels like IT causes a thousand new interesting things like on the one hand, it's it's not the thing that you are describing earlier in this conversation, which is something that is not full of personality. IT doesn't say I A lot. It's not this is not a tool anymore.

This is too slightly weird people hanging out in my ears for ten minute and they are like, whatever dial you have that says, like, make them say puns and try and be kind of kind if it's like that dial is at fourteen out of ten on most of the ones that i've listened to. And I kind of get why right you if you take the IT should have no personality IT should just be very short forward approach that makes sense on showing me a bunch of text and applied to a podcast that's a bad podcast. But IT does feel kind of in congress with a lot of the stuff you've been saying is like court to .

what you've been trying to do. Well, I such a great question. So it's let's just play I want to play this case.

People haven't heard IT. okay. So get this picture, a world where the hottest tech is that microchips. So the internet but dynamic.

Yeah, dynamic.

And it's not just blown stuff up, the skyscraper, railroads, massive engineering projects, but also, I you guess, the assiniboin, mpt, terrorism pts, all that volunteers.

Ff S A wild period.

IT is an understatement. And that's where our deep dive today comes in. We're cracking open Stephen Johnson's the inferno machine, a book that diggs into this explosive era where dynamite, anarchists and the birth of believe, IT or not, modern data collection all collided. Yeah.

get ready to have your mind blown because the connections he makes.

our mind blowing. absolutely.

So everything after yeah dinny just kills me.

I love, you know, one of the things is it's they are enthusiastic and so people have been upload their cbs and it's just like if they were feeling down about yourself like you're like, oh, john smith, look at to .

make you very .

a cloudy from. So I think when we first talked about what was then project talent um for the verge like you know right and I we're talking about how we don't the model doesn't have A A A A persona really like IT doesn't use the subjective. I in general, it's not trying to be your friend.

It's just try to get you the information you need. And that was that was kind of the house style that we felt was appropriate for what we are trying to build. But if you want to have a conversational audio format, there is no way to do that without they are having a sense of human personality. IT just won't work. No, no one wants to listen to two series talk to each other, right?

That is i'm certain there's a lot of that on the internet .

and it's not good. The new, but i'm sure but dollar story right know you want listen to two robots talking to each other. And initially, IT was a um another team inside of labs that had built this prototype that was a created document.

They're actually a couple of projects like this. A google is another wonderful one called illuminate that has a sliding more like kind of scholarly turn to IT. But IT was just one of these things were like IT became possible to do this.

And so some folks were expLoring this. And IT was a very good demand like we've seen from the last few days with audio overviews like people are impressed, people want to share IT, but IT IT didn't kind of have a place to live. And so kind of write before I O actually there was this idea like, well, maybe we could actually put IT into know what that makes sense inside of no book and know we were just rolling out the new book guides.

And when I first heard IT that the the podcast stuff was like, actually even Better, like the whole side names and things like that. And so, you know, but I like definitely make, you know, I think, rise. And I immediately saw that he was a continuation of the physics of the guides that we will take whatever you give us, and we will turn IT into the format that makes IT easiest for you to understand.

And we know, we know from, you know, the success of about that in general, that people do like to earn that way also. Also, they like to travel with audio and drive and, you know, walk around the city listening to IT. So we know, we knew there are a lot of reasons why audio conversation would make sense.

Are you starting to have similar conversations about things like video? I can just imagine like starting to think about what no book L M looks like when you're trying to be like tiktok native and youtube native and instagram real native is like it's just what a wild kind of road and a heading down in terms like how do we communicate to people where they are yeah but sort of in this house style this is one of the .

reasons why it's good that I am like fifty six year old grey. Heard, seen is not actually like driving things like pigs. I would be terrible at something with that. But like, yes, I think that could be part of our future.

One of the things I think is really interesting about IT in the response IT takes like four, five minutes to sometimes three to five minutes to generate one of and that's because there is a really complex series of of kind of know compute and inference is going on. basically. You can think about that kind of an editorial way, right? It's kind of drafting a version of that and then it's um filling in the details of the script and then it's revising the script based on the overall goal.

Like there's there are multiple cycles of an basically an edit cycle and that takes time. And then crucially, there's a stage where um what my favorite new word is where diflucan sees are added, right? So that IT IT deliberately makes that kind of noise and more like the the way that people talk when they are in conversation, where they overlap and they have partial phases that complete each other doses like that, all the things that make a traditional transactions to read.

If it's not been cleaned up, if you don't have those things in, it's on the wrong side of the uncle ny valley, but it's right in the middle of Kenny bali. I guess you can debate and obviously, like we're going to you can imagine we would introduce abilities to steer in different directions and dial up down, down. Things are focus on different topics. But the basic structure of kind of doing this long edit cycle and then humanizing yeah the .

thing I called wrong on this one um was that I thought audio overviews would be one of those things that people were like. All this is silly, right? It's like every time a new image generator comes out a bunch, people make wild stuff for that for forty eight hours and move on.

And we don't talk about anymore until some new and comes, uh, this one doesn't seem to have gone like that. And I think get IT somewhere in between the like is IT remarkable. That is possible verses.

Is this actually useful? I think that is more indeed, this is actually useful camp for people already than I expected IT to be like it's definitely remarkable that is possible. You shows a sort of sincerely new thing you can do with A I and whenever that happens, people get really excited on twitter, right like that. It's just one way of people really round up on two or four twelve hours. But I think there was you you overlap the this actually does something for people side of things more than I expected when I first heard, you can make an a ipod catch out of your note.

Well, I wanted thank you one for changing mind. But too, remember how jobs is to have that kind of apple is the intersection of liberals and technology, whatever like what you just described, the intersection of what is newly possible and what is genuinely useful, like that's what we're trying to do and no book and that's kind of what labs is trying to do, right? Like it's just like what can we do and what would actually be like what was unthinkable six months ago and what would would to be.

And you know, part of IT is no, I think in some cases, people are using IT like, I want to study, and this is a Better way for me to take my documents, just learn or learn on the go. The other thing is like creating things that are effectively podcast, where the no podcast would possibly ever exist in the real right. People like, here's this very obscured niche topic that not like the economics of the fog gas. This is will never support the.

I want the podcast on my d nd. Game that I play with my friends.

Ah yeah yeah, yeah no totally. And and or it's like, you know um we seen people doing like again thinking about no book and in a kind of a team contact exit work. Like okay like the weekend review, like give IT all your documents. I always say to people with the one of the best ways to get to know new book is to if you are a google docs insides drive user, just grab could know but and grab the last like twenty dogs that you've created and just create a new book with that and then just ask your questions and its its ability to kind of grasp like what you're working on the issues and things like do that in increate audio overview and it's like this is this is what even was working on for the past week you know here's too happy and enthusiastic people to discuss like the things even working on um and sharing that with your team. Like it's pretty nice like that's a great way to like look back on the week and think about what you ve been working on and and you can imagine other ways to explore that as well.

Right couple more questions. I have long time but i'll I leave you here. I could talk to you about this forever.

Yeah you're even Better than talking to bar.

Great anytime i'm here for I don't like skin me there. So um do you think there's a like big mainstream use case for something like notebook? I am obviously there are people and industries and jobs and schools like I can imagine a bunch of people for whom this is useful. Do you think there is like everyday everybody use kiss for something like this.

everybody everyday I don't know.

but I mean, be like, let me be this little different context, right? Google is a company that is famous for making things that a billion is there? Is there a, is there a billion note alam users out there?

I think if you define IT as there is an AI that is an expert in the information that is really relevant to you that you you've created and that you can engage within different forms, whether it's chat or listen to a review um and you can you can basically cultivate your own personal A I with a lot of information. Maybe it's all your journals, maybe it's you your company's history, whatever IT is.

And just by like dragging, dropping files in there, like IT develops this kind of acknowledge of everything that has happened to you or your organization and is able to kind of deliver advice or help you make decisions or just recall a fact that you're trying to remember and create these new documents. Is that a big like will maybe a billion people like be doing some variation of that in five years? I think that's pretty plausible because there are so many different things you could do in that kind of context.

Um you know I could be like a write working on a book, but IT could be an executive trying to make a complicated decision or IT could just be somebody trying to remember things that have happened in their lives. And it's like just keeping a journal only way you've got an A I that's helping you do IT. So I I I think there's a big market for IT.

okay? You don't even thinking about as as you are saying that there's this APP called my mind.

have you? It's a great APP.

You should play with IT. I think you of all people would enjoy some of the a but, uh, i've started using that APP and i've gotten the habit of basically, every time I encounter something I like, I just put IT there you like. If I love podcasting a video or a thing I read a photo I take or a good that I see, I just I just pouring IT all in there with no organization, no nothing and there's nothing particularly special about that APP.

It's just like it's pretty to look at. So like using IT um and already just the thing where you you open IT up and IT has a bunch of mode or you can just type in like movies and also show you all the movies and movie a jacon stuff you've been saving and. There's something really powerful about like here is just a competition of stuff that I find IT and there's just enough, man, you will work in that, that I think it's it's there's something to solve in the like how do you collect that data from people in a way that is both sort of easy and frictionless but also not like growth and privacy invading ah that is like the eternal google question. I have high hopes that somebody figured that .

eventually that is a bias that i've always had probably to a fault that you like, we really don't have days. We don't even have kind of fodders for your sources or your notes are things like that, like basic kind of low level things that we will no doubt ad, but we've been focused on these kind of coller features. That part of me is like the beauty like this, the complex systems of organization where you're tagging things and putting them into the right folder and suffer that are not as necessary now because the A I does that and makes the connections for you and find something that you're looking force or just have one place, make IT easy to grab as much stuff from as many different places and just dump IT into a notebook. And then what did the magic and figuring out the connections to finding the information you want after that.

Again, if you said IT correctly before, like the closest thing to this before wikipedia like I start here in this wikipedia page bt elephants and then I follow IT following these interesting parts but a dialogue is just even Better way to do that um and if if it's trustworthy and so the mode that I really love is and and again, this is only possible with conversation history and long context is to go on one of those like exploratory ory conversations through an idea and then at the end you're like, okay, this has been great. Will you format at all as A A single document that just says kind of key takeaway and insights from IT? So I can just capture IT for later because I don't nearly want to read the whole conversation again, but I I want to get the like great pieces from IT and so IT doesn't and you save that and that's you're kind of record of of what you just did like that I mean, like that's a beautiful way to to walk through information space and full of surprise and and unexpected turns and discovery so that that feels like a keeper.

What's the next step in getting Better at that? Like one one thing that that seems to me that we need pretty badly in that way of thinking, which I love like I love the idea of just like i'm to see here and spend three hours learning about something and then at the end going to deliver me like a handy summer. If everything amazing, that's a trip.

IT does drink me that one thing. We desperately need all of these tools to get Better out, to pull that off his multimedia, right? Like, I wanna know at the end of IT like listen these three podcasts in this youtube, you're going you're gna love and go check that out and this personal instagram soon there's like there's the internet is such a messy place in the best way now that he feels like that's one thing.

And I know a lot of folks are working on that. Uh, are there other things that you look at, you know you you're working on this stuff to? Is there are next kind of turn coming that's going to make .

all that have even Better right now? No book's ability to help you discover information to put in your new book is exactly zero but no nothing in that you are on your own. You need this of life resources um we will not help you discover anything um across the intern in any form IT turns out just as a coincident that google is really good at that stuff like I did not know, I learned them when I got that.

They have the search thing that is of a lot of people use. So it's obviously like a place where, you know, I would love that I wanted to be source grounded, but I would love you to stay in the book to be able to find things. And you know, we we would be really interested in in expLoring. And I think you'll see that in twenty twenty five for sure. right?

That is IT for the verge. Yeah, thanks again even for being on the show. And thank you, as always for listening. There's lots more on everything we talked about at the vegetable com. I'll puts some links in the shower notes to some of the of Stephen has written about this to his new book is really great. Go to the version that com, lots of shown notes, lots of no book coverage, oh, kinds of good stuff.

As always, if you have thoughts, questions, feelings or other ideas for air generator podcasts, you can always email us at verge cast at first shot com or c six six verge one we love caring from you want to hear our thoughts sender mall can't wait to hear this shows produced by liam James s. Wilton and arena cases production part of the box media. Podgorica will be back with our regularly scheduled programing on tuesday and friday. This week. We have a lot of headsets in particular to talk about this week, so get ready, see them back.

Support for this episode de comes from A W S A W S gender A A I gives you the tools to power your business forward with the security and speed of the world's most experienced club.