ChatGPT for Absolute Beginners - What is it and how does it work?
24:08 Share

This episode is brought to you by Shopify. Forget the frustration of picking commerce platforms when you switch your business to Shopify, the global commerce platform that supercharges your selling wherever you sell. With Shopify, you'll harness the same intuitive features, trusted apps, and powerful analytics used by the world's leading brands. Sign up today for your $1 per month trial period at shopify.com slash tech, all lowercase. That's shopify.com slash tech.

This episode is brought to you by Jira. Jira is the only project management tool you need to plan and track work across any team. So if you're a team of developers, Jira better connects you with teams like marketing and design so you have all the information you need in one place. Plus, their AI helps you knock out the small stuff so you can focus on delivering your best work. Get started on your next big idea today in Jira. Hey everybody, David Shapiro here with another video.

It occurred to me that there's a lot of people that are new to GPT and chat GPT and so you might have questions about how it works. Before we get started in the video, I wanted to direct your attention to my Patreon page. Now, I put a lot of content out entirely for free. I want to help make the world a better place.

by sharing my knowledge. And in exchange, I'm hoping that I can get a little bit more support for the work that I do. So if you find my content valuable, please jump over to Patreon and consider supporting me. One advantage that you get, or there's two advantages that you get for supporting me on Patreon. One, you get access to my exclusive blog. And two, I've started uploading Patreon exclusive videos. So with all that being said,

Let's go ahead and jump into today's presentation. What is ChatGPT and how does it work?

So, ChatGPT was built by a company called OpenAI. OpenAI was established a few years back as an open source consortium, or not really a consortium, just a company, an open source company with the goal of creating safe AGI. That was the primary original purpose of OpenAI. Now, it has since changed. It is now closed source, and it is also for-profit.

So, obviously, it has gotten some criticism for this because it kind of runs contrary to its founding purpose. But they do still release open source code every now and then. Okay, well, first, before we get into chat GPT, we have to answer the question, what is GPT? GPT means Generative Pre-Chained Transformer.

It is a technology that basically just reads and generates text. That is the long and short of what GPT does. It was trained, it's a deep neural network that is trained to predict the next token. Now, a token is just a fancy way of saying a few characters. And of course, you put characters together to make words. Everything that you read on a page is a series of characters, new line periods, spaces, letters, and so on.

So it's a little bit deceptive that it was only trained to predict the next character. Now, you might say, okay, well, how is it so powerful if that's all that it does? So the reason that it's so powerful is that because in order to predict the next character accurately, you need to have a lot of knowledge and other capabilities. And that is what it has learned to embed in its neural network over time.

And so you might hear that word embedded. You might also hear latent space. We'll talk about that in a couple slides. So for instance, it knows how to write a checklist by virtue of having read millions of checklists. It also knows how to write code because it's read lots of code. And so just by virtue of predicting what comes next, it learns to figure out what comes next. It was trained on billions and billions of tokens.

Now, the easiest way to think about it is that it is an autocomplete engine like what you might have on your phone, but it's on steroids. It doesn't just predict the next word kind of, you know, stochastically. Some people do say that it's a stochastic engine. Not really. It's a little bit more complicated than that.

But it is an autocomplete engine on steroids. Now you might say that a human brain is an autocomplete engine too because we have the ability to predict and generate patterns. But that's a topic for another video. The next thing that you need to know is that GPT comes in flavors. So there's two kinds of flavors that it comes in. One is that it comes in different sizes. So there's larger ones and there's smaller ones and it's all measured by the parameter count.

So the parameter count is basically the number of connections inside of that neural network. And there was a paper that came out a year or two ago that said that for these neural networks, it's roughly a thousand parameters in a deep neural network are equivalent to the processing power of one human neuron.

So with GPT, the largest model that we know of being 176 billion parameters, this is roughly equivalent to 176 million neurons in a human brain. So it's still much smaller than a human brain in terms of raw processing power. And that, of course, is if that paper holds up. We typically adjust how we think of human brain power over time.

Now GPT, you know, okay, 176 billion parameters. How much compute power does it take to run?

It takes roughly 700 gigabytes of VRAM. I think it's 768 or something like that. So that's roughly 90 Xboxes or maybe up to 100 Xbox Ones in order to run ChatGPT or GPT. Now, that's not necessarily what they're using, but that's just a rough approximation. Again, you know, take it with a grain of salt. It could be more, could be less. Oops.

So, there's flavors in terms of size, but there's also flavors in terms of what they are trained to do or fine-tuned to do. So what I mean by that is that the original model that was trained just to predict the next token is what we call the vanilla model or the foundation model. Now foundation models are really powerful, but they also tend to go off the rails.

And that is because they just predict the next token and it's very haphazard. They're not trained to do any one thing. They just predict the next tokens and it allows them to confabulate very deeply or kind of invent their own tasks. And so what we've done is we've had fine-tuned datasets

that have given us Codex, Instruct, and now ChatGPT. And we'll talk about how these fine-tuned data sets are created in just a moment. But the key thing to know about fine-tuning

is that it's based on a technology called transfer learning, where you have the pre-trained model, and you take it apart, and you slap a new layer or two on the end, and then you train it on that one new task with a new data set. But here's the thing, rather than training it on billions of tokens, it only takes a few thousand to train it on one specific new task. Thousands or millions. Give me just a moment.

Sorry, I've been sick and I'm recovering, so I'm drinking Pedialyte. Okay, so it comes in flavors. Now, how is the ChatGPT flavor created? The ChatGPT flavor uses RLHF, or reinforcement learning with human feedback. So the way that this is trained

is that you have a reinforcement learning model that uses a signal from humans that basically say, like, it gives, it tries a generation, it generates some text and asks you, did you like this, yes or no? And people say yes or no, good or bad. And the RLHF model then learns to predict what people will want.

So then once you have a reinforcement learning model that can accurately predict what people will want It'll it basically just says will you like this? Yes or no that allows you to then label lots and lots and lots of data automatically very quickly And so what they did was they used that this method RLHF to create a new data set so above and beyond instruct and codecs which are what most people are familiar with and

Now there's a data set for ChatGPT. And so basically, I've got it right here at the end, people preferred long thorough responses. And so that's how ChatGPT learned to communicate. Just by virtue of it gave a response and people gave it a thumbs up or a thumbs down. And that is the direction that it went. Now, how does its memory work?

because one of the most remarkable things about chat gpt is that you can have pretty long conversations with it and it seems like it has a pretty long memory this is one of the things that makes it very powerful so one thing to know about gpt technologies is that it has a window size and so the window size is the total amount of text that it can read and generate so for instance uh text da vinci uh o3 one of the larger models right now has a window size of 4 000 tokens

ChatGPT is rumored to have a token window size of 8,000 tokens. So when you think that it's three to four tokens per word on average, that includes spaces and hyphens and white space, that equates to about 10 pages of text, give or take. Could be 15, depends on what's on the page. So the most obvious way

is that ChatGPT just has what's called a rolling window.

where it reads the last 10 pages of your chat, which if you have a short chat, that means it can read the entire chat log and continue the conversation just as the same with the same paradigm of autocomplete, of just predicting the next text because it has read lots and lots of chat logs and it has a particular pattern that it follows where it gives very, very verbose responses.

Now, there's a couple other possibilities of how its memory works. Again, OpenAI is no longer open. This is closed source, and so it's proprietary technology. It's also for-profit technology. So this last part is pure speculation on the part of the AI community. It might use search or a scratchpad. So what we mean by search is that every chat log that you give it is searchable. So once you have very long chat conversations,

it's not going to fit in the window anymore. But based on what's going on currently in the conversation, it could use that to queue up and look back in older messages in the conversation to figure out what's going on. It could also use what's called a scratch pad, which is basically a running summary that holds out on the side

or an ongoing summarization that holds on the side that it can use to keep track of information regardless of how far back it goes. Again, this is entirely hypothetical. We're not sure if that's what it does. There's another possibility that I didn't put on here that I want to try and implement, which is that it can build a knowledge graph of the conversation as you go.

which means that it's constantly updating and keeping track of new topics and stuff, and then it can traverse that knowledge graph and extract information from it as it goes. I don't think that it does that, but a future version absolutely could. Now, why is this so powerful? Why is ChatGPT so incredibly powerful that it has taken the internet by storm?

Well, one of the things to keep in mind is that it has a lot of latent space. By virtue of the fact that it has read a significant chunk of the internet means that we don't even know what it knows. We have a good idea of what it knows, but we don't even have good benchmarks about how to measure the power of these models. In fact, there are new benchmarks coming out all the time because old NLP benchmarks don't really matter. It doesn't measure intelligence the right way.

Because what we have now, those old benchmarks were for NLP, natural language processing. What we're doing now is called natural language understanding and natural language generation. So it's an entirely different paradigm category of technology. So this latent space or these embeddings is what I mentioned earlier in that by virtue of figuring out what it takes to predict the next token, it has also embedded a lot of knowledge or has a lot of latent capabilities.

So that is one aspect of why it's so powerful. Another aspect of why it's so powerful is because your brain is interacting with a machine. So it's kind of like a utility droid, which is why I picked a picture of R2-D2. R2-D2 on his own doesn't really do that much, but he has capabilities that you don't. And similarly, ChatGPT has capabilities that you don't. And so you complement each other.

And so your brain has better, faster memory than ChatGPT, so you can remember what's going on in the conversation. You also have the ability to spontaneously come up with directives. ChatGPT does not. So ChatGPT does a different kind of work than you do, but it does it faster. And so by doing that different kind of work and doing it faster, it takes a lot of mental work off your shoulders, which is why it is so powerful.

Same idea behind having a utility droid is it can do something that you can't, like R2D2 can hack into computers. It does it much faster than any human can, and then R2D2 will follow you around. So is ChatGPT going to evolve into R2D2? Maybe. That'd be kind of cool. Now, let's talk about what has ChatGPT changed for us. The biggest thing is that ChatGPT is the first AI technology to take the world by storm. It is the biggest proof that AI is ready.

So the first thing that's going to happen is a lot of investment. So what I mean by this is that once a technology is commercially ready, once it's commercially viable, then you get a lot of money being put into it. We saw the same thing with electric vehicles and solar power, because for the longest time, things like EVs and solar were not cost effective. But now they are. And of course, there's a little bit of debate over whether or not EVs are actually cost effective, but solar absolutely is.

which is why the rate of investment in solar is accelerating. And so now that the world knows that AI is real and it works because chat GPT is easy to use and the value is obvious, the money is coming.

Okay, so that's the first thing that it changes. It will take a little while to prove it out. And it will take a little while to implement and deploy it because ChatGPT is just a prototype. It's not ready for commercial purposes yet. It's very useful as it is. I use it all the time. And this is just version one. Imagine version two or version 10. It's going to get exponentially more powerful. Now there's a lot of other problems to solve though.

Mostly safety. How do we use it correctly? How do we use it without doing any harm? How do we make sure that it is not going to do more damage than good?

So there's a lot of improvements that need to be made. This is the big thing that changes is, so there's the window size, the length of memory, some of those topics I already mentioned. It needs to be able to follow instructions a little bit better because sometimes if you use ChatGPT, you might notice that it kind of gets stuck in a rut still where you can correct it and say, no, this isn't the way to do it. Sometimes it'll listen, sometimes it won't.

And one of the biggest things is going to be integrations with external sources of information or other APIs, because right now it's self-contained in a tiny little bottle. But one of the biggest things that technologies like ChatGPT could change is that everything might go faster. All science, all education, all creativity, all business, everything could go faster because of the cognitive offload that this technology offers.

All right, so what are the limitations and downsides of ChatGPT? First and foremost, it's expensive to run. As we mentioned earlier, it would require about 90 Xbox Ones to run it. Obviously, that's not what they're using. The computers to run these are very expensive though. And OpenAI is not open about it. That's another big downside. But because this technology is so valuable, there are many, many up and coming competitors.

So that is, we're gonna see a huge investment in 2023 in people trying to make clones of ChatGPT. I have a video series where I started this and there's dozens, hundreds of other people already working on ChatGPT clones.

There is a huge potential for disruption, such as lost jobs, new jobs coming out too, and even new ways of living. And the biggest downside is probably going to be safety and privacy, such as data security. The conversations that you have with ChatGPT, if they get leaked, it could be used against you. Or at the very least, it could be very embarrassing. All right, last slide. What's next in 2023? Well, we're off to the races.

2023 is going to be the first year of the singularity. Mark my words, we are going to remember 2023 as the year that the singularity began. Another more boring term for that is the fourth industrial revolution.

There's going to be lots of money being invested in these technologies, lots of new products and services. We're going to see a lot of change very quickly because we're at a tipping point. So if you look back through time, from the introduction of mass-produced cars, it took, I think it was 14 years from the introduction of mass-produced cars to where basically horses were not used anymore.

We're going to see change even faster than that because we're at a tipping point. And because these technologies are very quick and easy to deploy, relatively speaking, you know, building a million cars takes a long time. Getting a million users on ChatGPT took three days. So the rate of change is going to be very fast and it's very difficult to predict where we're going to be a year from now in the first week of January in 2024.

Alright, well that's it. Thanks for watching. Again, please consider supporting me on Patreon. It's patreon.com slash Dave Schapp. My goal is to be able to do this full time so that I can continue putting out content for free. Thanks for watching and have a good one.