Explained simply: How does ChatGPT actually work?

Key Takeaways

chat GPT explained simply so technically chat GPT is just the website that we see here instead what we really care about is the engine or the model that drives the website and that is called GPT 3.5 3.5 is just a version number we can ignore that what we really care about is this GPT stands for generative pre-trained Transformer and this is basically a fancy word for a model that is trained to generate content but still what does this actually mean to explain let me go back to chat gbt and ask how do I start learning to code now when you enter in an input the input will go through two steps encoding and decoding and these two steps are what make up the Transformer model now encoding is basically the fancy word for breaking down into a smaller sentence possible to use an analogy think of Kevin from the office so me think why waste time say lot word when viewer do track yes so the aim of the game is to try reduce this into less words for example how learn code it's not grammatically correct but it conveys the same meaning as the original prompt each word is then given some mathematical values so that a computer can process them and by simplifying this this helps the model work out what is the context and the true meaning of the prompt this is where we get to the decoding stage which is a fancy process for evaluate context and predict best sequence of words as response now computers don't actually understand words in the same way that humans naturally understand them so to work out the context we have to use a technique called self-attention This is basically a complex mathematical formula to work out the importance of each word to the meaning of the sentence in this case the word code is the most important token because it sets the topic of the sentence and once the computer knows that the topic is about code it can then know what the words how and learn mean because they are the words that surround the word code and once the model has got in a context it then produces a list of words which together read out as a sentence and these words have the highest rate of relevance to the subject matter as well as to each other remember the model sees all these words as numbers and so it can work out whether a particular word is relevant to the context in terms of percentages and the model has already been trained to know what these percentages are hence why it's called pre-trained and basically GPT was trained on the entire internet so every dialogue every sentence on the internet has been broken down into either an input such as a question or an output such as an answer and gbt was exposed to billions of these questions and answer Pairs and that's how it learns which answers best relate to which questions and vice versa and after this process GPT it was then put through another training process where it got a question that had four possible answers like a multiple choice question and gbt had to guess which one was the correct answer if it got the answer correct it got a point if it didn't it got penalized and basically gbt learned a strategy to find the correct answer for most of the time that's basically how chat gbt Works follow me for more simple AI tutorials