Transformers, explained: Understand the model behind ChatGPT

Key Takeaways

hi everyone and welcome to this lesson of GPT in a nutshell where I'm basically going to try and make you understand exactly how GPT models work by understanding their underlying architecture which is Transformers so usually you would need a master's degree or years of studying to try and understand these so I'm going to try and attempt to teach you a summary of this in just one quick video let's go so for years scientists have been trying to model the human brain and the reason for that is because if we are able to model the human brain in a digital form we'll be able to process complex data such as language or images or video or even sound and music and stuff like that so for years they've been trying to develop what you call artificial intelligence so that you are able to solve these more complex problems so a human brain kind of looks like this where you have a whole lot of neurons in these yellow dots and then they're all connected by synapses which transfer information from one neuron to another so they were able to digitize this with something called an artificial neural network also um referred to as Ann which basically has all these nodes which represent neurons and those sinapsis are represented by these weighted edges which basically contain a number or a parameter value so as you can see here the blue the darker the blue the more negative the number and the more red the more positive the number so it starts off with an input layer and that is basically these nodes are receiving information from the outside world all right so that's like call it input data and then it passes through a whole lot of hidden layers right in this case this a&n has two hidden layers and then it gets passed to the output layer which is usually something like a prediction so with this spefic specific model you're able to solve a whole lot of complex problems and a whole field of study emerge from this called Deep learning and it's called Deep learning because the same way the human mind learns from its surroundings these models also learn so this process of assigning weights or parameters between all these different nodes is called learning and with learning you are able to build more intelligent models the larger the base model size the larger the memory so if we look at these examples of different language models we have gpt1 which was developed back in 2018 then we have gpt2 which was more intelligent gpt3 which was even more intelligent and GPT 4 which is even more intelligent and the intelligence of these models are directly proportional to the amount of parameters or those weighted connections between the nodes so the more of those parameters that you have the larger the model and as a result it's able to Lear learn more and it performs much better and it can solve much more complicated tasks so we can see back gpt1 was a tiny little dot only had 117 million parameters gpt2 had 1.5 billion gpt3 had 175 billion then GPT 4 according to a few resources online I found that it has 175 trillion parameters don't quote me on that because open AI haven't publicly said how many parameters it was trained on but this is what they estimate an expert estimate online so how do these large language models get trained how do they actually learn all right so in this case of a large language model it's trained on a whole lot of input data and this input data is literally just text in raw format gathered from resources like the internet like Wikipedia or books and they take all that text and it basically trains on that data and this is how it works so so it'll take the input training as data for example let's say one sentence in that massive Corpus of text said something like once upon a time and a land Far Far Away blah blah blah what it then does the model is set with initial random parameters so remember we saw over here there's like 117 million or 175 trillion parameters so this would go on to like four trillions of different parameters and they're all randomly initialized in this case they're all just set to one okay then what the model does is it tries to guess the next word okay because remember how these neural networks work it tries to predict an outcome by going through all those hidden layers and it tries to make a prediction so it'll take say for example the word once and it then needs to predict the next word so let's say for example the model then predicts the next word that comes after once is the word there all right here's the word there then what the model does is it tries to calculate the difference between the word that it predicted and the word that it should have predicted based on the training data so what I have here in step four it calculates a difference between the predicted word there and the actual word in the training data upon so if we look at the training data it says Once Upon all right so it predicted there but it should have been upon so what it does is it tries to calculate the difference mathematically it's able to quantify these words using math and it tries to minimize what they call a loss function fun so don't worry too much about how that actually works but basically they're trying to calculate the difference between what was predicted and what should have been predicted based on the next word and what it then does is it auto adjusts the parameters so for example in this case it adjusted parameter number three to 12 and parameter 6 to minus5 now as a hum these parameters we aren't able to interpret them but the model by adjusting these parameters well now the next time it tries to predict what word comes after the word once it'll have a higher probability of choosing the word upon as opposed to choosing the word there because these parameters have now been adjusted now as it runs through these uh the more and more training data and adjusts more and more of these parameters these models get smarter and become better at predicting what the next word should be based on the training data all right so GPT stands for generative pre-trained Transformer so the generative part means that it refers to generative AI which involves creating or generating new content for example generating text generating images generating videos generating music Etc then we have pre-trained and pre-trained means the model was previously trained on data and then lastly we have Transformer and and this refers to the model's architecture called a Transformer and this is what a Transformer looks like all right this is just a diagram of what it looks like internally and this was the biggest breakthrough of what made chat GPT possible this was developed in a scientific research paper called attention is all you need from 2017 and from that research paper gpt1 was then developed in 2018 and then ever since then these models have just getting better and better the same Transformer architecture is what Google use in their AI called Bard theirs is called Palm uh instead of GPT they use Palm but it's also based on the Transformer architecture so this Transformer architecture sits between two layers called tokenization and dett tokenization and I'm going to explain every step through this whole process and it's quite interesting to actually understand how these models work in internally so the first step is tokenization so what is that tokenization is converting words to numbers all right also referred to as token IDs because remember these are mathematical models so in order to understand these words uh you convert those words into numbers which are much easier to process so this is what tokenization looks like you get something called a tokenizer which converts text fragments into token ID s and if you want to experiment with a tokenizer you can go to platform. open.com tokenizer that's where I got the screenshot from and what it does is it uses what you call a tokenizer dictionary to correspond the words or tokens to numbers which is token IDs so this is what the dictionary would look like okay and we' have every single word and this goes on for about a trillion there's no reference to how many tokens there are in open air I tokenize a dictionary or GPT tokenize a dictionary but there's billions if not trillions of them so for example the word the with a lowercase T because this is all case sensitive refers to token ID 1,169 and that's static the letter a lowercase refers to the number 64 the word and is 322 the word hello with a capital H is 15492 ETC all right so then when these models take as input for example the fisherman caught the fish with a net what it does is it then converts this text into an array of token IDs so it'll take the and assign it to 464 notice it's a capital T so that's why it's not this number the with a capital t is 464 and that's where it gets the value from then the word fisherman and it's that number then the word Court the fish with the net even full stop over here references token ID number 13 now that's tokenization we've now gone through the step from converting the words into token IDs or numbers the next step is to convert these token IDs into what you call a vector okay and that is done in this embedding stage over here okay so a token embedding is a high-dimensional space where each token is mapped to a unique position called a vector so when I say hi dimensional as a human we only able to visualize three dimensions all right maximum three dimensions we we can visualize two dimensions and we can even visualize three dimensions in this case we have a zed Dimension X Dimension and Y Dimension just like real life we look around everything's in three dimensions so this is just a diagram to represent what a token embedding would look like but in reality there's a lot more there's a few hundred Dimensions so in this space each token ID is mapped to a position in space and these are also predetermined just like the tokenizer dictionary this high-dimensional space is a static space where every token exists somewhere in space all right and that's how you're able to quantify a word to a number so if we look at the word fish and fishermen because they both have to do with fish they're quite similar so in the embeded space they're quite close to each other the word net is also kind of close to fish and fisherman it's a little bit further away but usually you can catch fish with the net so that's why they might be close to each other then you might have the word ball which maybe doesn't have anything to do with fish and fishermen so it's a little bit further away but the word ball and C can be closer together or the word caught and fish and fisherman are also kind of close together because you can cat catch fish then you have things like sound now sound is very far away from maybe the word but it's Clos to to the word ball because balls make a sound possibly if you kick it or something like that so basically all these words represent a position in space relative to every other word and the closer they are the higher semantic similarity they have with each other all right so here I have an example with the word fish and in this specific example the coordinate of the fish in the X Y and Z plane is 12.5 so whatever unit that is let's say 12.5 M all right it's not actually meters but let's say 12.5 M and then Y is 9.4 M and then Zed is 3.2 M high and that's the position in space that it represents but remember like I said it's a high dimensional space so for example in the tension is all you need paper they use 512 dimensions and I know chat GPT they use a much larger amount of dimensions in their embedding space all right so that is how you're able to mathematically understand language you're able to calculate the distance between two vectors using the sum of the absolute difference between elements at position I in each Vector so let me demonstrate an example of this let's take the word cat and dog right and this is what the vector looks like let's assume there's 512 Dimensions so instead of just having three 1 2 3 like we had before this carries on comma another number comma another number comma and this carries on for 500 112 Dimensions now what you can do is you can subtract two vectors one vector minus another Vector you'll be able to quantify the distance between those two vectors and then you'll be able to know how similar or how different those words are so for example if you take the word cat and dog because they're both domestic pets they'll probably be quite similar to each other quite close to each other in the vector embedding space so the way you'd calculate that you'd take 1.5 minus 1.7 and then you'd say plus you'd say Min - 0.4 plus oh sorry minus - 0.3 + 7.2 minus 6.9 + 19.6 Etc right so what would happen is because these numbers are quite similar and you sum all these numbers together that number is not going to be too large because they're quite close together whereas if you try to calculate the distance between the word cat and PS4 otes because they're quite different things you can see even just by looking at these numbers they're much higher numbers as opposed to these they're not they're much different so because of that the distance between those two points in space are going to be much higher and that is how you mathematically are able to understand language so now that we have embeddings we can then use those embeddings to calculate the position of every token in the sequence of tokens so this is what you call positional embeddings so that PE over here stands for positional embedding and this is what it looks like so positional embeddings there's a vector for every single token and this is calculated with some fancy calculations using S and cosine and what this basically represents is the position of each word in a prompt relative to all the other words so this way it's able to understand context so this is why these GPT models are so good at understanding language because in language the order of the words are very important in order to translate it if you try translate English into Greek you can't just translate the first word or translate each word and then just uh linearly uh translate each word because the grammar won't make sense so with understanding the position of each word it's able to actually translate it much better and that's why if you think back in the day with Google Translate Google Translate wasn't really good at translating languages because it was kind of translating each word sequentially and it didn't understand grammar or not just grammar just in general word order for different languages whereas with positional embeddings it made that possible so what the model then does it takes the token embedding which we learned which is a position in space of that token ID and it adds it with a positional embedding all right so you're basically summing two vectors together and then you get this final input embedding over here all right so this Vector basically just represents a position in space that contains both the information of what the token means the definition for example the word net that's a token embedding and then the positional embedding it also represents that which is where the word net is relative to the other words in the prompt or in the input text so this was quite a big breakthrough these positional embeddings because like I said it it allowed the model to understand the position of words relative to other words in an input it then takes those final embeddings and it gets fed into what you call the self attention layer all right inside the encoder and decoder and what this self attention does is it assigns weights to every word relative to every other word so let me show you what that looks like so let's take for example the sentence the fisherman caught the fish with a net let's say this is the prompt okay uh it's not really a question but let's just assume it's a prompt what it does is it assigns parameters remember what I showed you earlier with the neural network where those lines in between the different nodes are assigned different weights uh you also call those weights parameters it's just a synonym so between each token and every other token so you can see it will assign maybe a light line between the and the word the but a thicker line between the word the and the word fisherman because it's basically saying there's a a strong importance between the word the and fisherman so it know n model knows the thing that the the article the is referring to the noun fisherman and then it assigns another way between the and Court which is a bit thinner it's not as important as fisherman and so on and then it assigns all these weights now it doesn't just do that with the first word it does that with every single word relative to every single other word in the entire prompt okay and then what you get is you get this net of all these parameters between every word and a prompt relative to every other word and these are all weights and numbers okay and with that you're able to understand the importance of the context of what the prompt is saying and now it doesn't just do this once it does this multiple different times and that's why it's called multiheaded self attention if we look here remember the self attention layer had multiple different layers and that's because it's multi-headed and by multi-headed what I mean is that each self attention head so this is one self attention head this this is another self attention head Etc and usually there'll be somewhere of a range of 12 to 100 of these self attention heads and each one represents something different for example in this one the parameters or these weights might be based on the people relationships between the different words this head might be based on the activities that are performed this one might be based on Words that rhyme so words that rhyme might have a higher or a thicker line or higher parameter value between them and others may not be as easy to interpret so that's something important to note these different self attention heads on something that the developers pre-programmed beforehand the AI basically just creates these different self attention heads that represent different things in the neural network so it's it's usually quite difficult to actually analyze these different self attention heads and figure out what they mean the next step step five is it predicts the next word by assigning scores for each word so that's how these models work as I mentioned earlier the same way how the models get trained it tries to predict the next word that's how it basically runs I'm sure you've seen when you run a GPT model and you click submit it starts generating one word at a time that's because it's busy predicting the next word as it goes so this is done in the feed forward Network so this feed forward Network it takes all of the attention weights as input from the previous step and then what it does it makes a prediction for what it thinks the next word could be based on its training data and it does this for every possible word it knows in its vocabulary which is that token dictionary that I showed you earlier so these predictions are not probabilities yet they're just scores or what they call logits and these scores together form a vector of logits right now all these scores they basically just numbers and the and they're in proportion to the probability of which word should be chos and next so in simple terms a vector of logits is a list of scores that the model gives to each possible next word in a sentence so for example if the model is trying to predict the next word in the sentence the cat is the vector of logit is an array like this for example it'll open square bracket 2.3 comma 4.97 comma 3584 point21 Etc and this carries on for all the trillions or however many tokens there are in the tokenizer dictionary and what it would do it might give a high score to the Token for example sleeping a medium score to the Token running and a low score to the Token Apple Etc and this because that's a word that would come after the phrase the cat is so it's more likely that the cat is sleeping as opposed to the cat is Apple so these scores are then called logits what then happens those logits or scores get fed into what you call the soft Max output and it converts those scores into probabilities so the model then turns the scores into probabilities between Z and one using a mathematical function called Soft Max so that they all add up to one so for example this is what the um Vector would look like and all these numbers instead of being scores are now a probability of likelihood of what the next word should be this gives a probability distribution over all possible next words and the word with the highest probability is chosen as the prediction once we've got that Vector of probabilities that Vector of probability then gets converted to token IDs or words all right so that's what you call DET tokenization it's the opposite of tokenization where it takes the token ID and it converts it back into a word so how does it do this if we look at the probability Vector as we had in the previous case index zero over here would refer to token ID number one and token ID number one refers to a specific word all right or a token index position one which is this next one over here uh refers to this probability and that's token ID number two Etc so for example let's assume that the highest probability was 01244 which when you multiply that by 100 is 12.44% so let's assume it was in index position 50,4 And1 then that index position is a token ID then the output token ID in the token dictionary that corresponds to 5041 would return the token sleeping and this is called DET tokenization then the response would then be the cat is sleeping and then this reruns steps 2 till 6 to predict every next word so what it it will then do it will take this whole input and it'll run it through the Transformer again and it'll then predict the word that comes up to the that it'll then take that full sentence run it through the transform again and predict the next word and that is GPT in a nutshell