Model Size and Hyperparameters Explained

Hello and welcome to this video. In this video, I want to explain you one simple concept in AI which is kind of overlooked but does mean a lot of things that is model sizes and hyperparameter. But I'm not going to explain you the technical details of what is hyperparameter. Let's take one step back. You all know hugging phase, right? Hugging phase is like GitHub but for models. If you look into the hugging phase, just go to hugging phase UI, open any of the model page, you want to find one parameter that looks interesting. That is model size is equal to 12b [music] or model size is equal to 1 or 2B. What is model size? What is 1 or 2B? Does model size mean it is the size of the model when you download it? And what is 1 or 2B mean? What does B mean here? I'm going to explain you this in detail in this whiteboarding session. So let us get started. [music] The concept that we want to understand today is the model size. So model size is equal to let's say 12b. This is exactly what you're going to see in the hugging phase UI if you open any model. So the question now we have is what is B here? What is the number 12 here? And what does the model size really mean? This is what we want to understand in this particular video. And I'm going to explain this in detail for you. Let's start with this one. So 12 and B. Let's let's try to understand this particular concept in bit more detail. And then I'm going to come to the model size. Now 12b here. Let's start with B. Now B here is billions. Okay. And then 12 of course is a variable or a number. That's it. But what billions? What exactly is this billion? Is it is it billion values, billion data? What is it? Well, in fact, this is a billion parameter or in other words also known as hyperparameter. That is what this B means. Now, what is hyperparameter? Imagine this. And think of a Formula 1 car. It is assembled every time before the race. Why do they do that? Because in every race the race track is different. So the car has to be fine-tuned to that particular track, that particular humidity and the fuel usage of that particular week. Of course, how long the track is. Now each Formula 1 car would have thousands of small parts and each one of it is fine-tuned every single time in every single race. Same concept apply to the ML model. ML model is trained on hundreds of data and hence it also has these knobs what we call as hyperparameter which has to be tuned in a way so that the model gives you the best result. Now what is the best result that comes out of the training of course but there are hundreds of parameters but in this case we have billions of parameters. Would you believe it? Well you should because the topmost frontier models have these billions of parameters in them. So now we understood what is this B mean and that means billions and billions of parameter. How many billions of parameters? You took a right guess. That's 12 billion hyperparameter tuning has been done on this given model. That's what 12b here means. Now let's take the example of a very famous model that is deepseek. So deepseek v3 has an hyperparameter tuning of 82b. Okay. And the llama models usually have a hyperparameter tuning of 8b. Now you know when I say 82b it's 82 billion and when I say llama is equal to 8b it's 8 billion. Now the thing is you as a end user usually won't get to fine-tune this. Okay. If you're thinking of models in chart GPT or for example cloud these hyperparameters tuning are mentioned like it does mention on how many parameters it has been tuned but you never get to see those parameters nor you get to fine-tune those parameters only in the case of open weight and open source you get to see these parameters. Now open weight and open source means different thing. If you're curious about it, let us know in the comment box and we will create a separate video explaining what is open source and what is open weight exactly. But now this 82B and 8B usually are mentioned up front in hugging phase. As I mentioned hugging phase is a place for ML models just like how we store code in GitHub. So all the model parameters are actually displayed here and you'll be able to see the 82B or 8B. That's the wonderful part of it, right? So deepse v3 was trained on 82 billion parameter. Can you imagine? And usually when you get these models, these models would have already been locked with these parameters. You don't get to tune them. But certain parameters can be tuned and that is the difference between fixed and actual parameters. Now that we understood what is this 12b or what does this b mean? What does model size mean now? Okay. Does it mean 12 billion bytes? Does it mean 12 billion bits? What what is what is a model size here? We need to understand this one. For that we need to look into one more concept in hugging phase where if you look into the hugging phase model again beside the model size is equal to 12b you find something interesting which is called as tensor type. Tensor type is a precision type that explains how many bytes were used for each hyperparameter. I'm going to repeat again. Tensor type says how many bytes were used for each of these hyperparameters. There are different kinds of tensor type. For example, FP 32, FP16. Then there is for example BP16 etc. Now they define how many bytes or the tensor type or the precision type each of these parameters are. So they are linked. Okay. So when you see the 12B size that says okay this is the hyperparameter side but to get the actual model size how big the model is when you download it you need the tensor type. The tensor type is what is used to finally get the size of the model. Let's take a very simple example to make this calculation much much easy for us. We'll take an example of llama scout model which is having 1.9b if I'm not wrong that was trained on 1.9 billion hyperparameter. Now if you were to come with a calculation of how big this model is let's go with bp16 as a tensor type. Now was this model built to support this? You can again see this in the hugging phase website if the model allows you to download this in the BP16 format. Now BP16 or BF16 is actually two bytes. So it says it is two bytes per hyperparameter. The way you're going to calculate this is now very simple. You take BF16 and you're going to take the 1 or six bytes that we have shown. So let me highlight that now. So you take this one and you take this one and you are just going to multiply both. So what will happen is we have 109 billion parameters and each one of them is BF16 format that is two bytes. Okay. So this calculation turns out to be how much? Can you take a good guess? This calculation turns out to be 218 GB if I'm not wrong. That is the size of the model. Now you would need a GPU and a CPU that supports 218GB of model size to run this particular model. Isn't that great? Because if you think of it inference that does happen with Chad GPT or cloud, we never think about this. But the moment you want to build something yourself, the moment you want to download an open source model and run it yourself, which many of us have thought about and you might be thinking why can't I simply run this model locally, look at the complexity around this. It is not that straightforward that we can do. But I am very sure in the near future we will have models that are capable of running in our mobile phones and laptop. And you might say like we already have it now. Well, we do but they don't have the precision as we expect, right? So we are moving in the direction and very soon in my opinion not even an year from now we will have these models capable of running them on our laptop. How do you run this in real life? As of now if you were to do it in fact deep can be downloaded llama can be downloaded locally you can run it. What do you need to run it? As I mentioned you need GPUs and that's the war of the GPU. That's where people are fighting about GPUs. That is why we need GPUs to run these models. If you're doing inference, if you're trying to build something from scratch, they need GPUs to run and do inference for you. Well, let me tell you where we started. We started with a simple question. What is a model size is equal to 12b? When you open hugging phase website and click on a model. This one little line took us into this journey of understanding what is parameter, what is tensor type and end up with knowing what is a model size of a given model. I hope you enjoyed this whiteboarding session. If you have any questions, you can let us know in the comment box. But do check out our AI and ML courses in codecloud.com. That is it for this particular video. Speak with you in the next one. Thank you.