How Do We Get MASSIVE Model To Run On Device? Quantization Explained.

Every time I do a video about a model I get a comment saying "Well you never said what it takes to run it!" Well since I am not inside of your computer the answer is - "it depends". In this video I want to take a simple explainer or crash course on Quantization for LLMs. That is those Q_ files you see on HuggingFace. Where do they come from? How are they made? What is really going on with this process. This is a brief explainer video about this whole process so that the next time a model drops - you can looks at these quants with confidence and ultimately know what it might take to run them. *Links* AnythingLLM: https://anythingllm.com About GGUF: https://github.com/ggml-org/ggml/blob/master/docs/gguf.md Example Config model: https://huggingface.co/Qwen/Qwen3.5-27B/blob/main/config.json *Chapters* 0:00 What even are these files? 1:30 What is GGUF? 2:32 What is a quant (or Quantization) 7:08 Who is making these files? 8:40 Looking at a real example 9:39 All about Quants and How They Work] 18:00 About Perplexity 19:39 Choosing the Right Quant 20:39 How memory is used 21:38 Calculate Memory for Model 22:51 Calculate the KV Cache size (Context window memory) 26:19 Crash Course is over