📰 Dev.to · Tech_Nuggets

An explanation of the KV cache memory problem in production LLM serving and how PagedAttention (the technique behind vLLM) solves it with OS-inspired virtual me

A practical walkthrough of the Model Context Protocol — what it is, how JSON-RPC 2.0 transports work, and how to build an MCP server in Python with FastMCP.

A practical comparison of three approaches to getting structured data from LLMs: prompt-only JSON, API-level JSON mode and function calling, and grammar-constra

MoE explained for practitioners: how the router works, load-balancing loss, why Mixtral has 45B params but activates 13B, and when not to use it. Practical, no

A production-oriented comparison of LLM sampling parameters -- how temperature, top-p, top-k, and min-p reshape the output distribution, what combos actually wo

A practical comparison of the four major LLM weight quantization formats — which one to use for CPU, GPU serving, and fine-tuning, with current version numbers

A practical walkthrough of LoRA and QLoRA -- how low-rank adaptation works, what NF4 quantization brings, and when to use each.

FP8 and INT8 KV caches cut attention state ~50%, but they shift the target model's logit distribution — and that can quietly halve the gains from speculative de