▶ Videos →

📰 Dev.to · Jahn

5 articles · Updated every 3 hours · View all reads

All Articles 167,707 Blog Posts 160,036 Tech Tutorials 44,479 Research Papers 32,781 News 21,438 ⚡ AI Lessons

DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings

Dev.to · Jahn 18h ago

DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings

Got a DGX Spark and want vLLM on it without the NGC container? Here is what actually happens on real...

SGLang outputs endless repetition on NVFP4 models: the FP8 lm_head bug

Dev.to · Jahn 🧠 Large Language Models ⚡ AI Lesson 21h ago

SGLang outputs endless repetition on NVFP4 models: the FP8 lm_head bug

Your NVFP4 model serves fine on vLLM but outputs an endlessly repeated phrase on SGLang, from the...

The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell

Dev.to · Jahn 🧠 Large Language Models ⚡ AI Lesson 1d ago

The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell

If you run vLLM with --kv-cache-dtype fp8 on a DeepSeek-family (MLA) model and your GPU is a GB10, an...

Did FP8 make the model dumber? A per-prompt regression check for quantized serving

Dev.to · Jahn 1d ago

Did FP8 make the model dumber? A per-prompt regression check for quantized serving

FP8 gave us a clean 1.5x on Qwen3-8B serving throughput on an RTX PRO 6000 Blackwell (1,725 to 2,597...

Qwen3-8B on workstation Blackwell: vLLM vs SGLang vs llama.cpp, plus an FP8 pass

Dev.to · Jahn 1d ago

Qwen3-8B on workstation Blackwell: vLLM vs SGLang vs llama.cpp, plus an FP8 pass

Benchmarks of the same model on the same GPU across three serving stacks, then an FP8 pass on the...