📰 Dev.to · Christopher Maher

Articles from Dev.to · Christopher Maher · 5 articles · Updated every 3 hours · View all reads

All ⚡ AI Lessons (9392) ArXiv cs.AI Dev.to · FORUM WEB Forbes Innovation Dev.to AI OpenAI News Hugging Face Blog

I tested speculative decoding on my home GPU cluster. Here's why it didn't help.

Dev.to · Christopher Maher 6d ago

I tested speculative decoding on my home GPU cluster. Here's why it didn't help.

I spent Saturday night testing n-gram speculative decoding on consumer GPUs. The claim: speculative...

Google Released Gemma 4 Yesterday. I Had It Fixing Real Bugs by Lunch.

Dev.to · Christopher Maher 1w ago

Google Released Gemma 4 Yesterday. I Had It Fixing Real Bugs by Lunch.

Google released Gemma 4 yesterday. By the time I went to bed, I had it deployed on my home lab,...

I Tested TurboQuant KV Cache Compression on Consumer GPUs. Here's What Actually Happened.

Dev.to · Christopher Maher 1w ago

I Tested TurboQuant KV Cache Compression on Consumer GPUs. Here's What Actually Happened.

I spent this weekend testing TurboQuant KV cache compression on my home lab Kubernetes cluster. The...

The $0 Problem: Why Every Tool Says Your On-Prem Inference is Free

Dev.to · Christopher Maher 2w ago

The $0 Problem: Why Every Tool Says Your On-Prem Inference is Free

If you run LLMs on your own hardware, every cost tracking tool in the ecosystem has the same answer...

llama.cpp on Kubernetes: The Guide I Wish Existed

Dev.to · Christopher Maher 3w ago

llama.cpp on Kubernetes: The Guide I Wish Existed

It started at my kitchen table. I was spending an evening on my laptop, fascinated by how LLMs...