All
Articles 113,178Blog Posts 122,632Tech Tutorials 28,892Research Papers 23,285News 16,818
⚡ AI Lessons
Dev.to · Christopher Maher
1w ago
A local model opened 41 of our pull requests in five weeks. The model is the least interesting part.
Between May 21 and June 25, a fleet of local models running on a Mac and an AMD mini-PC opened 41 pull requests we merged into our Kubernetes operator, at zero
Dev.to · Christopher Maher
2w ago
A 27B model on an AMD mini-PC fixed a bug in our operator. Then it overreached.
A 27B local coder on a consumer AMD Strix box fixed a real LLMKube bug. The gate verified it compiled and passed; I caught where it overreached. Trust the harne
Dev.to · Christopher Maher
🤖 AI Agents & Automation
⚡ AI Lesson
2w ago
Trust the harness, not the model: a weekend of local agents building their own guardrails
Cross-posted from the LLMKube blog. A local 27B coding model, running on hardware in my house, is...
Dev.to · Christopher Maher
🧠 Large Language Models
⚡ AI Lesson
3w ago
Making a fleet of self-hosted LLM agents trustworthy
Running one local LLM node is easy. Running a fleet of them, off-cluster, and trusting it to stay current and stay honest, is the hard part. This is the work th
Dev.to · Christopher Maher
🧠 Large Language Models
⚡ AI Lesson
2mo ago
TurboQuant on a MacBook Pro: two findings the upstream discussion missed
Built TheTom's TurboQuant fork of llama.cpp for Metal, ran the bench overnight on M5 Max, and surfaced two findings the upstream community thread didn't have: t
Dev.to · Christopher Maher
🧠 Large Language Models
⚡ AI Lesson
2mo ago
62.2% on Aider Polyglot from a MacBook Pro. Then the other model we tried scored 4%. Here's what actually happened, with a working cost loop attached.
Qwen3.6-35B-A3B Q8 on a MacBook Pro M5 Max scored 62.2% on Aider Polyglot, beating Claude Sonnet 4 with 32k thinking. Then Devstral 2 scored 4% on the same harn
Dev.to · Christopher Maher
🧠 Large Language Models
⚡ AI Lesson
2mo ago
We ran Qwen3.6-27B on $800 of consumer GPUs, day one: llama.cpp vs vLLM
A Kubernetes-native bake-off on 2× RTX 5060 Ti, with reproducible manifests and a cost-per-token number neither cloud nor OSS FinOps tools will tell you.
Dev.to · Christopher Maher
3mo ago
I tested speculative decoding on my home GPU cluster. Here's why it didn't help.
I spent Saturday night testing n-gram speculative decoding on consumer GPUs. The claim: speculative...
Dev.to · Christopher Maher
🛠️ AI Tools & Apps
⚡ AI Lesson
3mo ago
Google Released Gemma 4 Yesterday. I Had It Fixing Real Bugs by Lunch.
Google released Gemma 4 yesterday. By the time I went to bed, I had it deployed on my home lab,...
Dev.to · Christopher Maher
☁️ DevOps & Cloud
⚡ AI Lesson
3mo ago
I Tested TurboQuant KV Cache Compression on Consumer GPUs. Here's What Actually Happened.
I spent this weekend testing TurboQuant KV cache compression on my home lab Kubernetes cluster. The...
Dev.to · Christopher Maher
🧠 Large Language Models
⚡ AI Lesson
3mo ago
The $0 Problem: Why Every Tool Says Your On-Prem Inference is Free
If you run LLMs on your own hardware, every cost tracking tool in the ecosystem has the same answer...
Dev.to · Christopher Maher
🧠 Large Language Models
⚡ AI Lesson
3mo ago
llama.cpp on Kubernetes: The Guide I Wish Existed
It started at my kitchen table. I was spending an evening on my laptop, fascinated by how LLMs...
DeepCamp AI