📰 Dev.to · Marcus Chen
15 articles · Updated every 3 hours · View all reads
All
Articles 87,379Blog Posts 108,009Tech Tutorials 21,637Research Papers 18,891News 14,400
⚡ AI Lessons
Dev.to · Marcus Chen
2d ago
A 9-point eval gain vanished when we deduped train against test
TL;DR: We fine-tuned an 8B model for an enterprise ticket-routing task and saw accuracy jump from 71%...
Dev.to · Marcus Chen
5d ago
Our voice agent passed every test and still woke me up at 3am
Replaying real call transcripts as your test set is a trap. The failures come from the...
Dev.to · Marcus Chen
1w ago
The 4-layer voice-agent latency stack, traced with OTel spans
** How I instrument ASR, LLM, TTS, and the client with OpenTelemetry, and which number in...
Dev.to · Marcus Chen
2w ago
Provider drift broke our regression evals. We pinned versions through Bifrost.
TL;DR: Our nightly agent regression suite dropped 4 points on a tool-calling metric with zero code or...
Dev.to · Marcus Chen
2w ago
Aggregate eval scores hid a 14-point regression in one user segment
TL;DR: Our agent eval suite reported 87% pass rate before and after a fine-tune. The aggregate didn't...
Dev.to · Marcus Chen
2w ago
Shadow-testing a fine-tuned 8B against gpt-4o-mini through Bifrost
TL;DR: We fine-tuned a Llama 3.1 8B for invoice line-item extraction. Before flipping production...
Dev.to · Marcus Chen
2w ago
Continuous batching wrecked our p99 latency. Here's the trace.
TL;DR: We turned on vLLM continuous batching for a throughput win and watched p99 latency 8x in the...
Dev.to · Marcus Chen
2w ago
Virtual keys per tenant: ditching our custom LLM billing layer
TL;DR: We had 11,247 lines of Python middleware handling per-tenant LLM cost attribution, rate...
Dev.to · Marcus Chen
3w ago
LLM-as-judge variance broke our DPO training signal for 3 weeks
TL;DR: Our DPO pipeline used a single LLM as the preference judge. Training reward climbed every run....
Dev.to · Marcus Chen
3w ago
Token-level eval harness for tool-calling agents: what we wired up
TL;DR: We replaced our "did the agent finish the task" pass/fail eval with a token-level harness that...
Dev.to · Marcus Chen
3w ago
Prefix caching in vLLM under multi-tenant agent traffic
TL;DR: We turned on vLLM's prefix cache for our agent workloads at Nexus Labs and watched TTFT drop...
Dev.to · Marcus Chen
3w ago
We Audited Our Agent Tool-Call Traces. Half Our Eval Data Was Garbage.
TL;DR: We pulled 41,000 production agent traces at Nexus Labs to build a fine-tuning dataset. After a...
Dev.to · Marcus Chen
3w ago
Why Your LLM Eval Harness Is Lying to You (And How to Fix It)
TL;DR: Most eval harnesses I see in production are measuring the wrong thing. They report 87% pass...
Dev.to · Marcus Chen
3w ago
Measuring AI Gateway Failover: 30 Days of Production Data
TL;DR: We measured failover latency across three AI gateways (Bifrost, LiteLLM, Portkey) during 30...
Dev.to · Marcus Chen
2mo ago
What Gemma 4's multi-token prediction head actually means for your eval pipeline
Gemma 4 dropped with a multi-token prediction (MTP) head and immediately every benchmark thread on...
DeepCamp AI