✕ Clear all filters
20 articles

📰 Dev.to · Marcus Chen

20 articles · Updated every 3 hours · View all reads

All Articles 95,637Blog Posts 112,496Tech Tutorials 24,091Research Papers 20,254News 15,320 ⚡ AI Lessons
Bootstrap confidence intervals for your LLM eval metrics
Dev.to · Marcus Chen 6h ago
Bootstrap confidence intervals for your LLM eval metrics
TL;DR: A single eval number hides its own uncertainty. Eval confidence intervals from bootstrap...
Read article → + Playlist
Benchmarking 5 LLM providers on one eval set, no SDK per vendor
Dev.to · Marcus Chen 21h ago
Benchmarking 5 LLM providers on one eval set, no SDK per vendor
TL;DR: We run a 1,200-case eval suite for enterprise agent automation at Nexus Labs. Comparing models...
Read article → + Playlist
temperature=0 didn't make our LLM evals reproducible
Dev.to · Marcus Chen 1d ago
temperature=0 didn't make our LLM evals reproducible
TL;DR: We set temperature=0 and seed=42 and still got different eval scores on the same 800-prompt...
Read article → + Playlist
Harvesting a regression test set from gateway logs with a plugin
Dev.to · Marcus Chen 📐 ML Fundamentals ⚡ AI Lesson 1d ago
Harvesting a regression test set from gateway logs with a plugin
TL;DR: Our eval sets went stale because a human wrote the test cases by hand once and never updated...
View lesson → + Playlist
Perplexity held flat after INT4. Task accuracy dropped 7 points.
Dev.to · Marcus Chen 5d ago
Perplexity held flat after INT4. Task accuracy dropped 7 points.
TL;DR: We quantized a fine-tuned 14B agent model to INT4 with GPTQ. Perplexity moved 0.04. We almost...
Read article → + Playlist
A 9-point eval gain vanished when we deduped train against test
Dev.to · Marcus Chen 1w ago
A 9-point eval gain vanished when we deduped train against test
TL;DR: We fine-tuned an 8B model for an enterprise ticket-routing task and saw accuracy jump from 71%...
Read article → + Playlist
Our voice agent passed every test and still woke me up at 3am
Dev.to · Marcus Chen 1w ago
Our voice agent passed every test and still woke me up at 3am
Replaying real call transcripts as your test set is a trap. The failures come from the...
Read article → + Playlist
The 4-layer voice-agent latency stack, traced with OTel spans
Dev.to · Marcus Chen 2w ago
The 4-layer voice-agent latency stack, traced with OTel spans
** How I instrument ASR, LLM, TTS, and the client with OpenTelemetry, and which number in...
Read article → + Playlist
Provider drift broke our regression evals. We pinned versions through Bifrost.
Dev.to · Marcus Chen 3w ago
Provider drift broke our regression evals. We pinned versions through Bifrost.
TL;DR: Our nightly agent regression suite dropped 4 points on a tool-calling metric with zero code or...
Read article → + Playlist
Aggregate eval scores hid a 14-point regression in one user segment
Dev.to · Marcus Chen 3w ago
Aggregate eval scores hid a 14-point regression in one user segment
TL;DR: Our agent eval suite reported 87% pass rate before and after a fine-tune. The aggregate didn't...
Read article → + Playlist
Shadow-testing a fine-tuned 8B against gpt-4o-mini through Bifrost
Dev.to · Marcus Chen 3w ago
Shadow-testing a fine-tuned 8B against gpt-4o-mini through Bifrost
TL;DR: We fine-tuned a Llama 3.1 8B for invoice line-item extraction. Before flipping production...
Read article → + Playlist
Continuous batching wrecked our p99 latency. Here's the trace.
Dev.to · Marcus Chen 3w ago
Continuous batching wrecked our p99 latency. Here's the trace.
TL;DR: We turned on vLLM continuous batching for a throughput win and watched p99 latency 8x in the...
Read article → + Playlist
Virtual keys per tenant: ditching our custom LLM billing layer
Dev.to · Marcus Chen 3w ago
Virtual keys per tenant: ditching our custom LLM billing layer
TL;DR: We had 11,247 lines of Python middleware handling per-tenant LLM cost attribution, rate...
Read article → + Playlist
LLM-as-judge variance broke our DPO training signal for 3 weeks
Dev.to · Marcus Chen 4w ago
LLM-as-judge variance broke our DPO training signal for 3 weeks
TL;DR: Our DPO pipeline used a single LLM as the preference judge. Training reward climbed every run....
Read article → + Playlist
Token-level eval harness for tool-calling agents: what we wired up
Dev.to · Marcus Chen 4w ago
Token-level eval harness for tool-calling agents: what we wired up
TL;DR: We replaced our "did the agent finish the task" pass/fail eval with a token-level harness that...
Read article → + Playlist
Prefix caching in vLLM under multi-tenant agent traffic
Dev.to · Marcus Chen 4w ago
Prefix caching in vLLM under multi-tenant agent traffic
TL;DR: We turned on vLLM's prefix cache for our agent workloads at Nexus Labs and watched TTFT drop...
Read article → + Playlist
We Audited Our Agent Tool-Call Traces. Half Our Eval Data Was Garbage.
Dev.to · Marcus Chen 4w ago
We Audited Our Agent Tool-Call Traces. Half Our Eval Data Was Garbage.
TL;DR: We pulled 41,000 production agent traces at Nexus Labs to build a fine-tuning dataset. After a...
Read article → + Playlist
Why Your LLM Eval Harness Is Lying to You (And How to Fix It)
Dev.to · Marcus Chen 1mo ago
Why Your LLM Eval Harness Is Lying to You (And How to Fix It)
TL;DR: Most eval harnesses I see in production are measuring the wrong thing. They report 87% pass...
Read article → + Playlist
Measuring AI Gateway Failover: 30 Days of Production Data
Dev.to · Marcus Chen 1mo ago
Measuring AI Gateway Failover: 30 Days of Production Data
TL;DR: We measured failover latency across three AI gateways (Bifrost, LiteLLM, Portkey) during 30...
Read article → + Playlist
What Gemma 4's multi-token prediction head actually means for your eval pipeline
Dev.to · Marcus Chen 2mo ago
What Gemma 4's multi-token prediction head actually means for your eval pipeline
Gemma 4 dropped with a multi-token prediction (MTP) head and immediately every benchmark thread on...
Read article → + Playlist