Agentic infrastructure needs agentic observability | ODSP933

Key Takeaways

Once upon a time, humans wrote software, humans operated software, and humans debugged software. Observability was built around this. And I'm here to tell you that our own category is breaking. Logs, traces, dashboards, alerts, all of it assumed a human investigator sitting in front of the data trying to figure out what happened. But that world is changing. Now agents are writing code, operating systems, making decisions, debugging failures, and increasingly interacting with infrastructure without a human in the loop. And once the consumer of observability changes from humans to agents, the assumptions underneath the category start to fail. That's the core idea I want to talk about today. Agentic infrastructure needs agentic observability. My name's Jimmy. I lead the solutions engineering team here at Groundcover. And I want to share today that it's it's actually not really a product pitched. It's more of a diagnosis. Uh something fundamental stopped working in observability once AI systems became part of the software life cycle itself. And this is not a feature problem. It's not a we need better dashboards or more AI features problem. This is a systems problem. The operating assumptions underneath observability are changing. The first problem is logs. Logs were built for events. Agents produce decision breadcrumbs. Traditional logging assumes discrete events, stable schemas, and human-readable meaning. Request started, request completed, database failed, cache missed. Those are events. But agents don't behave like that. An agent retrieves documents, scores relevance, supplies policies, retries tool calls, rewrites outputs, reevaluates context. What gets produced is not an event stream. It's a reasoning trail. And that creates a completely different problem. Now the question isn't just what happened. The question becomes, why did the agent make this decision? Why did it choose these documents? Why did it block this output? Why did it retry this tool five times? The reasoning happens across long time horizons and wide context windows. And honestly, no observability platform today was really designed for this. Traditional observability assumed systems were mostly deterministic. You could usually reconstruct the story afterward. A request failed, a dependency timed out, a deployment introduced latency. Agent systems don't fail that cleanly. The same input can produce different different behaviors. The same workflow can make different decisions. And increasingly, understanding the reasoning path matters more than understanding the infrastructure path. The second problem is traces and sampling. Sampling breaks and status codes lie. APM was built for a world that no longer exists. Let's start with head-based sampling. A random sample of a non-deterministic agent flow tells you almost nothing. The one trace you keep may completely miss the reasoning path that actually mattered. Tail-based sampling isn't a real answer, either. Because now you're trying to hold tens of thousands of spans in memory waiting to decide if something failed. At 50,000 spans per session, that falls apart pretty quickly. And then there's the bigger issue. The 200 OK means absolutely nothing now. I'll say it again, 200 OK means nothing. In traditional systems, a successful status code usually meant the system behaved correctly. In agent systems, it only means nothing crashed. The infrastructure succeeded, the outcome can still be completely wrong. Agent may have used the wrong context, selected the wrong tool, hallucinated an answer, or misunderstood intent. No answer does not mean correct anymore. And these failures are incredibly difficult to capture because they're often rare and non-deterministic. Sampling guarantees you miss the exact journey you needed to debug. And this changes the economics completely. Historically, observability vendors made money by charging you to ingest more telemetry. That worked when telemetry growth was relatively predictable. But agent systems explode telemetry volume. One workflow can generate thousands of spans, hundreds of tool calls, and massive amounts of contextual data. So now teams are forced into a bad trade-off. Either keep the data and absorb the cost or reduce visibility and hope the important signals survive. Neither option really works. The third problem, sensitivity and instrumentation. Your telemetry now contains things you never meant to store. Prompts, customer conversations, PII, financial information, internal business logic. Telemetry stopped being just technical metadata. Now it's potentially the most sensitive data in the company. That changes the risk model completely. Shipping all of that to a SaaS vendor is no longer just a cost discussion. It becomes a liability discussion. And at the same time, manual instrumentation is collapsing under AI velocity. Humans used to maintain pipelines manually because the systems evolved at human speed. Now AI generates services and workflows faster than teams can keep instrumentation current. So the implications become pretty obvious. The observability layer itself has to learn to instrument automatically. If you zoom out, all three of these problems point to the same thing. Everything we built in observability was optimized for the one thing agents don't have. Human limits. We optimized for less data, simpler dashboards, smaller cardinality, reduced telemetry, human readable abstractions. But agents don't need the same simplifications humans need. Agents actually improve with more context, which means a lot of the old optimizations become constraints. This is why I say we built the whole industry for the wrong consumer. For 15 years, observability tooling was shaped around human cognition. Humans can't process 50,000 spans in a session. Agents can. Humans need summaries. Agents need full context. Humans simplify systems to understand them. Agents often perform better when when can access the entire system state. So, right now, we're doing something fundamentally broken. We're feeding agents a summary and expecting them to understand the entire book. That's not a model problem. It's a data completeness problem. And honestly, I think this is one of the biggest misconceptions in AI infrastructure right now. A lot of people assume better models automatically solve observability. But smarter models operating on the incomplete telemetry still make bad decisions. Context quality matters just as much as model quality, possibly more. And this is where I'll make a slightly controversial point about MCP-style demos. An external agent is always a guest. It doesn't control the API surface. It doesn't control indexing. It doesn't control correlation primitives. It operates on partial context. And partial context is exactly what creates hallucinated diagnosis. What agents actually need is an architecture built for data abundance. And this is where Groundcover happens to align very naturally with this shift. Agents on Groundcover reason over complete system state, not a curated slice. That's an important distinction. Because there's a massive difference between an agent confidently hallucinating a fix and an agent actually understanding what's wrong. A few architectural bets ended up mattering a lot here. First, DYOC, bring your own cloud. Everything stays inside your cloud. That matters for control, governance, and sensitive telemetry. Second, zero instrumentation and zero friction. You cannot depend on humans to manually keep instrumentation current in AI-generated systems. And third, one platform regardless of whether the code was written by a human or an agent. Because increasingly, it's both. This leads to a new model for observability. Move the LLM up, push the analysis down. The model is not the execution engine. The LLM's job is to understand intent, decide investigation strategy, and generate deterministic operations, and interpret the results. The back end's job is the actual analytical work. Close to the data, accurately, cheaply. That separation matters. You don't want the model brute forcing analysis over complete context. You want the model orchestrating investigation over a deterministic analytical system with complete telemetry. That's a very different architecture than what most observability systems were originally designed for. Once you accept that model investigation changes completely, investigation starts from intent, not navigation. In the old world, engineers click through dashboards manually, trying to piece together what happened. In the new world, you start with a goal. Why did this fail? What changed? Which customers were impacted? The agent runs the investigation, gathers the right signals, materializes the right outputs, and returns an answer plus next actions. Not a dashboard, an answer. And this changes development workflows, too. Right now, teams literally screenshot dashboards and paste them into Claude. That should not exist. Production telemetry should flow directly into coded coding agents and CI/CD systems as structured context. And eventually, these systems don't just observe, they start acting. Raising collection fidelity dynamically, instrumenting on the fly, supporting the remediation during incidents. And I think this is where the category starts to look fundamentally different. Observability stops being a passive debugging interface. It becomes an active operational system. Not just showing humans what happened, but helping systems understand, respond, and eventually recover automatically. Now, obviously, this introduces hard questions. And honestly, I think the industry still underestimates how important these questions are going to become. How much autonomy should your observability layer actually have? If an agent fixes a production incident at 2:00 in the morning, who approved that action? What guardrails govern systems that can act directly in production? If the agent remediates incorrectly, who owns that outcome? And how do you validate telemetry normalized correctly when no human explicitly defined the schema? Because ultimately, the agent's reasoning is only as good as its data. I'll say it again. The agent's reasoning is only as good as its data. These are not edge cases. These are foundational operating questions for the next generation of infrastructure systems. My view is that the companies that get observability right in the agent era will have a structural advantage. Not just operationally, competitively. AI cannot fix what it cannot see. And it cannot build effectively without full context. We definitely do not have every answer yet, but we know the direction that this is going. And we know the questions teams need to start asking now. Observability is becoming the operating system for the agentic SDLC, the data layer everything else depends on. This is the shift happening underneath the industry right now. I'll leave you with the core idea one more time. AI can't fix what AI can't see. The agentic infrastructure needs agentic observability. Thank you.