Lightning Talk: When Humans Stop Thinking: Agentic AI

Key Takeaways

Good afternoon. It seems like listening to the audience today and some of the speeches that we've been preparing for AI failure for quite some time. Right? Seems like quite some time. Modeling version, prompt injection, data poisoning, you name it, and it all matters. But I want to take the discussion somewhere different today. Because I believe something that we're less prepared for is something that's quieter, something harder to to trace because it doesn't always show up as a failure condition. AI didn't fail, but human judgment, well, that's starting to drift out of frame. And that's what I want to put light on today. Because many systems um because many systems uh are operate differently, right? And it's not always about what the model gets wrong. Sometimes it's about what Sometimes it's about what people Sometimes it's about what people um The model it's not about what we what the model gets wrong. Sometimes it's about what people um forget um to challenge the model on or what it gets right. So, um and that is that is the danger point, right? So, I want to talk about the process. A system starts out as a tool, right? Humans use tools. Humans are firmly in control of those tools. Then the then the then the tool becomes a trusted tool. It's useful, um it's generally right, and um it starts to, you know, impart confidence to to the to the user. But then if we're not careful, that trusted tool becomes something more. It starts to become decision authority, right? It starts to take on authority. And that is the real risk that we want to talk about today is when we give up our our um our autonomy and our decision um authority over these systems. And that's where the seams start to unravel. Because in this agentic age, we have to re- remain in charge of the decision process when we're dealing with these tools. We have to understand who's recommending, who's approving, and who's executing when we're actually using these systems. We may think that we're in charge, and we may think that we have decision ownership, but what's really happening underneath is that the model is assuming all of that authority. And we're just abdicating our responsibility for that. Let me give you a simple example. A um a human accepts an analyst, I'll say an analyst accepts a uh an AI recommendation. Right? So, they accept the recommendation, everything seems fine, and then the human approves, and then the system executes. But then we have to ask ask ourselves, who actually owned that decision at the end of the day? Because if we can't distinguish between where the human judgment ended and where the model momentum picked up, then we're not talking about automation anymore. We're talking about accountability blur. And that's important. Because if we can't firmly attest to what the system is doing, then we've lost control. Because nothing really broke. But the still but the system still but but control was still lost. Right? It doesn't require a catastrophic failure. Nothing has to trip. Nothing has to degrade, but then we still lose control. Control still eroded. And so we have to ask ourselves, who still owns that decision? And that brings me to a genetic role mapping. We need more than capability maps. We need authority maps. We need to understand who recommends, who approves, who executes, and ultimately who's accountable for the decision at the end of the day because visible approval is not always meaningful review. Just because you can click the approve button really really fast and you can clear out your backlog of all of these requirements that are coming at you and decisions are coming at you at machine speed, the operator is still operating at human speed. And so that is why I say visible approval is not always meaningful authority. And with that from the end of the conclusion at the end of the presentation here. So if you want to learn more about AI conversation take the take the take the survey scan the QR code and join the survey and we can find out more about decision authority.