Why RAG Systems Failed

Key Takeaways

And so we see two common failure patterns on agent authorization. Pattern number one is what we've all saw with rag systems which is you give the agent its own identity. Uh vendors will call this nonhuman identity, right? It's an identity. It's like a human but it's different. So we need to treat it like a new person. It's a new worker. It's a new knowledge worker. It's just automated. It's intelligent. That didn't work. That's it's never worked. This is why rag systems really struggled. What's the level of permission you give the rag system? Any level of permission above zero, then you run into a second problem is, oh, who do we give access to the agent? So, let's say that we give we create an agent that's doing compensation management at an organization. Does it get access to the CEO's compensation? Does it get access to the intern's compensation? What level of the organization can it see? We define whatever that's going to be. Now, the intern comes in to work for HR and he has access to the agent. What can the what can the intern see? If the intern's access is lower than the agent, you've just created what's called an authorization bypass vulnerability. The CISO will freak out. And so when you look at rag systems, the mass majority of them are working on public information. They're getting the lowest level of permission possible to avoid this problem. This is why all those really smart support apps that you interact with on the websites, you can ask all the questions that it regurgitates the public knowledge base, but the moment you ask it where your order is, it cannot answer the question. It's because they can't figure out how to authenticate and authorize as you and pull your information and prove that it's you asking for it and not you asking for his information. Huge problem. The other failure pattern we see is what is common in most MCP servers, which is you use the user's credentials. I put in my own credentials and the and the MCP server is acting as me and that is secure. That is legitimately secure, but it's very unsafe. I have seen cursor try and delete my root directory. It couldn't because it doesn't have pseudo access. Got blocked. It apologized profusely. It was very kind. Uh but thankfully it didn't have access. That's why giving it your credentials is a terrible idea because you have the ability to delete directories and drive. You have the ability to delete emails. You have the ability to do all kinds of different things that you may or may not want the agent to also be able to do. The right way to do this is what's called delegated authorization. It is to take the intersection of what the agents register to do not as an identity but as an application and then take the user's identity and what they're allowed to do and then take the intersection if the agents authorized to do it because they pre-registered as an as an OOTH application on the downstream service and requested scopes and claims as part of their registration process and the user is also permissioned to do it then the agent should be allowed to do it and if one of those two statements is not correct, the agent should not be allowed to do it. This is very difficult to do. So far as I know, I think we might be the only service that can do it, but that's going to change. Um, in the MCP specification, this has just been accepted. We were the authors of that contribution. Um, so my hope is it'll be merged into the spec by the end of the year. And so regardless of which vendor you use, this should be possible soon.