SLMs, LLMs, and Model Routing in Agents | Amazon Web Services

Hello, I'm Nolan Chen, partner solutions architect at AWS. And I'm Andrew Wilco. I lead the field engineering team at RCAI. Andrew, another topic we hear a lot about these days is AI agents. Can you tell us what is an AI agent? Absolutely. Everyone has a different definition for what an AI agent is. In our definition, it's where you're able to take additional context. So this can be additional information, data, whatever it might be, and be able to supply that to a model which it can then utilize to take action upon that information. And when we really get to true agents, this can all happen autonomously. You build a system or a construct that's able to retrieve the correct information to complete the task, supply that to the model which is then able to conduct its analysis or whatever action it needs to take and then an action can be taken upon that result. Thanks Andrew. So looking at your diagram, it looks like the AI model is at the center of an AI agent. Now, in our previous videos, we talked about small language models, large language models, and model routing. Can you tell us how all those components together help us build these agents? Yeah, absolutely. And in those videos, we talked about when each one would make sense and when we should use each one. And the way to think about it is the same way that you would think about building a team or even a company. If you were putting a company together, you wouldn't have just one type of person that you went to for every request. You would have your marketing team. You would have your development team. You would have your finance team. so on and so forth. And each one of these teams is able to coordinate together, right? And you have many, many more teams that are all interacting together and able to coordinate amongst one another. Agents are the same way. You don't want to have just one model that you use for every agent, every step. And that's really where intelligent model routing, small language models, and large language models all work really well together. So you can use the domainspecific small language models that we talked about before for certain roles. You can use the general large language models for certain roles and for ones that might change where in some cases you need an SLM, some cases you need a large language model, that's when you can use the intelligent model. [Music] routing and each one of these components fits into this type of system. So you have your SLMs, your LLM and in certain cases your intelligent model routing where each one is able to work together in order to build your overarching agentic system. Okay. So I understand why sometimes you want to use a general LLM versus a domainspecific SLM. Go back going back to your agent here. When we look at this model in the middle is the model router in here and when we put model here is it really look is it were we looking at multiple possible models in here and the routers in inside the agent? Yeah, absolutely. And there's a couple of benefits you'd actually achieve from intelligent model routing if you were to put it in that model placeholder. One is overall improved accuracy and that's because you're able to use the right model for the right task. And then the second really big benefit is cost reduction because in certain cases instead of just putting a large language model for every task, you're able to utilize SLMs where it makes sense. And in fact, some of our own customers that have utilized this technique here have seen upwards of a 64% reduction in cost within their systems by utilizing SLMs and intelligent model routing within their aentic networks. Does that mean depending on the prompt that the end user sends in, the same agent could actually be running different models different every time? Absolutely. Awesome. Well, thank you, Andrew. It's a fascinating journey today, not just about SLMs, but also model routing and agent and agentic applications. Absolutely. Thank you, Nolan. And I'm excited to see what uh the industry will keep doing. This is changing day by day and we're continuing to improve. It's been great chatting with you today. Likewise. Thank you very much.