SLMs and LLMs: When to use them? | Amazon Web Services

Key Takeaways

Hello, I'm Nolan Chen, partner solutions architect at AWS. And I'm Andrew Welco, lead the field engineering team at RCAI. Andrew, can you give us a quick recap recap again as to what is a small language model or SLM? Absolutely. So, when we're talking about small language models versus large language models, there's a couple categories that we're focusing on. With LLMs, you typically are able to handle more complex use cases and be able to handle complex data types. However, they're also very expensive to run and that means that you cannot run them in your own environment. However, with small language models, these have a lower parameter count, which means they are able to be run with lower latency, lower cost, can be run in your own environment, and additionally have an increased ability to be adapted or fine-tuned. Thanks Andrew. So at least in theory SLMs are less complex, less costly, more easily adaptable and can run in your own environment. But could you now give us some examples of real world use cases or workloads that would be ideal for an SLM? Absolutely. So there's really two categories of use cases that we would think about. one is the generalpurpose small language models that are out there and then the other is domain adapted small language models. So let's first start with general purpose ones and the most common use case here is for chat bots. Small language models do a great job at acting as chat bots for businesses and specifically chat bots that are using retrieval augmented generation because with retrieval augmented generation or rag, you're actually passing in that information to the model when asking a question. And this means that the SLM has the context it needs to answer that question and can do it in a very quick and inexpensive manner. We've also seen a lot of use cases do really well around data labeling or data tagging as well as sentiment analysis and many others. And this is perfect for general purpose models that you don't actually need to tune the model for. However, one of the large benefits with small language models is the fact that they can be adapted or fine-tuned. And when we're looking at fine-tuned models, this is where businesses can get real benefits because you can be a financial institute that has a model that's fine-tuned to be able to do financial analysis. You can be a healthc care company that is fine-tuning a model that does really well at summarizing uh doctor transcripts for patients and the list goes on. You're able to fine-tune these models to be really performant at the tasks that's most relevant for your business. Got it. So I think there are a lot of companies out there that can definitely benefit from having a chatbot doing labeling and doing sentiment analysis. And you listed two important industries here, finance and healthcare that can benefit from SLM. But that said, you talked about all these benefits of SLM. When should companies still still use LLMs instead of SLM? Yeah, great question. And we don't think that LLMs are pointless, right? It's just that they should be used for the right task. So for example, if you have really complex use cases, let's say for example that you are a research lab and you are doing research around protein synthesis for example. This is where an LLM can come into play and there's lots of news stories around about how companies have been able to utilize them effectively. Additionally, if you're doing really complex analysis, this is where LLMs do a good job. Also, if you need to utilize a massive context window, so you are sending in lots of documents, and I'm not talking hundreds of documents, but I'm talking about like you're getting to thousands of documents that you are sending. There are LLMs that allow you to use to be able to pass in that much information. So depending on the use case, you're able to use LLMs for those really really highly complex tasks. Are there any tasks or workflows where you have uh hybrid approach where you have both LM and SLM working together? Yeah, absolutely. And that's actually one that we see all the time. So let's take for example you have a document processing workflow. You might start with your document and you want to do a couple of things. Maybe you want to extract some data. So you want to do a little bit of data extraction here. You can pass that to an SLM to do that task. Then you might also want to do let's say sentiment analysis. [Music] You can use an SLM for that task as well. But then let's say that you actually want to pass the extracted data, the document itself, the sentiment analysis, and let's say that there is some data that you scrape from the web. [Music] Let's say it's some research and you want to pass all of that to a model. you can then pass that to your LLM to do overall analysis of this output and then that output can have a task done with it. So here we see a perfect example where you can combine small language models with large language models and one of the things that actually works well here is when we get into the topic of int intelligent model routing. Awesome. So, we'll get to that topic next, but thank you for showing how companies can benefit from LLMs or SLMs on their own or even put them together in a hybrid solution. Absolutely. Thanks, Dolan. Thank you, Andrew.