How to Choose an Embedding Model?

Key Takeaways

model to use in the demo we'll go over using two different types so we go over using a open source model with modern embed b or modern bird base and then we'll go over using one with open AI but there's lots and lots and lots and lots and lots of different models out there there's closed Source models from open AI they have a whole bunch there's coher models um and there's so many different open source models that it's gets it's really really overwhelming when you try to figure out which one to choose so how do you pick there's two main things that you should think about here the first one is data performance and there's several subcategories here where you need to correspond it to your actual use case that you're building so the first one is language specificity are you using this on Purely English data are you using this for multimodal data are you using this for code do you need a really large context window that all falls under language specificity the second one is domain specificity are you using this on sort of General data or do you have a specific domain like medical legal e-commerce um where you need specific performance tailored towards that third one is does it actually work um so certain use cases will require a higher amount of accuracy than other use cases or maybe you care more about um accuracy on certain things than others um so you need to figure out how it works in general um and also how it works in your own data so that's where benchmarking and comparison comes in the second side of things is infrastructure so as much as we would like it to be building machine learning applications is not free um and so this is where you need to think about costs so what is it going to cost to store and interact with your model um bigger models will cost more here smaller models will cost less storage costs so when you actually create your vector embeddings they can be different dimensions um and when you're storing those Vector embeddings larger amounts of or larger Dimensions will cost more for storage and the third one is the latency and throughput so if you need a higher amount of latency if you're doing more Qui varies if you need it to be faster um then you're going to need more infrastructure so those are the kind of two sides of what you need to consider when you're choosing an embedding model the mte leaderboard is a great place to sort of explore open source options and also explore all the different sort of tuning modules you can use and look at when you're choosing a model so um I actually just went on this site today and and it looks a lot different than what it did when I made this presentation a couple days ago so