What are text embeddings?

Key Takeaways

so Jason what are we going to break down in this video well AA let's go for embeddings great because sometimes I get asked what the heck is an embedding and the really simple answer is that it's a way to represent something interesting like text or an image or video as a big AR R of numbers and we call those raised vectors because math but why use those numbers what do they actually represent well let's say you're making a vector for a cat and a vector for a dog the vector encodes attributes of things like does it have legs and does it have fur but if you added another Vector for a house it would not have fur or legs but it would have a roof each Vector has Dimensions which are essentially the number of numeric components in the vector representing those attributes if we imagine our vectors is only two numbers we can also plot them on a graph and you could see that similar things like dog and cat are closer together and house is far away from both of them in reality embedding vectors can have thousands of numbers and embeddings are generated using a model usually by calling an API and different models return different embeddings and work on different kinds of data which one you should use depends on your data oh and one important note embeddings are deterministic for a given input embedding model will always return the same output it sometimes helps to think of them as hashes this is different from the inference models you may be more familiar with like Gemini that are non-deterministic if you supply the same prompt to an inference model you aren't guaranteed to get the same response back every time it gets even cooler when you think about multimodal embeddings so we talked about data science for a while but how do I convert images and pixels into vectors and capture their meaning based on content and not just the pixels in it this is the power of embedding algorithms they can organize unstructured data into what we call a semantic space so great we've established that embedding are simple arrays of numbers that represent something they're often very very large arrays but at their core they're just arrays of numbers but what are they actually good for well think about searching through unstructured data if you've encoded the meaning of a thing into the embedding you can find all of the pictures of dogs or even the pictures of a specific breed of dog this process allows you to view unstructured elements in a structured way enabling you to search within what we call that semantic space and it works because all of them use the same model for embedding I didn't fully Gra this concept until I built out a rag system for the first time and I started retrieving relevant information from a vector database to help the llm generate more relevant responses you can use your embedding code to generate an embedding of the user's input and then use that embedding to find similar items in your database and the similarity is not just based on pixels or words but can also be based on meeting depending on your embedding model all own up to the fact that when I did my first app that needed embeddings I went and found the first tutorial that looked like it would work and followed the instructions uh I think it used gecko however there are many embedding models and choosing the right one involves several factors such as how you will break up or chunk your input these decisions are crucial it's funny because a lot of the questions would boil down to what are you actually trying to achieve and come back to the use case if you're using something like llama index it offers a number of chunking strategies that you can choose from and I like to think of embeddings here as layers of information so imagine you're going to break down a book you might have embeddings at the sentence level at the paragraph level and even the chapter level analyzing the chapter might yield very different insights compared to analyzing individual sentences and this is where the manage search is valuable however you still really have to understand your data and what it impacts your data and also your goals your data chunks need to be useful for the specific application let's look at some code to demonstrate how you can create embeddings there are many ways to create embeddings and one of the cool ones is to use PG vector and alloy DB to create embeddings directly from the database here's a simple SQL oneliner to do just that you can also create embeddings in a language like python here's an example of creating a multimodal embedding with python and then showing the vector that it returns if you want to learn more about embeddings you can check out a great blog post on the Google Cloud blog that breaks down the concepts even more than we did here there's also a code lab on creating embeddings with gecko that you can check out to get a walkthr of creating your first embeddings for text Data check out the links below with that signing off this is AA and Jason in real terms for AI thanks for joining