What are Vectors? | Vector Databases for Beginners | Part 2

Key Takeaways

The research aspects, the academia aspects started maybe 2003 and they started doing a lot of research into vector embeddings and how you can represent words um meanings with vector representations and then this paper came out around 2013 about wordtovec and this was the first time that the sort of usefulness of vector representations was seen um in sort of more industry applications. Stuff started coming out and this sort of um snowballed into the attention paper and everything we know nowadays which we'll go into a little bit more um coming up. So you can do a bunch of cool things with vectors. Um this is what really got me interested in machine learning. So let's take a look at some weird cool things that vectors can do. Okay, so I mentioned before you can do math with vector embeddings of words. So if we take the vector embedding for queen and we subtract the vector embedding for king, weirdly enough, it's very similar to the value if you take the vector embedding for woman and you subtract the vector embedding for man. Kind of interesting. Um let's look at another example. So if you take uh this is sort of a visual representation of vector embeddings. Um and so you can think of each little box as a color coordinated of the number of a vector embedding. So immediately what stands out is the vector for water looks a lot different than the vector for any of the other terms. Right? And also if you look a little closer like your vector for woman and girl look very similar. Boy and man look very similar. King and queen look a little similar. In water you're missing this blue streak that is in everything else. Um for these terms you have this green one but that's different here. Um super interesting visually sort of encompassing what we humans have learned about these terms. So, when babies learn what a word means, how are they figuring it out? Um, how do they know what a woman corresponds to, what that word corresponds to? So, what are these two things sort of mean? And this is sort of saying that these vectors actually are encoding the meaning of words in some way. Super interesting. Okay, let's take a break. go back to this in a second. For now, I want to talk about a different type of vector um RGB color codes. [clears throat] So, RGB color codes are basically three numbers and each number corresponds to how much red, green or blue in a color. So, every color is made up of various amounts of red, green and blue. And then we use this basically in um any design applications to determine different colors. You can make any color with any with the differing the values of red, green, and blue. So if you set every value to zero, you get black, right? There's no color. If you set all the values to the max value, 255 in RGB color codes, you get white. Um, if you set red to the max value, you obviously get a red. Um, if you set both red and blue to the max value, you get this lovely purple color. And if you set sort of a combination, you can get this color of emerald. Right? So if we look at this in a 3D space, you can see here that similar colors are being grouped together obviously because they have similar values, right? Um so all the yellows are grouped together, all the greens are grouped together, the blues are grouped together, and more different colors are further apart. Okay, so each number is representing how much red, green, or blue is in the color. Funny enough, this is exactly what a vector embedding is when you're talking about vector embeddings in machine learning. It's a sequence of numbers that's representing the meaning. The main difference with RGB color codes is it's it's three numbers, right? And we know exactly what each of those numbers corresponds to, the red, green, or the blue. Vector embeddings are much higher dimensions. They can be thousands of dimensions and we don't exactly know what number corresponds to what feature of the meaning of the word. There's been some interesting research done about this but we we can't be quite sure. But for theoretical purposes, for understanding purposes, you can think of each of the dimensions as encoding sort of a feature of whatever it's encode or whatever it's embedding. So maybe in the vector for cat, one of the one of the numbers corresponds to if it's furry or not. Maybe one of them corresponds to if it's an animal or not, or if it has ears or not. Different things like this. And the same thing happens with images or audio. Um each of these numbers will encode a meaning of the vector or of of the word of the thing it's trying to embed. So that's sort of a highle overview of what vectors are. Um, this freaked me out a little bit. It took me a while to understand. So, don't be um concerned if you don't get it straight away at first. But for now, all you need to know, we can turn words and text and audio and video and images into the string of numbers that encodes its meaning. So that way we can use it in machine learning applications.