How does an AI model search through information?

Key Takeaways

Vector databases are the way that large language models search through information traditional databases are what you're used to you have a document and you have an organized set of keywords that reference this document when you search these keywords you pull up the document this works if you have really structured data or you want to go through that process of actually assigning docu keywords but that's not how our brains work if I had a document about cats I could search up the word cat or cat food and this document probably show up because I've ascribed the keyword cat to this document but if I searched up something similar to cat so for example dog or pet the document probably wouldn't show up unless I've actually gone and said that these are key words that should make this document show up you can say the same thing about memes or gifs for us all of these things are related in our brain so when we think about cast we can pull all of these things up but traditional databases don't work that way so that's where Vector databases come in because it's looking at comparing concept so the concept of cat to the concept of a tiger or a concept of a meme and seeing how similar they are