HIGHLY Scalable Vector Search Tutorial in 12 Mins!!!

Key Takeaways

building a scalable generative AI solution is always a challenge and that is where it is very critical to pick the right Vector database for you to build a scalable solution and that's where a solution like Kendra comes into picture Kendra being in the database world for a very long time has a strong Foundation to be the scalable Vector database through Aster DB so in this tutorial we're going to use Aster DB with Caso coupling them with Lang chain to build a fully scalable Q Anda system let's get started let's start the process of building a fully scalable Q&A system with Cassandra backend there are a lot of libraries that we need but also in this Google collab notebook we are using a lot of libraries primarily because this notebook is optimized to use open AI gcp Microsoft Azure if you're going to use one open AI you may not require all these libraries the main libraries that we need is Lang chain of course and also Casio that is going to help us connect the cassendra back end with asra DB so after we have in installed all the required libraries in this this Google collab notebook is going to restart I mean you're going to get a message saying your session is Crash just do not panic this is to ensure that your dependencies are installed correctly if you're doing this in a production environment of course you are not going to do it on Google collab then in that case you may not face it because everything is going to be either inside a Docker image or just a requirements. dxt file as mentioned the session or the libraries are installed so the session is going to restart now you're going see the session is crashed automatically restarting and your kernel is going to restart on Google collab notebook after it is done the next thing that we need is we need Astra DB's keyspace and Astro DB's application token even before this we need to make sure that we have already created a DB on as data Stacks so go to data stacks and go click databases and create a Victor database you have two options serverless database vctor database Vector database is what we need to create to have an AI application or an agent give a database name and then give a keyspace name it should be something that is unique and once you have the database name and keyspace name then you can select the cloud provider that you prefer like for example your organization could be using a particular cloud provider select that select the region and then click create database I'm on the free plan that's why you would see certain options like upgrade but once you create the database then your database is going to be ready and you would also notice that the database is currently under setup like it is still getting initialized once it successfully initializes you would see this pending changing to active just like this after it has changed to active that means your database is ready up and running for you to start using as a vector DB or vector database with which you can build any sort of vector DB application that you want we are going to build a question and answering for that we need to get the token get the token paste it here and the key space is something that you entered once you know the key space just enter the keyspace here PG V search after that your setup for Astro DB is successfully done now you need to upload the secure bundle from asra DB which is again you have to go to data Stacks get it from there get the bundle you have to download it once you download it it is going to come as azip file after you get it as azip file go to the Google kulab notebook and upload it on the Google collab notebook it's a very simple upload choose files upload the dotzip file once you upload it it is going to verify everything and it is going to be ready for you to use then you are going to set up the cas Center cluster at this point as you can see this code is quite straightforward all you have to do is run once you do that the next step is for you to Define your llm provider who are you going to use I want to use gcp are you going to use open AI or are you going to use Microsoft Azure in this particular case I'm using open AI I thought most of you would do that so you have to set up the open a APA key once you set up the open AI APA key now is the time for you to download some text files I'm using the very famous text file the Paul Gams text file that is part of the Lama index documentation I'm literally downloading it from their GitHub repository after you download the text files you're going to put it inside a folder called text so right now you don't have the folder now you got the folder text and you have put the pogram .txt there and you can also upload it locally which we'll see very shortly once you have downloaded the text file now you have you're going to start the vector search procedure the QA procedure so you're going to load certain libraries that you want especially from the Lang chain and you are also going to load Lang chain. stores. cendra and import cendra because we are of course using the cendra back end now you're going to create the DB connection at this point the DB connection is ready once the DB connection is successfully ready then you are going to go specify the llm resources once again this code is optimized for all the three providers but you can just specify open a and then run and the next next section we are going to create something called an index Creator and overall in this section we're going to create the store and then fill it with data and then retrieve or query later on whenever we need and this index Creator knows what type of vector store the embedding and the pre-processing of the input text to do and as you notice we have appended the llm provider name as part of the table name that is because different embedding requires different stores and that is why it has been written like that all you have to do is run that particular cell and the next section you can upload your own file like the way we already have got PR Graham text now you can upload one more Tex so right now we have got two different tick one is the great work. txt the second one is Paul graham. .txt so now after we load the load like the file now is the time we're going to chunk the text and then create embedding vectors for that and as you know that we have selected only the great work at this point and as you can see I'm creating the embedding and you can see that it it is chunking the text depending upon the default size anything that is longer than that it is going to chunk and then create the embedding for that and at this point our DB the cassendra DB that we are using has got the embeddings and the vectors are ready for us to use and you can also go monitor your progress there to see how much data transfer has happened let's quickly do a quick SQL to see the rows and you can see the text the body blob and you can also see the vectors and you can see the metadata the source of it at this point we just displayed three rows you can display five rows you can display 10 rows whatever you want to do you can just change it in the SQL I mean of course because you are using cendra you can change it in the SQL and you can see how how how many rows you have got and what kind of sources it has got it also helps you later on when you want the source as well so that's quite handy and once you see that now you can go ahead ask the question you can see that I've asked a question who did the first batch of YC include and it said I do not know it's a very important aspect of any Vector search is because you don't want the llm to hallucinate and that's why you know you see it says I don't know and you say what are the references of Sam Alman in the document it says I do not know and the reason it says is because right now we vectorized only one document of the two that we uploaded and the document that we vectorized and created embedding it does not have any of the information if you need those information then you need to vectorize the other document as well or you ask question only from this particular document which is what kind of assumption the author make for the recipe and then it says the author assumes that you need boldness but not much planning which you can exactly see in this document and you can go on and ask more questions just to make sure that your system is working completely fine we vectorized the document we created embedding and we can question and it is available for us you can verify it in the document so what we have so far seen is that we uploaded two documents but still we vectorized and created embeddings of only one document and that's why any question that is related to the other document actually shows that I do not know which is honestly a good thing a lot of people have reached out to me in the past and asking the same question and any question that is from this document easily shows the response and it also points out like we can go validate from where this is coming from so now what if we want the other document as well all you have to do is copy the path of the document go to the part cell where we are doing the load text loader paste the path of the document and then run the code once you run the code it is part of the loader object then you run it again the second cell that is going to create embeddings and it is going to chunk and load it and that's what exactly is happening now when you run the SQL code just to see the table you are also going to see the Paul Graham text and also you see the great work text you can see both the Paul Gamay and also the great work now when you run the question about exactly what we ask before which was like the first one who were in the first batch of Y for which already said I do not know now you are going to get the right answer the first batch of Y included Reddit Justin can imer and all these people so you can go ahead and then check in the document where all these people's names are mentioned so that is one important thing for you to keep in mind the document that you upload is very critical and the good thing here in this particular system that we have built a scalable Vector search system it is going to answer one from the document and it is not going to let the llm hallucinate in this particular case what are the qualities of doing great work now it is going to get the answer from the second document for you so the first thing it got from the first document now the second thing comes from the second document so as you can see that you uploaded both the documents now Vector DB that you have got has got data from both the documents and you can also get the metad data also and not necessarily only one response you can have like K response 1 2 3 4 five responses this is quite important or easy if you are going to use a ranking algorithm right now you just getting the response back but sometimes what you want to do also is you want to have like a reranking algorithm with the source so that you can give weightage to a particular Source or you can have your own algorithm to rank the responses this is quite handy in a lot of business cases especially if you're in e-commerce solution or somewhere you have a lot of items that are quite similar this is quite handy to use a retriever and get like more than one responses and then use your own logic or business rules to do the reranking after you have deployed your vector DB you can always go to the data stacks and check your Astra DB request that you have got the how much read request how much write request what is the storage that you have consumed how much is the data transfer this is quite handy for you to see how your app is performing and also you know it helps you think about scaling and all the other aspects that you would typically want to do when you have got a production solution so so far in this video we have totally learned how to build a fully scalable Q&A system with cassendra backend Cassandra's strong support is going to help you in building a scalable Q&A system a vector DB based system with the modern generative AI stack using Lang chain and Cas IO hope this tutorial was helpful to you in building a fully scalable system if you have any question let me know in the comment section otherwise all the required links including the Google collab and also the data Stacks a Deb link will be in the YouTube description for you to check it out see you in another video Happy prompting