Streamlit for ML #2 - ML Models and APIs

Key Takeaways

okay in the last video we had a look at how to build what you can see on the screen right now a very simple sort of interface using streamlight now what we want to do in this video is go through how we actually build the smart part behind the open domain q a system that we're going to put together here so i said before there are a few components to open the main q a we're going to stick with the first two for now so the vector database which we're going to use pinecone for and the retriever model which we are going to download from hook and face model hub and we're going to use the sentence transformers library to actually implement that now the first thing we are going to want to do is create our vector database or our index now to do that there are three parts or three steps we need to take first we need to download our data we're going to be using the squad data set from hogging face data sets then we want to encode those vectors um encode those paragraphs or what we call context into context vectors and we use sentence transformers and a retrieval model for that and then the next part is uploading or pushing all of those vectors into our pinecone vector database so to do all of that we're just going to very quickly go through that code because it is a lot and i don't want to focus on it too much so here we have a script i'm going to maybe zoom out a little bit so you can see so the first thing we do is import everything you don't need tqdm here but you can pip install tqvm if you do want to use that um so we are from dataset so this is hugging face it says you will need to install this so that is just a pip install data sets we're going to first initialize our retriever model so we're using the pinecone mpnet retriever squad 2 um retrieve model so this is a retrieval model that is based on the mpnet model from microsoft and it's been trained on the spot 2 data set and first thing we need to do is initialize our connection to pinecone so this is where we're going to store all of our vectors now to do that you do need an api key so i wouldn't i wouldn't write it in your code but i'm going to just do that for the sake of simplicity here so i'm going to go to this app.pinecone.io and this is free right away you don't have to pay anything um so we just go to app.pinecone.io and then you will have to sign up so you create an account i already have one so i don't need to to worry about that and i have this default api key over here like i could use that um and yeah i'm just going to use that so we can see the key if we want we want to zoom in a little bit i'll chris it's a little bit bigger um so we can see that and we can see the value there you can just copy or just press over here and copy that across and then i'm just going to paste it in here okay now this script by the way i'll leave a link to this in the description so you can just download it instead of writing it all out um because this isn't essential to our app it's just how we build our we encode all of our context and actually saw them in our in our vector database so we have that we have the cloud environment that we're using there switches back to the app um we want to check if the index already exists so i'm going to create this qa index now actually you can see in mind i already have it because i've run this code already so qa index already exists so it's not going to create a new index and instead it's just going to connect that index here right so we've just connected we've already created our our index our vector database index and now what i want to do is i'm going to switch back to our data and i'm going to run through that so i'm going to load the data search and the squad data set from hugging face now the i'm going to use a validation split because the model has been trained on the training data for squad um i want to make it at least a little bit hard so we're going to use a validation split that it hasn't seen before i'm removing any unique um or duplicate uh context in there so zoom out a little bit here um squad therefore using this filter so this is all uh hugging face data sets so syntax here and then we're encoding it so this model dot encode so this is our sentence transformer we're encoding it to create a load of sentence vectors for our context and we're converting these to list because we are going to be pushing these uh through an api request uh to pinecone again we need a list not a numpy railways you're gonna get going to get an error okay then back to the pinecone side of things we want to create a list of uh it's basically a list of two boards and those two boards include the id of each context so there's a unique id for each context uh we want the vector or the encoding uh the context vector and then we also have this uh dictionary here now this is metadata so metadata and pinecone is like any other information about your vectors that you want to include and this is really good if you want to use metadata filtering which is super powerful in pinecone and i definitely want to you know leave the option open later on i'm not sure if we'll use it or not we'll probably put something in there just so we can play around with it now that creates the format that we need to upset everything which means just like push or upload everything to pinecone so then i do that in chunks of 50 at a time it just makes things a little bit easier on the on the api request rather than sending everything at once okay so that's like how we create the index so now what we're going to do is actually um integrate that a little bit in in our app so let's switch back to our app here let's uh view it so first let's just remove this we don't need that okay save will automatically reload um so first thing we want to do here is let's initialize the pine cone connection so i'm going to just take let's just take this part of the code just copy it and we'll remove what we don't need in a minute um so we don't need we do need sentence transformers in a minute we don't need data sets uh we do need pancake so actually here we're initializing our retrieval model it's the same as what we did before so we do want to keep that in there a lot bigger api key again just saw this somewhere else um or if you are using a streamlit cloud they have like a secrets management system and something we'll look at in the future for sure but for now i'm just putting it in here so we have our api key environment and we're just doing the same thing we did before but actually we don't want to create an index we're assuming we've already created an invasive web in our app so we're just going to connect to it okay so with that we've uh kind of set up the like the back end part of our app i've smart part that's going to handle the open domain q a but it's going to be a little bit slow and we we will have a look at how to solve that pretty soon but for now what we're going to do is actually just implement this and we're going to actually query and see what we will return so i'm going to save this we won't see anything change in our app now other than the fact that it takes longer to load because it's uh downloading the uh the driver model that's the main part of the the slowness here and then obviously connecting to pinecone also takes a second as well so so now we're going to deal with how slow it is um but we will we will fix that pretty soon and now i actually want to do is i would say okay if the query is not empty because by default it is empty that's why we add that in there so i'm going to actually remove this enter if it is not empty so if query is not equal to nothing we're going to a query uh pinecone for whatever is in that query so the first thing we need to do is create our context vector so i'm going to write hq just shorthand for context selector um it's pretty standard especially if you use files before they tend to use this and i say i said content selector element query vector um so we're going to do model and code and we need to put this in square brackets and we have query okay and then we're going to convert that to a list okay so this is going to create our query vector let's write it down create query vector and then the next thing we want to do is query pinecone with this query vector so to do that we want to write first let's get relevant context and we're going to sort these in x c so like query i like context vector similar thing to the um query vector that we used before with xq but this time we're going to write index dot query and we're going to pass x cube so our query vector and we're going to say how many results we want to return now later on we're going to use streamlit a little like a slider bar to decide how many we would like to return but for now we will hard code it and another thing that we want to include here is we want to tell pinecone to return the metadata because by default it will not return metadata so return metadata equals true so these are like the extra little bits i mentioned before so included our uh the title so like uh the topic wikipedia topic that the context is coming from and also the text itself okay so we're going to return the relevant context and then we're going to loop through each of those now when we do this there's a particular format that we need to follow so our contacts are actually going to be stored so for context in xc results and resources are going to return a list and we just want the the first item in that list the reason it returns a list is because if you are querying pancake with multiple queries it will turn less of you know your answers um for each query but in this case we are only ever going to query with one uh query vector so we always enter our position zero here and then in there we will have all of our returned matches inside this matches um key value value so for contacts in there all we're going to do is write st dot write context and then we want to go into the metadata that we're returning and we have title and text here we don't want title we want text okay so let's save that and check that it actually works so again this is going to take a little while to load because we're initializing like the full pipeline with our vector database and the retrieval model so every time we run this it's downloading the full retrieval model which takes quite a bit of time okay so this is just rerun our app and now we can say who are the normans okay again it's going to reload everything so it's going to take a while we're going to fix this in the next video so we should be returning five contacts and if we scroll down we can see we have these five paragraphs now each one of these paragraphs is a single context so we could maybe we can inspect the element okay so we can see down here it's pretty horrific to look at actually so if i zoom up we can see each one of these is a single um a single one of our contents right he's here cool so i think that is that's it for this uh video so where now we have these back end working and the next one what we'll do is fix this issue with it taking forever to load reload everything every time uh which is actually super easy um but will make a big difference to our app so thank you very much for watching and i will see you in the next one bye