Extractive Q&A With Haystack and FastAPI in Python

Key Takeaways

okay so we're going to go through putting together our reader model and the fast api wrapper around all of this so originally i was just going to do the reader model in this video but i realized it's actually super easy so we'd almost be done already if we were just doing the reader model so what we're going to do wrap everything up in the api and then we'll add the reader model into that so this is where we got to before we have this which is our jupiter notebook and this has everything that has our document store the retriever and we're just retrieving a few things so we've got a few sort of context here now obviously notebooks are fine for when we're putting things together and testing things but we need to switch over to actual python files so we're going to switch across to vs code over here so this is our project directory on the left here so we have the code and the labs in here which we've already put together we have our data which is meditations and i also put together this requirements text file so in here these are all the libraries that we need to pip and source so we have fast api from haystack which we already installed and you've gone they're the two that i want to focus on so fast api obviously we're using that to build our api and uvicorn we're going to use to spin up the server that our api will be hosted on so that's our requirements.txt i'm going to create a new folder in here now i'm just going to call it qa service i mean we can change these later on i'm not really sure what to call them all so we're not in code okay and then in here we'll create our first python file and this is just going to be our api so we'll call it api dot pi and now we'll start putting together our actual api so let's switch across to the window and the first thing we need to do is import fast api so do from fast api import fast api and then we want to initialize our api here so just call it app equals fast api let's need to fix that there and that initializes our api and then whenever we want to create a http method all we need to do is this so we do app and then in here we write the method that we'd like to add to our api so the only one that i think we're going to need at least at the moment is a get method so we'll add that and then we also need to add the path to this method and in this case we're going to be querying our q a model so we'll just create the query path so it's pretty good and then we do async and here we have our function so this is just going to be the query function and we want this query function to at some point accept a query but for now we'll just leave it and we're going to add that in a moment now this will just return [Music] no hello world so that we know that it's working so let's spin this up so i'm opening the terminal in this folder at the moment so at the moment we're at the high level of this so we'll go down a few items so we have we want to go into the code directory and then in the code directory it's qa service which we made just a moment ago and from inside here we initialize our guvicorn service which will host our api so for that we just write uvcorn and then we want our file name which is api and then we want whatever we've called our fast api instance in our code which is app up here and then we'll use the reload flag as well so this means that whenever we edit our code we don't need to re-initialize the api instance in order for changes to load okay so now we can see that here we have our instance so i'm going to copy that and open it in my browser it's over here enter that and we see this detail not found so that's because we just went straight to the index so let me just have the query endpoint and there we get hello world so this is the end point that we're going to use for our fast api so we know that's working and now let's add in the rest of our code so that we can begin querying our qa stack so if we just check what we did in our jupyter notebook all we actually need to do is basically copy loads of this across so let me just create a new cell here and i'm going to take all this and we're just going to copy it across and honestly this should really be all we need okay so we'll copy this and take it into our code here so before we initialize our api we're going to initialize the rest of our code so we have our document store we initialize that it accesses the aurelius index and then we have our retriever here do we so import them as well and then as well as the retriever we also have the reader model which is the next step so first obviously we need to import that and we do from haystack reader farm import farm reader and then when we put all these together we're also going to use something called a pipeline and to use that we just want to put from haystack pipeline import extractive qa pipeline so the reason it's extractive qa is because it's question answering and we're extracting the answer from the text rather than generating it with model so that's why we're using the extractive qa pipeline now that should be everything that we need to import and then all we need to do here is we initialize our reader model which is farm reader and then in here we want to include our model name or path and that's going to be equal to deep set let me do this on another line so deep set bert base case squad 2 this is a i don't know if we covered this already i don't think we did so this is a pretty standard q and a model that i think i think a lot of people use so it's it's pretty safe uh i want to start with and then after we've initialized these three parts of our stack we need to initialize our pipeline object so for that we do pipeline and then we're taking our extractive qa pipeline and then we have our reader which is going to be equal to our reader and we also need to pass the retriever which is equal to our retriever so that should be everything that we need to do in order to initialize our model so now what we want to do is we need to query our data so we do pipeline dot run and then in here i'm just going to put like a random query so i was the one we used before it was what what did your grand father teach you okay and just pass that as the query and then this will return this is going to return a dictionary so then we can just return it straight away now let's see if our api is still running i think it should be okay um yep so it's just starting up again now okay looks good so i think that's good and let's try and open it up over here we should get loads of text yeah so that's cool so now we're getting the query what did your grandfather teach you and this is pretty hard to to see what you see we get this list and then this goes all the way through here and get all of these different answers so the first like the top rated answer is what did your grandfather teach you and it's act and speak okay uh in sleep scenes act and speak it's all like children from their parents okay well it was interesting um okay so it's not amazing so let me try instead of using [Music] instead of using the dpr retriever let's try swapping it out for a bm25 retriever which is it's basically just an algorithmic retriever and it should actually run faster as well elastic search retriever and all we do is swap those around and i think the only parameter we need is the document saw yeah so okay let's let that start up again okay just waiting let's try this one see how it compares and then we'll go one of these i think so let's let's go and just refresh the page again okay yeah this is a lot better which is weird i kind of thought the other one would do better but that's fine maybe we need to fine-tune it a bit more than what we have so what we have here is what do your grandfather teach you and we get good morals and the government of my temper which is i think pretty exactly what i wanted it to return so that's good i see here there's like a new line character as well which i think we need to format the text before we put it in to remove those but that's cool it's pretty cool so that's really good it's working but we're not actually making our own queries so we need to add that and to do that all we need to do is add q in here and this is our query string and in here we just make that equal to query and now if we do that rather than opening in the browser i'm gonna i'm going to open insomnia here so insomnia what you can see now is a it's just a http client so it allows us to send requests really easily and just form everything in a nice way i should have opened this before i'm not sure why i didn't um i kind of forgot about it so localhost 8000 we're going to the query endpoint then in our query we have the q parameter and then here i want to say what did your grandfather teach you okay and then we send this and we should get a response on the right cool and this is a lot better so it's good and let's try something else um so i think i've read a lot of time he just like talks about the universe so what is the universe and send that takes a while to run at the moment cool so we get a query and then we get answers and it's a well-arranged universe the universe loves to make whatever it is about to be interesting that which knows beginning and end and knows the reason which pervades all substance and through all time by fixed periods revolutions administers the universe yeah it's pretty deep yeah that's cool so i think that's i think it's everything i don't want to include anything else so that's pretty cool so now if we have a look at this again we've just done like this entire section here and the api i mean we'll probably make changes in the future but that's that's kind of all this done so you can cross that off and then the next bit is going to be probably the most difficult bit at least for me which is going to be the front end in angular so should be pretty interesting um so let's see let's see how that goes but for now i think that's it so i'll see you again in the next one