5 Steps to Build a Question Answering PDF Chatbot: LangChain + OpenAI + Panel + HuggingFace.

Key Takeaways

hello in this video Let's Make a question answering PDF chatbot just like this we upload a PDF file enter our openai API key you will need to set up billing and open AI because opener API is now free so yeah make sure you do that otherwise it won't return anything and enter your question here for example how many AI Publications in 2021 and click run so the example.pdf file I actually got it from the AI index report 993 I downloaded the first chapter of this report it has more than 50 pages so it's a good length Okay now we can see our result 500 000 AI Publications and you can see it also returned two relevant Source text chunks and the relevant information is actually in the second text Chunk right here okay so how about let's ask another question when was GPT to released let's change the number of text Chunk to be one so that we're only passing in one relevant text Chunk to our language model now we get the answer I think that was correct you can also see this app in the hagenifer Space page directly I will add the links in the description so you can give it a try and let's see just to make sure it works here as well okay cool we'll get the same results now let's take a look at the code and see how to build this app with only five simple steps and I'll show you how to do that okay before we get started we need to install needed packages and import packages we're actually using panel as our dashboarding solution I am a big fan of panel I wrote a blog post before um three ways to build a panel Dashboard please feel free to check it out I'll also Link in the description the first step of building our dashboard is to build widgets I have a file input widget which you can see is pn.widgets file input uh so this is where you can choose our PDF file file input dot value should return the information of our PDF and then open AI key is a password input widget this is where you can type in your password and also we have this widgets object which combines a lot of different widgets which is we have a prompt The Prompt uh I guess this is a little hard to see let me move it up here the layout is we want to organize things into one row the first element of the row is two elements first element is prompt which is a text editor where we enter a question for example how many AI Publications in 2021 um and then we have a run button which is a button widget is this run button and then we have some advanced settings uh so in the advanced settings we have a select chain type so this is a radio button group where you can choose one of the options so in my previous video I talked about four ways to do question answering inline chain and I talked about what each chain type is so basically stuff is you put everything into your language model all of your text map reduce is you separate things into batches and input each by separately and then I have the final widget is number of chunks which is the select K widget a integer slider if you want to try on other types of widgets you can check out panel documentation so this is actually runnable on the web with the magic of pile died and Pi script so it's actually quite nice as you can see we have a text input widget here the default starting value is a string but you can change the value to whatever you like hello world and now if you get widget.value you will get hello world and then you can change the value and update the content of the widget so there are a lot of different types of widgets so if you want to play around this is a place for you to take a look so now we have talked about widgets the next step is to define a question answering function to get our answers of a question according to our PDF this is what the function looks like if you have what in my previous video you may notice this is a method two that I mentioned in the video in this function we load our document we use the pi PDF loader because I mostly want to interact with my PDF if you have a txt file if you have a CSV file you can use different file types accordingly you can even write a if else statement to check okay so if my file is ending with a PDF I use PDF loader if my file is ending with a txt I use a text file loader and then the second step is to split your documents into different chunks select different embeddings you want to use create Vector stores to use as index and use the retriever interface here I want to search for k text chunks that are most similar to my question text Vector so that's the similarity search here and then we create a retrieval QA chain to answer our questions as you can see we have four parameters the first parameter is file this is the file we want to read from the PDF loader the second parameter is query which is our question for example What's the total number of AI Publications chain type here I'm using stuff but you can change it to mapreduce or other things and K is a number of relevant chunks and I want to pass into the language model to so here's an example running this function to run this function we need to set up the open AI API key here the result is we have a query we have the result we have two relevant Source documents oh by the way here I'm using openai models which is not free by the way but there are many other language model providers all the models on hanging face I believe is free but it's basically the same code so that's that let's comment this out because I don't actually need it in my app okay so now we have to to find the question answering function as you can see this is purely Lane chain it has nothing to do with our Panel App yet we want to show the output of this function in the app so how do we do that so this is step three where we show the output as a panel object so that we can see the output actually in the app so okay so here's how we did it first of all because this is chatbot interface I want to store all my piano objects in the list and we have a question will show up as a question and then we get an answer you will append to this list and then if I have another question it will append as another question this list will keep all the chat history so that's why I have a empty list over here and this function actually I Define the openai API key from the value of my openai widget and then I save my PDF file to a temp file file input is the the file info widget and we can select a file we like if we have upload loaded a file which means file input value is not none I want to save this file into a temp.pdf file and then I saved a prompt value as prompt text as you remember hopefully prompt is a text input widget prompt value right now is the value we typed in the box if there is text in this prompt text I want to run the QA function that we just defined in step two where the file is the cam.pdf we just saved and the query is the prompt text chain type is the value of Select chain type right now it should be stuff K which is the number of relevant text chunks we want to retrieve is the value of the select K widget as you can see it's is two right here so we want to convert the result into a pen object combo's list extend by this object so let's take a look at this object this first chunk right here returns our question this is the Emoji corresponding to this prompt text is our question prompt now let's take a look at the second chunk here we have again our Emoji which is a robot Emoji I thought that was kind of cute and then we have the result that was returned from the language model 500k relevant Source text and then this whole thing I know it might look a little confusing if we have more than one document in the source documents for example if we have two text chunks I want to separate those two text chunks by a line here so this is what this is in the end we just do a PM column with all elements in this list so basically we want to display everything in a column is that clear I'm hoping that's clear so just to remind you what we just did in step one we defined our widgets step two we defined a question answering function that's basically all line chain code there's nothing panel step three we created a panel object that contains all of the answers questions and the source text chunks so this function actually returns a pin object that we can use later in a panel app now is step four now you might wonder how do I combine this function with my widget I only want to run the function and show the output when we click the Run button so we want to be able to somehow bind this function with the wrong button so that's our step four where we bind the Run button with this QA result function so that only when the Run button is clicked this function will run this is also the reason why we have kind of a placeholder for a parameter here we're not using this parameter at all at all but this is just the parameter as to be able to add a widget on button right here in pn.bind and I actually want to format my output into a box uh just you can see here that's why I'm using another another widget called PM widget box to show my output in a box but of course you don't need to do that if you just do interact QA interactive it should show up and the final step step five is to define the layout you can write markdown in the panel app and then we have some descriptions on how does this app work and then you can see pn.row which is the third row here I guess we have a file input widgets and open API key here and the output box and this output box and then the widgets is the screwball widgets you need to add the severable here because you want to make sure your app is servable to to serve this app let's just do panel serve name chain and this file name okay so this is our app just to double check it works yeah yeah it works great okay so now you have a panel app okay so in the final section of this video I want to show you how you can deploy this app to your hiking bit space so that you can share your app with the world okay so you can see all my files here let's first take a look at the requirements.txt so in the requirements.txt this is all the packages you will need for your project and then here is The Notebook file as you can see it's the same as what we have just gone through we need to save this PDF file into a DOT cache directory um because it is because I'm going to show you right now in the docker file we made a DOT cache directory and we give permission to this dot cache directory so that um so that we can save a PDF and read from a PDF from this directory what are the rest of the coding this Docker file first of all we get a 3.9 a python 3.9 Docker image we set out working directory AS Slash code we pip install all the needed packages from this requirement.txt file and then we copy everything else into the directory everything is in the code directory right now now this is the important bits where we run the panel app we ran panel serve to serve our app here is exactly the same but you need to kind of organize everything into a list I know it looks a little strange but it's the same you do panel serve remember we saved everything in the code directory that's why this file is in the code directory right now you need to define the address and the port allow websocket origin so this is actually typing this address here this is where you can see the whole app uh you will need to change this bit into your corresponding this is the username this is your app name as you can see piano PDF QA so this way you will get a separate page for your hiking face app the final thing that's special about about this app is because we're using a chroma Vector store it actually will need permissions to this directory that's why here I made a directory called dot chroma and give permissions to the chroma directory yeah so this is our app I hope you find this video helpful thank you see you next time