Create a PDF Search and Summarization Tool in less than 100 Lines of Code: GPT-Index and Streamlit

Key Takeaways

hello everyone welcome to AI anytime channel in today's video we are going to talk about semantic search and summarization and why they are so important in today's world of big data now with the sheer amount of information available to us it's becoming difficult to find the exact piece of information and this is where techniques like semantic search and summarization comes in you know by understanding the meaning behind the queries by understanding the context we can quickly and easily find the information that we need from the large set of document or large scale of documents to help accomplish this task today we are going to use large language model gpt3 and also one of the services that has been built on top of that called GPT index I will talk about GPT index in bit as we all know that gpt3 is the one of the most capable language model available to us gpt3 has you know created this revolution in the last few months that we have seen in the aai community and after that we saw GPT 3.5 and gpt4 gpt3 was extremely crucial you know on and on building on top of that we saw many other large language models coming in okay and then in this in this uh application we'll also use something called GPT index now GPT index is nothing but you know it's a kind of a library that provides a central interface to connect large language model to the external data now we have custom data or some data that we have with us how can we you know ingest those data and how we can know uh use these lla models to connect with the data that I have right so for that we need GPT index here to perform this semantic search or applications or you know graph based algorithms or summarization or question and answer retrieval or any kind of task that's where we are going to use GPT index to connect all these Technologies and techniques we are going to use a web framework called streamlit and I hope we all aware of extremely so it's a very uh simple web framework that provides you know uh very high level functions and methods to build web applications where you can integrate all of your models with ease and it also provides a minimalistic and simple UI as well so let's dive in so if you see currently you know I am on this uh documentation which says welcome to Lama index you know so Lama index is is it's a new name it's also known as GPT index which is very famous for that it says as we discussed provides an interface to connect your llm with external data if you come down you will see you know they have all this documentation guides and Etc that we are not focusing on that from documentation right now if you see the context it says llms are of phenomenal piece of technology that we all know for you know knowledge generation and reasoning they are pre-trained on large amount of publicly available data now how do we best augment llm you know with our own private data now if I have some data which you know open AI hasn't trained their large language models because they will not have the access to my data that I have I can have my academic records I can have some set of data you know some you know historical data that I have you know at my workplace you can have your clients data for example how will you use this private data how can you integrate this private data with large language models not only with you know open AI gpt3 based model or any other large language models as well that's where this kind of you know uh features comes in that that is given by GPT index here that you can see it says data in in data ingestion data indexing and what not they have multiple you know solutions that you can see they have a full stack a web app with llama index they have very good documentation as well you can go and you can have a look at their documentation so what I will do I will head back to VA score you can see I am this is my vs I have created a folder called uh semantic search streamlit and in that I have created a virtual environment in that virtual environment I have installed all these requirements there are only four requirements okay this will of course install other dependencies as well so we we need to streamlit because we are going to use extremely to build the web UI where we have this you know interface where this model will be working together on a PDF file and then we have Lama index also known as GPT index that we discussed and then we need Pi pdf2 so Pi pdf2 is a dependency required by Llama Or GPT index to handle a PDF files because we are going to upload a PDF file and on top of that PDF file we are going to ask some queries and we'll see if these techniques are able to you know generate those responses or not from that PDF and then we have dot EnV where we'll be handling our open AI API Keys let's head back to app.5 so as I said we need to have a extremely application so I'll just import streamlit import streamlit as HD and I'll just first work with the streamlined code and then I will go back to the functions for GPT index and I will see what kind of functions that we need so uh let's first set the page config so there is a functional method in streamlit that that helps you you know uh handle the page layout so we have a page layout if you want to full width if you want a container you can handle that with set underscore page config so I'm just going to do that set underscore page config and inside that I will pass a parameter that's called layout equal to White so I need a wide layout that's what I'm going to do here say Dot Page config now let's also have a title for this app I'm going to call hd.title and let's call it you know semantic search or and summarization app and let's call it application application this is uh well not applications application this is my title and just to give you some information set underscore page configured will be the first line of your HTM unit code if you are using this uh method as well you know to build the application so now what I'm going to do now okay I'm going to have some let me just charge it so what I'm going to do I'm going to have a a file uploader we will upload our PDF file you know so we will will create this uh semantic search feature for that PDF okay so now for that what we can do let's call it you know uploaded PDF or something so uploaded PDF and in this uploaded PDF I'm going to use file uploader method of streamlit so SD dot file uploader and in file uploader what I'm going to do I will say okay let's label it so in or upload your PDF file upload your PDF upload your PDF and then let's call it type also which is a list so in type I can pass you can pass multiple actions and I'm just going to pass PDF here so when I say it as a list let's keep it the list if you want to also work with docs you can also work with docs as well so I'm just going to work with the PDF here so type equals PDF and now I'll do a uploaded PDF is not known if they upload the file if uploaded PDF is not none what I want to do I want to divide the page into three columns okay I'll use three different columns okay so that's how you can also use extremely dot columns not to divide this so column one column to column three and in this column one column to I'll use HD dot columns so HD dot columns and you can also uh again you can handle the width like if you want to now we have three columns if I want to give more weightage to the column one that column one should be bigger in the width so you can also handle that so I'm just going to divide into two is to 1 is to one wave Co so all these columns column one two three this looks good now so with column one so whatever I am going to do in the First Column we have to start like this with column one and in with column one I'll have an input file so let's let's first see so what I will do with column one let's just do a pass here by now okay let's do a path let's first run this application so how do we run my stream net application guys so you can see I am in this virtual environment on this uh folder where I have installed all these dependencies and libraries so this is how you run a streamlit app you say extremely run app.pi which is your file name so I will do extremely to run app.pi I'll hit enter what it will do it will open a you can see it's open the app in a browser it says semantic search and summarization application we have a very good uh file uploader which says limit 200 mb per file a PDF file that you have to upload and what we will do will have you know uh we want to show the PDF so we need a PDF viewer okay so so I because these all PDFs are I'm assuming that you upload uh PDF from public domain or you know which is available on internet for at least for this video I'm not using any confidential data so I would like to have a PDF viewer and this also a stimulate feature now what I have already done I have functions that available from on streamlit community in the extremely discussion forum I have used these functions a lot in my previous videos as well and my my at my workplace so how can you save a uploaded file and how can you display that PDF so what I'm going to do I'm going to use this functions so if you see the first function I will explain this let me first copy this function and I will give this gist link it is GitHub gist I will give the link in the description as well so if you come back here what you have to do in the vs code before this I'm just going to use this so let me import OS for that by the way so we need to import OS I don't know we also need to import of course base 60 for another for this other function as well but it's okay so import OS for now if you see the first function which says function to save a file so what we have to do guys we have to save the this file that we are uploading to a local directory and the directory is something called Data in that data folder we are going to pass this as well to the GPT index as well that will load the data from there I will show you what I'm talking about in bit for now what we are doing we have this data folder where we are storing this uploaded file you know that we are uploading it from our local machine okay so what I'm going to do now so let's do one thing so in this you see that function name is save underscore uploaded file we have uploaded file as a parameter and then we are using OS library to save this in this data file we are taking the file name by the way uploaded file.name and then we are getting the get using the get buffer method okay and then we are saving the file here okay hd.success it will also show you in the UI now with column one pass I will remove the pass here so in column one what we are going to do guys we are going to Let's Define a variable called input file that is going to be our input file and in this input file now I will use that function so save underscore uploaded file and here I am going to pass the uploaded PDF you see this variable where we have storing this file uh storing this PDF then we are uploading it from HD dot file uploader so input file equals to save underscore uploaded file and here it goes uploaded file uploaded PDF bloated PDF input file now we have storing this in this input file now let's create a variable so we can also use it later on so PDF file and I'm going to pass this path so the path is if you see it's in a data folder and in this data folder what we have is uploaded file uploaded PDF dot name if you see here we were having uploaded file dot name so now our operated file is nothing but the uploaded PDF now so I'm just going to use the uploaded PDF dot the name that's it and now this file is now assigned to this variable so now we can use this variable wherever we want in this entire code base okay so now PDF file now just hit enter and let's view this PDF so I'm just going to do PDF view okay how can we use this PDF view now guys okay so for that we also need a function uh that we can store these PDF files on a streamlined UI okay which is very interesting uh by the way if you want to use this function please use it for your other tasks as well where you have a requirement to show the PDF file so what I will do I will go back to my gist you see I have a function called Define display PDF that I have taken from the discussion forum of streamlit community Whoever has created this function kudos to them credits to them by the way what I'll do I'll use this function Define display PDF file function to display the PDF of a given file we are opening the file again the file operations basic python file and we have base64 encodings have been done here you see and now we are using iframes you know to Source us on the wave uh stimulate wave UI okay so iframe we are passing this and now using markdown which is supported by streamlit PDF display and you have to do this unsafe allow HTML equals to true you have if you have if this is false you will not view the PDF there okay so hd.markdown PDF display now now we'll use this display PDF function to show this PDF file that has been uploaded by the end user which is me in this case so PDF file PDF view I'll use this function called display PDF and in the display PDF I'll pass this variable now which is to our file so display PDF and this will go PDF underscore file so PDF underscore file and that's it so now let's see if we are getting any you know uh changes here so you have to do rerun once you do the rerun now what we have to do what we can also do let's upload a PDF file so in this data I will upload this PDF file it says name base64 is not defined cool so what we have to do we have to import base64. import base64 and I think okay let's see this should do input base 64. I'll do a rerun so you can see now we have this save PDF and you can see a beautiful PDF viewer okay on the streamlit application itself guys it's very interesting can streamlit is becoming so powerful now you also you might have seen that there is a recent development in the wave frame of something called Pi script has been officially released you might have seen some videos on internet as well but stream it is also becoming uh you know a stronger day by day the community is becoming very strong so you can see we have this PDF viewer and we're also getting a status message which says saved file PDF to directory and you can see your PDF so this PDF is about sustainable development goals or sdg you know by the United Nations and we are going to use this PDF to interact with this you know we'll have some queries you know you want to retrieve some information from this PDF file you know with help of generative AI That's why we are going to use gpt3 and we'll use GPD index to do this indexing you know we have embeddings what it does it it took where you will see what I'm talking about in the terminal okay we have word embeddings it kind of creates the embedding for the tokens and then it kind of creates a Json file so that I can that you can also use it Json file and then it you can ask your query then it will retrieve the information okay so you see this is a PDF file so our first function is complete or the First Column sorry we have couple of functions here for the file operations and showing the PDF on the stimulate UI but our first column is done completed so we have two other columns where we are going to use you know uh GPT index now how can we uh semantic uh search and summarizations for that let's import the library so what we have to do let's first do from Lama index which is from Lama index import GPT the first thing is simple vector so Vector index we need Vector index we also need directory to load the data from directory so simple directory reader this is right and then for summarization we need list index because I will show you that's little different than the vector index so GPT list index that's it we need these three modules or classes from Lama index GPT simple Vector index simple Library reader and GPT list index okay and now we also need something called from dot EnV we need to uh we need python load uh dot EnV to handle the open AI API so from dot EnV we need import load dot EnV this is what we need and the first line goes like this now load underscore dot EnV which is a method that also Define the environmental Keys OS environment path okay this path is nothing but the the dot EnV the way you have stored it so OS dot environment okay open AI and this go underscore API and now it goes key and now this will be OS dot get EnV so OS dot get EnV and this is not there this should go inside this open AI the same thing open air API key this makes sense so now we are done with open AI API key and stuff so now what we're going to do guys in this now we are going to use GPT index and in GPD index we need couple of functions the first is semantic search and the second is the summarization so what I'm going to do let me just Define that function so the function will be let's call it semantic so dip diff semantic let's call it you know sword and in that we have to pass the query right so let's call it query as a parameter and depth semantic search query and now what I'm going to do let's call it a variable as documents and in this documents we need to load the data okay so how we can do that so simple directory reader you can see simple directory reader and you have to pass the path of your data folder so in my case it is nothing but called Data you can see this data folder over here and now we use this method called load underscore data you see this is how you load the data you know when you are using GPT index or Lama index okay so now this is document now we're going to do the indexing here so let's call it index and what I'm going to do we're going to use GPT simple sector index very high level apis guys simple to use GPT is simple Vector index and now from documents from documents and will pass these documents which is correct which is our variable name as well so we are handling we are operating on a doc file or a PDF file so from documents document index and now we'll have our response so response is nothing but index dot query index dot query and uh index.query and let's pass this query so we are going to pass this query which is a parameter which will which will take the input from the end user on the streamlit UI so this is our response index.query and now you say return response so we have this semantic search query document simple directory reader we are loading the data here then we are indexing it and then we are getting the query uh from the end user and then we are doing performing this query method on this index okay on this response you can also save this you know in a Json file guys if you want to do it but you can figure it out from the documentation as well and you can extend this app further now the second function that we are going to use is let's call it summarize so I'm going to use summarize and in summarize what we are going to what we have to pass we have to we have to pass the file here the PDF file so let's call it file let's summarize and again the same thing so I'm just going to use this same let me just copy this this is how we're going to load the data documents and now in index it will not be Vector ended so we need to do the list index so I'm just going to use list index from documents and here it will be document so this is okay documents and now what we are going to do let's response and index dot query here you'll have our default prompt and this default default prompt goes like this summarize you know summarize the uh summarize the document okay okay now this is my default prompt summarize the document and let's call it response mode either tree summarize okay they have a compact they have different uh parameter here that you can use so I'm going to use three underscore summarize because we are working with a summary task three underscore summarize and knowledge let's just return this response pretty simple right so first function is for semantic search where we are passing a query as a parameter and then in the second is the summarize where we are passing the file or the file path that will pass so response index dot query you can see this is a default prompt you can change the prompt you can also have this prompt from the end user as well but I don't want to do that 300 summary this looks nice now what I'm going to do guys now next is let's let's work with the column two so with column 2 and in the column 2 will have a success message as we have in column one that from the function itself will say you know search area okay this is going to be the search area so let's call it search area search area actually dot success search area and this is a status message now we'll have a query underscore search and here it comes what you're going to do we'll have it you know we'll have something like text area okay that where end user on streamlit application can ask their query so we are going to use this method called text underscore area they also have something called text underscore input but that looks that's a single line This is multi lines you can have questions on multi-line as well so St dot text area and in this text area let's have something called No search your query and now query search search will query what we are going to do we need a check box let's have a checkbox you can also have a button inside it but let's have a check box so I'm going to do if St dot check box checkbox is a method I'll just have this option called search let's assign a name to this checkbox and now here it goes your where we'll use the function first function which is semantic search that's where we're going to use so let's call it let's first have a informations okay so hd.info and we call it your query we print the query okay your query and let's have a space and that here will have the query search this looks good query search hd.info is done now we'll use that function so let's store that in a result excuse me reserved so let's call this result and result equals that goes the function name which is semantic search and on this query search so this is going to be excuse me this is going to be the query underscore search that's it now what I'm going to do I'm going to uh let's write this maybe so let's write this HD dot write result this is so this is column two now what I'm going to do for column three where we let's use that function as well quickly because it's pretty simple and straight so AC dot success here we are doing let's also use success again guys so success I'll say summarization or your summary or your summary something like this if you dot success ah this ran let me do a control C by the way you can okay actually dot success and now here we'll use a summary result and in summary result we'll pass this function so summarize and we'll use the PDF file as I told you that we are restoring that file now in this variable so we can use this variable wherever we want in this code base do not have to load every time this complete path so summary result summarize let's use this PDF file passing this PDF file I want GPT index to summarize this PDF file with the help of course the ppt3 and we are using WC3 here so summarize PDF file again HD router right let's write this summary result that's it now let's just let's just try it and then we'll explain the functions once more the entire code from top to bottom you can see it's around 70 lines of code let's see what kind of applications we have built here what kind of application we can you can build using this now if you come here we'll just do a rerun okay now this is okay so what we will do just give me let me see so the file extension may be the problem here guys so just uh let's see so you can see uh OS dot environment open a I think that's the file extension earlier it was not an uh correct file extension there was no dot no hidden file so what I'm going to do now let's come back and see so localhost 8501 on the port which is streamlit appearance now here we'll upload this file and now you can see we have few things here guys the first thing is again the PDF viewer that you already have seen and then we also have something called two other columns so see this column the First Column is twice the width right it's taking more weight than these two other columns okay if you come back here and you will see it what I'm saying this again embedding token images I was discussing about GPT index and no no let's ask some questions the first question that I would like to ask you know the information that I want to retrieve from this document is like how many you know hdgs goals are there you know according to the U.N now this is my first question I will click on this search when I click on this search this is the check box so it's a checkbox and this is my question I have made a typo there as well okay but anyway your query you see this is my query you will query how many hdgs goals are they uh there according to the United Nation you can see it's running on top and I have got my response we say there are 17 sustainable development goals according to the United Nations including gender equality you know ending discrimination against women and girls eliminating violence against women and girls eliminating harmful practices and blah blah blah so there are seven teams like this okay so what I'm going to do now and if you come back here you will see something called total llm token ugs right for all your queries the number of times you are making this queries or you can find it over here all the logs in the terminal okay how many llm tokens have been used and all those things over here okay now let's come back let's ask some other questions here how many hdgs goals are there according to you maybe I can ask this question you know gender equality okay which HDD goal is related to and you can see we also got our summarization guys so before asking this queries let me also explain summarization it looks beautiful uh in this three set of columns so in the First Column we have our PDF file you can upload any PDF anyone to retrieve any information you go ahead retrieve you can have a download button as well you can go and watch my whisper and transcription video you know we have a minute submitting and a whisper video where I have this option of you know downloading some files how you can download this result this stick in a text format or something like that you can also extend this app further please do that and very helpful you know to reduce the manual efforts and if you are doing some mundane or TDS task you can also use this kind of application you can also have a CSV file with list of questions where you can upload and it will generate the answers for you please go ahead and extend this app further and please drop a comment in the comment box if you have done something you know uh with this if you have extended this further having option to upload CSV files with list of questions and you automatically get all your answer in the CSV file as well with questions answer and questions answer now you see this summary it says this document outlines the sustainable development goals or sdgs and the actions and indicators that the food and agriculture organization is taking to support the monitoring of these goals it talks about these goals like 2 5 6 15 and it's talking about why is this important and how this can help the developing countries or the underdeveloped countries you can read the summary here entirely right you can also use uh you know some methods if you come here down you see the response mode you can have a compact you can have tree summarize you can have different type of response mode you can read this in GPT index documentation now if you come back let's ask some other questions as well which HTC goal is related to you know uh oh it's it's already showing this goal seven ensure access to so it's already you can see it's running to gender equality so let me just uh let me just click on this search and now click on the search now this is my query your query which hdg goal is related to gender equality let's see if it's able to give us the right answer here okay so you can see this it says goal five this is goal five which is the correct answer guys which HTC goal is related to gender equality goal five which is gender equality so this kind of questions you can ask you can see this is document that I uploaded it's a big document around 70 Pages now you can upload multiple document as well it depends what you want to do with this app you know or this kind of techniques or technology like large language models with GPT index what kind of applications you want to build some cool applications you can build it in your personal capacity or you are doing some freelancing or you're working at your workplace you can combine all these things to build uh you know presentable solution to the end user if you come down let's talk about zero hunger you know let me first uncheck this but can you tell me for example what can you tell me on zero hunger for example let's see if it's able to you know give us some response now if I click on that search now this is my query now what can you tell me on zero hunger okay so we have our summary there we are you can see the PDF file and this code will be available on the GitHub repository okay I'll share the link of the GitHub repository you can go you can just change your API key you can also add your if you want to make it completely private you can also create an instance on Azure Services go ahead and create an Azure service use that Azure API Keys the model endpoints Etc just replace this line it should work fine if you come here and you'll see wow we got the response guys on our query what can you tell me on zero hunger so this is zero hunger is one of the sessionable development goals sdg is adopted by the United Nations you know uh 93 193 member states in 2015 you know including hdgs 2 which is zero hunger which is httg2 by the way and it's giving you all this uh responses here you can also if you want a bigger response just increase the parameters you can have some parameters over here Define like maximum number of tokens as etcetera that you have already seen with open on open AI playground as well if you have tested open AI playgrounds so I hope you liked the video guys I think a good output that we got in in last and I think that's it for today's videos uh guys on building a semantic search and summarization app I hope you found this tutorial helpful in understanding how this you know uh technology can be combined to make searching and summarizing much easier you know if you enjoyed please give it a thumbs up and if you haven't subscribed the channel yet please do subscribe and feel free to share your suggestions in the comment box uh thanks for watching see you in the next one