Summarize Papers with Python and ChatGPT

Key Takeaways

what's up guys welcome back to the channel in this video you're going to learn how to summarize papers using the judge PT API so on my left I have my Jitter Notebook on my right I have the paper that we're going to be summarizing this paper is called quantifying attention flow in Transformers the process to do that is relatively simple we're going to start by importing our dependencies we're going to use OS to handle the file stuff ipdf to parse the text in the PDF and green Jacks in case we need some text packaging and open AI to handle the calls for the change API so I'm going to run this and now in order to summarize the paper all we're going to do is first we're going to set a string that will contain the summary so that's what I'm doing here and then we're going to open the PDF file by first we set the path to the PDF in this case it's all my internet folder called PDFs and it's called paper.pdf because I'm very original and now we're going to read the PDF using the pipe PDF true package so I'm open the PDF I'm reading the PDFs and I'm going to Loop through all the pages in The PDF file and so for each page in the PDF we're going to be extracting the text from the page so here I am accessing the pages of the PDFs that I read with the PDF reader from pipe dev2 I am accessing the index for each of the pages I'm extracting the text and I'm making it all lowercase after that we're gonna get we're gonna make our alls to the judge PT API so here I'm making the call to the charging pdpi I'm creating a message in this case I'm using the GPT 3.5 turbo which is the default model uh the message is going to Define first a row for the system and this is telling judge PT what is the behavioral profile that I wanted to incorporate in this case I wanted to act like a helpful research assistant because we're talking about summarizing scientific papers and then we're gonna send the message in this case this message is very simple I just say summarize this and I give my bad itch page of that PDF so this is a very specific type of summarization where I'm just summarizing all the pages and then I'm concatenating that together and then after we've said that we're gonna get that summary summary from that page and we're going to add that to the empty string that we set in the beginning so here I'm just adding the summary of that page into that empty string which will now have a summary for all the pages of the PDF now essentially an evolution of this approach would be to divide to make the summaries by section by header but I found that you know in practice this works pretty well so now I'm I'm gonna I'm gonna write the summary to file so in here I'm just replacing the name of the PDF with summary.txt so that I know where the paper is and where the summary is and finally I'm saving the active file and we're closing the PDF file which we opened here so now if bye so it's maybe I found up close so now I can run this and here I'm going to be opening that file and reading it to the prompt reading it to my Jupiter notebook so that you guys can see the summary there you go there we have it a summary of this paper which upon reading I found this to be a very competent summary of that paper so it discusses attention rates the transformative explain model decisions specifically a reliable sort nature of attention rates as explanations yup that makes total sense for the paper so pretty happy with the summary and that's how you summarize a paper using the changing API thanks for watching don't forget to like And subscribe and see you next time cheers