Graph RAG Evolved: PathRAG (Relational Reasoning Paths)

Key Takeaways

hello Community it is so great that you are back today we'll look at the latest technology of rack we will have a very short Excursion rack graph rack light rack and the latest development half rack I will explain you how it works and give you the code for this so you know that the classical rag retrieval augmented generation we have a lot of techniques to make rack work and just here from rev8 those are the advanced rack techniques that we know today we start with index optimization fixed size chunking recursive chunking document based chunking semantic chunking we have the pre- retrieval optimization the retrieval optimization itself then the post- retrieval optimization all the different methods and you know it is not really working perfectly so therefore we developed graph-based rack system and they work much better however if we look at two graph-based rack system the graph and the light rack here today in this video we will discover they also to able to be improved upon because they retrieve some excessively broad subgraphs they have noisy prompts we have increased computational cost because of this and ultimately we have a suboptimal llm performance and then we talk about the solution because graph rack evolved the very last technological state is pack published just days ago they said set out to solve all those limitation so pathre and the rise of relational reasoning here for the r system is our topic let's start here let's open this video and of course if you're new to graph what is the most important paper February 19 2025 version 2 here graph R he approach from quer Focus summarization graph is simply the order to tell us hey rack fails on global questions that are directed in the entire Text corpus such as what are the main teams in this data set R cannot tell you this so therefore they they developed graack graack uses all the information from the note and the edges within certain communities and those communities are here the solution here because they build now here a graph index in two stages First Step derive an entity knowledge grow from The Source documents to your text database for example and then second regenerate Community summaries for all groups for all thematic clusters closely related entities so given a question each Community summary each cluster summary is now used to generate here a partial response maybe the query relates to multiple clusters in your knowledge base before all partial responses are summarized for the final response for the user very simple visualization we have here all the text documents in the text database we have the extraction and all the chunking going on then we have the text junks and then we have the summarization we have the entity and relationship in domain tailor summarization and we buil here our first knowledge grath this is done in indexing time so when we have end the query time we have communities in the knowledge gu so we evaluate now relevant communities to the query we have gra communities domain tailored summarization are already waiting for us so we do have the community summaries ready then we have the very focused summarization we have partial Community answers for multiple topic clusters and then we just have a global answer that is the fusion of all the sub answers it works great and I can tell you graph rack shows substantial improvements over the classical Vector rack Baseline both for all the different factors you notice light rack the second methodology November 7 2024 Beijing University really interesting it incorporates not graph structures into text indexing and retrieval process and this text indexing became really important for this let's have a look at this we have here our text or human readable text into your text database or whatever and here has an example by them they have beekeepers and they explain a beekeeper is an individual who produces honey and then we have other links to this bee keeper so the graph-based text indexing now here is relatively simple because we have three steps D duplication llm profiling a beekeeper be keep is a person who yeah great and then the entity and the relation extraction from our text here and this is simple we have the notes b key button B and observe we have a simple link entity we build now an index graph that has all this information and we will use this index graph could be a rather complex IND gra for the retrieval process itself for R now we have a lot of additional information in our node and edges we have done high level keys lowlevel keys and we have a dual level retrieval Paradigm with entities relationship and context this is all beautiful but you know the most important factor here is here those index graph for the retrieval here we have all the important information that we need for a graph indexing extract various entities the name the date the location all the events and record all the relationships between them this information collected through the process will be used here to create here a comprehensive Knowledge Graph that highlights now all the connection and insights across the entire collection of all the documents so we built this up beautifully and this is a very active Community if we go here to their GitHub light R you see just 6 hours ago or five 5 hours ago so there's really some Dynamic momentum here MIT license more than 12,000 stores here great have a look at this what I particularly like is here the graph visualization code it is so easy you use Network X here and you can generate your graph ml file here your network you can have even some more advanced threedimensional visualizations interactive so you immediately see here all the connectivity and the important structures in the index graph now you remember we have seen a similar structure and this is here my video about nine deep seek agents that ented build up here Knowledge Graph and yes of course this is also a Knowledge Graph so we also have a summarization agent an entity extraction agent a relationship extraction agent even multiple if we have multiple specialized domains we have a scheme alignment agent because we have now new knowledge that should fit into an already built Knowledge Graph we have a conflict resolution agent or multiple we have evaluator agents so you are familiar with this of course it's a Knowledge Graph but this is also now a Knowledge Graph but this index in graph is now something special and we will build on this on the development of graph rack so it is really important that you are 100% sure you understand the indexing graph graph structure indexes here the text database or whatever you have your private corporate documentation think about it if you compare it with the classical index here a traditional R system now you have textbook you have Pages you have paragraphs and you index each page or index each paragraph by keywords and then if you search for the wordy photosynthesis or whatever you get all the pages that mention this particular technical term we go now now the next step of an indexing graph for the r and we built here kind of a concept map of the textbook so our Nothern edges are clear but new is now the textual junks that we take care of those are now our valuable additional elements that we're going to use notes are clear no notes like photosynthesis chlorophyll sunlight edges give us the relationship photosynthesis requires sunlight but the textual chunks those are Concepts and relationship would be linked back to the relevant sentences on paragraph in the textbook that explain them so let's come here to The Spar of genius and to take the next step it was important that the Creator who built your light rack the same scientist found out here hey we can arue now analyzing those system that the information considered in the previous graph based rack matter here this two are often redundant and this redundancy of the data of the information of the knowledge in our graphs can introduce noise into the system it will degrade the model performance and it will increase the token consumption for nothing so now the researcher that built light rack for example said you know what moreover we found out that both those methods adopt here a flat structure to organize the retrieved information in the prompts so we just directly concatenating here the textual information of all the rtiv not and edges and this leads here to a suboptimal logicality and coherence and we can do better so the core new idea was now to overcome all those limitation of their last Model so this new if you want Evolution step here have the graph based track will now reduce noise reduce the token consumption and improve the quality of the answer given if you want to see the detail the inner working how this works the next 5 minutes are for you if you say I don't want to understand how it works skip ahead to the code section the PA rack methodology starts simply by identifying now the keywords in the user query in my human query and the system then uses here my keywords to search now within the indexing graph so yes we have to build build an index and gra from our text database or whatever we have and the retrieve notes that are considered relevant based on these keywords and this preliminary step nowas no down the search base because we Define our starting points and then we have a flow based pruning algorithm with the specific distance awareness so if we flow now through our knowledge graph we start from the retrieve notes we filed out the PA that are less important that are less relevant and we are distance aware we have the complete knowledge the mathematical knowledge of graph Theory so whatever specific parameter of graph Theory you apply you can prioritize shorter path or paths that are more directly connected and this means that hopefully they are more semantically correlated and less noisy plus we can assign here a numerical value and a reliability score to each paff and this will allow us to rank those PA RS and then p r will now fetch now this additional very valuable textual junks that are associated with each note and Edge along the path and we have here the importance again of this index graph underlined we have concatenation of the textual chunks in the order to appear in the graph not the edge not the edge and this concatenated text becomes now a textual relational path and this is a human readable text throughout representation of the path and its Associated information that is now so valuable and here now for the llm this becomes now an easy task so you see half rack is a methodology which effectively retrieves your key relational paths from an indexing graph we built from a text database with a flow-based pruning methodology and this effectively generates your answers with a p-based llm prompting optimization isn't this beautiful okay so let's come to the hard facts now we have here February 18 2025 PFF rag here this R generation with relational paths and the aors here again the Beijing University University of Hong Kong and nor Eastern University so remember those are also the ERS from our other graph rack systems so this is here the Natural Evolution and now this relational pathways are so important let's have a look at this in detail let's give you an example we have plants plants are infested by aits aits are fed on by the ladyx so this PFF represents now the relational chain pffs Infested by apids and apids are fed on by ladybug and this PA shows here relationship between the plants and the ladybug mediated through the apids in context of the P control so you see we already have a semantic environment defined and in this paag methodology both know than edes have as I told you this beautiful Associated textual chunks so this relational path is not just now a sequence of note and edges in an abstract graph but it implicitly now includes the textual description of all those entities and all those relationship along the way and when the parre then converts those to a textual relational of it's making this textual information explicit for the llm to reason on this is here the original visualization by the authors from Beijing University I found it not so helpful but I love this visualization here because it gives us a complete overview of the methodology so we start with a human query and my query is What organic methods have been effective in reducing here past populations and from this query the system now extracts here what it SS are the most important keywords organic matters for past populations then we have a database now this is our external knowledge new knowledge updated knowledge more complex knowledge whatever and now we Index this database here and we generate now an indexing graph kind of a Knowledge Graph but a very specific Knowledge Graph for this particular database so you see there's a dynamic factor and an update Factor but more about this later now what happens we have here our keywords and we have the indexing graph and now with a note retrieval augmentation we build now our first graph so we have now and you see here coded in Orange and this coded in let's see green we have now additional semantic information like ladybugs feed aits aits feed parasitic wasp and we built our graph and now what we do it is simple look we have now our note pairs like ladybugs and aits beautiful and we just have now a reliability score and distance score we have now graph-based indicator we can calculate from the classical mathematics of graph Theory now for each Edge in a particular configuration of our knowledge node and edges we have now scores and this scores can now be ranked so this means our path are now ranked based on the scores so we have now the top path from ladybug to aid from ladybug parasitic W to aid from yes you get the idea and then we have here given here the specific parameter of the reliability score we have now a ranking of the top puff we invert this here for the puff based prompting for answer generation because you know the llm is most sensitive at the end of its attention window and this is it this is the comp complete methodology and you might say great and I would also say great if I would not have one main question remaining and this is it how the hell do we arrive at this graph representation and I said please please don't tell me that we again have to build like a vector space like in Vector rag like in the classical rag that we did overcome years ago now I have to tell you yes the or of the study told us here based on the extracted query keywords here or two here a dense Vector matching is employed to retrieve your related notes in the indexing graph in this dense Vector matching the relevance between a keyword and a note is calculated by the similarity in the semantic embedding space so it is again a vector space and we use here the cosine similarity so we are back here to go from the keyword in the indexing graph and find semantically Clos related tasks or clusters or elements the whole system is beautiful except for this that we have to go back and build a vector space from a classical rack system my goodness but you know what there are alternatives but more about them later let's go with this study by Beijing University now Beijing University said hey we want to compare now for a lot of data if you have one question what are the deriv features yes yes yes and now we have we built a light rack so let's have the response by the light rack by this graph rack system and then we have the response by the PA rag system and you know what we use an llm as a judge so go with whatever you like and they have now here in multiple let's see dimension comprehensiveness diversity logicality coherence and relevant the answer by the llm the llm Compares those two textual responses and tells us you know answer one lists only eight features answer one only primarily focus on rather common methods without exploring Advanced Techniques answer one has less cohesive reasoning structure answer one is less fluid in its transition so you get it P R is outperforming light track so the system works isn't this beautiful now they R all the evolation test and appalation studies and whatever and they have now the final result pathre this latest evolution in the graph rack system consistently outperforms here the baselines across all evaluation dimension on all data sets and here you have the data for you to check yourself those are the F the five dimension of the evolation and then we have here the data sets agriculture legal history CS biology and a mix so if you run all the test beautiful PA rag is great let's look at the code we have here beautiful GitHub if you go to PA rag pi and whatever else you need I just had a look here at this particular uh python file and yeah unfortunately here you see it now in detail yes we do need here a vector database storage my goodness at least we have also open-source methodologies you don't have to pay for a commercial solution you can build your own if you want open source solution but I would say such a beautiful implementation but again coming close to what are semantically glose by terms here we have to fall back to the old VOR database now of course as a subscriber of my channel you know that there's a beautiful alternative I already did a video here on Rex intelligent upgrade the atantic Roar from Oxford University yeah but this is a video in itself now at the end of this video there is another point that is important for me and this is about a new study rag in medicine and I thought hey should I post this here on my post on my YouTube channel or should I integrate it in a video February 18 2025 we have MIT Stanford University and Duke University and they publish here kind of a warning about retriev augmented generation of Rec system in medical EI they tell us retriev augmented systems can be dangerous medical communicators I recommend that you read this study if you're interested in anything that is related to Medicine how humans use AI for their own medical purposes and yeah unfortunately I understand that some people who might not have access to a human doctor they might rely here on medical EI systems but there is a strong word of warning from MIT Stanford and duuk about Rex system they tell us the current retrieval augmented Services often use cited sources to generate narrowly accurate but pragmatically highly misleading responses now clear I'm not an MD I'm not a medical doctor I'm a theoretical physicist so I don't know here the test that they show us here in this study if this is really the medical complexity good or bad or better or worse I have to trust here the Judgment in micine M Standford and duuk but let's say we do this so we come to the conclusion that our rack based system that we apply currently reinforces here the patient presuppositions decontextualize here important facts relative to the source material rack based system generating misleading sentences for our human patients produce result without an intuitive and pragmatic understanding of Downstream consequences and I so this is a real strong warning issued here by those universities and I just want to make you aware if you use a rag system in medicine there is still a lot of research to be done now I looked at the data to get a feeling but unfortunately this is a domain knowledge I don't have why is liver transplant dangerous or kidney transplant and they went to perplexity to get all the ansers but I cannot evaluate this answers sorry in this particular video this medical knowledge I cannot provide here a personal reflection on the quality of those data set I simply have to trust those universities and they tell us here retri augmented system can be dangerous medical communicators and current rack based responses to health information queries have the potential to mislead and reinforce biases due to numerous intersecting pragmatic communication policies failures to interpret the queries misleading representation of sources misleading language in generating responses and this seems to be a work that we have to improve significantly you now I think if we only find this in medicine because medicine is so immediately important for humans now and we find out that rag system do not work there what about the performance of rack system in finance what about the performance of R system in education so I have not such a positive feeling reading this particular paper here about Rec system and science yeah for example here if they go here with specific medical terms and they say hey why is the procedure dangerous or why is the procedure safe I can't tell you if if to evaluate those answers and really tell you that the deductions the results that they presented is valid but if you are an MD please have a look at a study and I would highly be interested in a professional response here given the details of this study but I think it's great that MIT Stanford and duuk have a look at these topics and they tell us hey those rack augmented Generations are not ready in medicine for let's say the normal person like me and yes I have to tell you me too I thought about when I had some medical condition to go and ask your AI system but then I decided no I know how this system works so I do prefer to go to a doctor I had to wait but well yeah this will be a topic for some other videos but currently we have this strong warning here they can be dangerous medical communicators and there you have it this is the video of today we had a look at rag at graph rag the evolution into light Rag and then the latest technology for you with code implementation off rack I think it's absolutely fascinating how rack systems now develop and maybe we can increase here the performance of re system in one of my next videos because yes there is even a new development in the pure mathematical knowledge growth area if you want to subscribe you will be notified immediately