NEW Knowledge Graph based RAG: SimGRAG (no training)

Key Takeaways

hello Community today we look at brand new Knowledge Graph Daven rack we have four methods and a brand new one called similar graph rack with some really impressive new performance data so let's start and you know the problem that we face is is how to align quy text and knowledge gr structures for our optimized rag retrieval and you know caping this was in 202 three knowledge augmented language model prompting for serot knowledge graph question and answering then a little bit later we had your knowledge graph GPT the framework for reasoning on Knowledge Graph using here llms and our GPT mods where we had three steps the sentence segmentation the graph retrieval and the inference step if you want to see this here in a visualization and then we had KP the knowledge grph enhanced llm models via PA selection were we trained here the path text encoding over two specific rules and here you have the visualization now the last one that we talked about was about G retriever so this is a retrieval augmented generation for textual graph understanding and questional answering our typical topic here and this was here by meter and National University of Singapore it integrated here the string of a graph new network of some large language M for our rag retriever system but now we have here those four classical methods but there's something new in town yes and we're going to have a look at it because it will outperform those classical methodologies so here you have them if you want here caping G retriever knowledge C GPT and CP and now you might think hey wait a minute you showed us already this video no that Howard presented here a month ago a new Knowledge Graph agent for medical AI system where we have this new interplay on an agent level no this is too complex here today we're on this more simple pathway so you say okay so it was this video now where we add an llm to a knowledge GRA with the new GI methodology by UC Berkeley or you say wait a minute maybe it was agenda graph where we added multiple agents to a knowledge graph or you say hey wait maybe it was that we Empower here in llm to decode arm GRS with the new methodology VI MIT and then you say okay I give up maybe it was somewhere here in the knowledge 12 graph embedding playlist on your YouTube channel where you have 29 YouTube videos to learn here exactly how to do a graph embedding how to go beyond here the topological methods passing of a graph newal Network how you use your the code with djl how you go with a caras implementation and so on and even go beyond here the GNN here with some new homological ideas no it is much simpler today it is simple and beautiful look we have now a fifth methodology and the idea is beautiful we have a quy to pattern alignment procedure by the llm and I will show you the prompt of the llm and it's really simple and then we just have a pattern to a subgraph alignment where the subgraph is as it says a a subgraph of our main Knowledge Graph and then we just add you a new metric a new distance a graph semantic distance that we going to Define together then we know exactly then we have here a score that we know exactly here about the pattern to subgraph alignment so our query to pattern alignment and then our pattern to subgraph alignment two simple steps for this new methodology and here we have the new publication second half of December 20 24 Zim G Rack or I call it similar graph rack systems and this solves in a beautiful way how to align the text queries and the knowledge graph structures so let's have a deep dive what do we want we want to be fast know we want to have tens of millions of note and edges in our knowledge graph and this should not take more than let's say one second latency plus we want also to have kind of AE selection because we want to focus only on the most relevant and essential node and edges for our subgraphs and and this is a big one we don't want additional training or additional fine-tuning on some complex configuration so therefore we want just a simple plug and play without additional training or fine-tuning methodologies short fast and cheap and this brings us now this beautiful but also critical challenge of aligning now our user text query with the underlying structures because we do have a knowledge graph and a Knowledge Graph has an inherent structure as a graph and now we have to bring those together we have to align those in the most optimal way and the solution by the new team here is a two-step Approach at first we will use utilize the an llm to generate a pattern graph as I showed you that aligns now with the query text and then to retrieve the best multiple subgraphs from our specific Knowledge Graph that schematically aligns with the pattern graph the ORS introduced here a new evaluation metric and they call this matric the graph semantic distance matric which is a very nice defined new evaluation matric so here we have it you know it couldn't be simpler no we have a quy and we generate from from our query now a particular pattern alignment not we the llm does it here in green so we utilize here the power of an llm and you see here exactly in green this is now our pattern graph and here in this lower manifold we have the main graph the knowledge graph and there we have now a particular subgraph that we identify and you see this here in blue and you see here Virginia with Virginia and L&B mle and lore bore mall and then you see a georan architecture and Georgian and here College of William and Mary and D College of William Mar and you see we have here calculated semantic new distances so yes beautiful those are the values and you see this is here a beautiful visualization how easy this new methodology is and how simple it is before you have a deep dive because if you do a deep dive like I just did I have to tell you there are some beautiful ideas deep dive in the formalism in the mathematical formula but more about this in a minute yeah so pattern green subg graph in blue couldn't be easier now simple question how to build this pattern graph given that we have on a particular domain knowledge manifold we have now a particular query and we want to generate this pattern graph and then how do we find here the best matri the best evaluation matric to have here the perfect fit let's start so the improvements that the ERS tell us that they have against the other four classical methodologies are simple they say fundamentally differing here from the caping and the g retriever that do not explicitly constrain subra structure or size I show you how we do this then they say here they are also diverging here from CP that trains your P selection model that is limited to one hop or two hop off we will have an evaluation on sop and then finally to retrieve here the top K similar subgraph with respect to the pattern graph with the smallest metric deviation we further develop an optimized algorithm and they put a lot of work into this optimization of the algorithm and it is really impressive I will show you the code and the scallet how we do this optimization on this algorithm now here from their original publisher you see here if you want another structural diagram so at first the step one we have the query to pattern alignment step two the pattern the pattern to subg grth alignment beautiful and here you see comes now into play here our semantic guided subgraph retrieval functionality our retrieval from Rag and here they go now with two Vector structures Vector mathematical spaces where they have an entity Vector space and a relationship dependent Vector space and you are familiar with this because we have done this all the time and then I'm going to show you this they have a verbalized subg augmented generation I give you the code for this and you see here the top K subgraph the top one subgraph the top two subgraph and finally the correct answer given here for our knowledge graph induced rack system let's have a look look at this I leave this here on the left side so you can reference it immediately where we are in the process so the pattern to subgraph alignment methodology uses entity and relation Vector spaces here and I thought hey wait a mean do you mean need the classical Vector embeddings that we have here from our cosine similarity like generated with our beautiful sentence Transformer our espert models that were based on the bird model so we have Vector spaces Knowledge Graph and then Knowledge Graph based rack retrievers yes and you remember just days ago I showed you there's now a new bird system we have now the modern bird this is a new high performance Transformer architecture only or specifically designed for an optimization of the rack so therefore we can implement this immediately it is not in the original literature because this just happened days ago so they could could not implement the modern bird they go with the classical bird architecture hey but we can implement the modern bird so talking about code here you have your on GitHub the official U Reaper from similar graph rack and you have everything that you want if you have here look at the retriever you see yes we really go here with the embedding model and if you go deeper you see then really we have if we look here at the Bing model itself from the center Transformer Library we have here the S bird the sentence Transformer and we have here the classical SD implementation which is beautiful because you might remember that this is exactly what we already had a look at if we did the knowledge craft rthm rag systems now just to be sure the ORS work here with a particular data set this is the meta question also data set meter has nothing to do with meta company this is another abbreviation and what I wanted to show you before you jump to the code I want to show you the query to pattern alignment prompt that you ought to use because this is quite clever however if you're not familiar here with the Met questional answer data set this is here the literature for you to study now here you have it this is here the quy to pattern alignment ProMed very easily I did it here on jb4 and this is now particular for the structure of my meta Q&A data set so if you have your domain specific knowledge somewhere else in another configuration please align this accordingly and as you see I have here example one query divided actor and so on so we use here in context learning so a few input output example to facilitate the in context learning with two or three few short examples and then this is it this is all you need in you are coherent from the structure so great let's have here a short recap a short summary what we have we have it first Dear to pattern alignment now let's do it a little bit more precise from the original paper body ORS give a quit Tex Q we prompt you the llm to generate you a pattern graph P consisting of a set of triplets that I just showed you that align with the query semantics then we come to the pattern to subgraph alignment so given the generated pattern graph P our objective is to assess the overall similarity between p and the subgraph s in or from the knowledge graph G since the pattern P defines the expected structure of a subgraph we leverage here a graph isomorphism see my videos to enforce the structural constraints on the desired sub graphs y little bit of a notation and then we have here the definition by the ERS and they tell us here the graph semantic distance is exactly defined with this mathematical formula you also find it in the GitHub code implementation what they also did and this is beautiful a generalization to unknown entities or relation so either you have an unknown object or you have an un unknown relationship and you do not want the system that it crashes on this so they have here further specification that they generalize not the graph distance and they say we exclude particular items from the GSD computation so given the isomorphic mapping yes you no this and then this is this is where I have a feeling that a lot of effort went into the optimization of the top K retrieval algorithm by this research group because there are some really detailed analysis how to get back the top 20 or the top 100 uh sub graphs from the knowledge graph that really align with our text query so this retrieval algorithm at first I thought it is simple but wait a second let's have a deeper look so recent subgraph isomorphism algorithms often follow here filter ordering a numeration paradigm and to narrow down the potential search space here the ERS here first apply the sematic embedding to filter out unlikely candidate notes and relation but then they come and further Define this and here you have then the published top K retrieval algorithm and you see this pido code gives you an idea that it is not that trivial but there is some real inherent hidden Beauty in this so therefore I would recommend please read the original paper it would take another 10 to 20 minutes to explain this further but let's say we have it we have the code we understand here the primary functional groups so let's have a look at the The Benchmark and here we have it from the oras here you have here for our particular data set MAA Q&A or fact uh Knowledge Graph accuracy here you have the data and let's look only at the knowledge graph driven rack system without training and the llm is here a llama 370b system as specified by the order and here you have caping and here Knowledge Graph GPT system and then you have here the new sim graph rack system and I think the performance jump is impressive just look at these numbers especially if you have from a multihop this re hop evaluation I think close to 98% this is something where I have to tell you hey this looks interesting especially if you have a more complex domain knowledge like theoretical physics or organic chemistry or whatever you're working for so you see the performance jump is there and the are quite proudly present here this results so just to recap the two main points here the query to pattern alignment and the pattern to subgraph alignment where to introduce you this new graph semantic distance theatric to quany the alignment between the design pattern and the underlying subgraph and the knowledge graph ensuring that they have a context Consciousness and a preventing your entity leaks but of course remember you are working here on a specialized domain knowledge and the beauty is we can build our Vector spaces ourself with bir you know if you buy a vector store or a VOR space whatever and this was trained on I don't know 10,000 different domains but if I work only ontical physics or chemistry I just need two or three domains so therefore I can build my small but smart vacor bases highly specialized only for my single task for my single prompt for my single query and I have an absolute expert system and I don't have to pay anything to any commercial Vector stores and there you have it I think this is a very short introduction to this new this fifth methodology that we talked about here on this channel the knowledge graph driven rack the retrieval augmented generation here with external data how we combine this here with a knowledge graph we seen we have an implementation here with a large language model with we build new Vector spaces we build new Vector embeddings that are domain specific based on our training data set and the complete system is called either a Sim graph rag or a zim G Rag and I think it is a highly interesting new methodology if you want to optimize your R system and you are faced by the company that provides you with a Knowledge Graph so this is the latest logic implementation I hope this video you enjoyed it you found some new information some new code that you want to try out and it would be great if you subscribe and I see you in my next video