9 DeepSeek AGENTS Augment Knowledge Graph (KARMA)

Key Takeaways

hello community so great that you are back today we look at the interplay between knowledge gra enrichment by agents so I have a practical problem I do have a knowledge graph of all my EI literature and daily I get about 400 to 800 new eii research paper so this is now a problem because you know what I want to do I don't want to build your daily new knowledge C but I simply want to update to augment to refine the knowledge the new knowledge and bring it over into my knowledge C well of course I'm not working with 800 but this is here the main problem that I encounter and I would like to show you here the latest solution the latest AI research for this so let's open up here this video and let's see if I want to bring it over to refine it you know we have options and I know a lot of my colleagues they take note they have hundred and thousand of little notes or Little Pages or whatever they build databases or if you want to have here notebook llm is beautiful you can even generate podcast and whatever but you know for me I'm interested to build knowledge growth because knowledge growth are built from the reasoning traces here in a dynamic memory of our llm and what I love to do is have the knowledge graph have all the specific subgraphs and then an llm can plan here on the graph the best actions for reasoning topics for robotics or also for complex reasoning or in the simplest case I just take here a model like R1 and say hey r one here that's my knowledge graph with my particular preferences met my particular way how I reason how I like to reason what is important to me use this simply Knowledge Graph here to solve the specific query so I can break it down to about 15 or 20 AI research paper that are really outstanding every day and then I want to update here to knowledge gra and you might say hey we don't need any spark because this is simply straightforward yeah we just take here the best performant reasoning model in mathematics and we have four number one 01 o03 deep seek R1 and Gemini flash sinking great and then say hey just read the new PDF files including the graphs and the tables then the AI should understand your the main new thematic topics and the new methodology then the eyes should generate a short technical summary of the main implementation of each of every PDF then thei should identify the existing Knowledge Graph nodes that are similar now to the new technical summary then extract the the reasoning traces from the established Knowledge Graph and compare this to the reasoning to the sematic reasoning after new T summary identify all all weaps and all contradictory tricks riplets of the new T summary calculate the probabilities for the correct reasoning traces including new research because the new research might contradict established knowledge that is already mapped to the knowledge graph then the AI should simply delete specific reasoning traces in the knowledge graph and note by the way all dependencies because now we insert the new knowledge triplets into this augmented knowledge gra and then we have to validate the coherence and the old the dependencies are met so we have to validate the the Matic coherence of the new subgraphs that we just established and inserted into the knowledge graph and then there are some other technical points so you see no problem at all and you know what if I do this to my absolute amazement the performance is not there the complexity of this and this are already the first 10 points that the has to perform here to update my existing knowledge grph this complex is a rather little bit more on the complex side so what we do we build an agent of course it's February 2025 hello agents so we have an EI agent so which means we have an llm not just rule based or something and we have memory and function calling everything that we need and then we build specialized agent no multiple agents of course no problem multi-agent system or easy to learn but now for my particular task to update your my knowledge graph with the new scientific publication what is not a perfect number of AI agents because remember it is not just the complexity of my task in science but this also handles Now new unseen research data new unseen research terminology new relations in those unseen data and this is now if even increases the complexity further now you might say hey no problem in one of your last videos you showed us apology DS Pina where we have multi-agent and this is a simple mathematic optimization problem we have three steps 1 2 three you showed us here in the video my goodness this video was 30 minutes long and you said hey it's a simple mathematical problem and the system will figure out here the perfect topology of all the agents either they're in a linear chain or they in an interconnected chain and then we evaluate it and we have the perfect topology for the multi-agent system and you are absolutely right so let's come now to the core new idea we will build agents and we will build agents now not for the complete complexity but for subtask that have a lower complexity so that it is much easier and the performance is higher so at first we have to retrieve all the raw documents normally PDF or HTML then we split the documents in sections core the segments that are relevant using it an Knowledge Graph context and simply filter out all the nonrelevant context we condense the text segment into summaries while we preserve it a complete entity relationship manifolds and you know entity extraction agents they identify the entities via F shot LM prompts normalize them to knowledge grph canonical forms using our onology guided embedding alignment but of course we need also the relationship no what does it cause what is the base so we have a relationship extraction agent between the entity pairs using now multi-label classification because it is so much more funny if we have multi-level classification allowing now for overlapping relations well we want to align our schema of course to the knowledge graph schema so we have a schema alignment agent map novel entities or relation to the knowledge graph schema or flag them for a complete ontology expansion which is really interesting but the next two agents are really important we will have conflicts because I will put in new research data that might create a conflict might have complete new ideas go against established knowledge so we need a conflict resolution agent yeah it resolves here the contradictions here via llm that llm now start to communicate start to have a discussion a debate here about it and an Evidence aggregation and our last agent is an evaluator because everything what we have done until now should be evaluated before it becomes a ghost that propagates you through our pipelines so we have an evaluate agent compute your integration confidence using your weighted signals like a confidence the relevance a Clarity a coherence and whatever we'd like to have and here we have it now finally what is the perfect number of Agents nine and you say hey we show out eight yeah but I need of course for the simplest multi-agent case a central controller so there we have it we have nine agents with an alarm at its core and they are now given dedicated tasks so how do we build those specialized agent how do I build them now let's come here to the beautiful facts of a new research and they are done here this visual is done here by biomedical e College of futer Technology ping University and they have your new methodology for automated knowledge grth enrichment they Define all those eight beautiful agents they go here in the simplest case with a central controller but yeah I know in my last video I was talking about self learning I systems but unfortunately this one here this one here no this is a hierarchical system because it is the simplest system so let's start with the simplest system and then yeah if you want we can have a peak preview to self how we integrate now self learning into those systems yeah we need data no we have to have a data set this is easy to or say hey we go with genomics protonics and metabolomics we have everything in a PDF format and this is about 1,200 scientific papers and if you start fresh if you do not have a Knowledge Graph already established from this just to give you a feeling for this 1,200 scientific papers they could Define 38,000 separate entities technical terms relation whatever so let's let's start now the first question is of course hey what Central Intelligence will we use now the arst did a lot of experiment here with a single age and a real small llm a gbd4 Omni or the Deep seek and it turns out deep seek is just beautiful for this task so we have now specialized agents nine and in the heart of every little agent we have a deep seek version 3 llm that has all their intelligence to make decision to have your prediction what's going to happen now simply deciding on the next actions and I know what you ask you say but how do we build those specialized agent now that we have how we construct them but W how we code them well here's the prompt for the first age and you see it's so simple you have title a rule description the system instruction what is going to happen how you do error handling then you have the llm prompt template in an example then you have a sample input and a sample output given but notice this is here for the PBM this is really domain specific you can either do this once here manually or you use either DSP or tax red or whatever you like then we go to the next agent here this is simply the VA agent you see you have an input an output you have some scoring euristic you have here the template the sample input sample output so we go on here we have the summarizer agent this is here the complete prompt that they used in this example here the ERS for the entity extraction agent domain specific then we have the relationship extraction agent you know the edges that we need then we have the schema Alignment Plus the conflict resolution agent here you can improve as you like for your particular theod problems plus we have now the first prompt where we have an evaluation and we go here with the evaluator agent prompt for the confidence parameter so whatever is important in your domain be aware and then you simply give here an example that you have here you enabled the llm that it could give you a value between zero or one or very good or very bad you need here somehow the scores we do the same for the evaluator now on Clarity and on the relevance given that we already have a knowledge gr and only if those parameters are above a certain threshold we will accept it now you know at this point I ask myself hey normally I work with two or three agents and now I said hey do we really need all those nine EI agents you know it's not that uh non-expensive n well or one is open source so this is a beautiful thing very powerful in its reasoning capabilities but the authors also seem to have the same idea because they did some test they said okay if we go with here with the full version but what if we leave out just one of the agent the summarizer what if we leave out just a conflict resolution agent what if we leave out the evaluator agent what happens to the performance in the dyamics in the promic and metabolomics data and here you see for two parameter this here just is the llm based correctness parameter and this is question and answer coherence parameter and they calculated this and you see each every agent is valuable now the Chumps are you can see significant or not significant if you go from 0.8 to 0.7 or 0 .79 but you know it has an effect and they did this for all the other agent and they really found hey this is something to get the best performance so if you're really budget limited you have here indicator what agent you might be not so that you have to integrate it but otherwise you see sometimes you go from 0.77 to 0.63 or 0 .66 so I would recommend it really in the beginning you go with old agent and then you can try to switch off one agent and see about the performance of your system if you really have to reduce your on budget wasn't this simple wasn't this beautiful I never thought that I would do my next update of my knowledge graph with nine EI agents I saw two or three are enough well I was wrong and I Lear learned and I hope that you learned from my mistakes so what we did let's have a short summary here so I reduced the complexity of the overall task that I showed you with all the bullet points here by creating here less complex subtasks and for each subtask we employed now an eii agent we Define the specialized agent we defined a prompt you can go with prompt engineering thep whatever you like and get the optimal prompt that are for this specialized domain specific agent you have domain adaptive prompts and then you have those agents and there will be conflict so you have to provide a solution so that the agent can debate and have a conflict resolution schema it can be a very simple one you can go crazy on this but those are important topics to be above a certain threshold so you have multi-agent verification and cross agent validation just notice that if you work with science and you know that whenever you open a new AI research paper you have new technical terms you have at least here the methodology is give him a particular name like Serius or whatever so you do have an extended vocabulary sometimes and you get new relationship types because we discover new dependencies now those agent can be periodically retrained which is really recommended here after let's say a month or maybe some weeks or the prompt updates to handle new emerging entities you have new drug classes or whatever you have plus you will get more complex multimodal relationship so we have to build this system that it is able to grow to increase its complexity but the system structure must be able to provide here a way to receive high performant answers so so model of structure of the prompt eases here integration of these updates because now we can simply go and pick one or two agent update just one of the nine agent or maybe we have to update three agent that are related to each other but you can you see I don't have to do get a complete system update where I do not know if I have nine agents which of those agent goes first in the update what dependen is and so on so here we are now and you might say here at the end of this video do we have an outlook here yes of course because we found here a new hierarchical multii agent system and you might say hey this is great but you know the boss of this system here the Central Intelligence the Central Command we have to train those agent on training data so we have to provide this agent a lot of training data where I show that hey for this problem look this was the best solution step one 575 so we have to create a lot of synthatic training data for this controlling agent that controls now eight other sub agents the quality of the synthetic data especially if you have eight subagent is not yet established here to a degree where I would say hey I'm satisfied with the quality so of course what would be great we just give it here I don't know 10 20 ideas and then we say hey system are you able to develop a self-learning algorithm can you bootstrap here yourself here from the very few examples given and you develop a self- learning methodology in this AI system that consists of nine now then eight agents think about you have eight deep seek or one communicating and learning at the same time just to increase here the quality of your knowledge graph I'm absolutely fascinated by this idea and yes I have to go and try it out and if you are maybe I don't know could be theoretically that you are interested in this kind of videos hey why not subscribe