Knowledge Graphs w/ AI Agents form CRYSTAL (MIT)

Key Takeaways

hello Community today we talk about the latest in Knowledge Graph and and you would say why are you talking about a knowledge Crystal well I need here some specific feature and this is a self-organizing principle so what I want to talk today is about building an AI system that just does not process knowledge but it can grow and organize knowledge in a particular way that suddenly it becomes a structured knowledge and we will look here at the emergent of an intelligent pattern so let's have a look now you notice in nature we we see this every time in a cave we do have some crystalline structures growing here some water is dripping from the ceiling and everything is great and you know what we have we have a complex crystalline structure not by carefully designing every atom's position but it just happens in nature because we have simple rules and we have something like a self assembling mechanism where some specific atoms just form here in a perfect let's say crystalline lettuce and this is what we going to use and explore now this same idea in knowledge so what do we have we do have a knowledge CFT typical knowledge craft that you like that you know and then we say okay we have an EI agent and this EI agent is reasoning on a particular topic so given the inherent knowledge and the inherent training of this EI agent it produces now a reason in Trace and we transfer this reasoning Trace now in a little knowledge graph a little tiny Knowledge Graph only on the complex reasoning of our EI agent and maybe there's another topic for this EI agent so we have another subn Knowledge Graph and then we are faced with the exercise we want to integrate those knowledge graphs is sub graphs here in the main Knowledge Graph so you see we have here a crystalline knowledge structure and we have here now an eii agent and EI agent is trying here to self build to self assemble here more knowledge let the knowledge Crystal grow here but we do not give it here some specific centralized commands we want to see this happening here this self assembly this self know knowledge buildup by its own is this possible or are we completely on the wrong track so we want to grow our knowledge now like a crystal but smarter so what we need let's build the eye system to grow the knowledge Network organically so a kind of iterative crystallization here after information this is our game that we want to play today so instead of predefining now the complete Knowledge Graph structure we say hey we just give it you nucleus and then we just give it you some very basic instruction so that the knowledge can self organize in this beautiful crystalline shape of a Knowledge Graph and surprisingly it will end up looking remarkably like how human knowledge is structured well of course because we will use Ani agent this was that was trained on human knowledge now you might say but we know this now for example this video where I showed you here multiple agent and here a gr rack system yes we can build something when I give you commands where I tell here the graph rack system here and the agent particular task but what if I want to build a system that self assembles over time that continuously builds up new knowledge and integrates this knowledge here with the reasoning traces of the knowledge generation into the knowledge graph and you see this is what we're looking for we will have here a beautiful Knowledge Graph and you see this in this video here with a definite agent that we have here and we looked at Harvard and medical Ai and then we have the problem here of a blood language model the T has a ro a Transformer like structure no an ual Network structure but this crystalline structure of the knowledge graph there is a difference and this difference this is the real fascinating topic because one of those network has a characteristic the other might not have so let's have a look at this so now we equip here simple AI agent with two mathematical Frameworks work that we need because we want to go the next step no we want to go to self learning AI agent that build knowledge crystals so what do we do yes you guess that we go here for a mathematical category Theory and a complexity Theory approach because those are the two things that we need if you're interested why this is my explanation categor Ser now formly structured a knowledge representation we have there Concepts like objects and relationships that are like morphism and this is nothing else than a mapping so this gives us here a rigorous way to represent knowledge in a more structured in a relational format notice this is the relational format it's like having a mathematical grammar for our knowledge and then we have the complexity Theory and this deals with system made of many interacting parts that self organize and exhibit here some emergent Behavior think about your team and your in your office or if you are working within a team here there are something of a self organization that a team member find itself for particular task they reconfigure themselves in their team configuration to find here the optimal and stable configuration of team members this is explained by the mathematics of complexity Theory so we will need this category Theory and this complexity Theory because where order arises from decentralized interaction and feedback this is where we we have our AI agent here as The crucial element and now the combination of both of them is really what fascinates me so we do have category Theory provides you the blueprint for the structured knowledge and the complexity Theory explains how this structure can emerge dynamically and autonomously so interested let's have a look if we say the emergent of sophisticated structured knowledge from a relative simple iterative process this is where we have to look at because our AI agent is a simple agent no yes it can do some reasoning no problem but it is just performing a simple iterative process I get a query is EI reasons about it transform the reasoning pattern into a subn Knowledge Graph and then we have it so what is our limitation limitation In This Moment is of course the complexity of the knowledge graph but also the reasoning complexity of our EI agent but let's say our AI agent is not that intelligent so we have a relative simple iterative process of just adding knowledge in the reasoning form of this simple AI agent and the fascinating thing is in the knowledge graph we will see an emergent of a more sophisticated more structured knowledge let's have a look at this let's open up this video now this was just the intro to get you interested in so the central ideas and we have a beautiful research paper from MIT that a complex intelligent like knowledge structures they can emerg spontaneously from relative simple iterative processes we know this in theoretical physics we know this here in in geal processes but now we do it with knowledge so what we need let's stick with this we only need an itive Loop so an AI agent reasons adds then this reason to a Knowledge Graph and then reasons again based now on this updated graph so suddenly we don't have only the internal reasoning of the AI agent but it also takes now the knowledge that it just added and encoded now in the complete main Knowledge Graph and uses now the knowledge of the main Knowledge Graph for this next task so you see we have no Central multi-agent boss here in this configuration there's no Central planning eii Vision language model or large language model this is here dictated by the overall structure and this is why we need here specific entropy functions but more about this later so this arises here from the repeated local reasoning interactions from our little e agent that just adds here some local reasoning knowledge okay now we are heading of course towards some autonomous knowledge Discovery and all the things that you see here for deep research from open ey and Google and whatever yes we are heading in this direction but we're doing it in a different way you know we want to have some intelligence behavior from some iteration so this particular system now shows that an intelligent like Behavior like a knowledge organization or just generating some insights can emerge from this iterative process without needing some explicit top down design for the knowledge graph self-organization is here key so our knowledge structure can self-organize just like any complex system in nature think about ons so this shift here the focus from some naturally grafting the knowledge base to designing system that can grow their own knowledge base organically again again why should we as humans code your knowledge graph we just give it here the nucleus and a main idea how to organically grow their own knowledge this is what we are looking for here now let's take here Gro 3 because it's available today gr 3 beta and yeah beautiful and I ask it here hey search for and I do this deep search or deep reasoning or whatever you call it here remember you have here the Deep search and the deep thinking so I go now here and want the literature search so I don't go for deep search here on Gro 3 and I say search for category theory in the literature and provide detailed scientific explanation of the most important methods and insights we can apply category Theory to Scientific problems plus explain category Theory so you see deep search goes on 30 sources exploring this those are the sources we have Wikipedia we have Stanford we have again Wikipedia and then more so real focus on Wikipedia because this is a trusted source of information this is a good idea not just to go to some internet articles or some Reddit articles that are just plain crazy so selects the sources and comes back and Gro 3 tells us now hey category here is great for science because it's simplify complex ideas it turns messy details into clear abstract patterns like using a functors to map one system to the other more about functors in a minute we connect different fields physics biology computer share similar structures we use categories to model Quantum fields or program behavior and we can solve our problems more efficiently isn't this beautiful now the key Point according to gr is a mathematical tool for studying structures in their relationship this is so important that we are focusing here on the relationship between different objects used in science to find patterns and solve problems unify Concepts across field like physic Computer Science Biology key matters include functors adjunctions and Universal constructions making complex problems simply simpler useful in real world science beautiful what is it it's a branch of mathematics looks at the big picture how different mathematical structures relate to each other imagine a map that shows how numbers shapes or even computer programs connect connected here create a 20th century beautiful so we have objects things like sets groups and spaces yes set theory yeah group Theory Lee algebra yes you go there algebraic topology yes absolutely you are on the right track then we have morphisms errors showing relationship between objects like functions or transformation and we have rules because morphism can be composed linked together in a way that's associative and every object has an identity morphism and you exactly see here the group structure and the symmetries and so on so nice idea back walk yeah this this is something I can work with I hope you can work with this main matters in inside this was the part the key part of my question here funs they are like translators mapping one category to another while keeping the structure okay that's uh that's an interesting view help compare different scientific models like linking physics theory well yeah okay a junction so dual relationship like free groups relate to groups understanding of symmetry in physics yep Universal constructions biology system modeling equivalence High category Theory Quantum field yeah there's quite some other mathematics so okay so now comes the spark you know in this videos we already have a spark here a little bit genius and this is this paper by MIT now normally I don't show you papers where we have only a single order but I know this order I've seen the work of this order for now multiple years I read this study it's more than 100 Pages this is really an unusual study just want to make this clear right from the beginning published February 19 2025 MIT we are here Center for computational Science and Engineering and it is about a gantic deep graph reasoning that it yields your self-organizing knowledge Network it is a lot of experiment especially here with Material Science here and I just look at the keywords like graph Theory category Theory Material Science language modeling reasoning isomorphisms and Engineering so this 100 pages are great I just look at a very tiny element of this I want to look at here at the Dynamics of my knowledge Crystal that self forming so I just have a structure knowledge gra and an EI agent can this build the world now Marcus tells us here hey we present en gentic autonomous graph expansion Network that iteratively structures and refines knowledge in C2 in the knowledge graph and unlike conventional graph construction method that rely on static extraction and single pass learning they have here exactly the couple here the reasoning native llm our AI agent with a continually updated graph representation so this is exactly what I showed you here Al a topic that I I'm currently thinking about a knowledge gra agent add knowledge now please remember an agent is defined in itself that it has a context to the external environment the agent is not that it is isolated but with function calling or whatever we have we have direct interaction it interacts here per definition of an AI agent with the environment therefore the environment is something where this agent can see new facts can see new structures emerging can detect new objects and maybe come up here with a reasoning path for unseen objects with unseen reasoning traces so here the feed in of this new data of this new relationship especially from category Theory our AI agent should be able to learn so this means there is a certain minimum threshold in the complexity of this ER agent that this ER agent has to have to be able to deduct further data from the environment and understand the reasoning process okay you see it's easy if you want to see my video this is already months old that I did on Swarm intelligent very simple multi-agent ecosystem how does AI swarm intelligent build how does it grow just an example of nature look at this this is absolutely amazing now let's come to the core idea of this video according to MIT study by marus we have an agentic system no it's not passive it actively participates in the knowledge construction itself it modifies the the knowledge representation based on its own evolving understanding as an EI agent so we do have a real dense interaction between our knowledge graph and our EI agent plus we have now a graph as the native knowledge representation careful you might saying yeah but we also have a knowledge representation within our eii agent yeah but this is exactly the topic of this video now we have two entities we have a dynamic knowledge graph and a little eii agent that is also intelligent but now feeds iteratively pieces of more information of more relation of new unseen objects into this knowledge GR graph into this knowledge Crystal that grows now with time and this is crucial graphs inherently relational allowing here to for the explicit encoding of Concepts and their relationships so now the most important sentence this structure enables the system to capture higher order patterns like hubs or Bridges or communities in the graph structure which are often implicit in the text or other unstructured data so now we have an emergence but not as super intelligent just that in a graph we see suddenly higher order patterns emerge here when we look at a graph structure we see hubs Bridges and communities clusters emerging now this is nothing special for the graph special is that higher order patterns emerge by an iterative process that maybe is less intelligent and now to the main question can matter can a Transformer based EI agent self-develop higher order knowledge patterns as an agent or is the agent not able is the agent the wrong object to do higher order knowledge the agent is just feeding in iterative processes here and here we have the emerging of higher order patterns given that we have here this specific two mathematical models that were working with we integrate here this is our crystalline structure here of the knowledge so where is the complexity now we can reformulate this if an agent can self-train itself on a higher complexity input from the environment so it sees new data higher complexities are there we generate now we see this we notice it we want to data sets and we want to self-train an agent we know that an agent with a high complexity data set if we train this the agent will perform this higher complexity task but the point is can an agent with its given intelligence detect higher complexity in the environment or is it limited inherently by its maximum complexity knowledge that it can detect analyze and understand understand this is something that is not as easy as you might think because can anen really detect higher complexity in the real world in the environment or will it be limited to its own complexity its inherent complexity level and then there's the other question if we say no it cannot detect this so it takes here its little sub knowledge graphs it's puts in here but here now in the graph higher order patterns emerge and you remember I told you that for the next task the little AI agent now takes the complexity takes everything here of this new Knowledge Graph including higher ORD patterns as a base for the next decision so we have maybe a limited internal complexity structure but with the graph we have access to a higher order pattern is this now enough for our little EI agent to reason on the next complexity level although it is not an inherent knowledge of the a agent so can this EI agent detect higher complexity in the graph and if it can detect it in a graph then it should theoretically be able to detect it in a real environment or are there mathematical limitations that we have to be careful of now we are talking here about self-organizing knowledge networks and you remember the structure in our crystalline structures emerge here from the feedback loop itself from this little iteration think about here the complex systems in nature like a colonies so self-organization into functional units without central control is something that we know but again since we have no design top down approach is the system does it have natural thresholds that it cannot cross given a certain complexity level in the reasoning process I mean we we humans can detect higher order structure in a graph we see it we see that new objects emerge but can any eye agent realize and detect it and use it so the question is can an i and extract higher complexity patterns from a graph and design higher complexity training data sets for the samei agent to selft Trin itself for a higher complexity level or is the AI agent simply unaware that is extracting a high complexity pattern from a graph because there day exist and he build it this little agent and thereby now designing artificial higher complexity training data set although not realizing about a complexity threshold it can self-train itself now on a high complexity level I find this questions interesting I'm I'm currently running some tests on this but you see this is is where EI research is currently heading and I love this article by MIT okay let's come to the facts the main hypothesis was the following for the Mit paper and I'm referring now to the Mit paper a recursive growth expansion is our topic and this recursive graph expansion by our little AI agent limited or not enables now some self-organizing knowledge formation leading here to an intelligence like quotation mark Behavior without predefined ontologies external supervision or centralized control now the process schema is rather simple now we generate some graph native reasoning tokens we have here an llm MIT here with a specific llm where I have here is a reasoning token with beginning of thinking and end of sying you notice this is classical so this explicitly marks you the model reasoning process making it here extractable interpretable we can compute further on this from the lm's reasoning token a local graph G local is now extracted built simple we notice involves identifying Concepts in the node in the relationships in the edges so we have now transformed some unstructured textual reasoning into a structured graph format and then we merge the extracted graph with the larger graph we know how to do this no problem and then then comes now this interesting point yeah we generate now a new question so basic on the newly added notes and edges in our local graph a new follow-up question is now generated and this is now the crucial feedback mechanism so the system uses now its own recently acquired knowledge that it put in the graph to guide its next step of exploration ensuring here an iterative refinement and preventing here some aimless wandering around in an empty space so this system now is more intelligent because it doesn't wait here for a new prompt by the user for the next more complex task but the system now generates its own question so it's designed to have a feedback mechanism where it rechecks now hey do I really understood this acquired knowledge now that I extract now from the graph and this is such a beautiful idea and then we have iterative freeing in the loop it goes on and on and on and this more or less is our engine of the self organization allowing here our main graph to grow and evolve dynamically by the knowledge fed in by our little AI agents and there are hundred of question that remain of course because remember we are talking here about the growth potential if we go to higher complexity levels of a knowledge grow structure as a network and of a neural network so what are the differences what are the growth trajectory regarding here a complexity Evolution absolutely fascinating okay let's come to an end so this study also provides more than 100 Pages have a look yourself it is a really fasc it gives you so many brand new ideas but they also have some let's call it experimental data and content is we have some scale-free Network structure structure that emerge so from all the test that they did they exhibit some scale-free characteristics so this means that the degreed distribution follows here a power law indicating the presence of few highly connected Hub Concepts this structure we know common in real networks including our knowledge our social networks our biological networks beautiful but this means now that this suggests that our system naturally organizes now knowledge hierarchically with a central hub acting here as organizing principle mirroring here kind of a human knowledge organization plus we have Bridge notes graphia exhibit here emergence of Highly connected Hub notes and the bridge note connecting disparate knowledge clusters so the hubs represent the Core Concepts around which knowledge clusters while the bridge note facilitate here if you want an interdisciplinary connection and kind of a knowledge synthesis so it builds here the bigger picture plus we have yeah modity Small World Properties we have distinct conceptual communities emerging Community formation yeah beautiful we have linear growth we have a nonsaturation interestingly even if you go to thousands of iteration this shows us here more the open-ended nature of this knowledge expansion process in the knowledge gra itself the system seems to continue to learn and learn and learn integrate continuously new information without becoming redundant or collapsing on its own complexity of course here we depend on the reasoning capability of our e agent so you should not go with an agent that is maybe not there in its intelligent level yet we do have a dynamic evolution of the network properties you should have a look here at modularity short as PA length diameter you have all the new network centrality uh figures beautiful and the interpretation is the system under goes phases of structural reorganization of the knowledge which is amazing in itself suggesting here that a self-regulating process that leads to a stable and a more efficient knowledge structure this is something now especially when you read the next one that there is sudden breakthroughs with new Concepts so there seems to be that the system becomes more and more and more a little bit more unstable a little bit more unstable the more we add Mass to this logical concept this knowledge graph then suddenly it finds a new configuration that is more stable so the knowledge if you want reconfigures itself and builds your hybrid knowledge expansion model it is not that easy to explain on a pure knowledge level but hey this more optimal configuration of knowledge complexity it seems that they might exist this is all I can tell you for a moment just remember the complexity level is the one of the most important factors in this video about llms and their breaking points we looked at some beautiful exploration and we had here the complexity of a task and we had a small complexity medium larger models and ex large models and we have here these reasoning models 01 and 03 and deeps R1 and non- reasoning models like gbd 40 or claw Sonet 3.5 and you see the moment we go up in the complexity you see the non reasoning mods the performance goes to zero and even if you look at a large model and this is a 6 * 3 Matrix complexity if you want have a look at my video if you want to know more you see even here the o1 mini comes really down and only the R1 and the o1 have here the perform and then if it just go from 6 * 3 to 6 * 6 in a matrix structure you see even the biggest model here fail to solve this so the level of complexity if it increases would a little agent be able to build a knowledge graph that would inherently form stable configuration of new reasoning traces that are able to solve higher complexity task I think it's it's an absolute fascinating question and I don't have any answer so if you have an answer please leave here a comment let's look at a work by MIT and you will see or I feel that there's a certain limitation to it you know like let's have a look here at this image and you see building up complex idea from simpler components in a hierarchical and interpretable manner so we have your Atomic Concepts then we have pairwise compositional Fusion of those and then we have bridge synergies and then we have the final discuss now it's simplified but you get the idea so we have here I don't know materials for infrastructure design or biodegradable microplastic and then suddenly the pairwise fusion is ecor resilient infrastructure design I'm not sure that you see the question is let me for reformulate it the question is is this here A Low complexity is this here A Higher complexity well are those not complexities that are more or less in the same level in the same group in the same set of complexities because I have a feeling that there's not a jump over the borders of a complexity into a new set of higher complexities I think this is more or less own on the same complexity level and now the question is is this complexity level determined by the complexity capabilities of our EI agent or is it somewhere in between the I agent and the emerging features on the knowledge graph I can't give you here even a feeling from my side because I don't know it the interpretation here is highlights here the systems capacity for a systematic and logically sound knowledge construction going Beyond Simple pattern recognition and absolutely agree this goes beyond simple pattern recognition but we are talking here about more complex task and here complexity levels so no way it goes beyond pattern recognition but does it really cross a complexity border I'm not sure about this yeah let's come to the summary we have something beautifully we have a recursive graph expansion in this paper of the MIT as a model for self-organizing LGE and this knowledge itself can self-organize and grow organically if you just put here an EI agent with a contact to an external real World environment where the agent can learn can view new things new objects and learn new relations self Trin itself and build a knowledge gra second compositional reasoning for systematic knowledge synthesis so building up this complex knowledge ideas from simpler components as provided by our AI agent the graph can do this and address the limitations of the llm in its own limited systematic reasoning because it seems to be that the knowledge grph can represent higher complexities easier than a newal n transform architecture and this beautifully aligns here with the growing field that we see with AI driven scientific discovery in medicine and Mathematics in physics in Material Science in space science wherever you go those efforts to automate your hypothesis generation and knowledge synthesis and Science and Engineering so this is fascinating especially if you think about space and so on so this could theoretically offer you a novel approach to self-organizing and emergent knowledge rather than just the classical AI of data analysis and pattern recognition so maybe you understand now why I'm kind of excited about this new idea because I think this is the way for eii to go and there are so many open questions I don't have even an idea how to start to answer this well maybe I have an idea but more about this in a later video and what I wanted to show you just yesterday February 19 2025 Google researched a new publication accelerating scientific breakthroughs with Ani co- scientist so Google tells us here hey we developed a new multi-agent system with of course Gemini 2.0 hopefully as a virtual scientific collaborator to help human scientists generate new hypothesis and research proposal especially also then for bi iCal discoveries so I think this might be the context of my next video because I will have a look at this I will have to read it in another time and then if I decide that this is really a this is kind of a follow-up video yeah you will see it as a subscriber right here on this beautiful Channel