The Math That Kills Trillion-Parameter AI Models

Key Takeaways

Hello community. So great that you are back. Let's talk about the latest research in artificial intelligence. You know I declared in one of my last video we destroyed AI and then we just rebuild the eye in my very last video with a new algebra and you're not going to believe in the very next day Princeton University published a beautiful paper. So let's talk about this. March 14, 2026 an alternative trajectory for generative AI Princeton. Now Princeton focuses here on this topic and you know reinforcement learning our post training it is everything beautiful if we have verifiable reward structure either mathematic or either code that we can calculate but open-ended we are absolutely here in the open end we do not know how to handle this especially if we talk about reward function. So new idea is what about we have abstraction as implicit reward models for our post-raining our reinforcement learning. Let's have a look. They tell us current LLMs and heavy some genuine reasoning path only in domains mathematic and coding where we have rigorous pre-existing abstraction that provides structural grounding. either this is a Python environment C++ or you go here with the mathematics and they propose now something they call and stay with me don't run away domain specific super intelligence it's not really the most intelligent title but the idea is absolutely fascinating so do you argue we do not need this monster proprietary AI system where we have one system for 8 billion people but what about we build the expert system. What about we argue to achieve a robust reasoning in an open world domain, we just have to construct explicit symbolic abstraction such as knowledge graphs, pure ontologies and implementation of formal logic because formal logic served us very well if we went into mathematics and coding. You know I showed you here when we extracted here the skills from a chemistry textbook or any other documents we saw that our skill MD those configuration would be here the solution for everything. Turned out it is not because I showed you just here half an hour later that skills are great. skills are task level workflow templates. But what we are missing and I showed you this in this particular video in this study was X skill that we we lack experience and action level tactical knowledge exactly on how and when to use skills. And I defined experiences in this video as kind of an episodic prototype compressed generalizable traces of specific encounters. And now you immediately understand when they say generalizable reasoning traces. Yeah, knowledge graphs of course. So you see it did not appear just out of the blue sky. But we already had here an other publication the same thing. Skills are not enough. We have Princeton tells us instead of experience we need something like knowledge graphs. Now the idea is more or less the same. No we have domain specific corpus our text. We have as a cold start some semantic source a seed knowledge graph. We have our sentences here. We have then the support of our triples. We build a unified syntactic and semantic representation. And we have a graph mer. What is it? It is a distillation of factual and autological valid domain specific knowledge graphs from raw text. So what I showed you here from pure text from your chemistry textbook or maybe some visuals. We have a huge vision language model that extracts now all the skill MD files, markdown files. Now we do more or less the same thing here. And now we extract here complete knowledge graphs. And they call this graph mer beautifully. And they tell us why we have to do this. Because if we just do supervised finetuning alone, this is insufficient for elicit robust zero shot compositional reasoning. And they tell us, hey models trained only on supervised data can still default to superficial pattern matching when faced with high complex unseen multihop scenarios. So SFT is not enough. Therefore we have reinforcement learning but our traditional reinforcement environments are too rigid. So Princeton said okay what should we do now? And Brston tells us the solution lies in using now abstractions. Abstractions themselves as implicit reward models. And they say this works because a knowledge graph or a general graph encodes now the causal and the relational links between all the entities of the domain of the domain knowledge and therefore the multihop paths within a knowledge graph can now serve as a ground truth. This is a verifiable logical chain. You see what we achieved with this? We don't need anymore a process reward model. We don't need a second LLM that gives us here the reward structure back. If we have our knowledge graph built, we can use now the structure of the cause and our relation that links between the entities. But you know what the entities those entities are special? And if you want to see this here in a even higher mathematical solution, yeah, yesterday in my video we talked about the algebraic causality for NEI and I solved exactly this of Princeton on a higher level. But yeah, this is a screenshot from the study. But let's let's stay at a higher level. Let's say a knowledge graph transforms now a domain knowledge with the objects and the relation into let's say a verifiable complex structure. Let's call it elaborate. And now we want to find here the solution. Maybe this is a particular graph, a path forward that we lay over the labyrinth. No. And remember we higher levels are highly abstract. So the nodes of this graph are now if you want concept abstract concepts and the edges of the graph are logical rules. Do you see what we have? Can you feel it? Yes. the node and the edges of a knowledge graph can form an algebra. And now I close here the circle back to my video from yesterday. So this means for II researcher viewing knowledge graph on a higher level as an algebra changes everything about how you build architectures in AI because if reasoning is now just an algebraic structure, you can leverage centuries of mathematical optimization. You have a knowledge of thousands and thousands of mathematical expert who build algebbras and found out everything to know about an algebra. Let me give you a simple example. Graph embeddings. If you see a knowledge graph as an algebra, you cannot normally easily do calculus on discrete words. No in our classical auto reggressive predict the next token transform architecture. But if you map these algebraic primitives into a continuous vector space, let's say we do some geometric deep learning, you can now use our classical gradient descent to solve logical equations. But careful, you understand immediately, hey, we have to be absolutely careful how we build our algebraic primitives for a particular domain. Let's say medicine or finance or theoretical physics. Okay, another example by treating now edges also as causal primitives you can now implement some calculus inside a neural network so for example you can now solve a problem we cannot do today define counterfactuals let's say what happens to the graph if we surgically intervene and change a node x what is the implementation for the node y z and so on you know this what if scenarios these hypothetical scenarios we want to have a deep mathematical solution for those we can do this if we view also edges as causal primitives or let's just say you multiply here infinite chain of numbers any trained on relational algebra can now compose infinite chain of causal hops without degrading an accuracy because we we the scientific community knows almost everything about algebbras. So this is how you can beat for example the context window limitation of our classical standard LLMs. Do you see what opens up if we just have a reframing how we see EI and the sequence and the complexity of an EI as a knowledge graph. You're familiar with this. But now let's go from a knowledge graph to a mathematical algebra that we have to define. Because if we do this by explicitly grounding the lambda in an algebra of causal primitives similar to you, we give now the model the algebraic operator itself. And this makes life so much easier if you build no models. If you want to see how we turn a graph into an algebra, just leave a comment because I think it would be too much for this video. But if you want to do it yourself, I'd give you a hint. Because if we embed an algebraic structure into a highdimensional continuous space, we can treat the logical rules as geometric transformation. So leave me a comment if you have solved this. Okay. So you see a knowledge graph and algebbras are real close together for the development of future AI systems. But let's come back because here this is another study by Princeton University. This was already published March 6, 2026. Knowledge graph as implicit reward model. So let's come back here to the problem that we have here. Our reinforcement learning here with nonverifiable reward structures. So how we built them? And the answer is simple. We have knowledge graph as our implicit reward structures. You have here the GitHub. Beautiful. Everything is available for you. And the idea is easy. No, you have into your training phase. You have your one hop, two hop, three hop. Beautiful. You have your supervised fine tuning. You have your reinforcement learning. You have your post-training phase. You have knowledge path inspired corrected reward signal comes back. And then you have your base model, your supervised fine tuning. You do it on a Laura adapter. And then you have your reinforcement learning. You go with your GPO policy optimization. And then if the real run happens, the inference phase. Huh? Suddenly the system is able because we did have RL is able to solve a four hop or a five hop complex reasoning structure. Although it has never seen in the training data a four or five hop task. This is the beauty. This is the absolute beauty if we go with algebraic operators because then we don't have to put tons and tons and thousands of training examples here in the post- training and the pre-training data set. But then we define an algebra and we just give it the algebraic operation and the I could solve this and of course we will use lean four or whatever we have. But this will make it so much easier. So this means if we treat now here in this paper here by Princeton because yeah let's say with Princeton so we treat knowledge graph as implicit reward models no and use now the path derived rewards in a scalable fashion to teach now to learn those models how to compose domain primitives and you notice domain primitives were critical elements and how to compose those domain primitives into a long reasoning chain those are our reasoning LLMs Now you see exactly that we have now a mapping. If we have domain primitives in my last video we talked algebraic causality and if you want causality primitives you immediately see the link why I show you now this study after this video. So as I told you combined as supervised fine-tuning compact reinforcement learning we have a knowledge graph path aligned reward structure an implicit reward model to synthesize model then can generalize from one to three hop training to unseen four to five hop training that are even robust under format pertupation and OD out of distributions. How beautiful is this? Now the orers tell us here now from Princeton. Okay, so we did our um TRPO setup. Beautiful. So we have now our LSM policy pit, our strategy if you want. We have a prompt X from a grazy human and then we generate a multi-step chain of sort reasoning trajectory, a causal reasoning trajectory Y. So if you go now with a knowledge graph, we pass Y onto a sequence of discrete logical steps or hops. Beautiful. And instead of a neural reward model a second model here we define a deterministic algorithmic reward function based on the graph topology. So how do they do this? Now it turns out it is really simple. There are for the total reward function this is a composite of three signals. So let's have a look at this. The first signal is the aimatic validity the local reward your step reward. So for every reasoning step si claiming from a to b we query the knowledge graph and does the edge actually exist. If true our reward for this particular step is plus one and if it was a hallucination guess what we say minus 5. So some harsh penalty to disenourage here the guessing by the eye framework. Second signal is the chain continuity. So the structural reward does the step logically connect to the step before it. I mean in the simplest formulation the tail node of step one must be the head node of step two. If this is true plus two not minus to the reward. And the third one is the terminal grounding the global reward. Yeah. Does the final node reached at the end of the trace satisfy the prompt condition. So you see if you put everything together you have the final objective for the reinforcement learning algorithm. And this is here our optimization exercise. Now what is really nice and we know this for years if you want the scientific magic. Why this causes here a zero shot scaling? Because the reinforcement learning did not teach the model the medical facts like in the supervised engineering. This knowledge graph derived reward if you want taught them all the invariant mechanics of a logical deduction. It learned here not just to mimic a sequence but hopefully to understand here the logical part of the sequence. So being forced to construct some unbroken hopefully graph step by step the model hopefully internalizes our AI an algorithmic loop and the loop is simple. Identify the entity that we are talking about. Find all the valid relation for this entity going out to in of this entity. Retrieve the next entity that is connected or maybe within an epsilon environment around this entity and then check the continuity and then go on. And the hope is that once the neural network learns this algorithm of a graph traversal, walking five steps now require the exact same one parenthesis cognitive machinery as walking two steps. So therefore the model hopefully learn to reason rather than just to recall step one to three. So this would be a solution out of out of distribution problems. Nice. So you see we're working now or we're operating now with learned abstractions in neural network and by turning now an abstraction that we build now as a knowledge graph into an implicit reward model we transform the domains that lack the formal compilers the C++ and the Python solutions like medicine law or science into perfectly verifiable mathematical problems either with a knowledge graph or on a higher level with an algebra and this is not a cornerstone of Princeton's new paper on the domain specific super intelligence by the way it's quite a heavy paper there's a lot of studies and a lot of insights into energy consumption water consumption so it is kind of a grand survey so if you are interested in this I highly recommend this study personal sorts yes of course by defining Meaning now abstract concept in AI as geometric vectors and our logical rules as mathematical operators let's say transformation translation or tensor products we can decompose any complex reasoning task into a sequence of computable algebraic operation. Is this correct? And if it is correct, for what domain is it correct? Do you find uh an example where this would be not valid? Leave a comment. So therefore, I hope you had a little bit of fun in this video. I've showed you here there is some beautiful current research going on in EI about a new algebra. No, it was about here the knowledge graph as an implicit reward model. But if you reframe this from a little bit a higher mathematical perspective, Princeton studies becomes suddenly that we are talking about a new algebra in the eye. And now we have the absolute fascinating event of an algebra discovers a new network. And this opens up complete new possibilities for the next generation of AI. I hope you enjoyed it. Become a member. See you in my next video.