Distill 5 AI Agents into ONE (w/ CODE)

Key Takeaways

Hello community. So great that you are back. Yes, you know we have here from Open a new product placement. February 5th we have here OpenI Frontier and Frontier. Finally we have here multi-agent system for enterprise customers. And we have another product on the very same day February 5th 2026 GPD 5.3 Codeex. Also here a commercial product release. And you might say okay what is codeex here? Now they tell us here hey we can combine the performance of 5.2 codecs and the reasoning and the professional knowledge capabilities of GPT 5.2 and it will be 25% faster. So we have our codeex 5.2 and our reasoning 5.2 now combined into a brand new system. And you might say hey this is absolutely amazing. And they show us here also here strong professor knowledge work matching now GPT 5.2 and you see the model performance is here for 44 occupations. The task includes things like making presentations, spreadsheets and other work products. So we are moving here a little bit away here from a pure coding exercise to a presentation understanding spreadsheet analysis data topology and other 44 occupational trends. So you clearly see this is a commercial product release for specific industrial branches. And you may say interesting so now finally we have GPD 5.2 performance. No, and if you look at this video, I show you here what is the performance of GBD 5.2 on reasonings. But let's come back to Frontier because Frontier is a multi- aent system and finally OpenAI also has here multi- aent system for enterprises. And you might say, but wait a minute, you just showed us that Stanford University did here a beautiful study here a week ago. They tell us, "Hey, careful, don't use multiple agents because they will fail." And they have a complete uh benchmark, they call the Koopa benchmark here where they really demonstrate this. What are the current problems if you have two agents working together and they recommend don't do it? And now we have a commercial product here by OpenI for enterprises that may be not familiar here with the details of agents. Is not this a contradiction? No. On the left side you see here a marketing and on the right side you see here a technical if you want scientific insight from AI research. Those two systems have nothing to do complete different groups they addressed. Yeah, until you see this study of today, February 3rd, 2026. This is from Kim Mal University, beautiful Yah University, William Maria, I think. They're in Virginia, Georgia Institute of Technology, Amazon. Hello, Amazon and University of British Columbia. And they tell us something completely different to the product placement by OpenAI. They go in a different direction. They say you know what we're interested in distilling a multi- aent intelligence into a single LLM agent. So this is exactly the opposite what the product of openui is offering now to the enterprises globally Amazon Carneg University of British Columbia they say well we we know that it's not working yeah here Stanford University it has so many communication problems here topological problems here in the knowledge manifold so what we want we want to bring it back into a single agent okay so they have a Beautiful GitHub. By the way, just wanted to tell you you have here the complete setup, the quick start, all the key arguments that you want to have. And you will remember in my last video I told you, hm, Google warning was in context learning here. If you just prompted with the data here, this AI systems are fundamentally flawed for complex reasoning. And this was shown on a GPD5 system. And you remember I ended here part one. Well, I'm going to have a look for a solution. And guess what? Today in this video, we solve everything. If you want to have the code here, you find it here in the GitHub rep of Kagimal University, we will use a process reward mold training code is already available for you completely. And then we will have a reinforcement learning with a GRPO policy optimization. The code is completelyable for you. just yeah you know a gear up so let's have a look at what is the main idea here what does the authors show us here they tell us okay if the transient context like young showed you in my last video is fragile and inert this agent arc now argues that the solution is simple no you know it burn the dynamics of reasoning traces into the model the weights And we don't make it too easy now. So we don't take one agendantic system but we take a multi-agentic system here but not as a deployment strategy which is slow and expensive but as a data generator now for process aware distillation and if you want this pad methodology is the new innovation of this paper. So we are forcing now a single student I model. This can be a really tiny local model on your PC to predict the gradients of the debate of all our multi- aent system. This means all the correction all the conflict that those EI agents might have and the final consensus the reasoning traces and it transforms if you want the ghostly context of a multi- aent debate into a single agent and it will be a distillation of course. So you can say okay let's not go with a single agent system let's go with a multi- aent system and then we just condense it into one agent and you know what it will be so much cheaper from the computation cost. Now remember in my last video I showed you that Google warning ICL context is inert. ICL context is insufficient for complex robust reasoning because the resulting representation in the activation space are ungrounded. And today today we have this beautiful new study that will show us that the parameter efficient distillation of the reasoning process and now of course we don't go with ICL we have to go with a modification of the tensor weight in the layer. So we go here with a GRPO optimization and a very particular reward model a process reward model. This will be the solution. This will bridge the gap turning our abstract reasoning traces into reliable inference type comput system. Now you see the connection to last video. This is here the solution. We can't use ICL for reasoning processes with a two topology. We have really to go and have here a post training and the autostellus GRPO with PRM currently a very good way. Now I know that's one of my subscriber will ask me again but hey is there not a addition to me now because this is here the first commercial product here February 5th here so we have openi frontier multi- aent system for enterprises and now you come and tell us that the research community is already looking here for condensing multi- aent system back to one yeah of course now you know instead of paying here let's say 10 times for a GBD system like in this commercial product offering where you have let's say 10 agent no in AI research the community has already moved on but we now distill this multi- aent system down to a single agent system and guess what therefore for us it is 10 times cheaper than using here openi frontier 10 agent system where we have 10 parallel copies of GPT working in parallel and I'm sure that yeah openi just in its um profit optimization forgot to tell you about this late. So let's have a look at the visualization. Here we go. So what we do we have now a single agent and this single agent knows everything about here the collective here of multi- aent interaction on a particular domain knowledge on a particular process knowledge here. So we are shifting here the inference time compute which is expensive slow and transient here from this multi- aent communication into a efficient and permanent training time distur agent learns everything here from the group dynamic and the best process also with internalized reasoning. So great what we need we need training data from the group we have to filter out the best one about generalizability efficiency reasoning multi-perspective robustness cyber security and we distill everything back to a single agent and this is then really inexpensive to run and I will show you we can do this here with real tiny LLMs so we can run them locally and you don't have to pay anybody anything now [clears throat] they look at V progressively deeper distillation strength and I want to have it that you have it here in one view. So we have a reasoning enhanced outcome based super fine tuning with particular reasoning traces training here the eye model and the final consensus reached by the agent group while leveraging here the reasoning trajectory to ensure a single agent can reach those high quality. Second is a reasoning trajectory based data augmentation. So we are distilling here the diverse reasoning chain from the multi- aent communication allowing you the eye model to learn a variety of logical strategies and way of thinking and third if you want the main innovation of the paper a process aware distillation here we have now a gio really our reinforcement learning a process reward model to train the single agent to internalize the complex critique and revision dynamics of the multi- aent group enabling thereby a single agent to emulate the dialectic reasoning of a multi- aent debate within a single forward pause of our transformer architecture. So what are the key findings? Key findings is simple. This new methodology enables a single agent to acquire multi-agent reasoning ability and therefore operate here on a much cheaper financial basis. Second, our process reward model capacity matters more than the model size. While student capacity bounds your multi- aent gains, a high capacity process reward models enable strong improvement even for small EI models like a 1.7 billion free trainable parameter model or a 3B model that you can run locally. So reasoning quality is of course absolutely important because the reasoning traces are exactly here the objects that we will train our AI student models on. So simply adding more reasoning trajectory does not improve the performance. We need here a very particular maybe increasing reasoning quality for your particular domain or your particular job complexity. And here we have it. So this is the complete overview. So let's start. We have here on the left side the data generation. In the middle we have the knowledge extraction here from the multi- aent groupings. And then we have here if you want the hierarchical distillation with all the methods we just discussed easy data generation through multi- aent debate to produce diverse reasoning trajectory. So you have let's say you pay here for openi. you have 10 agents and you think okay I found here configuration I like you get all the data back and then you decide hm I don't want to pay forever for 10 times your GPD system or whatever yeah so you can have a knowledge extraction from the data you filter if you want here for high quality reasoning traces you have a distillation with data augmentation reasoning enhanced supervised finetuning I will show you this contrastive learning specific loss function And of course this process reward model your PO. Great. And you arrive here at your perfect little small LLM that can do what here the complex group of LLM agent did. Okay. So what we do we train a process reward model under the bed traces to score a stepbystep logic and then use our classical GPO to update here the policy. So therefore we optimizing now suddenly the trajectory and not just the next token probability. So I told you we have three methodologies. Let's start with the first one. Reasoning enhanced supervised fine. You know this this is our classical reasoning traces as supervision. This student model is trained to maximize the likelihood of a raw multi-agent reasoning traces and the objective is simply given here by this formula. So we have two terms. The first is the reasoning loss which optimizes the model's ability to generate a coherent sequence of intermediate rationals. And the second is here the answer loss which ensures that the final prediction yar is grounded in both the input context and the preceding reasoning of simple. The second is data augmentation distillation with data augment. So here we have a teacher LLM a big one 32B model and this is prompted to select here the best reasoning traces that employ distinct mathematical identities different logical uristics or even better varied sorting assumptions. So you want to have a real diverse set of logical constraints. So fully exploited the diversity in the multi perspective here they implement what they authors call a correctness first diverse extraction strategy. So if you want this is here the single simple optimization and simply forces you the model to learn multiple valid path to the same solution. You just want to increase the diversity of your reason and augmentation and of your reasoning manifold thereby improving the robustness of the solution and the generalizability of the optimization. Now the third one is the most interesting the process area distillation where we have PM with our GRPO reads now a distillation as a reinforcement learning problem using here the specific process reward model for a granular step level stepbystep supervision that reinforces the intermediate logical transition found in those multiate fates and we train here PRM to predict the probability that the reasoning step is correct classical nothing specific only the orers tell us hey we found if you do a two-stage curriculum we get the best results okay stage one feature alignment so we initialize here the reward model with the weight of the student model and freeze all layers except for the final one and the reward head so simply this prevents you the loss of the pre-trained linguistic features while the reward head learns to map existing representation to the correctness labels. Okay. And then of course we unfreeze the backbone. We unfreeze the entire backbone for end to end fine-tuning. And this now allows our model to develop specialized internal attention patterns for detecting here logical fallacies. So simple you know gpo fine-tune the student with gpo optimizes the policy by comparing the rewards of the output within the group removing the need to separate value function and yes can reuse the mics of an older video what are the results now the results I have to tell you are not so clearcut let's have a look we have here distillation from a Q3 32B to different student mall student moles is from a Q13 8B, a llama 38B, a gamma 7B, a Q13 1.7B and a Q3.06. So now we have here the in domain performance on the left side and the out ofd domain performance on the right side and you see on the y-axis the improvement in percent and for all three distillation methodology. Now you see that there's extreme sensitivity for the model because if I go with a Q138B the improvement is almost zero and even negative on all three methods. So this means this mall is not really sensitive to this three distillation probe. The same is true for the llama 38B. Look, there's a little bit but a little bit but not really. But now to the surprise. Look at a more modern gamma 7B improvement is above 20%. 30%. Above 20%. What a surprise. A QN3 1.7B almost nothing here. And if you go for the dark one and then a QN3 0.6B 6B in yellow. Real strange. No, what a if you want reasoning supervised fine tuning, we have a negative added value. Then we have an improvement here in the distilled data. And for PAT, we have a + one plus 2% improvement. So extreme sensitive to the molded. The sync changes for out of the main. As you see, suddenly a llama 38B has an improve u improvement here. Careful, we have now a different scaling on the yaxis here of 7%. But also here a 3 0.6b is also plus 7 percentage points. So this is quite significant just for a better distillation and augmentation of your training data. and PAD is not so famous look I would say above 2%. So interesting this is the in domain performance out of domain performance and those are the results of the re distillation process. Now if you look at the next one, this is even more interesting. Yeah. And now we look here for the different malls here on the Y-axis. You see here we have our Q3. Here we have our llama 3. Here we have our gamma 7B and our Q3 1.7 and our Q 0. And you see four different data set for different matter or for different other benchmarks here. The improvement in percent is really dependent heavily depending on the model. Interesting. You see here again gamma 7B is here real sensitive. So this is it. If you have here a separate singular distillation improvement here RSFTDA or PAT but what about if we would combine those now here we have it now so experiment is still from a Q132B source model to a Q13 1.7B student model 1.7B run beautiful on a local PC so let's have a look if we would go here with the base so only our SF FT we would have for particular benchmarks never mind here let's say a base performance of 68.61 if we add now here two-step training with here our augmented data this now improves but the improvement is not so famous look we just go from 68.6 to 70.2 two. So okay you might say okay there is an improvement. So this is here RSFT plus then of course we can look here at PAT and DA. So you see we go from 70.03 to 70.69. So yes, it is an improvement here. But yeah, I mean if you think about here we had here this combination of pad and data augmentation. I personally think the jump is not as significant as you might expect. But let's look here at the difference between our RSFT and our PAD or GRPO. I would have expected that SFT would be not as good or PAT would be much significant better. But look at this. This performance data are real close. No 70.27 compared to 70.69. So in the study it is mentioned that PAD has an improvement. Yes, I can see this improvement. But is it really worth what we do here? If we just do here reasoning supervised fine-tuning with data augmentation, we are coming real close here. And look, if we go here for a different data benchmark one, we even have a better performance with our SFT and a data augmentation 43.9 compared to PAD 43.68. So yeah, I think you have to be a little bit careful here. What is your outstanding performance? If you look here at the real performance data and thank you here to the authors of the paper to be so transparent and give us your all the perfect data then you have to judge if you want to go here for a process reward. Anyway, I think scale is not enough. You cannot simply prompt a small LLM to be smart. There are simple limitation and it is not the context. It is not a prompt engineering. You simply must distill if you want the intelligence of the process of being smart for a particular job on a particular domino. And yeah, agent arc proves that you can distill the intelligence a five agent enseormal running here multiple rounds of debate into a single 8b parameter model that requires only a forward pass. Now I think this is a beautiful hint for lowering the cost of a system to reasoning model because you can do this recursively. Anyway, this method really allows a Q3 8B student AI model to outperform here after some intense training here even the QN3 32B teacher AI model in specific reasoning benchmark it was really trained on to where it could effectively caching the debate computer and by using multi- aent system to generate explicit traces of correction self-reflection or critique by other AI agent and then distill filling this via our process reward model on GPO. We are forcing our student model to learn here also the computation that is required to navigate the map and topology. Therefore, this is really a possible solution to the problem I ended [clears throat] my last video on. So your PO and process seem to be currently the answer for really learning complex reasoning tracing outlet to the future and this can be very short as I already mentioned maybe it is recursive distillation. So we train here a massive multi- aent system if it's really necessary if there really some benefit but Stanford tells us maybe not then we distill this complex multi- aent system into a single agent then we use this single agent as the base for a new larger multi- aent system and then we just repeat and the beauty is in the end we end with a single agent that can be really small 8b all that is able to run on your local PC. You don't have to pay open or I don't know 10 huge proprietary EI agents here or whatever. We have distilled the knowledge of the multi- aent group into single agent you can run locally. So, I do have to hope that we have smarter, smaller, real intelligent LLMs and VLMs coming up real soon. I hope you enjoyed it. I hope you had a little bit of fun. Why not subscribe? Maybe you leave a like. Anyway, I hope to see you in my next