Kimi 2.6 Test | The Best Agentic Coding Open Model? | Coding, OCR, Image Understanding | 🔴 Live

Key Takeaways

Hello everyone. Can you guys confirm that the mic is okay? Okay, thank you so much. All right. So, welcome everybody. So, today we got a new open model. And this one is Gemini K 2.6. Uh Hero, thank you so much for confirming. And this model, unfortunately, is a bit too large, at least for my machine to run on. So, we're going to have a look at what the model is, what are the specifications on it, and I'm going to be using the official UI from Moonshot AI to try and test the model on a couple of cases. So, I'm going to be able to use their UI platform. And in some of the next possible live streams, we can take this model, for example, with open code or another harness, once the model is more freely available within their APIs. Even on open router, since this is like a very new model, it appears that they have like a lot of interruptions to the Moonshot API still. So, if we get this model to run maybe tomorrow or the day after that, and the model is running pretty good, the throughput is good, and we can hook up our open code to the open router API and try from there. So, for today, we're going to be looking at the model itself, and then uh we're going to try it out within the web UI, try some of the coding capabilities, some of the OCR and image understanding capabilities of this model, since this is a multimodal one. And I'm going to try at the end to push some of the PRD, or the template that we have started into the last live stream, the AI project template, and I'm going to put the information of the PRD in Gemini K 2. Uh sorry, Gemini K 2.6, and we're going to try and see whether or not this model is going to perform, let's say, a bit better compared to Gemini 3.6 or Gemma 4, the models that we have been running locally. So, again, welcome to everyone that is currently watching. Happy to see you guys here. Let's see what this model is, and how it is different compared to Gemini K 2.5. Uh this is was something that I was extremely interested in. And these are the benchmarks that are available on the Gemini blog. I'm going to paste the link to the blog itself into the chat, if you want to check it out. And what the authors from Moonshot AI are stating here is that generally this model has been optimized for agents or agentic workflows and coding. As you can see, these are the benchmarks that they're starting with. And this model has been further improved for visual understanding. Uh you can see here on the bottom right that we have math vision with Python and vision with Python tests. So, you can see those. What I usually do when a such model is presented to us is not compare it to, for example, GPT 5.4 or Claude Opus 4.6, since these benchmarks are, let's say, can be a lot of benchmarks, and I'm not really sure if I can trust the results from these benchmarks. So, what I I usually do is to go into the blog post itself, and get the results for Gemini K 2 .5, which are these. And let's try to compare some of the benchmarks. I'm not really sure if within the blog post they have comparisons between Gemini K 2.5, which was the previous model. So, let's try and see, do we have any of the benchmarks that are essentially repeated somewhere around these benchmarks, and see whether or not the improvements are as huge as the authors are claiming. So, the first thing that we can do see here that for Gemini K 2.5 on the Bro Comp benchmark, it got uh roughly 79 75%, while here on the K K 2.6, actually, let me do this for you guys. Uh we can do this a bit better to compare the results. Okay. Okay, let's do that. Okay, so, Bro Comp from 74.5, and this is Gemini K 2.5, to 83%. Here you can see that. Humanities Wast Exam with tools. Uh we have improvement of uh roughly 4%, as you can see here. Uh Deep Search QA from 77% to 92%. Okay, for coding, uh let's see that. SWE Bench Pro, but here it's not the pro version of it, so not a direct comparison. SWE Bench Multilingual, okay. From 73% to 76.7. Okay, this is again a large improvement. And on the image understanding, Math Vision with Python, I'm not really sure if this is the same test, but here it is 84% for Gemini K 2.5, and here it is 93% on 2.6. Okay, so, now that we have this information, we can assume that there is some improvement into the 2.6 model. And most of that improvement, at least in my opinion, is coming on from further fine-tuning and reinforcement learning of the base Gemini K, for example, 2 2 or 2.5 model. So, I think that this model is essentially another checkpoint that the authors have given us during the training. So, again, optimized for long horizon tasks, they have different ways to understand and look at the possibilities of this model to be given some large task, and this task is going to be left with the agent for, for example, here, 12 hours of continuous execution, and there were over 4,002 codes uh given by this model. And you can see the task. This is something that most of us can now understand what is the capability of this model in somewhat of a realistic task or environment. So, they have given Gemini 3.5, the 800 million parameter model, locally, and it was running on a Mac machine. And they have told the Gemini K 2.6 to optimize the inference of the model in Zig, and Zig, I believe, it's a programming language. I'm not really familiar with it, but the task was for the model itself to write the optimization of the inference in this Zig language, a highly niche programming language. It demonstrated exceptional out-of-distribution generalization. So, what I they mean by that is that probably most of the training data didn't contain a lot of Zig programming language specific tasks and data sets, etc. While this was the case, they say that even though the model was able to produce a very nice optimization that was written in the Zig language, and the optimization in general have improved the throughput from 15 tokens per second from the baseline to 193 tokens per second, uh achieving speeds 20% faster than LM Studio. Okay, I'm not really sure if LM Studio is a benchmark for inference speed in particular, but this is a great real-world task for such a model, so congrats on the team. And they're also given multiple other like tasks and what the model has been performing. Then, they're talking about the harness that is available in this design bench. So, here they have some examples of beautiful front-end designs. Okay, let's try some of it. Okay. So, this is supposedly a landing page that was created by Gemini K 2.6. And yeah, I can agree that uh this is a very nicely designed bench. Of course, this this might be like from hundreds and thousands of examples. Okay. And these might be just cherry-picked examples, but even so, it looks like that the overall quality of the design or the front-end capabilities of this model are very impressive for particularly on open model. Okay, so you can see that they're also talking about agentic harnesses and how you can use this model into Open Qwen, Hermes, and other agentic environment and harnesses. Of course, we're going to be mostly interested first in understanding what this model is capable of within this their chat interface, and then I'm going to be also interested in how well does this model perform, for example, compared to Qwen 5.1 in Open Code or another open agentic coding harness. So, here they're saying something very interesting that all of the experiments were conducted with temperature of one. So, this is the let's say the temperature that they're recommending or think that the performance increase performing best at. And the context length from here we can see that it is only 262K tokens. So, it's a bit on the smaller side compared to now models that have over like 1 million tokens in window. But, it's an open model, so it's okay. Okay, so let's go to the Hugging Face webpage for this model, and I'm going to link down into the chat as well. So, here is the open-source part of this model. The model actual weights were provided into the Hugging Face repository, and you can see here that the model size, at least according to Hugging Face, is 1.13 trillion parameters. Uh here in the bottom chart or the model the model summary table itself, you can see that the authors are claiming that this model is uh 1 trillion parameters. I'm not really sure, probably somewhere slightly above 1. something trillion parameters. But, what we can see here is that first I'm going to compare this to Qwen Qwen 2.5. So, you can see that essentially we don't have any changes here. So, it is probably safe to assume that this is just a further training checkpoint or a checkpoint with more training data compared to Qwen Qwen 2.5. So, we have the total parameters, the activated parameters or active parameters are 32 billion. So, it is like a very big model. And then we have also the context length presented here, the same number that we have seen within the blog post, and the vocabulary size is relatively large, not the largest that they have seen, but still a large one, 160K. Also, we have number of experts, which is relatively low compared to the amount of total parameters, 384. But, this is the architecture that they're using here. And one thing that I was pretty surprised about was that the parameters of the vision encoder are only 400 million. So, we have 1 trillion parameter model, and the visual encoder is only 400 million. I'm not really sure if other architectures are having the same ratio between the model parameters or the LLM part of the model and the vision encoder having such small amount of parameters compared to the larger LLM. Uh here we can also see the summary of what this model is tuned for. Significant improvement on complex end-to-end coding tasks. So, again, probably mostly optimized model for coding and agentic coding, especially for Rust, Go, Python, DevOps, and performance optimization. Okay. Coding-driven design, capable of transforming simple prompts and visual inputs into production-ready interfaces and lightweight full-stack workflows generating structured layouts, interactive elements, and rich animations with deliberate aesthetic precision. Okay, I would be very interested in trying this out. I am just want to try and see whether or not this model is actually going to be able to perform and create some nicely designed webpages for us. So, we're going to try that. Also, I wanted to show you something that is let's say a bit better compared to what I have seen from Minimax 2.7, but still something that the more it looks like that more and more of the open model providers are going for this tactic. And here we have a modified MIT license for this model. So, this is not a straight MIT license, unfortunately. And I'm going to pass this license to Qwen Qwen 2.6 and see what it thinks about this license and what e why would they have actually modified it. And here at the end you can see part of the modification or pretty much the same modification. Our only modification part is that if the software is used for any of your commercial products or services that have more than 100 million monthly active users or more than 20 million USD in monthly revenue, you shall prominently display Qwen Qwen 2.6 on the user interface of such product or service. So, basically very large corporations are not going to be able to use this model without advertising what the model is under the hood. And the rest of it seems to be like that it is like MIT license. Okay, so the authors have also put their tweets here on Twitter, and one of the prompts that I have seen with this model was this. Generate ultra-realistic water in 3GS with a boat on it and occasional waves hitting it, real physics. And I'm going to show you the results that is linked here for this model. So, this is the result that we got for this simulation. And we can change the wave intensity. Okay, the follow camera. Okay, it has normalized. The night camera is like a bit too dark. But, overall, look at that simulation. It is pretty nice. Yeah, it looks it looks pretty good. The actual simulation with the light. There is even a cloud, I believe. Okay, the we do have some clouds. Okay, nice. Okay, so these are some of the highlights, let's say, of this model. And I wanted to show you the actual interface that we are going to work with for this Qwen model. Uh this is on Qwen.com. And here you can log in for with your account and try the model on your own. And the important thing here is that the authors are giving us the ability to use Qwen Qwen 2.6 instant, which is going to be probably thinking turned off. Then we have Qwen Qwen 2.6 thinking, deep thinking for complex questions. Then we have an agent for research, slides, websites, docs, and sheets. Okay, we might try this one as well. And they have this agent swarm. World scale search, long-form writing, and batch tasks. This one is very interesting. I haven't read about it yet. So, uh here on the left you can see that they're also having these products like agent swarm, Qwen Qwen, Qwen Code. I haven't tried like any of these, but probably with the release of Qwen Qwen 2.6, these are going to be much more pushed by the authors. Okay. So, the first thing that I'm going to try here for this model, and I'm going to start with the simple stuff. I'm going to start, of course, with the Bulgarian fridge test. Okay, I'm just going to paste an image into the UI. And I'm going to be using the thinking for this model enabled. You're an expert chef. Look at the ingredients inside my fridge. Let me make this even a bigger for you guys. Look at the ingredients inside my fridge. List the main ingredients. Give me a step-by-step recipe for a quick delicious meal. Uh keep it practical and easy. Okay. And I'm going to send this. I'm not really sure how much of a limit do we have into this free version of the model. Yeah, I let I let let it cook. And one important thing here compared to what we have within closed models is that we can look at the thinking itself. Uh most of the the closed models these days are actually summarizing their thinking. So, they're not showing you the actual thinking or their reasoning steps under the hood. They are just giving you the summary uh because they're afraid that others might use that information as training data. Uh but here I hope that this is the actual output of the thinking process for the model. So, the first thing that we can see here is that the throughput is like not that great. Uh this looks like maybe like 20 to Sorry, maybe 30 to 40 tokens per second. Uh other than that, I am seeing that probably the structure of the thinking is nicely formatted. As you can see, it is using a lot of bullet points, but the thinking itself is like very Let's say detailed. Okay, let's see what it has come up with. The main ingredients I can see Bulgarian yogurt, several white cups of Okay. Watch pictures on the top shelf. Block of white brined cheese like sirene or feta on the middle shelf. Okay. Okay, so it has first enumerating what it's seeing that we have into the fridge itself. Quick Bulgarian style sauteed vegetables with yogurt sauce. Okay. So, this is something different that than what I have seen into previous models. We have seen like a lot of just sauces provided by those models as a recipes, but here we're seeing vegetables Bulgarian style sauteed vegetables. Uh one red bell pepper, tomato, red onion, garlic. Okay. Okay. Tablespoon of oil, yogurt. And a step-by-step introduction of how to create this. Total time 15 minutes. And very Bulgarian in spirit. Priyaten appetit. Okay, that was a nice touch. Kimi is back to reclaim open source crown. Uh yeah, I I really hope so, actually. I'm not really sure if you guys have like used any of the previous models, but I'm under the impression that these models especially Kimi K 2.5 was the secret backup model for most of the people that were using Codex or cold code when their limits are getting like hit. So, after their usage is like fueled in many and many like a lot of the people are essentially going to Kimi K 2.5 and they are working with it on their code bases. Okay, so the next one is going to be our very own web shop. This one is pretty much done very nicely. Uh let me show you the picture itself. Okay, this is like a screenshot from I believe something like uh maybe something like uh cart or shopping software, maybe. An image from such an UI. And what I have seen is that models such as Gemma 4 and Quant 3.6 even running on five or four quantized models were able to reproduce this very nicely. And I'm going to just expect that this model is going to do the same or even better in terms of reproduction into the HTML that is going to write for us. You're an expert front-end designer. I'm giving you a screenshot of a modern UI component. Recreate this exact UI using HTML and Tailwind CSS. Okay. And again, we have a lot of thinking. So, one important thing in the thinking part here is that the model is uh like describing pretty much the whole HTML into the thinking step. Okay, so it is doing some review etc. So, uh yeah, this is probably a bit too much like an overthinking, if you will. So, I hope that this is not going to take like a lot of time. Uh so, it made a correction. Let Let me check out the thinking process cuz I'm really interested they have like how do they do that? Wait, I need to look more carefully at the layout. The review section appears to be below the image on the left side, not spanning the full width. Looking at the screenshot again. Okay, so it might be doing something like interleave thinking. Actually, looking more carefully at the screenshot, it seems like image left, details right, reviews left under the image. Okay. So, it has made a mental map of the layout on two rows. And then it says actually no. Looking at the screenshot again, the description section and the two buttons are on the right side below the buy button. Okay, so it is adjusting on the fly. It seems like it. A lot of HTML is written. Then it is talking about Tailwind classes, as you can see here. A logo shop in bold purple. All right. So, talking about the ratings. Nice. And then reconsidering again. Yeah, it is it is looking like it is going like, you know, a lot of thinking in order to produce something. Okay, it is now providing each individual SVG icon. Okay, great. Uh let's get just a sample of this. And do like online SVG viewer. Do we have something? Okay, so this is the truck icon. So, it seems like seems like that it is going to do it like pretty nicely. Okay, so this is the truck SVG. Uh a lot of people are actually interested in like just asking these models to create SVGs for us. Uh this one looks particularly good. If you guys want me to try some SVG coding, uh let me know. So, is the 256 context window getting on the small side for modern agentic coding? Yeah, in my experience, you can do like a lot in 256K context window, especially if you are like context window conscious. And for uh like real-world use cases, you are using most likely uh like between, let's say, 70 to 150K tokens. Of course, as the project is becoming larger and larger, this is not the case and it would probably uh like not going to be enough, especially if you are like using a lot of designs, a lot of skills, a lot of MCPs, etc. So, uh maybe it is not going to be enough in let's say larger code bases. Hello. Hello. Hello. Looks clean. Yeah, I agree. SVG by memory is nuts. Okay. Uh let's see. Okay, so it has now started to look into Yeah, this is the actual HTML that is being given. So, this is the final response. I'm going to leave it as like just to to finish it. You can see that it is also using the SVGs right here. So, probably most of those are the SVGs that we got at from the thinking patterns themself. Okay. Okay, yeah, I think that it is finishing with the reviews. Yeah, the throughput speed is still not like amazing. Uh but overall, it is good enough. Uh it is just a released model, so you might expect this is this is going to get better and better over time, hopefully. Uh have any of you guys checked the model on OpenRouter? And like what's the price of it? I'm going to do that right now. Let me actually do that. And let's see how does it compare to uh let's say Opus or some of the other models. Okay, so we have it on OpenRouter now. Uh let me zoom in the this one. So, we have a roughly a dollar or 0.95 dollars or 95 cents for input. And 4 dollars for a million of output. So, it is five times cheaper compared to Opus 4.7 on the input. And yeah, roughly six times like give give or take six times cheaper on the output. So, compared to Opus, of course, it is like much much cheaper. Uh let's compare it to GLM 5.1. So, the input for GLM is a bit more expensive. Huh, seems cheap compared to Opus. Yeah, pretty much everything compared to Opus is like pretty cheap, but of course Opus is Opus. And the output from GLM 5.1 is actually a bit cheaper. The one for the Yeah, so the the Kimi K2.6 output is a bit more expensive compared to GLM 5.1. Okay. So, I would think that this is like the competitor that you're going to be working with if you are going to be using those types of models. Uh here also the context window is in a similar range. Uh the Minimax models are vastly cheaper. Uh but of course, there are like a what like they have like 200 million 200 billion something like that parameters. So, they might be like a what weaker compared to what you might get from the GLM models. So, these are the current prices for K2.6. Okay, so interesting we have GPT-4 Omni TTS. I haven't used this model. Interesting. Okay, let's check do we have here GPT-5.4? Okay. So, here we have 2.5 dollars for a million input. Okay, so it is roughly two and a half times more expensive, but here we have 1 million tokens. And the output is roughly again like let's say four times more expensive. Okay, so GPT-5.4 seems to be like very competitively priced especially uh compared to Opus 4.7 at least on OpenRouter. So, let's check and see what is the preview of this. Uh how can I like get it? Cuz I don't want this to be Okay. Let me copy the code. Uh let me open this in an HTML so you can guys check it out a bit better. Just make sure that we are giving Kimi here a proper chance for the HTML part. Okay. So, this is the HTML and I'm going to show you again the image and that we have given. So, this is the image. And let me remove my camera for a second so you can guys see it better. Okay. So, this is the HTML on the back. Here you can see uh even the logo seems to be quite well done. Okay. Here this like all shape hasn't been redone, but other than that uh the HTML seems to be quite well done. Uh here the shipping calculator that check out is flipped around compared to to this one. But that's okay. Uh let's check the buttons. Yeah, add to cart, buy now, Shop Pay. And let's check the the reviews. Okay. Okay, I would say that this is like very close to perfect actually. Let's see if anything is clickable. No. But the buttons have like hover effects. Nice. How does the model reasoning compared to GLM 5.1? In my experience, GLM 5.1 is a bit better than Kimi K2.5. Yeah, I'm still testing it, but it seems like that this model likes to think like a lot. So, compared to what I have seen for example with GLM, I think that the thinking there was a bit less. Okay. So, uh we have done this. Let's let's try the next one. The one that I'm like very interested in. Uh let me get the test for that. Of course, we can like refresh the notebook here, but I want to I want to give the Kimi 2.7 this PDF file which is an excerpt from Embarrassingly Simple Self-Distillation Improves Code Generation. This is a paper by Apple and it was released on April 2nd of this year. So, it is a relatively new paper. Uh let's see if we can just uh drag and drop the PDF here. Okay. It is parsing it. I'm not really sure what is the parsing that is happening under the hood. Uh let's see. Okay, so this was able was we were able to actually do this. Uh let me just get the prompt for this one. Cuz I really want to try and see something very specific. Yeah. So, this is the prompt and I'm going to show you uh where this is coming from and why I'm interested in this. So, I want to see if this model is going to be able to understand the PDF file and can it actually go into this chart visually that is or I hope and take a specific value from this graph. And the value that I'm asking for is the T eval 1.2 and T train 0.9. So, T eval 1.2 and 0.9 for T train. So, it should be 55.6. This is going to be the value that I'm expecting from this particular graph that is going to be extracted. Uh let me double check this. Yes, T eval 1.2 and T train 0.9. Okay, so this is like a pretty hard test especially for this model since like uh it is going to get overwhelmed with a lot of context from the text from this paper. There is a very like there are very uh complex tasks here presented uh very complex charts and diagrams. So, I just want to make sure that this is going to extract the correct value. What I have seen from uh for example, Qwen 3.6 and Gemma 4 that was that were running on my local machine, both of those models weren't able to understand this value and was giving me like some values from the table, but unfortunately not the correct ones. So, I'm going to be very interested if this model is going to be better compared to them. So, I have talked a bit about that the visual encoder is only 400 million parameters. Hopefully, this is going to be enough to understand the charts that we are seeing, and I'm going to look into the questions that you guys might have. Uh just a second, please. And let's see uh what do we have so far? Okay, so we have a lot of thinking. Okay, so it seems that it is looking at the table actually cuz most of those Okay. Uh this is the incorrect number so far. Uh but it seems like it is looking at very similar numbers. Okay, so it might actually be on the right path. Uh but let's see. It is uh going through the cells, uh which is good. Okay, so how the So, every China AI are free and without limit. Yeah, unfortunately, this one is like only free if you are able to run it, but I'm not able to run it and pretty much like this model is like very heavy. Okay, so it is uh saying that the T eval value of uh 1.2 in train of 0.9 is 56. uh 8, uh which unfortunately is incorrect. So, again, it is this value right here. Uh let me zoom that in for you guys so you can uh see it. Yeah, I have uh probably scrolled a bit too much. So, T eval 1.2 and T train 0.9 is 55.6, while the model it states that it is 57.8. Uh let's see, where do we have like 56.8? I can't like see close this value. Okay, so at least from this one shot with the PDF, uh we're not able to get the correct result. Well, uh that's a bit disappointing, but yeah. I'm going to try with some of the other models. I'm really interested. I might try this test with Opus 4.7 and uh Gemini 3.1 Pro since uh Gemini models are pretty much the best on uh document understanding. Okay, so uh if you guys want to me to check some like uh some interesting thing with this model, do let me know into the chat. Otherwise, I am just trying to uh like do one SVG uh just to try it test it. Uh create an SVG. Okay, so uh somebody's saying 0.6 and 1.5. Okay, so uh probably those were the values for the train and the eval uh that the Yeah, that the model was referring for. Uh create SVG on SVG with a penguin running through the power lifting. If I can write, it would be awesome. Uh create an SVG with a penguin running through the power lifting like a rock and and being ready to deadlift. Let's Let's make it a bit easier. It should be smooth and cyber punk y. Let's say. HTML only. Let's see. This is going to be like either an amazing SVG or something like completely wacky. Okay. After this, I'm going to uh I'm going to leave this now running, and I'm going to show you guys what is the next test that I want to run with this model. Uh let me open the project. Uh let's leave this running. So, this is the AI project starter or template that I'm uh building or uh I I what I was building into the previous live stream. Uh we were using Open Code and Qwen 3.6 uh running locally within llama.cpp, and I was able to create this uh template or this starter of a template. And within this template, we have this PRD file or the project specification. And this is like the overview of the whole project. I want to give the PRD or this file to the uh Qwen 2.6 model and just to make sure that it is understanding it and uh tell me what is good, what is bad about it. And I would hope that it is going to be also able to create a sample or a mock-up HTML, CSS, and JavaScript uh like front end for this particular project. So, we're going to test the front end capabilities of this model with something that is a bit, let's say, much more realistic. And I'm going to be very interested in understanding how well it is going to perform. Uh and after that, I'm going to uh or before that, I'm going to ask what it thinks about this particular project or this particular specification that we have. And I'm going to ask it to create like a list of pros and cons and possible improvements that we can make in order to make this like a 9.5 out of 10 at least starter template uh for you guys. So, I'm expecting that this model is going to perform quite well on such a task since it is optimized for coding capabilities. Uh so, let's see what is the current thinking process. Okay, so we already have an HTML. So, we have a background gradient. Penguin body, belly. Okay, what do you guys think? Do you think that this is going to be something that like is going to be like pretty cool or maybe a complete fail? I'm really interested to see it. Okay, it is uh drawing the power rack. Safety bars. Okay. The barbell. Wow, okay. The comments are pretty good. Let's see what the overall design is going to be. Uh Uh why don't every AI web UI have a context window indicator, bro? Yeah, I know. It would be uh it would have been so much easier to know how much tokens we are eating, especially with uh the thinking the thinking that is happening for this model. So, uh on the next test, I'm going to actually start a new chat uh just in case that uh this context is getting uh pretty much filled up. Uh we have like a lot of HTML and a lot of thinking, a lot of SVGs so far. I What I'm What I'm seeing so far is that the model is like it reminds me a lot of the thoroughness of uh Qwen Qwen models uh or Opus models. Um maybe not so much of 4.7. I'm not really sure what your uh guys' opinion of it is. Okay, so Okay, this is the SVG. Okay. So, not really sure if this is a penguin. >> [snorts] >> But yeah, it seems like at least it's not like completely distorted. I mean, like overall, I would say it is like let's say cool. Yeah, it is cool actually. Okay, nice. Yeah. For such a wacky, let's say, prompt, uh can we can we actually test this with a with another model? Like Do you guys have like What would happen if I give this to What other model can we try that is both powerful and let's say somewhat free? Let's try. Let me try this. I'm going to run this uh same prompt with Gemini 3.1 and uh just look at the result. Uh and I'm going to compare it in a second. Uh but let's continue with a new chat here. I'm going to leave the other one at the backside. And as I have already promised you, I'm going to get the PRD file. Uh let me zoom in this for you guys. Uh this is available on the Google uh the GitHub repository. I'm going to link it into the chat if you want uh guys to check out the starter template. Uh ChatGPT, yeah, I can throw it into ChatGPT as well. But I unfortunately I only have like the the free version. Uh let's see what will happen if I open it here. So, I'm not really sure what I'm not really sure what model I'm using under the hood. Okay, so it has started. Of course, probably not a fair comparison since I really don't know which version of ChatGPT this is like the free version. Okay. Okay. Okay. So, uh at least compared to the free version, this is like uh much worse. Let's go to this one. Yeah, this is ChatGPT, at least the free version. And this is the Kimi K 2.6. Okay. Uh Yeah, but unfortunately I will have to log in like in order to choose the model. Yeah, and I don't really have like a paid account. Okay. So, uh this is it for this one. And uh when the Gemini 3.1 Pro is ready, I'm going to show you that as well. Uh so, back to the No, it looks like a Pokémon card. Yeah. Do you have Opus? How does Opus do with the penguin? Uh okay, good points. Uh let me check. Uh let me check the Opus 4.7. Uh since you guys are so interested into the penguin test, which I am as well, uh let me just get the Opus output uh for this one. Okay. Let me get the prompt here. I'm going to run this with Opus 4.7 just to check and see. And I'm going to show you. Oh, okay. You might like this. So, this is from Gemini 3.1 Pro. Okay. So, it's a bit more cyberpunky as you can see. Okay, it has some like parts here. All right. So, what do you guys think like this simple version or the one that we got here? Uh okay. Uh we can do like G 15.1. Uh let's see what is their chat Z AI. Okay, G 1. Okay, so if you can run this without the login, uh let's check it out. I don't think Kimi has sub uh so subscription pricing. I think that they have like uh Kimi code or something like that. Kimi code. Uh so, this is the CLI. And okay. It So, I think that for their quote code alternative, they have this pricing. Uh like I'm going to share that link into the chat. I'm uh in no way affiliated with that, uh but you guys can check it out. Okay. Yeah, yeah, exactly. Kimi has a subscription, but for code. Yeah, I think that this is the subscription that is uh being discussed here. So, you can check it out. Uh not really sure how like what are the limits, how much uh you can use it, etc. And they have pretty much pricing that is competitive with Quad Code or Codex, I guess. All right. So, uh I'm going to close the AI Studio from Google. This is the original one. And uh since we're wasting uh waiting waiting for a G 15.1, I'm going to start the chat with the PRD that we have. I'm going to just paste it here. Okay, so it is adding it as a text file. And my prompt is going to be uh very simple. What do you think? Pros and cons? How to make it 9.5 out of 10? Okay, so I mean I pay for the $10 Minimax subs and I can't no one use the inference that subs gives. Okay, so you mean that uh you're getting like a lot of inference for the money? So, uh you're essentially given a lot of tokens for the 10 bucks. Okay, this is still thinking. Okay, so we can see that actually G 15.1, at least on this prompt, is taking also a lot of time to think things through. As you can see, the thinking here is like very large. Interesting. Okay. Uh and like have you guys like tried Minimax 2.7? I have uh like I I have been gifted like a 2-month subscription. And uh I'm also eager to try it out. Of course, I would expect that the inference is going to be uh much faster compared to what we're seeing here since it is like a much smaller model. Uh but I'm really interested in like working with the model. Uh what is like your experience so far with the Minimax 2.7? Like do you like it? Do you like uh have you been using it with like open code or other harness? Uh yeah, it's insane how much inference that sub gives takes a lot of hand-holding, but yeah. Also, you mean that you have to essentially baby sit it a lot in order to do the the correct things, I would assume. Okay. Also, here on this particular prompt that we got for uh the PRD, it says that it is a strong 8.5 out of 10 starter template. The scope discipline is excellent. The stack choices are modern and opinionated without being exotic. And the TDD mandate is generally differentiating signal of quality. Okay, it respects the starter constraint while leaving clear extension hooks. Here's my structured critique and concrete roadmap for the improvement essentially. I really like it. I use it for 90% of my coding to be honest. Oh, it's that good. Wow, interesting like and what do you like what do you code like what are like your main like back-end front-end like applications or maybe something a bit different. Like what is the domain? Okay, so if it is that good we can definitely have a run with it. I wonder if you'd get more success with the PDF parsing using the Doc Link MCP. Yeah, it is it is actually possible. I'm just like looking into uh what I'm mostly interested in is like getting the PDF then from there creating images from each page and allowing the model to essentially look through each of the image pages and understand what it's looking for. So this is probably the easiest workflow to work with and on the other hand very practical. Probably yeah, but would that not defeat the purpose of the test itself? Okay, so you're responding to the Doc Link MCP uh comment. Yeah, yeah, I would agree that the test is like to just give it a PDF like handle it in some way. Okay, web apps. I just make a dedicated media player for Minimax to generate music and stuff thinking of up in to the 20 bucks plan I would assume. So I can use their image services too. Okay, very interesting. So definitely we have some competition for let's say the larger or the American web apps with those subscriptions. This is great. Okay, thank you for letting me know. So here are the highest severity issues. Can I like unzoom this a bit so you can guys like have a better look at it. So you can see that the overall representation and style of the model again is very similar to what you might see into quote or GPT 5.4. The overall aesthetic and formatting of the text is I would say pretty nice and easy easy to read. So it says that we have we don't have any any course specified and we don't have any context window guard. And it says that like we have we can have documents that are over 100k characters. Okay, valid valid points. Okay. Fix the corner run experience, add a back-end config and document course. Okay. Add a context budget. Okay, interesting. Harden the streaming contract. Okay, okay, okay, minimal observability. Okay, I would I would say that these are very valid points especially for understanding the PRD that we have and now for the final test I want this model to write a complete HTML JavaScript and CSS dashboard for the front-end of the app. So I have essentially given it the complete uh specification or PRD of the app and I just want to see how well it is going to design it using the understanding of the functional parts of the apps. So here note that I'm not like I'm using any uh let's say additional wording around what I want to see as a design. Uh I just want to give like a very white instructions. I want to support dark theme be minimal and have very easy to read and understand typography. And hacky art without too much bold course. Okay, let's try to go into that. And let's see what we are going to get as an output. So I would assume that this is going to be Oh wow, this is still thinking. So I have told you guys before that I think that Geom 5.1 is not such a large thinker compared to what we're seeing from Kimi Kimi K 2.6 but uh this SVG prompt has proven me wrong. Nice. Wow. I haven't seen like such a thinking trace. I mean haven't seen this such a thinking trace like uh since the times that I have tried Coin 3.5. Maybe I have tried like a very small coin there and the thinking like went completely ballistic and it was stuck in a loop. Maybe a resource issue. Let's see the SVG. Yeah. Yeah, I'm definitely going to wait for the SVG with so much thinking. I would definitely want to see the end result of that. So it is discussing the X shape of the body. This remind me of the cute yeah. One of the first thinking models from Coin. This was like like over the top. Over thinker. So you mean this one? Wow, it's like it's like so old now. This model came around like 2024 I believe. Yeah. So it was one of the first thinking models that have Yeah, 2024. That have came out from Coin. I think that this came after Deep Seek, right? Yeah, Deep Seek R1 even the bigger one. Yeah. The good old times of thinking models. Those were the days. Yeah, correct. Uh you you did any back-end tests or only front-end to now? Well, I hope that I'm going to be doing back-end tests with within the open code harness once the open router is getting a bit better since the throughput right now is like let's say not great. So I'm mostly doing UI tests, document understanding and images. So mostly front-end so far and actually comparing some results with SVG generation from other models and pretty much everybody's eagerly waiting for the response from a Geom 5.1. And okay. Here we are pretty much into the thinking parts of the both models. Here we can see that we are getting some state definition from Kimi K 2.6. I haven't really seen what is the training cut off training data cut off point for the model. Let's check like how old the data for this model is. Uh do they have like a paper here? Uh do they have a paper for Kimi K 2.6? Maybe not. Paper for No, this is the old paper. And it's for February 3rd. Do they share like cut off some training cut off? Okay. Let's check the table of contents. >> Okay, so they have actually built the 2.5 on a checkpoint from Kimi K2. So, I would again assume that 2.6 is another step on top of 2.5. And the visual ingestion layer, if you will. So, I would assume that the training data is now relatively old. Mhm. Okay. Uh most of the of the new models that I'm seeing have uh this becoming somewhat of an issue. Since we have a lot of, let's say, explosion within the open-source coding libraries. And uh coding tasks, for example, LangChain, Next.js, uh React, pretty much everything that is surrounding the AI field, or whatever it is, a front-end, back-end. Uh if you want Docker, Lama CPP, uh pretty much all of those models are trained now with data that is like, let's say, 6 months old, an year old. And most of the new like changes to the APIs, uh the new tutorials that are available, etc., are not going into the training data. Uh which is somewhat of an a problem, uh since you are essentially recommended old approaches for those libraries. And you have to fill in your context window with additional API information uh just to get the model to give you something that is more relevant and using newer libraries. Okay, so we are starting to see the actual UI for our PRD. Let's check uh to see what we are Okay. So, the Geom 5.1 have essentially stopped. Okay. So, it just hang there. So, unfortunately, the Geom model is not giving us a PNG. Okay, so now it is generating directly an HTML file for some reason. Which is great. Okay, so it might be working. We're going to wait for that. And here on the front with Kimi K2.6, we are already receiving the HTML dashboard. Maybe I should make like uh something like a command center for these models, running them into in parallel with different tweaks while we are testing stuff. Okay, those websites they showed on the demo looked pretty promising with the cloth physics and animations. Yeah, I would uh I am agree 100% with that. And the the overall aesthetic of the web pages they uh were showing on the landing page or the blog post for Kimi K2.6 uh look pretty amazing. Uh if you guys find like any of their prompts they were using for those, uh do let me know. I want to play around with some changes and see actually how well using correct prompts those models are able to create uh [snorts] like front-end designs. Uh Geom 5.1 or Kimi K2.6 or Minimax 2.7 or Gemma 4 or Gwen 3.6 or ChatGPT or something like that. Well, uh in this case, it looks like that uh Kimi [snorts] K2.6 is performing quite well. I haven't done like a complete evaluation against Geom 5.1. And I haven't even play around with uh Minimax 2.7. Uh next on my list is the Minimax 2.7. And I'm really interested in trying them out into uh let's say a more real-world coding environment. Uh thank you for showing. I'm off to test it out myself. Okay, good luck. Uh let us know into the comments, please, if you find anything interesting. Uh I would choose Quad Metus wall. Yeah, probably most of us would choose that if we had access to that. Not really sure if Quad Metus is optimized for front-end design, but I don't know. It's probably optimized for everything. Uh so, there is a question. Okay. Uh just a second and I'm going to answer. I just want to just run play on this. Okay, so this is the result for Geom 5.1. Uh you can see it here. Well, I would say that definitely Kimi K2.6 has given us like a better response. What do you guys think? Yeah. It has some glow to it. It has more more details on the rack. Of course, the penguin is like somewhat crooked. Maybe the deadlift was too much for it. Not really sure. With animations, yeah. With animations, that's correct. With the glowing animations. Okay. So, let's check now the preview of the Wow, okay. From the PRD. Uh let me guys do you a solid actually, and get this into an HTML so we can like check it out. And I have also promised you that we're going to check the penguin prompt with Opus 4.7. Uh just a second. Uh this is for the AI dashboard AI projects.html. I just want to see the final result of Kimi K2.6. Okay. So, this is the final output of Wow, the PRD. I'm like I'm pretty impressed with that. What do you guys think? So, this is the UI from the PRD of the AI app starter template with the addition of the prompt that I have given which is essentially, let me show you the prompt again. It's a relatively simple. I wrote a complete HTML dashboard for the front end of the app. I want to support dark theme, be minimal, and have very easy to read and understand typography. Must be monochromatic and hacker inspired without too much swash bold colors. So, this is like this looks like very much like Vercel inspired. Uh if you think so, guys, yourself. And I can select some of these mocked documents. So, and click start chat. Uh what do you guys think? Should we make this like the UI for the AI app starter template? Do you guys like it? Here we have the different threads. Okay. Uh this looking quite cool. I mean like this is Wow. Okay. What do you guys think about the UI capability of Kimi K2.6? >> Okay, so it even has like a streaming support. It has code highlight highlighting inside. Okay. Nice. Nice. I really like this. Pretty good. Okay, and I have promised you guys that we're going to check out the penguin from Opus. This is the penguin from Opus 4.7. The SBG. How is it replying inside the HTML? Well, I'm going to look at the code in a second, but I would assume that it has like in lined JavaScript inside for the testing for the test streaming. And it has just a single response. So, this is the result for Opus 4.7 with X height thinking, I believe. Well, this is not a penguin, yeah. Yeah. This is just like something like a robot or something like that. So, I would say that Kimi the the penguin especially from Kimi K 2.6 is a bit like wackier. But yeah, a bit goofier, but it looks like pretty cool. And this one this penguin is like absolutely jacked. All right. Yeah, I I would say that this is a very successful release for Kimi K 2.6 2.7 2.6, yeah. And the overall style of this template I really enjoyed it. I mean, like this is cool. Like maybe a bit more color. But overall and the the overall aesthetic and quality of the fonts, etc. seems to be looking pretty good. At least for a starter template that we're going to be building. Okay, so I'll try to answer some questions if you guys have those. Also, please like and share and subscribe to the channel. I would be very like thankful to you guys. Also, please join the free Discord channel that we have. So, I'm going to paste a link to that if you want to join there. We're talking about Open AI and other models. Inside, let me see what were the previous questions. I'll try to answer. So when I work with you, you use mostly Open Code as coding agent. Have you tried leaked code code Open Code from Git Wow? No, so I mostly or for my day job, I use primarily Cloud Code with Opus 4.7 currently. And for pretty much all of the open models that we are working with, I primarily have tried Open Code and used mostly Open Code since seems at least to me that this is a very nicely default setup coding harness. And I have also tried Pi agent. Pi agent, I of course I really like it as well, but there it seems that I have to do a lot more configuration in order to get it to the same feature set that Open Code is providing by default. And I honestly don't have like whole day to play around with different agentic harnesses. Again, I play or use mostly for my day job Cloud Code. And from there probably I'm more experienced at the second place with Open Code. I mean, not official code, but the leaked one. I haven't even run the code code leaked version. I have only run code code with for example with Ollama. Just launched the model. For example, I think I have tried Gemma 4 and Qwen 3.5, I believe. Or as a local setup. But unfortunately, it doesn't look like that those models are enough to be competitive with of course code code, especially for architectural designs and deep backend work for example. Most of those models are still lacking. And especially if you have like a much larger project as well, those models are not good enough. I really hope that Kimi K 2.6 and models such as GLM 5.1 and Minimax 2.7 are going to be able to support most of the coding tasks that you guys can have. Okay. I hope that this is answering the question. You going to try the physics animation now or something else is planned? Okay, no, probably we can actually Yeah, we can actually finish with the physics animation. I just want to to try this. Maybe as an ending prompt. Let's try it and see what we're going to get while I'm trying to answer questions. So, I'm going to get pretty much the same prompt with a boat on it and occasional waves hitting it hard real physics. I want to be able to move the boat and have enemies trying to kill my boat. Let's make it a bit more interesting. Probably this is going to get a bit longer to respond. Liked and shared, but please engage to technical discussion with people here. Okay, I'll try to do that more. Or we'll cover in next stream. Yeah, in the next one I would hope to Well, so we have started with the response already. In the next stream, I hope that I'm going to be working a bit more on the coding site with a proper coding agentic harness, not just the web UI that you guys are seeing here. Yes, please and answer regarding coding agents. I hope that I have answered uh for the coding agent. Thanks for visiting questions. No worries. I mean, Open Code unlimited. Open Code unlimited. Java 4 is good. Yes, code code Opus yet unbeatable to this point. GLM is close enough. Okay. I think that even the original version of this demo was interactive. So, it was able to free form the the camera around. GLM is not close in a million years. It's horrible. Uh Yeah, I haven't I haven't really tried the GLM model. So, like so much compared to for example the Gemma 4 and the Qwen 3.6 models. So, I can't really comment on that. But if any of you have guys like tried the Opus 4.7 model for let's say a couple of days, how what do you think compared to Opus 4.6? Do you think that 4.7 is like a such a huge improvement as the benchmarks are suggesting? Maybe it's just me, but in some cases I'm a bit >> [snorts] >> like skeptical to see the largest improvements. Code code GLM. Okay. So, you can Yeah, you can essentially run the code code harness with the open models. Open model, is this better than Qwen 3.6? So, I would say that this the the thing that I'm seeing from Kimi K 2.6 so far it looks like that this is a comparatively very much better model compared to quant 3.6. Uh Kimi K 2.6 is actually great. No, Opus 4.6 is better. Yeah. And this is something strange to me as well. Like it seems to me that uh cool Opus 4.6 was like a better model. Or at least it was like let's say cheaper and faster. Uh of course a lot of people are discussing the uh the options of having this model being further quantized the 4.6. And maybe from there we were seeing much higher inference speed. But unfortunately people are speculating that the model at the end was a bit let's say dimmer dimmer down. It's one 1 trillion more parameters than quant quant 3.6. I use Gemini high for my normal coding and when something is broken I tell quote Opus to fix it occasionally. Okay. Uh so there's a question is this small enough in to run in consumer GPU? Unfortunately the model that we are looking at today Kimi K 2.6 is like very very large and even people with let's say uh 10K 20K 30K machines are might not be able to run it. Yeah. So unfortunately this is like a very large model. Okay. So it seems like that we're getting the output of the model. Uh let's check it out. I just I just 2 terabyte of VRAM. Okay, yes. That would be enough. A Kimi releases fat models. Yeah. It looks like it. It looks like that this is the case. Okay. So uh let's try the boat experiment. Okay. Let's see if it runs at all. Okay. Wow. What? So I'm moving this. So it seems like I'm moving to the back. Okay. But overall So if I use I don't think that I can move this forward. Uh let's try to give it an update. Yeah. It seems like But overall the the quality is like pretty amazing for an open model. Like wow. Okay. I'm going to try to do something. Or maybe fix part of it. But other than that uh let me refresh this. Okay. So I can move left and right or rotate. But not front and back. As you can see I'm getting hammered. Let's see. Okay. So I have queued somebody. Very interesting game. Okay. Uh this is very funny. Okay. Uh let's see if the update is going to give us something. Yeah. It seems like that the the cannon is actually pushing me backwards. That's exactly correct. Uh but also the boat is able to move left and right. Not front and back. Okay. So we have sunk four enemies. Uh recoil mechanics here. Probably. Probably recoil mechanics. But still I would love to be able to And it says that the mouse is for aiming. But maybe maybe it's not. Okay. So it is making the final adjustments if you will. So we're having like existential conversations into the chat. It seems like it with this physics represented here with our little boat game. Uh let's check the comments. Okay. Need something like M3 Ultra with 2 512 unified memory. Yeah. This would be like a beast machine. Uh do you guys think if M5 is going to have an ultra version? I mean it's hard to judge quality without real projects. I made a Mamba RNN. Okay. Test. Yeah. I would completely agree with that. And this is probably why I am mostly trying to do let's say a multi-step project with some of the open models just to judge how well they handle increasing complexity. Cuz it's very easy for most of those models to be given like a simple prompt. They have been able to generate let's say 50K tokens in some code HTML CSS etc. And then you might have some back and forth and then create this like very cool looking demo. But I don't care so much about that. It is really important at least for me to be able to have these models perform let's say classification tasks tool calling correctly create some analysis over documents or create code that is aligned with some of the specifications that I'm giving an example into open code or other like harnesses. So this is what I'm more interested in. Okay. Guys who asked about the boat open code I'm answering yes the limits and safety was removed from leaked code code. You can see repo it works now with all models. Uh can you guys like link the repository for this project? Uh that you guys are talking about. Uh if I this one is on GitHub I I would assume. Okay. Nope. Uh so this is probably about the M5 Ultra. No, probably not because of the RAM apocalypse that they ever stopped selling uh 512 gigabytes of unified memory max to the only 256 gigabytes are available. Uh but be aware Anthropic releases legal threats to those who copy it. And try to take down all copies from GitHub. Yeah. If you feel that you don't want to paste it here that's okay. Uh let me get the new version and try it out. And just see what we are getting here. Let's say boat improved if this is improved. Hello. Let's hope it is. Okay. Here is the improved version. Okay. So now we are moving. As you can guys see. All right. Okay. Front back. Taking some damage. Uh but the recoil mechanics are still there. All right, but the boat is moving. We also have speed in knots here on the bottom. All right. That's enough. Okay, so we can see that the model is improving based on feedback. I'm going to check the comments. I'd like to build an AMD cursor with framework desktop starting with two connected via HDMI and scale up if they're good. Have any of you guys been using AMD like GPUs or CPUs in order to do to run LLMs on top of that? Have you been like successful with that and like have you been able to get some good throughput? The Korean guy had his boat rewrite it while it was on plane. Now this is boat racing. >> [laughter] >> Okay, you can drift LLM. Yeah. Yeah. Okay. I would say that this model is like a lot of fun. I was particularly impressed with the UI capability. Yes, I looked up and all tested all versions including the Korean guy version. His version was rewritten to Python then to Rust. So you're talking about the export from quad code. I mean like the leak from quad code and then the rewrite that was done in Python and then it was done in Rust, correct? Macs are I would say economically better for local LLMs. Yeah. I currently think that pretty much nothing is beating let's say an M4 or M5 machine with roughly 48 gigabytes of unified memory. You can run the Gemma models, the Gemma 4 models and the quant 3.6 models on a laptop you can carry around. So this is looking pretty good for the Macs. Okay, yes the leaked code rewrite, correct. Okay, okay, thank you for confirming that. Maybe someday I would try to run it and compare it to the original quad code. One might say. Okay. So I would say that we are ready with the test so far. I would encourage you guys to go and subscribe to MLXpert Pro. If you guys want to become better AI engineers, there is a complete AI engineering academy that very heavily focuses on private and local LLMs. I'm trying to get my hands on M1 Max with 64 gigabytes of unified memory. Yeah, this would be also a very good machine especially for the money. And you'll probably be able to run a lot of models for that on top of it. Okay guys, please in the comments below after this stream is complete, let me know what you want to see next. Do you want to see Minimax 2.7 or another test with Kimiko 2.6 in open code environment. And let's put these models to a test and compare them and see whether or not these are better compared to or are they close to Opus 4.7 and ChatGPT 4.4. And this would be it for this stream. BTW, do you need a video editor? I'm looking for a client. Well, currently no, I'm not looking for a video editor at the moment. If I like get too much overwhelmed with the videos and the streams, I might be able to uh ask for some help, but if you guys are interested, just join the Discord and follow the essentially the journey. Okay guys, this is going to be it for this stream. Thank you so much for watching. I'll try to stream pretty much tomorrow at the same starting time. So if you want to join again, again please like, share and subscribe this video. And I'll see you