Kimi K2 0905 for Agents

Key Takeaways

Okay, so earlier today the Moonshot AI team from China have released Kimmy K2 0905. So this is basically an update to the Kimmy K2 model. Now I didn't make a video about the Kimmy K2 model cuz I was traveling at the time at the ICML conference, but my tests for it were really impressive and I was actually surprised at how good that model was with some of the Gentic skills and stuff like that. So what I thought now is I'll just make a quick video about this because the key things that they seem to have enhanced with this update seem to be things related to both coding but also some of the agentic tool calling etc. They've also extended the context window out to 256,000 tokens. And it seems that they've actually fine-tuned it to work better with some of the agent systems, the coding agent systems that are out there like chord code, rue code, etc. So, let's jump in and have a look at it. So, unlike a lot of the companies out there, we don't have a huge blog or anything for this. Actually, if you come and look at their site, you'll see that literally you've just got this small post that when you click it takes you to the hugging face page in here. So, not even a sort of big blog post or anything like that. They've basically just dropped the weights and dropped links for people to actually use this. So, we can see that the architecture hasn't changed at all from the Kimmy K2. This is basically just a weights update. We've still got a mixture of experts model with a trillion parameters but 32 billion active parameters which is the key thing in here in hanging face. The key features that they're listing are enhanced agentic coding intelligence which seems to me so far from my testing is that it's basically better at working with clawed code and also better at doing sort of tool calling and making use of that longer context window that's in there. So, we can see that the model architecture is pretty much the same as it was before. I'm still kind of surprised that their vocabulary is like 160,000. And perhaps that does limit it for certain languages with tokenization, etc. They've published a bunch of benchmarks in here. Honestly, I don't care too much about the benchmarks against other models, per se. I'm more interested in like the difference between the version that came out in July versus the September version. And we can see that it's got a nice bump on a lot of these different things. And my guess is that yes, certainly they're doing some hill climbing with these benchmarks, but even in my own sort of Gentic evaluations, this model does seem to be doing better at some of the tool calling, etc. So interesting. In here, they've basically got a bunch of things of how you can set this up if you do want to run it yourself. I don't think many people are going to run this themselves, even with a quantized version. It's just so big it doesn't really make a lot of sense for people to run it on their own computer. But it is interesting to see that they've got an anthropic compatible API here showing how they're actually computing that if you're using the enthropic API that they've got. Another thing like I mentioned earlier that this is really focusing on and even Kimmy 2 was actually very good at were these agentic tool calling capabilities. And they've got some stuff in here about setting it up and being able to do multiple calls as you go through this. Now, looking at their own API, we can see that the price is pretty damn good, right? We're looking at something that's basically 60 cents in, $2.50 out per million tokens. If you happen to c something, then the input is only a quarter of the 60 cents. So, you're looking at 15 cents. And we can see that they've got the K2 turbo mode in there, which looks like that has already been updated to the latest 0905 weights as well. Now, Moonshot are not the only people that are serving this model. We can see it's already up on places like base 10 on together AI and even Grock with a Q is basically serving this. And you see that they're claiming roughly 500 tokens per second with the full 256k context, which is pretty awesome if you wanted to use this with clawed code or with some sort of other coding agent for just getting responses back really quickly. One of the challenges always with claude code is it can often be very slow and you have to go off and get a coffee while it's actually running to do some stuff for you. All right, if you want to try just chatting to the model itself, you can basically just come into kimmy.com and try the model out yourself there. You can just loging in with a Google login, etc. Let's jump into some code and see how well this is actually doing by using the OpenAI endpoint for a bunch of different agentic frameworks. Okay, so here you can see that I'm going to be using Open Router to access the model. Now, the version that I'm going to be using is actually the version from Moonshot AI, but you can come in here and get the Grock version if you want to change that. And you see, basically, that's just going to be a different name for the model going through. All right. So, to get that set up, we're going to use the OpenAI SDK in here. And then I'm just going to bring in my Open Router T. One of the reasons I like Open Router is that it lets me test a bunch of different providers to see are they actually providing the same thing. You often find that the way people set up the model means that they're not actually providing the same thing. This is a way that I can compare the Moonshot AI version to Gro to other people in here. All right, I'm going to then use that OpenAI endpoint and just do a call to it. So you can see here I've got very simple call what is the meaning of life go through gives me a whole bunch of text back for that obviously one of the things that they talk about as being better in this version of the model is its ability to generate front-end code right stuff that actually looks good so I basically threw in generate some HTML that will work in a collab HTML cell for an ad actually that should just be 1D but anyway I think it's got it for Kimmy K2 to from Moonshot AI. And you can see that it generates out a whole bunch of different HTML. Now, if we take that HTML and copy it into the cell here, there's quite a decent amount of HTML and CSS code in there. I'm pretty amazed that the response that it actually came out. So, this ran perfectly first time. Just basically copied what came out above. And sure enough, it's created this ad for Kimmy K2 in here. It's got a whole bunch of different stuff for popups, for all this kind of stuff. Now, obviously, it's not actually a live website where it's going to click and take me to any other pages. It's just generated one page, but for me, this is pretty amazing that it's been able to do something that quality. And you can try playing around with it as well. So this is the way that I like to test some of these models through the open router endpoint. It just makes things very quick in here. So for me the most interesting thing is to see how this is going to work with different agentic frameworks and with being able to extract data out etc. So this is using just their standard sort of tool call. I'm trying to extract out a name and an email. A lot of the models will get it right and pull out the first one, but they won't, you know, actually go through and realize that there were multiple names and emails in there. So, we can see this has done a really good job of pulling out both of the names and both of the emails. So, it's gone through and done one and then it's gone on to do the second one. All right. If we try it with Pantic AI, we can set this up with the open router provider with Pardantic AI. We can see this one has no problems running the agent and returning what we expect. This one is going to be true. This one's going to be false. That's what we want it out. If we give it something like a search tool, it's able then to basically come up with a a keyword, go off, do the search, process the search back, give us something useful in a nice markdown format here. The same is the case for lang chain. It's working really nicely. and for lang graph where we can basically set up a graph do a similar kind of thing in in langraph and it's you know working fine so often I will find that a lot of the models will work well with one of the agentic frameworks but they don't really transfer over that well the second kind of test that I run these through for aentic things is basically giving it lots of tools to see how well it can handle it so even the top models you know tend to start falling down when you've got lots of tools. So, this is basically a finance agent with a whole bunch of different tools in here. And I'm using the ADK framework and you can see that this Yahoo Finance tools, we've got, you know, all these different tools in there. And we're basically getting the model to go through and use these tools and select out the appropriate tool for this. So, the agent I'm using is very simple. It's got a very simple, you know, system prompt. There's not a lot going on here except that it's got a lot of tools. And then what I do to check it is for each call, can it actually call out the right tool here. So I'm actually printing out what the call is, what the function is it's using, and it should match up here. So we can see get stock info. Okay, that one's pretty simple. Get historical prices. It's used the tool properly. It's given us back a nice result. Calculate moving averages. get financial statements. You can see here it's gone through and gotten multiple calls on that function. When we wanted to compare multiple stocks, it's going to basically call them out getting dividend information, calculating volatility. And generally what I do is start off with prompts that are really simple and then gradually make the prompts get harder over time. So you can see for each of these it's basically calling the right function, you know, for these. And for some of the tools, actually the API's changed. As you can see here, it's supposed to be calling get earnings calendar. It is calling get earnings calendar. But it looks like there's an error because part of the the API has been deprecated. Then we have a whole bunch of functions where okay, in in this case, it's you know calculating support resistance etc. going right through. Now I don't think it was perfect going through. I think it some of them although it's looking like it's gotten most of these right, some of them will be wrong. So, okay, we can see here that it supposed to be getting technical sentiment and it's gone for a function that's very similar to that in here. Then, basically, I just go through and test new prompts getting gradually getting harder and checking it out as as I go along. So, while I have this, you know, in scripts that I can run, I like to actually often just walk through something like this in a collab with a new model so I can get a sense of like, okay, what's the output like? How is it using the tool? you know, looking at all those different things. So, you can see there using the Google ADK framework, it's been able to handle 21 different tools, not perfectly, but I think it's only gotten one wrong out of all of these. So, for me, this is a really good sign for a model that when it can basically work out what tools to call and work out how to basically repeat calls or make multiple tool calls, we know that, okay, we've got something good there. So overall, I would say this is definitely a model that you should check out if you're using Claude code. I haven't shown any of that here, but you can certainly set that up to work with this and basically get something that's cheaper. And then depending on which provider you're using, you can get a much faster model as well, which just allows you to get a lot more done. I certainly need to play with it in quad code a lot more before I would say that it's necessarily better than the the sonnet model, but I do think for a large number of things just being quicker and cheaper often can be far more important if you're doing things like front-end code, like Nex.js, like stuff that the model knows pretty well already. So anyway, check it out. new version of Kimmy K20905 from Moonshot AI. Definitely a really nice model. Definitely one of the leading models that we're seeing coming out of China and competing very strongly with the proprietary models that are out there. Anyway, as always, if you found the video useful, please click like and subscribe and I will talk to you in the next video. Bye for now.