opencode vs pi agent. simple benchmark with Local LLM.

Key Takeaways

Hi everyone. In this video, we'll do a test with local large language model, so open code versus Pi. These are probably the two most more known project. Uh and the first one it's an alternative to open code. It's mainly an alternative to cloud code. Uh while Pi Mono it's a more minimal terminal interface with MCPs or better skills. And it is also the core of open claw, which is the project that many knows. And this one, if you go to the dependencies, you will find that open claw use Pi in its core. So, which is the point? The point it's to uh to see which one works better with local LLMs. So, um I won't use the TUI because it's difficult to to benchmark. So, I will use the terminal interface to do it. So, let's see the difference between the two terminal terminal interfaces. So, this is the first command. And it's uh P um minus minus model and I will use Qwen 3.5 35B A3B. And yeah, let's see the difference of these two uh terminal interface. And this is the prompt that I will use. So, which Linux kernel do you do I have? So, uh it will use a command on my system to to answer to this question. So, let's try with P. And as you can see, I don't have the clock. Let's put this here. And yeah, this is uh CPU. So, we have the answer in 5 seconds. Let's try again. So, [snorts] start. And as you can see here, I don't see the Pi because it's it's very fast, but it's about 200 megs of RAM usage. And let's do the same test with Open Web UI without the um TUI. So, just normal interface. So, open code the same model and the same answer that need also the command run to to have the reply. So, let's do also this. And start. And as you can see, um open code open Open code use more RAM. It's almost 300 megs. But the main issue isn't the the RAM usage. It's that the context that open code provide isn't just my question. Uh it gives more information uh for the agent in the context. This is why it is slower. And yeah, as you can see, it's the same model, which is Qwen 3.5, and I disabled the uh thinking part. So, I will put it in pause and we will see that it is also working. So, I pause. Let's see when it completes. Okay, we have now the answer after 2 minutes and 40 about the the same answer. And yeah, it's uh much faster with local models. Pi. So, let's try again this question. So, Pi. This is the context, the default context, the skills that aren't loaded. And this is a different model. So, let's change it to this one, which is the one I used before. Which Linux kernel do you do I have? And here you can see actually the commands that it runs to have the reply. So, as you can see, uh if you can use Pi, it is much much faster than open code. Yeah. Let me know in the comments which system do you use. And this is the chart that I made with open code benchmark dashboard. And for me, the best trade-off that I have it's with GPT-OSS 20B. And also with Nemotron, you you can have a good trade-off between correctness and speed. And also with uh Qwen 3.5 35B A3B. There are different quantizations, but uh it's uh it's fine. And yeah, this is my benchmarks. And if you want to try something faster, you can also try uh Yan code 4B, but uh it depends on your use cases. Um yeah, because smaller models with the tool usage could be faster. That's it. See you in another video. Bye.