Claude Opus 4.6 in 8 mins!

Key Takeaways

Anthropic just launched Opus 4.6 and I'm not here to tell you that it is mind-blowing or gamecher rather LLM updates are becoming very boring at this particular point. Mundane updates lot of benchmarks. So here are three things that you should know about OPUS 4.6. The first thing this is the first OPUS model that comes with 1 million context window. Later on the video I'll explain why that matters. Second agent teams. This is a model that can power a swarm of agents and that is a great thing to do. And third, this model is really good at running long context task. So where it requires to run longer and also it requires to understand large amount of things. These three things really make OPUS 4.6 a developer dream. But what doesn't make it a developer dream the cost. As you know that anthropic has got three classes of model. Haiku which is like the cheapest but also the fastest and the dumbest model. Second, you have got Sonnet, which is kind of the balance in between. And there is a rumor that Sonnet 5 is going to come very soon. And then third, their flagship and the smartest model, which is also the most expensive and also the slowest model. So, Opus 4.5 was already a really great model for a lot of tasks, agentic tasks particularly and programming tasks. And now we have an upgrade to their smartest model, which is Opus 4.6. And Opus 4.6 six comes with crushing benchmarks as usual. First thing is there is a benchmark called GDP val. So the idea of this is like can LLMs do economically important tasks rather than going behind MMLU, MMLU Pro and other these kind of tasks like GSM8K which are like at this particular point kind of saturated. This benchmark evaluates if the LLMs can do knowledge work which is typically what humans do in a lot of offices. And this model OPUS 4.6 has scored 1,600 points. It's like a ranking ELO point which is higher than the second best programming model in this world which is GBT 5.2. The next important benchmark where the model is doing good is computer use. So this is a benchmark called OS world. So idea here is that can you use LMS to control a computer and this model has scored 72.7%. There is not a huge amount of improvement in terms of bench verified. Seems like they're not optimizing for programming tasks to be solved. Exactly. But rather they're making it easier for the models to understand the problem, plan the problem, solve the programming in itself, not just the final solution. So I kind of understand the SWE bench verified has not improved a lot. The next benchmark that really caught my eye is AR a2. This is a benchmark which is like honestly like I tried to solve ARGA2, I couldn't solve it. And this model has scored 68.8% 8% which is kind of amazing given that you know this is an LM that is trying to solve some generic related tasks. So this is another benchmark that kind of impressed me. There are other agentic benchmarks where the model has done very well. Agentic search, agentic tool usage. These are benchmarks where the model has done well. But now moving forward the most important thing because the model supports 1 million context window. It is very important for us to understand why it matters and how the model is doing. See if you have ever used LLMs for programming. One important problem that you would face is while you're chatting with the LLM, the context rot might happen. What is a context rot? When the LLM is about to hit the context window, the LLM performance would degrade. That's one thing. Second thing, if you have got a code base that is large enough, then the LLM would not do a good job of understanding everything in place. For example, it might ignore some part of code. It might mess up some part of code. So whether you are retrieving data from a codebase or you're trying to give the LLM a codebase and then making it understand and then code it long context always matters a lot. I personally have always recommended Gemini wherever long context is required because Gemini has one of the best in-class long context model but now the fact that Opus 4.6 has got 1 million context window. The anthropic team has released two benchmarks. One is a long context retrieval benchmark where you retrieve from a longer context. The second one is a long context reasoning. So do you hit context rot? And if you see this so opus 4.5 at a 256,000 context window has scored 93% match rate mean match rate average match rate and with uh 4.6 opus 4.6 with 1 million context window it has scored 76%. And you can see the previous version the 1 million context window sonnet 4.5 had only 18% match rate. So it's a huge improvement for such a long context. And even for 256,000 context window this is huge. And even in the long context reasoning opus 4.6 has scored much better than the previous 1 million tokens benchmark. So if you see here the previous context window 1 million context window benchmark was sonnet 4.5 with bfs has scored 25.6 while opus 4.6 6 has scored 38.7. So I think overall this is a great model for long context reasoning. Along with these context window improvements, Anthropic has done two more things that will make this model much better for long context task. One is called adaptive thinking. Adaptive thinking makes the model make a decision whether it should use thinking or not. So why is it important? It is important because if the model cannot decide to make the thinking or non-thinking, it can always enable the thinking mode and then consume a lot of tokens. So for you to save the context window and also have a faster inference, adaptive thinking could be extremely helpful. There are like four modes. So one is like low, medium, high and also there is a max mode and then there is something called a context compaction. So context compaction basically means that whenever you have got a longunning conversations the model will use something called context compaction to automatically summarize and synthesize everything that you have discussed before so that that context is not lost at the same time the newer conversations are stitched coherent with the past conversations. This is again like very helpful to use clot code or some other tool to have a longer context and longer duration task. I think these two things make the longer context 1 million context window much more compelling. Finally, the most important thing that I'm really really excited about is this LLM companies trying to do agent swarm or what anthropic is here calling as agent teams. So rather than having one agent do a particular task, can we fire up like 10 or 20 agents, ask them to work towards a common goal? And that is exactly what Anthropic is trying to do with agent teams. It's a feature that is going to be part of cloud code. Let me know in the comment section if you want me to make a separate video about that. But for now, they have shown a demo where in this case they asked agent teams to build a C compiler from scratch. Like literally from scratch. All they had to do was just make a C compiler. Like as much as it is a joke, they just told it to make a C compiler and the model actually in two weeks the model managed to build a C compiler that executed that ran on a Linux kernel. It's not like you know local host joke but the compiler actually ran and it worked. So it's very interesting. I'll link the blog post in the YouTube description. You can go try it out. So basically it fired up a bunch of agents and using agent teams. There was like a not start that it has to do and then it managed to successfully complete it. And while agent teams are working there are like different roles for different models, different agents and everything finally comes together and then finally gives you the output. I'm really really excited to try out agent teams. See Opus 4.6 Six is a great model. There is no question about it. And the model is extremely extremely good for agentic task, coding tasks. So anything that you want to do, the model can obviously do it for you within the limits of what the LLM can do today. But the question is, is it your everyday model? Obviously not because the model is expensive. It is one of the most expensive models in the market right now. So there is a premium pricing that gets applied whenever you exceed 200,000 tokens. So, as much as it is good for longer context, but whenever you exceed the 200,000 tokens in context window, then it is going to be more expensive. And you can see the pricing here is the $10 per million input tokens and $37 per million output token. Like I said, this is very expensive model. You may not need this every time. But if do not mind spending this much money or you've got tasks where this much money can provide a good return on investment, then this is the model that you should use. You can go use it on cursor. You can go use it in windsurf. You can go use it in cloud code and claude anthropic ecosystem. Let me know what you feel about this model. See you in another video.