Anthropic Sonnet 3.7 - The Thinking Sonnet

Key Takeaways

Okay, so Anthropic has dropped their latest model Claude 3.7 Sonnet. Now, a lot of us were wondering whether this was going to be Claude 4 and people are very excited to see how Claude 40 was going to turn out. It still seems Claude 4 is around the corner, right? This is not that yet, but this model is a a quite substantial boost for coding and for reasoning tasks, etc. So, before I jump in and look at the model, there's something else I wanted to point out that I think is really interesting about the Anthropic story at the moment. Recently, The Information published an article about how Anthropic was raising more money at a new valuation. And in there, they had these projections of Anthropic's earnings for the next few years going right out to 2027. And they're projecting a possible huge increase in the use of their products. Now, clearly one of the things that has been coming to attention is that Anthropic is by far the number one coding model that people are using out there. So, if you look at Cursor, you look at all the different coding tools and stuff like that, people have been using the Anthropic Sonnet 3.5 version 2, which rightly so it seems now many of us were calling 3.6 based on the fact that the new version isn't called 3.7, but it's just fascinating to look at. This is the the area that Anthropic seems to be seeing this huge boom in. So, it's not surprising today to see that the the latest model and new product that they released relate to this sort of coding use. So, the two things that they've released today are the new model, which I'll talk about in depth, the fact that the new model has a reasoning or thinking mode to it as well, and they've also released this Claude code, which you could think about is their first step into taking on things like Cursor, like Bolt, like Lovable, all these sorts of coding tools that are out there. So, the first thing to look at in here is that Claude 3.7 is Anthropic's first release in this sort of reasoning and thinking mode here. And they've basically got this done in two ways where you can come in here and select that you want extended thinking with this model, and that will basically give you more tokens with the sort of thinking style in it. And then in code, you've actually got this idea of where you can turn on the thinking and you can also give it a budget of tokens. So, you can see here that like we would just normally call Claude, now we've got this thinking property in here, which we can enable and then we can assign a budget of the sort of how many tokens do we want of thinking. So, this is really fascinating in that they're allowing you to decide, "Okay, you know what? I want 30,000 tokens of thinking or something like that. Let's see how long this is actually going to be able to go out." Or, as we'll see when we actually look at their thinking mode, how they're actually doing it in parallel. And in fact, we can see that in the blog post that you're actually able to go up to 128,000 tokens of thinking doing this. So, this is a really big sort of trade-off of speed and cost versus quality of answer, which is great that they're giving you the possibility to decide this. Okay, so along with the new model and this new thinking, we of course going to get different benchmarks. And it's really interesting to see that, okay, here they're giving benchmarks of showing just how much better this model is at the SWE-bench, software engineering benchmark, and also showing that they get different results depending on how much of the thinking or how much of their particular way of using the model. And this is something that I talk about in the other blog post where they talk about this extended thinking, etc., is that really you need to rethink benchmarks now because you're not really just doing a benchmark on what is the answer out of the model, you're basically going to need to benchmark what is the answer out given certain amounts of compute in here. And I found this to be really interesting in that blog post of where they're basically talking about this parallel test time compute where they are sampling multiple independent thought processes and then they either go with some kind of majority or consensus voting for this of to work out, "Okay, this is the best one." Or they go for a sort of LLM as a judge strategy where they have a different version of Claude establish these out. Now, this is not going through to the end. This is doing it multiple steps along the way in much more like the paper let's verify step by step, which is quite different than what DeepSeek did for this kind of thing. So, it's going to be really interesting to see how this plays out and will we see the new version of DeepSeek that they actually work out to do something like this rather than the way they did the scoring with GRPO for their RL process out there. I think the simplest sort of takeaway that we can take from this is just look at what they're showing us here that basically giving more tokens gets us better answers. And we saw the exact same kind of thing come out with DeepSeek for the exact same benchmark in here. So, this really shows that at the moment that if you want to have really high-quality thinking out of one of these models, you want to basically get it to generate a lot of tokens in that thinking before it comes to its final answer. Another thing that Anthropic is clearly doubling down on is that this model is aimed at being used for agentic use cases. And mostly that's going to be through agentic tools. And you can see here them showing this new model benchmarked against their old, I'm going to call it the 3.6 model, and OpenAI's 01. Now, recently OpenAI mentioned, I think, that they're not going to release 03 as a sort of API, etc. So, I don't know if we're going to have to wait for GPT-4.5, which is rumored to be perhaps later this week or next week, or we have to wait till GPT-5 comes out in a few months' time to actually see how that's going to compare for these kind of things. But this is certainly interesting to see just how much better this is doing compared to the previous version of Claude that everyone's been using for these kind of things. If you look at the more general benchmarks, I feel like there's not a huge amount of surprise here. It is interesting that they've actually benchmarked against DeepSeek R1 with 32K of extended thinking and they've basically shown their scores for no extended thinking versus the much longer thinking in here. Also interesting to see that on some things like visual reasoning, Grok 3 beta actually is ahead of them. So, that's interesting. The reason why I haven't made a video about that is just that there's no API for us to test it properly, but hopefully that's coming in the next few weeks. So, I'm going to save the Claude code thing for its own video and I think we're going to see that tool get a lot better as they get feedback and iterate through this over the next couple of months, etc. But clearly this model is built for coding and I think we can certainly test it very well already in Cursor. We can test it on some coding tasks to see how it's going to do, etc. Just quickly going through the other blog post that I thought was some really interesting things in here. So, one of the things that I really like and I kind of feel that Anthropic is definitely one of the classiest providers out there at the moment in that they are a lot more transparent than a lot of the other frontier labs in what they're doing and explaining that. And this kind of shows here with them declaring that they're going to make the thought processes visible in here in the raw form. So, you can see here they're basically say, "We've decided to make its thought process visible in raw form." And then they give some reasons why. I think the reasons why here are actually very good. First one being trust of just when people can actually see this, people will actually be able to trust the outputs when you can actually see how it came to that conclusion. I think that sort of removes some of the elements of just having a black box. And I don't think the way that some of the labs are just giving sort of summaries of the thought processes is really a great sort of alternative for this. The second one being alignment. And this is interesting that they also acknowledge something that OpenAI admitted that really you want to be able to try and identify when the thinking and the output are misaligned. And it's interesting to see that they didn't do alignment training on the thinking for this. And while I'm pretty sure that they've done it on the outputs of this, the actual inputs and stuff like that, they haven't done this. The third one, which I think is actually a really valid thing, is actually just interest. And the the fact that it's actually interesting to look at how these things come to their decision and to see that in here. Now, I like that they prepare people by saying that first users may notice that the revealed thinking is more detached and less personal sounding than Claude's default outputs. Meaning that we're actually going to get us a more of a sense of how this thing does its sort of internal, in inverted commas, thinking as it goes through this. And you'll be able to see where it perhaps it's going wrong with its thinking and where it's basically misleading itself or having incorrect thoughts along the way. So, I'd encourage you to come and read this blog post cuz they talk about a number of really interesting things as they go through this whole thing about Claude's extended thinking, etc. All right, let's jump in and have a play with the model, see it how it actually performs, and see how does the extended thinking going, and how does the actual coding go when we ask it to do some real-world tasks. Okay, so I couldn't resist coming in the first thing and trying out the strawberry question. But it is interesting that how it actually does it in a visual way. I found this amusing that it's not just giving you some answer, it's actually showing it with code and in a visual way. And I think this kind of sets the tone for this whole model is just the way that it uses code to basically show you things and with its reasoning. And this was without the reasoning turned on, by the way. Okay, so here I'm going to give it a little bit more of a real-world thing that I was actually interested in recently. So this is basically a blog post from Sam Altman talking about some really interesting ideas about how to be successful, et cetera. And what I wanted to do was get the LLM to extract out the sort of rules and how to turn this into some way of being able to evaluate things. Now the previous model did very well at this. I'm curious to see, okay, how does this come out? And then does it actually write any sort of code in relation to this? The previous model it was able to give out a nice but react checklist and sort of some things around that, which I thought was interesting. The other thing is this is much more of a an analysis of some kind of real-world things, which thinking for quite a while. We're now like 20, 30 seconds into the actual thinking here. Okay, so it got to 32 seconds of thinking. And this has been able to extract out a number of key things. What if we ask it, can you build me a piece of software to actually use to do this? Okay, we can see sure enough it's basically building a piece of software now for the opportunity evaluator. And it does look like it's coming up with a waiting system, which is interesting and much more detailed than what the previous model was able to do. Okay, so it went on for quite a while there. If we look at the code, it's a shame that they don't show you lines of code, but it's written a lot of code in here. And I think this is really reinforcing the whole sort of use of this model and just how much this model has been built around different coding things. So you can see in here we've got new opportunities where we can put in an opportunity. We can change the waiting for it. So we've basically taken these rules out of a blog post, converted it into software now where it's different waiting and stuff like that. Okay, so you can see that I've filled this out. If we save this now, it's because of this where it's giving a score. It's then able to export to a CSV and to I guess compare opportunities if I've got multiple ones. Okay, we can see now we've got multiple ones in here. We can click edit and go back and edit one of the ones and update. Wow, it's like already got code for all of this. I'm not seeing the ability to Ah, okay, compare. Yes, here we can compare the selected ones and we can see, okay, where they lie on the different things in here. This is insane for the amount of that literally I put some some short prompts in here and it's able to come up with something that's detailed and coded up to this. And I think rather than me typing in a bunch of different reasoning examples, let's let's some sort of ideation and code examples in here. Okay, so I've given it a prompt that I want to explain basically how intent and slot filling models work in the past. So let's see if it can come up with something that not only explains this, but does it in a visual manner is what I'm looking for here. Cuz I kind of feel that this is one of the areas that these things really have the ability to do cool things is explain concepts that people don't know in a way often that would be visual that would get people to understand this. So it's writing a massive amount of code for this. Let's see, is it actually going to work? And what are we actually going to get out of this? So we can see that sure enough it understands that intent recognition is one thing, slot filling extracting specific parameters needed to fulfill that intent. All right. So we're into a lot of code, as you can see this going through. It's also interesting now that they've got a publish button in here. So I guess we can publish this for other people to see. Now honestly, with this amount of code, I would expect things to fail. All right, most models they would have some kind of bug in some point and stuff like this. Okay, so how this works, we're going to put in something in here. So let's put in this one and we're going to process. And I guess the intent would be booking a flight. Okay, so sure enough after it generated the first time it didn't work, but all I basically said to it, okay, looks good, but when I press process, nothing is happening. It went off, it did some more thinking and it updated the demo. And let's see, if I pick uh book a flight, it's worked out that this is an intent system, which we would want to get from this. We wanted to get the origin, which actually would be New York, and destination would be London, that's correct, and date and time. It's basically gotten all of those except that its neuro system hasn't been great. Well, let it go on. I think that it is something. Let's look at, okay, the example of the weather. Sure enough, we can see that, okay, it's doing really well at getting the intents and the slots. And to be honest, I spent a bunch of time of thinking how I could show this to people who only knew sort of LLMs how things were done in the past. So this is pretty amazing that it's basically been able to work these out. And I'm definitely going to publish this so that I can show this in my talk. Okay, so another thing I've been looking to explain to people is voice activity detection. So let's try same prompt, and this is not a huge prompt, by the way, how these things have evolved, what sort of goes into this kind of thing, and can they actually put together a demo for this? So it's given us a lot of different ways that this can be done, which is totally correct. I'm curious to see just what the actual demo is going to be like that comes out of this. All right, what you can actually do, cuz I got to say this is pretty exciting if suddenly we can just have really complex topics and stuff explained to us both not just in a text way, but in a visual way. One of the issues I've always found with things like ChatGPT and Claude is that if you want to have a diagram, want to have something like that, it's often very hard to actually get the model to create something. All right, so we've got this demo and we can see that, okay, we can change the threshold on it. This is nuts. This is absolutely nuts. Uh where we've not only got like the energy based, we've got a statistical based way of how it's done, and then we've got an ML way of how it's done with the various detection things. And we've got an explanation. All right, that is also going to go into my presentation for this. This is pretty nuts that this kind of thing now really can explain so much to us in such a visual sort of way. Okay, so now what I want to do is get it to make a landing page, but I kind of want to restrict it. So there was this nice graphic that I saw recently on Twitter, posted I guess by Oliver Kenyon, which just talked about what you want to have in a good homepage landing design and stuff like that. And I thought this was really nice about the sort of do's and don'ts of this. So what I've done is taken that image and dropped it in here. I've asked it, okay, make me code up a landing page for my hydroponic preparation for a nuclear winter site. Make the marketing and copy really strong and show that this can help people get ready for what is coming. I have also asked it to do it in Tailwind. So let's kick it off. I'm using the thinking mode for this. I think it will think probably not too long, but it's probably going to think about both the marketing copy and the actual code in here. And so sure enough it is using Tailwind straight away, so that's a good sign to go. I didn't tell it the name of the business or anything like that. Let's just see what it actually comes up with for something like this. And I'm curious to see like how it actually followed the instructions. I didn't really I should have told it to pay attention maybe more to the image. So let's see what we get out and then I'll come back and maybe give it some more instructions. Okay, it's come back not with great design features, but it's certainly come back with the different things. And the thing that I find interesting in here, and let me let's try publishing it see if it looks any better. Okay, so even after publishing it, it still looks like your CSS is off a bit with this. But it is interesting that it's definitely followed the instructions according to this. So it says the divine part was all the best practices from the guide. So clear benefit focused headlines, which is, if we look back in the guide, we can see is actually one of the key things that they mentioned. And all the other things are actually things that they've mentioned. So I really like this idea of now being able to just take some kind of guide or some kind of thing where we can steal this and have it be able to write out all this code. Now my guess is that actually looking at it, it looks like it's not pulling the actual tailwind in here. So, I might try later on playing around with this. The other cool thing is, don't forget this is the model that's being used in Claude Code to actually drive that. And if we come across to Cursor, we can see that this model's already available in Cursor, both the thinking version and the non-thinking version in here. So, you're going to think that this is definitely going to solidify Anthropic's lead with coding models and building things that help people with coding agents and that kind of thing. Okay, so just to wrap up, if you want to run the Claude Sonnet in code etc., it's quite easy to do. In Colab etc., you can just bring it in. This is the key bit is you want to basically set the number of tokens out and the thinking mode to be enabled and then the number of sort of budget for thinking tokens etc. Once you've basically done that, you can just run it through and you can see your thinking as a separate output and then you're just a standard text output of the answer that you get in here. The final answer coming out in markdown from what I'm seeing in here. You can also see the amount of tokens that it actually used. I'm not 100% sure if this is including all the different thinking tokens or just the ones that we got back out. So, if it generated multiple parallel trees and then filtered them out, I'm not sure if we how that works for getting charged for that at the moment. I But certainly you can get started with this and try it out. Overall, I got to say that the Claude's extended thinking for code, for doing a variety of different tasks is really impressive here. And I do think it's going to change a lot of people who are on the fence about whether this stuff is really going to take off in coding and stuff like that. I think it's that's totally clear now. And I'll probably be making some follow-up videos about Claude Code, about maps using it in Cursor and stuff like that as well. As always, I'd love to hear what you think in the comments. Let me know your own thoughts. What prompts have you tried that have turned out really well? What hasn't worked? Of course, not everything is going to be working at this stage. So, it'd be really interesting to hear from people what they've tried that hasn't worked and what sort of results they're getting there. And as always, if you found the video useful, please click like and subscribe and I will talk to you in the next video. Bye for now.