This VIBECODING LLM Runs LOCALLY! 🤯

Key Takeaways

You can probably start vibe coding locally without internet using the 7 billion parameter model. This model is so good that the 32 billion parameter version of this model scores 37.2% on swb bench. That means it managed to solve 37% of kdub issues. And you're going to see some of the mind-blowing examples that I got from this model is only with the 7 billion parameter model that is running very perfectly on my Mac which is a 36GB M3 max machine. So I'm going to show you about this model f first of all then I'm going to show you how to use this model locally and then start with your coding related question and answer. This model is from a company called All Hands and this model is called Open Hands. The company name is All Hands. The model name is Open Hands and I'll link the model link in the YouTube description below the like button. Also, if you appreciate the video, share it with your friends and like the button so you can see more updates from the channel. This is a smaller model. The way they have designed this model is quite interesting. They've taken the quen coded 2.5 instruct 32 billion parameter model. They took the base model of it and they used some principles of reinforcement learning that were described in S swim. It's a paper and they trained this model particularly to be good on coding related task and the result is that we have got 128,000 context window model that is particularly really really good with programming related task. Like for comparison, if you were to compare this with Deepseek V3, Deepseek R1, even DeepSeek V3, the latest version, you can see that even those models scored from somewhere between 32% to 38% on SWE. And this model is particularly very good with that kind of a task. And to put it in a perspective, you can see the 27 billion parameter GMA 3 model, which is only 10%. The score of it is only 10% on SWE bench while this model has scored 37%. Which is like a really good number on a paper and it's not like your typical MMLU or the normal benchmarks that you would typically see. This is a really good benchmark and on this benchmark this model is scored 37% which is like quite promising. The other thing that I wanted to highlight is like whenever you see something like 128,000 context window doesn't necessarily mean you're going to use 128,000 context window locally cuz see these models require a lot of compute for you to hit the max of context window. You need lot more compute than what you your mate uh your machine might give you. So just keep that in mind. I'm not going to run it on 128,000. I don't even think that my machine can offer 128,000 context window for this particular model. But if somebody were to like deploy this model and offer this model or you have got a VM running on cloud and then you can do it then probably you should be able to scale it up to 128K and also 128,000 context window doesn't mean for the entire 128,000 context window this model is going to be topnotch. There's going to be ups and downs. So just a note that I wanted to share uh with you all. So now if you see the model is available on hugging face. Uh so I'm going to show you how you can use the model. So the description I gave you is for the 32 billion parameter model. But what we are going to use is the toned down version which is a 7 billion parameter model which was trained with the same principles as we just discussed above. So you can copy the model name. I'm going to use it on LM studio. So all you have to do is go to the discover one and then just paste the model name. You can see the model from Barski which was like 11 days ago. The model is 4.4GB. You can download the model. The version that I'm downloading is Q4 the 4bit quantized model but depending upon the kind of compute that you have got the minimum requirement that is required you can download any different version quantized version. So if you have got a lesser compute then download a smaller size model if you have got more compute then download a bigger size model even like up to B bf6 which is like the B float 16. Once you download the model you can just go here and then say start use in chat. So in my case, the model is already loaded in my current session. So it says use in a new chat. But if you were to completely like brand new loading the model, it would say load the model. So after I've loaded the model, there are a couple of examples that I thought like this model is like really good and rock solid. The first one is give me a single HTML 5based landing page that's beautifully designed. I mean beautifully designed is quite subjective in this particular case for a company called Little Coder Labs. So what I'm trying to understand this model is first of all can it give me an HTML 5 script that works without any error. Second does it understand the company little go to lab and it can create copy for that. And third obviously I want a decent enough HTML 5 page. So in a very short time you can see that I've clocked 46 tokens per second. It gave me a,436 tokens and in this case I've got the HTML 5 page. I can just copy this and I can go back to my browser. I've got the HTML 5 playground. If I paste it, you can see this is what it gives me. Is it beautiful? No, it's definitely not beautiful. But is it a full working HTML 5 page that kind of has the basic skeleton of it? I would say obviously it has got for a 7 billion parameter model, I would strongly appreciate that this is good. It's it's managed to pick up the right uh emoticons or flat icons. It's got a little copy here. It's got the hero page here which is I think there is some CSS issue. So I think this is a really good first zeroshot landing page for such a model. But can we push the model further? That's my question with this model. And you would see that I have managed to push the model further. And then this model is again like a little bit dicey here and there. Okay. So I went and asked the model create a p5 js animation to see a square rotating and a ball bouncing inside. So what I'm imagining is like the square should rotate or the ball should bounce inside. You'll be surprised what it gave me. Okay. So it gave me this. The square is rotating and the ball is is it bouncing. I mean like I don't know was I not specific enough like the ball should be bouncing inside the box. It was so funny to see this thing. I mean I can't say that this is absolutely rubbish. Maybe I can say it is rubbish. it still managed to give me a working code and that was pretty interesting to see that it managed to give me a working code in p5.js. So yeah, just just maybe try with a different prompt. The next thing that I tried to ask is I said okay give me surprise me with a nice Pygame Python code. So I took the Python code here. So it gave me simple code. It says it creates moving rectangle. So I'm going to go to my Visual Studio code. I'm going to paste it, save it, and then I'm going to just run this. Once I run this. Yeah. So, you can see here I've got the keyboard control. So, I can go right, down, go up. I don't know. I mean, it's not much of a surprise, but at least like it's a de decent enough Python UI where I've got like pi game and then I can just like go play with that. Uh, I could have asked for snake game. I didn't ask for a snake game, but this is what it gave me. Next, I thought, okay, why shouldn't I ask it actual Stack Overflow questions that were posted recently? And that gave me lot of interesting ideas. Okay. So I've got this um question here. So you can see I've got three stack overflow question that was like posted today. And the first question is about you know I've got a data frame mostly like pandas related question because I'm a data scientist so I understand some bit of it. Other things I wouldn't understand necessarily. For example I've got the data frame in one column. So like this and then I wanted three columns. So like this is a typical exercise that you would do on Excel. So you you split it by comma and then you would get it and then you've got like uh the boolean value. So it's like wide long to wide that kind of a concept. So it says okay you can do this with strr.get dummies and it gives me the right answer. So this is stack overflow answer and I went ahead and then asked the same question. So I said convert the string to booleans and I gave this example. It did not work. It gave me an error. So I said okay is there a pandas method? So I got another code. I'm going to copy the code and then paste it here. I just wanted like something very simple and easy and as you can see here we have got a simple and easy solution just after one follow-up. I mean sometimes these models like it's not just this one even with cursor it just blindly tries to follow you what you ask not like suggesting you something else different. So it's always good to have that a slight bit of domain knowledge and then you ask okay do I do I have a pandas method and of course it it kind of gave me an answer that uh gives exactly what we wanted sum one sum two sum three and then you've got the boolean values here which works perfectly fine so next moving on to the next question so you can see that I've got another question very similar concept so you have got a data frame and then somebody's trying to do something with group by and then the first one answer the accepted answer on stack overflow says Why do you want to use group by dofilter? You don't need that. So I'm going to ask the same question again. And you can see I've got the same question. And uh it doesn't tell me exactly that I don't need group by. So you can see here it says group by.filter is used for groups which is exactly the first line on stack workflow. This question is like just posted 23 hours ago. And it gives me a different solution. So I'm going to copy the solution and go here. So this is the ideal output. I'm going to go to my Google collab notebook and I'm going to paste it and then see if it can. Yep, it uh gave me it gave me a b c. So this is ideally returning a a b c. Uh so there is a bit of let's say duplication and I asked like is there any other way you can do it? And then it um it gives me one more way. So let me copy paste this and yeah we got ABC and uh I think this is the right answer. Let's hope it's the right answer. Yeah, ABC 111. So this is another question directly from Stack Overflow and it managed to do a particularly good job. And the third one I got from pandas like stack overflow again is slightly related to regular expression rejects reg I don't know how do you say it and also pandas data frame. Once again, I pasted the question. I got an answer back. So, the question is about extracting entities from like that given text. We not doing any not named entity recognition, no language models, just simply Python. And I'm going to just copy the code and paste it here and then expect it to just run. Um, I think the first most important thing like literally it should run without throwing an error. Like you if you remember like the first time it showed me an error and anytime you have a complicated code it necessarily ends up in some kind of an indentation error or comma here and there or an open or closed pattern. This is logical error. So you always opt for a smaller code like easier code and here in this particular case this is exactly what that person wanted. So you have got XYZ XYZ ABC DPAC XYZ ABC DPAC it missed XYZ here but still it managed to do the job partially for us. I could have like prompted more. Overall I felt that this model is particularly good with programming related task. You can couple this model either through no lama model or like through lm studio endpoint. You can use it with uh plugins like uh continue on visual studio code and then you have got your local coding assistant without sending data to any server at all. Let me know what you feel about this model. I felt it is good but if you do the testing and then let me know for your testing if this is a good vibe coding model. See you in another video. Happy prompting.