DeepSeek Coder V2 - Quick Look!

Key Takeaways

the most underrated llm for coding is a deep seat coder V2 I don't know why not a lot of people talk about this model primarily probably it is from China but I believe this is one of the best models that are available out there that has a decent open license and also really good for coding I mean you don't have to believe what I say I'm going to show you a couple of benchmarks and we are also going to do a couple of test to start with this is a model from a group called Deep seek AI if I remember correctly this is probably somewhere related to the Alibaba Group which is from China so this model the latest version is deep seek coder V2 this model performs a better than gp4 turbo in some tasks for example like human evil mbbp plus which is a python test and in fact like there are other benchmarks where this model performs really well but again like this is a benchmark that has been given by the model creators themselves but if you want to take an independent Benchmark for example there is AER which is an llm based coding framework so AER helps you in creating the entire llm based coding projects so if you see aers llm leaderboard uh the second best model in terms of the code or project completion with 75.2% completion is deep C coder V2 the first model is CLA 3.5 Sonet the second model is deepy coder V2 this is in in fact slightly above the GPT 40 model and in fact it is a lot more above than gp4 model and it is not just this this particular Benchmark you can see the next Benchmark that they've got which is for a code refactoring even there it is on par almost with uh the gp4 turbo and gp4 preview model and if you go to the popular Benchmark that we have got is LMS arena in terms of in this is at the fifth position with uh 16 position Improvement so deep C coder V2 model is again scoring really good in coding why is this model really good that is the first question that we should be asking at this point I hope I made you believe or establish the fact and believe that this is a good model this is a good model primarily because this is not one model it is a mixture of experts model so this is a mixture of experts model just like the kind current popular Trend we have got mixl we have got probably gp4 I'm not sure then we have got like other models which are also mixture of experts model so this model comes in four different flavors so we have got the deepy coder the light model deeps coder light base and instruct deeps coder the whole full model with base and instruct in the light model you have got 16 billion parameters totally the mixture of export setup and 2.4 billion parameter models are active for every single token for the full model 236 billion parameter model and the active model is 21 billion parameter the most important thing or the most exciting thing is youve got 128,000 tokens active the context window I think this is extremely important for coding related task and you can see that an aers Benchmark if you see aers Benchmark one of the reason that this model is doing good because this model is not just doing code diff edit it is doing the code whole edit and uh it has got like 100% the percentage of correct edit format so this is one of the secret Source probably like I believe context window is one of the important aspects why this model is doing extremely well otherwise I guess uh they have used extremely really good uh training tokens that has made it possible for this model to have a really good um score in terms of all the benchmarks like human evil mbbp math GSM K so in all these benchmarks if you see DC coder V2 has been scoring really insanely good thewe bench out of box is where this model is not as good as let's say um gbd4 turbo Gemini 1.5 Pro uh This Is The Benchmark which uses the models to solve GitHub issues the actual GitHub issues in fact this is a benchmark which made divin quite popular D is still not there so anyways we'll leave that out uh deep seek also provides an endpoint uh if you want to use APA and that is probably the cheapest Endo that I've ever seen the input token is .14 the output is for million tokens the output is 28 so you can see this chart like ridiculously insanely cheap cheaper than Claude 3 hu so I'm not sure how they're managing to do it but you never know what kind of Technology they've got but if you want to run the model locally you can run the model locally I'm not going to run this locally but I'm going to jump directly into to their platform which is called cod. Deep seek.com if you go to their platform you can start using this code so I just went simply asked okay develop a simple HTML CSS JavaScript website that takes user input and displays an appropriate Emoji so the code is really good the only catch here is that the code is basically a set of k switch uh statement so for this command it's going to give you this output so anything that is not part of this this is going to this give this but again this is is a decent code like for example I can go here and also run the HTML and I can say something like angry and it would show me angry emoticon but again if I say anger it is not going to show anything because it's literally keyword based I'm not judging the quality of the application in itself but it does a pretty good job so I can go ahead and then ask it to improve things like for example I can say can you add a logo make the website more beautiful so I can just say that and send it and just like CL 3 Sonet 3.5 Sonet within artifacts you would be able to run this HTML within a Sandbox environment so ideally what we would do we would copy this go to a platform like code pin and then run this but you don't have to do this it is all available for you within one single interface and this at this point this model is available for you to use free I'm not sure about the rate limits but you can use this model within this uh platform again um if you're going to use any sensitive data I would I would suggest like you should probably not do it because this is a hosted platform and again um it it depends like how much you are bothered by that this is a Chinese llm not Chinese llm at this point I'm not very bothered um and again this model comes with open license so if I want to run I can still run it locally so run HTML it it gave me a placeholder logo here um which I can replace it with some other image and I can just go say anger angry and it is going to generate and you can see it is giving me more effect and I H it is giving me zoom and all these things okay that's well and good let me go ahead and ask another question I can click clear the context here so you can go up and then see the context but I'm going to click clear the context and ask a very simple question I'm going to say create a simple gradio application that takes two inputs and then creates a new stable diffusion based image AG using diffusers maybe I shouldn't say diffusers cool so I'm expecting it to understand that it can use a diffusers which it very correctly did it and you can also see that it installs all the right Frameworks in this case diffusers even though it is there Transformers and torch would be required for you to run it on Google collab and then it starts using stable diffusion pipeline which is also correct the only catch here is that it uses stable diffusion 1.4 model which still a lot of people like and uh it also uses if you have GPU um what to do Cuda if you have CPU what to do the code is really good it uses stable uh gradio older class which is interface and I can say it seems you are using gradio interface do you know to use blocks let's see if it can change the code with blocks and um as you can see here it kind of seems like understanding what I just said so the rest of the code St same and this is absolutely a code that would ideally work so the model is being downloaded the model is being sent to the device and then you have got prompt and it uses the blocks and this is how you should ideally write the blocks and you have got all the things that are available for you so it gives you the explanation as well so so far it seems like this is one of the best models from uh my testing as well but I'm going to spend more time explaining this model and what kind of things that you can build with this model let me know in the comment section what do you think about it and also if you have any hypothesis why this model is not being discussed a lot but this is honestly one of the best probably I can say the best open model with open license that is available for us to use in terms of coding model so deep C coder we do check it out see you in another video Happy prompting