How to use Hugging Face models on VS Code Copilot

Key Takeaways

All right, that's going to bring us straight into our next segment um with Selena from Hugging Face. >> Hi Olivia, how are you? >> Super excited to be here. Thanks for having me. >> Oh, we're so happy to have you here. Um yeah, I did a quick intro that you're from Hugging Face, but if you could just do an intro of yourself and then what you'll be showing today, that would be >> Yeah. Yeah. Um so I'm Selena. I'm a software engineer at Hugging Face for more than a year now. I'm the co-maintainer of the hugging face SDKs and I also work on our serverless uh inference service called hugging face inference providers uh that I will be talking today um and how to use it uh inside VS code. Uh so quick word on inference providers if you can show my screen. Yeah. Um so quick word on this uh it's basically uh the easiest way to get access to state-of-the-art open weights models via one single uh API and the inference runs on uh worldclass uh inference providers like Siri brass grog fireworks and a bunch of others and you get fast reliable access to pretty much any uh open weight uh model you can think of and because we uh support providers you get flexibility in term of um cost uh speed and availability without being tied um like to a single vendor and uh it's also cost effective in the sense that we don't uh add any uh extra markup on top of uh provider. Now the the part I'm especially excited about uh is that you can use this open weight uh models directly uh inside VSC copilot chat uh with the uh extension called tuginface uh provider for GitHub copilot that you can find on the marketplace. Uh I think you cannot miss it. It's the uh the extension with the cute uh the cutest logo. Uh so yeah uh by the way huge thanks to the VSC code team uh for implementing the tooling uh that make this super easy for us uh to build. It's based on on the bring your own key feature that uh Logan was presenting before and as he was explaining it's basically allows you to contribute your language models um through an API to VSC compiler chat and uh VSC being open source is such a great a great thing for us as well. It helped uh build this kind of uh of uh features. Um so yeah um another thing about the the extension uh we think it's uh very cool uh for us because we constantly release uh new features on our side and uh the extension being outside of VS Code allows us to iterate faster um add new features integrations bug fixes and um all of that without relying on on VS Code releases. Uh so yeah a quick demo setting this up is straightforward. You just need to install the hugging face um provider extension. Once it's installed uh go to copy chat uh click on the model picker then manage models click on add models and then you will find the hugging face in the list. Click on hugging face and then it will ask you for your hugging face uh API key. So this is your hugging face access token and you can find this in your uh hugging face settings. just make sure to um add permissions to reference providers. So I will copy my um token here. Click enter and then you can select any available models uh to use uh inside uh copilot chat. Uh so yeah uh a couple of useful things here uh is that you can actually filter uh models uh by provider uh that are that supports this model. For example, if I want to um show all models that are supported by fireworks, I can just uh look for uh fireworks and uh it will give me all the models that are served to fireworks. Um you can also pick uh either the fastest mode uh which will select uh the fastest provider for that model uh mean the highest support uh for for that model. We have also a cheapest mode which will select the the most cost efficient um provider for that model. So yeah so let's select uh for example the latest deepseat model which should be a deepse v3.2 two uh with the fastest provider and let's select GLM 4.6 six another uh open weight models that has uh great performance for coding and once you uh enable visibility here you will find uh the models here directly uh to use um yeah so let's try um a quick uh demo here um let's build something real actually so uh I have this feature request uh I was supposed to implement today so on on the hugging face hub repo which is the hugging face uh python SDK. So yeah, let's just ask uh GLM to implement uh this this feature directly. >> So just ask um GLM to to implement this by giving the URL directly to uh to the GitHub uh issue. Uh let's add a bit of context. And let's hope I don't get any uh demo effect here. But it should be [laughter] hell. >> Yeah. Uh actually I for this kind of features I really like using local agents um instead of assigning I don't know the issue straight to GitHub uh copilot because it feels much more uh interactive and I get to steal the implementation step by step. uh I can refine the prompt and uh uh I just like watch I just like watch it like uh until it completes uh its work. >> Uh so yeah um I also estimated roughly how much it cost for this kind of session. I mean this kind of request and it roughly around uh 20 to 30 cents uh if you want to like have JLM implement this uh feature and I'm not even uh using the cheapest uh provider. So, it's quite cheap compared to other uh other models. And with the free HuggingFace account, you get some uh free credits to try things out. And then with a pro account, you get uh even more um monthly credits, free credits, and then it's pay as you go. Um after that, um yeah, so I think it's it's going really well for now. It takes like one to two minutes. Um, and as of the extension, the implementation is obviously open source. Uh, so you can just check out uh the code and open issues if you have any uh bug reports or feature requests. Um, we're super uh happy to to get like some feedback on that. >> Yeah, we'll definitely make sure to share that um repo link so people can um contribute to that. Um yeah, I love like there's a couple things I just want to highlight they showed. First, I love the field to filter for cheapest and fastest because I mean like at the end of the day, right, a lot of people don't necessarily know what all these models are, but know what's what's something I can try that's cheap or what's something I try that's going to get my task done fastest. So, I think that that's a really really great user experience. Um, and then to your point about um just this being in its own extension, um Logan touched on this a little bit too. Um, but it just really allows y'all to be able to get the latest updates out to folks as quick as possible rather than to your point waiting for maybe stable release to go out. >> Yeah. Yeah. I actually for the cheapest and fastest uh mode we shipped that a couple of weeks ago after releasing the extension. So I just had like to release another um version uh of of the extension and then uh ship that into uh copilot chat. So it's uh it's very very cool. Um there is a question um on if these models can be handed to the cloud agent um or your local sessions. >> Yeah. Yeah. Yeah. I think I think it can be used to cloud agent. I tried this uh I think a couple of days ago. So it should be working fine also for with cloud agents background agents. Um I'm more of a local agent to be honest. But yeah it can be used as well. we see this kind of um like the other thing I think that's really great to show with the extension too is then it kind of gets all the error handling from y'all as well too. So that way you know it's a model that's supported by hugging face um and that you can make sure that all the responses are being uh uh requested properly um and that you can see kind of that great user experience there. >> Yeah. Yeah. Yeah. Um I think it helps also that we have like um we are open AAI uh compatible which makes it really easy to implement this uh for the VS code compiler chat. So um yeah it's it's um it's it's really a good uh a good thing to have uh this uh this thing of the bring your own key uh feature uh in VS Code and the way it's implemented as well. >> Mhm. Absolutely. >> Cool. >> Let's see. It looks like it's Yeah. All done successfully. Yeah. >> Yeah. I think it did the job. Um Yeah. And and then I can I mean using this I can just iterate ask for test or if I don't like something uh I can just like um do do the job uh locally and then uh push a final version of of it uh after reviewing the the agent work. >> Yeah. I I like I just think it's really cool how it's you still get kind of this native experience used to in chat, but you get to bring all these extra models in. Um and I know hugging faces like that was like one of the uh top ones that people are like, "Oh, when can I put in hugging face models here?" Um so uh that's a really great I'll put out the um link again real quick um so people know where to look for. Um, you can either search in the extension marketplace in VS Code for Hypingface inference provider um for copilot chat I think it's called or we have the link up um right now too for how to get to that extension page. Awesome. All right. Is there anything else that you want to share or not? That hugging face will be um coming in the future. >> Um I think though that that's it. We will uh I think we're communicating about it like every time we have like new features and we we try to uh to improve that and see what next features new features of VS Code and how to integrate that on our side. >> Awesome. Cool. Well, thank you so much Lina for being here. Really appreciate it. Um have a great day.