Github Models: Is this the end of Hugging Face Models?

Key Takeaways

one place to access many different models that is GitHub models having their own playground to test these models you are able to integrate that with your own application with their SDK you can use this in code space directly without any authentication is this an replacement to hugging face what are the models available in this how to use these models and integrate that in your own application and what is the pricing that's exactly what we're going to see today let's get started hi everyone I'm really excited to show you about GitHub models a new generation of AI Engineers building on GitHub enabling more than 100 million developers to become AI engineers and build with industry-leading AI models you can test and compare different models in the playground you can quickly prototype in code spaces I'm going to take you through step by step about the models playground how you can integrate this with your own application and also how you can use that in code space but before that I regularly create videos in regards to Artificial Intelligence on my YouTube channel so do subscribe and click the Bell icon to stay tuned make sure you click the like button so this video can be helpful for many others like you first we are going to see how you can use the playground so this is how the models Marketplace looks like and you can see the different models available here so when we see the models list you can see you got jamb instr different coair models llama 3.1 models myal models open AI models and 53 models so you can use any of these models quickly compare these models and prototype so let me take llama 3.1 and if I want to quickly ask a question I can just ask a question like this and it's going to respond accordingly you can change the maximum number of tokens the temperature the top p and other parameters here change this and see what works best in regard to your own use case there's easy option to integrate that with your own application just click get started there you got python implementation and it's using Azure AI inference python package you can also integrate that with cop JavaScript and rest for cop you got the instruction here for JavaScript you got the instruction here and for rest you got the instruction here rest is nothing but curl commands which means you can run these commands on your terminal so you just need to copy this as the first step you need to create a personal access token so go to personal access token there you can click generate new token and then generate new token here now I can set the expire date here how long you want to give access to this API token give a name to the APA token public repo and then generate token now copy the token and you need to export it like this in your terminal so coming to my terminal export GitHub token like this and then click enter now to test this you can directly use the curl command so we copied that ear earlier and we can just paste it here we can see we are using LL 3.1 and then click enter now we got the answer so the question we asked is what is the capital of frons and the capital of frons is Paris that's the response so this is a quick way to test using curl command now how you can integrate this with your application so there are different ways that you can integrate this with your own application just as a basic usage you can integrate tools that is function calling or tool call with this you are able to stream the response and also you are able to do multi-turn so I'm going to take you through all of this let's create a file called appp and inside the file import OS then from open AI import open AI then getting the GitHub token that will be our API key next setting the endpoint so this endpoint is same for all the models next setting the model name now we are using open AIS SDK with the base URL and APA key now we are using client chat completions create function to ask a question to the large language model so the question we we are going to ask is what is the capital of front we are providing the model name temperature max tokens and top P now after this we just printing the response that's it so it's just the basic open a SDK now I'm going to run this code in your terminal pip install open AI to install the open a package then python AB p and then click enter now we got the response the capital of France's Paris based on the question which we asked here next tool call or function calling so it's same as before but in this we are adding a parameter to call tools generally if you ask a lar language model what is the next flight from Seattle to Miami it's going to say I don't know because it doesn't know the real time data when we ask the same question it's going to use function calling method or tool call and call the function get the live information so this function is your own tool or own software and that is going to respond the accurate value and then it is sent to the large language model to respond like this in this way you are able to make a large language model respond in real time and you are able to integrate that with any of your tools or softwares same thing is happening here so we are assigning the tool to the large language model so that it can use the tool and answer the question flight from Seattle to Miami here we need to provide the tool definition so you can see the tool definition here get flight info is the function from and to they are required and get flight info function is here so this could be your own application or own software which is able to return the realtime flight information so once we call this function it'll trigger the tool call then we are going to pause the tool call then run the tool append The Returned result and send it to the large language model again to get the final response now I'm going to run this code in your terminal python tools. p and then click enter now you can see first is calling the tool that is function calling the origin City destination city then the function return this response the flight date time and the flight details then that is sent to the L language model and here is the final response the next flight from Seattle to Miami is operated by Delta Airlines and here are more information next we're going to see about streaming and multi-turn through code space so if you come to the playground click get started there you got run code space click on that and it'll bring up the code space like this so here you can see all the code here samples python code JavaScript and C command so in regards to python I'm going to try with myal AI multi-turn so instead of Mr small I'm going to change that to Mr Large and every SDK is pre-installed in this code space and you don't need to authenticate everything is taken care of behind the scenes so you can see the multi-turn which means first we are providing an example question and example answer then we are asking what about Spain that means based on the previous context we going to ask the capital of Spain now I'm going to run this code so in your terminal just type python samples python mral Ai and then multi-turn click enter and here is the response the capital of Spain is mrid now let's see how you can stream the response so it's same as before but we are using chat stream function and we are looping through the output so now I'm going to run this code so this is completely running in the browser python samples python Mr Ai and streaming so you can just copy the code and then test it for yourself you can just copy the code and test it for yourself you can you can see the response got generated so we've completed basic tools streaming and multi- turn next we are going to see about the pricing to use the playground there's a rate limit the number of requests per minute number of requests per day tokens per request and concurrent requests but when you go to production we need a Azure AI account and you need to get the personal token from Azure AI the pricing is listed in Azure AI so here you can see the pricing for Mr Lodge cair that's like 015 per th000 tokens then for open AI you can see the pricing here it's listed in Asher AI portal next finally we are going to see about how this can be run using CLI so here's the CLI command GH models run and then provide the model name and the question you want to ask probably this is in beta version so I was not able to run that locally on my computer similarly you are able to combine the GH models run with other commands let's say if you want to summarize the commit history that is the last 10 commit history you should be able to type the command like this and get a summary this is really good the final question is that is this a replacement to hugging face not yet because most of the models which we see here are close SCE and we don't have any open models weights available in this yet and it's all linking back to Azure AI overall this is a simple way to use Azure AI models using GitHub personal token apart from that it's similar to any other sdq provider I'm going to create more videos similar to this so stay tuned I hope you like this video do like share and subscribe and thanks for watching