How to Access Llama 3.1 LLMs

Key Takeaways

this is a very short and quick tutorial to explain how to download and use a llama 3.1 models of course we cannot use the 405 billion parameter model because we need a lot of ram insane amount of RAM is required for this for some context just for local inference if you were to run the 405 billion parameter model with full Precision 16 uh bit full Precision you need 8810 GB model with 8bit Precision you need 405g B and with quantization awq gptq bits and bytes integer for quantization you still need 203 GB of RAM it depends upon whether it is a vram or Graphics memory or Ram depending upon how do you want to run it so either way this is almost an impossible task for us to run it so if you were to run the other models the 8 billion parameter and the 70 billion parameter or let's say you have got the hardware to run this model the first thing that you have to do is you have to go to the link that I'll give it in the YouTube description that will take you to hugging face if you do not have an account in hugging face please make sure that you create one so after you come here llama 3.1 landing page this is a collections page on hugging pH you can go pick any model that you want here so in my case I'm going to just go select the 3.18 billion instruct model and after you go here you will see a form in my case I've already fill the form but in your case you will see a form here the form will actually have certain details like name affiliation date of birth and uh country so you have to fill in that once you do that your request will be submitted that I think right now it takes a bit of time for you to get the approval I'm not sure why it should be ideally automated but let's say you want to get the model access this is what the first step that you have to do once you do that you'll be able to access the model you should be able to download the model um you can access it with Transformers just like this so it's a very simple Transformers code all you have to do is import Transformers and talk the model ID is this you should be able to pretty much run this on Google collab without even any quantization and this is the first thing so first thing go ask for approval request approval once you get the approval you can download the model and use it if you actually want a separate Google collab notebook maybe I'll create a separate tutorial with all the details about how you can run the model with Google collab but then let's say now you have got this model and you know how to run this with hugging face Transformers but if you want to run this model with let's say a cloud version so MAA AI has made it really easy for you to run this met. a I have not logged in because I don't have a Facebook account it seems like I'll be forced to create a Facebook account that'll be a shame so you can go to this particular platform and you can just start chatting with this model can you create a snake game in Python so all I have to do is send this message okay it says I have to log in um continue without loging in cool when did I finish when was I born I was born in 1974 cool go on and you can see that it is starting to create the snake game it's pretty good the latency is pretty good I'm not sure I'm there is no information about what is a model that they're running it's not very easy but MAA has said that they are running the 405 billion parameter model this particular link so we can believe and do it if you are on WhatsApp and if you in the US you can try out this model also using your WhatsApp where you would see the same icon meta AI as one of your contacts and you can check out this model the next option is you can go to hugging chat HF doco chat and the default model at this point is metal Lama 3.1 405 billion parameter model instruct fp8 so you can just go here and then say can you create a joke about Elon Musk and uh the model will have uh some issues right now because everybody's trying model is overloaded but technically this should ideally work uh in the other scenario where it is not overloaded it is also available as part of other API providers like grock together AI fireworks Ai and all the other platforms but if this this is like the easiest way for you to get started with the model the quickly reminder uh first thing first and foremost thing just get an access otherwise it's a big pain and after you do that then you can just download the model you want or good met or good hugging chat then start using this model model and I'm going to put together a separate Google collab tutorial please let me know if there is an interest otherwise see you in another video Happy prompting