How to Set Up LLaMA 3.1 Locally & Use OpenAI Python Pkg to Write Code with Your Own LLM | #OwnYourAI

Key Takeaways

hello everyone my name is vishar today I'm going to show you all how you can use locally hosted llm version that is Lama 3.1 by meta with AMA and how you can write a code to create a chat GPD like interface with it this is Lama 3 is a llm model provided by meta you can visit this website and try for more AMA is a way to uh have llms on a like a local machine like your MacBook or a Windows machine with uh good GPU or even a Linux machine uh you can go to ama.com click on download Based on your operating system you can choose the version and download that once you download uh you want to go to the models section which is right over here you can go to models and you can uh look for Lama 3.1 then you just need to copy this command open a terminal and paste this command now I have already downloaded and uh set this up so it will directly let me use the Lama 3 on my local machine here you can I can just say hi tell me a joke and it will write me a joke uh just thing to note that this is using my totally locally hosted uh model and nothing on the internet but you might want to write a code around it to use it you can use open AIS package directly uh with locally hosted um uh your LM that is Lama 3 in this case you need to import the package then you need to set the base URL as the local host and these are some default details which comes with AMA you can specify an AMA like a dummy API key then you you want to specify the message similar to what you would do to an open AI API call let me open a quick terminal new terminal now uh here I have uh created a package so I'm just going to do this uh this is uh this is a python uh environment which I have created you guys can create the same and then you can run like something like this python uh test open. and uh now what it is doing it's going to go back to the my locally hosted model it's going to uh ask it to tell tell us a joke and uh it's going to return us the response and you can see that the entire code has been written using open as default package there is nothing specific that we have done or changed and you can see here that now that code is executed we have gotten the response from a local US model you can use this uh approach to write the codes which are propri proprietary to you and you don't want to use um Enterprise or uh uh third party providers like opener Claud coher or any other I hope this information for useful stay tuned for more and thank you