Getting Started with Azure AI Foundry in Python | Connect Deepseek, llama, Phi, OpenAI from Azure AI

Key Takeaways

hey everyone welcome back to the Technologies so we are in AI agent series and today we going to get started with Azure AI Foundry SDK we'll explore both the programming languages that is c as well as python for connecting to Azure AI Foundry SDK and Azure AI Foundry SDK is a set of libraries that connect to various other Azure AI services in this case it is going to connect to Azure open AI Azure AI inference aure AI search and many more okay so let's get started in our last video we created a new project and a new Hub in Azure AI Foundry I'll go to ai. azure.com this was the last project that we created PG JD Bots and here you can see the project connection string before getting started I'll go to the vs code let's create a new environment first we'll start with python and then we can move on to the cop code okay so I'm going to create a new folder instead Python and C perfect next for python let me reveal in file explorer open in terminal and let's open the vs code in that okay so I have opened the vs code in my python folder perfect now first thing first let's create a new environment so I'll go to view command pallet and create a new environment V EnV I'll choose the latest version that is 313.2 and here you can see it is getting created the new EnV M so the following environment is selected and here you can see venv is also created this here you can see this folder is also created my environment folder and if you open up a new terminal you'll see this V EnV uh here on the path so that confirms that your environment is created as well as it is is also activated okay now we are going to install few packages here and we'll be following this documentation getting started with Azure a Foundry SDK first thing first we're going to log in to Azure C and if you do not have a your CLI what you can do you can download it for your machine if you're using Mac windows or Linux you can install based on your operating system okay either Windows Linux or Mac you can install Azure CLI once you have installed Azure CLI what you need to do you need to come to the terminal or you can also go to the terminal on your Mac or on your windows uh at all the places this a your CLI will be accessible so you just need to do a z login this is a command to log in to your Azure subscription if you have multiple of them it will ask you which subscription you want to connect to okay so I'll just say enter it will open a web browser it will ask me to log in and you have logged into Microsoft Azure if you go back to your terminal it will retrieve the list of your 10 tenants and subscription that you have access to for some of the tenants it might not be able to log in uh if you want to log to a specific tenant then you need to provide some extra Flags to this command you can ask Co pilot if you want to log to your specific tenant as well as specific subscription okay so here it is giving me the list of subscription that I can choose from currently is selected as a third subscription but you can enter the number you want to choose okay just verify that the project that you created in Azure AI Foundry is part of what subscription you can see on the overview page so I'm going to select one okay there we have our subscription selected you can also verify what account you're already logged into just type AZ account show this will tell you that currently you are on this subscription and this is the account that you're logged in and this is the tenant that you currently logged into okay now let's clear this out next we need to install these two packages aurei project and Azure identity this is the package that will be used for Azure AI Foundry SDK and Azure identity will be used to log to your Azure account from your code basically it will be used to authenticate with all the client libraries with your enter ID account okay it has been installed you can verify by going to your environment folder and here you can see Azure identity is added and as your AI project is added and what version also you can see it here okay once that is done let's create a new file so here in the documentation they have mentioned both the ways synchronous and asynchronous code if you're developing as an API I would recommend you to go with a synchronous they also have all the samples in both synchronous and asynchronous programming okay so for this demo we're going to use synchronous but when we create an API out of this then we are going to go with as synchronous okay now let's import this package first let's create a new file okay so I'm going to name this file as main.py okay from Azure do identity import default asure credentials if you use default asure credential then it can either pick up your ca credential or logged in vs code credentials if you're working on Visual Studio then it can pick up your CLI your loged in or even your Azure service authentication uh it depends the I think one of the documentation they have mentioned the priority that it goes if it finds the suitable credentials then yeah it authenticates with that let's suppose you want to only use aure CLI credentials then you can mention it as your CLI credential in my case I'm going to use as your CLI credential because I don't want to get conflict with my other services okay next next is uh we're going to import Azure projects from Azure project client aure AI j. a. projects from Project client AI project client okay perfect next let's store the connection stream uh if you want to follow the best practices I would recommend you to create EnV file let's follow the best practices so I'm going to create a new EnV file and here I'm going to name this my project connection string project connection string okay it's not sqlite let's go to Azure a Foundry copy this project connection string and paste it here all right next we're going to import that let's ask goil it get configs from do EnV file okay so it is going to give me all of these okay so this is how you can can get the configurations okay and you need to install this package fix using copilot so this is the package you need to install pip install python. EnV well I'm not a core python developer that's why I'm utilizing uh co-pilot for my development so the best part about co-pilot now is uh it can ALS o put those commands directly in your terminal and then you just need to enter okay and now this error will be gone and here I'm going to use this variable project connection string os. getem okay uh let's verify if we are getting the connection string or not so I'm going to use print let's run this project if you have set python as a default for Python 3 then you can just use Python but in my machine I have not set it as python as an alas so I'm going to use Python 3 and I'm going to give a name of of my python file and enter so here you can see project connection string and it is and printing out my connection string perfect so now we know that it is getting the connection string from myv file perfect now we can get rid of this next next we're going to create this project client and AI project client Dot from connection string and project connection string and uh we need to provide credentials as well that is azure CLI credential perfect that is all now we can go on and do all the stuff that we want either connect to Azure open Azure AI inference Azure AI search we can do all those stuff now okay let me put a comment over here create a client using that's fine the new feature of copilot grub copilot in vs code allows you to autoc complete previously it was there but it was not that efficient in Visual Studio it was very efficient but now in vs code as well it is kindly highly uh efficient like you start typing something and it is going to autocomplete it for you okay either commands either code everything and uh if you want to enable that uh this one Auto compete not autoc complete let me show you that option EXT let me show you thatt ension I think the name was intell code yeah this one this is a very powerful extension that allows you to see all the methods that are part of any libraries okay previously it was completely random if you just click dot then it would give us the methods from different libraries as well like as a t but now it is going to give you only from that particular Library you know you must have noticed that if I have typed right as your dot it has only given me intelligence for AI only so that's the power okay now now we are going to connect to Azure open service okay let me install this package first and let's create that open EG object project dot inference get aure open there you can see if I put dot right so it is going to give me all the methods that are applicable and I need to pass the API version let's see let's use the same version that it is giving us I'm going really step by step I could have copied it but I like to write code uh rather than copy pasting okay it helps me understand things and uh yeah if I need to bug fix uh then it really helps me that so response equals to next open a chat completions create open a chat complet there you see that is the power of intelligence okay and including the co-pilot edit completions okay uh well it's not deployment ID it's model and let's use GPT 40 that's a good model that's the latest model we have and we have 01 O3 as well but yeah as compared to pricing and speed as well as uh good results GPT 4 is a good option then we have messages the first is R system let's add the system let me add one more object here roll system you're a helpful assistant and I'm going to give a question that's when that's a good question if you want to add temperature max tokens you can I'll keep it as is let me see what all options we have do we have a temperature yes we do have a temperature we can pass that do we have Max tokens yes and we have top P that also we have perfect so we have all the parameters that are applicable for open AI okay after that we can print the response response. choices messages content perfect let's print and run this application Python 3 main.py so why it's taking time because it is first authenticating it and we have an error I know what could be the error because we haven't deployed that model yet so um create model gp4 I want to see that error here yeah deployment not found the API deployment for this resource does not exist if you create a deployment within last 5 minutes please wait movement so yeah I knew it this was the error was coming up but I want to show you the error as well and how you can fix those errors okay I'll go to Azure Foundry I'll go to deployments uh where is my deployments model catalog and I'm going to use GPT 40 deploy okay you can choose the deployment type well data Zone standard I think it's the faster as compared to global standard uh let's use that data Zone standard or maybe yeah best to use global standard just in case okay and I'll use the connected AI resource you can configure this tokens per minute limit everything model version Let me see uh 2011 2024 and what is the version that we use here six let's see if it's going to work I'm not sure what version is that for this one is it a model version or is it what version I'm not sure yet parameters API version anyway let's see if we end up with another error we can find it out okay not found error connection default Azure open can't be found in this workspace that's fine uh let's see let's run this again okay the capital of France is Paris so it is now giving us the response perfect so we are successfully connected aure Foundry SDK to our Azure and you can also create embeddings uh create images audio so here you can see the documentation here you can connect to models such as 40 mini gp4 turbo di whisper iddings all those capabilities you can do it all right so now coming to Azure AI model inference this is another way of calling chat GPT uh this was using one way only specific to aure open but in Azure AI inference it can connect to any of your open source models with which you might have deployed such as the models from open AI Microsoft meta and many more even mral five models from Microsoft or even deep seek models so all those models you can connect to using this particular library that is azure AI inference let's install this uh in net it's still in preview I'm not sure about python let's install this package okay oh yeah here also it's still in beta I guess I think this is the naming convention that they might have used on the version B9 I'm pretty sure it might be on the beta okay now uh let's use this inferencing uh I can change this open AI uh object okay this was for aure open as your AI inference okay and chat inference uh I'm using um the naming convention from my net so let me make use of python naming conventions that is I can use underscores well the name I used earlier even that will work fine but uh yeah I would prefer to use the language specific naming conventions okay chat project dot what we have inference get chat completion client get chat completions client here we don't have any model next we have I'm going to change this response open response inference next chat. complete ah chatore inference do complete next is model so it is using the GPT 40 model only if you want to use other models you can use okay then uh I'm going to try this something here what is the capital of India okay let me try with Azure openi model well you can connect to Azure openi as well I'll show you the different uh model as well let's run this first the capital of France is Paris the capital of India is New Delhi perfect so it is giving us a response now I'm going to deploy some more models here not specifically the azuro model but we're going to use deep seek model this time so I'll go to model catalog and let's use deep seek V3 deploy uh if you're worried about the pricing right so I would suggest you go to this uh link see the pricing details and before trying this model out they have specifically mentioned the pricing uh per tokens per million tokens I guess so you can go through that pricing okay so let's deploy this and the deployment name is this one deep seek V3 and let's change this here okay and let's ask again there it is the capital of India is New Delhi it is located in the northern part of the country and serves as administrative political and Cultural Center of India well it has given us some extra content as well that's good deep seek is very good I guess uh if you're also looking for a faster model you can also go with f from from Microsoft let me show you that as well or I think Nvidia N I am well I was going through these models uh yesterday but I was not able to deploy them let me try Lama 3B 8B instruct yeah this one okay let's see oh this is instruct model I'm looking for chat model anyway that's fine uh yeah let's try 54 uh not mini instruct I just need 54 multimodel this one okay let's try this okay let's copy the deployment name paste it here I'm just showing you what all capabilities it has like you can connect to any model you want using aurei inference okay let's run this again okay the capital of India is New Delhi it serves as a seat of all three branches of government of India including the executive legislative and Judiciary the New Delhi is located within the capital Cent national capital territory of Delhi perfect it is giving some extra answer as well greater than our deep seek model okay so this is good uh we have successfully connected to deep seek models aopi models F model from Microsoft uh let's also connect to one more famous model that is meta so from meta what's the famous one I think llama is the famous one but I cannot see it here let me search it llama Vision instruct instruct let's see use this one let's use some instruct model as well agree and proceed it's really easy uh for you to connect to the various models uh in our llm industry so let's change the model let's see how this is going to respond perfect the capital of India is New Delhi this is very precise and straightforward nothing extra okay so we are good here now let's move on to the next one prompt templates or or maybe we can try this in our next video that will be better I think this video this is more than sufficient I guess you must have understood lot of stuff how you can connect to various models from using Azure AI inference or if you only target to Azure open a then you can only use Azure open a models it's your choice okay um yeah the all the models which are part of azure a Foundry apart from Azure Opa they have their own pricing so I would suggest you to go to that prizing model and use based on that okay all right so see you then thanks everyone see you in the next video