MuleSoft inference Connector - [Agent] Define Prompt Template

Hello everyone. So as part of this video uh I'm going to start the new series the play playlist which is related with the new soft interference connectors. Okay. So the latest version which is available is the 1.2. Okay. If we will go through this definitions like whatever the description has been. Okay. So any point connector for the new soft okay provide access to the imperance offering for the LLM which is nothing but the large language models from multiple providers including OpenAI, OpenAI compatible endpoints, open router, Heroku AI, Azure AI foundry, Azure OpenAI and many others. Okay. So this connector provides operations to interface directly with the API of the various imperance providers. Okay. enabling seamless integrations to AI capabilities into your new applications. So uh in summary if you are going to summarize if suppose we are uh we have a requirement in our dual applications to connect with uh uh this one of our the LLM like the take the example of the open AAI. Okay. So it has the uh this connector uh uh ability we can able to connect. So once we are going to connect so our interaction could be related with the image okay like the image related problem our interaction could be like the text related prom or something else okay so basically it has the few supported operations so that we are going to see okay what all the supported operations are there okay so coming to this okay [snorts] so if you have noticed or Like while we are uh looking into our any point studio we can able to see like new soft inference connector is broadly going to divided into the five categories. Okay, one is basically going to I have given the names or maybe it is also there. Okay, like the agent. Okay, which means nothing but it is the defining the prompt template. Okay, which means uh we can consider our LLM as a kind of the template. Okay, so like suppose you are the like customer support template. Okay. Or maybe the customer case related, customer service related, any kind of the template we can ask like okay please like we are going to treat as part of the prompt as a this template. Okay. And then we can write the next prompt like what is the action like this uh any LLM model need to use it. Now second things we can able to consider as a chat one. Okay we can categorize. Okay. So here in the chat we can see like the two categories. One is the chat answer prompt which is nothing but that is the we can do the continuous conversation with our LLM. Okay. Which means that uh like suppose in a context we can able to communicate with the LLM. Okay. The second part as part of the this chat category. This is the chat completion which means once you are going to send the query okay that we are going to uh uh get the response directly. Okay. So as and okay so these are related with chat. Okay. Similar to this is for the tools. Okay. which we are going to see like the reasoning only what we can able to achieve it. Okay, this is for the image related. Okay, we will come to know uh the details about these tools. Okay. Image related also like uh whenever we are going to generate uh from a prompt we can able to generate or different size of the image like suppose somewhere we have to like upload uh upload the image as a avatar kind of things or some of the like the profile kind of things. So on the very like uh a specific requirement we can able to promp the prompt to generate an image. This is something we can able to read the image which is located somewhere. Okay. As per our prompt. So we will see like one by one like whenever we are going to the videos. Okay. This is the toxicity like suppose whenever any prompt which is coming and which is containing the some like the harm or like abusive abusive languages. Okay. So it has the capabilities can also able to detect it. Okay. So if you'll go like in details okay so broadly this agent or chat if you we are going to further uh generalize we can consider as a text generation. So these all are going to call as a text generation these from there we can able to see these are the image related. Okay. So agent define template this chat answer prop this chat all are the text generations. Okay. So coming to your first okay as part of our videos okay so we are going to first deal with our agent one okay let's see what is the purpose of this agent okay okay so what this agent is going to do okay so agent define prompt template like suppose a human tries to slow during p incident Okay, like I have just considered okay during the outage or security events or the incidents kind of okay to doing the tries with the human little bit time consuming. Okay. So how we can able to automate triest okay and on the basis of the any case or service request or complaint has been raised by customer we can able to automate it and we can before it's reaching to the uh this human it can suggest some response from the agent okay so that user can able to like uh follow that uh instructions okay if that is going to fulfill that that is fine like If it's um solving the issue that is okay. If it is not solving then the last step will be to involve the human or the like to take the help on that. Okay. So it can like perform like few of the use cases which uh I think like it can produce a concise assembly for an SN. Okay. recommends routing priority and tags like suppose uh some different different channels are there okay some like slack executive okay email or some things like the creating the incidents okay service now okay so these are the on the basis of priority different routing is required for the different priority kind of the incident so it has also like looking into priority it can redirect it okay uh generate a ready to send customer reply. Okay. Uh like whatever the suggested response are there, it can able to send the response. Okay. All in real time when a case is created, updated. So agent get immediate conceptual help. Okay. So define prompt template is used to register the LLM prompt patterns case summary routing reply once then all the inference operation in runtime flows to every element event. Okay. So let's see how we can able to basically uh implement these things into this anyoint studio. Okay. So let me quickly go over here. Okay. Simple I have defined uh this agent one. Okay. One of the API which is the define prompt template. Okay. This is the basically the loggers I have put here. I have creating the request builders. Okay. Okay. So from postman we are getting uh some of the instructions. So that I'm storing into instruction where into template uh I am storing this template which is coming into template where here if you see the data set I have modifying the some of the change because as part of the uh postman request I'm sending the JSON object which is nothing but the case details. Okay. Like uh for the example, okay, and on the basis of Okay. So um which is the u customer as part of the postman. Okay. Like considering uh the postman is a client which is going to send the case summary details. Okay. Along with the not the case summary like case related entire payload. Okay. We will see. Okay. and using that I'm forming one of the data set uh prompt okay so I'm telling that you are a helpful customer service agent customer okay payload is data set so this is the contact number so like this is say the payload so customer okay so customer is this is the name okay has an issue with uh subject okay and description this and issue priority is this so this is I have formatted one of the prompt okay if you see let's align here this is the prompt okay which I'm sending here okay so what I'm sending like template you are a customer support agent who analyze the customer case details in the data set okay so this is the customer case details which is I'm sending as a part of the data set which is going to analyze what the instruction I'm ing this LLM. Okay. Answer via plain text output with all recommended personalized personalized response. Do not repeat the response directly. Start the conversation with the formal greetings. Okay. So I'm just instructing uh giving the instruction to the data models uh sorry this LLM. Okay. So on the basis of that I I did this form uh uh formatting that prompt. Okay. Okay. Now this is the uh supported operations where we are supposed to pass this informations. Okay. So here you can see I'm passing the template. So what kind of the template you are supposed to it should act. Okay. So that template I'm passing and this template is nothing but this value. Okay. Now this is the instructions which is coming from this okay and this is the data set although I'm not passing directly but the whatever this I have created here that I'm passing here so these three things are required okay in order to like make sure uh your LLM is going to work against your this prompt template okay this is the response I have done okay now apart from that I haven't did any uh others uh configurations. Looking into your see this is why I told this is the text generation because for this the connector configuration is required called the text generations. Okay, if you see here so this is the text generation. So which is going to support multiple connections. Okay, like OpenAI, okay, if you see almost whatever this data like LLM models are available, okay, almost everything which is they are supported. Okay, I have selected this open AI. I have chosen this open AI. Inside that you have the also the different model names. Okay, so whatever the model names you are going to because each model has some of the like enhancement. Okay, someone is working fine. Someone is working like the with the very less token generate. Someone like the big token generate. Okay, so because I have chosen this GPT 4 mini because I want to the less token generation because you all know like the billing of this is going to happen on the basis of the token. I have also instructed uh generate max token 500. Okay, this is the temperature value and top P is also zero. If you see here control diversity by limiting choice of the most probable options. Okay. If you see the control randomness low is predictable high high is random. So these are the something like how your response is going to generate. This is the important one API key. Without this it is not going to work and how you are going to get this key. Okay. So you are supposed to okay it is not going to work with your free tiers okay you are supposed to register like suppose here I have taken the subscription for the open you are supposed to take the sub subscription of the any lm okay and at API keys you are supposed to generate a new secret similar to this okay and this key you are going to use it otherwise it's not going So make sure prior to going to explore those okay you are if you go to the billing okay so currently it is I I have chosen pay as you go model okay so I have done lot many uh this things okay for my P and to learning purpose so currently my uh remaining balance is the $4 because at least $10 or maybe the $5 you are supposed to invest in the earlier Okay, because whenever I'm going to hit as per the token which is going to generate this balance will be going to reduce. Okay, so this is the this information I haven't put any this proxy on this. Okay, so let's quickly uh deploy this. Okay, and we are going to see. Okay, so it's already running. Okay, so that is good. Okay, so it's already running. Okay, let me hit this. Okay, through the postman. Okay, so I'm sending here. See, you are the customer. Okay, Salesforce or unable to login Salesforce job. Okay, so this information I'm sending here. Okay, so once I'm hitting this locally, let's see how this is going to behave. Okay, what all the information we are going to get? It's running. So if you see it is following the actual instruction whatever we have given. Okay. So it's saying just analyze this. Okay. Dear Rajiv. So how we got? So looking into this contact name. Okay. Thank you for reaching out to us regarding the login issue you are experienced with the Salesforce development organization. I understand that encountering a 500 error can be quite frustrating especially when it uh hinders your access to important resource to assist you effectively. Could you please provide a few additional details especially it would be helpful to know if you have experienced this issue consistently or if it is this in the meantime you might try clearing your browser attempting okay so it is asking for additional details okay along with it is also suggesting here okay you might try clearing your browser cache or attempting to login in is using different browser or device to see if basu persist we are committed to resolving this matter promptly so please Rest assured that we'll work diligently to help you regain access of your sales for looking forward your response. So see how politely just like a human being has written this response. Okay. I I am using the YL to make it more readably. Okay. So and if you see this is coming nothing from but this one. Okay. So it hit hit this one and this is the response we got it. Right. So you you can see like how intelligently or very well manner way okay uh this is going to so similar way like suppose whenever a particular case is going to register for the first time okay although we are going to register the case into the salesforce okay or maybe like suppose if the case is already registered into Salesforce and we are going to receive through the platforms event we can maybe able to connect with this interference okay and get the resolutions and update to Okay. Directly as a case comment. So can executive or we can send the message or we can send the email or we can say so doing the like uh little bit orchestration and choreography. Okay. While we are doing the integrations we can able to achieve like uh if any uh suggestions or which can basically like uh expedite this any uh case issue. Okay. Uh rerouting kind of things. Okay. And to make it a more in a efficient or effective way. Okay. So, hope you are able to like understand the purpose of the agent defined template uh as part of your this new interference characters. Okay. So, uh thanks for your time. Okay. Uh I will recommend you just explore by yourself to make your hand dirty to write the prompt uh to just see like how this is going to behave. Okay. So, thanks for your time. Sorry.