Mistral AI Introduces Agents! (Tutorial)

Key Takeaways

Mel AI just made some exciting new announcements this specific announcement is about making it easier to develop more complex generative AI applications with large language models the great thing about Mell AI is that they are developing a range of language models that you can utilize today for a variety of use cases however it's really hard to customize those language models for your specific use cases so you may want to for instance change or improve the system message to improve how the model responds to whatever task you're interested in in addition it will be useful answer to be able to quickly prototype improve not only the quality of the outputs and how customized those outputs of those language models are but the ability to also reduce latency and experiment with a variety of models as we develop more complex workflows like agents for instance with these llms what Mally I is announcing here is their Vision to how they are enabling this for developers they announce here they're making it simpler more efficient to customize models and you can customize models like mro large 2 and cool as well and you can do this customization in different ways you can customize them by fine-tuning these models you can customize them by using things like f shot prompting and improving the system instruction and so forth so they're making available all these very strong reference models as I mention here so the idea now is for developers to have these really good strong Foundation models that you can fine tune or whatever use case you're working on and they're trying to make it as easy as possible so you can iterate and continue to prototype really quickly for your application can provide specific domain knowledge context tone whatever it is that you need to customize that particular model so the fine tuning feat Fe is available via their console I am going to show you an example of how to fine-tune a model really quickly and how to use that for instance as an agent how you can also test with it really fast using lechat as well so you can serve that mod really quickly using their chat interface another exciting part of this release is this idea of Agents so they're mentioning here that they're introducing an early version of their agents this wraps models with additional context and instruction for exposure on the chat or API so I think offering both is really useful so that you can continue exploring and experimenting with your language models and you can share with your team and so on it just makes things a lot more quicker and productive to experiment with the language modes that you're customizing the idea of Agents I don't think it's necessarily new I believe this is very similar to custom gpts where you're trying to customize Behavior by defining a set of instructions system instructions and you're providing it also examples of the type of outputs that you want focusing on the quality and so on so you define that you build your agent you serve it and you can use it for whatever component you're interested in building let's say You're Building this agented workflow that needs a very good research analyst you can develop a research analy customize that to whatever behavior and type of output that you want and now you have this customized model that you can leverage and you can leverage it again using the apis and you see what they mention here is that now you can use this mistal large two which is their really large model very capable model I think this is one of their strongest models that you can now layer on increasingly complex workflows with multiple agents that are easy to share within your organization so you can share them you can test them evaluate whatever PRS you're evaluating with that's much easier to do now with mol eyes I think this direction makes a lot of sense it's very developer friendly as well as you will see with some of the demos that I will cover in a bit so they mention here that one of their goals is to eventually enable these agents and have the ability to integrate with tools and data sources so let's say you wanted to connect your agents with some external database or PDF files or whatever it may be that you're interested in using with the agent or maybe some external tool like a search engine that should be easy to do I think the customization piece is important and then how to integrate with tools external tools is the next part and that's what's coming soon so I'm very excited about that I'm now going to jump into the console the mral AI console where I'm going to cover two examples so how to create an agent and how to find you the model I will touch on the first example which is creating an agent I think this is really interesting you can create an agent really quickly you can specify your instructions you can add demonstrations to it you can really customize how that particular language model performs and then you can also test it on the chat so I'm going to show you the whole process with one use case that I've been working on recently so I'm going to click here again I'm on the homepage of console. m.i and Le platform and then I'm going to click here on create an agent so I have already created an agent I'm just going to pull that agent back again here just to show you how it works so this paper summarizer is one that I just developed like a couple of minutes ago and you can see here that I'm using this particular mod so mrra large 2 and you can select any of the models that they have available you can even select finetune models that you have so that's really neat when you even have the Legacy models as well but I'm choosing this general purpose language model which is Mel 2 and then once you have that you can Define the temperature I'm setting temperature to zero I don't want any Randomness and then here is where you define how this mod will behave right similar to what you do with custom GPT here you are defining the instructions on how you want that particular mod to behave and this is a system prompt and what I've done here is I'm just creating a simple technical AI writer I will give this agent or assistant a technical AI paper abstract and it should convert that into a clear concise and Technical summary as the examples provided I'm going to provide examples below ensure you're using the same formats as in the example that's really important as a custom behavior that I want from this model and so I've provided demonstrations here which is really neat so these are the few short prompts so I have the user input and model output so you have to provide the pair and I'm giving it for instance I'm giving it the title of the paper and also the abstract you can see here I've given it a couple of examples so you can see this is one example I have given it another demonstration here and I've given it a third demonstration here so it's the same format so it's a title and abstract and then the model output is this quick summary the summary is basically like a short title and then a concise summary of what the abstract entails so this is just to simplify the use case right I'm just using abstract but for this you want to use abstract and other components of the paper such as results findings and so on this is just to simplify it and if you interested I actually have a data set for this so in one of our repos where we maintain ml papers of the week I'll provide a link in the description you can check out many summaries that I've done myself so all of these summaries are things that I put together I'm not using an AI for this my idea is how can I automate this whole process and make it much more efficient and productive for me to be able to do this type of work and you can also find the papers so you can copy the abstracts or these are just links to Archive papers so you can copy the abstract and the title of that paper and then you can bring it back into the agent here that I'm creating so that's how I kind of did this whole process and once you have that then you can create it and when you create it you deploy it as API so it's available in the API to use and then you can also deploy it in Le chat which makes this model or this particular customized agent available in the chat so this is what I've done so in order for you to test it now um you can test it down here or you can test it in the chat so I will click on this button here and this will take me right into lechat so right now I'm in lechat and I have available the mod itself right so I have the mods from Mel and then I also have the agents that are available that I have customized as well so what I can do here is I can just give it an input based on the type of input that it expects right this particular agent I've customized it to summarize papers given the title and Abstract that I'm passing to it so I've done that already I'll give you an example of how it looks so this is the example I've generated this actually saves the chats or previous chats as I was experimenting with it I noticed that it really struggle to give me this particular output that I really wanted so this is the output that I wanted right and something like a nice little title or short title and then the summary in this format finally I got it to work after iterating on the demonstrations a bit and the system instruction the system instruction that you saw is already kind of tuned so I went through a few iterations it was really quickly for me to iterate on it because because I can just go back to the summarizer here this agent and I can do changes and then update really quickly and the changes are reflected right away here and now I can continue experimenting one thing I noticed though when I was experimenting with this you have to be really specific with the instructions I think this is common sense but you also have to provide it more demonstrations so I tested with two demonstrations it didn't work quite so well and then I tested with three demonstrations and finally got the format a bit better so I think to improve this more and to make it more reliable to always give me that format that I want I need to give it a few more examples maybe like five or 10 examples so that's something to experiment with if you're working on this particular example or any example that you're working on just keep that in mind so that's the agent example I'm now going to jump into the fine tuning example here so I can go back here to overview and we have the option to finda model I'm going to find tun a model and then I can select any of these models so I've already fine tuned a few models as well so I'm just going to show you the process here the first thing you need to do is you need to have some data sets so you go to data sets you upload data sets if you're interested in data sets I have data sets available you can use my data set so the data set that I'm using I can go back here to the ti I'm uploading a few data sets just to make it easier for for folks to experiment and replicate some of the things that I'm showing here and in particular I'm using an emotion classification task here so I have some emotion data sets you can use this for training and it has to be in JSL format so this is the format that it's using it uses messages then you create a list of the role which is user role and then pair it with the assistant role so you have here the classification or label and the input itself which goes under the user role and I have a bunch of these that I've provided to the model and so I download this data set you can download it yourself if you want to replicate this particular experiment and then what you do is you go back here and then upload it to this data set it should identify this as a type instruct because that's the format that I'm using in the documentation you can find how you can upload different kinds of data sets for instance for function calling and so on so this is the valid data set which you can also find here I have a valid which is the validation data set that I'm using this one here and it consists of a few examples the only thing to note is that it will show you an error if you're using more than 20% of the training data set so make sure that's a smaller data set and it doesn't crosses that 20% threshold so training validation data set are already here now I can go to F tune models and then I already find tune model but I'll show you again the process so I go to finetuning of a model I select the model that I want I can select postal if I'm interested in a coding use case I can select Mell large 2 which is the more capable model but I can also select the other smaller models like Mel Nemo which is a 12 billion parameter model which was developed in collaboration with Nvidia but I'm just going to keep it to Mell 7B just for demonstration purposes now you can go to training data sets select the training data set you can go to validation data set you can select validation data set and this one is optional you can select your learning rate here that's something that you can adjust this is nice because this is not something that I saw in the opening I fine tuning interface and you can also select even the EPO as well so you can select 10 seven whatever that may be obviously the more EPO the longer it will take and then you select next and it will take you through the process uh this takes a little bit of time but I'm going to go back to F tune models here and you will see that this is the model that I fine tune I will click on it and here are the specifics of this particular fine tune model so it has a base model here I'm using this particular base model uh no function calling no fill in the middle those are things that you can fine tune on and then here's the cost as I was saying it's like 9 us this is the number of training tokens EPO learning rate and support one option that you have here you can archive it and you can create an agent so if you click on create an agent right it takes you to the agent interface and here's where you can create an agent on a specific finetune model when they say customization this is what they refer to as customization you can find your n you can further customize it you can add instructions to it further more and you can even add more F shop prompts examples as well if that makes sense for you but what I'll do here is I'll just going to show you this agent that I built on this F tun model that I just showed you so this emotion classifier is an agent and I wanted to create an agent because I wanted available in the chat so this is the reason why I created one and this one is based on the fine tune model that I just showed you which is you will see it right here so I can select that and here is the particular agent and this agent has a system prompt just to guide it a bit more and that's something that you cannot hear why not and the more you can customize it the more you can give it clear instructions the better the small is going to per firm and so this is the instruction here and then now you can select Le chat right and you can it's available in API well I'm going to select the chat and now you will see that it's available the paper summarizer example and then you have the emotion classifier example now I'm going to select emotion classifier and then just going to quickly test here so I'm going to say I'm feeling right very happy so what I told this model is to specifically output just the emotion label alone and it was important for me to provide it that specific instruction because when I fine tune this model I just gave it the input and output paer I did not give it any Specific Instructions but because now you have the ability to do that using this agent interface you can further customize how you want this particular mold to behave which I I think is really amazing and really interesting way of building an experimenting with these language models so I'm going to say I'm very sad and say sadness sadness is one of the labels okay that's okay I guess and then this one is surprise this one is usually harder for the model and from here you can continue experimenting you can run better evaluations by using the model via the API so this is something that I plan to do with this particular model just to compare with other models that I'm experimenting with there's also another video that will do that's based on the open AI fine-tuning interface so I was about to do that when I saw this launch so that's very good timing and I'll compare how all of these models perform right how the missal fine tune models perform compared to the openi models and I also want to kind of discuss as well these trends of building better interfaces to allow you to customize this models better which we are seeing from all these companies that will be it for the video let me know if you have any questions if you need any of the data sets if you need links to anything that I covered in the demos please let me know other than that that'll be it for this video thank you so much for listening please consider leaving a like And subscribe to the channel if you haven't it really helps the channel and I'll see you in the next one