Agentic Model Customization on Amazon SageMaker AI | Amazon Web Services

Key Takeaways

Hey everybody, in this video I'll walk you through Amazon SageMaker AI's new capabilities with agent guided model customization using SageMaker AI studio and its serverless model customization features. Myself Sepra, a specialist essay at Amazon Web Services. Customizing large language model with your propriated data and domain expertise requires a series of coordinated decisions. Traditionally it's been a complex time consuming required deep ML expertise. Now Amazon SageMaker AI has simplified that with agent guided model customization features through an agentic code first experience. The new features takes you from a conversational description that is prompts of your LLM use case to a deployable model guided every step of the way by coding agents. Sage Mikai now comes integrated with coding agents and agentic skills. The agentic skills are specialized skills that guide the workflow from data preparation, evaluation all the way up to deploying serving the model. These agents and agent skills are enabled by default in SageMaker Studio. The agent skills themselves are open source built using the open agent skills standard giving you full flexibility to extend or modify them to fit to your team's workflows, governance standards and compliance requirements. Everything these coding agents produce is fully yours. You get fully editable Jupyter notebooks. The code you can inspect, modify, version control and you can plug it into your existing setups. You can interact with this coding agent through many ways in SageMaker's AI studio. Kirro and agent skills come preconfigured out of the box. You open your Jupyter lab and you're ready to go. Furthermore, the studio Jupyter Lab also has built-in ACP support agent client protocol, which means you can work with many other coding agents like cloud code all within the same studio environment. If you prefer working in your own ID of choice, that's also fully supported. the coding agent and agent skills with Kirro IDE supported also in cursor and VS code. Now let me show you what it looks like in practice. I'm going to walk you through a real example fine-tuning a large language model using the agentic experience in SageMaker AI Studio. Now I'm going to flip over to my AWS console. I'm here in my SageMaker Studio landing page to access my coding agents. I will open the Jupyter Lab space. I can either create a brand new Jupyter Lab space or I can use any of my existing Jupyter Lab space. The important note for us here is we need to make sure we use the latest SageMaker distribution image that is of version 4.1 or higher. With that I can open Jupyter Labace two ways. One is in the browser itself or I can open it in the IDE of choice. We have VS code hero and cursor through which these ids will use remote access to connect with Jupyter lab space and the agent agent skills are configured there for you. Now I have the Jupyter lab space open in the browser here. When I come into my Jupyter Lab space, I can access the coding agent by clicking on the Jupyter Lab chat in the launcher or I also have the option to click on open the Jupyter Lab chat in the sidebar. Click on the chat. Open the new chat. Give it a name. It opens the chat window and Ko is preconfigured for you. So I can call Kirro by typing the name and it's going to say hi to power it up. And there you go. Ko is available for you to use any other third party coding agent that's also possible. Like I have cloud code installed here. It's a similar experience how I'm going to use cloud code with kirao. I have to install cloud code and I have done that using the npm install method and the instructions for you to install will be provided in the link in the description below. Now talking about the skills that are there, I'm going to ask Kira to tell me the skills it has. So I'm going to send a prompt asking it to list the skills and KO list all the skills with the description from the planning skill which is the one which creates the step-by-step customization plan to the finetuning related skills, data preparation, model evaluation, deployment. So all the skills are available and to use the skills I can get started with a very simple prompt. I'm going to now ask it a simple prompt to fine-tune a model. And we can see Ko is typing and it has first activated the planning skill and now it has come up with a plan for this very specific prompt. Now I'm going to walk you through this plan with a more end to- end use case. But the important behavior for us to note here is this behavior where KO or the coding agent always confirms with me on the plan and the outputs before proceeding. You will see this behavior throughout as we work with the KO agent. Now I'm going to flip over to a chat where I have used used KO as the coding agent to fine-tune a quen model using RLVR technique. Here I send a prompt for with very specific instructions including the instructions on which very model to use, which hugging face data set to use and with the prompt we can see the KRO has activated the planning skills, defined the plan from use case definition, finetuning setup, data set related activities all the way up to model evaluation deployment. I'm going to walk you through the outputs soon. Now the important behavior as I was noting is it verifies these plans and the output with you. So after my confirmation, it moves ahead to the next step. The next step here is to create the use case. It activates the very specific skill, creates the business plan, the success tenants and again it verifies whether the output looks correct and then moves to the next step. Next step is to do the finetuning setup that is make sure we select the right model and we select the right model comes back with the very specific plan. Now on the base model it aspect and the finetuning technique confirms with me and moves ahead to the next step. The next step now being preparing the hugging face data set we gave it. It reads the data set, loads the data set and creates the target RLVR schema. Now we want to fine-tune it using RLVR schema and it uh produces the data set and also gives a sample output and for us to see and confirm. Once we confirm it moves ahead to the next step that is here it produces the Jupyter notebook for the data set transformation and the notebook is generated here side by side. I even ask it to then execute the notebook. It even executes that notebook. Similarly, I after I execute this notebook, confirm the output, I ask it to move to the next task of fine-tuning the model. I work with Jupyter Lab in the chat window with Kira and it produces the outputs that is the fine-tuning notebook followed by that I work with hero to produce the model evaluation notebook here it helps me select the evaluator model and the metrics which are more appropriate for my use case produces the model evaluation notebook and the model deployment code. Now this is all fully editable notebooks. I can edit any aspects of it but they are the notebook and the code is fully executable. I execute all these notebooks and the outputs are all generated in the serverless model customization. Now let me go back to the SageMaker Studio and see the outputs. The outputs are all in the model section. I go to my models and here's the Quen model which we fine-tune and I can see the fine-tuning results like the performance and training related statistics. I can see the full statistics and results in the MLFlow app that's associated with it. I can see the evaluation job and its results that are also captured here and the deployment where this fine-tune model was deployed in my case to bedrock. I get a full end to-end lineage snapshot here from the data set that was used, the finetuning job all the way up to the deployment. Throughout the course of this execution, there are logs which are generated in real time for my verification. And these logs are all captured here. Overall, this covers the full model customization life cycle from data preparation, evaluation, fine-tuning, deployments, end to end. This is the new agent guided model customization in SageMaker AI. To recap, you describe your use case in natural language prompts. The agent skills help you through the best practices and you get fully editable code that you own and control. And all of this you can do it in the Jupyter Lab space in the browser or in the IDE. To learn more, check out the links in the description below and get started today in SageMaker AI Studio. Thank you.