Deploy a Hugging Face Transformers Model from the Model Hub to Amazon SageMaker

Key Takeaways

hi everyone my name is philip i'm a machine learning engineer at hugging phase and also the tech lead for our partnership with aws i will show you in a few minutes how you can deploy hiking phase transformers from dehydrating phase hub to amazon sagemaker for inference amazon sagemaker is a fully managed machine learning service that provides everyone with the ability to build train and deploy machine learning models quickly together with aws we have built an inference optimized solution to deploy a transformer model as sagemaker endpoints let's get started we are going to use the sagemaker notebook instance to deploy our model using a sagemaker notebook instance is not a requirement to deploy your model you can also use sagemaker studio or deploy a model from your local machine the first step we need to do is to choose a model from the hugging phase hub the hiking phase hub offers over 10 000 different models fine-tuned on nlp vision and speech in this example we want to use a model fine tune on question answering and we also want to be it quick therefore we select the digital bird base cased distal squad model we copy the model id and go back to our notebook instance to be able to deploy a model from the hub to sagemaker we need to create a hugging face model class this model class contains all the configuration for our hub model as the model id and our task in this case question and the ring after that we can create our hugging face model and run the dot deploy method dot deploy method will create our sagemaker endpoint and we can pass in the initial instance count basically we can define on how many instances we want our endpoint to deploy it and the instance type in this case i went with the m5 instance which is a cpu based image the deploy will now create our sagemaker endpoint which takes around 5 minutes the endpoint has been successfully deployed to sagemaker and can now be used for inference the benefit of using the python sagemaker sdk is that the deploy method automatically returns predictor class which we can use to request our endpoint with the dot predict method therefore we need to define our inputs in this example since it's a question answering model we have our context my name is philip and i live in nuremberg this model is used with stage maker for inference and our question what is used for inference and our request has been successfully executed and the model successfully predicted the correct answer what is used for inference its sagemaker after you are done testing or requesting your endpoint you can run predictor dot delete endpoint this will clean up everything on sagemaker and make sure everything is deleted properly if you want to learn more about packing face on amazon sagemaker check out huggingface.co maker