Deploy a Hugging Face Transformers Model from S3 to Amazon SageMaker

Key Takeaways

hi everyone my name is philli I a machine learning engineer at hugging phas and DET Tech lead for our partnership with AWS I will show you in a few minutes how you can deploy hiking F Transformers from S3 to Amazon sagemaker for an inference Amazon sagemaker is a fully managed machine learning service that provides everyone with the ability to build train and deploy machine learning models quickly together with a s we have built an inference optimized solution to deploy Transformer models as s maker endpoints let's get started we are going to use a sagemaker notebook instance to deploy our model using a sagemaker notebook instance is not a requirement to deploy your model you can also use sagemaker Studio or you can deploy it from your local machine to be able to deploy a model from S3 to Amazon sagemaker you need to create an archive called model. tower. GC which contains all the relevant model and tokenizer files if you have trained your model before using the Amazon sagemaker and the hugging F DLC for training this archive will be created automatically for you at the end of your training and upload it to as free so if you have already trained a model you can use this model for deployment in this case I will use a distal Bird model fine-tuned on the sst2 data set to deploy our model we first need to create a hugging face model class the hugging face model class contains all the relevant information for our endpoint as our model data where the model archive is sored on S fre a role for the permissions our Transformers version and the pych or tensorflow version the huging phase inference solution is um can be used with py toch or tensor flow after we created our huging face model class we can run deploy the dot deploy method will then create our stagemaker endpoint therefore we Define the initial instance count and the instance type the initial instance count defines how many instance will be used for our endpoint and the instance type defines which instance will be used in our example I use the M5 instance which is a CPU based instance after executing this s maker will create our endpoint which takes around 5 minutes the endpoint has been successfully deployed to sagemaker and can now be used for inference the benefit of using the python sagemaker SDK is that the deploy method automatically returns predictor class which we can use to request our endpoint therefore we can run predictor predict and provide our inputs as an input I have a yeah simple sentence phrase um executing predictor predict our endpoint successfully predicts the correct class of forour input and works at the end you can run predictor delete endpoint which makes sure everything will be deleted on Sage maker properly if you want to learn more about hugging face on Amazon sagemaker check out huging F.C stagemaker