HuggingFace 🤗 on Trainium
A (not so deep) exploration of 🤗 Transformers training on AWS Trainium.
📢 UPDATE: 🤗 on Trainium is now part of the AWS Iberia AI/ML workshops
Inspired by Julien Simon’s post on how to Accelerate Transformer training with AWS Trainium 🙌, I’ve created a small project with a some Terraform configurations and PyTorch training scripts to help you get started ASAP.
The goal is to show how little one has to change (code-wise) to run training jobs on Trn1 instances, starting from Hugging Face’s demo on how to fine-tune a pretrained model.
It is still a WIP 🚧 so feel free to reach out if you have any suggestions on how to improve it.
📚 Want to learn more about AWS Trainium? Check out the References section in the repo docs.
