1. On the opportunities and risks of foundation models;Bommasani,2021
2. Exploring the limits of transfer learning with a unified text-to-text transformer;Raffel;The Journal of Machine Learning Research,2020
3. Vatt: Transformers for multi-modal self-supervised learning from raw video, audio and text;Akbari;Advances in Neural Information Processing Systems,2021
4. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity;Fedus;The Journal of Machine Learning Research,2022
5. Parameter-efficient transfer learning for NLP;Houlsby