1. Longformer: The long-document transformer;Beltagy,2020
2. Unlimiformer: Long-range transformers with unlimited length input;Bertsch,2023
3. Token merging: Your vit but faster;Bolya,2022
4. Recurrent memory transformer;Bulatov;Advances in Neural Information Processing Systems,2022
5. Scaling transformer to 1 m tokens and beyond with rmt;Bulatov,2023