1. David Chiang , Peter Cholak , and Anand Pillay . 2023 . Tighter Bounds on the Expressivity of Transformer Encoders . In Proc. ICML. David Chiang, Peter Cholak, and Anand Pillay. 2023. Tighter Bounds on the Expressivity of Transformer Encoders. In Proc. ICML.
2. Social processes and proofs of theorems and programs
3. Nouha Dziri and Ximing Lu. 2023. Faith and Fate: Limits of Transformers on Compositionality. arXiv preprint arXiv:2305.18654. Nouha Dziri and Ximing Lu. 2023. Faith and Fate: Limits of Transformers on Compositionality. arXiv preprint arXiv:2305.18654.
4. On the naturalness of software
5. Bingbin Liu and Jordan T Ash. 2022. Transformers learn shortcuts to automata. arXiv preprint arXiv:2210.10749. Bingbin Liu and Jordan T Ash. 2022. Transformers learn shortcuts to automata. arXiv preprint arXiv:2210.10749.