Abstract
AbstractGiven the impressive performances of LLM-derived tools across a range of tasks considered all but impossible for computers until recently, the capabilities of LLMs seem limitless. However, there are some fundamental limitations to what they can or cannot do inherent to the current architecture of LLMs. I will attempt to review the most notable of them to give the reader an understanding of what architectural modifications will need to take place before a given problem is solved. Specifically, I discuss counterfactual generation, private information leakage, reasoning, limited attention span, dependence on the training dataset, bias, and non-normative language.
Publisher
Springer Nature Switzerland
Reference29 articles.
1. Ross Taylor et al. Galactica: A large language model for science. CoRR, abs/2211.09085, 2022.
2. Benj Edwards. New meta ai demo writes racist and inaccurate scientific literature, gets pulled. Ars Technica, 2022.
3. Amelia Glaese et al. Improving alignment of dialogue agents via targeted human judgements. CoRR, abs/2209.14375, 2022.
4. Kurt Shuster et al. Language models that seek for knowledge: Modular search & generation for dialogue and prompt completion. CoRR, abs/2203.13224, 2022.
5. OpenAI. Gpt-4 technical report. CoRR, abs/2303.08774, 2023.