Affiliation:
1. Department of Media Studies, University of Amsterdam, Amsterdam, the Netherlands
Abstract
In recent years, AI research has become more and more computationally demanding. In natural language processing (NLP), this tendency is reflected in the emergence of large language models (LLMs) like GPT-3. These powerful neural network-based models can be used for a range of NLP tasks and their language generation capacities have become so sophisticated that it can be very difficult to distinguish their outputs from human language. LLMs have raised concerns over their demonstrable biases, heavy environmental footprints, and future social ramifications. In December 2020, critical research on LLMs led Google to fire Timnit Gebru, co-lead of the company’s AI Ethics team, which sparked a major public controversy around LLMs and the growing corporate influence over AI research. This article explores the role LLMs play in the political economy of AI as infrastructural components for AI research and development. Retracing the technical developments that have led to the emergence of LLMs, we point out how they are intertwined with the business model of big tech companies and further shift power relations in their favour. This becomes visible through the Transformer, which is the underlying architecture of most LLMs today and started the race for ever bigger models when it was introduced by Google in 2017. Using the example of GPT-3, we shed light on recent corporate efforts to commodify LLMs through paid API access and exclusive licensing, raising questions around monopolization and dependency in a field that is increasingly divided by access to large-scale computing power.
Subject
Library and Information Sciences,Information Systems and Management,Computer Science Applications,Communication,Information Systems
Reference78 articles.
1. Abdalla M, Abdalla M (2021) The Grey Hoodie project: big tobacco, big tech, and the threat on academic integrity. arXiv:2009.13676 [cs]. Available at: https://arxiv.org/abs/2009.13676.
2. Abid A, Farooqi M, Zou J (2021) Persistent anti-muslim bias in large language models. arXiv:2101.05783 [cs]. Available at: http://arxiv.org/abs/2101.05783.
3. Ahmed N, Wahed M (2020) The de-democratization of ai: deep learning and the compute divide in artificial intelligence research. arXiv: 2010.15581. Available at: http://arxiv.org/abs/2010.15581.
4. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473 [cs, stat]. Available at: http://arxiv.org/abs/1409.0473.
Cited by
28 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献