The great Transformer

Examining the role of large language models in the political economy of AI

Bibliographic Data

ID5260696
AuthorsDieuwertje Luitse (0000-0003-0652-3315, University of Amsterdam, corresponding author), Wiebke Denkena (0000-0003-2728-7024, University of Amsterdam)
Year2021
Volume8
Issue2
Publication date2021-07-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueBig Data & Society (JOURNAL)
Journal identifiersISSN: 2053-9517 • E-ISSN: 2053-9517
PublisherSAGE Publications Inc (PUBLISHER)
DOI10.1177/20539517211047734
OpenAlexW3202773593
LanguageEN
Citations received36
References cited30

In recent years, AI research has become more and more computationally demanding. In natural language processing (NLP), this tendency is reflected in the emergence of large language models (LLMs) like GPT-3. These powerful neural network-based models can be used for a range of NLP tasks and their language generation capacities have become so sophisticated that it can be very difficult to distinguish their outputs from human language. LLMs have raised concerns over their demonstrable biases, heavy environmental footprints, and future social ramifications. In December 2020, critical research on LLMs led Google to fire Timnit Gebru, co-lead of the company's AI Ethics team, which sparked a major public controversy around LLMs and the growing corporate influence over AI research. This article explores the role LLMs play in the political economy of AI as infrastructural components for AI research and development. Retracing the technical developments that have led to the emergence of LLMs, we point out how they are intertwined with the business model of big tech companies and further shift power relations in their favour. This becomes visible through the Transformer, which is the underlying architecture of most LLMs today and started the race for ever bigger models when it was introduced by Google in 2017. Using the example of GPT-3, we shed light on recent corporate efforts to commodify LLMs through paid API access and exclusive licensing, raising questions around monopolization and dependency in a field that is increasingly divided by access to large-scale computing power

Economics · Political economy · Political science · Politics · Sociology · Adversarial Robustness in Machine Learning · Ethics and Social Impacts of AI · Law · Topic Modeling

Tools

Open DOIOpen Access