Misalignment or misuse? The AGI alignment tradeoff

Bibliographic Data

ID21370547
AuthorsMax Hellrigel-Holderbaum (0000-0002-7236-7552, Friedrich-Alexander-Universität Erlangen-Nürnberg), Leonard Dung (0000-0003-4154-5560, Ruhr University Bochum, corresponding author)
Year2025
Publication date2025-10-10
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenuePhilosophical Studies (JOURNAL)
Journal identifiersISSN: 0031-8116 • E-ISSN: 1573-0883
PublisherSpringer Science and Business Media LLC (PUBLISHER)
DOI10.1007/s11098-025-02403-y
OpenAlexW4415015395
LanguageEN
References cited85

Creating systems that are aligned with our goals is seen as a leading approach to create safe and beneficial AI in both leading AI companies and the academic field of AI safety. We defend the view that misaligned AGI – future, generally intelligent (robotic) AI agents – poses catastrophic risks. At the same time, we support the view that aligned AGI creates a substantial risk of catastrophic misuse by humans. While both risks are severe and stand in tension with one another, we show that – in principle – there is room for alignment approaches which do not increase misuse risk. We then investigate how the tradeoff between misalignment and misuse looks empirically for different technical approaches to AI alignment. Here, we argue that many current alignment techniques and foreseeable improvements thereof plausibly increase risks of catastrophic misuse. Since the impacts of AI depend on the social context, we close by discussing important social factors and suggest that to reduce the risk of a misuse catastrophe due to aligned AGI, techniques such as robustness, AI control methods and especially good governance seem essential

Catastrophic failure · Control (management) · Corporate governance · Field (mathematics) · Philosophy of language · Philosophy of mind · Reinforcement Learning in Robotics

Tools

Open DOIOpen Access