A world without violet

Peculiar consequences of granting moral status to Artificial Intelligences

Dados Bibliográficos

ID17675920
AutoresSever Ioan Topan (McGill University, autor correspondente)
Ano2026
Volume41
Fascículo5
Páginas5103-5117
Data de publicação2026-01-19
Open AccessSim
TipoARTICLE
PeriódicoAI & Society (JOURNAL)
Identificadores do periódicoISSN: 0951-5666 • E-ISSN: 1435-5655
EditoraSpringer Nature (PUBLISHER • SG)
DOI10.1007/s00146-025-02834-5
OpenAlexW7124746472
IdiomaEN
Referências citadas56

Granting artificial intelligences (AIs) moral status creates a set of ethical dilemmas stemming from our ability to control their preferences. As an illustrative example, if we could program an artificial system worthy of moral agency to experience suffering when encountering the color violet, then we appear to have created a scenario where we have a moral imperative to remove any violet objects from our surroundings. AIs in this situation can effectively “hijack” our moral landscape, altering the moral valence of arbitrary world states. In this work, we formalize the phenomenon of moral hijacking, analyze its implications from the perspective of several ethical systems, and propose recommendations for constraining novel preference creation in moral agents

Agency (philosophy) · Moral agency · Moral dilemma · Moral disengagement · Moral reasoning · Perspective (graphical) · Phenomenon · Social cognitive theory of morality · Ethics and Social Impacts of AI · Neuroethics, Human Enhancement, Biomedical Innovations · Psychology of Moral and Emotional Judgment