A world without violet

Peculiar consequences of granting moral status to Artificial Intelligences

Datos Bibliográficos

ID17675920
AutoresSever Ioan Topan (McGill University, autor de correspondencia)
Año2026
Volumen41
Número5
Páginas5103-5117
Fecha de publicación2026-01-19
Open AccessSí
TipoARTICLE
RevistaAI & Society (JOURNAL)
Identificadores de la revistaISSN: 0951-5666 • E-ISSN: 1435-5655
EditorialSpringer Nature (PUBLISHER • SG)
DOI10.1007/s00146-025-02834-5
OpenAlexW7124746472
IdiomaEN
Referencias citadas56

Granting artificial intelligences (AIs) moral status creates a set of ethical dilemmas stemming from our ability to control their preferences. As an illustrative example, if we could program an artificial system worthy of moral agency to experience suffering when encountering the color violet, then we appear to have created a scenario where we have a moral imperative to remove any violet objects from our surroundings. AIs in this situation can effectively “hijack” our moral landscape, altering the moral valence of arbitrary world states. In this work, we formalize the phenomenon of moral hijacking, analyze its implications from the perspective of several ethical systems, and propose recommendations for constraining novel preference creation in moral agents

Agency (philosophy) · Moral agency · Moral dilemma · Moral disengagement · Moral reasoning · Perspective (graphical) · Phenomenon · Social cognitive theory of morality · Ethics and Social Impacts of AI · Neuroethics, Human Enhancement, Biomedical Innovations · Psychology of Moral and Emotional Judgment