The conversational action test

Detecting the artificial sociality of artificial intelligence

Bibliographic Data

ID6443382
AuthorsAlbert (0000-0003-1043-1237, Loughborough University), W Housley (0000-0003-1568-9093, Cardiff University), Rein Ove Sikveland (0000-0002-6322-5800, Norwegian University of Science and Technology), E Stokoe (0000-0002-7353-4121, The London School of Economics and Political Science, UK)
Year2025
Volume27
Issue10
Pages5592-5621
Publication date2025-10-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueNew Media & Society (JOURNAL)
Journal identifiersISSN: 1461-4448 • E-ISSN: 1461-7315
PublisherSAGE Publishing (PUBLISHER • US)
DOI10.1177/14614448251338277
OpenAlexW4414812188
LanguageEN
Citations received5
References cited91

Drawing on the “Voigt-Kampff Empathy Test”—a science fiction version of Turing’s famous thought experiment—we propose the Conversational Action Test (CAT): a new approach to evaluating conversational artificial intelligence (AI) voice agents. We compare social actions in a range of telephone service encounters where one party is an artificial conversational agent to a range of similar human-human calls. The CAT demonstrates a novel paradigm that addresses long-standing theoretical and methodological problems for ostensible “tests” of conversational AI by (a) revealing the conceptual confusion of attempting to “detect” an AI in routine service interactions and (b) focusing, instead, on the situated interactional practices through which an AI “passes” for human. We discuss the implications of the CAT for the design and evaluation of conversational AI, and for the notion of “humanness” as a goal or benchmark for such systems. Data include publicly available human/AI service calls and comparable human-human calls in British and American English

Deception detection and forensic psychology · Language, Discourse, Communication Strategies · Language, Metaphor, and Cognition

Tools

Open DOIOpen Access