Majority voting of doctors improves appropriateness of AI reliance in pathology

Bibliographic Data

ID21642323
AuthorsHongyan Gu (0000-0001-8962-9152, University of California, Los Angeles, corresponding author), Chunxu Yang (0000-0002-9684-7534, University of California, Los Angeles), Shino Magaki (0000-0003-0433-5759, University of California, Los Angeles), Neda Zarrin-Khameh, Neda Zarrin‐Khameh (0000-0001-7506-156X, Baylor College of Medicine), Nelli S Lakis (University of Kansas Medical Center), Inma Cobos (0000-0002-2043-1890, Stanford Medicine), Negar Khanlou (0000-0002-7308-9363, University of California, Los Angeles), Xinhai R Zhang (0000-0001-7524-2547, The University of Texas Health Science Center at Houston), Jasmeet Assi (University of Kansas Medical Center), Joshua T Byers (0000-0001-5064-4603, University of California, San Francisco), Ameer Hamza (0000-0002-4054-9730, University of Kansas Medical Center), Karam Han (0000-0002-7913-8293, University of Wisconsin–Madison), Anders Meyer (0000-0003-2036-3207, University of Kansas Medical Center), Hilda Mirbaha (University of California, Los Angeles), Carrie A Mohila, Carrie Mohila (0000-0002-0261-0406, Baylor College of Medicine), Todd M Stevens (0000-0002-2313-463X, University of Kansas Medical Center), Sara L Stone (0000-0002-1689-8148, Hospital of the University of Pennsylvania), Wenzhong Yan (0000-0002-3711-0807, University of California, Los Angeles), Mohammad Haeri (0000-0001-8509-4525, University of Kansas Medical Center), Xiang Chen (0000-0002-5045-9253, University of California, Los Angeles), Xiang ‘Anthony’ Chen
Year2024
Volume190
Pages103315
Publication date2024-10-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueInternational Journal of Human-Computer Studies (JOURNAL)
Journal identifiersISSN: 1071-5819 • E-ISSN: 1095-9300
PublisherElsevier BV (PUBLISHER)
DOI10.1016/j.ijhcs.2024.103315
OpenAlexW4399910920
LanguageEN
Citations received1
References cited53

As Artificial Intelligence (AI) making advancements in medical decision-making, there is a growing need to ensure doctors develop appropriate reliance on AI to avoid adverse outcomes. However, existing methods in enabling appropriate AI reliance might encounter challenges while being applied in the medical domain. With this regard, this work employs and provides the validation of an alternative approach – majority voting – to facilitate appropriate reliance on AI in medical decision-making. This is achieved by a multi-institutional user study involving 32 medical professionals with various backgrounds, focusing on the pathology task of visually detecting a pattern, mitoses, in tumor images. Here, the majority voting process was conducted by synthesizing decisions under AI assistance from a group of pathology doctors (pathologists). Two metrics were used to evaluate the appropriateness of AI reliance: Relative AI Reliance (RAIR) and Relative Self-Reliance (RSR). Results showed that even with groups of three pathologists, majority-voted decisions significantly increased both RAIR and RSR – by approximately 9% and 31%, respectively – compared to decisions made by one pathologist collaborating with AI. This increased appropriateness resulted in better precision and recall in the detection of mitoses. While our study is centered on pathology, we believe these insights can be extended to general high-stakes decision-making processes involving similar visual tasks

Cognitive psychology · Majority rule · Management · Medical education · Pathology · Political science · Recall · Second opinion · Voting · AI in cancer detection · Artificial Intelligence in Healthcare and Education · Computer Science · Explainable Artificial Intelligence (XAI · Medicine · Psychology · Artificial Intelligence

Tools

Open DOIOpen Access