When Is It Safe to Introduce an AI System Into Healthcare? A Practical Decision Algorithm for the Ethical Implementation of Black‐Box AI in Medicine
Bibliographic Data
| ID | 21504115 |
|---|---|
| Authors | Jemima Winifred Allen (0000-0002-3950-9950, Department of Paediatrics, Faculty of Medicine, Nursing and Health Sciences Monash University Clayton Victoria Australia), Dominic Wilkinson (0000-0003-3958-8633, Department of Paediatrics, Faculty of Medicine, Nursing and Health Sciences Monash University Clayton Victoria Australia), Julian Savulescu (0000-0003-1691-6403, Uehiro Oxford Insitute University of Oxford Oxford UK, corresponding author) |
| Year | 2026 |
| Volume | 40 |
| Issue | 1 |
| Pages | 61-72 |
| Publication date | 2026-01-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Bioethics (JOURNAL) |
| Journal identifiers | ISSN: 0269-9702 • E-ISSN: 1467-8519 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1111/bioe.70032 |
| PMID | 40964933 |
| OpenAlex | W4414341447 |
| Language | EN |
| Citations received | 2 |
| References cited | 43 |
There is mounting global interest in the revolutionary potential of AI tools. However, its use in healthcare carries certain risks. Some argue that opaque (‘black box’) AI systems in particular undermine patients' informed consent. While interpretable models offer an alternative, this approach may be impossible with generative AI and large language models (LLMs). Thus, we propose that AI tools should be evaluated for clinical use based on their implementation risk, rather than interpretability. We introduce a practical decision algorithm for the clinical implementation of black‐box AI by evaluating its risk of implementation. Applied to the case of an LLM for surgical informed consent, we assess a system's implementation risk by evaluating: (1) technical robustness, (2) implementation feasibility and (3) analysis of harms and benefits. Accordingly, the system is categorised as minimal‐risk (standard use), moderate‐risk (innovative use) or high‐risk (experimental use). Recommendations for implementation are proportional to risk, requiring more oversight for higher‐risk categories. The algorithm also considers the system's cost‐effectiveness and patients' informed consent
Clinical decision making · Decision support system · Generative grammar · Health care · Healthcare system · Precision medicine · Artificial Intelligence in Healthcare and Education · Ethics and Social Impacts of AI · Ethics in Clinical Research
Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum
Human Decisions and Machine Predictions
International evaluation of an AI system for breast cancer screening
Performance of ChatGPT on USMLE
AI in health and medicine
Consent-GPT
Beyond algorithmic trust
Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI
The selective deployment of AI in healthcare
Artificial Intelligence and Black‐Box Medical Decisions
Artificial Intelligence and Patient-Centered Decision-Making
Transparency in Algorithmic and Human Decision-Making
| Citations per year | 2 |
|---|---|
| Citation span | 2026 - 2026 (1) |
| Highly cited | No |