Assessing LLM-generated vs. expert-created clinical anatomy MCQs
A student perception-based comparative study in medical education
Bibliographic Data
| ID | 17720689 |
|---|---|
| Authors | Maram Elzayyat (0009-0004-5955-8663, Qatar University), Mohammad Javed Ansari (0000-0002-8718-3078, Qatar University), Janatul Naeim Mohammad (Qatar University), Sami Zaqout (0000-0002-3298-4425, Qatar University, corresponding author) |
| Year | 2025 |
| Volume | 30 |
| Issue | 1 |
| Pages | 2554678-2554678 |
| Publication date | 2025-08-30 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Medical Education Online (JOURNAL) |
| Journal identifiers | ISSN: 1087-2981 • E-ISSN: 1087-2981 |
| Publisher | Taylor & Francis (PUBLISHER • GB) |
| DOI | 10.1080/10872981.2025.2554678 |
| PMID | 40884796 |
| OpenAlex | W4413850072 |
| Language | EN |
| References cited | 30 |
Large language models (LLMs) such as ChatGPT and Gemini are increasingly used to generate educational content in medical education, including multiple-choice questions (MCQs), but their effectiveness compared to expert-written questions remains underexplored, particularly in anatomy. We conducted a cross-sectional, mixed-methods study involving Year 2-4 medical students at Qatar University, where participants completed and evaluated three anonymized MCQ sets-authored by ChatGPT, Google-Gemini, and a clinical anatomist-across 17 quality criteria. Descriptive and chi-square analyses were performed, and optional feedback was reviewed thematically. Among 48 participants, most rated the three MCQ sources as equally effective, although ChatGPT was more often preferred for helping students identify and confront their knowledge gaps through challenging distractors and diagnostic insight, while expert-written questions were rated highest for deeper analytical thinking. A significant variation in preferences was observed across sources (χ2 (64) = 688.79, p < .001). Qualitative feedback emphasized the need for better difficulty calibration and clearer distractors in some AI-generated items. Overall, LLM-generated anatomy MCQs can closely match expert-authored ones in learner-perceived value and may support deeper engagement, but expert review remains critical to ensure clarity and alignment with curricular goals. A hybrid AI-human workflow may provide a promising path for scalable, high-quality assessment design in medical education
Medical education · Medical physics · Perception · Anatomy and Medical Technology · Artificial Intelligence in Healthcare and Education · Innovations in Medical Education · Medicine · Psychology
How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment
A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment
Performance of ChatGPT on USMLE
Large Language Models in Medical Education
Keeping humans in the loop efficiently by generating question templates instead of questions using AI
A scoping review of artificial intelligence in medical education
Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education
| Highly cited | No |
|---|