Large language models in medical education
A comparative cross-platform evaluation in answering histological questions
Bibliographic Data
| ID | 15327919 |
|---|---|
| Authors | Volodymyr Mavrych (0009-0009-1159-4573, Alfaisal University), Einas Yousef (0000-0002-8935-0183, Alfaisal University), Ahmed Yaqinuddin (0000-0001-7536-910X, Alfaisal University), Olena Bolgova (0009-0002-9496-9754, Alfaisal University, corresponding author) |
| Year | 2025 |
| Volume | 30 |
| Issue | 1 |
| Pages | 2534065-2534065 |
| Publication date | 2025-07-12 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Medical Education Online (JOURNAL) |
| Journal identifiers | ISSN: 1087-2981 • E-ISSN: 1087-2981 |
| Publisher | Taylor & Francis (PUBLISHER • GB) |
| DOI | 10.1080/10872981.2025.2534065 |
| PMID | 40651009 |
| OpenAlex | W4412367130 |
| Language | EN |
| Citations received | 2 |
| References cited | 28 |
Large language models (LLMs) have shown promising capabilities across medical disciplines, yet their performance in basic medical sciences remains incompletely characterized. Medical histology, requiring factual knowledge and interpretative skills, provides a unique domain for evaluating AI capabilities in medical education. To evaluate and compare the performance of five current LLMs: GPT-4.1, Claude 3.7 Sonnet, Gemini 2.0 Flash, Copilot, and DeepSeek R1 on correctly answering medical histology multiple choice questions (MCQs). This cross-sectional comparative study used 200 USMLE-style histology MCQs across 20 topics. Each LLM completed all the questions in three separate attempts. Performance metrics included accuracy rates, test-retest reliability (ICC), and topic-specific analysis. Statistical analysis employed ANOVA with post-hoc Tukey's tests and two-way mixed ANOVA for system-topic interactions. All LLMs achieved exceptionally high accuracy (Mean 91.1%, SD 7.2). Gemini performed best (92.0%), followed by Claude (91.5%), Copilot (91.0%), GPT-4 (90.8%), and DeepSeek (90.3%), with no significant differences between systems ( p > 0.05). Claude showed the highest reliability (ICC = 0.931), followed by GPT-4 (ICC = 0.882). Complete accuracy and reproducibility (100%) were detected in Histological Methods, Blood and Hemopoiesis, and Circulatory System, while Muscle tissue (76.0%) and Lymphoid System (84.7%) presented the greatest challenges. LLMs demonstrate exceptional accuracy and reliability in answering histological MCQs, significantly outperforming other medical disciplines. Minimal inter-system variability suggests technological maturity, though topic-specific challenges and reliability concerns indicate the continued need for human expertise. These findings reflect rapid AI advancement and identify histology as particularly suitable for AI-assisted medical education. Clinical trial number : The clinical trial number is not pertinent to this study as it does not involve medicinal products or therapeutic interventions
Mathematics education · Medical education · Medical physics · AI in cancer detection · Artificial Intelligence in Healthcare and Education · Computer Science · Medicine · Psychology · Radiomics and Machine Learning in Medical Imaging
How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment
From human writing to artificial intelligence generated text
Instructional Design Made Easy! Instructional Design Models, Categories, Frameworks, Educational Context, and Recommendations for Future Work
| Citations per year | 2 |
|---|---|
| Citation span | 2025 - 2026 (2) |
| Highly cited | No |