A randomised cross-over trial assessing the impact of AI-generated individual feedback on written online assignments for medical students
Bibliographic Data
| ID | 21448801 |
|---|---|
| Authors | Leon Nissen (0009-0009-3884-0850, Institute for Digital Medicine, University Hospital Bonn), Johanna Flora Rother (0009-0004-3526-3211, Institute of Medical Education, University Hospital Bonn, corresponding author), Marie Heinemann (0009-0001-1402-0876, Institute for Digital Medicine, University Hospital Bonn), Lara Marie Reimer (0000-0002-6546-8531, Institute for Digital Medicine, University Hospital Bonn), Stephan Jonas (0000-0002-3687-6165, Institute for Digital Medicine, University Hospital Bonn), Tobias Raupach (0000-0003-2555-8097, Institute of Medical Education, University Hospital Bonn) |
| Year | 2025 |
| Volume | 47 |
| Issue | 9 |
| Pages | 1544-1550 |
| Publication date | 2025-09-02 |
| Peer Reviewed | Yes |
| Open Access | No |
| Type | ARTICLE |
| Venue | Medical Teacher (JOURNAL) |
| Journal identifiers | ISSN: 0142-159X • E-ISSN: 1466-187X |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/0142159x.2025.2451870 |
| PMID | 39831699 |
| OpenAlex | W4406620367 |
| Language | EN |
| Citations received | 7 |
| References cited | 17 |
Purpose Self-testing has been proven to significantly improve not only simple learning outcomes, but also higher-order skills such as clinical reasoning in medical students. Previous studies have shown that self-testing was especially beneficial when it was presented with feedback, which leaves the question whether an immediate and personalized feedback further encourages this effect. Therefore, we hypothesised that individual feedback has a greater effect on learning outcomes, compared to generic feedback.Materials and methods In a randomised cross-over trial, German medical students were invited to voluntarily answer daily key-feature questions via an App. For half of the items they received a generalised feedback by an expert, while the feedback on the other half was generated immediately through ChatGPT. After the intervention, the students participated in a mandatory exit exam.Results Those participants who used the app more frequently experienced a better learning outcome compared to those who did not use it frequently, even though this finding was only examined in a correlative nature. The individual ChatGPT generated feedback did not show a greater effect on exit exam scores compared to the expert comment (51.8 ± 22.0% vs. 55.8 ± 22.8%; p = 0.06).Conclusion This study proves the concept of providing personalised feedback on medical questions. Despite the promising results, improved prompting and further development of the application seems necessary to strengthen the possible impact of the personalised feedback. Our study closes a research gap and holds great potential for further use not only in medicine but also in other academic fields
Medical education · Artificial Intelligence in Healthcare and Education · Clinical Reasoning and Diagnostic Skills · Innovations in Medical Education · Medicine · Psychology
The impact of generative artificial intelligence on clinical skills and knowledge acquisition in medical undergraduates
A scoping review of the use of generative artificial intelligence tools in health profession education
The impact of integrating generative artificial intelligence into medical education on short-term learning outcomes
Comparison of AI ‐Generated and Instructor Feedback
Development and validation of a GPT-based rater for assessing communication skills using the Gap-Kalamazoo Communication Skills Assessment Form
Using large language model to aid in teaching medical imaging report writing
Impact of AI-generated individual feedback on written online assignments for medical students
| Unique citing works | 7 |
|---|---|
| Citations per year | 7 |
| Citation span | 2025 - 2026 (2) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 7 |