Utility of large language models for creating clinical assessment items
Bibliographic Data
| ID | 21448939 |
|---|---|
| Authors | George Lam (0000-0001-6011-0154, Imperial College London), Yusra Shammoon (0000-0003-2630-4443, Imperial College London), Anna Coulson (0009-0007-3451-579X, Imperial College London), Felicity Lalloo (0000-0002-5698-4120, Imperial College London), Arti Maini (0000-0002-0951-5604, Imperial College London), Anjali Amin (0000-0002-0416-6505, Imperial College London), Celia Brown (0000-0002-7526-0793, Imperial College London), Amir H Sam (0000-0002-9599-9069, Imperial College London, corresponding author) |
| Year | 2025 |
| Volume | 47 |
| Issue | 5 |
| Pages | 878-882 |
| Publication date | 2025-05-04 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Medical Teacher (JOURNAL) |
| Journal identifiers | ISSN: 0142-159X • E-ISSN: 1466-187X |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/0142159x.2024.2382860 |
| PMID | 39186054 |
| OpenAlex | W4401883137 |
| Language | EN |
| Citations received | 3 |
| References cited | 9 |
PURPOSE: To compare student performance, examiner perceptions and cost of GPT-assisted (generative pretrained transformer-assisted) clinical and professional skills assessment (CPSAs) items against items created using standard methods. METHODS: We conducted a prospective, controlled, double-blinded comparison of CPSA items developed using GPT-assistance with those created through standard methods. Two sets of six practical cases were developed for a formative assessment sat by final year medical students. One clinical case in each set was created with GPT-assistance. Students were assigned to one of the two sets. RESULTS: = 15) of respondents to an examiner feedback questionnaire felt GPT-assisted cases were appropriately difficult and realistic. GPT-assistance resulted in significant labour cost savings, with a mean reduction of 57% (880 GBP) in labour cost per case when compared to standard case drafting methods. CONCLUSIONS: GPT-assistance can create CPSA items of comparable quality with significantly less cost when compared to standard methods. Future studies could evaluate GPT's ability to create CPSA material in other areas of clinical practice, aiming to validate the generalisability of these findings
Generative grammar · Medical education · Transformer · Artificial Intelligence · Computer Science · Engineering · Innovations in Medical Education · Medicine · Psychology · Psychometric Methodologies and Testing · Student Assessment and Feedback
| Unique citing works | 3 |
|---|---|
| Citations per year | 3 |
| Citation span | 2025 - 2026 (2) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 3 |