Reliability and validity of simulation-based electrocardiogram assessment rubrics for cardiac life support skills among medical students using generalizability theory
Bibliographic Data
| ID | 15328288 |
|---|---|
| Authors | Sethapong Lertsakulbunlue (0000-0002-9349-2088, Phramongkutklao College of Medicine), Kaophiphat Thammasoon (Phramongkutklao College of Medicine), Anupong Kantiwong (0000-0003-1353-3650, Phramongkutklao College of Medicine, corresponding author) |
| Year | 2025 |
| Volume | 30 |
| Issue | 1 |
| Pages | 2479962-2479962 |
| Publication date | 2025-03-23 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Medical Education Online (JOURNAL) |
| Journal identifiers | ISSN: 1087-2981 • E-ISSN: 1087-2981 |
| Publisher | Taylor & Francis (PUBLISHER • GB) |
| DOI | 10.1080/10872981.2025.2479962 |
| PMID | 40122072 |
| OpenAlex | W4408756343 |
| Language | EN |
| References cited | 40 |
Simulation-based learning (SBL) is effective for EKG interpretation training in the advanced cardiac life support (ACLS) context, enhancing motivation, confidence, and learning outcomes. However, research on the psychometrics of assessment rubrics for ACLS skills among pre-clinical students is limited. This study investigates the validity and reliability of assessment rubrics for ACLS skills, including EKG interpretation, scenario and pharmacological management, and teamwork. An SBL course that integrates basic EKG interpretation into ACLS Stations was conducted at Phramongkutklao College of Medicine, utilizing high-fidelity mannequins to simulate realistic scenarios, enrolling 96 medical students. The course consisted of five independent stations, and each student was assessed once by two raters using ten-item assessment rubrics. The rubrics included three domains: (1) EKG and ACLS algorithm skills, (2) management and mechanisms of action, and (3) affective domains. Validity evidence on the content was gathered, and construct validity was confirmed with confirmatory factor analysis (CFA). Inter-rater and internal consistency reliability were calculated. Generalizability theory was utilized to analyse the data. Three expert reviews yielded an item-objective congruence index of 0.67-1.00, with iterative validation through alpha and beta tests. The CFA demonstrated a good fit, but two questions with loading factors below 0.30 were removed, resulting in an eight-item assessment form. An inter-rater correlation of 0.70 (p < 0.001) and a Cronbach's alpha of 0.76 was demonstrated. To achieve a Phi-coefficient ≥0.80, three raters and at least 10 items are required in a p×i×r crossed design. With eight items, r:(p×i) nested design reliability was 0.69, 0.79, and 0.83 for one, two, and three raters, respectively. While a single rater with 10 items achieved a Phi-coefficient of 0.74. The rubrics for assessing ACLS skills among pre-clinical students demonstrated acceptable validity and reliability. A condensed eight-item rubric with acceptable reliability is proposed as a practical tool for optimizing assessment in future evaluations relevant to the pre-clinical context
Generalizability theory · Mathematics education · Medical education · Psychometrics · Reliability (semiconductor · Reliability engineering · Rubric · Validity · Clinical Psychology · Computer Science · Emergency Medicine Education and Research · Engineering · Innovations in Medical Education · Medicine · Psychology · Simulation-Based Education in Healthcare
Introducing Lisrel
Generalizability Theory
Features and uses of high-fidelity medical simulations that lead to effective learning
Programmatic assessment
The use of scoring rubrics for formative assessment purposes revisited
G*Power 3
Reliable Assessment of Surgical Technical Skills Is Dependent on Context
Peer assessment in the digital age
Assessment of Clinical Competence
Impact of tailored feedback in assessment of communication skills for medical students
Having a fit
| Highly cited | No |
|---|