Do entrustment scales make a difference in the inter-rater reliability of the workplace-based assessment
Bibliographic Data
| ID | 15328339 |
|---|---|
| Authors | Ayat Eltayar (0000-0001-7177-1126, Alexandria University, corresponding author), Soha Rashed Aref (Alexandria University), Hoda Khalifa (0000-0003-1842-8286, Alexandria University), Hoda Mahmoud Khalifa (Alexandria University), Abdullah Hammad (0000-0002-3058-1817, Alexandria University), Abdullah Said Hammad (Alexandria University) |
| Year | 2022 |
| Volume | 27 |
| Issue | 1 |
| Publication date | 2022-03-21 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Medical Education Online (JOURNAL) |
| Journal identifiers | ISSN: 1087-2981 • E-ISSN: 1087-2981 |
| Publisher | Taylor & Francis (PUBLISHER • GB) |
| DOI | 10.1080/10872981.2022.2053401 |
| OpenAlex | W4220703428 |
| Language | EN |
| References cited | 10 |
Background A workplace-based assessment (WBA) is used to assess learners’ competencies in their workplaces. Many workplace assessment tools are available and validated to assess various constructs. The implementation of workplace-based assessment requires proper training of the staff.Objective This study aimed to explore the impact of staff training on WBA practices and evaluate the inter-rater reliability of these practices while using entrustment scales, performance descriptors, and personal judgment.Design A quasi-experimental study, in which the staff members of the orthopedic department were invited to participate in a training program on the use of entrustment scales and assessment descriptors within the WBA tools. As a response to the training, subjective judgment was replaced by entrustment scales and performance descriptors in a trauma course offered by the orthopedic department. The inter-rater reliability of the WBA was evaluated using various rating scales.Results The entrustment scales had higher inter-rater reliability of the assessment tools than performance descriptors and the personal judgment.Conclusion The inter-rater reliability was highest when using entrustment scales for WBAs, which could indicate that the entrustment scales achieve good psychometric properties as regards consistency among different raters. Thus, they decrease the confounding effect of differences in assessors. They may also give a clearer image of the actual academic level of the learners
Consistency (knowledge bases · Inter-rater reliability · Medical education · Rating scale · Reliability (semiconductor · Scale (ratio · Applied Psychology · Clinical Reasoning and Diagnostic Skills · Computer Science · Innovations in Medical Education · Medical Education and Admissions · Medicine · Psychology · Artificial Intelligence
| Highly cited | No |
|---|