Utility of a multimodal computer-based assessment format for assessment with a higher degree of reliability and validity

Datos Bibliográficos

ID21448895
AutoresJohan Renes (0000-0003-3758-7602, Maastricht University, autor de correspondencia), Cees Van Der Vleuten (0000-0001-6802-3119, Maastricht University), Carlos Fernando Collares (0000-0003-0914-3430, Maastricht University)
Año2023
Volumen45
Número4
Páginas433-441
Fecha de publicación2023-04-03
Open AccessSí
TipoARTICLE
RevistaMedical Teacher (JOURNAL)
Identificadores de la revistaISSN: 0142-159X • E-ISSN: 1466-187X
EditorialInforma UK Limited (PUBLISHER • GB)
DOI10.1080/0142159x.2022.2137011
PMID36306368
OpenAlexW4307475379
IdiomaEN
Citas recibidas1
Referencias citadas44

Multiple choice questions (MCQs) suffer from cueing, item quality and factual knowledge testing. This study presents a novel multimodal test containing alternative item types in a computer-based assessment (CBA) format, designated as Proxy-CBA. The Proxy-CBA was compared to a standard MCQ-CBA, regarding validity, reliability, standard error of measurement, and cognitive load, using a quasi-experimental crossover design. Biomedical students were randomized into two groups to sit a 65-item formative exam starting with the MCQ-CBA followed by the Proxy-CBA (group 1, n = 38), or the reverse (group 2, n = 35). Subsequently, a questionnaire on perceived cognitive load was taken, answered by 71 participants. Both CBA formats were analyzed according to parameters of the Classical Test Theory and the Rasch model. Compared to the MCQ-CBA, the Proxy-CBA had lower raw scores (p < 0.001, η2 = 0.276), higher reliability estimates (p < 0.001, η2 = 0.498), lower SEM estimates (p < 0.001, η2 = 0.807), and lower theta ability scores (p < 0.001, η2 = 0.288). The questionnaire revealed no significant differences between both CBA tests regarding perceived cognitive load. Compared to the MCQ-CBA, the Proxy-CBA showed increased reliability and a higher degree of validity with similar cognitive load, suggesting its utility as an alternative assessment format

Cognition · Developmental psychology · Formative assessment · Mathematics education · Multiple choice · Proxy (statistics) · Rasch model · Raw data · Raw score · Reliability (semiconductor) · Reliability engineering · Statistics · Clinical Psychology · Clinical Reasoning and Diagnostic Skills · Computer Science · Engineering · Innovations in Medical Education · Mathematics · Psychology · Radiology practices and education