Comparison of cognitive workload between very short answer questions and multiple-choice questions
An eye-tracking experiment
Bibliographic Data
Very short answer questions (VSAQs) have gained attention for their superior psychometric properties compared to multiple-choice questions (MCQs). While VSAQs require knowledge recall, MCQs primarily involve knowledge recognition. This difference in cognitive processes may lead to varying cognitive workloads, defined as the amount of mental processing in working memory. Previous studies have not demonstrated consistent differences, likely due to reliance on self-reported measures. Eye tracking provides objective, process-level indicators of cognitive workload. This study investigated whether answering VSAQs requires a higher cognitive workload than answering MCQs. In a within-subject randomized crossover experiment, sixth-year medical students answered both VSAQs and MCQs. Cognitive workload was measured using screen-based eye tracking, focusing on the number of fixations and revisitations as objective indicators of mental effort. Data were analyzed using mixed-effects models. Thirty-four medical students participated, yielding 1,326 observations, which is the multiplication of the number of students by the number of questions (39 questions). Mixed-effects models showed a significant effect of question type on both workload indicators: VSAQs elicited more fixations and revisitations than MCQs (β_std = 0.30-0.39, p p < .01). Heatmaps confirmed these findings, showing denser fixations on key diagnostic features for VSAQs and on answer options for MCQs. Answering VSAQs imposed a higher cognitive workload than MCQs. The presence of answer options in MCQs may reduce workload by providing unintentional cues, while VSAQs require active retrieval. Eye tracking proved valuable for distinguishing cognitive workload across assessment formats
Cognition · Cognitive Load · Cognitive test · Eye movement · Eye tracking · Tracking (education · Workload · Clinical Reasoning and Diagnostic Skills · Educational Strategies and Epistemologies · Visual and Cognitive Learning Processes
The Critical Importance of Retrieval for Learning
A review of using eye-tracking technology in exploring learning from 2000 to 2012
Cognitive Load Theory
Desirable difficulties in theory and practice.
The Power of Testing Memory
Fitting Linear Mixed-Effects Models Using lme4
Utility of a multimodal computer-based assessment format for assessment with a higher degree of reliability and validity
Use of eye tracking in medical education
Identifying the response process validity of clinical vignette-type multiple choice questions
Testing (quizzing) boosts classroom learning
A power primer
| Citation velocity | historical |
|---|---|
| Highly cited | No |