Cognitive Validity
Can Multiple-Choice Items Tap Historical Thinking Processes
Bibliographic Data
| ID | 8223447 |
|---|---|
| Authors | Mark Smith (0000-0002-1048-7786, Stanford University, corresponding author), Mark D Smith (0000-0002-4188-2324, Stanford University) |
| Year | 2017 |
| Volume | 54 |
| Issue | 6 |
| Pages | 1256-1287 |
| Publication date | 2017-12-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | American Educational Research Journal (JOURNAL) |
| Journal identifiers | ISSN: 0002-8312 • E-ISSN: 1935-1011 |
| Publisher | American Educational Research Association (AERA) (PUBLISHER) |
| DOI | 10.3102/0002831217717949 |
| OpenAlex | W2760153335 |
| Language | EN |
| Citations received | 8 |
| References cited | 45 |
Cognitive validity examines the relationship between what an assessment aims to measure and what it actually elicits from test takers. The present study examined whether multiple-choice items from the National Assessment of Educational Progress (NAEP) grade 12 U.S. history exam elicited the historical thinking processes they were designed to measure. Think-aloud data from 27 accomplished high school students revealed that in no instances did students engage in the intended processes. Rather, the items typically elicited three construct-irrelevant processes: factual recall/recognition, reading comprehension, and test-taking strategies. Further, findings revealed that although the items often prompted students to engage in factual recall, they were often not sound indicators of student knowledge. Implications for history testing and alternative forms of assessment are discussed
Cognition · Cognitive psychology · Cognitive science · Comprehension · Construct (python library · Construct validity · Developmental psychology · Historical Thinking · Multiple choice · Pedagogy · Protocol analysis · Psychometrics · Reading (process · Reading comprehension · Recall · Test (biology · Test validity · Think aloud protocol · Computer Science · Educational Assessment and Pedagogy · Educational Strategies and Epistemologies · Educator Training and Historical Pedagogy · Psychology · Social Psychology
Academic majors of social studies teachers and student achievement in the U.S
Assessing What Matters in Civics
New Multiple-Choice Measures of Historical Thinking
U.S. history state assessment items
How do we know what they know? A case study of classroom-based assessment with multilingual learners
A Sociological Lens on Linguistic Diversity
Improving university students' web savvy
Illuminating the Black Hole
Protocol Analysis
Analysis of Expert Readers in Three Disciplines
Are We Asking the Right Questions? Using Cognitive Interviews to Improve Surveys in Education Research
Validating the Interpretations and Uses of Test Scores
Writing Multiple-Choice Items to Measure Higher-Order Educational Objectives
New Directions in Assessing Historical Thinking
Teaching History in the Digital Age
Assessing Historical Thinking and Understanding
Teaching Disciplinary Literacy to Adolescents
Historical problem solving
The real test bias
Let's Improve Multiple-Choice Tests
Try, Try, Try Again
Testing Historical Knowledge
Uncoverage
The History Learning Project
Three faces of intellect
Validity of psychological assessment
Reading Abraham Lincoln
A Deafening Silence
| Unique citing works | 8 |
|---|---|
| Citations per year | 1 |
| Citation span | 2018 - 2026 (9) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 8 |