Concept map analysis of reading comprehension assessment
Tracing situation models in multiple-choice and constructed-response items
Bibliographic Data
| ID | 21067754 |
|---|---|
| Authors | Erik Roelofs (0000-0003-1797-7451, corresponding author), Mohammadreza Farrokhnia (0000-0002-0150-5372), Maryam Amir Haeri (0000-0002-9638-9481) |
| Year | 2026 |
| Volume | 89 |
| Pages | 101599 |
| Publication date | 2026-06-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Studies In Educational Evaluation (JOURNAL) |
| Journal identifiers | ISSN: 0191-491X • E-ISSN: 1879-2529 |
| Publisher | Elsevier BV (PUBLISHER) |
| DOI | 10.1016/j.stueduc.2026.101599 |
| OpenAlex | W7147559868 |
| Language | EN |
| References cited | 44 |
Reading comprehension assessments are often criticized for focusing on discrete text segments rather than on deeper comprehension processes such as situation-model construction. This exploratory study investigated whether concept map analysis can be used to evaluate how well comprehension items reflect the conceptual structure of an expert situation model underlying a text. The study was not intended as a generalizable empirical evaluation of existing items, but as a methodological demonstration of an analytic approach. Accordingly, 18 selected-response items from Dutch vocational education reading exams were analyzed using expert-constructed concept maps as proxies for situation models. For each item, the number, implicitness, and centrality of required concepts were examined to estimate the extent to which items engage the underlying conceptual structure. In addition, three constructed-response versions of selected items were developed and compared with their selected-response counterparts. Results showed that most selected-response items addressed only small portions of the concept maps, typically less than 10% of the mapped concepts, although summarization items showed somewhat broader coverage. The constructed-response versions engaged substantially larger portions of the maps than the selected-response items. Exploratory correlational analyses further indicated that items requiring more concepts, particularly more central concepts, tended to be more difficult. These findings suggest that concept map analysis provides a useful method for examining construct representation in reading comprehension assessment and for informing the design of items that better capture deeper comprehension processes
Comprehension · Concept learning · Concept map · Reading comprehension · Tracing · Educational and Psychological Assessments · Psychometric Methodologies and Testing · Reading and Literacy Development
Pisa 2018 Assessment and Analytical Framework
Constructing inferences during narrative text comprehension.
A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment
Chapter 9 Toward a Comprehensive Model of Comprehension
Implications of Evidence‐Centered Design for Educational Testing
Somatotopic Representation of Action Words in Human Motor and Premotor Cortex
The role of knowledge in discourse comprehension
Centering predictor variables in cross-sectional multilevel models
Roles, applications, and trends of concept map-supported learning
The automated model of comprehension version 4.0 – Validation studies and integration of ChatGPT
The Roles of Initial Mathematics, Reading, and Cognitive Skills in Subsequent Mathematics Performance
Inference skills for reading
A Coefficient of Agreement for Nominal Scales
How assessing reading comprehension with multiple-choice questions shapes the construct
Inter-Coder Agreement for Computational Linguistics
The Effects of Surface Structure Variables on Performance in Reading Comprehension Tests
The Contribution of General Reading Ability to Science Achievement
Determining the Passage Dependency of Comprehension Questions in 5 Major Tests
A Twelve-Year Longitudinal Study of Science Concept Learning
The Subskills of Reading
Cognitive Structure and the Facilitation of Meaningful Verbal Learning1
| Highly cited | No |
|---|