Validating a partial-credit scoring approach for multiple-choice science items
An application of fundamental ideas in science
Bibliographic Data
| ID | 21589367 |
|---|---|
| Authors | Xiaoming Zhai (0000-0003-4519-1931, University of Georgia, corresponding author), Min Li (0000-0002-1163-8961, University of Washington) |
| Year | 2021 |
| Volume | 43 |
| Issue | 10 |
| Pages | 1640-1666 |
| Publication date | 2021-07-03 |
| Peer Reviewed | Yes |
| Open Access | No |
| Type | ARTICLE |
| Venue | International Journal of Science Education, Part B (JOURNAL) |
| Journal identifiers | ISSN: 2154-8455 • E-ISSN: 2154-8463 |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/09500693.2021.1923856 |
| OpenAlex | W3159989888 |
| Language | EN |
| Citations received | 3 |
| References cited | 60 |
This study provides a partial-credit scoring (PCS) approach to awarding students’ performance on multiple-choice items in science education. The approach is built on fundamental ideas, the critical pieces of students’ understanding and knowledge to solve science problems. We link each option of the items to several specific fundamental ideas to capture their mastery patterns when an option is selected. Using these mastery patterns to order the options of items and accordingly assign credits to students, we measure students’ cognitive proficiency without including additional measures (e.g. times of trial) to the test or requiring extra support (e.g. technology). Using many-facet Rasch analysis, we find that the ordered options students selected were aligned with their ability measures. The PCS yields robust psychometric quality; compared to the dichotomous scoring of multiple-choice items, it generates better item fit and separation parameters. Besides, this PCS approach helps address construct validity by modelling student responses at the option level to reflect students’ mastery of fundamental ideas
Big Five personality traits · Cognition · Cognitive psychology · Data mining · Developmental psychology · Item response theory · Mathematics education · Multiple choice · Polytomous Rasch model · Psychometrics · Rasch model · Science education · Significant difference · Statistics · Computer Science · Mathematics · Psychology · Psychometric Methodologies and Testing · Reliability and Agreement in Measurement · Social Psychology · Student Assessment and Feedback
A Framework of Construct-Irrelevant Variance for Contextualized Constructed Response Assessment
Employing a Force and Motion Learning Progression to Investigate the Relationship between Task Characteristics and Students’ Conceptions at Different Levels of Sophistication
Assessing high‐school students' modeling performance on Newtonian mechanics
A Rasch Model for Partial Credit Scoring
Tracing physics content knowledge gains using content complexity levels
On the Use of Objective Examinations
Ability to Avoid Gross Error as a Measure of Achievment
Evaluation of construct-irrelevant variance yielded by machine and human scoring of a science teacher PCK constructed response assessment
Scoring methods for multiple choice assessment in higher education – Is it still a matter of number right scoring or negative marking
Changing Conceptual Change
Some Modifications of the Multiple-Choice Item
A Comparison of Empirical Differential Option Weighting Scoring Procedures as a Function of Inter-Item Correlation
The Assessment of Partial Knowledge
Reliability and Validity of Item Option Weighting Schemes
The Effect on Test Reliability and Validity of Scoring Aptitude and Achievement Tests With Weights for Every Choice
Reducing the need for guesswork in multiple-choice tests
A rating formulation for ordered response categories
Construct validity in psychological tests
| Unique citing works | 3 |
|---|---|
| Citations per year | 0,6 |
| Citation span | 2021 - 2023 (3) |
| Citation velocity | historical |
| Highly cited | No |
| Citation types | Neutral: 3 |