Modeling the interaction between resilience and ability in assessments with allowances for multiple attempts
Bibliographic Data
| ID | 21563967 |
|---|---|
| Authors | Susu Zhang (0000-0003-0751-6467, University of Illinois Urbana-Champaign, corresponding author), Yoav Bergner (0000-0001-7738-4290, Department of Administration), Jack DiTrapani (Munich Re (United States)), Moongee Jeon (0000-0002-5880-4146, University of California, Los Angeles) |
| Year | 2021 |
| Volume | 122 |
| Pages | 106847 |
| Publication date | 2021-09-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Computers in Human Behavior (JOURNAL) |
| Journal identifiers | ISSN: 0747-5632 • E-ISSN: 1873-7692 |
| Publisher | Elsevier BV (PUBLISHER) |
| DOI | 10.1016/j.chb.2021.106847 |
| OpenAlex | W3158262759 |
| Language | EN |
| Citations received | 3 |
| References cited | 48 |
Many computer-based assessments, in particular those administered in learning settings, provide immediate item-level feedback and permit respondents to retry questions following incorrect responses. Doing so supports learning and facilitates the measurement of partial mastery. However, persistence through attempts may introduce new factors, such as resilience, affecting a student's test score. The current study proposes a tree-based approach to jointly model item responses and reattempt decisions in multiple-attempt assessments, allowing for the separate measurement of latent ability and reattempt propensity, or resilience. The proposed model is implemented on two datasets collected from online homework assignments, one from a university for-credit course and the other from a massive open online course . The results shed light on the relationship between resilience and ability both at the individual level and at the item level, where threshold parameters can be interpreted as difficulty and repeatability. Importantly, resilience was found to affect certain commonly used performance scores, such as eventual correctness proportion, posing a validity threat for summative assessment use-cases. The tree-based model not only resolves the problem but also opens the door to ecologically valid measures of resilience that are not reliant on self-reports
Cognitive psychology · Computer Science · Grit, Self-Efficacy, and Motivation · Innovative Teaching and Learning Methods · Online Learning and Analytics · Psychology · Social Psychology
Statistical Theories of Mental Test Scores
Multidimensional Item Response Theory
Feedback in written instruction
The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems
The relation of strength of stimulus to rapidity of habit‐formation
Development of a new resilience scale
Focus on Formative Feedback
Expectancy–Value Theory of Achievement Motivation
Immediate Feedback and Opportunity to Revise Answers to Open-Ended Questions
Has student learning been improved by the use of online and offline formative assessment opportunities
The Mass Implementation and Evaluation of Computer‐based Assessments
Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being
Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being
Affordance, conventions, and design
Immediate Feedback during Academic Testing
Meaning and Values in Test Validation
Grit
The Power of Feedback
Anxiety in academic achievement situations
| Unique citing works | 3 |
|---|---|
| Citations per year | 0.75 |
| Citation span | 2022 - 2025 (4) |
| Citation velocity | recent |
| Highly cited | No |
| Citation types | Neutral: 3 |