Common Structural Design Features of Rubrics May Represent a Threat to Validity
Bibliographic Data
| ID | 9732191 |
|---|---|
| Authors | Stephen Mark Humphry (The University of Western Australia, Crawley, WA), Stephen Humphry (The University of Western Australia, corresponding author), Sandra Allison Heldsinger (The University of Western Australia, Crawley, WA), Sandra Heldsinger (The University of Western Australia) |
| Year | 2014 |
| Volume | 43 |
| Issue | 5 |
| Pages | 253-263 |
| Publication date | 2014-06-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Educational Researcher (JOURNAL) |
| Journal identifiers | ISSN: 0013-189X • E-ISSN: 1935-102X |
| Publisher | American Educational Research Association (AERA) (PUBLISHER) |
| DOI | 10.3102/0013189x14542154 |
| OpenAlex | W2081387623 |
| Language | EN |
| Citations received | 17 |
| References cited | 26 |
Rubrics for assessing student performance are often seen as providing rich information about complex skills. Despite their widespread usage, however, little empirical research has focused on whether it is possible for rubrics to validly meet their intended purposes. The authors examine a rubric used to assess students’ writing in a large-scale testing program. They present empirical evidence for the existence of a potentially widespread threat to the validity of rubric assessments that arose due to design features. In this research, an iterative tryout-redesign-tryout approach was adopted. The research casts doubt on whether rubrics with structurally aligned categories can validly assess complex skills. A solution is proposed that involves rethinking the structural design of the rubric to mitigate the threat to validity. Broader implications are discussed
Empirical research · Mathematics education · Rubric · Statistics · Computer Science · Evaluation of Teaching Practices · Mathematics · Psychology · Psychometric Methodologies and Testing · Student Assessment and Feedback
Pedagogical mediation with ICT for the development of critical thinking in primary education
Agnosticism in instructional observation systems
Using global observation protocols to inform research on teaching effectiveness and school improvement
Challenges and opportunities for classroom-based formative assessment and AI
A Two-Stage Method for Obtaining Reliable Teacher Assessments of Writing
Judging similarity versus judging difference
Formative Assessment of Social-Emotional Skills Using Rubrics
Relationship between Students’ Perception of a Rubric for Oral Presentations and Their Academic Characteristics
Mark scheme design for school- and college-based assessment in VTQs
How dominant language influences rubric reading and task performance
What’s in a score? Problematizing interpretations of observation scores
Development and psychometric properties of rubrics for assessing social-emotional skills in youth
Effects of rubric quality on marker variation in higher education
Rubricizing the assessment practice
Same grade for different reasons, different grades for the same reason
Analysis of online rubric platforms
Improving assessment of procedural skills in health sciences education
The use of scoring rubrics
The Concept of Validity.
A Rasch Model for Partial Credit Scoring
A constant error in psychological ratings.
A framework relating outcomes based education and the taxonomy of educational objectives
A review of rubric use in higher education
Indeterminacy in the use of preset criteria for assessment and grading
A rating formulation for ordered response categories
The Interplay of Evidence and Consequences in the Validation of Performance Assessments
What Good Is Polarizing Research Into Qualitative and Quantitative
Assessment of Complex Performances
Meaning and Values in Test Validation
| Unique citing works | 17 |
|---|---|
| Citations per year | 2,83 |
| Citation span | 2020 - 2026 (7) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 17 |