Rater Connectedness Affects Student Achievement Estimates and Ordered Rankings in Formal Music Performance Assessments
Bibliographic Data
| ID | 10201504 |
|---|---|
| Authors | Stefanie A Wind (0000-0002-1599-375X, University of Alabama, Tuscaloosa, AL, USA, corresponding author), Brian C Wesolowski (0000-0003-0615-9277, University of Georgia, Athens, GA, USA), Eli Jones (0000-0001-9273-9627), Eli A Jones (0000-0002-0320-6341, University of Memphis, Memphis, TN, USA) |
| Year | 2025 |
| Volume | 74 |
| Issue | 1 |
| Pages | 110-131 |
| Publication date | 2025-09-28 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Journal of Research in Music Education (JOURNAL) |
| Journal identifiers | ISSN: 0022-4294 • E-ISSN: 1945-0095 |
| Publisher | SAGE Publishing (PUBLISHER • US) |
| DOI | 10.1177/00224294251363327 |
| OpenAlex | W7083623986 |
| Language | EN |
| References cited | 27 |
A persistent concern in music performance assessments is the quality of ratings assigned by judges. Differences in rater judgment are most concerning in cases when student performances are scored by different raters (i.e., disconnected rating designs)—bringing into question the comparability of scores between raters and students. We used data from a formal solo music performance assessment to demonstrate and explore the impact of different data collection designs and a statistical adjustment procedure on the estimates and rank-ordering of student performances. Our results indicated notable discrepancies in conclusions about individual student performances between conditions where all raters scored all students, designs with no common performances between raters, designs that included overlapping performances between raters, and the results from a post hoc adjustment procedure for disconnected designs. We discuss the implications of our results for the design and interpretation of music performance assessments
Academic achievement · Comparability · Data collection · Inter-rater reliability · Interpretation (philosophy · Music education · Quality (philosophy · Social connectedness · Student achievement · Agriculture Sustainability and Environmental Impact · Agriculture, Water, and Health · Bioeconomy and Sustainability Development
Applying the Rasch Model
Test Equating, Scaling, and Linking
Three Generations of DIF Analyses
The Science of Training and Development in Organizations
Music Criticism Reconsidered
“Don’t forget Shakespeare”
Rater fairness in music performance assessment
A many-facet Rasch analysis of the second language group oral discussion task
The Effect of Music Contest Format on Self-Concept, Motivation, Achievement, and Attitude of Elementary Band Students
The Development of a Secondary-Level Solo Wind Instrument Performance Rubric Using the Multifaceted Rasch Partial Credit Measurement Model
Validity of psychological assessment
| Citation velocity | historical |
|---|---|
| Highly cited | No |