An Analysis of the Ratings and Interrater Reliability of High School Band Contests
Bibliographic Data
| ID | 10199481 |
|---|---|
| Authors | Phillip M Hash (0000-0002-3384-4715, Calvin College, Grand Rapids, MI, USA, corresponding author) |
| Year | 2012 |
| Volume | 60 |
| Issue | 1 |
| Pages | 81-100 |
| Publication date | 2012-02-16 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Journal of Research in Music Education (JOURNAL) |
| Journal identifiers | ISSN: 0022-4294 • E-ISSN: 1945-0095 |
| Publisher | SAGE Publishing (PUBLISHER • US) |
| DOI | 10.1177/0022429411434932 |
| OpenAlex | W1973227215 |
| Language | EN |
| Citations received | 10 |
| References cited | 31 |
The purpose of this study was to examine procedures for analyzing ratings of large-group festivals and provide data with which to compare results from similar events. Data consisted of ratings from senior division concert band contests sponsored by the South Carolina Band Directors Association from 2008 to 2010. Three concert-performance and two sight-reading judges evaluated each band to determine a final rating. Research questions examined (a) frequency distributions of ratings; (b) interrater reliability as measured by Spearman correlation of individual judges’ ratings, internal consistency (α), and two forms of interrater agreement (IRA); and (c) differences in mean ratings among individual adjudicators, contest sites, years, and classifications. The average final rating for all bands ( N = 353) was 1.73, with 86.7% ( n = 306) earning a I/Superior or II/Excellent. Interrater correlation, IRA, and internal consistency were higher for sight-reading versus concert performance. Each of these measures rose above the .80 benchmark for good reliability, except interrater correlation and average pairwise IRA in the concert portion of the contest. Data indicated significant differences in 8 out of 18 judging panels, in contest sites in 2010, and among ensemble classifications. In this study, the author demonstrated an effective procedure for analyzing ratings of large-group festivals and identified implications for improving these events
Consistency (knowledge bases · CONTEST · Correlation · Developmental psychology · Inter-rater reliability · Internal consistency · Pairwise comparison · Psychometrics · Rating scale · Reliability (semiconductor · Statistics · Diverse Music Education Insights · Mathematics · Music History and Culture · Neuroscience and Music Perception · Psychology
“Failing in spite of wonderfulness”
Beyond the Music
A PRISMA Review of Large Ensemble Adjudicated Performance Literature
An Examination of Differential Item Functioning in a Rubric to Assess Solo Music Performance
Rater fairness in music performance assessment
Effects of Ensemble Size and Repertoire Difficulty on Ratings of Concert Band Performances
A Comparison of Virginia Band Performance Assessments in Relation to Director Gender
Relationships Among Band Festival Ratings, Director Gender, and Other Selected Factors
Large Group Performance Evaluation in the United States
A Theoretical Structure of High School Concert Band Performance
Content analysis
Reliability and Validity Assessment
Averaging correlation coefficients
Judge-Group Differences in the Rating of Secondary School Trumpet Performances
Influence of Selected Variables on Solo and Small-Ensemble Festival Ratings
Criteria-Specific Rating Scales in the Evaluation of High School Instrumental Performance
Measurement of Instrumental and Vocal Undergraduate Performance Juries Using a Multidimensional Assessment Rubric
Reliability and Perceived Pedagogical Utility of a Weighted Music Performance Assessment Rubric
Effects of Authority Figure Biases on Changing Judgments of Musical Events
Performer, Rater, Occasion, and Sequence as Sources of Variability in Music Performance Assessment
An Examination of the Reliabilities of Two Choral Festival Adjudication Forms
Internal Consistency of Performance Evaluations as a Function of Music Expertise and Excerpt Familiarity
Stability of a Model Explaining Selected Extramusical Influences on Solo and Small-Ensemble Festival Ratings
Effects of Special Education Labels on Peers' and Adults' Evaluations of a Handicapped Youth Choir
Conductor and Ensemble Performance Expressivity and State Festival Ratings
The Effect of Conductor Expressivity on Ensemble Performance Evaluation
Faculty Interjudge Reliability of Music Performance Evaluation
Selected Influences on Solo and Small-Ensemble Festival Ratings
| Unique citing works | 10 |
|---|---|
| Citations per year | 0,91 |
| Citation span | 2015 - 2026 (12) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 10 |