Not Just Generalizability
A Case for Multifaceted Latent Trait Models in Teacher Observation Systems
Bibliographic Data
| ID | 9735556 |
|---|---|
| Authors | Stefanie A Wind (0000-0002-1599-375X, The University of Alabama, Tuscaloosa, AL, corresponding author), Eli Jones (0000-0001-9273-9627), Eli A Jones (0000-0002-0320-6341, Columbus State University, Columbus, GA) |
| Year | 2019 |
| Volume | 48 |
| Issue | 8 |
| Pages | 521-533 |
| Publication date | 2019-11-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Educational Researcher (JOURNAL) |
| Journal identifiers | ISSN: 0013-189X • E-ISSN: 1935-102X |
| Publisher | American Educational Research Association (AERA) (PUBLISHER) |
| DOI | 10.3102/0013189x19874084 |
| OpenAlex | W2973074601 |
| Language | EN |
| Citations received | 3 |
| References cited | 17 |
Teacher evaluation systems often include classroom observations in which raters use rating scales to evaluate teachers’ effectiveness. Recently, researchers have promoted the use of multifaceted approaches to investigating reliability using Generalizability theory, instead of rater reliability statistics. Generalizability theory allows analysts to quantify the contribution of multiple sources of variance (e.g., raters and tasks) to measurement error. We used data from a teacher evaluation system to illustrate another multifaceted approach that provides additional indicators of the quality of observational systems. We show how analysts can use Many-Facet Rasch models to identify and control for differences in rater severity, identify idiosyncratic ratings associated with various facets, and evaluate rating scale functioning. We discuss implications for research and practice in teacher evaluation
Developmental psychology · Facet (psychology · Generalizability theory · Inter-rater reliability · Item response theory · Observational study · Psychometrics · Quality (philosophy · Rasch model · Rating scale · Reliability (semiconductor · Scale (ratio · Statistics · Test theory · Trait · Variance (accounting · Variance components · Applied Psychology · Clinical Psychology · Computer Science · Educational Assessment and Improvement · Psychology · School Choice and Performance · Social Psychology
Applying the Rasch Model
A Rasch Model for Partial Credit Scoring
An Argument Approach to Observation Protocol Validity
Teacher evaluation
Trends in Classroom Observation Scores
Effect of Observation Mode on Measures of Secondary Mathematics Teaching
A systematic review of methods for evaluating rating quality in language assessment
The Rasch wars
Building a More Complete Understanding of Teacher Evaluation Using Classroom Observations
When Rater Reliability Is Not Enough
| Unique citing works | 3 |
|---|---|
| Citations per year | 0,6 |
| Citation span | 2021 - 2026 (6) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 3 |