Enhancing generalizability theory with mixed‐effects models for heteroscedasticity in psychological measurement
A theoretical introduction with an application from EEG data
Bibliographic Data
| ID | 22410156 |
|---|---|
| Authors | Philippe Rast (0000-0003-3630-6629, Department of Psychology University of California Davis Davis California USA, corresponding author), Peter E Clayson (0000-0003-4437-6598, Department of Psychology University of South Florida Tampa Florida USA) |
| Year | 2026 |
| Publication date | 2026-01-18 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | British Journal of Mathematical and Statistical Psychology (JOURNAL) |
| Journal identifiers | ISSN: 0007-1102 • E-ISSN: 2044-8317 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1111/bmsp.70026 |
| PMID | 41549064 |
| OpenAlex | W7124727942 |
| Language | EN |
| References cited | 39 |
Generalizability theory (G‐theory) defines a statistical framework for assessing measurement reliability by decomposing observed variance into meaningful components attributable to persons, facets, and error. Classic G‐theory assumes homoscedastic residual variances across measurement conditions, an assumption that is often violated in psychological and behavioural data. The main focus of this work is to extend G‐theory using a mixed‐effects location‐scale model (MELSM) that allows residual error variance to vary systematically across conditions and persons. By modeling heteroscedasticity, we can extend the computation of condition‐specific generalizability () and dependability () coefficients to reflect local reliability under varying degrees of measurement precision. As an illustration, we apply the model to empirical data from an EEG experiment and show that failing to account for variance heterogeneity can mask meaningful differences in measurement quality. A simulation‐based decision study further demonstrates how targeted increases in measurement density can improve reliability for low‐precision conditions or participants. The proposed framework retains the interpretative character of classical G‐theory while enhancing its flexibility. We argue that it supports finer‐grained insights on conditions that influence reliability and better‐informed design decisions in psychological measurements. We discuss implications for individualized reliability assessment, adaptive measurement strategies, and future extensions to multi‐facet designs.
Dependability · Generalizability theory · Heteroscedasticity · Homoscedasticity · Reliability (semiconductor) · Residual · Variance (accounting) · Variance components · Mental Health Research Topics · Neural and Behavioral Psychology Studies · Psychometric Methodologies and Testing
Generalizability Theory
A Manifesto on Psychology as Idiographic Science
Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper)
Information Theory and an Extension of the Maximum Likelihood Principle
Practical Bayesian model evaluation using leave-one-out cross-validation and Waic
Using heteroskedasticity-consistent standard error estimators in OLS regression
Generating random correlation matrices based on vines and extended onion method
An Application of a Mixed‐Effects Location Scale Model for Analysis of Ecological Momentary Assessment (EMA) Data
The Disaggregation of Within-Person and Between-Person Effects in Longitudinal Models of Change
Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.
Centering predictor variables in cross-sectional multilevel models
BRMS
Analyzing Multivariate Generalizability Theory Designs within Structural Equation Modeling Frameworks
Modeling Individual Differences in Within-Person Variation of Negative and Positive Affect in a Mixed Effects Location Scale Model Using Bugs/Jags
A Mixed-Effects Location Scale Model for Dyadic Interactions
| Highly cited | No |
|---|