Relative Robustness of CDMs and (M)IRT in Measuring Growth in Latent Skills

Previous studies have demonstrated evidence of latent skill continuity even in tests intentionally designed for measurement of binary skills. In addition, the assumption of binary skills when continuity is present has been shown to potentially create a lack of invariance in item and latent ability parameters that may undermine applications. In this article, we examine measurement of growth as one such application, and consider multidimensional item response theory (MIRT) as a competing alternative. Motivated by prior findings concerning the effects of skill continuity, we study the relative robustness of cognitive diagnostic models (CDMs) and (M)IRT models in the measurement of growth under both binary and continuous latent skill distributions. We find CDMs to be a less robust way of quantifying growth under misspecification, and subsequently provide a real-data example suggesting underestimation of growth as a likely consequence. It is suggested that researchers should regularly attend to the assumptions associated with the use of latent binary skills and consider (M)IRT as a potentially more robust alternative if unsure of their discrete nature

Binary number · Cognitive psychology · Confirmatory factor analysis · Econometrics · Item response theory · Latent growth modeling · Measurement invariance · Psychometrics · Robustness (evolution) · Statistics · Structural equation modeling · Cognitive Abilities and Testing · Computer Science · Grit, Self-Efficacy, and Motivation · Mathematics · Psychology · Psychometric Methodologies and Testing

• Bayesian inference for dynamic Q matrices and attribute trajectories in hidden Markov diagnostic classification models

  Open Access•Chen‐Wei Liu•British Journal of Mathematical…•2026

  Incluir item

• LTM

  Open Access•Dimitris Rizopoulos•Journal of Statistical Software•2006

  Incluir item

• Mirt

  Open Access•R Philip Chalmers•Journal of Statistical Software•2012

  Incluir item

• A new look at the statistical model identification

  Open Access•Hirotugu Akaike•IEEE Transactions on Automatic…•1974

  Incluir item

• Estimating the Dimension of a Model

  Gideon Schwarz•The Annals of Statistics•1978

  Incluir item

• The Effects of Q-Matrix Design on Classification Accuracy in the Log-Linear Cognitive Diagnosis Model

  Open Access•Matthew J Madison, Laine Bradshaw et al.•Educational and Psychological…•2015

  Incluir item

• Assessing Change in Latent Skills Across Time With Longitudinal Cognitive Diagnosis Modeling

  Open Access•Yasemin Kaya, Walter L Leite•Educational and Psychological…•2017

  Incluir item

• A Latent Transition Analysis Model for Assessing Change in Cognitive Skills

  Open Access•Feiming LI, Allan S Cohen et al.•Educational and Psychological…•2016

  Incluir item