Standard errors and confidence intervals for cognitive diagnostic models

Parallel bootstrap methods

Bibliographic Data

ID19407699
AuthorsYanlou Liu (0000-0002-4229-7573, corresponding author)
Year2022
Volume54
Issue6
Pages703-724
Publication date2022-06-01
Peer ReviewedYes
Open AccessNo
TypeARTICLE
VenueActa Psychologica Sinica (JOURNAL)
Journal identifiersISSN: 0439-755X
PublisherChina Science Publishing & Media Ltd (PUBLISHER)
DOI10.3724/sp.j.1041.2022.00703
OpenAlexW4226087014
LanguageEN
Citations received2
References cited24

The model parameter standard error ( SE ; or variance-covariance matrix), which provides an estimate of the uncertainty associated with the model parameter estimate, has both theoretical and practical implications in cognitive diagnostic models (CDMs). The drawbacks of the analytic methods, such as the empirical cross-product information matrix, observed information matrix, and “robust” sandwich-type information matrix, are that they require the positive definiteness of the information matrix and may suffer from boundary problems. Another method for estimating model parameter SE s is to use the computer-intensive bootstrap method, and consequently, no study has systematically explored the performance of the bootstrap in calculating model parameter SE s and confidence intervals (CIs) in CDMs. The purpose of this research is to present two new highly efficient bootstrap methods to calculate model parameter SE s and CI s in CDMs, namely the parallel parametric bootstrap (pPB) and parallel non-parametric bootstrap (pNPB) methods. A simulation study was conducted to evaluate the performance of the pPB and pNPB methods. Five factors that may influence the performance of the model parameter SE s and CIs were manipulated. The two model specification scenarios considered in this simulation were the correctly specified and over-specified models. The sample size was set to two levels: 1, 000 and 3, 000. Three bootstrap sample sizes were manipulated: 200, 500, and 3, 000. Three levels of item quality were considered: high [ $P(\mathbf{0})=0.1$ , $P(\mathbf{1})=0.9$ ], moderate [ $P(\mathbf{0})=0.2$ , $P(\mathbf{1})=0.8$ ], and low quality [ $P(\mathbf{0})=0.3$ , $P(\mathbf{1})=0.7$ ]. The pPB and pNPB methods were used to estimate model parameter SE s and CIs. The simulation results indicated the following. (1) For the correctly specified CDMs, under the high- or moderate-item-quality conditions, the coverage rates of the 95% CIs of the model parameter SE s based on the pNPB or pPB method were reasonably close to the expected coverage rate, and the bias for each model parameter SE converged to zero, meaning that the estimated SE was almost identical to the empirical SE . The increase in the bootstrap sample size had only a slight effect on the performance of the pNPB or pPB method. Under the low-item-quality condition, the pNPB method tended to over-estimate SE , whereas a contrary trend was observed for the pPB method. (2) For the over-specified CDMs, most of the permissible item parameter SE s and almost all of the permissible structural parameter SE s exhibited good performance in terms of the 95% CI coverage rates and bias. Under most of the simulation conditions, the impermissible model parameter SE s did not exhibit good performance in approximating the empirical SE s. To the best of our knowledge, this is the first study in which the performance of the bootstrap method in estimating model parameter SE s and CIs in CDMs is systematically investigated. The pNPB or pPB appears to be a useful tool for researchers interested in evaluating the uncertainty of the model parameter point estimates. As a time-saving computational strategy, the pNPB or pPB method is substantially faster than the usual bootstrap method. The simulation and real data studies showed that 3, 000 re-samples might be adequate for the bootstrap method in calculating model parameter SE s and CIs in CDMs

Confidence interval · Econometrics · Robust confidence intervals · Standard error · Statistics · Bayesian Modeling and Causal Inference · Mathematics · Psychology · Statistical Methods and Inference · Statistical Methods in Clinical Trials

Tools

Open DOI