ISSN 0439-755X
CN 11-1911/B
主办:中国心理学会
   中国科学院心理研究所
出版:科学出版社

心理学报 ›› 2022, Vol. 54 ›› Issue (6): 703-724.doi: 10.3724/SP.J.1041.2022.00703

• 研究报告 • 上一篇    

认知诊断模型的标准误与置信区间估计:并行自助法

刘彦楼()   

  1. 曲阜师范大学教育大数据研究院, 山东 济宁 273165
  • 收稿日期:2021-10-14 发布日期:2022-04-26 出版日期:2022-06-25
  • 通讯作者: 刘彦楼 E-mail:liuyanlou@163.com
  • 基金资助:
    国家自然科学基金青年项目(31900794)

Standard errors and confidence intervals for cognitive diagnostic models: Parallel bootstrap methods

LIU Yanlou()   

  1. Academy of Big Data for Education, Qufu Normal University, Jining 273165, China
  • Received:2021-10-14 Online:2022-04-26 Published:2022-06-25
  • Contact: LIU Yanlou E-mail:liuyanlou@163.com

摘要:

认知诊断模型的标准误(Standard Error, SE; 或方差—协方差矩阵)与置信区间(Confidence Interval, CI)在模型参数估计不确定性的度量、项目功能差异检验、项目水平上的模型比较、Q矩阵检验以及探索属性层级关系等领域有重要的理论与实践价值。本研究提出了两种新的SE和CI计算方法:并行参数化自助法和并行非参数化自助法。模拟研究发现:模型完全正确设定时, 在高质量及中等质量项目条件下, 这两种方法在计算模型参数的SE和CI时均有好的表现; 模型参数存在冗余时, 在高质量及中等质量项目条件下, 对于大部分允许存在的模型参数而言, 其SE和CI有好的表现。通过实证数据展示了新方法的价值及计算效率提升效果。

关键词: 认知诊断模型, 标准误, 置信区间, 自助法, 并行计算

Abstract:

The model parameter standard error (SE; or variance-covariance matrix), which provides an estimate of the uncertainty associated with the model parameter estimate, has both theoretical and practical implications in cognitive diagnostic models (CDMs). The drawbacks of the analytic methods, such as the empirical cross-product information matrix, observed information matrix, and “robust” sandwich-type information matrix, are that they require the positive definiteness of the information matrix and may suffer from boundary problems. Another method for estimating model parameter SEs is to use the computer-intensive bootstrap method, and consequently, no study has systematically explored the performance of the bootstrap in calculating model parameter SEs and confidence intervals (CIs) in CDMs.
The purpose of this research is to present two new highly efficient bootstrap methods to calculate model parameter SEs and CIs in CDMs, namely the parallel parametric bootstrap (pPB) and parallel non-parametric bootstrap (pNPB) methods. A simulation study was conducted to evaluate the performance of the pPB and pNPB methods. Five factors that may influence the performance of the model parameter SEs and CIs were manipulated. The two model specification scenarios considered in this simulation were the correctly specified and over-specified models. The sample size was set to two levels: 1, 000 and 3, 000. Three bootstrap sample sizes were manipulated: 200, 500, and 3, 000. Three levels of item quality were considered: high [$P(\mathbf{0})=0.1$, $P(\mathbf{1})=0.9$], moderate [$P(\mathbf{0})=0.2$, $P(\mathbf{1})=0.8$], and low quality [$P(\mathbf{0})=0.3$, $P(\mathbf{1})=0.7$]. The pPB and pNPB methods were used to estimate model parameter SEs and CIs.
The simulation results indicated the following.
(1) For the correctly specified CDMs, under the high- or moderate-item-quality conditions, the coverage rates of the 95% CIs of the model parameter SEs based on the pNPB or pPB method were reasonably close to the expected coverage rate, and the bias for each model parameter SE converged to zero, meaning that the estimated SE was almost identical to the empirical SE. The increase in the bootstrap sample size had only a slight effect on the performance of the pNPB or pPB method. Under the low-item-quality condition, the pNPB method tended to over-estimate SE, whereas a contrary trend was observed for the pPB method.
(2) For the over-specified CDMs, most of the permissible item parameter SEs and almost all of the permissible structural parameter SEs exhibited good performance in terms of the 95% CI coverage rates and bias. Under most of the simulation conditions, the impermissible model parameter SEs did not exhibit good performance in approximating the empirical SEs.
To the best of our knowledge, this is the first study in which the performance of the bootstrap method in estimating model parameter SEs and CIs in CDMs is systematically investigated. The pNPB or pPB appears to be a useful tool for researchers interested in evaluating the uncertainty of the model parameter point estimates. As a time-saving computational strategy, the pNPB or pPB method is substantially faster than the usual bootstrap method. The simulation and real data studies showed that 3, 000 re-samples might be adequate for the bootstrap method in calculating model parameter SEs and CIs in CDMs.

Key words: cognitive diagnostic model, standard error, confidence interval, bootstrap, parallel computing method

中图分类号: