ISSN 0439-755X
CN 11-1911/B
主办:中国心理学会
   中国科学院心理研究所
出版:科学出版社

心理学报 ›› 2024, Vol. 56 ›› Issue (5): 670-688.doi: 10.3724/SP.J.1041.2024.00670

• 研究报告 • 上一篇    

CD-CAT中基于SCAD惩罚和EM视角的在线标定方法开发——G-DINA模型

谭青蓉1,2, 蔡艳1(), 汪大勋1(), 罗芬3, 涂冬波1()   

  1. 1江西师范大学心理学院, 南昌 330022
    2陆军军医大学医学心理系基础心理学教研室, 重庆 400000
    3江西师范大学计算机信息工程学院, 南昌 330022
  • 收稿日期:2022-09-26 发布日期:2024-03-06 出版日期:2024-05-25
  • 通讯作者: 蔡艳, E-mail: cy1979123@aliyun.com; 汪大勋, E-mail: wangda.xun@163.com; 涂冬波, E-mail: tudongbo@aliyun.com
  • 基金资助:
    国家自然科学基金项目(62167004);国家自然科学基金项目(32160203);国家自然科学基金项目(32300942);国家自然科学基金项目(31960186);国家自然科学基金项目(61967009)

Development of Online Calibration Method based on SCAD penalty and EM perspective in CD-CAT: G-DINA model

TAN Qingrong1,2, CAI Yan1(), WANG Daxun1(), LUO Fen3, TU Dongbo1()   

  1. 1School of Psychology, Jiangxi Normal University, Nanchang 330022, China
    2Department of Basic Psychology, College of Psychology, Army Medical University, Chongqing 400000, China
    3College of Computer Information Engineering, Jiangxi Normal University, Nanchang 330022, China
  • Received:2022-09-26 Online:2024-03-06 Published:2024-05-25

摘要:

G-DINA (the generalized deterministic input, noisy and gate)模型限制条件少, 应用范围广, 满足大量心理与教育评估测验数据的要求。研究提出一种适用于G-DINA等模型的同时标定新题Q矩阵与项目参数的认知诊断计算机化自适应测验(CD-CAT)在线标定新方法SCADOCM, 以期促进CD-CAT在实践中的推广与应用。本研究分别基于模拟题库以及真实题库进行研究, 结果表明:相比传统的SIE方法, SCADOCM在各实验条件下均具有较为理想的标定精度与标定效率, 应用前景较好; SIE方法不适用于饱和的G-DINA等模型, 其各实验条件下的Q矩阵标定精度均较低。

关键词: 认知诊断计算机化自适应测验, 在线标定, Q矩阵, G-DINA模型, SCAD惩罚

Abstract:

Cognitive diagnostic computerized adaptive testing (CD-CAT) provides a detailed diagnosis of an examinee’s strengths and weaknesses in the content measured in a timely and accurate manner, which can be used as a reference for further study or remediation planning, thus meeting the practical need for efficient and detailed test results. The successful implementation of CD-CAT is based on an item bank, but its maintenance is a very challenging task. A psychometrically popular choice for maintaining an item bank is online calibration. Currently, the research on online calibration methods in the CD-CAT that can calibrate Q-matrix and item parameters simultaneously is very weak. The existing methods are basically developed based on the deterministic input, noisy and gate (DINA) model. Compared with the DINA model, the generalized DINA (G-DINA) model has been more widely applied because it is less restrictive and can meet the requirements of a large number of test data in psychological and educational assessment. Therefore, if the online calibration method that jointly calibrates the Q-matrix and item parameters can be developed for models with few constraints such as G-DINA, its meaning is understood without explanation.
In current study, a new online calibration method, SCADOCM, was proposed, which was suitable for the G-DINA model. The construction of SCADOCM was based on the smoothly clipped absolute deviation penalty (SCAD) and marginalized maximum likelihood estimation (MMLE/EM) algorithm. For the new item j, the log-likelihood function with SCAD can be formulated based on the examinees’ responses in this item and the examinees’ attribute marginal mastery probability, and the q-vector of the new item can be estimated by the q-vector estimator based on SCAD. Then, the EM algorithm was used to estimate the item parameter of the new item j based on the posterior distributions of examinees’ attribute patterns, the examinees’ responses to new item j and the estimated q-vector.
To examine the performance of the proposed SCADOCM and compare it with the SIE method, two simulation studies (Study 1 and Study 2) are conducted. Study 1 is based on a simulated item bank while Study 2 is based on the real item bank (Internet addiction item bank; Shi, 2017). In these simulation studies, four factors were manipulated: the calibration sample size (nj = 50 vs. 100 vs. 500 vs. 1000 vs. 2000), the distribution of the attribute pattern (uniform distribution vs. high-order distribution vs. normal distribution), the item quality (U (0.05, 0.15) vs. U (0.1, 0.3)), and the online calibration methods (SCADOCM vs. SIE). The results showed that (1) SCADOCM has satisfactory calibration accuracy and calibration efficiency, and is superior to the SIE method. In addition, the traditional SIE method is not applicable for the G-DINA model, and its Q-matrix estimation accuracy rate is low under all experimental conditions. (2) The item calibration accuracy of SCADOCM and SIE increases with the increase of calibration sample and item quality under most conditions, and its item calibration accuracy in the uniform distribution/higher-order distribution is greater than that in the normal distribution. (3) The calibration efficiency of SCADOCM decreases with the increase of calibration samples, but it is less affected by the item quality and the attribute pattern distribution; the calibration efficiency of SIE decreases with the increase of calibration samples, but it is less affected by the item quality. Moreover, the calibration efficiency of the SIE method in the normal distribution is slightly slower than that of uniform distribution/high-order distribution.
To sum up the results, this study demonstrated that the SCADOCM has higher item calibration accuracy and calibration efficiency, and outperforms the SIE method; meanwhile, the traditional SIE method is not suitable for G-DINA model. All in all, this study provides an efficient and accurate method for item calibration in CD-CAT, and provides important support for further promoting the application of CD-CAT in practice.

Key words: Cognitive Diagnostic Computerized Adaptive Testing, Online Calibration, Q-matrix, G-DINA model, SCAD Penalty

中图分类号: