Please wait a minute...
Acta Psychologica Sinica
A Simulation Study to Compare Five Cognitive Diagnostic Models
CAI Yan;TU Dongbo;DING Shuliang
(1 School of Psychology, Jiangxi Normal University, Nanchang 330027, China) (2 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang 330027, China)
Download: PDF(406 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    

Cognitive diagnosis is an important topic in modern psychometric area. Now more than 70 cognitive diagnosis models (CDMs) are developed. There are some questions among these models: (1) When the attribute hierarchy structure is known, how to choose the most suitable model? (2) When the attribute hierarchy structure is unknown, and cognitive diagnosis is required, how to do it? These problems seems especially more puzzled for the practice workers. This paper only paid main attention on three international popular models. Therefore, five cognitive diagnosis models (RSM, AHM_A, GDD, DINA and DINA_HC) were compared corresponding to the above questions from psychometric opinion. In this paper, Monte Carlo simulation study was used. Although the number of slips and the hierarchy structure are two important factors that affect the performance on corrected match ratio of cognitive diagnosis, this study would pay attention on other three factors: the distribution of cognitive pattern, the sample size, the number of attributes. The findings identified: (1) When the characteristic of data was known, focusing on specific factor, the five methods had different advantages. a) For the distributions of cognitive pattern, although they have different effects on different methods, the same conclusion could find that the performance on negative bias distribution was the best, and that of DINA_HC and DINA were better than the rest methods on any discussed distributions. b) Considering the sample size, the performance of GDD with small scale assessment (100/20, persons/items)was the best one; with medium and large scale assessment (1000/60, 5000/100, persons/items), the performance of DINA_HC and DINA were better than the rest c) For the number of attributes, the more the attributes are the worse the performance will be. But for the methods, the performance the performance of DINA_HC and DINA were also better than the rest. All these reflected that the most suitable method could be adapt from the three methods: GDD, DINA_HC and DINA, corresponding to the real scenario. And the RSM was the worst cognitive diagnosis method. (2) When the characteristic of data was unknown, an unstructured attribute hierarchy is treated as a coarsened version of a structured one, and the DINA method has a similar performance under unstructured hierarchy with the GDD, DINA_HC methods under structured hierarchy. Thus if the hierarchy structure could not be identified clearly, and the test Q matrix was clear, then the DINA could be adapt.

Keywords cognitive attributes      cognitive diagnosis      cognitive diagnosis models      attributes Match Ratio     
Corresponding Authors: TU Dongbo   
Issue Date: 25 November 2013
E-mail this article
E-mail Alert
Articles by authors
TU Dongbo
DING Shuliang
Cite this article:   
CAI Yan,TU Dongbo,DING Shuliang. A Simulation Study to Compare Five Cognitive Diagnostic Models[J]. Acta Psychologica Sinica, 10.3724/SP.J.1041.2013.01295
URL:     OR
[1] LIU Yanlou; XIN Tao; LI Lingqing; TIAN Wei; LIU Xiaoxiao. An improved method for differential item functioning detection in cognitive diagnosis models: An application of Wald statistic based on observed information matrix[J]. Acta Psychologica Sinica, 2016, 48(5): 588-598.
[2] ZHAN Peida; BIAN Yufang; WANG Lijun. Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes[J]. Acta Psychologica Sinica, 2016, 48(3): 318-330.
[3] ZHAN Peida; CHEN Ping; BIAN Yufang. Using confirmatory compensatory multidimensional IRT models to do cognitive diagnosis[J]. Acta Psychologica Sinica, 2016, 48(10): 1347-1356.
[4] CAI Yan; MIAO Ying; TU Dongbo. The polytomously scored cognitive diagnosis computerized adaptive testing[J]. Acta Psychologica Sinica, 2016, 48(10): 1338-1346.
[5] KANG Chunhua; REN Ping; ZENG Pingfei. Nonparametric Cognitive Diagnosis: A Cluster Diagnostic Method  Based on Grade Response Items[J]. Acta Psychologica Sinica, 2015, 47(8): 1077-1088.
[6] TANG Xiaojuan; DING Shuliang; YU Zonghuo. Application of Rough Set Theory in Item Cognitive Attribute Identification[J]. Acta Psychologica Sinica, 2015, 47(7): 950-962.
[7] ZHAN Peida; LI Xiaomin; WANG Wen-Chung; BIAN Yufang; WANG Lijun. The Multidimensional Testlet-Effect Cognitive Diagnostic Models[J]. Acta Psychologica Sinica, 2015, 47(5): 689-701.
[8] CAI Yan; TU Dongbo. Extension of Cognitive Diagnosis Models Based on the Polytomous Attributes Framework and Their Q-matrices Designs[J]. Acta Psychologica Sinica, 2015, 47(10): 1300-1308.
[9] WANG Zhuoran; GUO Lei; BIAN Yufang. Comparison of DIF Detecting Methods in Cognitive Diagnostic Test[J]. Acta Psychologica Sinica, 2014, 46(12): 1923-1932.
[10] TU Dongbo;CAI Yan;DAI Haiqi. Comparison and Selection of Five Noncompensatory Cognitive Diagnosis Models Based on Attribute Hierarchy Structure[J]. Acta Psychologica Sinica, 2013, 45(2): 243-252.
[11] TU Dong-Bo,CAI Yan,DAI Hai-Qi. A New Method of Q-matrix Validation Based on DINA Model[J]. , 2012, 44(4): 558-568.
[12] TU Dong-Bo;CAI Yan;DAI Hai-Qi;DING Shu-Liang. A New Multiple-Strategies Cognitive Diagnosis Model: the MSCD Method[J]. Acta Psychologica Sinica, 2012, 44(11): 1547-1553.
[13] DING Shu-Liang;MAO Meng-Meng;WANG Wen-Yi;LUO Fen;CUI Ying. Evaluating the Consistency of Test Items Relative to the Cognitive Model for Educational Cognitive Diagnosis[J]. Acta Psychologica Sinica, 2012, 44(11): 1535-1546.
[14] CHEN Ping,XIN Tao. Developing On-line Calibration Methods for Cognitive Diagnostic Computerized Adaptive Testing[J]. , 2011, 43(06): 710-724.
[15] YU Xiao-Feng,DING Shu-Liang,QIN Chun-Ying,LU Yun-Na. Application of Bayesian Networks to Identify Hierarchical Relation Among
Attributes in Cognitive Diagnosis
[J]. , 2011, 43(03): 338-346.
Full text



Copyright © Acta Psychologica Sinica
Support by Beijing Magtech