Please wait a minute...
Acta Psychologica Sinica    2012, Vol. 44 Issue (11) : 1535-1546     DOI: 10.3724/SP.J.1041.2012.01535
Evaluating the Consistency of Test Items Relative to the Cognitive Model for Educational Cognitive Diagnosis
DING Shu-Liang;MAO Meng-Meng;WANG Wen-Yi;LUO Fen;CUI Ying
(1 School of Computer and Engineering, Jiangxi Normal University, Nanchang 330022, China) (2 College of Education, NanChang University, Nanchang 330031, China) (3 Centre for Research in Applied Measurement and Evaluation (CRAME), University of Alberta, CA)
Download: PDF(546 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  Attributes and their hierarchy may present a cognitive model. Building a cognitive model is one of key steps for cognitive diagnosis as it is directly related to the validity and usefulness of test results. It is very important to detect whether the test specification coincides with the cognitive model before administering the test. In this paper, an explicit index is given to measure the extent to which the cognitive model is represented by the test items of the diagnostic test. We call this index as the theoretical construct validity (TCV). In terms of TCV, the test reported by Tatsuoka and her colleagues (1988) is reanalyzed, and the TCV of the test is only 9/24. That is, 24 knowledge states are obtained from the theoretic cognitive model but the test specification could distinguish only 9 knowledge states. Cui and her colleagues established a person fit index named hierarchy consistency index (HCI) to detect the fitness of an examinee’s observed response pattern (ORP) to an expected response pattern (ERP). HCI is not defined well when an examinee mastered one attribute only and there is only one item measuring the same attribute in the test, and the examinee responses correctly to the item. The original HCI could not compute for the number of comparisons being zero, hence the denominator being zero. In addition, the HCI includes the slipping only. Combining the slipping and guessing in the response to an item, we proposed a new index called modified HCI (MHCI). MHCI could detect the consistency of the response data and the cognitive model that is obtained by experts.
Keywords cognitive diagnosis      cognitive model      cognitive diagnostic test     
Corresponding Authors: DING Shu-Liang   
Issue Date: 01 November 2012
E-mail this article
E-mail Alert
Articles by authors
DING Shu-Liang
MAO Meng-Meng
CUI Ying
Cite this article:   
DING Shu-Liang,MAO Meng-Meng,WANG Wen-Yi, et al. Evaluating the Consistency of Test Items Relative to the Cognitive Model for Educational Cognitive Diagnosis[J]. Acta Psychologica Sinica, 2012, 44(11): 1535-1546.
URL:     OR
[1] LIU Yanlou; XIN Tao; LI Lingqing; TIAN Wei; LIU Xiaoxiao. An improved method for differential item functioning detection in cognitive diagnosis models: An application of Wald statistic based on observed information matrix[J]. Acta Psychologica Sinica, 2016, 48(5): 588-598.
[2] ZHAN Peida; BIAN Yufang; WANG Lijun. Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes[J]. Acta Psychologica Sinica, 2016, 48(3): 318-330.
[3] ZHAN Peida; CHEN Ping; BIAN Yufang. Using confirmatory compensatory multidimensional IRT models to do cognitive diagnosis[J]. Acta Psychologica Sinica, 2016, 48(10): 1347-1356.
[4] CAI Yan; MIAO Ying; TU Dongbo. The polytomously scored cognitive diagnosis computerized adaptive testing[J]. Acta Psychologica Sinica, 2016, 48(10): 1338-1346.
[5] KANG Chunhua; REN Ping; ZENG Pingfei. Nonparametric Cognitive Diagnosis: A Cluster Diagnostic Method  Based on Grade Response Items[J]. Acta Psychologica Sinica, 2015, 47(8): 1077-1088.
[6] TANG Xiaojuan; DING Shuliang; YU Zonghuo. Application of Rough Set Theory in Item Cognitive Attribute Identification[J]. Acta Psychologica Sinica, 2015, 47(7): 950-962.
[7] ZHAN Peida; LI Xiaomin; WANG Wen-Chung; BIAN Yufang; WANG Lijun. The Multidimensional Testlet-Effect Cognitive Diagnostic Models[J]. Acta Psychologica Sinica, 2015, 47(5): 689-701.
[8] CAI Yan; TU Dongbo. Extension of Cognitive Diagnosis Models Based on the Polytomous Attributes Framework and Their Q-matrices Designs[J]. Acta Psychologica Sinica, 2015, 47(10): 1300-1308.
[9] FU Zhongfang;SUN Xunwei;LIANG Peipeng. Proactive Interference in Episodic Memory: Neuropsychopharmacology and Computational Cognitive Modeling Study[J]. Acta Psychologica Sinica, 2014, 46(4): 492-499.
[10] WANG Zhuoran; GUO Lei; BIAN Yufang. Comparison of DIF Detecting Methods in Cognitive Diagnostic Test[J]. Acta Psychologica Sinica, 2014, 46(12): 1923-1932.
[11] TU Dongbo;CAI Yan;DAI Haiqi. Comparison and Selection of Five Noncompensatory Cognitive Diagnosis Models Based on Attribute Hierarchy Structure[J]. Acta Psychologica Sinica, 2013, 45(2): 243-252.
[12] CAI Yan;TU Dongbo;DING Shuliang. A Simulation Study to Compare Five Cognitive Diagnostic Models[J]. Acta Psychologica Sinica, 2013, 45(11): 1295-1304.
[13] TU Dong-Bo,CAI Yan,DAI Hai-Qi. A New Method of Q-matrix Validation Based on DINA Model[J]. , 2012, 44(4): 558-568.
[14] TU Dong-Bo;CAI Yan;DAI Hai-Qi;DING Shu-Liang. A New Multiple-Strategies Cognitive Diagnosis Model: the MSCD Method[J]. Acta Psychologica Sinica, 2012, 44(11): 1547-1553.
[15] CHEN Ping,XIN Tao. Developing On-line Calibration Methods for Cognitive Diagnostic Computerized Adaptive Testing[J]. , 2011, 43(06): 710-724.
Full text



Copyright © Acta Psychologica Sinica
Support by Beijing Magtech