Please wait a minute...
Acta Psychologica Sinica
|
Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes
ZHAN Peida1; BIAN Yufang1; WANG Lijun2
(1 Collaborative Innovation Center of Assessment toward Basic Education Quality, Beijing Normal University, Beijing 100875, China)
(2 Department of Psychology, Zhejiang Normal University, Jinhua 321004, China)
Download: PDF(732 KB)   Review File (1 KB) 
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  
Diagnostic classification assessment (DCA) utilizes latent class models to provide fine-grained information about students’ strengths and weaknesses in the learning process. In the past decades, extensive research has been conducted in the area of DCA and many statistical models based on a probabilistic approach have been proposed. At present, several diagnostic classification models (DCMs) for dichotomous attributes exist, which include the deterministic inputs, noisy “and” gate (DINA; Junker & Sijtsma, 2001); the deterministic inputs, noisy “or” gate (DINO; Templin & Henson, 2006); and the linear logistic model (LLM; Maris, 1999). In contrast, only a few DCMs can be used to deal with the polytomous attributes, such as the model based on the ordered-category attribute coding (OCAC; Karelitz, 2004), and the polytomous generalized DINA (pG-DINA; Chen & de la Torre, 2013).
Polytomous attributes, particularly those defined as part of the test development process, can provide additional diagnostic information. The present research proposes three reparametrized reduced models of pG-DINA (Chen & de la Torre, 2013), which include the reparametrized polytomous attributes DINA (RPa-DINA), the reparametrized polytomous attributes DINO (RPa-DINO), and the reparametrized polytomous attributes LLM (RPa-LLM). Furthermore, to better understand the classification accuracy of the new models, the impact of 6 factors was investigated, namely, the number of polytomous attributes, the highest level of polytomous attributes, the correlations among polytomous attributes, the hierarchical structure, the sample size, and the number of items. Results of the simulation study indicated that:
(1) more polytomous attributes led to lower classification. Their effects, in descending order, were the RPa-LLM, the RPa-DINO, and the RPa-DINA. Less than 5 polytomous attributes used in empirical research is suggested;
(2) for the number of attribute levels, more levels resulted in worse performance. Less than 4 levels within one attribute used in empirical research is suggested;
(3) the higher the correlations among polytomous attributes, the higher the classification accuracy would be;
(4) different hierarchical structure had different influences on the classification accuracy. No matter what structure we had, the performance of RPa-DINA was quite well behaved. However, other 2 models, especially the RPa-DINO, were recommended for the analysis of response data from independent hierarchical structure;
(5) the sample size has little impact on the classification accuracy; and

(6) the number of items was inversely proportional to the classification accuracy.

Keywords cognitive diagnosis      polytomous attribute      polytomous Q matrix      diagnostic classification models      DINA      DINO      LLM     
Corresponding Authors: BIAN Yufang, E-mail: bianyufang66@126.com; WANG Lijun, E-mail: frankwlj@163.com   
Issue Date: 25 March 2016
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
ZHAN Peida
BIAN Yufang
WANG Lijun
Cite this article:   
ZHAN Peida,BIAN Yufang,WANG Lijun. Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes[J]. Acta Psychologica Sinica, 10.3724/SP.J.1041.2016.00318
URL:  
http://journal.psych.ac.cn/xlxb/EN/10.3724/SP.J.1041.2016.00318     OR     http://journal.psych.ac.cn/xlxb/EN/Y2016/V48/I3/318
[1] GUO Haiying; CHEN Lihua; YE Zhi; PAN Jin; LIN Danhua. Characteristics of peer victimization and the bidirectional relationship between peer victimization and internalizing problems among rural-to-urban migrant children in China: A longitudinal study[J]. Acta Psychologica Sinica, 2017, 49(3): 336-348.
[2] CHEN Leni; WANG Zhen; LUO Ben Nanfeng; LUO Zhengxue. Leader-subordinate extraversion fit and subordinate work engagement: Based on dominance complementarity theory[J]. Acta Psychologica Sinica, 2016, 48(6): 710-721.
[3] HUANG Bijuan; FENG Hongmin; SI Jiwei; ZHANG Jie; WANG Xiangyan. Dual-task coordination and task presentation mode influence arithmetic strategy execution in adults: Evidence from computational estimation[J]. Acta Psychologica Sinica, 2016, 48(6): 671-683.
[4] LIU Yanlou; XIN Tao; LI Lingqing; TIAN Wei; LIU Xiaoxiao. An improved method for differential item functioning detection in cognitive diagnosis models: An application of Wald statistic based on observed information matrix[J]. Acta Psychologica Sinica, 2016, 48(5): 588-598.
[5] ZHAO Ying; CHENG Yahua; WU Xinchun; NGUYEN Thi Phuong. The reciprocal relationship between morphological awareness and vocabulary knowledge among Chinese children: A longitudinal study[J]. Acta Psychologica Sinica, 2016, 48(11): 1434-1444.
[6] ZHAN Peida; CHEN Ping; BIAN Yufang. Using confirmatory compensatory multidimensional IRT models to do cognitive diagnosis[J]. Acta Psychologica Sinica, 2016, 48(10): 1347-1356.
[7] CAI Yan; MIAO Ying; TU Dongbo. The polytomously scored cognitive diagnosis computerized adaptive testing[J]. Acta Psychologica Sinica, 2016, 48(10): 1338-1346.
[8] AI Jiru; ZHANG Hongduan; SI Jiwei; LU Chun; ZHANG Tangzheng. The effects of presenting mode, reaction order of dual task on adults’ arithmetic strategy choice and execution[J]. Acta Psychologica Sinica, 2016, 48(10): 1248-1257.
[9] KANG Chunhua; REN Ping; ZENG Pingfei. Nonparametric Cognitive Diagnosis: A Cluster Diagnostic Method  Based on Grade Response Items[J]. Acta Psychologica Sinica, 2015, 47(8): 1077-1088.
[10] TANG Xiaojuan; DING Shuliang; YU Zonghuo. Application of Rough Set Theory in Item Cognitive Attribute Identification[J]. Acta Psychologica Sinica, 2015, 47(7): 950-962.
[11] ZHAN Peida; LI Xiaomin; WANG Wen-Chung; BIAN Yufang; WANG Lijun. The Multidimensional Testlet-Effect Cognitive Diagnostic Models[J]. Acta Psychologica Sinica, 2015, 47(5): 689-701.
[12] LV Jie; ZHANG Gang. The Mechanisms of Knowledge Heterogeneity on Creativity of Knowledge Teams: An Interactive Cognition Perspective[J]. Acta Psychologica Sinica, 2015, 47(4): 533-544.
[13] YU Xiaofeng; LUO Zhaosheng; GAO Chunlei; LI Yujun; WANG Rui; WANG Yutong. An Item Attribute Specification Method Based On the Likelihood D2 Statistic[J]. Acta Psychologica Sinica, 2015, 47(3): 417-426.
[14] YU Xiaofeng; LUO Zhaosheng; QIN Chunying; GAO Chunlei; LI Yujun. Joint Estimation of Model Parameters and Q-Matrix Based on Response Data[J]. Acta Psychologica Sinica, 2015, 47(2): 273-282.
[15] XU Peng; QI Lu; XIONG Jian; YE Haosheng. A Regression Analysis Model of Ordinal Variable to Psychological Data[J]. Acta Psychologica Sinica, 2015, 47(12): 1520-1528.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech