Please wait a minute...
Acta Psychologica Sinica
|
The Multidimensional Testlet-Effect Cognitive Diagnostic Models
ZHAN Peida1,2; LI Xiaomin3; WANG Wen-Chung3; BIAN Yufang2; WANG Lijun1
(1 Department of Psychology, Zhejiang Normal University, Jinhua 321004, China) (2 National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China) (3 Assessment Research Center, The Hong Kong Institute of Education, Hong Kong, China)
Download: PDF(587 KB)   Review File (1 KB) 
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

Cognitive diagnosis, which is also referred as skill assessment or skill profiling, utilizes latent class models to provide fine-grained information about students’ strength and weakness in the learning process. The outcome of cognitive diagnostic models (CDMs) is a profile with binary element for each examinee to indicate the mastery/nonmastery status of every attribute/skill. Therefore, one major advantage of CDMs is the capacity to provide additional information about the instructional needs of students. In the past decades, extensive research has been conducted in the area of cognitive diagnosis and many statistical models based on a probabilistic approach have been proposed. Examples of CDMs include the deterministic inputs, noisy and gate (DINA) model (Junker & Sijtsma, 2001), the deterministic input, noisy or gate (DINO) model (Templin & Henson, 2006), and the linear Logistic model (LLM) (Maris, 1999). In educational measurement, one of the most commonly used formats is the testlet design, which is a cluster of items that share a common stimulus (e.g., a reading comprehension passage or a figure). Under the framework of item response theory (IRT), various testlet response models (TRM) have been proposed, such as the Rasch testlet model (Wang & Wilson, 2005) and the multidimensional testlet-effect Rasch model (MTERM) (Zhan, Wang, Wang, & Li, 2014). However, limited efforts have been contributed to the development of testlet models for CDMs. A question then naturally arises is the searching for a way to account for testlet effect under CDMs. To address this issue, this study proposed two testlet-CDMs. One followed the compensatory approach and the other followed the noncompensatory approach: (1) the compensatory multidimensional testlet-effect CDM (C-MTECDM) was based on the combination of LLM and MTERM, while (2) the noncompensatory multidimensional testlet-effect CDM (N-MTECDM) was based on the combination of (logit)DINA model and MTERM, respectively. Model parameters can be estimated by the Bayesian methods with Markov chain Monte Carlo (MCMC) algorithms, which have been implemented with the freeware WinBUGS. In study 1, a series of simulations were conducted to evaluate parameter recovery of two new models, and results showed that the model parameters could be recovered fairly well under all simulated conditions. In study 2, the two new models were compared with the LLM and the (logit)DINA model, respectively. Results showed that ignoring testlet effect would result in biased item parameter estimations and worse person classification rates. Additionally, fitting a more complicated model (i.e., MTECDM) to data with a simpler structure did litter harm on parameter recovery. In conclusion, the new models is feasible and flexible.

Keywords cognitive diagnosis      testlet      item response theory      multidimensional testlet-effect      Logistic testlet framework      DINA     
Corresponding Authors: BIAN Yufang, E-mail: bianyufang66@126.com; WANG Lijun, E-mail: frankwlj@163.com   
Issue Date: 25 May 2015
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
ZHAN Peida
LI Xiaomin
WANG Wen-Chung
BIAN Yufang
WANG Lijun
Cite this article:   
ZHAN Peida,LI Xiaomin,WANG Wen-Chung, et al. The Multidimensional Testlet-Effect Cognitive Diagnostic Models[J]. Acta Psychologica Sinica, 10.3724/SP.J.1041.2015.00689
URL:  
http://journal.psych.ac.cn/xlxb/EN/10.3724/SP.J.1041.2015.00689     OR     http://journal.psych.ac.cn/xlxb/EN/Y2015/V47/I5/689
[1] GUO Haiying; CHEN Lihua; YE Zhi; PAN Jin; LIN Danhua. Characteristics of peer victimization and the bidirectional relationship between peer victimization and internalizing problems among rural-to-urban migrant children in China: A longitudinal study[J]. Acta Psychologica Sinica, 2017, 49(3): 336-348.
[2] CHEN Ping. Two new online calibration methods for computerized adaptive testing[J]. Acta Psychologica Sinica, 2016, 48(9): 1184-1198.
[3] MENG Xiangbin; TAO Jian; CHEN Shali. Warm’sweighted maximum likelihood estimation of latent trait in the four-parameter logistic model[J]. Acta Psychologica Sinica, 2016, 48(8): 1047-1056.
[4] CHEN Leni; WANG Zhen; LUO Ben Nanfeng; LUO Zhengxue. Leader-subordinate extraversion fit and subordinate work engagement: Based on dominance complementarity theory[J]. Acta Psychologica Sinica, 2016, 48(6): 710-721.
[5] HUANG Bijuan; FENG Hongmin; SI Jiwei; ZHANG Jie; WANG Xiangyan. Dual-task coordination and task presentation mode influence arithmetic strategy execution in adults: Evidence from computational estimation[J]. Acta Psychologica Sinica, 2016, 48(6): 671-683.
[6] LIU Yanlou; XIN Tao; LI Lingqing; TIAN Wei; LIU Xiaoxiao. An improved method for differential item functioning detection in cognitive diagnosis models: An application of Wald statistic based on observed information matrix[J]. Acta Psychologica Sinica, 2016, 48(5): 588-598.
[7] ZHAN Peida; BIAN Yufang; WANG Lijun. Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes[J]. Acta Psychologica Sinica, 2016, 48(3): 318-330.
[8] WANG Wenyi;SONG Lihong;DING Shuliang. Classification accuracy and consistency indices for complex decision rules in multidimensional item response theory[J]. Acta Psychologica Sinica, 2016, 48(12): 1612-1624.
[9] ZHAO Ying; CHENG Yahua; WU Xinchun; NGUYEN Thi Phuong. The reciprocal relationship between morphological awareness and vocabulary knowledge among Chinese children: A longitudinal study[J]. Acta Psychologica Sinica, 2016, 48(11): 1434-1444.
[10] ZHAN Peida; CHEN Ping; BIAN Yufang. Using confirmatory compensatory multidimensional IRT models to do cognitive diagnosis[J]. Acta Psychologica Sinica, 2016, 48(10): 1347-1356.
[11] CAI Yan; MIAO Ying; TU Dongbo. The polytomously scored cognitive diagnosis computerized adaptive testing[J]. Acta Psychologica Sinica, 2016, 48(10): 1338-1346.
[12] AI Jiru; ZHANG Hongduan; SI Jiwei; LU Chun; ZHANG Tangzheng. The effects of presenting mode, reaction order of dual task on adults’ arithmetic strategy choice and execution[J]. Acta Psychologica Sinica, 2016, 48(10): 1248-1257.
[13] KANG Chunhua; REN Ping; ZENG Pingfei. Nonparametric Cognitive Diagnosis: A Cluster Diagnostic Method  Based on Grade Response Items[J]. Acta Psychologica Sinica, 2015, 47(8): 1077-1088.
[14] TANG Xiaojuan; DING Shuliang; YU Zonghuo. Application of Rough Set Theory in Item Cognitive Attribute Identification[J]. Acta Psychologica Sinica, 2015, 47(7): 950-962.
[15] LV Jie; ZHANG Gang. The Mechanisms of Knowledge Heterogeneity on Creativity of Knowledge Teams: An Interactive Cognition Perspective[J]. Acta Psychologica Sinica, 2015, 47(4): 533-544.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech