Please wait a minute...
Acta Psychologica Sinica    2016, Vol. 48 Issue (7) : 903-914     DOI: 10.3724/SP.J.1041.2016.00903
|
New item selection methods in cognitive diagnostic computerized adaptive testing: Combining item discrimination indices
GUO Lei1,2,3; ZHENG Chanjin4; BIAN Yufang5; SONG Naiqing3,6; XIA Lingxiang1
(1 Faculty of Psychology, Southwest University, Chongqing 400715, China) (2 Postdoctoral Research Center for Statistics, Southwest University, Chongqing 400715, China) (3 Southwest University Branch, Collaborative Innovation Center of Assessment toward Basic Education Quality, Chongqing 400715, China) (4 School of Psychology, Jiangxi Normal University, Nanchang 330022, China) (5 Collaborative Innovation Center of Assessment toward Basic Education Quality, Beijing Normal University, Beijing 100875, China) (6 Center for Basic Education Research, Southwest University, Chongqing 400715, China)
Download: PDF(435 KB)   Review File (1 KB) 
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

Interest in developing computerized adaptive testing (CAT) under cognitive diagnostic models has increased recently. Cognitive diagnostic computerized adaptive testing (CD-CAT) attempt to classify examinees into the correct latent class profile so as to pinpoint the strengths and weaknesses of each examinee whereas CAT algorithms choose items from the item bank to achieve that goal as efficiently as possible. Most of the research in CD-CAT uses the posterior-weighted Kullback-Leibler (PWKL) index due to its high efficiency. The PWKL index integrated the posterior probabilities of examinees’ latent class profiles into the KL information, and thus improved item selection efficiency considerably. However, the PWKL index only used examinee-based information to assess the relative importance of each latent class profile. The current study attempted to take advantage of not only the examinee-base information but also the item-based information that could be readily obtained from items. In a sense, the PWKL index should be regarded as single-source index. This paper introduced four new multiple-source item selection methods, GIDPWKL, AIDPWKL, CIDPWKL, and KLEDPWKL respectively, which can be modified from the PWKL index by combining the item discrimination information. Two simulation studies were conducted to evaluate the new methods’ efficiency against the PWKL index and mutual information (MI) index in the DINA model with the exposure control. The effects of different factors were investigated: the Q matrix structure (simple vs. complex), item quality (high vs. low) and test length (moderate vs. short). Simulation results indicated that: (1) In most cases, the shorter the test length was, the higher AACCR and PCCR values the four new methods would have in the fix-length test. The GIDPWKL index had the highest average attribute correct classification rate and pattern correct classification rate among the six methods, and followed by AIDPWKL index. The performance among the CIDPWKL, KLEDPWKL, and MI depends on the experimental conditions. (2) In most cases, the higher the item quality was, the more advantage the four new methods would have in the fix-length test. (3) The structure of the Q matrix affected the performance of different item selection methods. (4) In the variable-length test, the mean of test length across all examinees for the four new methods and MI method were all smaller than those in the PWKL method. As a whole, the performance of the GIDPWKL index was the best, and should be recommended in practice where had the similar testing scenarios.

Keywords cognitive diagnostic computerized adaptive testing      item selection strategy      item discrimination      exposure control     
Corresponding Authors: GUO Lei, E-mail: happygl1229@swu.edu.cn   
Issue Date: 25 July 2016
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Cite this article:   
GUO Lei; ZHENG Chanjin; BIAN Yufang; SONG Naiqing; XIA Lingxiang. New item selection methods in cognitive diagnostic computerized adaptive testing: Combining item discrimination indices[J]. Acta Psychologica Sinica,2016, 48(7): 903-914.
URL:  
http://journal.psych.ac.cn/xlxb/EN/10.3724/SP.J.1041.2016.00903     OR     http://journal.psych.ac.cn/xlxb/EN/Y2016/V48/I7/903
[1] DAI Buyun; ZHANG Minqiang; JIAO Can; LI Guangming; ZHU Huawei; ZHANG Wenyi. Item Selection Using the Multiple-Strategy RRUM Based on CD-CAT[J]. Acta Psychologica Sinica, 2015, 47(12): 1511-1519.
[2] GUO Lei; ZHENG Chanjin; BIAN Yufang. Exposure Control Methods and Termination Rules in Variable-Length Cognitive Diagnostic Computerized Adaptive Testing[J]. Acta Psychologica Sinica, 2015, 47(1): 129-140.
[3] MAO Xiuzhen;XIN Tao. A Comparison of Item Selection Methods for Controlling Exposure Rate in Cognitive Diagnostic Computerized Adaptive Testing[J]. Acta Psychologica Sinica, 2013, 45(6): 694-703.
[4] WANG Wen-Yi,DING Shu-Liang,YOU Xiao-Feng. On-Line Item Attribute Identification in Cognitive Diagnostic Computerized Adaptive Testing[J]. , 2011, 43(08): 964-976.
[5] CHEN Ping,XIN Tao. Item Replenishing in Cognitive Diagnostic Computerized Adaptive Testing[J]. , 2011, 43(07): 836-850.
[6] CHENG Xiao-Yang,DING Shu-Liang,YAN Shen-Hai,ZHU Long-Yin. New Item Selection Criteria of Computerized Adaptive Testing with Exposure-Control Factor[J]. , 2011, 43(02): 203-212.
[7] LIU Zhen,DING Shu-Liang,LIN Hai-Jing. Item Selection Strategies for Computerized Adaptive Testing with the Generalized Partial Credit Model[J]. , 2008, 40(05): 618-625.
[8] Dai Haiqi,Chen Dezhi,Ding Shuliang,Deng Taiping. The Comparison Among Item Selection Strategies of CAT with Multiple-choice Items[J]. , 2006, 38(05): 778-783.
[9] Chen-Ping,Ding-Shuliang,Lin-,Zhou-Jie. Item Selection Strategies of Computerized Adaptive Testing based on Graded Response Model[J]. , 2006, 38(03): 461-467.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech