Please wait a minute...
   2006, Vol. 38 Issue (03) : 461-467     DOI:
|
Item Selection Strategies of Computerized Adaptive Testing based on Graded Response Model
Chen-Ping,Ding-Shuliang,Lin-,Zhou-Jie
Computer Information Engineering College, Jiangxi Normal University, Nanchang 330027, China
Download: PDF(593 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  Computerized Adaptive Testing (CAT) is one of the most important testing innovations as the result of the advancement of Item Response Theory (IRT). Consequently, many large-scale tests such the GRE and TOFEL have been transformed from their original paper-and-pencil versions to the current CAT versions. However, one limitation of these CAT tests is their reliance on dichotomous IRT models that require each item be scored as either correct or incorrect. Many measurement applications produce polytomous item response data. In addition, the information provided by a polytomous item is considerably more than that provided by a dichotomously scored item. Therefore, for the purpose of improving test quality, it is important to design CATs based on polytomous IRT models. This research is based on the Graded Response Model (GRM).
Item selection strategy (ISS) is an important component of CAT. Its performance directly affects the security, efficiency and precision of the test. Thus, ISS becomes one of the central issues in CATs based on the GRM. It is well known that the goal of IIS is to administer the next unused item remaining in the item bank that best fits the examinee’s current ability estimate. In dichotomous IRT models, every item has only one difficulty parameter and the item whose difficulty matches the examinee’s current ability estimate is considered to be the best fitting item. However, in GRM, each item has more than two ordered categories and has no single value to represent the item difficulty. Consequently, some researchers have used to employ the average or the median difficulty value across categories as the difficulty estimate for the item. Using the median value in effect introduced two corresponding ISSs.
In this study, we used computer simulation compare four ISSs based on GRM. We also discussed the effect of “shadow pool” on the uniformity of pool usage as well as the influence of different item parameter distributions and different ability estimation methods on the evaluation criteria of CAT. In the simulation process, Monte Carlo method was adopted to simulate the entire CAT process; 1000 examinees drawn from standard normal distribution and four 1000-sized item pools of different item parameter distributions were also simulated. The assumption of the simulation is that a polytomous item is comprised of six ordered categories. In addition, ability estimates were derived using two methods. They were expected a posteriori Bayesian (EAP) and maximum likelihood estimation (MLE). In MLE, the Newton-Raphson iteration method and the Fisher–Score iteration method were employed, respectively, to solve the likelihood equation. Moreover, the CAT process was simulated with each examinee 30 times to eliminate random error. The IISs were evaluated by four indices usually used in CAT from four aspects——the accuracy of ability estimation, the stability of IIS, the usage of item pool, and the test efficiency. Simulation results showed adequate evaluation of the ISS that matched the estimate of an examinee’s current trait level with the difficulty values across categories. Setting “shadow pool” in ISS was able to improve the uniformity of pool utilization. Finally, different distributions of the item parameter and different ability estimation methods affected the evaluation indices of CAT
Keywords graded response model      computerized adaptive testing      item selection strategy      shadow pool     
:  B841  
Corresponding Authors: Ding Shuliang   
Issue Date: 30 May 2006
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Cite this article:   
Chen-Ping,Ding-Shuliang,Lin-,Zhou-Jie. Item Selection Strategies of Computerized Adaptive Testing based on Graded Response Model[J]. ,2006, 38(03): 461-467.
URL:  
http://journal.psych.ac.cn/xlxb/EN/     OR     http://journal.psych.ac.cn/xlxb/EN/Y2006/V38/I03/461
[1] CHEN Ping. Two new online calibration methods for computerized adaptive testing[J]. Acta Psychologica Sinica, 2016, 48(9): 1184-1198.
[2] GUO Lei; ZHENG Chanjin; BIAN Yufang; SONG Naiqing; XIA Lingxiang. New item selection methods in cognitive diagnostic computerized adaptive testing: Combining item discrimination indices[J]. Acta Psychologica Sinica, 2016, 48(7): 903-914.
[3] LIN Zhe; CHEN Pin; XIN Tao. The Block Item Pocket Method to Allow Item Review in CAT[J]. Acta Psychologica Sinica, 2015, 47(9): 1188-1198.
[4] DAI Buyun; ZHANG Minqiang; JIAO Can; LI Guangming; ZHU Huawei; ZHANG Wenyi. Item Selection Using the Multiple-Strategy RRUM Based on CD-CAT[J]. Acta Psychologica Sinica, 2015, 47(12): 1511-1519.
[5] GUO Lei; ZHENG Chanjin; BIAN Yufang. Exposure Control Methods and Termination Rules in Variable-Length Cognitive Diagnostic Computerized Adaptive Testing[J]. Acta Psychologica Sinica, 2015, 47(1): 129-140.
[6] GUO Lei;WANG Zhuoran;WANG Feng;BIAN Yufang. a-Stratified Methods Combining Item Exposure Control and General Test Overlap in Computerized Adaptive Testing[J]. Acta Psychologica Sinica, 2014, 46(5): 702-713.
[7] MAO Xiuzhen; XIN Tao. A Comparison of Item Selection Methods for Cognitive Diagnostic Computerized Adaptive Testing with Nonstatistical Constraints[J]. Acta Psychologica Sinica, 2014, 46(12): 1910-1922.
[8] MAO Xiuzhen;XIN Tao. A Comparison of Item Selection Methods for Controlling Exposure Rate in Cognitive Diagnostic Computerized Adaptive Testing[J]. Acta Psychologica Sinica, 2013, 45(6): 694-703.
[9] LUO Fen,DING Shu-Liang,WANG Xiao-Qing. Dynamic and Comprehensive Item Selection Strategies for Computerized Adaptive Testing Based on Graded Response Model[J]. , 2012, 44(3): 400-412.
[10] TIAN Wei,XIN Tao. A Polytomous Extension of Rule Space Method Based on Graded Response Model[J]. , 2012, 44(2): 249-262.
[11] WANG Wen-Yi,DING Shu-Liang,YOU Xiao-Feng. On-Line Item Attribute Identification in Cognitive Diagnostic Computerized Adaptive Testing[J]. , 2011, 43(08): 964-976.
[12] CHEN Ping,XIN Tao. Item Replenishing in Cognitive Diagnostic Computerized Adaptive Testing[J]. , 2011, 43(07): 836-850.
[13] CHEN Ping,XIN Tao. Developing On-line Calibration Methods for Cognitive Diagnostic Computerized Adaptive Testing[J]. , 2011, 43(06): 710-724.
[14] CHENG Xiao-Yang,DING Shu-Liang,YAN Shen-Hai,ZHU Long-Yin. New Item Selection Criteria of Computerized Adaptive Testing with Exposure-Control Factor[J]. , 2011, 43(02): 203-212.
[15] LUO-Huan,DING Shu-Liang,WANG Wen-Yi,YU Xiao-Feng,CAO Hui-Yuan. Attribute Hierarchy Method Based on Graded Response Model with Different Scoring-Weight for Attributes[J]. , 2010, 42(04): 528-538.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech