ISSN 0439-755X
CN 11-1911/B
主办：中国心理学会
　　　中国科学院心理研究所
出版：科学出版社

心理学报 ›› 2015, Vol. 47 ›› Issue (1): 129-140.doi: 10.3724/SP.J.1041.2015.00129

• 论文 • 上一篇    下一篇

变长CD-CAT中的曝光控制与终止规则

郭磊1,2;郑蝉金3;边玉芳2,4   

  1. (1西南大学心理学部, 重庆 400715) (2北京师范大学认知神经科学与学习国家重点实验室, 北京 100875) (3伊利诺伊大学香槟分校教育心理学系, 香槟, 伊利诺伊州 61820 美国) (4中国基础教育质量评价与提升协同创新中心, 北京 100875)
  • 收稿日期:2013-10-30 发布日期:2015-01-26 出版日期:2015-01-26
  • 通讯作者: 边玉芳, E-mail: bianyufang66@126.com
  • 基金资助:

    高等学校博士学科点专项科研基金资助课题(20120003110002)的资助。

Exposure Control Methods and Termination Rules in Variable-Length Cognitive Diagnostic Computerized Adaptive Testing

GUO Lei1,2; ZHENG Chanjin3; BIAN Yufang2,4   

  1. (1 Faculty of Psychology, Southwest University, Chongqing 400715, China) (2 National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China) (3 Educational Psychology, University of Illinois at Urbana-Champagn, Champaign, IL, 61820, USA) (4 National Cooperative Innovation Center for Assessment and Improvement of Basic Education Quality, Beijing Normal University, Beijing 100875, China)
  • Received:2013-10-30 Online:2015-01-26 Published:2015-01-26
  • Contact: BIAN Yufang, E-mail: bianyufang66@126.com

PDF (PC)

1395

摘要：

本研究借鉴传统计算机化自适应测验的思想, 并结合认知诊断的特点, 在认知诊断框架下提出了4种变长CD-CAT的终止规则, 分别是属性标准误法(SEA)、邻近后验概率之差法(DAPP)、二等分法(HA)以及混合法(HM)。在未控制曝光和采用不同曝光控制条件下, 与HSU法及KL法进行了比较。研究结果表明：(1) 终止条件越严格, 平均测验长度越长, 按测验长度最大值终止的测验百分比越大, 模式判准率越高。(2) 当未加入曝光控制时, 4种新的终止规则均有较好表现, 与HSU法十分接近。随着最大后验概率预设值的增加或e的减小, 模式判准率呈上升趋势, 平均测验长度逐渐增加, 但在题库使用率方面均较差。(3) 当加入项目曝光控制时, 6种变长终止规则下的题库使用率有了极大的提升, 仍能保持较高的模式判准率, 并且不同的曝光控制方法对终止规则的影响是不同的。其中, 相对标准终止规则极易受到曝光控制方法的影响。(4) 综合来看, SEA、HM以及HA法在各项指标上的表现与HSU法基本一致, 其次为KL法和DAPP法。

关键词: 认知诊断计算机化自适应测验, 变长终止规则, 曝光控制, 判准率, DINA模型

Abstract:

Comparing to the nonadaptive testing, the major advantage of computerized adaptive testing (CAT) is that the examinees achieve the same degree of measurement precision (i.e., fixed precision). But few studies are devoted to the termination rules in variable-length cognitive diagnostic computerized adaptive testing (CD-CAT). Inspired by the termination rule research in traditional CAT, this paper proposed four termination rules for variable-length CD-CAT. The new termination rules were standard error of attribute method (SEA), difference of the adjacent posterior probability method (DAPP), halving algorithm (HA) and hybrid method (HM), respectively. Then, the four new termination rules were compared with the HSU and KL method under two scenarios: with and without item exposure control. Three exposure control methods were considered, i.e., simple, modified restrictive progressive (MRP) and modified restrictive threshold (MRT) method. The MRP and MRT methods were extension of the Wang et al.’s (2011) work to the variable-length CD-CAT scenario. The results indicated that: (1) When the criterion of variable-length termination rule was conservative, the mean of the test length and the percentage of examinees reaching the maximum test length were large, and the classification accuracy rate for examinees who finished the CAT using fixed precision was high. (2) Without the item exposure control, the four new variable-length termination rules had a similar performance compared to the HSU method. With the increase of maximum posterior probability and the decrease of e, the classification accuracy rate and the mean test length presented a increasing trend. But the item pool usage was unsatisfactory. (3) With the item exposure control, item pool usage was greatly improved in the six variable-length termination rules while the classification accuracy rates were maintained. Different exposure control methods had a different effect on the different variable-length termination rules. The relative criterion termination rules such as DAPP and KL methods were easily affected by the item exposure control. (4) Taken all together, the SEA, HM, and HA methods were comparable to the HSU method, and followed by the KL and DAPP method. Some future directions were suggested in the end of this paper.

Key words: cognitive diagnostic computerized adaptive testing, variable-length termination rule, exposure control, classification accuracy rate, DINA model