Please wait a minute...
心理学报
  论文 本期目录 | 过刊浏览 | 高级检索 |
结合a分层的兼具项目曝光和广义测验重叠率控制的选题策略
郭磊;王卓然;王丰;边玉芳
(1北京师范大学认知神经科学与学习国家重点实验室; 2中国基础教育质量评价与提升协同创新中心, 北京 100875)
a-Stratified Methods Combining Item Exposure Control and General Test Overlap in Computerized Adaptive Testing
GUO Lei;WANG Zhuoran;WANG Feng;BIAN Yufang
(1 National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China) (2 National Cooperative Innovation Center for Assessment and Improvement of Basic Education Quality, Beijing Normal University, Beijing 100875, China)
全文: PDF(293 KB)   评审附件 (1 KB) 
输出: BibTeX | EndNote (RIS)      
摘要 

测验安全和题库使用率在计算机化自适应测验中十分重要, 特别是高风险测验。传统的SHGT法兼具同时控制项目曝光率和广义测验重叠率的功能, 但题库使用率较差。a分层法能够提高题库使用率, 但对过度曝光的项目控制不足。本研究将a分层法的思想与SHGT法相结合, 各取所长, 提出了3种新的选题方法:SHGT_a法, SHGT_b法和SHGT_c法。研究结果表明:(1)与SHGT法相比, 新方法均可以在有效地控制项目曝光率和广义测验重叠率同时, 极大地提高题库使用率; (2)随着预设项目曝光率(rmax)和广义测验重叠率( )取值的增大以及共享人数a的减小, 新方法对被试能力估计的精度呈上升趋势。比起SHGT法, 新方法仍能保持很高的题库使用率; (3)当区分度和难度的相关(rab)较大时, SHGT_b和SHGT_c法在能力估计精度方面优于SHGT_a法; (4)在不同的测验考察内容比例下, 3种新方法对被试能力估计的精度均较好; (5)与SHGT法相比, 新方法能够有效地控制项目曝光率过度控制的问题。

服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
郭磊
王卓然
王丰
边玉芳
关键词 项目曝光率广义测验重叠率计算机化自适应测验a分层法选题策略    
Abstract

Test security and item pool utilization rate are very important in computerized adaptive testing (CAT), especially in the high-stakes tests. Most existing methods only focus on the item exposure rate, but rarely control the test overlap rate. Way (1998) suggested that the item exposure and the test overlap rate be two indices of test security. Following this reasoning, Chen (2010) proposed an on–line version of the Sympson-Hetter procedure with general test overlap control (SHGT) that didn’t need iterative simulations. Although the SHGT method could control item exposure and general test overlap simultaneously without iterative simulations, the item pool utilization rate was not very ideal when the item exposure or test overlap rate was slightly high or the number of examinees who shared the information with another examinee was small. Thus, the test security was threatened. To address the limitation of the SHGT method, we combined the a-stratified method with the SHGT method, and proposed three new methods: SHGT_a method, SHGT_b method, and SHGT_c method. Simulation results indicated that: (1) Compared to the SHGT method, these three new methods could not only improve the item pool utilization rate, but also maintain a very high precision of ability estimate in the same experiment condition; (2) With the increase of maximum item exposure rate (rmax) and maximum general test overlap rate ( ), and the decrease of a, the precision of ability estimation increased. Compared to the SHGT method, these three new methods could maintain a higher item pool utilization rate; (3) SHGT_b and SHGT_c outperformed SHGT_a in the aspect of the precision of ability estimation when rab was high; (4) The three new methods had a good performance in respect of the precision of ability estimation; (5) Compared to the SHGT method, these three new methods could also effectively solve the problem of item exposure over-control. Some future directions of study were suggested at the end of this paper.

Key wordsitem exposure rate    general test overlap rate    computerized adaptive testing    a-stratified method    selection strategy
收稿日期: 2013-06-14      出版日期: 2014-05-24
基金资助:

高等学校博士学科点专项科研基金资助课题(20120003110002)的资助。

通讯作者: 边玉芳   
引用本文:   
郭磊;王卓然;王丰;边玉芳. 结合a分层的兼具项目曝光和广义测验重叠率控制的选题策略[J]. 心理学报, 10.3724/SP.J.1041.2014.00702.
GUO Lei;WANG Zhuoran;WANG Feng;BIAN Yufang. a-Stratified Methods Combining Item Exposure Control and General Test Overlap in Computerized Adaptive Testing. Acta Psychologica Sinica, 2014, 46(5): 702-713.
链接本文:  
http://journal.psych.ac.cn/xlxb/CN/10.3724/SP.J.1041.2014.00702      或      http://journal.psych.ac.cn/xlxb/CN/Y2014/V46/I5/702
[1] 陈平. 两种新的计算机化自适应测验在线标定方法[J]. 心理学报, 2016, 48(9): 1184-1198.
[2] 郭磊; 郑蝉金; 边玉芳; 宋乃庆; 夏凌翔. 认知诊断计算机化自适应测验中新的选题策略:结合项目区分度指标[J]. 心理学报, 2016, 48(7): 903-914.
[3] 林喆;陈平;辛涛. 允许CAT题目检查的区块题目袋方法[J]. 心理学报, 2015, 47(9): 1188-1198.
[4] 罗照盛;喻晓锋;高椿雷;李喻骏;彭亚风;王 睿;王钰彤. 基于属性掌握概率的认知诊断计算机化自适应测验选题策略[J]. 心理学报, 2015, 47(5): 679-688.
[5] 郭磊;郑蝉金;边玉芳. 变长CD-CAT中的曝光控制与终止规则[J]. 心理学报, 2015, 47(1): 129-140.
[6] 毛秀珍;辛涛. 认知诊断CAT中具有非统计约束选题方法的比较[J]. 心理学报, 2014, 46(12): 1910-1922.
[7] 毛秀珍;辛涛. 认知诊断CAT中项目曝光控制方法的比较[J]. 心理学报, 2013, 45(6): 694-703.
[8] 罗芬,丁树良,王晓庆. 多级评分计算机化自适应测验动态综合选题策略[J]. , 2012, 44(3): 400-412.
[9] 陈平,辛涛. 认知诊断计算机化自适应测验中的项目增补[J]. , 2011, 43(07): 836-850.
[10] 陈平,辛涛. 认知诊断计算机化自适应测验中在线标定方法的开发[J]. , 2011, 43(06): 710-724.
[11] 程小扬,丁树良,严深海,朱隆尹. 引入曝光因子的计算机化自适应测验选题策略[J]. , 2011, 43(02): 203-212.
[12] 陈平,丁树良. 允许检查并修改答案的计算机化自适应测验[J]. , 2008, 40(06): 737-747.
[13] 刘珍,丁树良,林海菁. 基于GPCM的计算机自适应测验选题策略比较[J]. , 2008, 40(05): 618-625.
[14] 林海菁,丁树良. 具有认知诊断功能的计算机化自适应测验的研究与实现[J]. , 2007, 39(04): 747-753.
[15] 戴海琦,陈德枝,丁树良,邓太萍. 多级评分题计算机自适应测验选题策略比较[J]. , 2006, 38(05): 778-783.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
版权所有 © 《心理学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发  技术支持:support@magtech.com.cn