ISSN 0439-755X
CN 11-1911/B
主办：中国心理学会
　　　中国科学院心理研究所
出版：科学出版社

心理学报 ›› 2014, Vol. 46 ›› Issue (5): 702-713.doi: 10.3724/SP.J.1041.2014.00702

• 论文 • 上一篇    下一篇

结合a分层的兼具项目曝光和广义测验重叠率控制的选题策略

郭磊;王卓然;王丰;边玉芳   

  1. (1北京师范大学认知神经科学与学习国家重点实验室; 2中国基础教育质量评价与提升协同创新中心, 北京 100875)
  • 收稿日期:2013-06-14 发布日期:2014-05-24 出版日期:2014-05-24
  • 通讯作者: 边玉芳
  • 基金资助:

    高等学校博士学科点专项科研基金资助课题(20120003110002)的资助。

a-Stratified Methods Combining Item Exposure Control and General Test Overlap in Computerized Adaptive Testing

GUO Lei;WANG Zhuoran;WANG Feng;BIAN Yufang   

  1. (1 National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing 100875, China) (2 National Cooperative Innovation Center for Assessment and Improvement of Basic Education Quality, Beijing Normal University, Beijing 100875, China)
  • Received:2013-06-14 Online:2014-05-24 Published:2014-05-24
  • Contact: BIAN Yufang

PDF (PC)

967

摘要：

测验安全和题库使用率在计算机化自适应测验中十分重要, 特别是高风险测验。传统的SHGT法兼具同时控制项目曝光率和广义测验重叠率的功能, 但题库使用率较差。a分层法能够提高题库使用率, 但对过度曝光的项目控制不足。本研究将a分层法的思想与SHGT法相结合, 各取所长, 提出了3种新的选题方法：SHGT_a法, SHGT_b法和SHGT_c法。研究结果表明：(1)与SHGT法相比, 新方法均可以在有效地控制项目曝光率和广义测验重叠率同时, 极大地提高题库使用率; (2)随着预设项目曝光率(rmax)和广义测验重叠率( )取值的增大以及共享人数a的减小, 新方法对被试能力估计的精度呈上升趋势。比起SHGT法, 新方法仍能保持很高的题库使用率; (3)当区分度和难度的相关(rab)较大时, SHGT_b和SHGT_c法在能力估计精度方面优于SHGT_a法; (4)在不同的测验考察内容比例下, 3种新方法对被试能力估计的精度均较好; (5)与SHGT法相比, 新方法能够有效地控制项目曝光率过度控制的问题。

关键词: 项目曝光率, 广义测验重叠率, 计算机化自适应测验, a分层法, 选题策略

Abstract:

Test security and item pool utilization rate are very important in computerized adaptive testing (CAT), especially in the high-stakes tests. Most existing methods only focus on the item exposure rate, but rarely control the test overlap rate. Way (1998) suggested that the item exposure and the test overlap rate be two indices of test security. Following this reasoning, Chen (2010) proposed an on–line version of the Sympson-Hetter procedure with general test overlap control (SHGT) that didn’t need iterative simulations. Although the SHGT method could control item exposure and general test overlap simultaneously without iterative simulations, the item pool utilization rate was not very ideal when the item exposure or test overlap rate was slightly high or the number of examinees who shared the information with another examinee was small. Thus, the test security was threatened. To address the limitation of the SHGT method, we combined the a-stratified method with the SHGT method, and proposed three new methods: SHGT_a method, SHGT_b method, and SHGT_c method. Simulation results indicated that: (1) Compared to the SHGT method, these three new methods could not only improve the item pool utilization rate, but also maintain a very high precision of ability estimate in the same experiment condition; (2) With the increase of maximum item exposure rate (rmax) and maximum general test overlap rate ( ), and the decrease of a, the precision of ability estimation increased. Compared to the SHGT method, these three new methods could maintain a higher item pool utilization rate; (3) SHGT_b and SHGT_c outperformed SHGT_a in the aspect of the precision of ability estimation when rab was high; (4) The three new methods had a good performance in respect of the precision of ability estimation; (5) Compared to the SHGT method, these three new methods could also effectively solve the problem of item exposure over-control. Some future directions of study were suggested at the end of this paper.

Key words: item exposure rate, general test overlap rate, computerized adaptive testing, a-stratified method, selection strategy