ISSN 0439-755X
CN 11-1911/B

心理学报 ›› 2022, Vol. 54 ›› Issue (10): 1277-1292.doi: 10.3724/SP.J.1041.2022.01277

• 研究报告 • 上一篇    


钟小缘1, 喻晓锋1(), 苗莹1, 秦春影2, 彭亚风1, 童昊1   

  1. 1.江西师范大学心理学院, 南昌 330022
    2.南昌师范学院数学与信息科学学院, 南昌 330032
  • 收稿日期:2021-08-25 发布日期:2022-08-24 出版日期:2022-10-25
  • 通讯作者: 喻晓锋
  • 基金资助:

Exploration of change point analysis in detecting speededness based on response time data with known/unknown item parameters

ZHONG Xiaoyuan1, YU Xiaofeng1(), MIAO Ying1, QIN Chunying2, PENG Yafeng1, TONG Hao1   

  1. 1. School of Psychology, Jiangxi Normal University, Nanchang 330022, China
    2. School of Mathematics and Information Science, Nanchang Normal University, Nanchang 330032, China
  • Received:2021-08-25 Online:2022-08-24 Published:2022-10-25
  • Contact: YU Xiaofeng


相对于传统的离散作答数据, 作答时间作为连续数据, 可以提供更多信息。改变点分析(change point analysis)技术在心理和教育领域是一个比较新的技术。本文一方面对改变点分析在心理测量领域的应用进行了一个综合的总结和分析; 另一方面, 将基于作答数据的两种改变点分析统计量推广到作答时间数据, 将改变点分析技术应用到测验异常作答模式:加速作答speededness的检测上。采用两种检验方法:似然比检验和Wald检验, 分别在已知和未知项目参数的条件下, 实现异常作答模式的检测。结果表明, 所采用的方法对于加速作答行为的检测具有很高的检验力, 同时能够很好的控制I类错误率。实证数据分析进一步表明本文中所使用的方法具有应用价值。

关键词: 改变点分析法, 异常作答行为, 作答时间, 加速作答, 统计过程控制


In recent years, response time has received a rapidly growing amount of attention in psychometric research, likely due to the increasing availability of (item-level) response time data through computer-based testing and online survey data collection. Compared to the conventional item response data that are often dichotomous or polytomous, the response time is continuous and can provide much more information. Aberrant response behaviors are frequently encountered during testing. It could cause various negative effects. Change point analysis (CPA) is a well-established statistical process control method to detect changes in a sequence, and it has provided testing professionals a new lens through to understand test-taking behavior at both the examinee and item levels.

In this paper, we took test speededness as an example to illustrate how the CPA method can be used to detect aberrant behavior using item response time data. Response time under speededness was simulated using the gradual-change log-normal model for response time. Two CPA-based test statistics, the Likelihood Ratio Test and Wald Test, were used to detect aberrant response behaviors. The critical values were obtained through Monte Carlo simulations and compared with the approximate critical values in a previous study. Based on the chosen critical values, we examined the performance of the likelihood ratio test and Wald test in detecting speeded responses, specifically in terms of power and empirical Type-I error.

On the one hand, the critical values are almost identical for Wald and the likelihood ratio test. They vary substantially at different nominal α levels, but do not differ much across different test lengths. On the other hand, compared to approximate critical values, the critical values are not too far away from them but are different. That may be because the approximate critical values are suitable for situations where the change point appears in the middle of the test. Results indicate that the proposed method is much more powerful based on the critical values than conventional methods that use item response data. The power was close to 1 for most of the conditions while keeping the type-I error rate well-controlled. Real data analysis also demonstrates the performance of the method.

This study uses CPA with response time data and offers a very promising approach to detecting aberrant response behavior. Through the simulation study, we demonstrated that it is possible to use fixed critical values in different test lengths, which makes the application of the method straightforward. It also means that it is unnecessary to reconduct the simulation to update critical values when small changes occur in the test. CPA is very flexible. This study assumed that the log-normal model fits the response time data, but the method is not bounded by that assumption.

Key words: change point analysis, aberrant response behaviors, response time, test speededness, statistical process control