作答时间与反应依赖关系建模：基于双因子模型视角

doi:10.3724/SP.J.1041.2024.00352

摘要/Abstract

摘要：

在心理与教育测验中, 测验的计算机化越来越普遍, 使得被试作答的过程性数据的搜集也越来越便利。分层模型的提出为作答时间与反应的联合分析提供了一个基本的建模框架, 且逐渐成为当前最流行的方法。虽然分层模型被广泛使用, 但仅仅通过参数间的关系还不能很好地解释作答时间和反应之间的关系。因此, 一些研究者提出了一系列改进模型, 但仍然存在一些不足。基于双因子模型的新视角, 文中将测验的作答时间与反应分别视为测量被试速度和能力的两个局部因子, 而作答时间与反应又视为综合测量了被试的速度与准确率权衡的一般能力或全局因子。基于此, 文中提出双因子分层模型, 以探讨作答时间与反应的依赖关系。模拟研究发现Mplus程序能有效估计双因子分层模型的各参数, 而忽视作答时间与反应依赖关系的分层模型的参数估计结果存在明显的偏差。在实例数据分析中, 相较于分层模型, 双因子分层模型的各模型拟合指数表现更好。此外, 不同被试在不同项目上的作答时间与反应存在不同的依赖关系, 从而对被试的作答准确率与时间产生不同的影响。

关键词: 作答时间, 反应, 依赖关系, 分层模型, 双因子模型

Abstract:

In the realms of psychological and educational testing, the computerization of tests is becoming more prevalent, facilitating the acquisition of process data from test-takers. In the domain of process data, response time and response represent the two most commonly utilized variables. Responses provide critical insights into the answers provided by test-takers, while response time, as an essential source of information, is increasingly garnering attention from researchers. The proposal of hierarchical model (HM) has provided a fundamental modeling framework for the joint analysis of response time and response, and it is becoming increasingly popular in current research practices. However, relying solely on the association between item and subject parameters is insufficient to adequately explain the correlation between response time and response. Consequently, researchers have proposed various enhanced models to address these limitations, although some challenges persist.

The bifactor model explains common variance through a general or global factor, while a local or specific factor explains the common variance of additional partial items. In psychological and educational testing, it is possible to capture not only the test-takers’ response times on test items but also their responses. From the perspective of the bifactor model, response times and responses to test items measure different local factors. Specifically, a test's response time measures the test-taker's speed trait, while the response to the test measures their ability trait. Test-takers are also influenced by a combination of time and accuracy when responding to the test, known as general latent traits or global factors, or speed-accuracy trade-off ability. This test structure aligns well with the bifactor model and provides a new perspective on analyzing the relationship between test-taking response time and response dependence. Based on this, this study proposes a bifactor hierarchical model (Bi-HM) to explore the dependency between response time and response.

In the simulation study, it was found that the MPLUS program utilizing MLR (Maximum Likelihood Robust), could accurately estimate the parameters of the Bi-HM and was not influenced by the level of item parameter correlation. Conversely, when disregarding the relationship between response time and response in the HM, notable bias in the parameter estimates occured. In the empirical data analysis, the Bi-HM demonstrated significantly superior model fit indices compared to the HM. Moreover, the Bi-HM effectively captured the dependency between response and response time at both the participant and item levels. This dependency is closely associated with item difficulty and time intensity factors.

Based on the findings mentioned above, it is evident that the Bi-HM, which adopts a bifactor model perspective, excels in parameter estimation and data fitting, demonstrating excellent scalability.

Key words: response time, response, dependency relationship, hierarchical model, bifactor model

中图分类号:

B841

郭小军, 柏小云, 罗照盛. (2024). 作答时间与反应依赖关系建模：基于双因子模型视角. 心理学报, 56(3), 352-362.

GUO Xiaojun, BAI Xiaoyun, LUO Zhaosheng. (2024). Modeling the dependence between response and response time: A bifactor model approach. Acta Psychologica Sinica, 56(3), 352-362.

图/表 10

参考文献 33

[1]	Bolsinova, M., De Boeck, P., & Tijmstra, J. (2017). Modelling conditional dependence between response time and accuracy. Psychometrika, 82(4), 1126-1148. doi: 10.1007/s11336-016-9537-6 pmid: 27738955
[2]	Bolsinova, M., & Molenaar, D. (2018). Modeling nonlinear conditional dependence between response time and accuracy. Frontiers in Psychology, 9, 1525. doi: 10.3389/fpsyg.2018.01525 pmid: 30245650
[3]	Bolsinova, M., & Tijmstra, J. (2016). Posterior predictive checks for conditional independence between response time and accuracy. Journal of Educational and Behavioral Statistics, 41(2), 123-145. doi: 10.3102/1076998616631746 URL
[4]	Bolsinova, M., Tijmstra, J., & Molenaar, D. (2017). Response moderation models for conditional dependence between response time and response accuracy. British Journal of Mathematical and Statistical Psychology, 70(2), 257-279. doi: 10.1111/bmsp.2017.70.issue-2 URL
[5]	Cai, L., Yang, J. S., & Hansen, M. (2011). Generalized full-information item bifactor analysis. Psychological Methods, 16(3), 221-248. doi: 10.1037/a0023350 pmid: 21534682
[6]	Chen, H., De Boeck, P., Grady, M., Yang, C.-L., & Waldschmidt, D. (2018). Curvilinear dependency of response accuracy on response time in cognitive tests. Intelligence, 69, 16-23. doi: 10.1016/j.intell.2018.04.001 URL
[7]	Davison, M. L., Semmes, R., Huang, L., & Close, C. N. (2012). On the reliability and validity of a numerical reasoning speed dimension derived from response times collected in computerized testing. Educational and Psychological Measurement, 72(2), 245-263. doi: 10.1177/0013164411408412 URL
[8]	Guo, X., & Luo, Z. (2019). The trade-off between speed and accuracy: Evaluation and modeling for participants in the response states. Studies of Psychology and Behavior, 17(5), 589-595.
	[ 郭小军, 罗照盛. (2019). 速度与准确率权衡: 被试反应状态评价与建模. 心理与行为研究, 17(5), 589-595.]
[9]	Guo, X., Luo, Z., & Yan., J. (2022). Modeling of the effect on multidimensional latent speeds in the between-item multidimensional response time. Journal of Psychological Science, 45(5), 1222-1229.
	[ 郭小军, 罗照盛, 严娟. (2022). 项目间多维测验作答时间数据分析: 潜在特质速度间效应建模. 心理科学, 45(5), 1222-1229.]
[10]	Guo, X., Luo, Z., & Yu, X. (2020). A speed-accuracy tradeoff hierarchical model based on cognitive experiment. Frontiers in Psychology, 10, 2910. doi: 10.3389/fpsyg.2019.02910 URL
[11]	Klein Entink, R. H., Kuhn, J.-T., Hornke, L. F., & Fox, J.-P. (2009). Evaluating cognitive theory: A joint modeling approach using responses and response times. Psychological Methods, 14(1), 54-75. doi: 10.1037/a0014877 pmid: 19271848
[12]	Klein Entink, R. H., van der Linden, W. J., & Fox, J.-P. (2009). A box-cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62(3), 621-640. doi: 10.1348/000711008X374126 URL
[13]	Liang, K., Tu, D., & Cai, Y. (2023). Using process data to improve classification accuracy of cognitive diagnosis model. Multivariate Behavioral Research, 58(5), 969-987. doi: 10.1080/00273171.2022.2157788 URL
[14]	Loeys, T., Rosseel, Y., & Baten, K. (2011). A joint modeling approach for reaction time and accuracy in psycholinguistic experiments. Psychometrika, 76(3), 487-503. doi: 10.1007/s11336-011-9211-y URL
[15]	Lu, J., Wang, C., & Shi, N. (2021). A mixture response time process model for aberrant behaviors and item nonresponses. Multivariate Behavioral Research, 58(1), 71-89. doi: 10.1080/00273171.2021.1948815 URL
[16]	Luce, R. D. (1986). Response times: Their role in inferring elementary mental organization. New York: Oxford University Press.
[17]	Mao, X., Xia, M., & Xin, T. (2018). Full-information item bifactor analysis: Model parameter estimation and application. Advances in Psychological Science, 26(2), 358-367. doi: 10.3724/SP.J.1042.2018.00358 URL
	[ 毛秀珍, 夏梦连, 辛涛. (2018). 全信息项目双因子分析: 模型、参数估计及其应用. 心理科学进展, 26(2), 358-367.] doi: 10.3724/SP.J.1042.2018.00358
[18]	Meng, X.-B. (2016). A log-skew-normal model for item response times. Journal of Psychological Science, 39(3), 727-734.
	[ 孟祥斌. (2016). 项目反应时间的对数偏正态模型. 心理科学, 39(3), 727-734.]
[19]	Meng, X.-B., Tao, J., & Chang, H.-H. (2015). A conditional joint modeling approach for locally dependent item responses and response times. Journal of Educational Measurement, 52(1), 1-27. doi: 10.1111/jedm.2015.52.issue-1 URL
[20]	Molenaar, D., Tuerlinckx, F., & van der Maas, H. L. J. (2015). A generalized linear factor model approach to the hierarchical framework for responses and response times. British Journal of Mathematical and Statistical Psychology, 68(2), 197-219. doi: 10.1111/bmsp.2015.68.issue-2 URL
[21]	Muthén, L. K., & Muthén, B. O. (2019). Mplus: The comprehensive modeling program for applied researchers: user's guide. Muthén & Muthén.
[22]	Scherer, R., Greiff, S., & Hautamäki, J. (2015). Exploring the relation between time on task and ability in complex problem solving. Intelligence, 48, 37-50. doi: 10.1016/j.intell.2014.10.003 URL
[23]	van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287-308. doi: 10.1007/s11336-006-1478-z URL
[24]	van der Linden, W. J. (2009). Conceptual issues in response- time modeling. Journal of Educational Measurement, 46(3), 247-272. doi: 10.1111/jedm.2009.46.issue-3 URL
[25]	van der Linden, W. J., & Glas, C. A. W. (2010). Statistical tests for conditional independence in a hierarchical model for speed and accuracy on test items. Psychometrika, 75(1), 120-139. doi: 10.1007/s11336-009-9129-9 URL
[26]	Wang, C., Chang, H.-H., & Douglas, J. A. (2013). The linear transformation model with frailties for the analysis of item response times. British Journal of Mathematical and Statistical Psychology, 66(1), 144-168. doi: 10.1111/bmsp.2013.66.issue-1 URL
[27]	Wei, D., Liu, H., & Zhang, D. (2017). Multidimentional rasch testlet model: An extension and generalization of MRCMLM. Acta Psychologica Sinica, 49(12), 1604-1614. doi: 10.3724/SP.J.1041.2017.01604 URL
	[ 魏丹, 刘红云, 张丹慧. (2017). 多维题组反应模型:多维随机系数多项Logistic模型的应用拓展. 心理学报, 49(12), 1604-1614.]
[28]	Ranger, J. (2013). Modeling responses and response times in personality tests with rating scales. Psychological Test and Assessment Modeling, 55(4), 361-382.
[29]	Ranger, J., & Ortner, T. (2012). The case of dependency of responses and response times: A modeling approach based on standard latent trait models. Psychological Test and Assessment Modeling, 54(2), 128-148.
[30]	Zhan, P. (2022). Joint-cross-loading multimodal cognitive diagnostic modeling incorporating visual fixation counts. Acta Psychologica Sinica, 54(11), 1416-1423. doi: 10.3724/SP.J.1041.2022.01416
	[ 詹沛达. (2022). 引入眼动注视点的联合-交叉负载多模态认知诊断建模. 心理学报, 54(11), 1416-1423.] doi: 10.3724/SP.J.1041.2022.01416
[31]	Zhan, P., Jiao, H., & Liao, D. (2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71(2), 262-286. doi: 10.1111/bmsp.2018.71.issue-2 URL
[32]	Zhan, P., Wang, W.-C., & Wang, L. (2013). Testlet response theory: An introduction and new developments. Advances in Psychological Science, 21(12), 2265-2280. doi: 10.3724/SP.J.1042.2013.02265
	[ 詹沛达, 王文中, 王立君. (2013). 项目反应理论新进展之题组反应理论. 心理科学进展, 21(12), 2265-2280.] doi: 10.3724/SP.J.1042.2013.02265
[33]	Zheng, C.-J., Guo, C.-Y., & Bian, Y.-F. (2011). Using testlet DIF procedures to detect testlet DIF in chinese passage-based reading testing. Acta Psychologica Sinica, 43(7), 830-835.
	[ 郑蝉金, 郭聪颖, 边玉芳. (2011). 变通的题组项目功能差异检验方法在篇章阅读测验中的应用. 心理学报, 43(7), 830-835.]

参数	关系类型
参数	类型一	类型二		类型三		类型四		类型五
${{\lambda }_{1j}}$	0	< 0	> 0	> 0	< 0	> 0	< 0	> 0	< 0
${{\lambda }_{2j}}$	0	< 0	> 0	> 0	< 0	< 0	> 0	< 0	> 0
${{\eta }_{i}}$	——	< 0	> 0	< 0	> 0	> 0	< 0	< 0	> 0
关系内涵	独立	注重准确率, 忽视速度。		注重作答速度, 忽视准确率。		既注重准确率, 又注重速度。		作答速度慢且准确率低。

参数	关系类型
参数	类型一	类型二		类型三		类型四		类型五
${{\lambda }_{1j}}$	0	< 0	> 0	> 0	< 0	> 0	< 0	> 0	< 0
${{\lambda }_{2j}}$	0	< 0	> 0	> 0	< 0	< 0	> 0	< 0	> 0
${{\eta }_{i}}$	——	< 0	> 0	< 0	> 0	> 0	< 0	< 0	> 0
关系内涵	独立	注重准确率, 忽视速度。		注重作答速度, 忽视准确率。		既注重准确率, 又注重速度。		作答速度慢且准确率低。

项目参数	模型	${{\rho }_{b\beta }}$	N = 500				N=1000
			m = 30		m = 60		m = 30		m = 60
			MSE	Bias	MSE	Bias	MSE	Bias	MSE	Bias
a	HM	0	0.108	0.031	0.099	0.029	0.103	0.035	0.083	0.026
		0.4	0.123	0.021	0.106	0.041	0.091	0.023	0.085	0.034
		0.8	0.101	0.035	0.102	0.026	0.091	0.045	0.096	0.046
	Bi-HM	0	0.028	−0.012	0.024	−0.016	0.013	−0.012	0.011	−0.013
		0.4	0.028	−0.015	0.024	−0.004	0.012	−0.013	0.011	−0.009
		0.8	0.026	0	0.025	−0.015	0.012	−0.004	0.011	0.002
d	HM	0	0.028	0.002	0.027	0.002	0.02	−0.003	0.018	0.003
		0.4	0.026	0.007	0.025	0.003	0.018	0.008	0.018	−0.003
		0.8	0.027	0.009	0.026	0.013	0.019	0	0.018	−0.002
	Bi-HM	0	0.02	−0.003	0.02	0.005	0.01	0	0.009	0.003
		0.4	0.021	0.012	0.019	0.002	0.01	0.009	0.009	−0.004
		0.8	0.02	0.005	0.021	0.013	0.01	0	0.009	0
β	HM	0	0.006	−0.001	0.006	0	0.003	−0.01	0.003	0.012
		0.4	0.007	0.009	0.006	0.002	0.002	−0.002	0.003	0.003
		0.8	0.006	−0.001	0.006	0.01	0.003	−0.001	0.003	0.003
	Bi-HM	0	0.006	−0.001	0.006	0	0.003	−0.009	0.003	0.012
		0.4	0.007	0.009	0.006	0.002	0.003	−0.002	0.003	0.003
		0.8	0.006	−0.001	0.006	0.01	0.003	−0.001	0.003	0.003
α	HM	0	0.094	0.189	0.089	0.18	0.087	0.182	0.094	0.192
		0.4	0.085	0.175	0.088	0.182	0.089	0.182	0.099	0.196
		0.8	0.096	0.187	0.092	0.188	0.09	0.185	0.093	0.189
	Bi-HM	0	0.001	−0.003	0.001	−0.004	0.001	−0.004	0.001	−0.002
		0.4	0.002	−0.004	0.001	−0.004	0.001	−0.002	0.001	−0.003
		0.8	0.002	−0.005	0.001	−0.004	0.001	−0.002	0.001	−0.002
${{\lambda }_{1}}$	Bi-HM	0	0.02	0.003	0.019	−0.008	0.009	−0.008	0.01	−0.006
		0.4	0.021	−0.012	0.02	−0.004	0.01	0.001	0.01	−0.008
		0.8	0.019	0.003	0.021	−0.008	0.01	0.007	0.009	−0.002
${{\lambda }_{2}}$	Bi-HM	0	0.006	0.001	0.005	0.007	0.003	0.005	0.002	−0.001
		0.4	0.006	−0.002	0.005	0.004	0.003	−0.006	0.003	0.005
		0.8	0.006	−0.001	0.005	0.003	0.003	−0.003	0.003	−0.005

项目参数	模型	${{\rho }_{b\beta }}$	N = 500				N=1000
			m = 30		m = 60		m = 30		m = 60
			MSE	Bias	MSE	Bias	MSE	Bias	MSE	Bias
a	HM	0	0.108	0.031	0.099	0.029	0.103	0.035	0.083	0.026
		0.4	0.123	0.021	0.106	0.041	0.091	0.023	0.085	0.034
		0.8	0.101	0.035	0.102	0.026	0.091	0.045	0.096	0.046
	Bi-HM	0	0.028	−0.012	0.024	−0.016	0.013	−0.012	0.011	−0.013
		0.4	0.028	−0.015	0.024	−0.004	0.012	−0.013	0.011	−0.009
		0.8	0.026	0	0.025	−0.015	0.012	−0.004	0.011	0.002
d	HM	0	0.028	0.002	0.027	0.002	0.02	−0.003	0.018	0.003
		0.4	0.026	0.007	0.025	0.003	0.018	0.008	0.018	−0.003
		0.8	0.027	0.009	0.026	0.013	0.019	0	0.018	−0.002
	Bi-HM	0	0.02	−0.003	0.02	0.005	0.01	0	0.009	0.003
		0.4	0.021	0.012	0.019	0.002	0.01	0.009	0.009	−0.004
		0.8	0.02	0.005	0.021	0.013	0.01	0	0.009	0
β	HM	0	0.006	−0.001	0.006	0	0.003	−0.01	0.003	0.012
		0.4	0.007	0.009	0.006	0.002	0.002	−0.002	0.003	0.003
		0.8	0.006	−0.001	0.006	0.01	0.003	−0.001	0.003	0.003
	Bi-HM	0	0.006	−0.001	0.006	0	0.003	−0.009	0.003	0.012
		0.4	0.007	0.009	0.006	0.002	0.003	−0.002	0.003	0.003
		0.8	0.006	−0.001	0.006	0.01	0.003	−0.001	0.003	0.003
α	HM	0	0.094	0.189	0.089	0.18	0.087	0.182	0.094	0.192
		0.4	0.085	0.175	0.088	0.182	0.089	0.182	0.099	0.196
		0.8	0.096	0.187	0.092	0.188	0.09	0.185	0.093	0.189
	Bi-HM	0	0.001	−0.003	0.001	−0.004	0.001	−0.004	0.001	−0.002
		0.4	0.002	−0.004	0.001	−0.004	0.001	−0.002	0.001	−0.003
		0.8	0.002	−0.005	0.001	−0.004	0.001	−0.002	0.001	−0.002
${{\lambda }_{1}}$	Bi-HM	0	0.02	0.003	0.019	−0.008	0.009	−0.008	0.01	−0.006
		0.4	0.021	−0.012	0.02	−0.004	0.01	0.001	0.01	−0.008
		0.8	0.019	0.003	0.021	−0.008	0.01	0.007	0.009	−0.002
${{\lambda }_{2}}$	Bi-HM	0	0.006	0.001	0.005	0.007	0.003	0.005	0.002	−0.001
		0.4	0.006	−0.002	0.005	0.004	0.003	−0.006	0.003	0.005
		0.8	0.006	−0.001	0.005	0.003	0.003	−0.003	0.003	−0.005

被试参数	模型	${{\rho }_{b\beta }}$	N = 500				N = 1000
			m = 30		m = 60		m = 30		m = 60
			MSE	Bias	MSE	Bias	MSE	Bias	MSE	Bias
θ	HM	0	0.252	0.001	0.2	0.001	0.26	0.001	0.194	0.006
		0.4	0.283	0.009	0.206	0.002	0.259	0.006	0.194	−0.003
		0.8	0.263	0.004	0.202	0.011	0.253	−0.001	0.204	0
	Bi-HM	0	0.148	0.001	0.09	0.002	0.151	0.001	0.086	0.005
		0.4	0.157	0.009	0.087	0.002	0.151	0.006	0.087	−0.004
		0.8	0.152	0.004	0.088	0.011	0.151	0	0.084	0
τ	HM	0	0.158	−0.001	0.141	0.001	0.157	−0.011	0.135	0.008
		0.4	0.136	0.007	0.135	0.003	0.166	0	0.144	0.004
		0.8	0.156	0.004	0.122	0.007	0.122	−0.001	0.114	0.004
	Bi-HM	0	0.035	−0.001	0.02	0.002	0.035	−0.011	0.017	0.008
		0.4	0.038	0.007	0.019	0.003	0.034	0	0.017	0.003
		0.8	0.035	0.004	0.019	0.007	0.032	−0.001	0.017	0.004
η	Bi-HM	0	0.05	0.002	0.028	0.005	0.05	−0.002	0.025	−0.009
		0.4	0.057	−0.007	0.028	−0.001	0.051	0.006	0.024	0.003
		0.8	0.051	0.004	0.026	−0.008	0.051	−0.001	0.025	−0.002