心理实验数据的联合建模：反应与反应时的混合影响

doi:10.3724/SP.J.1041.2024.01619

摘要/Abstract

摘要：

混合效应模型(Mixed-Effects Model, MEM)将被试和刺激项目同时作为随机变量, 有效地分析实验效应和相关的被试(或刺激项目)差异, 从而避免了传统方差分析的随机效应固定化问题。基于此, 文中构建了混合MEM、独立MEM和速度MEM三个联合模型, 并与反应和反应时数据的分开建模(即分开MEM)进行比较。在IAT实验数据分析中, 分开MEM的模型拟合与参数估计均不如独立MEM, 而混合MEM的模型拟合优于独立MEM和速度MEM。模拟结果显示, 分开MEM参数估计的相对偏差普遍大于独立MEM, 且具有较高的第I类错误率; 而混合MEM比其他联合模型能更好地识别不同模拟情景的参数, 并且具有较佳的第I类错误率和统计检验力。因此, 在心理实验中, 联合建模方法比分开建模具有更大优势。

关键词: 心理实验, 反应时, 反应, 混合效应模型, 联合建模

Abstract:

Mixed-Effects Models (MEMs) have become a prominent trend in the analysis of psychological experiment data. MEMs can simultaneously treat both subjects and stimuli as random variables, effectively analyzing experimental effects and the associated differences between subjects (or stimuli). This approach avoids the issue of treating subjects or stimuli as fixed variables and consequently the high incidence of false positives, which is common in analysis of variance (ANOVA). Typically, in psychological experiments, reaction and reaction times are described and modeled separately. However, this separation hinders the full utilization and integration of different data from subjects to exploit maximum information from sample datasets.

Current psychometric and cognitive process models attempt to jointly analyze different data sources, providing insights for the joint modeling of psychological experiment data. In psychological experiments, a specific duration is typically set for each stimulus, and subjects are required to make a keystroke response within this period. This setup is similar to time-limited tests, but the tasks in psychological experiments are usually simpler. Based on this, the paper constructs three joint models: mixed MEM, independent MEM, and speed MEM. These models are compared with the separate model of reaction and reaction time data (i.e., separate MEM) in a series of studies.

In the analysis of IAT (Implicit Association Test) experiment data, the separate MEM was found to be inferior to the independent MEM in both data fitting and parameter estimation. The mixed MEM showed better model fit indices than both independent MEM and speed MEM. In the simulation studies, different comparisons were conducted using mixed MEM and speed MEM as benchmark models. The simulation results show that the relative bias in parameter estimation for separate MEM was generally greater than that for independent MEM, and it had a higher Type I error rate. Among the joint models, the independent MEM exhibited significant parameter estimate biases across benchmark models and also had high Type I error rates and statistical power. Similarly, the speed MEM was found to have comparable issues under mixed MEM simulation conditions. On the other hand, the mixed MEM was able to better identify parameters under different simulated scenarios compared to other joint models, and it had better Type I error rates and statistical power.

In conclusion, joint modeling is more advantageous than separate analysis in psychological experiments. Moreover, the reaction and reaction times in experimental tasks are more likely to have complex mixed influence relationships.

Key words: psychological experiment, reaction time, reaction, mixed-effects models, joint modeling

中图分类号:

B841

郭小军, 焦玉月, 柏小云, 罗照盛, 李弘. (2024). 心理实验数据的联合建模：反应与反应时的混合影响. 心理学报, 56(11), 1619-1633.

GUO Xiaojun, JIAO Yuyue, BAI Xiaoyun, LUO Zhaosheng, LI Hong. (2024). Joint modeling of psychological experimental data: Mixed effects of reaction and reaction time. Acta Psychologica Sinica, 56(11), 1619-1633.

图/表 13

参考文献 49

[1]	Ames, A. J., & Au, C. H. (2018). Using stan for item response theory models. Measurement: Interdisciplinary Research and Perspectives, 16(2), 129-134.
[2]	Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390-412.
[3]	Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255-278.
[4]	Bolsinova, M., & Tijmstra, J. (2018). Improving precision of ability estimation: Getting more from response times. British Journal of Mathematical and Statistical Psychology, 71(1), 13-38.
[5]	Brooks, S. P., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434-455.
[6]	Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57(3), 153-178. doi: 10.1016/j.cogpsych.2007.12.002 pmid: 18243170
[7]	Brown, V. A. (2021). An introduction to linear mixed-effects modeling in R. Advances in Methods and Practices in Psychological Science, 4(1), 1-19.
[8]	Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, 1(1), 1-20.
[9]	Bunji, K., & Okada, K. (2020). Joint modeling of the two-alternative multidimensional forced-choice personality measurement and its response time by a thurstonian D- diffusion item response model. Behavior Research Methods, 52(3), 1091-1107.
[10]	DeBruine, L. M., & Barr, D. J. (2021). Understanding mixed- effects models through data simulation. Advances in Methods and Practices in Psychological Science, 4(1), 1-15.
[11]	Dixon, P. (2008). Models of accuracy in repeated-measures designs. Journal of Memory and Language, 59(4), 447-456.
[12]	Gelman, A., Lee, D., & Guo, J. (2015). Stan: A probabilistic programming language for bayesian inference and optimization. Journal of Educational and Behavioral Statistics, 40(5), 530-543.
[13]	Guo, X., Bai, X., & Luo, Z. (2024). Modeling the dependence between response and response time: A bifactor model approach. Acta Psychologica Sinica, 56(3), 352-364. doi: 10.3724/SP.J.1041.2024.00352
	[郭小军, 柏小云, 罗照盛. (2024). 作答时间与反应依赖关系建模:基于双因子模型视角. 心理学报, 56(3), 352-362.] doi: 10.3724/SP.J.1041.2024.00352
[14]	Guo, X., Jiao, Y., Huang, Z., & Liu, T. (2022). Joint modeling of response accuracy and time in between-item multidimensional tests based on bi-factor model. Frontiers in Psychology, 13, 763959.
[15]	Hoffman, M. D., & Gelman, A. (2014). The no-U-turn sampler: Adaptively setting path lengths in hamiltonian monte carlo. Journal of Machine Learning Research, 15(1), 1593-1623.
[16]	Hoijtink, H., Mulder, J., van Lissa, C., & Gu, X. (2019). A tutorial on testing hypotheses using the bayes factor. Psychological Methods, 24(5), 539-556. doi: 10.1037/met0000201 pmid: 30742472
[17]	Hu, C. -P., Kong, X. -Z., Wagenmakers, E. -J., Ly, A., & Peng, K. (2018). The bayes factor and its implementation in JASP: A practical primer. Advances in Psychological Science, 26(6), 951-965.
	[胡传鹏, 孔祥祯, Eric-Jan, Wagenmakers, Alexander, Ly, 彭凯平. (2018). 贝叶斯因子及其在JASP中的实现. 心理科学进展, 26(6), 951-965.] doi: 10.3724/SP.J.1042.2018.00951
[18]	Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59(4), 434-446. doi: 10.1016/j.jml.2007.11.007 pmid: 19884961
[19]	Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54-69. doi: 10.1037/a0028347 pmid: 22612667
[20]	Judd, C. M., Westfall, J., & Kenny, D. A. (2017). Experiments with more than one random factor: Designs, analytic models, and statistical power. Annual Review of Psychology, 68(1), 601-625.
[21]	Klein Entink, R. H., van der Linden, W. J., & Fox, J. -P. (2009). A box-cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62(3), 621-640.
[22]	Kumle, L., Võ, M. L., & Draschkow, D. (2021). Estimating power in (generalized) linear mixed models: An open introduction and tutorial in R. Behavior Research Methods, 53(6), 2528-2543. doi: 10.3758/s13428-021-01546-0 pmid: 33954914
[23]	Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1-26.
[24]	Lee, W. Y. (2018). Generalized linear mixed effect models with crossed random effects for experimental designs having non-repeated items: Model specification and selection. Vanderbilt University, Nashville.
[25]	Lee, Y. H., & Ying, Z. (2015). A mixture cure-rate model for responses and response times in time-limit tests. Psychometrika, 80(3), 748-775. doi: 10.1007/s11336-014-9419-8 pmid: 25280442
[26]	Liang, K., Tu, D., & Cai, Y. (2023). Using process data to improve classification accuracy of cognitive diagnosis model. Multivariate Behavioral Research, 58(5), 969-987.
[27]	Liu, Y., Xu, L., Liu, H., Han, Y., You, X., & Wan, Z. (2024). Confidence interval width contours: Sample size planning for linear mixed-effects models. Acta Psychologica Sinica, 56(1), 124-138. doi: 10.3724/SP.J.1041.2024.00124
	[刘玥, 徐雷, 刘红云, 韩雨婷, 游晓锋, 万志林. (2024). 置信区间宽度等高线图在线性混合效应模型样本量规划中的应用. 心理学报, 56(1), 124-138.] doi: 10.3724/SP.J.1041.2024.00124
[28]	Loeys, T., Rosseel, Y., & Baten, K. (2011). A joint modeling approach for reaction time and accuracy in psycholinguistic experiments. Psychometrika, 76(3), 487-503.
[29]	Luo, Y., & Al-Harbi, K. (2017). Performances of LOO and WAIC as IRT model selection methods. Psychological Test and Assessment Modeling, 59(2), 183-205.
[30]	Ma, Z., Jia, J., & Wu, S. (2022). On the analysis of reaction time data through the use of mixed-effects models: Principles and practices. Foreign Language Learning Theory and Practice, 1, 35-46
	[马拯, 贾锦萱, 吴诗玉. (2022). 混合效应模型框架下反应时数据的分析: 原理和实践. 外语教学理论与实践, 1, 35-46.]
[31]	Man, K., Harring, J. R., Jiao, H., & Zhan, P. (2019). Joint modeling of compensatory multidimensional item responses and response times. Applied Psychological Measurement, 43(8), 639-654. doi: 10.1177/0146621618824853 pmid: 31551641
[32]	Martínez-Huertas, J. Á., Olmos, R., & Ferrer, E. (2022). Model selection and model averaging for mixed-effects models with crossed random effects for subjects and items. Multivariate Behavioral Research, 57(4), 603-619.
[33]	Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. (2017). Balancing type I error and power in linear mixed models. Journal of Memory and Language, 94, 305-315.
[34]	Meteyard, L., & Davies, R. A. I. (2020). Best practice guidance for linear mixed-effects models in psychological science. Journal of Memory and Language, 112, 1-22.
[35]	Miller, R., Scherbaum, S., Heck, D. W., Goschke, T., & Enge, S. (2018). On the relation between the (censored) shifted wald and the wiener distribution as measurement models for choice response times. Applied Psychological Measurement, 42(2), 116-135. doi: 10.1177/0146621617710465 pmid: 29881116
[36]	Park, J., Cardwell, R. L., & Yu, H. (2020). Specifying the random effect structure in linear mixed effect models for analyzing psycholinguistic data. Methodology, 16(2), 92-111.
[37]	Ranger, J. (2013). Modeling responses and response times in personality tests with rating scales. Psychological Test and Assessment Modeling, 55(4), 361-382.
[38]	Ranger, J., & Kuhn, J. -T. (2012). A flexible latent trait model for response times in tests. Psychometrika, 77(1), 31-47.
[39]	Ranger, J., Kuhn, J., & Gaviria, J. L. (2015). A race model for responses and response times in tests. Psychometrika, 80(2), 791-810.
[40]	Rietveld, T., & Van Hout, R. (2007). Analysis of variance for repeated measures designs with word materials as a nested random or fixed factor. Behavior Research Methods, 39(4), 735-747. pmid: 18183886
[41]	Rouder, J. N., Province, J. M., Morey, R. D., Gomez, P., & Heathcote, A. (2015). The lognormal race: A cognitive- process model of choice and latency with desirable psychometric properties. Psychometrika, 80(2), 491-513.
[42]	Sorensen, T., Hohenstein, S., & Vasishth, S. (2016). Bayesian linear mixed models using Stan: A tutorial for psychologists, linguists, and cognitive scientists. The Quantitative Methods for Psychology, 12(3), 175-200.
[43]	Suh, H. (2010). A study of bayesian estimation and comparison of response time models in item response theory. Unpublished doctoral dissertation. University of Kansas, Lawrence, KS.
[44]	van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287-308.
[45]	van der Maas, H. L. J., Molenaar, D., Maris, G., Kievit, R. A., & Borsboom, D. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118(2), 339-356. doi: 10.1037/a0022749 pmid: 21401290
[46]	Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27, 1413-1432.
[47]	Visalli, A., Montefinese, M., Viviani, G., Finos, L., Vallesi, A., & Ambrosini, E. (2024). lmeEEG: Mass linear mixed- effects modeling of EEG data with crossed random effects. Journal of Neuroscience Methods, 401, 109991.
[48]	Wen, Z., Tang, D., & Gu, H. . (2019). A general simulation comparison of the predictive validity between bifactor and high-order factor models. Acta Psychologica Sinica, 51(3), 383-391. doi: 10.3724/SP.J.1041.2019.00383
	[温忠麟, 汤丹丹, 顾红磊. (2019). 预测视角下双因子模型与高阶因子模型的一般性模拟比较. 心理学报, 51(3), 383-391.] doi: 10.3724/SP.J.1041.2019.00383
[49]	Zhan, P. (2022). Joint-cross-loading multimodal cognitive diagnostic modeling incorporating visual fixation counts. Acta Psychologica Sinica, 54(11), 1416-1423. doi: 10.3724/SP.J.1041.2022.01416
	[詹沛达. (2022). 引入眼动注视点的联合-交叉负载多模态认知诊断建模. 心理学报, 54(11), 1416-1423.] doi: 10.3724/SP.J.1041.2022.01416

反应时数据			反应数据
模型	AIC	BIC	模型	AIC	BIC
LMEM-S01I01	3456.1	3511.8	GLMEM-S01I01	1206.9	1256.5
LMEM-S0I01	3668.0	3711.3	GLMEM-S0I01	1245.9	1283.1
LMEM-S01I0	3452.3	3495.6	GLMEM-S01I0	1202.9	1240.1
LMEM-S0I0	3664.0	3695.0	GLMEM-S0I0	1242.5	1267.2

反应时数据			反应数据
模型	AIC	BIC	模型	AIC	BIC
LMEM-S01I01	3456.1	3511.8	GLMEM-S01I01	1206.9	1256.5
LMEM-S0I01	3668.0	3711.3	GLMEM-S0I01	1245.9	1283.1
LMEM-S01I0	3452.3	3495.6	GLMEM-S01I0	1202.9	1240.1
LMEM-S0I0	3664.0	3695.0	GLMEM-S0I0	1242.5	1267.2

模型	模型拟合指数
模型	WAIC	LOO
混合MEM	50684.6	50684.9
速度MEM	50920.3	50921.3
独立MEM	54696.6	54697.9
分开MEM	54697.2	54698.5

模型	模型拟合指数
模型	WAIC	LOO
混合MEM	50684.6	50684.9
速度MEM	50920.3	50921.3
独立MEM	54696.6	54697.9
分开MEM	54697.2	54698.5

数据	参数		混合MEM	速度MEM	独立MEM	分开MEM
反应时	固定效应	${{\beta }_{0}}$	656(24.59)	660(25.86)	643(24.54)	643(24.54)
	固定效应	${{\beta }_{1}}$	150(33.24)	159(36.83)	139(31.52)	140(32.52)
	随机效应	${{\sigma }_{\tau 0}}$	163(16.52)	173(17.25)	165(16.73)	164(16.76)
		${{\sigma }_{\tau 1}}$	207(22.35)	237(25.22)	199(21.86)	202(22.80)
		${{\sigma }_{\gamma 0}}$	13(9.07)	18(10.49)	16(10.35)	16(9.93)
		${{\sigma }_{e}}$	375(4.62)	384(4.78)	403(4.79)	403(4.79)
反应	固定效应	${{b}_{0}}$	5.67(0.33)	——	3.58(0.17)	3.59(0.18)
	固定效应	${{b}_{1}}$	−1.56(0.54)	——	−0.87(0.29)	−0.82(0.30)
	随机效应	${{\sigma }_{\theta 0}}$	0.87(0.15)	——	0.80(0.15)	0.84(0.15)
		${{\sigma }_{\theta 1}}$	1.22(0.14)	——	1.23(0.24)	1.28(0.25)
		${{\sigma }_{\lambda 0}}$	0.28(0.21)	——	0.22(0.14)	0.23(0.13)