Equivalence testing: A new perspective on structural equation model evaluation and measurement invariance analysis

doi:10.3724/SP.J.1042.2020.01961

Abstract

Abstract:

There are some limitations in the commonly used fit indices of structural equation model. For example, χ ², which is set up to reject the traditional null hypothesis, cannot be used to endorse a model, while descriptive fit indices such as RMSEA and CFI do not have inferential statistical properties. Equivalence testing can effectively compensate for the limitations mentioned above. In this paper, the way to use equivalence testing in evaluating the fit of a single model and its difference from the null hypothesis testing were introduced first. Then the approach to analyze measurement invariance by equivalence testing was described. Furthermore, empirical data was used to demonstrate the effectiveness of equivalence testing in single model evaluation and measurement invariance test, and to compare equivalence testing with traditional model evaluation method.

Key words: structural equation model, fit indices, equivalence testing, null hypothesis testing, measurement invariance

CLC Number:

B841

WANG Yang, WEN Zhonglin, FU Yuanshu. Equivalence testing: A new perspective on structural equation model evaluation and measurement invariance analysis[J]. Advances in Psychological Science, 2020, 28(11): 1961-1969.

Figures/Tables 1

References 34

[1]	王孟成 . (2014). 潜变量建模与Mplus应用(基础篇). 重庆大学出版社.
[2]	王阳, 王才康, 温忠麟, 肖婉婷 . ( 2017). 共情和同情量表在中国幼儿教师样本中的效度和信度. 中国临床心理学杂志, 25( 6), 1027-1030.
[3]	温涵, 梁韵斯 . ( 2015). 结构方程模型常用拟合指数检验的实质. 心理科学, 38( 4), 987-994.
[4]	温忠麟, 侯杰泰 . ( 2008). 检验的临界值: 真伪差距多大才能辨别?——评《不同条件下拟合指数的表现及临界值的选择》. 心理学报. 40( 1), 119-124.
[5]	温忠麟, 黄彬彬, 汤丹丹 . ( 2018). 问卷数据建模前传. 心理科学, 41( 1), 204-210.
[6]	温忠麟, 刘红云, 侯杰泰 . (2012). 调节效应和中介效应分析. 教育科学出版社.
[7]	吴明隆 . (2010). 结构方程模型——AMOS的操作与应用(第2版). 重庆大学出版社.
[8]	颜志强, 苏彦捷 . ( 2018). 共情的性别差异: 来自元分析的证据. 心理发展与教育, 34( 2), 129-136.
[9]	Alpizar, D., French, B. F., & Vo, T. T . ( 2020). Equivalence testing of a youth risk and needs assessment. Journal of Psychoeducational Assessment. Advance online publication. doi: 10.1177/0734282907303873 URL pmid: 26346633
[10]	Counsell, A., Cribbie, R. A., & Flora, D. B . ( 2020). Evaluating equivalence testing methods for measurement invariance. Multivariate Behavioral Research, 55( 2), 312-328. doi: 10.1080/00273171.2019.1633617 URL pmid: 31389729
[11]	Deng, L. F., & Yuan, K.-H. ( 2016). Comparing latent means without mean structure models: A projection-based approach. Psychometrika, 81( 3), 802-829. doi: 10.1007/s11336-015-9491-8 URL pmid: 26661857
[12]	Finch, W. H., & French, B. F . ( 2018). A simulation investigation of the performance of invariance assessment using equivalence testing procedures. Structural Equation Modeling: A Multidisciplinary Journal, 25( 5), 673-686.
[13]	Fu, Y. S., Wen, Z. L., & Wang, Y . ( 2018). The total score with maximal reliability and maximal criterion validity: An illustration using a career satisfaction measure. Educational and Psychological Measurement, 78( 6), 1108-1122. doi: 10.1177/0013164417738564 URL pmid: 30559516
[14]	Hancock, G. R., & Mueller, R. O . ( 2011). The reliability paradox in assessing structural relations within covariance structure models. Educational and Psychological Measurement, 71( 2), 306-324.
[15]	Jiang, G., Mai, Y. J., & Yuan, K.-H . ( 2017). Advances in measurement invariance and mean comparison of latent variables: Equivalence testing and a projection-based approach. Frontiers in Psychology, 8, Article 1823. URL pmid: 29416519
[16]	Jin, Y. ( 2020). A note on the cutoff values of alternative fit indices to evaluate measurement invariance for ESEM models. International Journal of Behavioral Development, 44( 2), 166-174.
[17]	Kelloway, E. K . ( 2015). Using Mplus for structural equation modeling: A researcher’s guide (2nd ed.). SAGE Publications.
[18]	Kim, Y. J., & Cribbie, R. A . ( 2018). ANOVA and the variance homogeneity assumption: Exploring a better gatekeeper. British Journal of Mathematical and Statistical Psychology, 71( 1), 1-12.
[19]	Lai, K. ( 2020). Confidence interval for RMSEA or CFI difference between nonnested models. Structural Equation Modeling: A Multidisciplinary Journal, 27( 1), 16-32. doi: 10.1080/10705511.2019.1631704 URL
[20]	Lakens, D. ( 2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8( 4), 355-362. doi: 10.1177/1948550617697177 URL pmid: 28736600
[21]	Marcoulides, K. M., & Yuan, K.-H. ( 2017). New ways to evaluate goodness of fit: A note on using equivalence testing to assess structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 24( 1), 148-153. doi: 10.1080/10705511.2016.1225260 URL
[22]	McNeish, D., An, J., & Hancock, G. R . ( 2018). The thorny relation between measurement quality and fit index cutoffs in latent variable models. Journal of Personality Assessment, 100( 1), 43-52. URL pmid: 28631976
[23]	Moshagen, M., & Auerswald, M. ( 2018). On congruence and incongruence of measures of fit in structural equation modeling. Psychological Methods, 23( 2), 318-336. URL pmid: 28301200
[24]	Shi, D., Lee, T., & Maydeu-Olivares, A . ( 2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79( 2), 310-334. doi: 10.1177/0013164418783530 URL pmid: 30911195
[25]	Shiskina, T., Farmus, L., & Cribbie, R. A . ( 2018). Testing for a lack of relationship among categorical variables. The Quantitative Methods for Psychology, 14( 3), 167-179.
[26]	Svetina, D., Rutkowski, L., & Rutkowski, D . ( 2020). Multiple-group invariance with categorical outcomes using updated guidelines: An illustration using Mplus and the lavaan/semTools packages. Structural Equation Modeling: A Multidisciplinary Journal, 27( 1), 111-130.
[27]	Swami, V., & Barron, D. ( 2019). Translation and validation of body image instruments: Challenges, good practice guidelines, and reporting recommendations for test adaptation. Body Image, 31, 204-220. doi: 10.1016/j.bodyim.2018.08.014 URL pmid: 30220631
[28]	Taasoobshirazi, G., & Wang, S. S . ( 2016). The performance of the SRMR, RMSEA, CFI, and TLI: An examination of sample size, path size, and degrees of freedom. Journal of Applied Quantitative Methods, 11( 3), 31-40.
[29]	Tóth-Király, I., Morin, A. J. S., Bőthe, B., Orosz, G., & Rigó, A . ( 2018). Investigating the multidimensionality of need fulfillment: A bifactor exploratory structural equation modeling representation. Structural Equation Modeling: A Multidisciplinary Journal, 25( 2), 267-286.
[30]	Wang, Y., Li, Y., Xiao, W. T., Fu, Y. S., & Jie, J . ( 2020). Investigation on the rationality of the extant ways of scoring the Interpersonal Reactivity Index based on confirmatory factor analysis. Frontiers in Psychology, 11, Article 1086. doi: 10.3389/fpsyg.2020.568723 URL pmid: 33132976
[31]	Wang, Y., Su, Q., & Wen, Z. L . ( 2019). Exploring latent profiles of empathy among Chinese preschool teachers: A person-centered approach. Journal of Psychoeducational Assessment, 37( 6), 706-717.
[32]	Wang, Y., Wen, Z. L., Fu, Y. S., & Zheng, L. L . ( 2017). Psychometric properties of a Chinese version of the Measure of Empathy and Sympathy. Personality and Individual Differences, 119, 168-174.
[33]	Yuan, K.-H., & Chan, W. ( 2016). Measurement invariance via multigroup SEM: Issues and solutions with chi-square-difference tests. Psychological Methods, 21( 3), 405-426. doi: 10.1037/met0000080 URL pmid: 27266799
[34]	Yuan, K.-H., Chan, W., Marcoulides, G. A., & Bentler, P. M . ( 2016). Assessing structural equation models by equivalence testing with adjusted fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 23( 3), 319-330.

模型	ε_t	RMSEA_t	RMSEA	CFI	ΔRMSEA	ΔCFI	Δχ²	Δdf	p (Δχ²)
形态不变性(男)	0.044	0.068	0.042	0.991
形态不变性(女)	0.051	0.073	0.055	0.981
形态不变性			0.048	0.987
单位不变性	0.017	0.074	0.046	0.986	0.002	0.001	8.352	6	0.213
截距不变性	0.006	0.043	0.042	0.987	0.004	0.001	3.439	6	0.752
潜均值不变性	0.017	0.130	0.043	0.986	0.001	0.001	5.373	2	0.068

模型	ε_t	RMSEA_t	RMSEA	CFI	ΔRMSEA	ΔCFI	Δχ²	Δdf	p (Δχ²)
形态不变性(男)	0.044	0.068	0.042	0.991
形态不变性(女)	0.051	0.073	0.055	0.981
形态不变性			0.048	0.987
单位不变性	0.017	0.074	0.046	0.986	0.002	0.001	8.352	6	0.213
截距不变性	0.006	0.043	0.042	0.987	0.004	0.001	3.439	6	0.752
潜均值不变性	0.017	0.130	0.043	0.986	0.001	0.001	5.373	2	0.068