认知诊断评估中主效应DIF与交互式DIF检测方法开发: 基于递归分割视角

doi:10.3724/SP.J.1041.2026.0995

心理学报 ›› 2026, Vol. 58 ›› Issue (5): 995-1014.doi: 10.3724/SP.J.1041.2026.0995 cstr: 32110.14.2026.0995

• 研究报告 • 上一篇

认知诊断评估中主效应DIF与交互式DIF检测方法开发: 基于递归分割视角

刘凯^1,2, 郭治辰¹, 王琴¹, 汪大勋¹, 蔡艳¹, 涂冬波^1,3,4

¹江西师范大学心理学院, 南昌 330022;
²辽宁师范大学心理学院, 大连 116029;
³江西省哲学社会科学实验室—江西师范大学数据科学与智能化心理测评及服务实验室;
⁴智能信息处理与情感计算江西省重点实验室, 南昌 330022

收稿日期:2024-04-26 发布日期:2026-03-04 出版日期:2026-05-25
通讯作者: 蔡艳, E-mail: cy1979123@aliyun.com; 汪大勋, E-mail: wangda.xun@163.com; 涂冬波, E-mail: tudongbo@aliyun.com
作者简介:郭治辰和王琴为本文共同第一作者。
基金资助:
国家自然科学基金项目(32300942, 62467002, 62167004, 32160203)、江西省科技创新基地计划—智能信息处理与情感计算江西省重点实验室(20242BCC32021)资助。

Development of main effect DIF and interactive DIF detection method in cognitive diagnosis assessments: A recursive partitioning-based perspective

LIU Kai^1,2, GUO Zhichen¹, WANG Qin¹, WANG Daxun¹, CAI Yan¹, TU Dongbo^1,3,4

¹School of Psychology, Jiangxi Normal University, Nanchang 330022, China;
²College of Psychology, Liaoning Normal University, Dalian 116029, China;
³Jiangxi Laboratory of Philosophy and Social Sciences—Data Science and Intelligent Psychological Assessment and Service Laboratory of Jiangxi Normal University, Nanchang 330022, China;
⁴Jiangxi Provincial Key Laboratory of Intelligent Information Processing and Affective Computing, Nanchang 330022, China

Received:2024-04-26 Online:2026-03-04 Published:2026-05-25

摘要/Abstract

摘要： 在认知诊断评估中, 项目功能差异(DIF)检测是评估其测验公平性以及测量效度的重要技术手段。然而, 现有的认知诊断DIF检测方法局限于单一协变量视角下的主效应DIF检测, 对于由多个协变量交互作用引发的交互式DIF尚缺乏有效的检测手段。针对这一局限, 研究借鉴递归分割技术的核心思想, 提出了一种能够在认知诊断评估中同时检测主效应DIF和交互式DIF的新方法(记为ISRPM)。模拟研究结果表明, ISRPM不仅在主效应DIF检测中的整体表现与传统方法大体相当, 更重要的是, 其在交互式DIF检测方面的性能表现优于传统方法。实证研究则进一步支持了该方法的可用性, 结果显示, ISRPM与传统DIF检测方法在检测结果上具有较高一致性, 并在识别交互式DIF方面展现出潜在优势。总体而言, ISRPM的提出有望进一步提升认知诊断DIF检测精度, 并促进认知诊断评估在心理与教育测评实践中的推广与应用。

关键词: 认知诊断评估, 项目功能差异, 主效应DIF, 交互式DIF, 递归分割技术

Abstract: With the growing recognition of the advantages of cognitive diagnosis (CD) in psychological and educational measurement, applying the CD framework to test development has become an important research direction in the field of psychology. In the development of cognitive diagnostic assessments, detecting differential item functioning (DIF) remains a crucial quality control procedure to ensure test fairness and validity. However, existing CD-based DIF detection methods typically focus on a single covariate at a time. While these approaches are effective for identifying main effect DIF induced by a single covariate, they are limited in detecting interactive DIF caused by the interaction among multiple covariates. Such limitations may compromise the fairness and interpretability of assessment outcomes. To address this issue, the present study integrates CD modeling with recursive partitioning techniques by proposing a novel DIF detection method, namely the Item-based Sequential Recursive Partitioning Method (ISRPM). Building on the core principles of recursive partitioning, the ISRPM allows the simultaneous consideration of multiple covariates within a single DIF detection procedure and facilitates the identification of both main effect DIF and interactive DIF in cognitive diagnostic assessments.
To evaluate the performance of the proposed method, a series of Monte Carlo simulation studies were conducted focusing on two key objectives: (1) examining how factors such as sample size per group, DIF magnitude, DIF type, item quality, correlations among attributes, and the influence of demographic covariates on attribute mastery distribution affect the performance of ISRPM; and (2) comparing ISRPM with several existing DIF detection methods across varied experimental conditions. In addition, to illustrate its practical utility, ISRPM was applied to a cognitive diagnostic version of the Schizotypal Personality Questionnaire (DC-SPQ) and compared with five established DIF detection methods.
The results showed that (1) sample size, DIF magnitude, and item quality substantially influenced the performance of all methods; and (2) when items exhibited interactive DIF, ISRPM achieved higher detection accuracy than the Wald, LR, FS-Wald, FS-LR, and Mantel-Haenszel (MH) approaches. When only the main effect DIF was present, the overall performance of ISRPM was comparable to that of the existing methods.
These findings suggest that ISRPM provides a flexible and effective framework for identifying both main effect DIF and interactive DIF in cognitive diagnostic assessments, thereby contributing to methodological advancements in fairness evaluation and the broader application of CD-based measurement in psychological and educational measurement.

Key words: cognitive diagnosis assessments, differential item functioning, main effect DIF, interactive DIF, recursive partitioning

中图分类号:

B841

刘凯, 郭治辰, 王琴, 汪大勋, 蔡艳, 涂冬波. (2026). 认知诊断评估中主效应DIF与交互式DIF检测方法开发: 基于递归分割视角. 心理学报, 58(5), 995-1014.

LIU Kai, GUO Zhichen, WANG Qin, WANG Daxun, CAI Yan, TU Dongbo. (2026). Development of main effect DIF and interactive DIF detection method in cognitive diagnosis assessments: A recursive partitioning-based perspective. Acta Psychologica Sinica, 58(5), 995-1014.

参考文献

[1] American Education Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). The standards for educational and psychological testing. Washington, DC: AERA Publications.
[2] Barnett, V., & Lewis, T. (1994). Outliers in statistical data. Hoboken: Wiley.
[3] Bauer, D. J. (2017). A more general model for testing measurement invariance and differential item functioning.Psychological Methods, 22(3), 507-526.
[4] Belzak, W. C. (2023). The multidimensionality of measurement bias in high-stakes testing: Using machine learning to evaluate complex sources of differential item functioning.Educational Measurement: Issues and Practice, 42(1), 24-33.
[5] Belzak, W. C. M., & Bauer, D. J. (2020). Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning.Psychological Methods, 25(6), 673-690.
[6] Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing.Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300.
[7] Bollmann S., Berger M., & Tutz G. (2018). Item-focused trees for the detection of differential item functioning in partial credit models.Educational and Psychological Measurement, 78(5), 781-804.
[8] Collins, P. H. (1990). Black feminist thought: Knowledge, consciousness, and the politics of empowerment. UnwinHyman.
[9] de la Torre, J. (2011). The generalized DINA model framework.Psychometrika, 76(2), 179-199.
[10] de la Torre J., van der Ark, L. A., & Rossi G. (2018). Analysis of clinical data from a cognitive diagnosis modeling framework.Measurement and Evaluation in Counseling and Development, 51(4), 281-296.
[11] DiBello L. V., Roussos L. A., & Stout W. (2006). 31a review of cognitively diagnostic assessment and a summary of psychometric models.Handbook of Statistics, 26, 979-1030.
[12] Finch W. H., Hernández Finch M. E., & French B. F. (2015). Recursive partitioning to identify potential causes of differential item functioning in cross-national data. International Journal of Testing, 16(1), 21-53.
[13] Holland, P. W., & Wainer, H. (1993). Differential item functioning. Hillsdale, NJ: Erlbaum.
[14] Hothorn T., Hornik K., & Zeileis A. (2006). Unbiased recursive partitioning: A conditional inference framework.Journal of Computational and Graphical Statistics, 15(3), 651-674.
[15] Hou, L. (2013). Differential item functioning assessment in cognitive diagnostic modeling: Applying the Wald test to investigate DIF in the generalized DINA model framework (Unpublished doctoral dissertation). University of Delaware.
[16] Hou L., de la Torre J., & Nandakumar R. (2014). Differential item functioning assessment in cognitive diagnostic modeling: Application of the Wald test to investigate DIF in the DINA model.Journal of Educational Measurement, 51(1), 98-125.
[17] Komboz B., Strobl C., & Zeileis A. (2016). Tree-based global model tests for polytomous Rasch models.Educational and Psychological Measurement, 78(1), 128-166.
[18] Leighton J. P.,& Gierl, M. (2007). Cognitive diagnostic assessment for education: Theory and Applications Cambridge, UK: Cambridge University Press Theory and Applications. Cambridge, UK: Cambridge University Press.
[19] Li, F. (2008). A modified higher-order DINA model for detecting differential item functioning and differential attribute functioning (Unpublished doctoral dissertation). University of Georgia.
[20] Li L., Zhou X., Huang J., Tu D., Gao X., Yang Z., & Li M. (2020). Assessing kindergarteners’ mathematics problem solving: The development of a cognitive diagnostic test.Studies in Educational Evaluation, 66, 100879.
[21] Li, X., & Wang, W.-C. (2015). Assessment of differential item functioning under cognitive diagnosis models: The DINA model example. Journal of Educational Measurement, 52(1), 28-54.
[22] Liu Y., Xin T., Li L., Tian W., & Liu X. (2016). An improved method for differential item functioning detection in cognitive diagnosis models: An application of Wald statistic based on observed information matrix.Acta Psychologica Sinica, 48(5), 588-598.
[刘彦楼, 辛涛, 李令青, 田伟, 刘笑笑. (2016). 改进的认知诊断模型项目功能差异检验方法——基于观察信息矩阵的Wald统计量.心理学报, 48(5), 588-598.]
[23] Ma, W., & de la Torre, J. (2020). GDINA: An R package for cognitive diagnosis modeling.Journal of Statistical Software, 93(14), 1-26.
[24] Ma W., Terzi R., & de la Torre, J. (2021). Detecting differential item functioning using multiple-group cognitive diagnosis models.Applied Psychological Measurement, 45(1), 37-53.
[25] Magis D., Béland S., Tuerlinckx F., & de Boeck P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning.Behavior Research Methods, 42(3), 847-862.
[26] Meade, A. W., & Wright, N. A. (2012). Solving the measurement invariance anchor item problem in item response theory.Journal of Applied Psychology, 97(5), 1016-1031.
[27] Mehrazmay R., Ghonsooly B., & de la Torre, J. (2021). Detecting differential item functioning using cognitive diagnosis models: Applications of the Wald test and likelihood ratio test in a university entrance examination.Applied Measurement in Education, 34(4), 262-284.
[28] Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance.Psychometrika, 58(4), 525-543.
[29] Nichols P. D., Chipman S. F., & Brennan R. L. (1995). Cognitively diagnostic assessment. Routledge.
[30] Paulsen J., Svetina D., Feng Y., & Valdivia M. (2020). Examining the impact of differential item functioning on classification accuracy in cognitive diagnostic models.Applied Psychological Measurement, 44(4), 267-281.
[31] R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.Rproject.org/
[32] Rupp A. A., Templin J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications Guilford Press Theory, methods, and applications. Guilford Press.
[33] Strobl C., Kopf J., & Zeileis A. (2015). Rasch trees: A new method for detecting differential item functioning in the Rasch model.Psychometrika, 80(2), 289-316.
[34] Strobl C., Malley J., & Tutz G. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests.Psychological Methods, 14(4), 323-348.
[35] Sun X., Liu Y., Wang S., Xin T., Song N., & Zhou M. (2022). Using information matrix-based method to detect differential item functioning with multiple groups in cognitive diagnostic test.Journal of Psychological Science, 45(3), 710-717.
[孙小坚, 刘彦楼, 王诗梦, 辛涛, 宋乃庆, 周蔓. (2022). 认知诊断测验中基于信息矩阵的多群组DIF检验.心理科学, 45(3), 710-717.]
[36] Tan Z., de La Torre J., Ma W., Huh D., Larimer M. E., & Mun E.-Y. (2023). A tutorial on cognitive diagnosis modeling for characterizing mental health symptom profiles using existing item responses.Prevention Science, 24(3), 480-492.
[37] Tay L., Huang Q., & Vermunt J. K. (2015). Item response theory with covariates (IRT-C): Assessing item recovery and differential item functioning for the three-parameter logistic model.Educational and Psychological Measurement, 76(1), 22-42.
[38] Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models.Psychological Methods, 11(3), 287-305.
[39] Tu D., Cai Y., Gao X., & Wang, D. (2019). Advanced cognitive diagnosis. Beijing: Beijing Normal University Publishing Group.
[涂冬波, 蔡艳, 高旭亮, 汪大勋. (2019). 高级认知诊断. 北京: 北京师范大学出版社.]
[40] Tutz, G., & Berger, M. (2016). Item-focussed trees for the identification of items in differential item functioning.Psychometrika, 81(3), 727-750.
[41] Wang D., Gao X., Cai Y., & Tu D. (2019). Development of a new instrument for depression with cognitive diagnosis models. Frontiers in Psychology, 10, 1306.
[42] Wang, X. (2019). Development and verification of cognitive diagnostic test for cross-grade pupils’ mathematics learning ability,Chinese Exam, 8, 71-78.
[王欣瑜. (2019). 跨年级小学数学学力认知诊断测验的开发与验证.中国考试, 8, 71-78.]
[43] Wang Z., Guo L., & Bian Y. (2014). Comparison of DIF detecting methods in cognitive diagnostic test.Acta Psychologica Sinica, 46(12), 1923-1932.
[王卓然, 郭磊, 边玉芳. (2014). 认知诊断测验中的项目功能差异检测方法比较.心理学报, 46(12), 1923-1932.]
[44] Xi C., Cai Y., Peng S., Lian J., & Tu D. (2020). A diagnostic classification version of schizotypal personality questionnaire using diagnostic classification models.International Journal of Methods in Psychiatric Research, 29(1), e1807.
[45] Yuan K. H., Liu H., & Han Y. (2021). Differential item functioning analysis without a priori information on anchor items: QQ plots and graphical test.Psychometrika, 86(2), 345-377.
[46] Zhang, W. (2006). Detecting differential item functioning using the DINA model (Unpublished doctoral dissertation). The University of North Carolina at Greensboro.

认知诊断评估中主效应DIF与交互式DIF检测方法开发: 基于递归分割视角

Development of main effect DIF and interactive DIF detection method in cognitive diagnosis assessments: A recursive partitioning-based perspective

PDF (PC)

评审附件

可视化

English Version

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 9

编辑推荐

Metrics

本文评价

[1]	郭磊, 周文杰. 基于选项层面的认知诊断非参数方法[J]. 心理学报, 2021, 53(9): 1032-1043.
[2]	刘彦楼;辛涛;李令青;田伟;刘笑笑. 改进的认知诊断模型项目功能差异检验方法 ——基于观察信息矩阵的Wald统计量[J]. 心理学报, 2016, 48(5): 588-598.
[3]	汪文义;丁树良;宋丽红. 认知诊断中基于条件期望的距离判别方法[J]. 心理学报, 2015, 47(12): 1499-1510.
[4]	王卓然; 郭磊; 边玉芳. 认知诊断测验中的项目功能差异检测方法比较[J]. 心理学报, 2014, 46(12): 1923-1932.
[5]	张勋;李凌艳;刘红云;孙研. IRT_Δb法和修正LR法对矩阵取样 DIF检验的有效性[J]. 心理学报, 2013, 45(8): 921-934.
[6]	刘红云,李冲,张平平,骆方. 分类数据测量等价性检验方法及其比较：项目阈值(难度)参数的组间差异性检验[J]. 心理学报, 2012, 44(8): 1124-1136.
[7]	郑蝉金,郭聪颖,边玉芳. 变通的题组项目功能差异检验方法在篇章阅读测验中的应用[J]. 心理学报, 2011, 43(07): 830-835.
[8]	曹亦薇. 项目功能差异在跨文化人格问卷分析中的应用[J]. 心理学报, 2003, 35(01): 120-126.
[9]	曹亦薇,张厚粲. 汉语词汇测验中的项目功能差异初探[J]. 心理学报, 1999, 31(4): 460-467.