ISSN 1671-3710
CN 11-4766/R

心理科学进展 ›› 2020, Vol. 28 ›› Issue (8): 1392-1408.doi: 10.3724/SP.J.1042.2020.01392

• 研究方法 • 上一篇    



  1. 陆军炮兵防空兵学院军政基础系, 合肥 230031
  • 收稿日期:2019-12-06 出版日期:2020-08-15 发布日期:2020-06-28
  • 通讯作者: 朱海腾

Data aggregation adequacy testing in multilevel research: A critical literature review and preliminary solutions to key issues

ZHU Haiteng()   

  1. Department of Military and Ideological Basic Education, PLA Army Academy of Artillery and Air Defense, Hefei 230031, China
  • Received:2019-12-06 Online:2020-08-15 Published:2020-06-28
  • Contact: ZHU Haiteng


组织管理领域的多层次研究经常需要测量共享单位特性构念, 常用方法是将单位内若干个体成员的评分聚合到单位层次, 确保聚合后的分数具有充分代表性的统计前提是通过聚合适当性检验。聚合适当性检验的常用指标是组内一致性rWG和组内信度ICC(1)、ICC(2), 但目前学界对于这两类指标何者更优、rWG的原分布选择和数据清理、各指标的划界值等关键问题存在诸多争议。为此, 首先对国内9份管理学、心理学期刊2014年以来发表的166篇包含聚合适当性检验的论文进行内容分析, 并以Journal of Applied Psychology上的85篇论文为对比, 查明常规实践中的共性问题, 进而提出实践建议:(1)明确功能定位, 将rWG作为聚合适当性指标, ICC(1)和ICC(2)分别作为效度、信度指标。(2)计算rWG时审慎选择原分布, 排除组内一致性过低的组。(3)为各指标设定更加合理、有适度灵活性的划界值, 停止使用武断、粗糙的经验标准。最后, 强调研究者在模型构建和聚合决策中应加强理论考量, 避免片面依赖统计检验结果。

关键词: 多层次研究, 共享单位特性, 聚合, 组内一致性, 组内信度


The measurement of shared unit property constructs is ubiquitous in multilevel organizational research, of which the most frequently used approach is to aggregate the ratings of several unit members to the unit level. The data aggregation adequacy testing (DAAT) is a statistical hurdle to ensure the validity and representativeness of aggregated scores. Well-established indicators of DAAT include within-group agreement index, rWG, and within-group reliability indices, ICC(1) and ICC(2); nonetheless, some key issues are still open to debate, for instance, the superiority of the two families of indicators, the null distribution and data screening decision of rWG, and appropriate cut-off values. To address the above questions, the current research firstly conducted a content analysis of 166 studies adopting DAAT procedure published on 9 Chinese journals in the field of management and psychology since 2014, coupled with 85 studies from Journal of Applied Psychology as a comparison. Common problems in routine practice of DAAT were identified and related suggestions were proposed as follows: (1) Disentangling and differentiating the role of DAAT indicators; specifically, rWG should be used as the exclusive indicator of aggregation adequacy, whereas ICC(1) and ICC(2) should be deemed as indices of validity and reliability, respectively. (2) Making prudent and justifiable decisions in choosing null distributions when calculating rWG index, and excluding groups with low within-group agreement. (3) Applying more reasonable and moderately flexible cut-off values instead of arbitrary and rough practical standards. Last but not the least, researchers should always prioritize theoretical considerations in the process of framework building and DAAT, and unload disproportionate dependence on statistical results.

Key words: multilevel research, shared unit property, aggregation, within-group agreement, within-group reliability