ISSN 1671-3710
CN 11-4766/R
主办:中国科学院心理研究所
出版:科学出版社

心理科学进展 ›› 2019, Vol. 27 ›› Issue (1): 171-180.doi: 10.3724/SP.J.1042.2019.00171

• 研究方法 • 上一篇    下一篇

创造力测评中的评分者效应

韩建涛1,2,3, 刘文令1, 庞维国1()   

  1. 1 华东师范大学心理与认知科学学院, 上海 200062
    2 安徽师范大学教育科学学院, 芜湖241000
    3 巢湖学院文学传媒与教育科学学院, 巢湖 238000
  • 收稿日期:2017-07-28 出版日期:2019-01-15 发布日期:2018-11-23
  • 基金资助:
    * 教育部人文社会科学研究青年基金项目(16YJC190008)

Rater effects in creativity assessment

HAN Jiantao1,2,3, LIU Wenling1, PANG Weiguo1()   

  1. 1 School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
    2 School of Educational Science, Anhui Normal University, Wuhu 241000, China
    3 School of Literature Media and Educational Science, Chaohu College, Chaohu 238000, China
  • Received:2017-07-28 Online:2019-01-15 Published:2018-11-23

摘要:

创造力测评中的评分者效应(rater effects)是指在创造性测评过程中, 由于评分者参与而对测评结果造成的影响.评分者效应本质上源于评分者内在认知加工的不同, 具体体现在其评分结果的差异.本文首先概述了评分者认知的相关研究, 以及评分者,创作者,社会文化因素对测评的影响.其次在评分结果层面梳理了评分者一致性信度的指标及其局限, 以及测验概化理论和多面Rasch模型在量化,控制该效应中的应用.最后基于当前研究仍存在的问题, 指出了未来可能的研究方向, 包括深化评分者认知研究,整合不同层面评分者效应的研究, 以及拓展创造力测评方法和技术等.

关键词: 评分者效应, 创造力, 主观评分, 评分者认知, 评分者一致性

Abstract:

Rater effects refer to the impact of different raters’ idiosyncrasies in their behaviors on the evaluation results in creativity assessment. Rater effects are due to the difference in raters’ cognitive process of the evaluation, which are externally reflected in the difference of their scorings. This article first summarizes the studies of rater cognition and other influencing factors on creativity assessment, including characteristics of raters, information of creators and socio-cultural factors. It further examines inter-rater reliability indexes and their limitations, as well as the applications of Generalization Theory and Many-Facet Rasch Model in quantifying and controlling of rater effects. Finally, this paper specifies directions of future research based on the existing limitations, including deepening the investigation on rater cognition in creativity assessment, integrating the studies of rater effects on different levels, and developing new methods and techniques of creativity assessment.

Key words: creativity, subjective scoring, rater effects, rater cognition, inter-rater agreement

中图分类号: