ISSN 1671-3710
CN 11-4766/R

Advances in Psychological Science ›› 2019, Vol. 27 ›› Issue (1): 171-180.doi: 10.3724/SP.J.1042.2019.00171

• Research Method • Previous Articles     Next Articles

Rater effects in creativity assessment

HAN Jiantao1,2,3, LIU Wenling1, PANG Weiguo1()   

  1. 1 School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
    2 School of Educational Science, Anhui Normal University, Wuhu 241000, China
    3 School of Literature Media and Educational Science, Chaohu College, Chaohu 238000, China
  • Received:2017-07-28 Online:2019-01-15 Published:2018-11-23

Abstract:

Rater effects refer to the impact of different raters’ idiosyncrasies in their behaviors on the evaluation results in creativity assessment. Rater effects are due to the difference in raters’ cognitive process of the evaluation, which are externally reflected in the difference of their scorings. This article first summarizes the studies of rater cognition and other influencing factors on creativity assessment, including characteristics of raters, information of creators and socio-cultural factors. It further examines inter-rater reliability indexes and their limitations, as well as the applications of Generalization Theory and Many-Facet Rasch Model in quantifying and controlling of rater effects. Finally, this paper specifies directions of future research based on the existing limitations, including deepening the investigation on rater cognition in creativity assessment, integrating the studies of rater effects on different levels, and developing new methods and techniques of creativity assessment.

Key words: creativity, subjective scoring, rater effects, rater cognition, inter-rater agreement

CLC Number: