Emotional processing of winning and losing facial expression and body posture
WANG Lili1, FENG Wenfeng2, JIA Lina3, ZHU Xiangru4, LUO Wenbo5, YANG Suyong6, LUO Yue-jia7,8
1 School of Educational Science, Huaiyin Normal University, Huaian 223300, China;
2 Department of Psychology, Soochow University, Jiangsu 215000, China;
3 School of Humanities, Jiangnan University, Wuxi 214122, China;
4 Institute of Cognition, Brain and Health, Henan University, Kaifeng 475001, China;
5 Research Center of Brain and Cognitive Neuroscience, Liaoning Normal University, Dalian 116029, China;
6 Key Laboratory of Exercise and Health Sciences of Ministry of Education, Shanghai University of Sport, Shanghai 200438, China;
7 College of Psychology and Sociology, Shenzhen University, Shenzhen 518060, China;
8 Shenzhen Key Laboratory of Affective and Social Cognitive Science, Shenzhen University, Shenzhen 518060, China
Abstract：For humans, both face and body play important roles in conveying emotional information. Previous studies showed that body postures rather than faces could provide more valid information about valence in the recognition of victory and defeat. The present study aimed to compare the processing of the faces and the bodies of victory and defeat.
The current study employed emotional expressions of Chinese professional players reacting to victory or defeat to compare the processing of emotional faces and body postures using behavioral and ERP recordings. 80 images (40 winners and 40 losers) were obtained through Google and Baidu image search, using the search keyword “reacting to winning a point” or “reacting to losing a point”, intersected with “tennis” or “table tennis” or “badminton”. In Experiment 1, the behavioral experiment asked participants to rate the valence and intensity of the faces and the body postures on a 9 point scale (valence: 1-the most negative and 9-the most positive; intensity: 1-the least intense and 9-the most intense). In Experiment 2, participants were asked to determine the type of emotion (neutral, happiness, sadness, anger, fear, and disgust) expressed by the faces and the body postures. In the ERP study (Experiment 3), participants were instructed to indicate the valence (positive or negative) of the faces and the body postures.
The behavioral results showed that body postures rather than faces facilitated the discrimination between victory and defeat. Compared to body postures, the faces were more complex and involved a variety of facial expressions. The behavioral result of the ERP study showed that body postures rather than faces could provide more valid information about valence. The ERP results showed that the emotional information of body postures could be detected earlier than faces, as reflected by larger N170 amplitudes for winning body postures than losing body postures. However, there was no significant N170 difference between winning faces and losing faces. The emotional effect of faces was reflected by the EPN component, and losing faces elicited larger negative EPN amplitudes than winning faces. On the contrary, winning body postures elicited larger negative EPN amplitudes than losing body postures. Moreover, victory elicited larger LPP amplitudes than defeat under both face and body conditions.
These data suggest that the higher rate of discrimination between winning body postures and losing body postures is possibly due to the stimulus evaluation and categorization of body postures at multiple stages of processing. It is hoped that the current results regarding the emotional processing of facial and body expressions will help us understand the mechanisms of the emotional brain.
Adolphs, R. (2002). Neural systems for recognizing emotion. Current Opinion in Neurobiology, 12(2), 169–177. Aviezer, H., Messinger, D. S., Zangvil, S., Mattson, W. I., Gangi, D. N., & Todorov, A. (2015). Thrill of victory or agony of defeat? Perceivers fail to utilize information in facial movements. Emotion, 15(6), 791–797. Aviezer, H., Trope, Y., & Todorov, A. (2012). Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science, 338(6111), 1225–1229. Bentin, S., Allison, T., Puce, A., Perez, E., & McCarthy, G. (1996). Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience, 8(6), 551–565. Blau, V. C., Maurer, U., Tottenham, N., & McCandliss, B. D. (2007). The face-specific N170 component is modulated by emotional facial expression. Behavioral and Brain Functions, 3, 7. Bogart, K. R., Tickle-Degnen, L., & Ambady, N. (2014). Communicating without the face: Holistic perception of emotions of people with facial paralysis. Basic & Applied Social Psychology, 36(4), 309–320. Borhani, K., Borgomaneri, S., Làdavas, E., & Bertini, C. (2016). The effect of alexithymia on early visual processing of emotional body postures. Biological Psychology, 115, 1–8. Borhani, K., Làdavas, E., Maier, M. E., Avenanti, A., & Bertini, C. (2015). Emotional and movement-related body postures modulate visual processing. Social Cognitive & Affective Neuroscience, 10(8), 1092–1101. Carroll, J. M., & Russell, J. A. (1996). Do facial expressions signal specific emotions? Judging emotion from the face in context. Journal of Personality and Social Psychology, 70(2), 205–218. de Gelder, B., & Hortensius, R. (2014). The many faces of the emotional body. In J. Decety & Y. Christen (Eds.), New frontiers in social neuroscience (Vol. 21, pp. 153–164). New York: Springer. de Gelder, B. (2006). Towards the neurobiology of emotional body language. Nature Reviews Neuroscience, 7(3), 242–249. Eimer, M. (2000). The face-specific N170 component reflects late stages in the structural encoding of faces. Neuroreport, 11(10), 2319–2324. Eimer, M., Kiss, M., & Holmes, A. (2008). Links between rapid ERP responses to fearful faces and conscious awareness. Journal of Neuropsychology, 2, 165–181. Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48(4), 384–392. Ekman, P., & Friesen, W. V. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1, 49–98. Ekman, P., & Friesen, W. V. (1978). Facial action coding system: Investigator’s guide. Palo Alto, CA: Consulting Psychologists Press. Gu, Y. Y., Mai, X. Q., & Luo, Y. J. (2013). Do bodily expressions compete with facial expressions? Time course of integration of emotional signals from the face and the body. PLoS One, 8(7), e66762. Hietanen, J. K., & Astikainen, P. (2013). N170 response to facial expressions is modulated by the affective congruency between the emotional expression and preceding affective picture. Biological Psychology, 92(2), 114–124. Kiss, M., & Eimer, M. (2008). ERPs reveal subliminal processing of fearful faces. Psychophysiology, 45(2), 318–326. Kissler, J., & Herbert, C. (2013). Emotion, etmnooi, or emitoon?--Faster lexical access to emotional than to neutral words during reading. Biological Psychology, 92(3), 464–479. Kok, A. (1997). Event-related-potential (ERP) reflections of mental resource?s: A review and synthesis. Biological Psychology, 45(1-3), 19–56. Kutas, M., McCarthy, G., & Donchin, E. (1977). Augmenting mental chronometry: The P300 as a measure of stimulus evaluation time. Science, 197(4305), 792–795. Lepp?nen, J. M., Moulson, M. C., Vogel-Farley, V. K., & Nelson, C. A. (2007). An ERP study of emotional face processing in the adult and infant brain. Child Development, 78(1), 232–245. Luo, Q., Holroyd, T., Jones, M., Hendler, T., & Blair, J. (2007). Neural dynamics for facial threat processing as revealed by gamma band synchronization using MEG. NeuroImage, 34(2), 839–847. Luo, Q., Holroyd, T., Majestic, C., Cheng, X., Schechter, J., & Blair, R. J. (2010). Emotional automaticity is a matter of timing. Journal of Neuroscience, 30(17), 5825–5829. Luo, W. B., Feng, W. F., He, W. Q., Wang, N. Y., & Luo, Y. J. (2010). Three stages of facial expression processing: ERP study with rapid serial visual presentation. NeuroImage, 49(2), 1857–1867. Macmillan, N. A., & Creelman, D. C. (2004). Detection theory: A user's guide (2nd ed.). New York: Psychology Press. Matsumoto, D., & Willingham, B. (2006). The thrill of victory and the agony of defeat: Spontaneous expressions of medal winners of the 2004 Athens Olympic Games. Journal of Personality and Social Psychology, 91(3), 568–581. Meeren, H. K. M., van Heijnsbergen, C. C. R. J., & de Gelder, B. (2005). Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the United States of America, 102(45), 16518–16523. Mehrabian, A., & Russell, J. A. (1974). An approach to environmental psychology. Cambridge, MA: MIT. Oatley, K., & Johnson-Laird, P. N. (1987). Towards a cognitive theory of emotions. Cognition and Emotion, 1(1), 29–50. Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective picture processing: An integrative review of ERP findings. Biological Psychology, 77(3), 247–265. Pegna, A. J., Landis, T., & Khateb, A. (2008). Electrophysiological evidence for early non-conscious processing of fearful facial expressions. International Journal of Psychophysiology, 70(2), 127–136. Pegna, A. J., Darque, A., Berrut, C., & Khateb, A. (2011). Early ERP modulation for task-irrelevant subliminal faces. Frontiers in Psychology, 2, 88. Pourtois, G., Grandjean, D., Sander, D., & Vuilleumier, P. (2004). Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex, 14(6), 619–633. Pritchard, W. S. (1981). Psychophysiology of P300. Psychological Bulletin, 89(3), 506–540. Righart, R., & de Gelder, B. (2007). Impaired face and body perception in developmental prosopagnosia. Proceedings of the National Academy of Sciences of the United States of America, 104(43), 17234–17238. Schupp, H. T., Flaisch, T., Stockburger, J., & Jungh?fer, M. (2006). Emotion and attention: Event-related brain potential studies. Progress in Brain Research, 156, 31–51. Schupp, H. T., Jungh?fer, M., Weike, A. I., & Hamm, A. O. (2004). The selective processing of briefly presented affective pictures: An ERP analysis. Psychophysiology, 41(3), 441–449. Schyns, P. G., Petro, L. S., & Smith, M. L. (2007). Dynamics of visual information integration in the brain for categorizing facial expressions. Current Biology, 17(18), 1580–1585. Stekelenburg, J. J., & de Gelder, B. (2004). The neural correlates of perceiving human bodies: An ERP study on the body-inversion effect. Neuroreport, 15(5), 777–780. Wang, L. L., Feng, C. L., Mai, X. Q., Jia, L., Zhu, X. R., Luo, W. B., & Luo Y. J. (2016). The impact of perceptual load on the non-conscious processing of fearful faces. PLoS One, 11(5), e0154914. Wang, L. L., Xia, L. S., & Zhang, D. D. (2017). Face-body integration of intense emotional expressions of victory and defeat. PLoS One, 12(2), e0171656. Wild-Wall, N., Dimigen, O., & Sommer, W. (2008). Interaction of facial expressions and familiarity: ERP evidence. Biological Psychology, 77(2), 138–149. Xia, M., Li, X. L., Ye, C., & Li, H. (2014). The ERPs for the facial expression processing. Advances in Psychological Science, 22(10), 1556–1563.
[侠牧, 李雪榴, 叶春, 李红. (2014). 面部表情加工的ERP成分. 心理科学进展, 22(10), 1556–1563.] Xiao, R. Q., Li, X. C., Li, L., & Wang, Y. M. (2016). Can we distinguish emotions from faces? Investigation of implicit and explicit processes of peak facial expressions. Frontiers in Psychology, 7, 1330. Zhang, D. D., He, W. Q., Wang, T., Luo, W. B., Zhu, X. R., Gu, R. L.,… Luo, Y. J. (2014). Three stages of emotional word processing: An ERP study with rapid serial visual presentation. Social Cognitive & Affective Neuroscience, 9(12), 1897–1903. Zhang, D. D., Luo, W. B., & Luo, Y. J. (2013). Single-trial ERP evidence for the three-stage scheme of facial expression processing. Science China Life Sciences, 56(9), 835–847.
[张丹丹, 罗文波, 罗跃嘉. (2013). 面孔表情加工三阶段模型的单试次ERP证据. 中国科学:生命科学, 43(8), 643–656.] Zhang, D. D., Wang, L. L., & Luo, Y. J. (2012). Individual differences in detecting rapidly presented fearful faces. PLoS One, 7(11), e49517. Zhang, D. D., Zhao, T., Liu, Y. Z., & Chen, Y. M. (2015). Comparison of facial expressions and body expressions: An event-related potential study. Acta Psychologica Sinica, 47(8), 963–970.
[张丹丹, 赵婷, 柳昀哲, 陈玉明. (2015). 恐惧情绪面孔和身体姿势加工的比较: 事件相关电位研究. 心理学报, 47(8), 963–970.]