ISSN 0439-755X
CN 11-1911/B

Acta Psychologica Sinica ›› 2021, Vol. 53 ›› Issue (1): 15-25.doi: 10.3724/SP.J.1041.2021.00015

• Reports of Empirical Studies • Previous Articles     Next Articles

The brain mechanism of explicit and implicit processing of affective prosodies: An fNIRS study

LEI Zhen1, BI Rong2, MO Licheng2, YU Wenwen2, ZHANG Dandan1,2()   

  1. 1China Center for Behavioral Economics and Finance & School of Economics, Southwestern University of Finance and Economics, Chengdu 611130, China
    2College of Psychology, Shenzhen University, Shenzhen 518060, China
  • Received:2020-03-03 Published:2021-01-25 Online:2020-11-24
  • Contact: ZHANG Dandan
  • Supported by:
    National Natural Science Foundation of China(31970980);Shenzhen basic research free exploration project(JCYJ20180305124305294);Shenzhen Hong Kong Institute of brain science innovation(2019shibs0003);Ministry of Education Humanities and social sciences research planning fund project(16XJA79004)


Emotional expressions of others embedded in speech prosodies are important for social interactions. Affective prosody refers to a way to express and convey emotions through the dynamic changes of various acoustic cues such as pitch, intensity, stress, and intonation in speech, without relying on vocabulary and grammatical structure. Previous studies have shown that STC, IFG, OFC, and other cerebral cortex and subcortical structures are involved in affective prosody processing, and gradually formed a hierarchical model. However, existing studies on the neural mechanism of affective prosody processing mostly focus on the difference between non-neutral affective prosody and neutral prosody, while the comparison between various non-neutral affective prosody is less investigated. Besides, the differences involved in brain regions of affective prosody processing under explicit and implicit tasks are still not clear. Furthermore, it is necessary to further accumulate experimental evidence based on noise-free brain imaging technology such as fNIRS, which has a noise-free feature and is especially suitable for speech processing research.

This study used functional near-infrared spectroscopy to investigate how speech prosodies of different emotional categories are processed in the cortex under different task conditions. A group of 25 college students participated in this study with a 3 (emotion: angry vs. fearful vs. happy) í 2 (task focus: explicit vs. implicit) within-participant factorial design. We manipulated task focus by adopting two different tasks, with emotional discrimination task as explicit condition and sex discrimination task as implicit condition. Ten phonological materials for each of angry, fearful, and happy prosody were selected from the Chinese Speech Emotion Database and consisted of the corresponding affective prosodies and neutral prosodies. The emotional explicit task was to count the affective and neutral sentences contained in each 10-second speech, and the affective implicit task was to count the sentences played by two women in each 10-second speech. A multi-channel fNIRS system was used to record brain activity in a continuous waveform. According to existing literature, the brain regions observed in this study are the bilateral frontal and temporal lobes. Therefore, we used 13 emitters and 15 detectors to form 37 effective observation channels.

We first adopted NirSpark-2442 software to preprocess the data, and then conducted general linear model analyses to calculate the cortical activation related to the task. The results showed that the brain activation was significantly higher when angry prosody was contrasted to fearful and happy ones in left frontal pole / orbitofrontal cortex, when happy prosody was contrasted to fearful and angry ones in left inferior frontal gyrus, and when fearful prosody was contrasted to angry and happy ones in right supramarginal gyrus. Importantly, there was an interaction between emotion and task. In the explicit task, cortex activity in the right supramarginal gyrus was more sensitive to fearful prosody than to angry and happy ones. But no similar results were found under angry and happy prosody. In addition, the brain activation in temporopolar, superior temporal gyrus, and middle temporal gyrus in the explicit task was greater than that in the implicit task.

The present study demonstrated the specific brain regions for processing angry, fearful and happy prosody were left frontal pole / orbitofrontal cortex, right supramarginal gyrus, and left inferior frontal gyrus, respectively, and revealed the important role of right superior temporal gyrus and right supramarginal gyrus in emotional explicit task. These findings partially support the hierarchical model of affective prosody and question the third level of the model.

Key words: emotion, phonetic prosody, superior temporal gyrus, orbitofrontal cortex, inferior frontal gyrus, supramarginal gyrus