外显和内隐情绪韵律加工的脑机制：近红外成像研究

doi:10.3724/SP.J.1041.2021.00015

摘要/Abstract

摘要：

准确识别言语中的情绪韵律信息对社会交往非常重要。本研究采用功能近红外成像技术, 探索外显和内隐情绪加工条件下愤怒、恐惧、快乐三种情绪韵律加工过程中的大脑皮层神经活动。结果表明, 对愤怒、恐惧、快乐韵律进行特异性加工的脑区分别为左侧额极/眶额叶、右侧缘上回、左侧额下回, 其中右侧缘上回脑区同时受到情绪和任务的调控。此外, 右侧颞中回、颞下回和颞极在情绪外显任务中的激活明显强于内隐任务。本研究的结果部分支持了情绪韵律的层次模型, 也对该模型的第三层次, 即“额区对语音情绪信息的精细加工需要外显性情绪加工任务参与”提出了质疑。

关键词: 情绪, 语音韵律, 颞上回, 眶额叶, 额下回, 缘上回

Abstract:

Emotional expressions of others embedded in speech prosodies are important for social interactions. Affective prosody refers to a way to express and convey emotions through the dynamic changes of various acoustic cues such as pitch, intensity, stress, and intonation in speech, without relying on vocabulary and grammatical structure. Previous studies have shown that STC, IFG, OFC, and other cerebral cortex and subcortical structures are involved in emotional prosody processing, and gradually formed a hierarchical model. However, existing studies on the neural mechanism of emotional prosody processing mostly focus on the difference between non-neutral emotional prosody and neutral prosody, while the comparison between various non-neutral emotional prosody is less investigated. Besides, the differences involved in brain regions of emotional prosody processing under explicit and implicit tasks are still not clear. Furthermore, it is necessary to further accumulate experimental evidence based on noise-free brain imaging technology, such as the noise-free features of fNIRS are especially suitable for speech processing research.
This study used functional near-infrared spectroscopy to investigate how speech prosodies of different emotional categories are processed in the cortex under different task conditions. A group of 25 college students participated in this study with a 3 (emotion: anger vs. fearful vs. happy) by 2 (task focus: explicit vs. implicit) within-participant factorial design. We manipulated task focus by adopting two different tasks, with emotional discrimination task as explicit condition and sex discrimination task as implicit condition. Ten phonological materials for each of anger, fearful, and happy prosody were selected from the Chinese Speech Emotion Database and consisted of the corresponding emotional prosodies and neutral prosodies. The emotional explicit task was to count the emotional and neutral sentences contained in each 10-second speech, and the emotional implicit task was to count the sentences played by two women in each 10-second speech. A multi-channel fNIRS system was used to record brain activity in a continuous waveform. According to existing literature, the brain regions observed in this study are the bilateral frontal and temporal lobes. Therefore, we used 13 emitters and 15 detectors to form 37 effective observation channels.
We first adopted NirSpark-2442 software to preprocess the data, and then conducted general linear model analyses to calculate the cortical activation related to the task. The results showed that the brain activation was significantly higher when anger was contrasted to fearful and happy prosody in left frontal pole / orbitofrontal cortex, and when happy was contrasted to fearful and anger prosody in left inferior frontal gyrus, and when fearful was contrasted to anger and happy prosody in right supramarginal gyrus. Importantly, there was an interaction between emotion and task. In the explicit task, cortex activity in the right supramarginal gyrus was more sensitive to fearful than to anger and happy prosodies. But no similar results were found under anger and happy prosody. In addition, the brain activation in temporopolar, superior temporal gyrus, and middle temporal gyrus with the explicit task was greater than that in the implicit task.
The present study demonstrated the specific brain regions for processing angry, fearful and happy prosody were left frontal pole/orbitofrontal cortex, right supramarginal gyrus, and left inferior frontal gyrus respectively, and the important role of right superior temporal gyrus and right supramarginal gyrus in emotional explicit task. These findings partially support the hierarchical model of emotional prosody and question the third level of the model.

Key words: emotion, phonetic prosody, superior temporal gyrus, orbitofrontal cortex, inferior frontal gyrus, supramarginal gyrus

中图分类号:

B842
B845

雷震, 毕蓉, 莫李澄, 于文汶, 张丹丹. (2021). 外显和内隐情绪韵律加工的脑机制：近红外成像研究. 心理学报, 53(1), 15-25.

LEI Zhen, BI Rong, MO Licheng, YU Wenwen, ZHANG Dandan. (2021). The brain mechanism of explicit and implicit processing of emotional prosodies: An fNIRS study. Acta Psychologica Sinica, 53(1), 15-25.

图/表 6

参考文献 55

[1]	Adolphs, R., Damasio, H., Tranel, D., Cooper, G., & Damasio, A. R. (2000). A role for somatosensory cortices in the visual recognition of emotion as revealed by three-dimensional lesion mapping. The Journal of Neuroscience, 20(7), 2683-2690. URL pmid: 10729349
[2]	Alba-Ferrara, L., Kochen, S., & Hausmann, M. (2018). Emotional prosody processing in epilepsy: Some insights on brain reorganization. Frontiers in Human Neuroscience, 12, 92. URL pmid: 29593517
[3]	Aryani, A., Hsu, C.-T., & Jacobs, A. M. (2018). The sound of words evokes affective brain responses. Brain Sciences, 8(6), 94. doi: 10.3390/brainsci8060094 URL
[4]	Bach, D. R., Grandjean, D., Sander, D., Herdener, M., Strik, W. K., & Seifritz, E. (2008). The effect of appraisal level on processing of emotional prosody in meaningless speech. Neuroimage, 42(2), 919-927. doi: 10.1016/j.neuroimage.2008.05.034 URL pmid: 18586524
[5]	Beaucousin, V., Zago, L., Herve, P.-Y., Strelnikov, K., Crivello, F., Mazoyer, B., & Tzourio-Mazoyer, N. (2011). Sex-dependent modulation of activity in the neural networks engaged during emotional speech comprehension. Brain Research, 1390, 108-117. URL pmid: 21439268
[6]	Belyk, M., & Brown, S. (2014). Perception of affective and linguistic prosody: An ALE meta-analysis of neuroimaging studies. Social Cognitive and Affective Neuroscience, 9(9), 1395-1403. doi: 10.1093/scan/nst124 URL pmid: 23934416
[7]	Ben-David, B. M., Multani, N., Shakuf, V., Rudzicz, F., & van Lieshout, P. H. H. M. (2016). Prosody and semantics are separate but not separable channels in the perception of emotional speech: test for rating of emotions in speech. Journal of Speech Language and Hearing Research, 59(1), 72-89. doi: 10.1044/2015_JSLHR-H-14-0323 URL
[8]	Beyer, F., Munte, T. F., Gottlich, M., & Kramer, U. M. (2014). Orbitofrontal cortex reactivity to angry facial expression in a social interaction correlates with aggressive behavior. Cerebral Cortex, 25(9), 3057-3063. URL pmid: 24842782
[9]	Brück, C., Kreifelts, B., & Wildgruber, D. (2011). Emotional voices in context: A neurobiological model of multimodal affective information processing. Physics of Life Reviews, 8(4), 383-403. doi: 10.1016/j.plrev.2011.10.002 URL pmid: 22035772
[10]	Calvo, M. G., & Nummenmaa, L. (2016). Perceptual and affective mechanisms in facial expression recognition: An integrative review. Cognition and Emotion, 30(6), 1081-1106. URL pmid: 26212348
[11]	Dieler, A. C., Tupak, S. V., & Fallgatter, A. J. (2012). Functional near-infrared spectroscopy for the assessment of speech related tasks. Brain and Language, 121(2), 90-109. URL pmid: 21507475
[12]	Enea, V., & Iancu, S. (2016). Processing emotional body expressions: state-of-the-art. Social Neuroscience, 11(5), 495-506. URL pmid: 26513592
[13]	Ethofer, T., Anders, S., Erb, M., Herbert, C., Wiethoff, S., Kissler, J., … Wildgruber, D. (2006). Cerebral pathways in processing of affective prosody: a dynamic causal modeling study. Neuroimage, 30(2), 580-587. URL pmid: 16275138
[14]	Ethofer, T., Bretscher, J., Gschwind, M., Kreifelts, B., Wildgruber, D., & Vuilleumier, P. (2012). Emotional voice areas: Anatomic location, functional properties, and structural connections revealed by combined fMRI/DTI. Cerebral Cortex, 22(1), 191-200. URL pmid: 21625012
[15]	Ethofer, T., Kreifelts, B., Wiethoff, S., Wolf, J., Grodd, W., Vuilleumier, P., & Wildgruber, D. (2009b). Differential influences of emotion, task, and novelty on brain regions underlying the processing of speech melody. Journal of Cognitive Neuroscience, 21(7), 1255-1268. URL pmid: 18752404
[16]	Ethofer, T., van de Ville, D., Scherer, K., & Vuilleumier, P. (2009a). Decoding of emotional information in voice- sensitive cortices. Current Biology, 19(12), 1028-1033. URL pmid: 19446457
[17]	Fox, K. C. R., Yih, J., Raccah, O., Pendekanti, S. L., Limbach, L. E., Maydan, D. D., & Parvizi, J. (2018). Changes in subjective experience elicited by direct stimulation of the human orbitofrontal cortex. Neurology, 91(16), e1519-e1527. URL pmid: 30232252
[18]	Frühholz, S., Ceravolo, L., & Grandjean, D. (2012). Specific brain networks during explicit and implicit decoding of emotional prosody. Cerebral Cortex, 22(5), 1107-1117. URL pmid: 21750247
[19]	Frühholz, S., & Grandjean, D. (2013a). Multiple subregions in superior temporal cortex are differentially sensitive to vocal expressions: a quantitative meta-analysis. Neuroscience and Biobehavioral Reviews, 37(1), 24-35. URL pmid: 23153796
[20]	Frühholz, S., & Grandjean, D. (2013b). Processing of emotional vocalizations in bilateral inferior frontal cortex. Neuroscience and Biobehavioral Reviews, 37(10), 2847-2855. doi: 10.1016/j.neubiorev.2013.10.007 URL
[21]	Frühholz, S., Hofstetter, C., Cristinzio, C., Saj, A., Seeck, M., & Vuilleumier, P., & Grandjean, D.(2015). Asymmetrical effects of unilateral right or left amygdala damage on auditory cortical processing of vocal emotions. Proceedings of the National Academy of Sciences of the United States of America, 112(5), 1583-1588. URL pmid: 25605886
[22]	Frühholz, S., Trost, W., & Kotz, S. A. (2016). The sound of emotions-Towards a unifying neural network perspective of affective sound processing. Neuroscience and Biobehavioral Reviews, 68, 96-110. URL pmid: 27189782
[23]	Goucha, T., & Friederici, A. D. (2015). The language skeleton after dissecting meaning: A functional segregation within Broca’s Area. Neuroimage, 114, 294-302. URL pmid: 25871627
[24]	Hartwigsen, G., Baumgaertner, A., Price, C. J., Koehnke, M., Ulmer, S., & Siebner, H. R. (2010). Phonological decisions require both the left and right supramarginal gyri. Proceedings of the National Academy of Sciences of the United States of America, 107(38), 16494-16499. URL pmid: 20807747
[25]	Hensel, L., Bzdok, D., Müller, V. I., Zilles, K., & Eickhoff, S. B. (2015). Neural correlates of explicit social judgments on vocal stimuli. Cerebral Cortex, 25(5), 1152-1162. URL pmid: 24243619
[26]	Herpertz, S. C., Nagy, K., Ueltzhöffer, K., Schmitt, R., Mancke, F., Schmahl, C., & Bertsch, K. (2017). Brain mechanisms underlying reactive aggression in borderline personality disorder-sex matters. Biological Psychiatry, 82(4), 257-266. URL pmid: 28388995
[27]	Hinojosa, J. A., Mercado, F., & Carretié, L. (2015). N170 sensitivity to facial expression: A meta-analysis. Neuroscience and Biobehavioral Reviews, 55, 498-509. URL pmid: 26067902
[28]	Johnstone, T., van Reekum, C. M., Oakes, T. R., & Davidson, R. J. (2006). The voice of emotion: an FMRI study of neural responses to angry and happy vocal expressions. Social Cognitive and Affective Neuroscience, 1(3), 242-249. URL pmid: 17607327
[29]	Kirby, L. A. J., & Robinson, J. L. (2017). Affective mapping: An activation likelihood estimation (ALE) meta-analysis. Brain and Cognition, 118, 137-148. doi: 10.1016/j.bandc.2015.04.006 URL pmid: 26074298
[30]	Knight, M. J., & Baune, B. T. (2019). Social cognitive abilities predict psychosocial dysfunction in major depressive disorder. Depression and Anxiety, 36(1), 54-62. URL pmid: 30211966
[31]	Kotz, S. A., Kalberlah, C., Bahlmann, J., Friederici, A. D., & Haynes, J.-D. (2013). Predicting vocal emotion expressions from the human brain. Human Brain Mapping, 34(8), 1971-1981. URL pmid: 22371367
[32]	Kotz, S. A., Meyer, M., Alter, K., Besson, M., von Cramon, D. Y., & Friederici, A. D. (2003). On the lateralization of emotional prosody: an event-related functional MR investigation. Brain and Language, 86(3), 366-376. URL pmid: 12972367
[33]	Köchel, A., Schöngassner, F., & Schienle, A. (2013). Cortical activation during auditory elicitation of fear and disgust: a near-infrared spectroscopy (NIRS) study. Neuroscience Letters, 9(549), 197-200.
[34]	Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L.,… Fox, P. (2000). Automated Talairach atlas labels for functional brain mapping. Human Brain Mapping, 10(3), 120-131. doi: 10.1002/1097-0193(200007)10:3<120::aid-hbm30>3.0.co;2-8 URL pmid: 10912591
[35]	Liebenthal, E., Silbersweig, D. A., & Stern, E. (2016). The Language, Tone and prosody of emotions: neural substrates and dynamics of spoken-word emotion perception. Frontiers in Aging Neuroscience, 10, 506.
[36]	Lin, Y., Ding, H., & Zhang, Y. (2018). Emotional prosody processing in schizophrenic patients: A selective review and meta-analysis. Journal of Clinical Medicine, 7(10), 363.
[37]	Lindquist, K. A., Wager, T. D., Kober, H., Bliss-Moreau, E., & Barrett, L. F. (2012). The brain basis of emotion: a meta- analytic review. Behavioral and Brain Sciences, 35(3), 121-143.
[38]	Liu, P., & Pell, M. D. (2012). Recognizing vocal emotions in Mandarin Chinese: a validated database of Chinese vocal emotional stimuli. Behavior Research Methods, 44, 1042-1051. URL pmid: 22539230
[39]	Matsui, T., Nakamura, T., Utsumi, A., Sasaki, A. T., Koike, T., Yoshida, Y., … Sadato, N. (2016). The role of prosody and context in sarcasm comprehension: Behavioral and fMRI evidence. Neuropsychologia, 87, 74-84. URL pmid: 27157883
[40]	Mitchell, R. L. C. (2007). fMRI delineation of working memory for emotional prosody in the brain: commonalities with the lexico-semantic emotion network. Neuroimage, 36(3), 1015-1025. URL pmid: 17481919
[41]	Mitchell, R. L. C., & Xu, Y. (2015). What is the value of embedding artificial emotional prosody in human-computer interactions? Implications for theory and design in psychological science. Frontiers in Psychology, 6, 1750. URL pmid: 26617563
[42]	Molavi, B., & Dumont, G. A. (2012). Wavelet-based motion artifact removal for functional near-infrared spectroscopy. Physiological Measurement, 33(2), 259-270. URL pmid: 22273765
[43]	Mothes-Lasch, M., Mentzel, H.-J., Miltner, W. H. R., & Straube, T. (2011). Visual attention modulates brain activation to angry voices. Journal of Neuroscience, 31(26), 9594-9598. URL pmid: 21715624
[44]	Patel, S., Oishi, K., Wright, A., Sutherland-Foggio, H., Saxena, S., Sheppard, S. M., & Hillis, A. E. (2018). Right hemisphere regions critical for expression of emotion through prosody. Frontiers in Neurology, 9, 224. URL pmid: 29681885
[45]	Patterson, R. D., Uppenkamp, S., Johnsrude, I. S., & Griffiths, T. D. (2002). The processing of temporal pitch and melody information in auditory cortex. Neuron, 36(4), 767-776. URL pmid: 12441063
[46]	Paulmann, S., Seifert, S., & Kotz, S. A. (2010). Orbito-frontal lesions cause impairment during late but not early emotional prosodic processing. Social Neuroscience, 5(1), 59-75. URL pmid: 19658025
[47]	Quadflieg, S., Mohr, A., Mentzel, H.-J., Miltner, W. H. R., & Straube, T. (2008). Modulation of the neural network involved in the processing of anger prosody: the role of task-relevance and social phobia. Biological Psychology, 78(2), 129-137. URL pmid: 18353521
[48]	Ross, E. D. (1981). The aprosodias. Functional-anatomic organization of the affective components of language in the right hemisphere. Archives of Neurology, 38(9), 561-569. URL pmid: 7271534
[49]	Schirmer, A., & Kotz, S. A. (2006). Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing. Trends in Cognitive Sciences, 10(1), 24-30. URL pmid: 16321562
[50]	Steber, S., König, N., Stephan, F., & Rossi, S. (2020). Uncovering electrophysiological and vascular signatures of implicit emotional prosody. Scientific Reports, 10(1), 5807. URL pmid: 32242032
[51]	Tong, Y., Hocke, L. M., & Frederick, B. B., (2011). Isolating the sources of widespread physiological fluctuations in functional near-infrared spectroscopy signals. Journal of Biomedical Optics, 16(10), 106005. URL pmid: 22029352
[52]	Witteman, J., van Heuven, V. J., & Schiller, N. O. (2012). Hearing feelings: a quantitative meta-analysis on the neuroimaging literature of emotional prosody perception. Neuropsychologia, 50(12), 2752-2763. URL pmid: 22841991
[53]	Zhang, D., Chen, Y., Hou, X., & Wu, Y. J. (2019). Near-infrared spectroscopy reveals neural perception of vocal emotions in human neonates. Human Brain Mapping, 40(8), 2434-2448. URL pmid: 30697881
[54]	Zhang, D., Zhou, Y., Hou, X., Cui, Y., & Zhou, C. (2017). Discrimination of emotional prosodies in human neonates: A pilot fNIRS study. Neuroscience Letters, 658, 62-66. URL pmid: 28842278
[55]	Zhang, D., Zhou, Y., & Yuan, J. (2018). Speech prosodies of different emotional categories activate different brain regions in adult cortex: an fNIRS study. Scientific Reports, 8(1), 218. URL pmid: 29317758

通道	脑区	F	p*	愤怒β值	恐惧β值	快乐β值
3	L Frontopolar/orbitofrontal area	12.51	0.001	0.21 ± 0.20	0.12 ± 0.23	0.06 ± 0.20
9	L pars triangularis/Broca’s area	24.24	< 0.001	0.10 ± 0.16	0.10 ± 0.15	0.21 ± 0.15
32	R Supramarginal gyrus	12.48	0.001	0.10 ± 0.56	0.36 ± 0.51	0.11 ± 0.45

通道	脑区	F	p*	愤怒β值	恐惧β值	快乐β值
3	L Frontopolar/orbitofrontal area	12.51	0.001	0.21 ± 0.20	0.12 ± 0.23	0.06 ± 0.20
9	L pars triangularis/Broca’s area	24.24	< 0.001	0.10 ± 0.16	0.10 ± 0.15	0.21 ± 0.15
32	R Supramarginal gyrus	12.48	0.001	0.10 ± 0.56	0.36 ± 0.51	0.11 ± 0.45

通道	脑区	F	p*	内隐β值	外显β值
27	R Temporopolar area	11.62	0.004	0.04 ± 0.36	0.32 ± 0.42
28	R Superior temporal gyrus	26.17	< 0.001	0.05 ± 0.45	0.37 ± 0.43
29	R Middle temporal gyrus	15.84	0.003	-0.03 ± 0.49	0.34 ± 0.53

通道	脑区	F	p*	内隐β值	外显β值
27	R Temporopolar area	11.62	0.004	0.04 ± 0.36	0.32 ± 0.42
28	R Superior temporal gyrus	26.17	< 0.001	0.05 ± 0.45	0.37 ± 0.43
29	R Middle temporal gyrus	15.84	0.003	-0.03 ± 0.49	0.34 ± 0.53