面孔表情和声音情绪信息整合对返回抑制的影响

doi:10.3724/SP.J.1041.2022.00331

摘要/Abstract

摘要：

返回抑制(inhibition of return, IOR)与情绪刺激都具有引导注意偏向、提高搜索效率的特点, 但二者间是否存在一定的交互作用迄今为止尚不明确。研究采用“线索-目标”范式并在视听双通道呈现情绪刺激来考察情绪刺激的加工与IOR的交互作用。实验1中情绪刺激以单通道视觉面孔或一致的视听双通道呈现, 实验2通过在视听通道呈现不一致的情绪刺激进一步考察视听双通道情绪一致刺激对IOR的影响是否是由听觉通道一致的情绪刺激导致的, 即是否对听觉通道的情绪刺激进行了加工。结果发现, 视听双通道情绪一致刺激能够削弱IOR, 但情绪不一致刺激与IOR之间不存在交互作用, 并且单双通道的IOR不存在显著差异。研究结果表明仅在视听双通道呈现情绪一致刺激时, 才会影响同一阶段的IOR, 这进一步支持了IOR的知觉抑制理论。

关键词: 返回抑制, 情绪, 视听觉整合

Abstract:

Both inhibition of return (IOR) and emotion have the characteristics of attentional bias and improving search efficiency. Previous studies mostly used a single modality presentation of emotional stimuli to investigate the relationship between the two, but the findings are inconsistent. Existing studies have shown that the congruent emotion of audiovisual dual modality can be integrated into the perceptual stage, which is the same as the processing stage of IOR. Therefore, the present study adopted the cue-target paradigm and used audiovisual dual modality to present emotional stimuli to further investigate the interaction between emotion and IOR.
Experiment 1 was a three-factor within-subject design. We manipulated the presentation of cue validity (cued vs. uncued), target modalities (visual vs. audiovisual), and emotion type (negative vs. neutral). The task of the subjects was to identify the emotional stimuli of visual modality. Experiment 2 was similar to Experiment 1, but the emotional congruency was changed. The audiovisual dual modality presented incongruent emotional stimuli (visual negative face-auditory neutral sound; visual neutral face-auditory negative sound) to further investigate whether the impact of the audiovisual dual modality emotional stimulus on IOR was caused by the emotional stimulus of the auditory modality, that is, whether the emotional stimulus of the auditory modality was processed.
In Experiment 1, the responses in the cued condition were slower than those in the uncued condition, which suggested that IOR occurred. More importantly, the interaction between emotion type and cue validity in the audiovisual dual modality condition showed that congruent negative emotion produces a smaller IOR effect (11 ms) than neutral emotion (25 ms). At the same time, the audiovisual dual modality condition produced a smaller IOR effect (18 ms) than the visual single modality condition (40 ms). We also found a larger multisensory response enhancement effect in the congruent negative emotion than in the neutral emotion. In Experiment 2, the results showed that there was no interaction between emotion and IOR under the condition of audiovisual dual modality, and there was no significant difference in IOR effect between single modality and audiovisual dual modality. This indicated that the IOR effect was not influenced by the presence of incongruent emotion in the audiovisual dual modality. In summary, the present study showed that the IOR effect was influenced only when the audiovisual dual modality presented the same emotion.
Our findings revealed that IOR and audiovisual dual modality congruent emotion in the same processing stage had a mutual influence. Audiovisual dual modality congruent emotion weakened the IOR effect, and the differences between the negative emotion and the neutral emotion showed the adaptability of IOR. At the same time, this study further supports the perceptual inhibition theory of IOR.

Key words: inhibition of return, emotion, audiovisual integration

中图分类号:

B842

张明, 王婷婷, 吴晓刚, 张月娥, 王爱君. (2022). 面孔表情和声音情绪信息整合对返回抑制的影响. 心理学报, 54(4), 331-342.

ZHANG Ming, WANG Tingting, WU Xiaogang, ZHANG Yue’e, WANG Aijun. (2022). Effects of integration of facial expression and emotional voice on inhibition of return. Acta Psychologica Sinica, 54(4), 331-342.

图/表 5

图1 实验1流程图

图2 各个条件下的反应时(注：*p < 0.05, **p < 0.01)

图3 (a)各个条件下的IOR效应量(RT有效–RT无效), (b) 各个条件下的相对多感觉反应增强(rMRE) (注：*p < 0.05, **p < 0.01, ***p < 0.001)

图4 各个条件下的反应时(注：n.s. p > 0.05)

图5 (a) 各个条件下的IOR效应量(RT有效–RT无效), (b) 各个条件下的相对多感觉反应增强(rMRE) (注：n.s. p > 0.05)

参考文献 54

[1]	Anderson, A. K., Christoff, K., Panitz, D., de Rosa, E., & Gabrieli, J. D. E. (2003). Neural correlates of the automatic processing of threat facial signals. The Journal of Neuroscience, 23(13), 5627-5633. doi: 10.1523/JNEUROSCI.23-13-05627.2003 URL
[2]	Anderson, A. K., & Phelps, E. A. (2001). Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature, 411(6835), 305-309. doi: 10.1038/35077083 URL
[3]	Baijal, S., & Srinivasan, N. (2011). Emotional and hemispheric asymmetries in shifts of attention: An ERP study. Cognition and Emotion, 25(2), 280-294. doi: 10.1080/02699931.2010.492719 URL
[4]	Belin, P., Fillion-Bilodeau, S., & Gosselin, F. (2008). The Montreal Affective Voices: A validated set of nonverbal affect bursts for research on auditory affective processing. Behavior Research Methods, 40(2), 531-539. doi: 10.3758/BRM.40.2.531 URL
[5]	Berdica, E., Gerdes, A. B. M., & Alpers, G. W. (2017). A comprehensive look at phobic fear in inhibition of return: Phobia-related spiders as cues and targets. Journal of Behavior Therapy and Experimental Psychiatry, 54, 158-164. doi: S0005-7916(16)30032-5 pmid: 27517673
[6]	Chen, Q., Wei, P., & Zhou, X. (2006). Distinct neural correlates for resolving stroop conflict at inhibited and noninhibited locations in inhibition of return. Journal of Cognitive Neuroscience, 18(11), 1937-1946. pmid: 17069483
[7]	Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., & Lepore, F. (2008). Audio-visual integration of emotion expression. Brain Research, 1242, 126-135. doi: 10.1016/j.brainres.2008.04.023 pmid: 18495094
[8]	de Gelder, B., & Vroomen, J. (2000). The perception of emotions by ear and by eye. Cognition and Emotion, 14(3), 289-311. doi: 10.1080/026999300378824 URL
[9]	Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using GPower 3.1: Tests for correlation and regression analyses. Behavior Research Methods*, 41(4), 1149-1160. doi: 10.3758/BRM.41.4.1149 URL
[10]	Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). GPower 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods*, 39(2), 175-191. doi: 10.3758/BF03193146 URL
[11]	Fox, E., Russo, R., & Dutton, K. (2002). Attentional bias for threat: Evidence for delayed disengagement from emotional faces. Cognition and Emotion, 16(3), 355-379. doi: 10.1080/02699930143000527 URL
[12]	Guan, J., Li, W. R., & Zhao, X. D. (2018). The competition between inhibition of return and emotional attention bias: Evidence from eye movements. Journal of Psychological Science, 41(06), 1353-1358.
	[ 关荐, 李文瑞, 赵旭东. (2018). 返回抑制和情绪信息注意偏向的竞争:来自眼动的证据. 心理科学, 41(06), 1353-1358.]
[13]	Hopfinger, J. B., & Mangun, G. R. (2001). Tracking the influence of reflexive attention on sensory and cognitive processing. Cognitive, Affective, & Behavioral Neuroscience, 1(1), 56-65.
[14]	Jessen, S., & Kotz, S. A. (2011). The temporal dynamics of processing emotions from vocal, facial, and bodily expressions. NeuroImage, 58(2), 665-674. doi: 10.1016/j.neuroimage.2011.06.035 pmid: 21718792
[15]	Jia, L., Wang, J., Zhang, K., Ma, H., & Sun, H. J. (2019). Do emotional faces affect inhibition of return? An ERP study. Frontiers in Psychology, 10(721), 1-8. doi: 10.3389/fpsyg.2019.00001 URL
[16]	Klasen, M., Kreifelts, B., Chen, Y. H., Seubert, J., & Mathiak, K. (2014). Neural processing of emotion in multimodal settings. Frontiers in Human Neuroscience, 8(822), 1-4.
[17]	Klein, R. M. (2000). Inhibition of return. Trends in Cognitive Sciences, 4(4), 138-147. doi: 10.1016/S1364-6613(00)01452-2 URL
[18]	Kokinous, J., Kotz, S. A., Tavano, A., & Schröger, E. (2014). The role of emotion in dynamic audiovisual integration of faces and voices. Social Cognitive and Affective Neuroscience, 10(5), 713-720. doi: 10.1093/scan/nsu105 URL
[19]	Lang, P. J., Davis, M., & Öhman, A. (2000). Fear and anxiety: Animal models and human cognitive psychophysiology. Journal of Affective Disorders, 61(3), 137-159. pmid: 11163418
[20]	Lange, W. G., Heuer, K., Reinecke, A., Becker, E. S., & Rinck, M. (2008). Inhibition of return is unimpressed by emotional cues. Cognition and Emotion, 22(8), 1433-1456. doi: 10.1080/02699930701809451 URL
[21]	Li, P., Zhang, M. M., Li, S. X., Zhang, H.Y., & Luo, W. B. (2019). The integration of facial expression and vocal emotion and its brain mechanism. Advances in Psychological Science, 27(7), 1205-1214. doi: 10.3724/SP.J.1042.2019.01205 URL
	[ 李萍, 张明明, 李帅霞, 张火垠, 罗文波. (2019). 面孔表情和声音情绪信息整合加工的脑机制. 心理科学进展, 27(7), 1205-1214.]
[22]	Lupiáñez, J., Klein, R. M., & Bartolomeo, P. (2006). Inhibition of return: Twenty years after. Cognitive Neuropsychology, 23(7), 1003-1014. doi: 10.1080/02643290600588095 URL
[23]	Miller, J. (1986). Timecourse of coactivation in bimodal divided attention. Perception & Psychophysics, 40(5), 331-343. doi: 10.3758/BF03203025 URL
[24]	Mineka, S., & Öhman, A. (2002). Phobias and preparedness: The selective, automatic, and encapsulated nature of fear. Biological Psychiatry, 52(10), 927-937. pmid: 12437934
[25]	Müller, V. I., Habel, U., Derntl, B., Schneider, F., Zilles, K., Turetsky, B. I., & Eickhoff, S. B. (2011). Incongruence effects in crossmodal emotional integration. NeuroImage, 54(3), 2257-2266. doi: 10.1016/j.neuroimage.2010.10.047 URL
[26]	Okon-Singer, H., Henik, A., & Gabay, S. (2020). Increased inhibition following negative cues: A possible role for enhanced processing. Cortex, 122, 131-139. doi: S0010-9452(18)30418-0 pmid: 30638583
[27]	Pan, F., Wu, X., Zhang, L., & Ou, Y. (2017). Inhibition of return is modulated by negative stimuli: Evidence from subliminal perception. Frontiers in Psychology, 8(1012), 1-8.
[28]	Pérez-Dueñas, C., Acosta, A., & Lupiáñez, J. (2009). Attentional capture and trait anxiety: Evidence from inhibition of return. Journal of Anxiety Disorders, 23(6), 782-790. doi: 10.1016/j.janxdis.2009.03.002 pmid: 19380211
[29]	Pérez-Dueñas, C., Acosta, A., & Lupiáñez, J. (2014). Reduced habituation to angry faces: Increased attentional capture as to override inhibition of return. Psychological Research, 78(2), 196-208. doi: 10.1007/s00426-013-0493-9 pmid: 23689476
[30]	Posner, M., & Cohen, Y. (1984). Components of visual orienting. Attention and Performance X: Control of Language Processes, 32, 531-556.
[31]	Pourtois, G., de Gelder, B., Vroomen, J., Rossion, B., & Crommelinck, M. (2000). The time-course of intermodal binding between seeing and hearing affective information. NeuroReport, 11(6), 1329-1333. pmid: 10817616
[32]	Pourtois, G., Schettino, A., & Vuilleumier, P. (2013). Brain mechanisms for emotional influences on perception and attention: What is magic and what is not. Biological Psychology, 92(3), 492-512. doi: 10.1016/j.biopsycho.2012.02.007 pmid: 22373657
[33]	Prime, D. J., & Jolicœur, P. (2009). On the relationship between occipital cortex activity and inhibition of return. Psychophysiology, 46(6), 1278-1287. doi: 10.1111/psyp.2009.46.issue-6 URL
[34]	Prime, D. J., & Ward, L. M. (2006). Cortical expressions of inhibition of return. Brain Research, 1072(1), 161-174. doi: 10.1016/j.brainres.2005.11.081 URL
[35]	Redden, R. S., MacInnes, W. J., & Klein, R. M. (2021). Inhibition of return: An information processing theory of its natures and significance. Cortex, 135, 30-48. doi: 10.1016/j.cortex.2020.11.009 pmid: 33360759
[36]	Reuter-Lorenz, P. A., Jha, A. P., & Rosenquist, J. N. (1996). What is inhibited in inhibition of return. Journal of Experimental Psychology: Human Perception and Performance, 22(2), 367-378. doi: 10.1037/0096-1523.22.2.367 URL
[37]	Rutherford, H. J. V., & Raymond, J. E. (2010). Effects of spatial cues on locating emotional targets. Visual Cognition, 18(3), 389-412. doi: 10.1080/13506280902787043 URL
[38]	Silvert, L., & Funes, M. J. (2016). When do fearful faces override inhibition of return? Acta Psychologica, 163, 124-134. doi: 10.1016/j.actpsy.2015.11.002 URL
[39]	Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9(4), 255-266. doi: 10.1038/nrn2331 URL
[40]	Stoyanova, R. S., Pratt, J., & Anderson, A. K. (2007). Inhibition of return to social signals of fear. Emotion, 7(1), 49-56. pmid: 17352562
[41]	Talsma, D., & Woldorff, M. G. (2005). Selective attention and multisensory integration: Multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience, 17(7), 1098-1114. pmid: 16102239
[42]	Tang, X., Gao, Y., Yang, W., Ren, Y., Wu, J., Zhang, M., & Wu, Q. (2019). Bimodal-divided attention attenuates visually induced inhibition of return with audiovisual targets. Experimental Brain Research, 237(4), 1093-1107. doi: 10.1007/s00221-019-05488-0 URL
[43]	Taylor, T. L., & Klein, R. M. (1998). On the causes and effects of inhibition of return. Psychonomic Bulletin & Review, 5(4), 625-643. doi: 10.3758/BF03208839 URL
[44]	Taylor, T. L., & Therrien, M. E. (2008). Inhibition of return for the discrimination of faces. Perception & Psychophysics, 70(2), 279-290. doi: 10.3758/PP.70.2.279 URL
[45]	Tian, Y., & Yao, D. (2008). A study on the neural mechanism of inhibition of return by the event-related potential in the Go/Nogo task. Biological Psychology, 79(2), 171-178. doi: 10.1016/j.biopsycho.2008.04.006 pmid: 18524452
[46]	Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., Marcus, D. J., Westerlund, A., Casey, B. J., & Nelson, C. (2009). The NimStim set of facial expressions: Judgments from untrained research participants. Psychiatry Research, 168(3), 242-249. doi: 10.1016/j.psychres.2008.05.006 pmid: 19564050
[47]	Troiani, V., Price, E. T., & Schultz, R. T. (2014). Unseen fearful faces promote amygdala guidance of attention. Social Cognitive and Affective Neuroscience, 9(2), 133-140. doi: 10.1093/scan/nss116 URL
[48]	Ulrich, R., Miller, J., & Schröter, H. (2007). Testing the race model inequality: An algorithm and computer programs. Behavior Research Methods, 39(2), 291-302. pmid: 17695357
[49]	van der Burg, E., Talsma, D., Olivers, C. N. L., Hickey, C., & Theeuwes, J. (2011). Early multisensory interactions affect the competition among multiple visual objects. NeuroImage, 55(3), 1208-1218. doi: 10.1016/j.neuroimage.2010.12.068 URL
[50]	van der Stoep, N., van der Stigchel, S., Nijboer, T. C. W., & Spence, C. (2017). Visually induced inhibition of return affects the integration of auditory and visual information. Perception, 46(1), 6-17. doi: 10.1177/0301006616661934 pmid: 27484341
[51]	Vizueta, N., Patrick, C. J., Jiang, Y., Thomas, K. M., & He, S. (2012). Dispositional fear, negative affectivity, and neuroimaging response to visually suppressed emotional faces. NeuroImage, 59(1), 761-771. doi: 10.1016/j.neuroimage.2011.07.015 URL
[52]	Wang, J. X., Jia, L. P., Zhang, K., & Zhang, S. (2013). Influence of emotional faces for different location on inhibition of return. Journal of Psychological Science, 36(02), 335-339.
	[ 王敬欣, 贾丽萍, 张阔, 张赛. (2013). 不同位置的情绪面孔加工对返回抑制的影响. 心理科学, 36(02), 335-339.]
[53]	Wang, Z., Zhang, K., & Klein, R. M. (2010). Inhibition of return in static but not necessarily in dynamic search. Attention, Perception, & Psychophysics, 72(1), 76-85. doi: 10.3758/APP.72.1.76 URL
[54]	Zhao, X., Li, X., & Shi, W. (2017). Influence of inhibitory tagging (IT) on emotional and cognitive conflict processing: Evidence from event-related potentials. Neuroscience Letters, 657, 120-125. doi: 10.1016/j.neulet.2017.08.014 URL