ISSN 0439-755X
CN 11-1911/B

中国科学院心理研究所

• 研究报告 •

### 面孔表情和声音情绪信息整合对返回抑制的影响

1. 1苏州大学心理学系, 心理与行为科学研究中心, 苏州 215123
2日本国立岡山大学健康系统统合研究院, 冈山 700-8530
3苏州科技大学心理学系, 苏州 215009
• 收稿日期:2021-06-24 出版日期:2022-04-25 发布日期:2022-02-21
• 通讯作者: 张明,王爱君 E-mail:psyzm@suda.edu.cn;ajwang@suda.edu.cn
• 基金资助:
国家自然科学基金(31871092);国家自然科学基金(31700939);教育部人文社会科学基金(17YJC190024);日本学术振兴会(JSPS)基础研究项目(20K04381)

### Effects of integration of facial expression and emotional voice on inhibition of return

ZHANG Ming1,2(), WANG Tingting1, WU Xiaogang3, ZHANG Yue’e1, WANG Aijun1()

1. 1Department of Psychology, Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou 215000, China
2Graduate School of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama 700-8530, Japan
3Department of Psychology, Suzhou University of Science and Technology, Suzhou 215009, China
• Received:2021-06-24 Online:2022-04-25 Published:2022-02-21
• Contact: ZHANG Ming,WANG Aijun E-mail:psyzm@suda.edu.cn;ajwang@suda.edu.cn

Abstract:

Both inhibition of return (IOR) and emotion have the characteristics of attentional bias and improving search efficiency. Previous studies mostly used a single modality presentation of emotional stimuli to investigate the relationship between the two, but the findings are inconsistent. Existing studies have shown that the congruent emotion of audiovisual dual modality can be integrated into the perceptual stage, which is the same as the processing stage of IOR. Therefore, the present study adopted the cue-target paradigm and used audiovisual dual modality to present emotional stimuli to further investigate the interaction between emotion and IOR.
Experiment 1 was a three-factor within-subject design. We manipulated the presentation of cue validity (cued vs. uncued), target modalities (visual vs. audiovisual), and emotion type (negative vs. neutral). The task of the subjects was to identify the emotional stimuli of visual modality. Experiment 2 was similar to Experiment 1, but the emotional congruency was changed. The audiovisual dual modality presented incongruent emotional stimuli (visual negative face-auditory neutral sound; visual neutral face-auditory negative sound) to further investigate whether the impact of the audiovisual dual modality emotional stimulus on IOR was caused by the emotional stimulus of the auditory modality, that is, whether the emotional stimulus of the auditory modality was processed.
In Experiment 1, the responses in the cued condition were slower than those in the uncued condition, which suggested that IOR occurred. More importantly, the interaction between emotion type and cue validity in the audiovisual dual modality condition showed that congruent negative emotion produces a smaller IOR effect (11 ms) than neutral emotion (25 ms). At the same time, the audiovisual dual modality condition produced a smaller IOR effect (18 ms) than the visual single modality condition (40 ms). We also found a larger multisensory response enhancement effect in the congruent negative emotion than in the neutral emotion. In Experiment 2, the results showed that there was no interaction between emotion and IOR under the condition of audiovisual dual modality, and there was no significant difference in IOR effect between single modality and audiovisual dual modality. This indicated that the IOR effect was not influenced by the presence of incongruent emotion in the audiovisual dual modality. In summary, the present study showed that the IOR effect was influenced only when the audiovisual dual modality presented the same emotion.
Our findings revealed that IOR and audiovisual dual modality congruent emotion in the same processing stage had a mutual influence. Audiovisual dual modality congruent emotion weakened the IOR effect, and the differences between the negative emotion and the neutral emotion showed the adaptability of IOR. At the same time, this study further supports the perceptual inhibition theory of IOR.