Abstract:

The integration of various emotional information from different modalities (e.g., face and voice) plays an important role in our interpersonal communication. In order to understand its brain mechanism, more and more researchers found that the interaction between facial expression and vocal emotional information begins in the early stage of perception, and the integration of emotional information content occurs in the late decision-making stage. In the early stage, the primary sensory cortex is responsible for encoding information; while in the late stage, the amygdala, temporal lobe and other advanced brain regions are responsible for cognitive evaluation. In addition, the functional coupling of oscillation activities on multiple frequency bands facilitates the integration of emotional information cross channels. Future research needs to explore whether facial expression and vocal emotional information integration is associated with emotional conflict, and whether inconsistent emotional information has advantages. Lastly, we should find out how the neural oscillations of different frequency bands promotes the integration of facial expression and vocal emotional information, so as to further understand its dynamic basis.

Key words: emotion, integration effect, ERP, neural oscillation, fMRI