Abstract:

Signals from different sensory channels can be integrated and processed in the brain. Compared with stimuli from a single sensory modality, individual responses are faster for multisensory target signals; this is defined as the redundant signal effect. One of the main theories explaining the redundant signal effect is the co-activation model. According to the co-activation model, the signal input through multiple sensory channels is integrated into specialized brain regions, such as the intraparietal sulcus, superior temporal sulcus, and prefrontal lobe regions. Related regions collaborate through neural oscillations. The strength of the integrated signal is stronger relative to the signal in single channel, which can trigger the reaction more quickly. However, the phase of cognitive processing at which the integration of the multisensory signals occurs is not confirmed.
Task switching is an important paradigm for studying cognitive control. Participants are slower when completing the switch task than when completing the repetitive task. At the same time, the error rate is greater because of the switching cost. When a shift occurs between different modal stimuli, there is also a cost in relation to the sensory channel changing—the modality shift effect. That is, owing to the modality shifting between successive sensory channels, individuals have a longer reaction time. Nevertheless, few studies have thoroughly explored the neural mechanisms of the modality shift effect. When the task is switched between different modalities, modal and task are all shifted. The cost of task switching associated with the cross-modal is more than the single modality switching cost or the task switching cost but less than the sum of the two kinds of loss. This provides evidence for the hypothesis that the switching cost associated with different sensory channels is derived from inertia and interference of the task set. In addition, the modality shift effect also exists in the paradigm of the redundant signal effect. Due to the shift of the sensory channel, the reaction time of the participants would be slower for the single sensory channel. Therefore, the existence of modality shift loss provides a different approach to the two classical theoretical models to explain the source of the redundant signal effect. Moreover, when modality switching occurs between single-modal and multi-sensory signals, the modality switching cost will decrease or even disappear, which is due to the multisensory integration offsetting a part of the loss. This supports the co-activation model.
Finally, if the stimulation in the task switch paradigm is present in a multisensory channel, there is simultaneous task shifting and multisensory integration. The redundant signal effect may offset the switching cost, affecting the synchronous changes in neural oscillations and the intensity of activation in the relevant brain regions. However, it is unclear how multisensory integration affects the neural processing of the task switch, which would require more brain and neural evidence to demonstrate. Further studies could try to solve this problem by combining the multisensory integration research paradigm with the classic task-switching paradigm, exploring whether task-switching and cross-modal shifting share a common cognitive processing center. Furthermore, future studies could help to determine the processing mechanism of the cross-modal shift, source of loss, and processing phase of multisensory integration.

Key words: multisensory integration, redundant signal effect, modality shift effect, task switching