关键词: Pip-and-Pop效应, 视听空间一致性, 视听时间一致性, 动态视觉搜索范式, 眼动技术

Abstract:

Previous studies usually used the dynamic visual search paradigm to explore the Pip-and-Pop effect, which suggests that the Pip-and-Pop effect is influenced by multiple factors. But the impact of temporal and spatial consistency of audiovisual stimuli on the Pip-and-Pop effect is still controversial. In the present study, we applied the eye movement technology to the dynamic visual search paradigm to systematically investigate the impact of temporal and spatial consistency of audiovisual stimuli on the Pip-and-Pop effect. Based on the results of previous studies, we expected that the spatial and temporal consistency of audiovisual modulated the Pip-and-Pop effect. The more consistent the spatial location of audiovisual stimuli are, the greater the Pip-and-Pop effect will be. The more consistent the temporal audiovisual stimuli are, the greater the Pip-and-Pop effect will be.

Experiment 1 was a 3 (set sizes: 36, 48, 60) × 4 (spatial consistency conditions: ipsilateral, contralateral, bilateral, no sound) within-subjects design, recruited 24 participants (5 males; age range: 19~28 years; mean age: 22.5 ± 2.3 years). The visual search displays consisted of 24, 36, or 48 red (13.9 cd/m²) or green (46.4 cd/m²) line segments (0.57°×0.17°) on a black (0.4 cd/m²) background. The auditory stimulus was 1000 Hz pure tone (65 dB, 60 ms, including 5 ms fade-in and 5 ms fade-out time), presented through speakers on the left and right sides behind the screen. Each trial started with a drift calibration point. The drift calibration point would not disappear until the participant gazed upon this point. This was followed by a central fixation point of 1000ms, and finally a search screen. The participants were required to find the target line segment and judge whether the target line segment was vertical or horizontal by pressing the key (Z key or M key) as quickly and accurately as possible. Experiment 2 was a 2 (set sizes: 36, 60) × 6 (temporal consistency conditions: -200 ms, -100 ms, 0 ms, 100 ms, 200 ms, no sound) within-subjects design, recruited 27 participants (5 males; age range: 18~25 years; mean age: 20.7 ± 2.4 years). These temporal consistency conditions represent the tone sounded before (-200, -100 ms), simultaneous with (0 ms), or after (100, 200 ms) the visual target event. The tone could also be absent (no sound). The experimental materials and procedures of Experiment 2 were identical to those in Experiment1.

Regarding the results in Experiment 1, compared with the baseline condition (no sound), the search response time was the shortest under the condition of ipsilateral, the mean fixation number was the least, and the mean saccade amplitude was the smallest, indicating that the search efficiency was the highest in the ipsilateral condition, the Pip-and-Pop effect was the largest in the ipsilateral condition. The bilateral condition was the second. The contralateral condition showed no significant difference in the above indicators compared with the no sound condition, indicating that no Pip-and-Pop effect was found in the contralateral condition. The results showed that the more consistent the spatial location of audiovisual stimuli were, the greater the Pip-and-Pop effect was. In Experiment 2, compared with the no sound condition, we found that the response time of the search was significantly shorter, the mean fixation number in search decreased, the mean saccade amplitude increased, and the mean fixation duration became longer under the conditions of 0 ms, -100 ms and 100 ms. Compared with the no sound condition, conditions -200 ms and 200 ms showed no significant difference in the above indicators. This indicated that the conditions of 0 ms, -100 ms and 100 ms produced the Pip-and-Pop effect, and the Pip-and-Pop effect was the largest under the condition of 0ms, while the Pip-and-Pop effect did not find under the condition of -200 ms and 200 ms. The results showed that the more consistent the temporal audiovisual stimuli were, the greater the Pip-and-Pop effect was.

In summary, the results showed that the more consistent the audiovisual stimuli were in space and time, the larger the Pip-and-Pop effect was. Therefore, the findings in the present study suggest that the temporal and spatial consistency of audiovisual stimuli modulates the Pip-and-Pop effect. The results provide evidence that the reason for the Pip-and-Pop effect is multisensory integration.

Key words: Pip-and-Pop effect, audiovisual spatial consistency, audiovisual temporal consistency, dynamic visual search, eye movement technique