不同SOA下视觉返回抑制对视听觉整合的调节作用

doi:10.3724/SP.J.1041.2019.00759

摘要/Abstract

摘要：

基于外源性线索-靶子范式, 采用2(线索-靶子间隔时间, stimulus onset asynchronies, SOA：400~600 ms、1000~1200 ms) × 3(目标刺激类型：视觉、听觉、视听觉) × 2(线索有效性：有效线索、无效线索)的被试内实验设计, 要求被试对目标刺激完成检测任务, 以考察视觉线索诱发的返回抑制(inhibition of return, IOR)对视听觉整合的调节作用, 从而为感知觉敏感度、空间不确定性及感觉通道间信号强度差异假说提供实验证据。结果发现：(1) 随SOA增长, 视觉IOR效应显著降低, 视听觉整合效应显著增强; (2) 短SOA (400~600 ms)时, 有效线索位置上的视听觉整合效应显著小于无效线索位置, 但长SOA (1000~1200 ms)时, 有效与无效线索位置上的视听觉整合效应并无显著差异。结果表明, 在不同SOA条件下, 视觉IOR对视听觉整合的调节作用产生变化, 当前结果支持感觉通道间信号强度差异假说。

关键词: 外源性空间注意, 视听觉整合, 返回抑制, 竞争模型, 线索-靶子范式

Abstract:

Both audiovisual integration and inhibition of return (IOR) can facilitate the processing of sensory information, such as enhancing the perceptual processing. Previous studies found that IOR decreased the audiovisual integration at previously attended locations. Several hypotheses have been put forward to explain the effect of IOR on audiovisual integration: perceptual sensitivity, spatial uncertainty, and differences in unimodal signal strength. In present study, we used cue-target paradigm and manipulated the SOA conditions (400~600 ms vs. 1000~1200 ms) to investigate how audiovisual integration would be modulated by IOR induced by visual exogenous spatial cues.
The current study was a 2 (SOA conditions: 400~600 ms, 1000~1200 ms) × 3 (target modalities: visual, auditory, audiovisual) × 2 (cue validities: cued, uncued) factorial design. Twenty-seven undergraduate students were recruited as paid volunteers from a university. The visual (V) target was a red and white block (1°×1°). The auditory (A) target (duration of 100 ms) was a 1000 Hz sinusoidal tone presented by speakers. The audiovisual (AV) target was composed by the simultaneous presentation of both the visual and the auditory stimuli. At the beginning of each trial, the fixation stimulus was presented for 800~1000 ms in the center of the display. Following the fixation stimulus, a visual white square served as a exogenous cue was presented for 50 ms at the left or right location randomly. Then, the fixation stimulus was randomly presented for 150~250/450~550 ms, which was followed by a central cue with a delay of 50 ms. Before the target (100 ms) occurrence, the fixation stimulus randomly appeared again for 150~250/450~550 ms. Thus, the SOA between the peripheral cue and the target was completed in 400~600/1000~1200 ms. The target (A, V, or AV) randomly appeared (6/7) for 100 ms in the left or right locations, or no stimulus appeared (1/7). During the experiment, participants were instructed to respond to the target stimulus at any possible locations by pressing a response button as quickly and accurately as possible.
The results showed that the responses to AV targets were faster than V or A targets, indicating the appearance of the bimodal advancement effect. A smaller magnitude of audiovisual IOR as compared to visual IOR was found whether it’s in short or long SOA conditions. In addition, visual IOR effect was significantly reduced under the long SOA condition compared with the short SOA condition while the audiovisual integration effect increased by SOA. The results of the relative multisensory response enhancement (rMRE), race model (probability difference) and positive area under the curve (pAUC) showed that audiovisual integration decreased at cued compared to uncued locations under the short SOA condition but not the long SOA condition.
Based on the aforementioned findings, it is assumed that visual IOR decreased the audiovisual integration in the short SOA, and audiovisual integration would be modulated by different SOA conditions. The current result supported the hypothesis of differences in unimodal signal strength.

Key words: exogenous spatial attention, audiovisual integration, inhibition of return, race model, cue-target paradigm

中图分类号:

B842

彭姓, 常若松, 李奇, 王爱君, 唐晓雨. (2019). 不同SOA下视觉返回抑制对视听觉整合的调节作用. 心理学报, 51(7), 759-771.

PENG Xing, CHANG Ruosong, LI Qi, WANG Aijun, TANG Xiaoyu. (2019). Visually induced inhibition of return affects the audiovisual integration under different SOA conditions. Acta Psychologica Sinica, 51(7), 759-771.

图/表 6

图1 实验示意图注：图左为刺激呈现位置的示意图, 图右为单个试次的流程图。图右中视觉线索(白色正方形)呈现在左侧, 目标(视听觉)也呈现在左侧(即, 有效线索位置), 要求被试对目标刺激进行既快又准的检测反应。其中, 目标刺激(V/A/AV)分别代表视觉(visual)、听觉(auditory)和视听觉(audiovisual)通道目标。ISI是指刺激间时间间隔(inter-stimulus interval)。ITI是指试次间的时间间隔(inter-trial interval)。SOA的计算是由视觉线索时间(50 ms), 2个ISI时间(150~250 ms/450~550 ms)以及视觉中心线索时间(50 ms)相加而得, 因此SOA为400~600 ms/1000~1200 ms。

图2 不同条件下的平均反应时和IOR效应注：图(a)中的反应时结果为平均中位数。图(b)中IOR效应由有效线索位置上目标的平均反应时减去无效线索位置上目标的平均反应时。* p < 0.05; *** p < 0.001。

表1 不同SOA条件下Cueing effect、rMRE、pAUC结果对比

条件	M	95% CI		t	p
条件	M	下限	上限	t	p
短SOA
Cueing effect (ms)
V	38.77	31.54	46.00	11.09	0.000
A	1.83	-1.43	5.10	1.16	0.257
AV	16.00	11.55	20.45	7.43	0.000
Cueing effect对比(ms)
AV vs. V	-22.77	-30.12	-15.43	-6.41	0.000
V vs. A	36.94	28.53	45.34	9.09	0.000
AV vs. A	14.17	9.06	19.28	5.74	0.000
rMRE对比(%)
无效vs.有效	3.20	1.53	4.87	3.97	0.001
pAUC对比(ms)
无效vs.有效	-4.30	-6.64	-1.96	-3.80	0.001
长SOA
Cueing effect (ms)
V	26.98	19.42	34.54	7.38	0.000
A	5.88	0.51	11.24	2.26	0.033
AV	10.85	5.88	15.83	4.51	0.000
Cueing effect对比(ms)
AV vs. V	-16.13	-26.51	-5.74	-3.21	0.004
V vs. A	21.10	14.24	27.97	6.36	0.000
AV vs. A	4.98	-2.88	12.84	1.31	0.203
rMRE对比(%)
无效 vs. 有效	0.4	-1.78	2.59	0.38	0.706
pAUC对比(ms)
无效 vs. 有效	2.14	-1.12	5.40	1.36	0.188

图4 不同SOA条件下不同线索有效性的rMRE/pAUC 注：* p < 0.05; ** p < 0.01。

图3 不同SOA条件下不同线索有效性的反应时竞争模型分析注：图(c)中加粗横线表示显著违反竞争模型(实际AV累积概率(CPAV)显著大于竞争模型预测累积概率(CPRace model))的时间窗口, 红色虚线代表有效线索位置, 黑色实线代表无效线索位置。*代表峰值(最大概率值)出现的时间。

图5 不同SOA条件下不同线索有效性的视、听觉反应时之差注：* p < 0.05。

参考文献 55

[1]	Agusti A. I., Satorres E., Pitarque A., & Meléndez J. C . ( 2017). Effects of SOA and age on the inhibition of return in a localization task. Current Psychology, 1-6.
[2]	Calvert G. A., Spence C.& Stein, B. E.. ,( 2004). The handbook of multisensory processes. MIT Press.
[3]	Carrasco M. , (2011). Visual attention: The past 25 years. Vision Research, 51( 13), 1484-1525. doi: 10.1016/j.visres.2011.04.012 URL
[4]	Chica A. B., Bartolomeo P., & Lupiáñez J . ( 2013). Two cognitive and neural systems for endogenous and exogenous spatial attention. Behavioural Brain Research, 237( 1), 107-123. doi: 10.1016/j.bbr.2012.09.027 URL
[5]	Chica A. B., Lupianez J., & Bartolomeo P . ( 2006). Dissociating inhibition of return from endogenous orienting of spatial attention: Evidence from detection and discrimination tasks. Cognitive Neuropsychology, 23( 7), 1015-1034. doi: 10.1080/02643290600588277 URL
[6]	Cohen J. , ( 1988). Statistical power analysis for the behavioral sciences (2nd Edition). . Erlbaum Associates.
[7]	Giard M.H., & Peronnet F. , ( 1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11( 5), 473-490. doi: 10.1162/089892999563544 URL
[8]	Hershenson M. , ( 1962). Reaction time as a measure of intersensory facilitation. Journal of Experimental Psychology, 63( 3), 289-293. doi: 10.1037/h0039516 URL
[9]	Klein R. , ( 1988). Inhibitory tagging system facilitates visual search. Nature, 334( 6181), 430-431.
[10]	Klein R. . ( 2000). Inhibition of return. Trends in Cognitive Sciences, 4( 4), 138-147. doi: 10.1016/S1364-6613(00)01452-2 URL
[11]	Koningsbruggen M. G., Gabay S., Sapir A., Henik A., & Rafal R. D . ( 2010). Hemispheric asymmetry in the remapping and maintenance of visual saliency maps: A TMS study. Journal of Cognitive Neuroscience, 22( 8), 1730-1738. doi: 10.1162/jocn.2009.21356 URL
[12]	Laurienti P. J., Burdette J. H., Maldjian J. A., & Wallace M. T . ( 2006). Enhanced multisensory integration in older adults. Neurobiology of Aging, 27( 8), 1155-1163. doi: 10.1016/j.neurobiolaging.2005.05.024 URL
[13]	Lippert M., Logothetis N. K., & Kayser C . ( 2007). Improvement of visual contrast detection by a simultaneous sound. Brain Research, 1173( 1173), 102-109. doi: 10.1016/j.brainres.2007.07.050 URL
[14]	Liu Q . ( 2010). The research on brain mechanism of the multisenory integration (Unpublished doctorial dissertation). Southwest University, China.
	[ 刘强 . ( 2010). 多感觉整合脑机制研究(博士学位论文). 西南大学.]
[15]	Lupiáñez J., Milán E. G., Tornay F. J., Madrid E., & Tudela P . ( 1997). Does IOR occur in discrimination tasks? Yes, it does, but later. Perception & Psychophysics, 59( 8), 1241-1254.
[16]	Lupiáñez J., Ruz M., Funes M. J., & Milliken B . ( 2007). The manifestation of attentional capture: Facilitation or IOR depending on task demands. Psychological Research, 71( 1), 77-91. doi: 10.1007/s00426-005-0037-z URL
[17]	Macaluso E., Noppeney U., Talsma D., Vercillo T., Hartcher-O’Brien J., & Adam R . ( 2016). The curious Incident of attention in multisensory integration: Bottom- up vs. top-down. Multisensory Research, 29( 6-7), 557-583. doi: 10.1163/22134808-00002528 URL
[18]	Martínarévalo E., Chica A. B., & Lupiáñez J . ( 2015). No single electrophysiological marker for facilitation and inhibition of return: A review. Behavioural Brain Research, 300, 1-10.
[19]	McDonald J. J., Tedersälejärvi W. A., Russo F. D., & Hillyard S. A . ( 2005). Neural basis of auditory-induced shifts in visual time-order perception. Nature Neuroscience, 8( 9), 1197-1202.
[20]	Miller J. , ( 1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14( 2), 247-279. doi: 10.1016/0010-0285(82)90010-X URL
[21]	Miller J. ( 1986). Time course of coactivation in bimodal divided attention. Perception & Psychophysics, 40( 5), 331-343.
[22]	Noesselt T., Tyll S., Boehler C. N., Budinger E., Heinze H. J., & Driver J . ( 2010). Sound-induced enhancement of low-intensity vision: Multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. Journal of Neuroscience, 30( 41), 13609-13623. doi: 10.1523/JNEUROSCI.4524-09.2010 URL
[23]	Otto T. U., Dassy B., & Mamassian P . ( 2013). Principles of multisensory behavior. Journal of Neuroscience, 33( 17), 7463-7474. doi: 10.1523/JNEUROSCI.4678-12.2013 URL
[24]	Posner M.I., & Cohen Y. , ( 1984). Components of visual orienting. Attention and Performance X: Control of Language Processes, 32, 531-556.
[25]	Posner M. I., Rafal R. D., Choate L. S., & Vaughan J . ( 1985). Inhibition of return: Neural basis and function. Cognitive Neuropsychology, 2( 3), 211-228. doi: 10.1080/02643298508252866 URL
[26]	Pratt J., & Fischer M.H . ( 2002). Examining the role of the fixation cue in inhibition of return. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimentale, 56( 4), 294-301. doi: 10.1037/h0087405 URL
[27]	Prime D. J., Visser T. A. W., & Ward L. M . ( 2006). Reorienting attention and inhibition of return. Perception & Psychophysics, 68( 8), 1310-1323.
[28]	Prime D.J., & Ward L.M . ( 2006). Cortical expressions of inhibition of return. Brain Research, 1072( 1), 161-174. doi: 10.1016/j.brainres.2005.11.081 URL
[29]	Raab D.H . ( 1962). Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences, 24( 5), 574-590. doi: 10.1111/j.2164-0947.1962.tb01433.x URL
[30]	Reuter-Lorenz P. A., Jha A. P., & Rosenquist J. N . ( 1996). What is inhibited in inhibition of return? Journal of Experimental Psychology: Human Perception and Performance, 22( 2), 367-378. doi: 10.1037/0096-1523.22.2.367 URL
[31]	Schmitt M., Postma A., & De H. E . ( 2000). Interactions between exogenous auditory and visual spatial attention. The Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology, 53( 1), 105-130. doi: 10.1080/713755882 URL
[32]	Senkowski D., Saint-Amour D., Höfle M., & Foxe J. J . ( 2011). Multisensory interactions in early evoked brain activity follow the principle of inverse effectiveness. Neuroimage, 56( 4), 2200-2208. doi: 10.1016/j.neuroimage.2011.03.075 URL
[33]	Slagter H. A., Prinssen S., Reteig L. C., & Mazaheri A . ( 2016). Facilitation and inhibition in attention: Functional dissociation of pre-stimulus alpha activity, P1, and N1 components. Neuroimage, 125( 6), 25-35. doi: 10.1016/j.neuroimage.2015.09.058 URL
[34]	Spence C. , ( 2010). Crossmodal spatial attention. Annals of the New York Academy of Sciences, 1191( 1), 182-200. doi: 10.1111/nyas.2010.1191.issue-1 URL
[35]	Spence C., & Driver J. , ( 2004). Crossmodal space and crossmodal attention. Politics.
[36]	Spence C., Lloyd D., Mcglone F., Nicholls M. E. R., & Driver J . ( 2000). Inhibition of return is supramodal: A demonstration between all possible pairings of vision, touch, and audition. Experimental Brain Research, 134( 1), 42-48. doi: 10.1007/s002210000442 URL
[37]	Stein B. E., London N., Wilkinson L. K., & Price D. D . ( 1996). Enhancement of perceived visual intensity by auditory stimuli: a psychophysical analysis. Journal of Cognitive Neuroscience, 8( 6), 497-506. doi: 10.1162/jocn.1996.8.6.497 URL
[38]	Stein B.E., & Meredith M.A . ( 1993). The merging of the senses. Journal of Cognitive Neuroscience, 5( 3), 373-374. doi: 10.1162/jocn.1993.5.3.373 URL
[39]	Stein B.E., & Stanford T.R . ( 2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9( 4), 255-266.
[40]	Talsma D., Doty T. J., & Woldorff M. G . ( 2007). Selective attention and audiovisual integration: Is attending to both modalities a prerequisite for early integration? Cerebral Cortex, 17( 3), 679-690.
[41]	Talsma D., Senkowski D., Soto-Faraco S., & Woldorff M. G . ( 2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14( 9), 400-410. doi: 10.1016/j.tics.2010.06.008 URL
[42]	Talsma D., & Woldorff M.G . ( 2005). Selective attention and multisensory integration: Multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience, 17( 7), 1098-1114. doi: 10.1162/0898929054475172 URL
[43]	Tang X. Y., Wu J. L., & Shen Y . ( 2016). The interactions of multisensory integration with endogenous and exogenous attention. Neuroscience & Biobehavioral Reviews, 61, 208-224.
[44]	Tang X. Y., Gao Y. L., Yang W. P., Ren Y. N., Wu J. L., Zhang M., Wu Q .(in press). Bimodal-divided attention attenuates visually induced inhibition of return with audiovisual targets. Experimental Brain Research. First online 15 Feb. 2019,
[45]	Tassinari G., Aglioti S., Chelazzi L., Peru A., & Berlucchi G . ( 1994). Do peripheral non-informative cues induce early facilitation of target detection? Vision Research, 34( 2), 179-189. doi: 10.1016/0042-6989(94)90330-1 URL
[46]	Ulrich R., Miller J., & Schröter H . ( 2007). Testing the race model inequality: An algorithm and computer programs. Behavior Research Methods, 39( 2), 291-302. doi: 10.3758/BF03193160 URL
[47]	van der Burg E., Olivers C. N. L., Bronkhorst A. W., & Theeuwes J . ( 2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34( 5), 1053-1065. doi: 10.1037/0096-1523.34.5.1053 URL
[48]	van der Burg E., Talsma D., Olivers C. N. L., Hickey C., & Theeuwes J . ( 2011). Early multisensory interactions affect the competition among multiple visual objects. Neuroimage, 55( 3), 1208-1218. doi: 10.1016/j.neuroimage.2010.12.068 URL
[49]	van der Stoep N., van der Stigchel S& Nijboer, T. C. W. ., ( 2015). Erratum to: Exogenous spatial attention decreases audiovisual integration. Attention Perception & Psychophysics, 77( 1), 464-482.
[50]	van der Stoep N., van der Stigchel S., Nijboer T. C., & Spence C . ( 2016). Visually induced inhibition of return affects the integration of auditory and visual information. Perception, 46( 1), 6-17.
[51]	Ward L. M., Mcdonald J. J., & Lin D . ( 2000). On asymmetries in cross-modal spatial attention orienting. Percept Psychophys, 62( 6), 1258-1264. doi: 10.3758/BF03212127 URL
[52]	Wascher E., & Tipper S.P . ( 2004). Revealing effects of noninformative spatial cues: an EEG study of inhibition of return. Psychophysiology, 41( 5), 716-728. doi: 10.1111/psyp.2004.41.issue-5 URL
[53]	WEN Z. L., FAN X. T., YE B. J., & CHEN Y. S . ( 2016). Characteristics of an effect size and appropriateness of mediation effect size measures revisited. Acta Psychologica Sinica, 48( 4), 435-443.
	[ 温忠麟, 范息涛, 叶宝娟, 陈宇帅 . ( 2016). 从效应量应有的性质看中介效应量的合理性. 心理学报, 48( 4), 435-443.]
[54]	Yang W. P., Chu B. Q., Yang J. J., Yu Y. H., Wu J. L., & Yu S. Y . ( 2014). Elevated audiovisual temporal interaction in patients with migraine without aura. The Journal of Headache and Pain, 15( 1), 44.
[55]	Yang Z., & Mayer A.R . ( 2014). An event-related FMRI study of exogenous orienting across vision and audition. Human Brain Mapping, 35( 3), 964-974. doi: 10.1002/hbm.22227 URL