内源性空间线索有效性对视听觉整合的影响

doi:10.3724/SP.J.1041.2020.00835

心理学报 ›› 2020, Vol. 52 ›› Issue (7): 835-846.doi: 10.3724/SP.J.1041.2020.00835

内源性空间线索有效性对视听觉整合的影响

唐晓雨¹(), 吴英楠¹, 彭姓², 王爱君³(), 李奇⁴

¹ 辽宁师范大学心理学院, 辽宁省儿童青少年健康人格评定与培养协同创新中心, 大连 116029
² 中国民用航空飞行学院飞行技术学院, 广汉 618307
³ 苏州大学心理学系, 心理与行为科学研究中心, 苏州 215123
⁴ 长春理工大学计算机科学技术学院, 长春 130022

收稿日期:2019-11-15 发布日期:2020-05-25 出版日期:2020-07-25
通讯作者: 唐晓雨,王爱君 E-mail:2006@163.com;ajwang@suda.edu.cn
基金资助:
* 国家自然科学基金项目(31600882);国家自然科学基金项目(31700939);国家自然科学基金项目(61773076);辽宁省教育厅高水平创新团队国(境)外培养项目(2018LNGXGJWPY- YB015)

The influence of endogenous spatial cue validity on audiovisual integration

TANG Xiaoyu¹(), WU Yingnan¹, PENG Xing², WANG Aijun³(), LI Qi⁴

¹ School of Psychology, Liaoning Collaborative Innovation Center of Children and Adolescents Healthy Personality Assessment and Cultivation, Liaoning Normal University, Dalian 116029, China
² Flight Technology College, Civil Aviation Flight University of China, Guanghan 618307, China
³ Department of Psychology, Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou 215123, China
⁴ School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, China

Received:2019-11-15 Online:2020-05-25 Published:2020-07-25
Contact: TANG Xiaoyu,WANG Aijun E-mail:2006@163.com;ajwang@suda.edu.cn

摘要/Abstract

摘要：

采用内源性线索-靶子范式, 操纵线索类型(有效线索、无效线索)和靶刺激通道类型(视觉刺激、听觉刺激、视听觉刺激)两个自变量, 通过两个实验, 分别设置50%和80%两种内源性空间线索有效性来考察不同空间线索有效性条件下内源性空间注意对视听觉整合的影响。结果发现, 当线索有效性为50%时(实验1), 有效线索位置和无效线索位置的视听觉整合效应没有显著差异; 当线索有效性为80%时(实验2), 有效线索位置的视听觉整合效应显著大于无效线索位置的视听觉整合效应。结果表明, 线索有效性不同时, 内源性空间注意对视听觉整合产生了不同的影响, 高线索有效性条件下内源性空间注意能够促进视听觉整合效应。

关键词: 线索有效性, 视听觉整合, 内源性空间注意, 线索-靶子范式, 竞争模型

Abstract:

Previous studies usually used the instruction to control the attention of the participants to different spatial locations to investigate the influence of endogenous spatial attention on the audiovisual integration, which found that endogenous spatial attention enhanced audiovisual integration. However, in reality, it is difficult to assign attention in an all-or-none manner. In present study, we used endogenous spatial cue-target paradigm to investigate the effect of endogenous spatial cue validity on audiovisual integration.
The current study was a 2 (cue validities: valid, invalid) × 3 (target modalities: visual, auditory, audiovisual) factorial design. A total of 62 undergraduate students were recruited as paid volunteers. Experiment 1 recruited 35 participants (5 males; age range: 18~26 years; mean age: 21.26 ± 1.77 years). Experiment 2 recruited 27 participants (6 males; age range: 19~25 years; mean age: 22.19 ± 1.75 years). The visual (V) target was a white pentagonal asterisk block (2°×2°). The auditory (A) target was a 1600 Hz sinusoidal tone presented by speakers. The audiovisual (AV) target was composed by the simultaneous presentation of both the visual and the auditory stimuli. At the beginning of each trial, the fixation stimulus was presented for 600~800 ms in the center of the monitor. Following the fixation stimulus, the cue was presented for 200 ms, which could predict (50% or 80%) the location of the target. Before the target that appeared for 100 ms, the inter stimulus interval (ISI) lasted for 700 ms. The target (A, V, or AV) randomly appeared for 100 ms in the left or right locations. At last, the fixation stimuli appeared for 600 ms to wait for the correspondence responses to targets. During the experiment, the participants were asked to locate targets by pressing buttons (F/J) as quickly and accurately as possible.
The results showed that the responses to AV targets were faster than V or A targets, indicating the appearance of the bimodal advancement effect in both experiments. Each target modality showed significant cueing effect. And the cueing effect in experiment 2 was significantly larger than the cueing effect in experiment 1. In addition, in experiment 1, relative multisensory response enhancement (rMRE), race model (probability difference) showed no significant difference at valid compared to invalid locations. However, in experiment 2, rMRE and race model (probability difference) increased at valid compared to invalid locations. The results suggested that endogenous spatial attention enhanced audiovisual integration in high spatial validity condition.
In summary, Endogenous spatial attention had different effects on the audiovisual integration under different spatial cue validity. High spatial cue validity enhanced audiovisual integration. The current study provides direct behavioral evidence for endogenous spatial attention to enhance audiovisual integration.

Key words: cue validity, audiovisual integration, endogenous spatial attention, cue-target paradigm, race model

中图分类号:

B842

唐晓雨, 吴英楠, 彭姓, 王爱君, 李奇. (2020). 内源性空间线索有效性对视听觉整合的影响. 心理学报, 52(7), 835-846.

TANG Xiaoyu, WU Yingnan, PENG Xing, WANG Aijun, LI Qi. (2020). The influence of endogenous spatial cue validity on audiovisual integration. Acta Psychologica Sinica, 52(7), 835-846.

图/表 7

图1 实验刺激示例图和实验流程图 (a)为靶刺激呈现位置的示意图; 图(b)为单个试次的流程图。图(b)中视觉线索(中央箭头)指向左侧, 靶刺激(视听觉)呈现在左侧(即, 有效线索位置), 要求被试对靶刺激进行既快又准的定位反应。其中, 靶刺激(V/A/AV)分别代表视觉(visual)、听觉(auditory)和视听觉(audiovisual)通道目标。ISI是指刺激间的时间间隔(inter-stimulus interval)。ITI是指试次间的时间间隔(inter-trial interval)。

表1 每个实验中不同条件下的反应时(RT/ms)、正确率(ACC/%)(M ± SD)

靶刺激通道类型	线索类型	实验1		实验2
靶刺激通道类型	线索类型	RT (ms)	ACC (%)	RT (ms)	ACC (%)
视听觉	有效线索	302 ± 34	99.2 ± 1.2	289 ± 42	98.7 ± 1.2
	无效线索	312 ± 36	99.2 ± 1.2	333 ± 51	95.8 ± 4.8
听觉	有效线索	341 ± 39	97.1 ± 2.8	318 ± 51	97.1 ± 2.6
	无效线索	353 ± 42	96.8 ± 2.7	363 ± 56	92.7 ± 4.5
视觉	有效线索	334 ± 42	98.0 ± 2.3	324 ± 41	97.5 ± 2.3
	无效线索	345 ± 47	97.7 ± 2.9	362 ± 46	95.4 ± 4.7

图2 实验1中不同条件下的反应时和relative MRE (rMRE) 注：图(a)为不同条件下的反应时, 反应时结果为平均中位数; 图(b)为不同条件下的rMRE。*** p < 0.001

图3 实验1中不同线索类型的反应竞争模型分析和违反竞争模型的时间窗口注：图(a)为不同线索类型的反应竞争模型分析, 图(a)中横轴表示显著违反竞争模型(实际AV累积概率(CPAV)显著大于竞争模型预测累积概率(CPRace model))的时间窗口; 图(b)为不同线索类型下违反竞争模型的时间窗口, 图(b)中*代表峰值, 箭头分别代表违反竞争模型时间窗口的起始时间、峰值发生时间。

图4 实验2中不同条件下的反应时和relative MRE (rMRE) 注：图(a)为不同条件下的反应时, 反应时结果为平均中位数; 图(b)为不同条件下的rMRE。*** p < 0.001, * p < 0.05

表2 不同靶刺激通道类型下的Cueing effect结果对比

实验	条件	M	95% CI		t	p
实验	条件	M	下限	上限	t	p
实验1	Cueing effect
	A	12.23	5.97	18.49	3.97		< 0.001
	AV	9.44	5.48	13.41	4.84		< 0.001
	V	10.23	5.62	14.84	4.51		< 0.001
	Cueing effect对比
	A vs. AV	2.79	-1.31	6.88	1.38		0.176
	A vs. V	2.00	-3.28	7.28	0.77		0.447
	AV vs. V	-0.78	-4.37	2.80	-0.45		0.659
实验2	Cueing effect
	A	45.04	29.77	60.30	6.06		< 0.001
	AV	43.54	29.75	57.32	6.49		< 0.001
	V	38.70	26.00	51.41	6.26		< 0.001
	Cueing effect对比
	A vs. AV	1.50	-3.14	6.14	0.67		0.512
	A vs. V	6.33	0.21	12.46	2.13		0.043
	AV vs. V	4.83	-0.88	10.54	1.74		0.094

图5 实验2中不同线索类型的反应竞争模型分析和违反竞争模型的时间窗口注：图(a)为不同线索类型条件下的反应竞争模型分析, 图(a)中横轴表示显著违反竞争模型(实际AV累积概率(CPAV)显著大于竞争模型预测累积概率(CPRace model))的时间窗口; 图(b)为不同线索类型下违反竞争模型的时间窗口, 图(b)中*代表峰值, 箭头分别代表违反竞争模型时间窗口的起始时间、峰值发生时间。

参考文献 48

[1]	Arjona, A., Escudero, M., & Gómez, C. M. ( 2016). Cue validity probability influences neural processing of targets. Biological Psychology, 119, 171-183.
[2]	Baluch, F., & Itti, L. ( 2011). Mechanisms of top-down attention. Trends in Neurosciences, 34( 4), 210-224. URL pmid: 21439656
[3]	Bertelson, P., Vroomen, J., de Gelder, B., & Driver, J. ( 2000). The ventriloquist effect does not depend on the direction of deliberate visual attention. Perception & Psychophysics, 62( 2), 321-332.
[4]	Buschman, T. J., & Miller, E. K. ( 2007). Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science, 315( 5820), 1860-1862. URL pmid: 17395832
[5]	Busse, L., Roberts, K. C., Crist, R. E., Weissman, D. H., & Woldorff, M. G. ( 2005). The spread of attention across modalities and space in a multisensory object. Proceedings of the National Academy of Sciences of the United States of America, 102( 51), 18751-18756.
[6]	Carrasco, I. ( 2014). Gender gap in innovation: An institutionalist explanation. Management Decision, 52( 2), 410-424.
[7]	Chica, A. B., & Lupiáñez, J. ( 2009). Effects of endogenous and exogenous attention on visual processing: An Inhibition of Return study. Brain Research, 1278, 75-85.
[8]	Dombert, P. L., Kuhns, A., Mengotti, P., Fink, G. R., & Vossel, S. ( 2016). Functional mechanisms of probabilistic inference in feature- and space-based attentional systems. Neuroimage, 142, 553-564. URL pmid: 27523448
[9]	Eimer, M. ( 1997). Uninformative symbolic cues may bias visual-spatial attention: Behavioral and electrophysiological evidence. Biological Psychology, 46( 1), 67-71. URL pmid: 9255432
[10]	Fairhall, S. L., & Macaluso, E. ( 2009). Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. European Journal of Neuroscience, 29( 6), 1247-1257.
[11]	Giard, M. H., & Peronnet, F. ( 1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11( 5), 473-490. URL pmid: 10511637
[12]	Jonides, J. ( 1983). Towards a model of the mind's eye's movement. Bulletin of the Psychonomic Society, 21( 4), 247-250.
[13]	Koelewijn, T., Bronkhorst, A., & Theeuwes, J. ( 2010). Attention and the multiple stages of multisensory integration: A review of audiovisual studies. Acta Psychologica, 134( 3), 372-384. doi: 10.1016/j.actpsy.2010.03.010 URL pmid: 20427031
[14]	Kuhns, A. B., Dombert, P. L., Mengotti, P., Fink, G. R., & Vossel, S. ( 2017). Spatial attention, motor intention, and Bayesian cue predictability in the human brain. Journal of Neuroscience, 37( 21), 5334-5344. doi: 10.1523/JNEUROSCI.3255-16.2017 URL pmid: 28450541
[15]	Laurienti, P. J., Burdette, J. H., Maldjian, J. A., & Wallace, M. T. ( 2006). Enhanced multisensory integration in older adults. Neurobiology of Aging, 27( 8), 1155-1163. doi: 10.1016/j.neurobiolaging.2005.05.024 URL pmid: 16039016
[16]	Lewald, J., & Guski, R. ( 2003). Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. Cognitive Brain Research, 16( 3), 468-478.
[17]	Li, Q., Wu, J. L., & Touge, T. ( 2010). Audiovisual interaction enhances auditory detection in late stage: an event-related potential study. Neuroreport, 21( 3), 173-178.
[18]	Lunn, J., Sjoblom, A., Ward, J., Soto-Faraco, S., & Forster, S. ( 2019). Multisensory enhancement of attention depends on whether you are already paying attention. Cognition, 187, 38-49. URL pmid: 30825813
[19]	McCracken, H. S., Murphy, B. A., Glazebrook, C. M., Burkitt, J. J., Karellas, A. M., & Yielder, P. C. ( 2019). Audiovisual multisensory integration and evoked potentials in young adults with and without attention-deficit/hyperactivity disorder. Frontiers in Human Neuroscience, 13, 95. doi: 10.3389/fnhum.2019.00095 URL pmid: 30941026
[20]	Mengotti, P., Boers, F., Dombert, P. L., Fink, G. R., & Vossel, S. ( 2018). Integrating modality-specific expectancies for the deployment of spatial attention. Scientific Reports, 8( 1), 1210. doi: 10.1038/s41598-018-19593-7 URL pmid: 29352145
[21]	Meredith, M. A., Nemitz, J. W., & Stein, B. E. ( 1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. Journal of Neuropathology and Experimental Neurology, 7( 10), 3215-3229.
[22]	Meredith, M. A., & Stein, B. E. ( 1986a). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Research, 365( 2), 350-354. doi: 10.1016/0006-8993(86)91648-3 URL pmid: 3947999
[23]	Meredith, M. A., & Stein, B. E. ( 1986b). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56( 3), 640-662.
[24]	Meyer, K. N., Du, F., Parks, E., & Hopfinger, J. B. ( 2018). Exogenous vs. endogenous attention: Shifting the balance of fronto-parietal activity. Neuropsychologia, 111, 307-316. doi: 10.1016/j.neuropsychologia.2018.02.006 URL pmid: 29425803
[25]	Miller, J. ( 1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14( 2), 247-279.
[26]	Miller, J. ( 1986). Timecourse of coactivation in bimodal divided attention. Perception & Psychophysics, 40( 5), 331-343.
[27]	Peelen, M. V., & Kastner, S. ( 2014). Attention in the real world: Toward understanding its neural basis. Trends in Cognitive Sciences, 18( 5), 242-250.
[28]	Posner, M. I. ( 1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32( 1), 3-25.
[29]	Raab, D. H. ( 1962). Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences, 24( 5), 574-590.
[30]	Riggio, L., & Kirsner, K. ( 1997). The relationship between central cues and peripheral cues in covert visual orientation. Perception & Psychophysics, 59( 6), 885-899. URL pmid: 9270363
[31]	Santangelo, V., Fagioli, S., & Macaluso, E. ( 2010). The costs of monitoring simultaneously two sensory modalities decrease when dividing attention in space. Neuroimage, 49( 3), 2717-2727. URL pmid: 19878728
[32]	Santangelo, V., Ho, C., & Spence, C. ( 2008). Capturing spatial attention with multisensory cues. Psychonomic Bulletin & Review, 15( 2), 398-403. doi: 10.3758/pbr.15.2.398 URL pmid: 18488658
[33]	Santangelo, V., & Spence, C. ( 2007). Multisensory cues capture spatial attention regardless of perceptual load. Journal of Experimental Psychology: Human Perception and Performance, 33( 6), 1311-1321.
[34]	Stein, B. E., Meredith, M. A., & Wallace, M. T. ( 1993). The visually responsive neuron and beyond: Multisensory integration in cat and monkey. Progress in Brain Research, 95, 79-90.
[35]	Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. ( 2010). The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences, 14( 9), 400-410. URL pmid: 20675182
[36]	Talsma, D., & Woldorff, M. G. ( 2005). Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. Journal of Cognitive Neuroscience, 17( 7), 1098-1114. URL pmid: 16102239
[37]	Tang, X. Y., Gao, Y. L., Yang, W. P., Ren, Y. N., Wu, J. L., Zhang, M., & Wu, Q. ( 2019). Bimodal-divided attention attenuates visually induced inhibition of return with audiovisual targets. Experimental Brain Research, 237( 4), 1093-1107.
[38]	Ulrich, R., Miller, J., & Schröter, H. ( 2007). Testing the race model inequality: An algorithm and computer programs. Behavioural Brain Research, 39( 2), 291-302.
[39]	van den Brink, R. L., Cohen, M. X., van der Burg, E., Talsma, D., Vissers, M. E., & Slagter, H. A. ( 2014). Subcortical, modality-specific pathways contribute to multisensory processing in humans. Cerebral Cortex, 24( 8), 2169-2177.
[40]	van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. ( 2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34( 5), 1053-1065.
[41]	van der Burg, E., Talsma, D., Olivers, C. N. L., Hickey, C., & Theeuwes, J. ( 2011). Early multisensory interactions affect the competition among multiple visual objects. Neuroimage, 55( 3), 1208-1218. URL pmid: 21195781
[42]	van der Stigchel, S., Meeter, M., & Theeuwes, J. ( 2007). Top-down influences make saccades deviate away: The case of endogenous cues. Acta Psychologica, 125( 3), 279-290. URL pmid: 17022930
[43]	van der Stoep, N., van der Stigchel, S., & Nijboer, T. C. W. ( 2015). Exogenous spatial attention decreases audiovisual integration. Attention Perception & Psychophysics, 77( 2), 464-482.
[44]	van der Stoep, N., van der Stigchel, S., Nijboer, T. C. W., & Spence, C. ( 2017). Visually induced inhibition of return affects the integration of auditory and visual information. Perception, 46( 1), 6-17.
[45]	Vossel, S., Mathys, C., Daunizeau, J., Bauer, M., Driver, J., Friston, K. J., & Stephan, K. E. ( 2014). Spatial attention, precision, and Bayesian inference: A study of saccadic response speed. Cerebral Cortex, 24( 6), 1436-1450. doi: 10.1093/cercor/bhs418 URL pmid: 23322402
[46]	Vossel, S., Mathys, C., Stephan, K. E., & Friston, K. J. ( 2015). Cortical coupling reflects Bayesian belief updating in the deployment of spatial attention. Journal of Neuroscience, 35( 33), 11532-11542. doi: 10.1523/JNEUROSCI.1382-15.2015 URL pmid: 26290231
[47]	Vossel, S., Thiel, C. M., & Fink, G. R. ( 2006). Cue validity modulates the neural correlates of covert endogenous orienting of attention in parietal and frontal cortex. Neuroimage, 32( 3), 1257-1264.
[48]	Zou, H., Müller, H. J., & Shi, Z. ( 2012). Non-spatial sounds regulate eye movements and enhance visual search. Journal of Vision, 12( 5), 2. URL pmid: 22562709

内源性空间线索有效性对视听觉整合的影响

The influence of endogenous spatial cue validity on audiovisual integration

RichHTML

PDF (PC)

评审附件

可视化

English Version

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 48

相关文章 8

编辑推荐

Metrics

本文评价

[1]	祖光耀, 李舒淇, 张天阳, 王爱君, 张明. 返回抑制对视听跨通道对应的影响[J]. 心理学报, 2023, 55(8): 1220-1233.
[2]	高玉林, 唐晓雨, 刘思宇, 王爱君, 张明. 内源性空间线索有效性对老年人视听觉整合的影响[J]. 心理学报, 2023, 55(5): 671-684.
[3]	张明, 王婷婷, 吴晓刚, 张月娥, 王爱君. 面孔表情和声音情绪信息整合对返回抑制的影响[J]. 心理学报, 2022, 54(4): 331-342.
[4]	唐晓雨, 佟佳庚, 于宏, 王爱君. 内外源性空间注意对多感觉整合的影响[J]. 心理学报, 2021, 53(11): 1173-1188.
[5]	唐晓雨, 孙佳影, 彭姓. 双通道分配性注意对视听觉返回抑制的影响[J]. 心理学报, 2020, 52(3): 257-268.
[6]	彭姓, 常若松, 李奇, 王爱君, 唐晓雨. 不同SOA下视觉返回抑制对视听觉整合的调节作用[J]. 心理学报, 2019, 51(7): 759-771.
[7]	孙远路,胡中华,张瑞玲,寻茫茫,刘强,张庆林. 多感觉整合测量范式中存在的影响因素探讨[J]. 心理学报, 2011, 43(11): 1239-1246.
[8]	陈文锋, 焦书兰. 客体注意在等级模式加工中的作用[J]. 心理学报, 2003, 35(增刊): 51-63.