客体类别差异调节视听跨通道冲突中反应水平的感觉主导效应

doi:10.3724/SP.J.1041.2025.1001

摘要/Abstract

摘要： 感觉主导效应是大脑面临多种感觉通道信息时, 优先加工某种感觉通道信息的现象。认知加工水平假说认为感觉主导效应的产生是由于不同认知加工水平所决定, 即早期知觉加工水平表现为视觉主导, 晚期反应加工水平表现为听觉主导。然而, 现有的研究并未关注认知加工水平中处于早期和晚期之间的中间加工水平如何影响感觉主导效应。研究操纵了位于中间加工水平的客体类别差异, 采用2-1 mapping (映射)范式通过3个实验考察介于早期知觉和晚期反应水平之间的客体类别表征如何影响跨通道冲突感觉主导效应。实验1结果发现, 客体类别差异能够调节反应水平的感觉主导效应。类别差异小时, 表现为视觉主导; 类别差异大时, 表现为听觉主导。实验2结果说明此效应的产生与视觉刺激不同加工深度无关, 证实了此效应的产生特异于视觉通道。实验3通过经颅直流电刺激技术抑制类别加工脑区, 即左侧颞前叶。结果发现, 反应水平的听觉主导效应消失。研究表明, 认知加工水平中的客体类别表征这一中间加工水平对感觉主导效应产生了调节, 完善了认知加工水平假说对于跨通道冲突感觉主导效应的解释。

关键词: 客体类别, 视觉主导效应, 听觉主导效应, 反应前水平, 反应水平

Abstract:

Sensory dominance is a phenomenon in which the brain selectively processes specific sensory information when presented with multisensory inputs, thereby enhancing human perception of external stimuli. Previous studies have discussed sensory dominance at perceptual and response levels. However, how the intermediate processing level between perceptual and response levels affects sensory dominance remains unknown. Therefore, the present study adopted the cross-modal 2-1 mapping paradigm, and object categories were manipulated through three studies to investigate the role of the intermediate processing level on sensory dominance in a cross-modal conflict.

In this paradigm, based on key mapping, cognitive processing levels can be defined into the preresponse level (including perceptual and semantic levels) and the response level. The difference between the audiovisual incongruent condition and the audiovisual congruent condition is called the conflict effect, and sensory dominance can be obtained by comparing the conflict effect of attention to vision and auditory. Experiment 1 manipulated the degree of difference in object categories to explore its impact on sensory dominance. Experiments 1a−1c involved animal objects (small differences), tool objects (moderate differences), and animal and musical instrument objects (large differences). A total of 30 participants were recruited for each experiment. Visual pictures reached perceptual representation early, whereas auditory sounds reached semantic representation early. Therefore, Experiment 2 (34 participants) changed visual pictures into visual words on the basis of Experiment 1c to explore the effects of the visual presentation way of object categories on sensory dominance. In Experiment 3 (20 participants), transcranial direct current stimulation (tDCS) was used on the left anterior temporal lobe, an important brain region responsible for processing object categories to study casually the effects of object category on the sensory dominance of the response level further.

The results of Experiment 1 showed that, regardless of the difference in object categories, the conflict effect of attention to auditory at the preresponse level was significantly greater than that of attention to vision, that is, visual dominance. However, visual dominance at the response level appeared when the object category difference was small (Experiment 1a). Moreover, no sensory dominance was observed when the object category difference was moderate (Experiment 1b), and auditory dominance appeared when the object category difference was large (Experiment 1c). The results of Experiment 2 and Experiment 1c were consistent, that is, auditory dominance, indicating that this behavior pattern was not affected by the bottom-up visual presentation way. The results of Experiment 3 showed that under the cathodal tDCS condition, the preresponse level still showed visual dominance. However, the response level no longer showed sensory dominance. This result showed the effects of object categories on the sensory dominance of the response level from the causal level.

The mechanism of sensory dominance is still under investigation. The present study was the first to find that object categories affected the sensory dominance of the response level. From the perspective of cognitive processing level, the intermediate processing level played a regulating role in the sensory dominance of the response level. This finding can enrich the explanatory theory of sensory dominance and can provide a new perspective for the study of sensory dominance in a cross-modal conflict.

Key words: object categories, visual dominance, auditory dominance, preresponse level, response level

中图分类号:

B842

周衡, 王爱君, 袁祥勇, 蒋毅. (2025). 客体类别差异调节视听跨通道冲突中反应水平的感觉主导效应. 心理学报, 57(6), 1001-1012.

ZHOU Heng, WANG Ai-Jun, YUAN Xiang-Yong, JIANG Yi. (2025). Object category differences regulate the sensory dominance of the response level in an audiovisual cross-modal conflict. Acta Psychologica Sinica, 57(6), 1001-1012.

图/表 4

图1 实验1c的刺激实例及实验流程图。视听刺激构成三种条件, 一致条件：视听刺激完全一致; 反应前不一致条件：视听刺激不一致但对应按键一致; 反应不一致条件：视听刺激对应按键不一致。反应前冲突 = 反应前不一致条件的反应时 − 一致条件的反应时; 反应冲突 = 反应不一致条件的反应时 − 反应前不一致条件的反应时。

表1 各实验中被试的原始反应时平均值和标准误(ms)

实验名称	注意视觉			注意听觉
实验名称	一致	反应前不一致	反应不一致	一致	反应前不一致	反应不一致
实验1a	494 ± 20	542 ± 29	539 ± 29	756 ± 25	850 ± 29	884 ± 32
实验1b	522 ± 32	571 ± 40	602 ± 45	787 ± 26	873 ± 339	907 ± 34
实验1c	496 ± 30	529 ± 37	562 ± 45	777 ± 27	862 ± 30	867 ± 30
实验2	503 ± 27	535 ± 33	559 ± 36	791 ± 24	874 ± 26	880 ± 29
实验3-伪	520 ± 22	522 ± 25	591 ± 28	704 ± 34	761 ± 38	807 ± 33
实验3-阴	510 ± 24	539 ± 27	580 ± 38	679 ± 26	734 ± 29	773 ± 27

图2 实验1和2的结果图。A~C分别为实验1a~c的结果图; D为实验1每个实验的感觉主导指数(听觉冲突与视觉冲突之差)再抽样得到的样本均值分布图; E为实验2的结果图。注：***p < 0.001, **p < 0.01和*p < 0.05。误差棒表示标准误。彩图见电子版, 下同。

图3 A为左侧颞前叶皮层在国际EEG 10-20系统标准的定位示意图, 位于T7和FT7之间; B为HD-explore软件模拟左侧颞前叶皮层在标准脑模型的电场模型, C为伪刺激条件下的实验结果; D为阴极刺激条件下的实验结果。注：***p < 0.001, *p < 0.05。误差棒表示标准误。

参考文献 57

[1]	Akbıyık S., Goksun T., & Balcı F. (2020). Cathodal tDCS stimulation of left anterior temporal lobe eliminates cross- category color discrimination response time advantage. Behavioural Brain Research, 391, 112682. doi: 10.1016/j.bbr.2020.112682 pmid: 32445777
[2]	Amedi A., Hofstetter S., Maidenbaum S., & Heimler B. (2017). Task selectivity as a comprehensive principle for brain organization. Trends in Cognitive Sciences, 21(5), 307-310. doi: S1364-6613(17)30050-5 pmid: 28385460
[3]	Baddeley A. D., & Hitch G. J. (2017). Is the levels of processing effect language-limited? Journal of Memory and Language, 92, 1-13.
[4]	Bi Y. C., Wang X. Y., & Caramazza A. (2016). Object domain and modality in the ventral visual pathway. Trends in Cognitive Sciences, 20(4), 282-290. doi: S1364-6613(16)00043-7 pmid: 26944219
[5]	Binney R. J., Ashaie S. A., Zuckerman B. M., Hung J. Y., & Reilly J. (2018). Frontotemporal stimulation modulates semantically-guided visual search during confrontation naming: A combined tDCS and eye tracking investigation. Brain and Language, 180-182, 14-23.
[6]	Butera I. M., Stevenson R. A., Gifford R. H., & Wallace M. T. (2023). Visually biased perception in cochlear implant users: A study of the McGurk and Sound-induced flash illusions. Trends in Hearing, 27, 23312165221076681. doi: 10.1177/23312165221076681 pmid: 37377212
[7]	Callan A., Callan D., & Ando H. (2015). An fMRI study of the ventriloquism effect. Cerebral Cortex, 25(11), 4248-4258.
[8]	Chen A. T., Liu Q., Chen C. M., & Li H. (2010). The effect of Intermediate processing level on response conflict in Stroop-like tasks. Journal of Psychological Science, 33(3), 569-572.
	[陈安涛, 刘强, 陈昌明, 李红. (2010). 中间模块加工水平对反应冲突的影响及其机制. 心理科学, 33(3), 569-572.]
[9]	Chen Q., & Zhou X. L. (2013). Vision dominates at the preresponse level and audition dominates at the response level in cross-modal interaction: Behavioral and neural evidence. Journal of Neuroscience, 33(17), 7109-7121. doi: 10.1523/JNEUROSCI.1985-12.2013 pmid: 23616521
[10]	Colavita F. B. (1974). Human sensory dominance. Perception & Psychophysics, 16(2), 409-412.
[11]	Dietze N., & Poth C. H. (2023). Vision rivals audition in alerting humans for fast action. Acta Psychologica, 238, 103991.
[12]	Diez, E., Gomez-Ariza, C. J., Diez-Alamo, A. M., Alonso, M. A., & Fernandez, A. (2023). The processing of semantic relatedness in the brain: Evidence from associative and categorical false recognition effects following transcranial direct current stimulation of the left anterior temporal lobe. Cortex, 93, 133–145.
[13]	Dirani J., & Pylkkänen L. (2023). The time course of cross-modal representations of conceptual categories. NeuroImage, 277, 120254.
[14]	Donohue S. E., Todisco A. E., & Woldorff M. G. (2013). The rapid distraction of attentional resources toward the source of incongruent stimulus input during multisensory conflict. Journal of Cognitive Neuroscience, 25(4), 623-635. doi: 10.1162/jocn_a_00336 pmid: 23249355
[15]	Faul F., Erdfelder E., Buchner A., & Lang A.-G. (2009). Statistical power analyses using GPower 3.1: Tests for correlation and regression analyses. Behavior Research Methods*, 41(4), 1149-1160. doi: 10.3758/BRM.41.4.1149 pmid: 19897823
[16]	Faul F., Erdfelder E., Lang A.-G., & Buchner A. (2007). GPower 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods*, 39, 175-191. doi: 10.3758/bf03193146 pmid: 17695343
[17]	Haselton M. G., & Buss D. M. (2000). Error management theory: A new perspective on biases in cross-sex mind reading. Journal of Personality and Social Psychology, 78(1), 81-91. pmid: 10653507
[18]	Hockley W. E., & Bancroft T. (2011). Extensions of the picture superiority effect in as-sociative recognition. Canadian Journal of Experimental Psychology, 65(4), 236-244.
[19]	Higdon K. F., Neath I., Surprenant A. M., & Ensor T. M. (2024). Distinctiveness, not dual coding, explains the picture- superiority effect. Quarterly Journal of Experimental Psychology. https://doi.org/10.1177/17470218241235520
[20]	Hirst R. J., Cragg L., & Allen H. A. (2018). Vision dominates audition in adults but not children: A meta-analysis of the Colavita effect. Neuroscience & Biobehavioral Reviews, 94, 286-301.
[21]	Hirst R. J., McGovern D. P., Setti A., Shams L., & Newell F. N. (2020). What you see is what you hear: Twenty years of research using the sound-induced flash illusion. Neuroscience & Biobehavioral Reviews, 118, 759-774.
[22]	Kang G. L., & Luo X. X. (2020). The integration and conflict control in audiovisual processing. Journal of Psychological Science, 43(5), 1072-1078.
	[康冠兰, 罗霄骁. (2020). 视听跨通道信息的整合与冲突控制. 心理科学, 43(5), 1072-1078.]
[23]	Kato M., & Konishi Y. (2006). Auditory dominance in the error correction process: A synchronized tapping study. Brain Research, 1084(1), 115-122. pmid: 16556436
[24]	Keil J. (2020). Double flash illusions: Current findings and future directions. Frontiers in Neuroscience, 14, 298-304. doi: 10.3389/fnins.2020.00298 pmid: 32317920
[25]	Koppen C., Levitan C. A., & Spence C. (2009). A signal detection study of the Colavita visual dominance effect. Experimental Brain Research, 196(3), 353-360. doi: 10.1007/s00221-009-1853-y pmid: 19488743
[26]	Lambon Ralph M. A., & Patterson K. (2008). Generalization and differentiation in semantic memory:Insights from semantic dementia. Annals of the New York Academy of Sciences, 1124(1), 61-76.
[27]	Laws K. R. (2000). Category-specific naming errors in normal subjects: The influence of evolution and experience. Brain and Language, 75(1), 123-133. pmid: 11023642
[28]	Li X., Cai S. Z., Chen Y., Tian X. M., & Wang A. J. (2024). Enhancement of visual dominance effects at the response level in children with attention-deficit/hyperactivity disorder. Journal of Experimental Child Psychology. 242, 105897.
[29]	Li Z., Gu R., Qi M., Cen J., Zhang S., Gu J., Zeng X., & Chen Q. (2019). Loss of vision dominance at the preresponse level in tinnitus patients: Preliminary behavioral evidence. Frontiers in Neuroscience, 13, 482. doi: 10.3389/fnins.2019.00482 pmid: 31139048
[30]	Martin A. (2007). The representation of object concepts in the brain. Annual Review of Psychology, 58, 25-45. pmid: 16968210
[31]	Martin A., & Chao L. (2001). Semantic memory and the brain: Structure and processes. Current Opinion in Neurobiology, 11(2), 194-201. doi: 10.1016/s0959-4388(00)00196-3 pmid: 11301239
[32]	Mayer A. R., Franco A. R., Canive J., & Harrington D. L. (2009). The effects of stimulus modality and frequency of stimulus presentation on cross-modal distraction. Cerebral Cortex, 19(5), 993-1007.
[33]	Mesulam M. M., Wieneke C., Hurley R., Rademaker A., Thompson C. K., Weintraub S., & Rogalski E. J. (2013). Words and objects at the tip of the left temporal lobe in primary progressive aphasia. Brain, 136(2), 601-618.
[34]	Molholm S., Ritter W., Javitt D. C., & Foxe J. J. (2004). Multisensory visual-auditory object recognition in humans: A high-density electrical mapping study. Cerebral Cortex, 14(4), 452-465. doi: 10.1093/cercor/bhh007 pmid: 15028649
[35]	Morgenstern Y., Storrs K. R., Schmidt F., Hartmann F., Tiedemann H., Wagemans J., & Fleming R. W. (2024). High-level aftereffects reveal the role of statistical features in visual shape encoding. Current Biology, 34(5), 1098-1106.
[36]	Nieznański M. (2020). Levels-of-processing effects on context and target recollection for words and pictures. Acta Psychologica, 209, 103127.
[37]	Odegaard B., Wozny D. R., & Shams L. (2016). The effects of selective and divided attention on sensory precision and integration. Neurosciences Letters, 614, 24-28.
[38]	Potter M. C., & Fox L. F. (2009). Detecting and remembering simultaneous pictures in a rapid serial visual presentation. Journal of Experimental Psychology: Human Perception and Performance, 35(1), 28-38.
[39]	Roberts B. R. T., MacLeod C. M., & Fernandes M. A. (2023). Symbol superiority: Why $ is better remembered than ‘dollar’. Cognition, 238, 105435.
[40]	Robinson C. W., Chandra M., & Sinnett S. (2016). Existence of competing modality dominances. Attention, Perception, & Psychophysics, 78(4), 1104-1114.
[41]	Schubert E. (2021). Creativity is optimal novelty and maximal positive affect: A new definition based on the spreading activation model. Frontiers in Neuroscience, 15, 612379.
[42]	Shams L., Kamitani Y., & Shimojo S. (2000). What you see is what you hear. Nature, 408(6814), 788-788.
[43]	Snodgrass J. G., & Vanderwart M. (1980). A standardized set of 260 picture: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 6(2), 174-215.
[44]	Stein B. E., & Stanford T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9(4), 255-266. doi: 10.1038/nrn2331 pmid: 18354398
[45]	Tang X. Y., Wu J. L., & Shen Y. (2016). The interactions of multisensory integration with endogenous and exogenous attention. Neuroscience & Biobehavioral Reviews, 61, 208-224.
[46]	Ubaldi S., & Fairhall S. L. (2021). fMRI response to automatic and purposeful familiar-face processing in perceptual and nonperceptual cortical regions. Journal of Neurophysiology, 125(4), 1058-1067. doi: 10.1152/jn.00481.2020 pmid: 33596739
[47]	Velasquez A. G., Gazzaley A., Toyoda H., Ziegler D. A., & Morsella E. (2021). The generation of involuntary mental imagery in an ecologically-valid task. Frontiers in Psychology. 12, 759685.
[48]	Visconti di Oleggio Castello M., Haxby J. V., & Gobbini M. I. (2021). Shared neural codes for visual and semantic information about familiar faces in a common representational space. Proceedings of the National Academy of Sciences, 118(45), e2110474118.
[49]	Vogler J. N., & Titchener K. (2011). Cross-modal conflicts in object recognition: Determining the influence of object category. Experimental Brain Research, 214, 597-605. doi: 10.1007/s00221-011-2858-x pmid: 21912929
[50]	Wang A. J., Huang J., Lu F. F. He J. Y., Tang X. Y., & Zhang M. (2020). Sound-induced flash illusion in multisensory integration. Advances in Psychological Science, 28(10), 1662-1677. doi: 10.3724/SP.J.1042.2020.01662
	[王爱君, 黄杰, 陆菲菲, 何嘉滢, 唐晓雨, 张明. (2020). 多感觉整合中的声音诱发闪光错觉效应. 心理科学进展, 28(10), 1662-1677.] doi: 10.3724/SP.J.1042.2020.01662
[51]	Wang A. J., Sang H. B., He J. Y., Sava-Segal C., Tang X. Y., & Zhang M. (2019). Effects of cognitive expectation on sound-induced flash illusion. Perception, 48(12), 1214-1234. doi: 10.1177/0301006619885796 pmid: 31808371
[52]	Wang A., Zhou H., Zhang F., Sang H., Yu W., Tang X., Zhang T., & Zhang M. (2021). Repetition suppression in visual and auditory modalities affects the sound-induced flash illusion. Perception, 50(6), 489-507.
[53]	Wang Y. H., Wang Y. F., & Zhou X. L. (2006). Conflict Control at different periods of processing in children with two subtypes of ADHD. Acta Psychologica Sinica, 38(2), 181-188.
	[王勇慧, 王玉凤, 周晓林. (2006). 注意缺陷多动障碍儿童在不同加工阶段的干扰控制. 心理学报, 38(2), 181-188.]
[54]	Wiggett A. T., Pritchard R. C., & Downing P. E. (2009). Animate and inanimate objects in human visual cortex: Evidence for task-independent category effects. Neuropsychologia, 47(14), 3111-3117. doi: 10.1016/j.neuropsychologia.2009.07.008 pmid: 19631673
[55]	Wong C., & Gallate J. (2012). The function of the anterior temporal lobe: A review of the empirical evidence. Brain Research, 1449, 94-116. doi: 10.1016/j.brainres.2012.02.017 pmid: 22421014
[56]	Wozny D. R., Beierholm U. R., & Shams L. (2008). Human trimodal perception follows optimal statistical inference. Journal of Vision, 8(3), 1-11.
[57]	Zhou H., Li S., Huang J., Yang J., Wang A., & Zhang M. (2022). Sound-induced flash illusions at different spatial locations were affected by personality traits. Attention, Perception, & Psychophysics, 85(2), 463-473.