语言和情境对具体概念感知运动仿真的影响

doi:10.3724/SP.J.1041.2022.00583

摘要/Abstract

摘要：

具体概念加工中的感知运动仿真是概念表征的核心加工过程。本研究系统探讨了语言因素(语言类型：第一语言和第二语言)和情境因素(知觉情境：空间知觉情境和语义知觉情境)对具体概念加工中感知运动仿真的影响。实验1采用语义相关判断范式探讨了第二语言加工过程中是否存在感知运动仿真, 以及第二语言中的仿真和第一语言中的仿真是否存在差异。研究发现, 第二语言加工中依然存在感知运动仿真, 但是第一语言的感知运动仿真具有一定的优势。实验2采用语义相关判断范式及其变式, 通过2个分实验分别探讨了个体加工概念时空间知觉信息和语义知觉信息对于概念表征过程中感知运动仿真的影响。研究发现, 无论是在较弱的空间信息知觉条件下还是在较浅的语义信息知觉条件下, 具体概念加工过程中都产生了感知运动仿真。本研究的发现弥补了知觉符号理论未对第二语言中的感知运动仿真提出针对性预测的不足, 表明感知运动仿真具有一定的跨语言稳定性。同时, 具体概念加工中的感知运动仿真不受空间信息和语义信息的调节, 这表明感知运动仿真能在一定程度上自动化产生。

关键词: 具体概念, 概念表征, 感知运动仿真, 知觉符号理论

Abstract:

How concepts are represented in the brain is an important topic in cognitive science. There are two different theories on this research question, i.e., propositional symbol theory and perceptual symbol theory. The difference between these two theories is how to treat the relationship between the internal symbol and the external archetype. The propositional symbol theory holds that the relationship between the two is arbitrary. Perceptual information does not participate in the processing of concepts. While perceptual symbols theory holds that the relationship between the two is similar. Perceptual information participates in the processing of concepts.
People learn conceptual information in the vertical space. The spatial information is stored in long-term memory along with the representation of concepts. This hypothesis has been supported by many studies and is in line with the perceptual symbol theory. However, few studies have tested what factors influence sensorimotor simulation in conceptual processing. In this study, the semantic relevance judgment paradigm is used to test this question. The present study examined whether the sensorimotor simulation participates in conceptual processing in the first and second languages. Then, the present study examined how situational information modulates conceptual processing, by manipulating the intensity of spatial information and the level of semantic processing.
In Experiment 1, the semantic relevance judgment paradigm is used to test whether sensorimotor simulation is involved in the second language processing and whether there is any difference between the first language and second language processing. The results of Experiment 1 show that the sensorimotor simulation has participated in the second language processing. However, there is an accuracy advantage in the sensorimotor simulation in the first language. In Experiment 2, two experiments are conducted to test the effect of the perceptual situation on sensorimotor simulation. Experiment 2a tests the influence of perceptual situation on sensorimotor simulation by changing the intensity of individual perceptual in vertical spatial axis. The results show that the sensorimotor simulation can be found in both strong and weak spatial perception. The effect of semantic processing level on perceptual motion simulation is tested in Experiment 2b. The experiment manipulates the level of semantic processing by using the semantic relevance judgment task and word judgment task. The results show that sensorimotor simulation participates in the processing of concepts, while semantic situational did not modulate this effect. The results of Experiment 2 suggest that sensorimotor stimulation may be automatic in conceptual processing.
The present study conducted two experiments to test the role of sensorimotor simulation in conceptual processing. Results found that the sensorimotor simulation participates in conceptual representation, supporting the perceptual symbol theory. Furthermore, the sensorimotor simulation takes place in both Chinese and English. This finding extends the perceptual symbol theory. Finally, results found that sensorimotor stimulation may be automatic in conceptual representation and is not affected by spatial information and semantic processing.

Key words: concrete concept, conceptual representation, sensorimotor simulation, perceptual symbol theory

中图分类号:

B842

石如彬, 谢久书, 杨梦情, 王瑞明. (2022). 语言和情境对具体概念感知运动仿真的影响. 心理学报, 54(6), 583-594.

SHI Rubin, XIE Jiushu, YANG Mengqing, WANG Ruiming. (2022). The influence of language and context on sensorimotor simulation of concrete concepts. Acta Psychologica Sinica, 54(6), 583-594.

图/表 12

参考文献 45

[1]	Barsalou, L. W. (1999). Perceptions of perceptual symbols. Behavioral and Brain Sciences, 22(4), 637-660. doi: 10.1017/S0140525X99532147 URL
[2]	Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59(1), 617-645. doi: 10.1146/annurev.psych.59.103006.093639 URL
[3]	Barsalou, L. W., Dutriaux, L., & Scheepers, C. (2018). Moving beyond the distinction between concrete and abstract concepts. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1752).
[4]	Baumeister, J. C., Foroni, F., Conrad, M., Rumiati, R. I., & Winkielman, P. (2017). Embodiment and emotional memory in first vs. second language. Frontiers in Psychology, 8(8), 394.
[5]	Bergen, B., Lau, T. T. C., Narayan, S., Stojanovic, D., & Wheeler, K. (2010). Body part representations in verbal semantics. Memory & Cognition, 38(7), 969-981. doi: 10.3758/MC.38.7.969 URL
[6]	Chen, D., Wang, R., Zhang, J., & Liu, C. (2020). Perceptual representations in L1, L2 and L3 comprehension: Delayed sentence-picture verification. Journal of Psycholinguistic Research, 49(1), 41-57. doi: 10.1007/s10936-019-09670-x URL
[7]	Costa, A., Santesteban, M. & Cano, A. (2005). On facilitatory effect cognate words in bilingual speech produefion. Brain and Language, 94(9): 94-103. doi: 10.1016/j.bandl.2004.12.002 URL
[8]	Dudschig, C., de la Vega, I., de Filippis, M., & Kaup, B. (2014). Language and vertical space: On the automaticity of language action interconnections. Cortex, 58, 151-160. doi: 10.1016/j.cortex.2014.06.003 pmid: 25016318
[9]	Dudschig, C., de la Vega, I., & Kaup, B. (2014). Embodiment and second-language: Automatic activation of motor responses during processing spatially associated L2 words and emotion L2 words in a vertical stroop paradigm. Brain and Language, 132, 14-21. doi: 10.1016/j.bandl.2014.02.002 pmid: 24681402
[10]	Dudschig, C., & Kaup, B. (2017). Is it all task-specific? The role of binary responses, verbal mediation, and saliency for eliciting language-space associations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(2), 259- 270. doi: 10.1037/xlm0000297 URL
[11]	Dudschig, C., Lachmair, M., de la Vega, I., de Filippis, M., & Kaup, B. (2012). From top to bottom: Spatial shifts of attention caused by linguistic stimuli. Cognitive Processing, 13(1), 151-154.
[12]	Dudschig, C., Souman, J., Lachmair, M., de la Vega, I., & Kaup, B. (2013). Reading “sun” and looking up: The influence of language on saccadic eye movements in the vertical dimension. Plos One, 8(2), e56872. doi: 10.1371/journal.pone.0056872 URL
[13]	Dunn, B., Kamide, Y., & Scheepers, C. (2014). Hearing “moon” and looking up:Word-related spatial associations facilitate saccades to congruent locations. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 36, No. 36
[14]	Eilola, T. M., & Havelka, J. (2010). Behavioural and physiological responses to the emotional and taboo stroop tasks in native and non-native speakers of English. International Journal of Bilingualism, 15(3), 353- 369. doi: 10.1177/1367006910379263 URL
[15]	Estes, Z., & Barsalou, L. W. (2018). A comprehensive meta- analysis of spatial interference from linguistic cues: Beyond Petrova et al. (2018). Psychological Science, 29(9), 1558- 1564. doi: 10.1177/0956797618794131 URL
[16]	Fodor, J. A. (1975). The language of thought. Cambridge, MA: Harvard University Press.
[17]	Glenberg, A. M., & Gallese, V. (2012). Action-based language: A theory of language acquisition, comprehension, and production. Cortex, 48(7), 905-922. doi: 10.1016/j.cortex.2011.04.010 pmid: 21601842
[18]	Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558-565. doi: 10.3758/BF03196313 URL
[19]	Grauwe, S. D., Willems, R. M., Rueschemeyer, S. A., Lemhöfer, K., & Schriefers, H. (2014). Embodied language in first- and second-language speakers: Neural correlates of processing motor verbs. Neuropsychologia, 56, 334-349. doi: 10.1016/j.neuropsychologia.2014.02.003 URL
[20]	Günther, F., Dudschig, C., & Kaup, B. (2018). Symbol grounding without direct experience: Do words inherit sensorimotor activation from purely linguistic context?. Cognitive Science, 42, 336-374. doi: 10.1111/cogs.12549 URL
[21]	Harnad, S. (1990). The symbol grounding problem. Physica D Nonlinear Phenomena, 42(1-3), 335-346. doi: 10.1016/0167-2789(90)90063-U URL
[22]	Harris, C. L., Ayçíçeğí, A., & Gleason, J. B. (2003). Taboo words and reprimands elicit greater autonomic reactivity in a first language than in a second language. Applied Psycholinguistics, 24(4), 561-579. doi: 10.1017/S0142716403000286 URL
[23]	Jared, D., Pei Yun Poh, R., & Paivio, A. (2013). L1 and L2 picture naming in Mandarin-English bilinguals: A test of bilingual dual coding theory. Bilingualism: Language and Cognition, 16(2), 383-396. doi: 10.1017/S1366728912000685 URL
[24]	Lachmair, M., Dudschig, C., Filippis, M. D., Vega, I., & Kaup, B. (2011). Root versus roof: Automatic activation of location information during word processing. Psychonomic Bulletin & Review, 18(6), 1180-1188. doi: 10.3758/s13423-011-0158-x URL
[25]	Lakens, D. (2012). Polarity correspondence in metaphor congruency effects: Structural overlap predicts categorization times for bipolar concepts presented in vertical space. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(3), 726-736. doi: 10.1037/a0024955 URL
[26]	Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of Chicago Press.
[27]	Lebois, L. A., Wilson-Mendenhall, C. D., & Barsalou, L. W. (2014). Are automatic conceptual cores the gold standard of semantic processing? The context - dependence of spatial meaning in grounded congruency effects. Cognitive Science, 39(8), 1764-1801. doi: 10.1111/cogs.12174 URL
[28]	Lewis, M. P., Simons, G. F., & Fennig, C. D. (2015). Ethnologue: Languages of the world (18th ed.). SIL International.
[29]	Liu, W. J., Li, Y., & Wang, R. M. (2015). The role of sensorimotor information in conceptual representation. Journal of Psychological Science, 38(6), 1347-1352.
	[ 刘文娟, 李莹, 王瑞明. (2015). 感知运动信息在概念表征中的作用. 心理科学, 38(6), 1347-1352.]
[30]	Liu, W. J., Shen, M. Q., Li, Y., & Wang, R. M. (2016). The interaction between emotional concept processing and emotional face perception. Acta Psychologica Sinica, 48(2), 163-173. doi: 10.3724/SP.J.1041.2016.00163 URL
	[ 刘文娟, 沈曼琼, 李莹, 王瑞明. (2016). 情绪概念加工与情绪面孔知觉的相互影响. 心理学报, 48(2), 163-173.]
[31]	Mo, L., Wu, L. M., & Wang, R. M. (2006). Spatial iconicity affects semantic relatedness judgment. Journal of Psychological Science, 29(4), 770-773.
	[ 莫雷, 伍丽梅, 王瑞明. (2006). 物体的空间关系对语义相关判断的影响. 心理科学, 29(4): 770-773]
[32]	Nielsen, A. K. S., & Rendall, D. (2013). Parsing the role of consonants versus vowels in the classic Takete-Maluma phenomenon. Canadian Journal of Experimental Psychology, 67(2), 153-163. doi: 10.1037/a0030553 URL
[33]	ÖTTL, B., Dudschig, C., & Kaup, B. (2017). Forming associations between language and sensorimotor traces during novel word learning. Language and Cognition, 9(1), 156-171. doi: 10.1017/langcog.2016.5 URL
[34]	Proctor, R. W., & Xiong, A. (2015). Polarity correspondence as a general compatibility principle. Current Directions in Psychological Science, 24(6), 446-451. doi: 10.1177/0963721415607305 URL
[35]	Pylyshyn, Z. W. (1984). Computation and cognition. Cambridge, MA: MIT Press.
[36]	Strozyk, J. V., Dudschig, C., & Kaup, B. (2019). Do I need to have my hands free to understand hand-related language? Investigating the functional relevance of experiential simulations. Psychological Research, 83(3), 406-418. doi: 10.1007/s00426-017-0900-8 URL
[37]	Vukovic, N., & Shtyrov, Y. (2014). Cortical motor systems are involved in second-language comprehension: Evidence from rapid mu-rhythm desynchronisation. NeuroImage, 102, 695-703. doi: 10.1016/j.neuroimage.2014.08.039 URL
[38]	Wang, R. M., Mo, L., Li, L., Wang, S. P., & Wu, J. (2005). Perceptual symbol representation and propositional symbol representation in language comprehension. Acta Psychologica Sinica, 37(2), 143-150.
	[ 王瑞明, 莫雷, 李利, 王穗苹, 吴俊. (2005). 言语理解中的知觉符号表征与命题符号表征. 心理学报, 37(2), 143-150.]
[39]	Wang, R. M., Mo, L., Wu, L. M., & Li, L. (2006). The influence of spatial information on semantic relatedness judgment. Journal of Psychological Science, 29(6), 1320-1323.
	[ 王瑞明, 莫雷, 伍丽梅, 李利. (2006). 空间信息表征对语义相关判断的影响. 心理科学, 29(6), 1320-1323.]
[40]	Wang, R. M., Zhang, J. T., Li, L., & Mo, L. (2010). The role of bilinguals^,second language in semantic access of third language. Journal of Psychological Science, 33(4), 853- 856.
	王瑞明, 张洁婷, 李利, 莫雷. (2010). 二语词汇在双语者三语词汇语义通达中的作用. 心理科学, 33(4), 853-856.
[41]	Xie, J. S., Shen, G. Y., & Huang, Y. L. (2022). Debates and integration of bilinguals’ cross-language orthographic transfer. Journal of Psychological Science, in press
	[ 谢久书, 沈光银, 黄艳利. (2022). 双语者跨语言正字法迁移的争议与整合. 心理科学, 印刷中]
[42]	Zhao, T. Y., Huang, Y. L., Chen, D. G., Jiao, L., Ramos, F. M., Wang, R. M., & Xie, J. S. (2020). The modality switching costs of Chinese - English bilinguals in the processing of L1 and L2. Quarterly Journal of Experimental Psychology, 73(3) 396-412. doi: 10.1177/1747021819878089 URL
[43]	Zwaan, R. A., & Madden, C. J. (2005). Embodied sentence comprehension. In D. Pecher & R. Zwaan (Eds.), Grounding cognition: The role of perception and action in memory, language, and thinking (pp. 224-245) Cambridge: Cambridge University Press.
[44]	Zwaan, R. A., Stanfield, R., & Yaxley, R. H. (2002). Language comprehenders mentally represent the shapes of objects. Psychological Science, 13(2), 168-171. pmid: 11934002
[45]	Zwaan, R. A., & Yaxley, R. H. (2003). Spatial iconicity affects semantic relatedness judgments. Psychonomic Bulletin & Review, 10(4), 954-958. doi: 10.3758/BF03196557 URL

语言类型	空间一致(ms)	空间不一致(ms)
中文	784.00 ± 82.32	799.02 ± 94.00
英文	1004.66 ± 110.66	1030.88 ± 144.82

语言类型	空间一致(ms)	空间不一致(ms)
中文	784.00 ± 82.32	799.02 ± 94.00
英文	1004.66 ± 110.66	1030.88 ± 144.82

语言类型	空间一致	空间不一致
中文	0.97 ± 0.04	0.95 ± 0.04
英文	0.92 ± 0.07	0.93 ± 0.07

语言类型	空间一致	空间不一致
中文	0.97 ± 0.04	0.95 ± 0.04
英文	0.92 ± 0.07	0.93 ± 0.07

空间知觉	空间一致(ms)	空间不一致(ms)
强空间知觉	732.53 ± 97.06	744.35 ± 88.56
弱空间知觉	672.34 ± 78.68	679.83 ± 75.84