语言符号与知觉符号表征对外语词汇习得的影响比较：来自行为与脑电的实验证据

doi:10.3724/SP.J.1041.2024.00542

摘要/Abstract

摘要：

通过行为与脑电实验技术, 比较外语词汇学习过程中, 语言符号与知觉符号表征对词汇记忆编码与再认的影响过程, 从而考察语义表征对词汇习得的影响机制。行为结果表明, 语言符号与知觉符号表征条件下被试对词汇学习效果的主观判断无显著差异, 但后者比前者有更高的词汇再认正确率。脑电结果表明, 对于词汇编码阶段, 相较语言符号表征, 知觉符号表征在编码晚期诱发更正的LPC成分; 对于词汇再认阶段, 知觉符号表征条件诱发更大N400成分, 且脑电时频分析表明该条件下出现更为明显的μ波抑制及θ波功率增强现象。研究结果综合表明, 与语言符号表征便捷但非模态化的语义加工相比, 知觉符号表征加深了词汇晚期编码的深度, 并通过知觉模拟, 利用多模态信息提高了词汇的形象化再认, 从而推动了再认阶段的语义检索, 最终内隐性地提高了词汇学习效果。

关键词: 语义表征, 外语词汇习得, 语言符号, 知觉符号, EEG

Abstract:

Semantic representation is a way to achieve semantic access, which plays an important role in foreign language vocabulary learning. The linguistic symbolic and perceptual symbolic representations constitute two subsystems of semantic representations. Previous studies have found that the linguistic symbolic representation can provide quick approximate processing, which is suitable for shallow language processing tasks, while the perceptual symbolic representation can create a complete situation model, which is important for deep language processing tasks. In view of these different cognitive processing patterns, the effect of each symbolic representation system on foreign language vocabulary learning were investigated in this study to explore the mechanism of semantic representation on vocabulary learning.
A total of 52 participants were randomly assigned to the two symbolic representation conditions and executed a vocabulary learning-recognition task. To manipulate the participants’ semantic representations effectively when learning foreign language vocabulary, two types of spatial semantics, i.e., “up” and “down,” were chosen as the learning materials. Furthermore, to investigate the learning process effectively, the foreign words expressing the meanings of “up” and “down” were selected from languages that the participants were completely unfamiliar with. In the learning stage, Chinese characters or spatial cues of “up” and “down” were presented first as semantic priming stimuli to initiate the participants’ linguistic symbolic or perceptual symbolic representations of the foreign words subsequently presented. The participants were then required to learn the foreign words. After every 20 words learned, a test on the semantic recognition of the foreign words was conducted. Behavioral and EEG data were collected to investigate the different effects of linguistic symbolic and perceptual symbolic representations on the learning and recognition stages.
The behavioral results showed that no significant difference in judgment of learning were found between the linguistic symbolic and perceptual symbolic representation conditions, but the latter showed higher recognition accuracy rate than the former. The event related potential results showed that during the learning stage, the perceptual symbolic representation induced more positive LPC components (in the time window of 400~800 ms) than the linguistic symbolic representation condition. During the recognition stage, in relation to the linguistic symbolic representation condition, the perceptual symbolic representation evoked larger N400 components in the time window of 200~400 ms after the onset of the recognition words. The results of EEG time-frequency analysis showed that during the recognition stage, the perceptual symbolic representation condition elicited lower μ band power and higher θ band power than the linguistic symbolic representation condition (the time windows of the two bands were 200~800 ms after the onset of the recognition words).
In conclusion, results indicated that compared with linguistic symbolic representation, perceptual symbolic representation had a delayed influence on vocabulary encoding. It promoted deep encoding processing of vocabulary and improved the efficiency of vocabulary semantic retrieval through perceptual simulation in the recognition process, thereby implicitly improving the semantic recognition of vocabulary.

Key words: semantic representation, foreign language vocabulary learning, linguistic symbol, perceptual symbol, EEG

中图分类号:

B842

任维聪, 杨婷, 王汉林. (2024). 语言符号与知觉符号表征对外语词汇习得的影响比较：来自行为与脑电的实验证据. 心理学报, 56(5), 542-554.

REN Weicong, YANG Ting, WANG Hanlin. (2024). Different effects of linguistic and perceptual symbolic representations on foreign language vocabulary learning: Evidence from behavioral and EEG data. Acta Psychologica Sinica, 56(5), 542-554.

图/表 5

图1 知觉符号表征(上图)与语言符号表征(下图)条件实验流程

图2 语言符号与知觉符号表征条件下被试对外语词汇记忆效果的主观评价(学习判断JOL评分, 左图)与客观测评(再认正确率, 右图)结果

图3 两种表征条件下词汇记忆编码阶段诱发LPC及其差异波(虚线−实线)地形图

图4 两种表征条件下词汇再认阶段诱发N400及其差异波(虚线−实线)地形图

图5 两种表征条件下θ频段(实线框)和μ频段(虚线框)的ERSP结果

参考文献 53

[1]	Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577-660. pmid: 11301525
[2]	Baumeister, J. C., Foroni, F., Conrad, M., Rumiati, R. I., & Winkielman, P. (2017). Embodiment and emotional memory in first vs. second language. Frontiers in Psychology, 8, 394. doi: 10.3389/fpsyg.2017.00394 pmid: 28386240
[3]	Bi, Y. C. (2021). Dual coding of knowledge in the human brain. Trends in Cognitive Sciences, 25(10), 883-895. doi: 10.1016/j.tics.2021.07.006 pmid: 34509366
[4]	Borghesani, V., & Piazza, M. (2017). The neuro-cognitive representations of symbols: The case of concrete words. Neuropsychologia, 105, 4-17. doi: S0028-3932(17)30239-7 pmid: 28648571
[5]	Brunsdon, V. E. A., Bradford, E. E. F., & Ferguson, H. J. (2019). Sensorimotor mu rhythm during action observation changes across the lifespan independently from social cognitive processes. Developmental Cognitive Neuroscience, 38, 100659. doi: 10.1016/j.dcn.2019.100659 URL
[6]	Capone, N. C., & McGregor, K. K. (2005). The effect of semantic representation on toddlers' word retrieval. Journal of Speech Language and Hearing Research, 48(6), 1468-1480. doi: 10.1044/1092-4388(2005/102) URL
[7]	Casasanto, D., & de Bruin, A. (2019). Metaphors we learn by: Directed motor action improves word learning. Cognition, 182, 177-183. doi: S0010-0277(18)30251-8 pmid: 30296659
[8]	Chen, Y., Guo, C., & Jiang, Y. (2011). Neural basis for successful encoding and retrieval of prospective memory. Science China Life Sciences, 41(4), 332-339.
	[陈燕妮, 郭春彦, 姜扬. (2011). 成功的前瞻记忆编码与提取的神经机制研究. 中国科学:生命科学, 41(04), 332-339.]
[9]	Cohen, J. (2013). Statistical power analysis for the behavioral sciences. Routledge.
[10]	Comesana, M., Perea, M., Piñeiro, A., & Fraga, I. (2009). Vocabulary teaching strategies and conceptual representations of words in L2 in children: Evidence with novice learners. Journal of Experimental Child Psychology, 104(1), 22-33. doi: 10.1016/j.jecp.2008.10.004 pmid: 19084237
[11]	Craik, F. I., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104(3), 268-294. doi: 10.1037/0096-3445.104.3.268 URL
[12]	Debnath, R., Salo, V. C., Buzzell, G. A., Yoo, K. H., & Fox, N. A. (2019). Mu rhythm desynchronization is specific to action execution and observation: Evidence from time- frequency and connectivity analysis. Neuroimage, 184, 496-507. doi: S1053-8119(18)31858-5 pmid: 30248457
[13]	Dove, G. (2009). Beyond perceptual symbols: A call for representational pluralism. Cognition, 110(3), 412-431. doi: 10.1016/j.cognition.2008.11.016 pmid: 19135654
[14]	Eschmann, K. C. J., Bader, R., & Mecklinger, A. (2020). Improving episodic memory: Frontal-midline theta neurofeedback training increases source memory performance. Neuroimage, 222, 117219. doi: 10.1016/j.neuroimage.2020.117219 URL
[15]	Foroni, F. (2015). Do we embody second language? Evidence for ‘partial’ simulation during processing of a second language. Brain and Cognition, 99, 8-16. doi: 10.1016/j.bandc.2015.06.006 URL
[16]	Fox, N. A., Bakermans-Kranenburg, M. J., Yoo, K. H., Bowman, L. C., Cannon, E. N., Vanderwert, R. E.,... van Ijzendoorn, M. H. (2016). Assessing human mirror activity with EEG mu rhythm: A meta-analysis. Psychological Bulletin, 142(3), 291-313. doi: 10.1037/bul0000031 pmid: 26689088
[17]	Garbin, G., Costa, A., Sanjuan, A., Forn, C., Rodriguez- Pujadas, A., Ventura, N.,... Ávila, C. (2011). Neural bases of language switching in high and early proficient bilinguals. Brain and Language, 119(3), 129-135. doi: 10.1016/j.bandl.2011.03.011 pmid: 21550652
[18]	Guan, Q. (2019). Embodied linguistics——Language science in the era of artificial intelligence. Beijing: Science Press.
	[官群. (2019). 具身语言学——人工智能时代的语言科学. 北京: 科学出版社.]
[19]	Guo, T., & Peng, D. (2002). The accessing mechanism of the less proficient Chinese-English bilinguals’ conceptual representation. Acta Psychologica Sinica, 35(1), 23-28.
	[郭桃梅, 彭聃龄. (2002). 非熟练中-英双语者的第二语言的语义通达机制. 心理学报, 35(1), 23-28.]
[20]	Hald, L. A., de Nooijer, J., van Gog, T., & Bekkering, H. (2016). Optimizing word learning via links to perceptual and motoric experience. Educational Psychology Review, 28(3), 495-522. doi: 10.1007/s10648-015-9334-2 URL
[21]	Herweg, N. A., Solomon, E. A., & Kahana, M. J. (2020). Theta Oscillations in Human Memory. Trends in Cognitive Sciences, 24(3), 208-227. doi: S1364-6613(19)30294-3 pmid: 32029359
[22]	Holcomb, P. J., Kounios, J., Anderson, J. E., & West, W. C. (1999). Dual-coding, context-availability, and concreteness effects in sentence comprehension: An electrophysiological investigation. Journal of Experimental Psychology: Learning Memory and Cognition, 25(3), 721-742. doi: 10.1037/0278-7393.25.3.721 URL
[23]	Hsieh, L. T., & Ranganath, C. (2014). Frontal midline theta oscillations during working memory maintenance and episodic encoding and retrieval. Neuroimage, 85, 721-729. doi: 10.1016/j.neuroimage.2013.08.003 URL
[24]	Hulstijn, J. (2002). Towards a unified account of the representation, processing and acquisition of second language knowledge. Second Language Research, 18(3), 193-223. doi: 10.1191/0267658302sr207oa URL
[25]	Kutas, M., & Federmeier, K. D. (2000). Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences, 4(12), 463-470. doi: 10.1016/s1364-6613(00)01560-6 pmid: 11115760
[26]	Lai, V. T., Howerton, O., & Desai, R. H. (2019). Concrete processing of action metaphors: Evidence from ERP. Brain Research, 1714, 202-209. doi: S0006-8993(19)30128-3 pmid: 30853628
[27]	Laszlo, S., & Federmeier, K. D. (2011). The N400 as a snapshot of interactive processing: Evidence from regression analyses of orthographic neighbor and lexical associate effects. Psychophysiology, 48(2), 176-186. doi: 10.1111/j.1469-8986.2010.01058.x pmid: 20624252
[28]	Linck, J. A., Kroll, J. F., & Sunderman, G. (2009). Losing access to the native language while immersed in a second language: Evidence for the role of inhibition in second-language learning. Psychological Science, 20(12), 1507-1515. doi: 10.1111/j.1467-9280.2009.02480.x pmid: 19906121
[29]	Louwerse, M., & Hutchinson, S. (2012). Neurological evidence linguistic processes precede perceptual simulation in conceptual processing. Frontiers in Psychology, 3, 385. doi: 10.3389/fpsyg.2012.00385 pmid: 23133427
[30]	Louwerse, M., He, X., Deng, Y., Zhao, X., & Xue, Y. (2017). Symbol interdependency in language processing:Interactions between language statistics and perceptual simulation. Journal of South China Normal University (Social Science Edition), (2), 51-60.
	[Louwerse, M., 何先友, 邓玉梅, 赵雪汝, 薛颖. (2017). 语言加工中的符号相互依存: 语言统计和知觉模拟的交互作用. 华南师范大学学报(社会科学版), (2), 51-60.]
[31]	Louwerse, M. M. (2011). Symbol interdependency in symbolic and embodied cognition. Topics in Cognitive Science, 3(2), 273-302. doi: 10.1111/j.1756-8765.2010.01106.x pmid: 25164297
[32]	Ma, Z. (2015). The motivation and path of conceptual reconstruction in second language vocabulary acquisition. Journal of PLA University of Foreign Languages, 38(3), 74-80.
	[马拯. (2015). 二语词汇习得中概念重构的理据及其路径. 解放军外国语学院学报, 38(3), 74-80.]
[33]	Macedonia, M., Repetto, C., Ischebeck, A., & Mueller, K. (2019). Depth of encoding through observed gestures in foreign language word learning. Frontiers in Psychology, 10, 33. doi: 10.3389/fpsyg.2019.00033 pmid: 30761033
[34]	Maouene, J., Hidaka, S., & Smith, L. (2008). Body parts and early-learned verbs. Cognitive Science, 32(7), 1200-1216. doi: 10.1080/03640210802019997 pmid: 21585449
[35]	Markman, A. B., & Dietrich, E. (2000). Extending the classical view of representation. Trends in Cognitive Sciences, 4(12), 470-475. pmid: 11115761
[36]	Muller, O., Duñabeitia, J. A., & Carreiras, M. (2010). Orthographic and associative neighborhood density effects: What is shared, what is different? Psychophysiology, 47(3), 455-466. doi: 10.1111/j.1469-8986.2009.00960.x pmid: 20102534
[37]	Mulligan, N. W., & Hornstein, S. L. (2003). Memory for actions: Self-performed tasks and the reenactment effect. Memory & Cognition, 31(3), 412-421. doi: 10.3758/BF03194399 URL
[38]	Nittono, H., Suehiro, M., & Hori, T. (2002). Word imageability and N400 in an incidental memory paradigm. International Journal of Psychophysiology, 44(3), 219-229. pmid: 12031296
[39]	Öttl, B., Dudschig, C., & Kaup, B. (2017). Forming associations between language and sensorimotor traces during novel word learning. Language and Cognition, 9(1), 156-171. doi: 10.1017/langcog.2016.5 URL
[40]	Paivio, A. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology/Revue canadienne de psychologie, 45(3), 255-287. doi: 10.1037/h0084295 URL
[41]	Pineda, J. A. (2005). The functional significance of mu rhythms: Translating "seeing" and "hearing" into "doing". Brain Research Reviews, 50(1), 57-68. doi: 10.1016/j.brainresrev.2005.04.005 URL
[42]	Pulvermüller, F. (2013). How neurons make meaning: Brain mechanisms for embodied and abstract-symbolic semantics. Trends in Cognitive Sciences, 17(9), 458-470. doi: 10.1016/j.tics.2013.06.004 pmid: 23932069
[43]	Reilly, J., Peelle, J. E., Garcia, A., & Crutch, S. J. (2016). Linking somatic and symbolic representation in semantic memory: The dynamic multilevel reactivation framework. Psychonomic Bulletin & Review, 23(4), 1002-1014. doi: 10.3758/s13423-015-0824-5 URL
[44]	Rosburg, T., Johansson, M., Weigl, M., & Mecklinger, A. (2015). How does testing affect retrieval-related processes? An event-related potential (ERP) study on the short-term effects of repeated retrieval. Cognitive Affective & Behavioral Neuroscience, 15(1), 195-210.
[45]	Rugg, M. D., Mark, R. E., Walla, P., Schloerscheidt, A. M., Birch, C. S., & Allan, K. (1998). Dissociation of the neural correlates of implicit and explicit memory. Nature, 392(6676), 595-598. doi: 10.1038/33396
[46]	Schwartz, A. I., Yeh, L.-H., & Shaw, M. P. (2008). Lexical representation of second language words: Implications for second language vocabulary acquisition and use. The Mental Lexicon, 3(3), 309-324. doi: 10.1075/ml URL
[47]	Shen, M., Xie, J., Zhang, K., Li, Y., Zeng, C., & Wang, R. (2014). The spatial metaphor of bilingual affective concepts processing. Acta Psychologica Sinica, 46(11), 1671-1681. doi: 10.3724/SP.J.1041.2014.01671
	[沈曼琼, 谢久书, 张昆, 李莹, 曾楚轩, 王瑞明. (2014). 二语情绪概念理解中的空间隐喻. 心理学报, 46(11), 1671-1681.]
[48]	Solomon, E. A., Lega, B. C., Sperling, M. R., & Kahana, M. J. (2019). Hippocampal theta codes for distances in semantic and temporal spaces. Proceedings of the National Academy of Sciences of the United States of America, 116(48), 24343-24352. doi: 10.1073/pnas.1906729116 pmid: 31723043
[49]	Wang, B., Li, Z., Wu, L., & Zhang, J. (2019). Effects of embodied simulation on understanding Chinese body action verbs. Acta Psychologica Sinica, 51(12), 1291-1305. doi: 10.3724/SP.J.1041.2019.01291
	[王斌, 李智睿, 伍丽梅, 张积家. (2019). 具身模拟在汉语肢体动作动词理解中的作用. 心理学报, 51(12), 1291-1305.] doi: 10.3724/SP.J.1041.2019.01291
[50]	Wang, H., Jiang, Z., Feng, X., & Lu, Z. (2020). Spatial iconicity of moral concepts: Co-dependence of linguistic and embodied symbols. Acta Psychologica Sinica, 52(2), 128-138. doi: 10.3724/SP.J.1041.2020.00128
	[王汉林, 蒋泽亮, 冯晓慧, 鲁忠义. (2020). 道德概念的空间形象性: 语言因素和具身因素的共同作用. 心理学报, 52(2), 128-138.] doi: 10.3724/SP.J.1041.2020.00128
[51]	Wang, R. M., Yang, J., & Li, L. (2016). Second language learning. Shanghai: East China Normal University Press.
	[王瑞明, 杨静, 李利. (2016). 第二语言学习. 上海: 华东师范大学出版社.]
[52]	Wu, L., Mo, L., & Wang, R. (2007). Dynamic mental simulations during comprehension of motion descriptions. Advances in Psychological Science, 15(4), 605-612.
	[伍丽梅, 莫雷, 王瑞明. (2007). 有关运动的语言理解的知觉模拟. 心理科学进展, 15(4), 605-612.]
[53]	Zhang, C., Yao, R., & Guan, Q. (2015). Brain mechanisms of foreign language learning difficulty: Evidence from ERP and fMRI studies. Chinese Journal of Special Education, (7), 39-46.
	[张冲, 姚茹, 官群. (2015). 外语学习困难的脑机制研究: 来自ERP和fMRI的证据及启示. 中国特殊教育, (7), 39-46.]