心理科学进展 ›› 2024, Vol. 32 ›› Issue (3): 499-513.doi: 10.3724/SP.J.1042.2024.00499
收稿日期:
2023-08-04
出版日期:
2024-03-15
发布日期:
2024-01-19
通讯作者:
蒋晓鸣, E-mail: xiaoming.jiang@shisu.edu.cn
基金资助:
HU Yanbing1, JIANG Xiaoming1,2()
Received:
2023-08-04
Online:
2024-03-15
Published:
2024-01-19
摘要:
嗓音模仿是言语交流中关键的认知过程, 即对话一方(说话人)将感知到的另一方(目标说话人)的嗓音信号映射为自己的发声器官运动表征, 达到发声器官运动表征复制和目标说话人嗓音再现的目的。成像结果表明, 嗓音模仿的认知加工涉及颞上回到左侧额下回, 再到发声相关初级运动皮层的神经网络, 并且基底神经节在该网络中发挥协调作用。嗓音辨别能力、嗓音信号至发声运动表征的映射能力以及发声器官肌肉的控制能力的个体差异都会影响嗓音模仿的认知加工。未来研究应该考虑将嗓音模仿与发声障碍以及侵入电极技术结合起来, 旨在共同揭示脑与行为的因果机制, 并进一步应用于言语的终身发展、认知可塑性以及言语预期领域。
中图分类号:
胡砚冰, 蒋晓鸣. (2024). 嗓音模仿认知神经加工的多阶段模型. 心理科学进展 , 32(3), 499-513.
HU Yanbing, JIANG Xiaoming. (2024). Multi-stage model of neurocognitive processing for vocal imitation. Advances in Psychological Science, 32(3), 499-513.
[1] |
蔡笑, 张清芳. (2020). 言语运动系统中前馈和反馈控制整合加工的作用机制. 心理科学进展, 28(4), 588-603.
doi: 10.3724/SP.J.1042.2020.00588 |
[2] |
Adank, P., Hagoort, P., & Bekkering, H. (2010). Imitation improves language comprehension. Psychological Science, 21(12), 1903-1909.
doi: 10.1177/0956797610389192 pmid: 21135202 |
[3] |
Belyk, M., Brown, R., Beal, D. S., Roebroeck, A., McGettigan, C., Guldner, S., & Kotz, S. A. (2021). Human larynx motor cortices coordinate respiration for vocal- motor control. NeuroImage, 239, 118326.
doi: 10.1016/j.neuroimage.2021.118326 URL |
[4] |
Belyk, M., & Brown, S. (2017). The origins of the vocal brain in humans. Neuroscience & Biobehavioral Reviews, 77, 177-193.
doi: 10.1016/j.neubiorev.2017.03.014 URL |
[5] | Belyk, M., Eichert, N., & McGettigan, C. (2021). A dual larynx motor networks hypothesis. Philosophical Transactions of the Royal Society B: Biological Sciences, 376(1840), 20200392. |
[6] |
Belyk, M., Johnson, J. F., & Kotz, S. A. (2018). Poor neuro-motor tuning of the human larynx: A comparison of sung and whistled pitch imitation. Royal Society Open Science, 5(4), 171544.
doi: 10.1098/rsos.171544 URL |
[7] |
Belyk, M., Lee, Y. S., & Brown, S. (2018). How does human motor cortex regulate vocal pitch in singers? Royal Society Open Science, 5(8), 172208.
doi: 10.1098/rsos.172208 URL |
[8] | Belyk, M., & McGettigan, C. (2022). Real-time magnetic resonance imaging reveals distinct vocal tract configurations during spontaneous and volitional laughter. Philosophical Transactions of the Royal Society B, 377(1863), 20210511. |
[9] |
Belyk, M., Pfordresher, P. Q., Liotti, M., & Brown, S. (2016). The neural basis of vocal pitch imitation in humans. Journal of Cognitive Neuroscience, 28(4), 621-635.
doi: 10.1162/jocn_a_00914 pmid: 26696298 |
[10] | Belyk, M., Schultz, B. G., Correia, J., Beal, D. S., & Kotz, S. A. (2019). Whistling shares a common tongue with speech: Bioacoustics from real-time MRI of the human vocal tract. Proceedings of the Royal Society B: Biological Sciences, 286(1911), 20191116. |
[11] |
Bernhold, Q. S., & Giles, H. (2020). Vocal accommodation and mimicry. Journal of Nonverbal Behavior, 44(1), 41-62.
doi: 10.1007/s10919-019-00317-y |
[12] |
Bono, D., Belyk, M., Longo, M. R., & Dick, F. (2022). Beyond language: The unspoken sensory-motor representation of the tongue in non-primates, non-human and human primates. Neuroscience & Biobehavioral Reviews, 139, 104730.
doi: 10.1016/j.neubiorev.2022.104730 URL |
[13] | Brown, S., Yuan, Y., & Belyk, M. (2021). Evolution of the speech-ready brain: The voice/jaw connection in the human motor cortex. Journal of Comparative Neurology, 529(5), 1018-1028. |
[14] |
Cartei, V., Oakhill, J., Garnham, A., Banerjee, R., & Reby, D. (2020). “This is what a mechanic sounds like”: Children’s vocal control reveals implicit occupational stereotypes. Psychological Science, 31(8), 957-967.
doi: 10.1177/0956797620929297 URL |
[15] | Chen, T., Lammert, A. C., & Parrell, B. (2021). Modeling sensorimotor adaptation in speech through alterations to forward and inverse models. Interspeech, 3201-3205. |
[16] | Chomsky, N., & Lightfoot, D. W. (2002). Syntactic structures. Walter de Gruyter. |
[17] |
Cohn, M., Segedin, B. F., & Zellou, G. (2022). Acoustic- phonetic properties of Siri- and human-directed speech. Journal of Phonetics, 90, 101123.
doi: 10.1016/j.wocn.2021.101123 URL |
[18] |
Cracco, E., Bardi, L., Desmet, C., Genschow, O., Rigoni, D., de Coster, L.,... Brass, M. J. P. B. (2018). Automatic imitation: A meta-analysis. Psychological Bulletin, 144(5), 453-500.
doi: 10.1037/bul0000143 pmid: 29517262 |
[19] |
Drake, E., & Corley, M. (2015). Articulatory imaging implicates prediction during spoken language comprehension. Memory & Cognition, 43(8), 1136-1147.
doi: 10.3758/s13421-015-0530-6 URL |
[20] |
Dufour, S., & Nguyen, N. (2013). How much imitation is there in a shadowing task? Frontiers in Psychology, 4, 346.
doi: 10.3389/fpsyg.2013.00346 pmid: 23801974 |
[21] |
Frühholz, S., & Schweinberger, S. R. (2021). Nonverbal auditory communication - Evidence for integrated neural systems for voice signal production and perception. Progress in Neurobiology, 199, 101948.
doi: 10.1016/j.pneurobio.2020.101948 URL |
[22] |
Galantucci, B., Fowler, C. A., & Goldstein, L. (2009). Perceptuomotor compatibility effects in speech. Attention, Perception, & Psychophysics, 71(5), 1138-1149.
doi: 10.3758/APP.71.5.1138 URL |
[23] |
Gambi, C., van de Cavey, J., & Pickering, M. J. (2022). Representation of others’ synchronous and asynchronous sentences interferes with sentence production. Quarterly Journal of Experimental Psychology, 76(1), 180-195.
doi: 10.1177/17470218221080766 URL |
[24] | Gandolfi, G., Pickering, M. J., & Garrod, S. (2022). Mechanisms of alignment: Shared control, social cognition and metacognition. Philosophical Transactions of the Royal Society B: Biological Sciences, 378(1870), 20210362. |
[25] |
Garnier, M., Lamalle, L., & Sato, M. (2013). Neural correlates of phonetic convergence and speech imitation. Frontiers in Psychology, 4, 600.
doi: 10.3389/fpsyg.2013.00600 pmid: 24062704 |
[26] |
Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105(2), 251-279.
pmid: 9577239 |
[27] |
Herbst, C. T. (2020). Electroglottography - An update. Journal of Voice, 34(4), 503-526.
doi: S0892-1997(18)30461-2 pmid: 30871855 |
[28] |
Heyes, C. (2001). Causes and consequences of imitation. Trends in Cognitive Sciences, 5(6), 253-261.
pmid: 11390296 |
[29] |
Heyes, C. (2011). Automatic imitation. Psychological Bulletin, 137(3), 463-483.
doi: 10.1037/a0022288 pmid: 21280938 |
[30] | Heyes, C. (2021). Imitation. Current Biology, 31(5), R228- R232. |
[31] |
Hickok, G., & Poeppel, D. (2000). Towards a functional neuroanatomy of speech perception. Trends in Cognitive Sciences, 4(4), 131-138.
pmid: 10740277 |
[32] |
Hickok, G., & Poeppel, D. (2004). Dorsal and ventral streams: A framework for understanding aspects of the functional anatomy of language. Cognition, 92(1-2), 67-99.
doi: 10.1016/j.cognition.2003.10.011 pmid: 15037127 |
[33] |
Janssen, N., Kessels, R. P. C., Mars, R. B., Llera, A., Beckmann, C. F., & Roelofs, A. (2023). Dissociating the functional roles of arcuate fasciculus subtracts in speech production. Cerebral Cortex, 33(6), 2539-2547.
doi: 10.1093/cercor/bhac224 URL |
[34] |
Kim, D., & Clayards, M. (2019). Individual differences in the link between perception and production and the mechanisms of phonetic imitation. Language, Cognition and Neuroscience, 34(6), 769-786.
doi: 10.1080/23273798.2019.1582787 URL |
[35] |
Kinzler, K. D. (2021). Language as a social cue. Annual Review of Psychology, 72(1), 241-264.
doi: 10.1146/psych.2021.72.issue-1 URL |
[36] | Kuhlen, A. K., & Abdel Rahman, R. (2023). Beyond speaking: Neurocognitive perspectives on language production in social interaction. Philosophical Transactions of the Royal Society B: Biological Sciences, 378(1875), 20210483. |
[37] |
Lin, J.-F. L., Imada, T., Meltzoff, A. N., Hiraishi, H., Ikeda, T., Takahashi, T.,... Kuhl, P. K. (2023). Dual-MEG interbrain synchronization during turn-taking verbal interactions between mothers and children. Cerebral Cortex, 33(7), 4116-4134.
doi: 10.1093/cercor/bhac330 URL |
[38] | Liu, Y., Zhao, Z., Xu, M., Yu, H., Zhu, Y., Zhang, J.,... Wu, J. (2023). Decoding and synthesizing tonal language speech from brain activity. Science Advances, 9(23), eadh0478. |
[39] |
Mai, G., & Howell, P. (2023). The possible role of early- stage phase-locked neural activities in speech-in-noise perception in human adults across age and hearing loss. Hearing Research, 427, 108647.
doi: 10.1016/j.heares.2022.108647 URL |
[40] |
Martin, C. D., Branzi, F. M., & Bar, M. (2018). Prediction is production: The missing link between language production and comprehension. Scientific Reports, 8(1), 1079.
doi: 10.1038/s41598-018-19499-4 pmid: 29348611 |
[41] | Mercado, E., Mantell, J. T., & Pfordresher, P. Q. (2014). Imitating sounds: A cognitive approach to understanding vocal imitation. Comparative Cognition & Behavior Reviews, 9, 1-57. |
[42] |
Mousikou, P., Strycharczuk, P., Turk, A., & Scobbie, J. M. (2021). Coarticulation across morpheme boundaries: An ultrasound study of past-tense inflection in Scottish English. Journal of Phonetics, 88, 101101.
doi: 10.1016/j.wocn.2021.101101 URL |
[43] |
Nguyen, N., & Delvaux, V. (2015). Role of imitation in the emergence of phonological systems. Journal of Phonetics, 53, 46-54.
doi: 10.1016/j.wocn.2015.08.004 URL |
[44] |
Pardo, J. S., Jordan, K., Mallari, R., Scanlon, C., & Lewandowski, E. (2013). Phonetic convergence in shadowed speech: The relation between acoustic and perceptual measures. Journal of Memory and Language, 69(3), 183-195.
doi: 10.1016/j.jml.2013.06.002 URL |
[45] |
Pardo, J. S., Pellegrino, E., Dellwo, V., & Möbius, B. (2022). Special issue: Vocal accommodation in speech communication. Journal of Phonetics, 95, 101196.
doi: 10.1016/j.wocn.2022.101196 URL |
[46] | Pardo, J. S., & Remez, R. E. (2021). On the relation between speech perception and speech production. In J. S. Pardo, L. C. Nygaard, R. E. Remez, & D. B. Pisoni (Eds.), The Handbook of Speech Perception (pp.632-655). Wiley Online Library. https://doi.org/10.1002/9781119184096.ch23 |
[47] |
Pardo, J. S., Urmanche, A., Wilman, S., & Wiener, J. (2017). Phonetic convergence across multiple measures and model talkers. Attention, Perception, & Psychophysics, 79(2), 637-659.
doi: 10.3758/s13414-016-1226-0 URL |
[48] |
Paroni, A., Henrich Bernardoni, N., Savariaux, C., Lœvenbruck, H., Calabrese, P., Pellegrini, T.,... Gerber, S. (2021). Vocal drum sounds in human beatboxing: An acoustic and articulatory exploration using electromagnetic articulography. The Journal of the Acoustical Society of America, 149(1), 191-206.
doi: 10.1121/10.0002921 URL |
[49] |
Perrachione, T. K., Del Tufo, S. N., & Gabrieli, J. D. (2011). Human voice recognition depends on language ability. Science, 333(6042), 595.
doi: 10.1126/science.1207327 pmid: 21798942 |
[50] |
Peschke, C., Ziegler, W., Kappes, J., & Baumgaertner, A. (2009). Auditory-motor integration during fast repetition: The neuronal correlates of shadowing. NeuroImage, 47(1), 392-402.
doi: 10.1016/j.neuroimage.2009.03.061 pmid: 19345269 |
[51] |
Pfordresher, P. Q., & Mantell, J. T. (2014). Singing with yourself: Evidence for an inverse modeling account of poor-pitch singing. Cognitive Psychology, 70, 31-57.
doi: 10.1016/j.cogpsych.2013.12.005 pmid: 24480454 |
[52] |
Pickering, M. J., & Gambi, C. (2018). Predicting while comprehending language: A theory and review. Psychological Bulletin, 144(10), 1002-1044.
doi: 10.1037/bul0000158 pmid: 29952584 |
[53] |
Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27(2), 169-190.
pmid: 15595235 |
[54] |
Pickering, M. J., & Garrod, S. (2013). An integrated theory of language production and comprehension. Behavioral and Brain Sciences, 36(4), 329-347.
doi: 10.1017/S0140525X12001495 pmid: 23789620 |
[55] |
Pisanski, K., Cartei, V., McGettigan, C., Raine, J., & Reby, D. (2016). Voice modulation: A window into the origins of human vocal control? Trends in Cognitive Sciences, 20(4), 304-318.
doi: S1364-6613(16)00020-6 pmid: 26857619 |
[56] |
Shmuelof, L., & Krakauer, J. W. (2011). Are we ready for a natural history of motor learning? Neuron, 72(3), 469-476.
doi: 10.1016/j.neuron.2011.10.017 pmid: 22078506 |
[57] |
Stansbury, A. L., & Janik, V. M. (2019). Formant modification through vocal production learning in gray seals. Current Biology, 29(13), 2244-2249.e4.
doi: S0960-9822(19)30685-2 pmid: 31231051 |
[58] |
Stoeger, A. S., Mietchen, D., Oh, S., de Silva, S., Herbst, C. T., Kwon, S., & Fitch, W. T. (2012). An Asian elephant imitates human speech. Current Biology, 22(22), 2144-2148.
doi: 10.1016/j.cub.2012.09.022 pmid: 23122846 |
[59] |
Virhia, J., Kotz, S. A., & Adank, P. (2019). Emotional state dependence facilitates automatic imitation of visual speech. Quarterly Journal of Experimental Psychology, 72(12), 2833-2847.
doi: 10.1177/1747021819867856 URL |
[60] | Waters, S., Kanber, E., Lavan, N., Belyk, M., Carey, D., Cartei, V.,... McGettigan, C. (2021). Singers show enhanced performance and neural representation of vocal imitation. Philosophical Transactions of the Royal Society B: Biological Sciences, 376(1840), 20200399. |
[61] |
Wilt, H., Wu, Y., Trotter, A., & Adank, P. (2023). Automatic imitation of human and computer-generated vocal stimuli. Psychonomic Bulletin & Review, 30(3), 1093-1102.
doi: 10.3758/s13423-022-02218-6 |
[62] |
Wu, Y., Evans, B. G., & Adank, P. (2019). Sensorimotor training modulates automatic imitation of visual speech. Psychonomic Bulletin & Review, 26(5), 1711-1718.
doi: 10.3758/s13423-019-01623-8 |
[63] | Yu, A. C. L., Abrego-Collier, C., & Sonderegger, M. (2013). Phonetic imitation from an individual-difference perspective: Subjective attitude, personality and “autistic” traits. PLOS ONE, 8(9), e74746. |
[64] |
Zellou, G., Cohn, M., & Kline, T. (2021). The influence of conversational role on phonetic alignment toward voice-AI and human interlocutors. Language, Cognition and Neuroscience, 36(10), 1298-1312.
doi: 10.1080/23273798.2021.1931372 URL |
[65] |
Zhang, J., Liu, D.-Q., Qian, S., Qu, X., Zhang, P., Ding, N., & Zang, Y.-F. (2022). The neural correlates of amplitude of low-frequency fluctuation: A multimodal resting-state MEG and fMRI-EEG study. Cerebral Cortex, 33(4), 1119-1129.
doi: 10.1093/cercor/bhac124 URL |
[66] | Zhang, L., Wang, X., Alain, C., & Du, Y. (2023). Successful aging of musicians: Preservation of sensorimotor regions aids audiovisual speech-in-noise perception. Science Advances, 9(17), eadg7056. |
[1] | 张凤翔, 陈美璇, 蒲艺, 孔祥祯. 空间导航能力个体差异的多层次形成机制[J]. 心理科学进展, 2023, 31(9): 1642-1664. |
[2] | 周广方, 金花. 精准功能磁共振成像揭示个体化脑功能网络组织[J]. 心理科学进展, 2023, 31(11): 2078-2091. |
[3] | 包寒吴霜, 蔡华俭. 姓名对个体心理与行为的实际影响:证据和理论[J]. 心理科学进展, 2021, 29(6): 1067-1085. |
[4] | 钱柳, 汝涛涛, 罗雪, 牛佳兴, 马永骏, 周国富. 睡眠限制对认知功能的影响及其潜在作用机制[J]. 心理科学进展, 2020, 28(9): 1493-1507. |
[5] | 赵鹤宾, 夏勉, 曹奔, 江光荣. 接触干预在减少精神障碍公众污名中的应用[J]. 心理科学进展, 2019, 27(5): 843-857. |
[6] | 陆静怡, 尚雪松. 为他人做决策:多维度心理机制与决策体验[J]. 心理科学进展, 2018, 26(9): 1545-1552. |
[7] | 叶晓燕, 张得龙, 常松, 刘鸣. 视觉表象个体差异及其神经基础[J]. 心理科学进展, 2018, 26(7): 1186-1192. |
[8] | 叶晓燕, 张得龙, 倪冰, 董霁月, 蔡红杰. 基于心理旋转范式探讨视觉表象的个体差异[J]. 心理科学进展, 2017, 25(suppl.): 37-37. |
[9] | 冯雪;彭凯平. 技能和风格:理性思维的两种测量途径[J]. 心理科学进展, 2015, 23(9): 1550-1559. |
[10] | 王妍;杨娟. 人格特质对个体心理性应激反应的调节作用[J]. 心理科学进展, 2015, 23(8): 1453-1460. |
[11] | 张秀平;杨晓虹;杨玉芳. 语篇理解中语义整合的神经机制及其影响因素[J]. 心理科学进展, 2015, 23(7): 1130-1141. |
[12] | 邱方晖;罗跃嘉;贾世伟. 面部表情的类别知觉及其个体差异[J]. 心理科学进展, 2015, 23(6): 946-958. |
[13] | 左婷婷;胡清芬. 空间认知风格及其与空间能力的关系[J]. 心理科学进展, 2015, 23(6): 959-966. |
[14] | 李彬;徐富明;王伟;张慧;罗寒冰. 决策的加工过程及个体差异:模糊痕迹理论视角[J]. 心理科学进展, 2015, 23(2): 316-324. |
[15] | 江俊;王梓梦;万璇;蒋存梅. 音乐时间加工的影响因素[J]. 心理科学进展, 2014, 22(4): 650-658. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||