口语句子的韵律边界：窥探言语理解的秘窗

doi:10.3724/SP.J.1042.2021.00425

摘要/Abstract

摘要：

韵律边界加工与言语理解紧密相关, 最近十几年来逐渐成为心理学和语言学的研究焦点。韵律系统包含若干由小到大的韵律单位, 不同单位的韵律成分其边界强度不同, 表现在音高、延宕和停顿三个声学线索上的参数也不同。句子的听力理解过程中, 听话人运用声学线索感知权重策略对韵律边界的声学线索进行加工。从神经层面上来看, 对于韵律边界的加工, 大脑显示出独立且特异性的神经机制。韵律边界的加工能力在婴儿出生后随年龄的增长而发展, 到了老年阶段则逐渐退化, 而且似乎能够对二语迁移。未来, 需要扩大对韵律边界声学表现的考查范围, 进一步明确韵律边界的加工过程, 进一步厘清韵律边界加工和句法加工之间的关系, 进一步关注二语者韵律边界加工能力的发展。

关键词: 韵律边界, 言语理解, 声学表现, 神经振荡

Abstract:

The perception of prosodic boundaries is critical to spoken language comprehension, and it has become a primary research topic among psychologists and psycholinguists in the past decade. Utterances are chunked into prosodic units of different strengths. The boundaries between prosodic units are mainly signaled by acoustic cues like pitch change, final lengthening, and pause. Previous cognitive, linguistic, developmental, and neuroimaging studies have significantly advanced our understanding of the processing of prosodic boundaries. We now know that listeners use a perceptual weighting strategy to process prosodic boundary cues, and there are specific brain mechanisms for prosodic boundary processing. The ability to processing prosodic boundaries steadily develops with age in young children and transfers to a second language, but it generally decreases with age in older adults. Future studies should expand the investigation of prosodic boundaries to more pragmatic genres and focus on revealing the cognitive mechanisms underlying prosodic boundary processing, the relationship between prosodic boundary and syntax processing, and the development of prosodic boundary perception in second language learners.

Key words: prosodic boundary, spoken language comprehension, acoustic characteristics, neural oscillation

中图分类号:

B842

方岚, 郑苑仪, 金晗, 李晓庆, 杨玉芳, 王瑞明. (2021). 口语句子的韵律边界：窥探言语理解的秘窗. 心理科学进展 , 29(3), 425-437.

FANG Lan, ZHENG Yuanyi, JIN Han, LI Xiaoqing, YANG Yufang, WANG Ruiming. (2021). Prosodic boundaries in speech: A window to spoken language comprehension. Advances in Psychological Science, 29(3), 425-437.

参考文献 74

[1]	杨玉芳. (2015). 心理语言学. 北京: 科学出版社.
[2]	周游, 刘方舟. (2017). 汉语韵律结构与语法结构的对比分析. 见:中国中文信息学会语音信息专业委员会(编), 第十四届全国人机语音通讯学术会议(NCMMSC’2017)论文集
[3]	Aasland, W., & Baum, S. (2003). Temporal parameters as cues to phrasal boundaries: A comparison of processing by left-and right-hemisphere brain-damaged individuals. Brain and Language, 87(3), 385-399. URL pmid: 14642541
[4]	Bögels, S., Schriefers, H., Vonk, W., Chwilla, D. J., & Kerkhofs, R. (2010). The interplay between prosody and syntax in sentence processing: The case of subject-and object-control verbs. Journal of Cognitive Neuroscience, 22(5), 1036-1053. doi: 10.1162/jocn.2009.21269 URL pmid: 19445602
[5]	Bögels, S., Schriefers, H., Vonk, W., Chwilla, D. J., & Kerkhofs, R. (2013). Processing consequences of superfluous and missing prosodic breaks in auditory sentence comprehension. Neuropsychologia, 51(13), 2715-2728. doi: 10.1016/j.neuropsychologia.2013.09.008 URL
[6]	Brodbeck, C., Hong, L. E., & Simon, J. Z. (2018). Rapid transformation from auditory to linguistic representations of continuous speech. Current Biology, 28( 24), 3976- 3983. e5.
[7]	Buxó-Lugo, A., & Watson, D. G. (2016). Evidence for the inﬂuence of syntax on prosodic parsing. Journal of Memory and Language, 90, 1-13. URL pmid: 30853752
[8]	Cason, N., Astésano, C., & Schön, D. (2015). Bridging music and speech rhythm: Rhythmic priming and audio-motor training affect speech perception. Acta Psychologica, 155, 43-50. URL pmid: 25553343
[9]	Clahsen, H., & Felser, C. (2018). Some notes on the shallow structure hypothesis. Studies in Second Language Acquisition, 40(3), 693-706. doi: 10.1017/S0272263117000250 URL
[10]	Clifton, C, & Duffy, S. A. (2001). Sentence and text Comprehension: Roles of linguistic structure. Annual Review of Psychology, 52, 167-196. URL pmid: 11148303
[11]	Clifton, C., Carlson, K., & Frazier, L. (2002). Informative prosodic boundaries. Language and Speech, 45(2), 87-114. doi: 10.1177/00238309020450020101 URL
[12]	Cole, J. (2015). Prosody in context: A review. Language, Cognition and Neuroscience, 30(1-2), 1-31. doi: 10.1080/23273798.2014.963130 URL
[13]	Cole, J., Mo, Y., & Bae, S. (2010). The role of syntactic structure in guiding prosody perception with ordinary listeners and everyday speech. Language and Cognitive Process, 25(7-9), 1141-1177. doi: 10.1080/01690960903525507 URL
[14]	Cumming, R., Wilson, A., Leong, V., Colling, L. J., & Goswami, U. (2015). Awareness of rhythm patterns in speech and music in children with specific language impairments. Frontiers in Human Neuroscience, 9, 672. doi: 10.3389/fnhum.2015.00672 URL pmid: 26733848
[15]	Dilley, L. C., & Pitt, M. A. (2010). Altering context speech rate can cause words to appear or disappear. Psychological Science, 21(11), 1664-1670. doi: 10.1177/0956797610384743 URL pmid: 20876883
[16]	Ding, N., & Jin, P. (2019). Low-frequency neural activity reflects rule-based chunking during speech listening. bioRxiv, Article 742585v1. https://www.biorxiv.org/content/10.1101/ 742585v1.full.pdf
[17]	Ding, N., Chatterjee, M., & Simon, J. Z. (2014). Robust cortical entrainment to the speech envelope relies on the spectro-temporal fine structure. Neuroimage, 88, 41-46. URL pmid: 24188816
[18]	Ding, N., Melloni, L., Zhang, H., Tian, X., & Poeppel, D. (2015). Cortical tracking of hierarchical linguistic structures in connected speech. Nature Neuroscience, 19(1), 158-164. doi: 10.1038/nn.4186 URL pmid: 26642090
[19]	Ding, N., Pan, X., Luo, C., Su, N., Zhang, W., & Zhang, J. (2018). Attention is required for knowledge-based sequential grouping: Insights from the integration of syllables into words. Journal of Neuroscience, 38(5), 1178-1188. doi: 10.1523/JNEUROSCI.2606-17.2017 URL
[20]	Doelling, K. B., Arnal, L. H., Ghitza, O., & Poeppel, D. (2014). Acoustic landmarks drive delta-theta oscillations to enable speech comprehension by facilitating perceptual parsing. Neuroimage, 85, 761-768. doi: 10.1016/j.neuroimage.2013.06.035 URL
[21]	Flaugnacco, E., Lopez, L., Terribili, C., Montico, M., Zoia, S., & Schön, D. (2015). Music training increases phonological awareness and reading skills in developmental dyslexia: A randomized control trial. PLOS One, 10( 9), Article e0138715. https://doi.org/10.1371/journal.pone. 0138715
[22]	Frazier, L., Clifton, C., & Carlson, K. (2004). Don't break, or do: Prosodic boundary preferences. Lingua, 114 (1), 3-27. doi: 10.1016/S0024-3841(03)00044-5 URL
[23]	Frazier, L., Carlson, K., & Clifton, C. (2006). Prosodic phrasing is central to language comprehension. Trends in Cognitive Sciences, 10(6), 244-249. doi: 10.1016/j.tics.2006.04.002 URL pmid: 16651019
[24]	Giraud, A. L., & Poeppel, D. (2012). Cortical oscillations and speech processing: Emerging computational principles and operations. Nature Neuroscience, 15(4), 511-517. URL pmid: 22426255
[25]	Gross, J., Hoogenboom, N., Thut, G., Schyns, P., Panzeri, S., Belin, P., & Garrod, S. (2013). Speech rhythms and multiplexed oscillatory sensory coding in the human brain. PLOS Biology, 11(12). Article e1001752. https://doi.org/ 10.1371/journal.pbio.1001752 doi: 10.1371/journal.pbio.1001732 URL pmid: 24339748
[26]	Halle, M., & Stevens, K. (1962). Speech recognition: A model and a program for research. IRE Transactions on Information Theory, 8(2), 155-159.
[27]	Hawthorn, K., & Gerke, L. (2014). From pauses to clauses: Prosody facilitates learning of syntactic constituency. Cognition, 133(2), 420-428. URL pmid: 25151251
[28]	Hidalgo, C., Falk, S., & Schön, D. (2017). Speak on time! Effects of a musical rhythmic training on children with hearing loss. Hearing Research, 351, 11-18. URL pmid: 28552493
[29]	Holliman, A. J., Gutiérrez Palma, N., Critten, S., Wood, C., Cunnane, H., & Pillinger, C. (2017). Examining the independent contribution of prosodic sensitivity to word reading and spelling in early readers. Reading and Writing, 30, 509-521. doi: 10.1007/s11145-016-9687-z URL
[30]	Holzgrefe, J., Wellmann, C., Petrone, C., Truckenbrodt, H., Höhle, B., & Wartenburger, I. (2013). Brain response to prosodic boundary cues depends on boundary position. Frontiers in Psychology Language Sciences, 4, 421.
[31]	Holzgrefe-Lang., J., Wellmann, C., Petrone, C., & Räling, R. (2016). How pitch change and final lengthening cue boundary perception in German: Converging evidence from ERPs and prosodic judgements. Language, Cognition and Neuroscience, 31(7), 904-920.
[32]	Honbolygo, F., Török, Á., Bánréti, Z., Hunyadi, L., & Csépe, V. (2016). ERP correlates of prosody and syntax interaction in case of embedded sentences. Journal of Neurolinguistics, 37, 22-33. doi: 10.1016/j.jneuroling.2017.10.001 URL pmid: 29422720
[33]	Howard, M. F., & Poeppel, D. (2010). Discrimination of speech stimuli based on neuronal response phase patterns depends on acoustics but not comprehension. Journal of Neurophysiology, 104(5), 2500-2511. doi: 10.1152/jn.00251.2010 URL pmid: 20484530
[34]	Hwang, H., & Steinhauer, K. (2011). Phrase length matters: The interplay between implicit prosody and syntax in Korean “garden path” sentences. Journal of Cognitive Neuroscience, 23(11), 3555-3575. URL pmid: 21391765
[35]	Ischebeck, A. K., Friederici, A. D., & Alter, K. (2008). Processing prosodic boundaries in natural and hummed speech: An fMRI study. Cerebral Cortex, 18(3), 541-552. URL pmid: 17591598
[36]	Kayser, S. J., Ince, R. A. A., Gross, J., & Kayser, C. (2015). Irregular speech rate dissociates auditory cortical entrainment, evoked responses, and frontal alpha. Journal of Neuroscience, 35(44), 14691-14701. URL pmid: 26538641
[37]	Kjelgaard, M. M., & Speer, S. R. (1999). Prosodic facilitation and interference in the resolution of temporary syntactic closure ambiguity. Journal of Memory and Language, 40(2), 153-194. doi: 10.1006/jmla.1998.2620 URL
[38]	Kösem, A., & van Wassenhov, V. (2016). Distinct contributions of low- and high-frequency neural oscillations to speech comprehension. Language, Cognition and Neuroscience, 32(5), 536-544.
[39]	Kreiner, H., & Eviatar, Z. (2014). The missing link in the embodiment of syntax: Prosody. Brain and Language, 137, 91-102. URL pmid: 25190329
[40]	Krivokapić, J., & Byrd, D. (2012). Prosodic boundary strength: An articulatory and perceptual study. Journal of Phonetics, 40(3), 430-442. URL pmid: 23441103
[41]	Li, W. J., & Yang, Y. (2009). Perception of prosodic hierarchical boundaries in Mandarin Chinese sentences. Neuroscience, 158(4), 1416-1425. URL pmid: 19111906
[42]	Männel, C., & Friederici, A. D. (2016). Neural correlates of prosodic boundary perception in German preschoolers: If pause is present, pitch can go. Brain Research, 1632, 27-33. doi: 10.1016/j.brainres.2015.12.009 URL pmid: 26683081
[43]	Männel, C., Schipke, C. S., & Friederici, A. D. (2013). The role of pause as a prosodic boundary marker: Language ERP studies in German 3- and 6-year-olds. Developmental Cognitive Neuroscience, 5, 86-94. doi: 10.1016/j.dcn.2013.01.003 URL
[44]	Marcus, M., & Hindle, D. (1990). “Description theory and intonation boundaries”. In G. Altman (Eds.), Cognitive Models of Speech Processing: Psycholinguistic and Computational Perspectives (pp. 483-512). Cambridge, MA, USA: MIT Press.
[45]	Maslowski, M., Meyer, A. S., & Bosker, H. R. (2019). Listeners normalize speech for contextual speech rate even without an explicit recognition task. The Journal of the Acoustical Society of America, 146(1), 179-188. URL pmid: 31370593
[46]	Meyer, L., Henry, M. J., Gaston, P., Schmuck, N., & Friederici, A. D. (2017). Linguistic bias modulates interpretation of speech via neural delta-band oscillations. Cerebral Cortex, 27(9), 4293-4302. doi: 10.1093/cercor/bhw228 URL pmid: 27566979
[47]	Nickels, S., Opitz, B., & Steinhauer, K. (2013). ERPs show that classroom-instructed late second language learners rely on the same prosodic cues in syntactic parsing as native speakers. Neuroscience Letters, 557, 107-111. URL pmid: 24141083
[48]	Nickels, S., & Steinhauer, K. (2018). Prosody-syntax integration in a second language: Contrasting event-related potentials from German and Chinese learners of English using linear mixed effect models. Second Language Research, 34(1), 9-37. doi: 10.1177/0267658316649998 URL
[49]	Pannekamp, A., Toepel, U., Alter, K., Hahne, A., & Friederici, A. D. (2005). Prosody-driven sentence processing: An event-related brain potential study. Journal of Cognitive Neuroscience, 17(3), 407-421. URL pmid: 15814001
[50]	Park, H., Ince, R. A., Schyns, P. G., Thut, G., & Gross, J. (2015). Frontal top-down signals increase coupling of auditory low-frequency oscillations to continuous speech in human listeners. Current Biology, 25(12), 1649-1653. URL pmid: 26028433
[51]	Pauker, E., Itzhak, I., Baum, S. R., & Steinhauer, K. (2011). Effects of cooperating and conflicting prosody in spoken English garden path sentences: ERP evidence for the boundary deletion hypothesis. Journal of Cognitive Neuroscience, 23(10), 2731-2751. URL pmid: 21281091
[52]	Pennington, M. C., & Ellis, N.C. (2000). Cantonese Speakers’ Memory for English Sentences with Prosodic Cues. The Modern Language Journal, 84(3), 372-389. doi: 10.1111/modl.2000.84.issue-3 URL
[53]	Petrone, C., Truckenbrodt, H., Wellmann, C., Holzgrefe- Lang, J., Wartenburger, I., & Hohle, B. (2017). Prosodic boundary cues in German: Evidence from the production and perception of bracketed lists. Journal of Phonetics, 61, 71-92. doi: 10.1016/j.wocn.2017.01.002 URL
[54]	Poeppel, D. (2003). The analysis of speech in different temporal integration windows: Cerebral lateralization as 'asymmetric sampling in time'. Speech Communication, 41(1), 245-255. doi: 10.1016/S0167-6393(02)00107-3 URL
[55]	Qin, Z., Chien, Y., & Tremblay, A. (2016). Processing of word-level stress by Mandarin-speaking second language learners of English. Applied Psycholinguistics, 38(3), 541-570. doi: 10.1017/S0142716416000321 URL
[56]	Roll, M., Lindgren, M., Alter, K., & Horne, M. (2012). Time-driven effects on parsing during reading. Brain and Language, 121(3), 267-272. doi: 10.1016/j.bandl.2012.03.002 URL pmid: 22480626
[57]	Schmidt, E., Pérez, A., Cilibrasi, L., & Tsimpli, I. (2019). Prosody facilitates memory recall in L1 but not in L2 in highly proficient listeners. Studies in Second Language Acquisition, 42(1), 223-238. doi: 10.1017/S0272263119000433 URL
[58]	Schön, D., & Tillmann, B. (2015). Short- and long-term rhythmic interventions: Perspectives for language rehabilitation. Annals of the New York Academy of Sciences, 1337, 32-39. URL pmid: 25773614
[59]	Schremm, A., Horne, M., & Roll, M. (2015). Brain responses to syntax constrained by time-driven implicit prosodic phrases. Journal of Neurolinguistics, 35, 68-84. doi: 10.1016/j.jneuroling.2017.10.001 URL pmid: 29422720
[60]	Seidl, A. (2007). Infants’ use and weighting of prosodic cues in clause segmentation. Journal of Memory & Language, 57(1), 24-48.
[61]	Seidl, A., & Cristià, A. (2008). Developmental changes in the weighting of prosodic cues. Developmental Science, 11(4), 596-606. URL pmid: 18576967
[62]	Selkirk, E. (2005). Comments on intonational phrasing in English. In S. Frota, M. Vigario, and M. J. Freitas (Eds.), Prosodies (pp. 11-58). Berlin, Germany: Mouton de Gruyter.
[63]	Slater, J., & Kraus, N. (2016). The role of rhythm in perceiving speech in noise: A comparison of percussionists, vocalists and non-musicians. Cognitive Processing, 17(1), 79-87. URL pmid: 26445880
[64]	Steinhauer, K., Abada, S. H., Pauker, E., Itzhak, I., & Baum, S. R. (2010). Prosody-syntax interactions in aging: Event-related potentials reveal dissociations between on-line and off-line measures. Neuroscience Letters, 472(2), 133-138. URL pmid: 20138120
[65]	Steinhauer, K., Alter, K., & Friederici, A. D. (1999). Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nature Neuroscience, 2(2), 191-196. doi: 10.1038/5757 URL pmid: 10195205
[66]	Truckenbrodt, H. (1995). Phonological phrases: Their relation to syntax, focus and prominance (Unpublished doctorial Dissertation), Massachusetts Institute of Technology.
[67]	van Ommena, S., Boll-Avetisyanb, N., Larrazaa, S., Wellmannb, C., Bijeljac-Babica, R., Höhleb, B., & Nazzi, T. (2020). Language-specific prosodic acquisition: A comparison of phrase boundary perception by French- and German- learning infants. Journal of Memory and Language, 112, 104-108.
[68]	Wagner, M., & Watson, D. G. (2010). Experimental and theoretical advances in prosody: A review. Language and Cognitive Processes, 25(7-9), 905-945. doi: 10.1080/01690961003589492 URL pmid: 22096264
[69]	Webman-Shafran, R., &Fodor, J. D. (2016). Phrase length and prosody in on-line ambiguity resolution. Journal of Psycholinguist Research, 45(3), 447-474. doi: 10.1007/s10936-015-9358-2 URL
[70]	Xu, G., Zhang, L., Shu, H., Wang, X., & Li, P. (2013). Access to lexical meaning in pitch-flattened Chinese sentences: An fMRI study. Neuropsychologia, 51(3), 550-556. URL pmid: 23262075
[71]	Yang, L. C. (2016, March). Optimizing pronunciation and prosody teaching in second language learning. Paper presented at the meeting of ISAPh 2016 International Symposium on Applied Phonetics, Nagoya, Japan.
[72]	Yang, X., Shen, X., Li, W., & Yang, Y. (2014). How listeners weight acoustic cues to intonational phrase boundaries. PLOS One, 9(7), Article e102166. http://doi.org10.1371/ journal.pone.0102166 URL pmid: 25080093
[73]	Zoefel, B., & VanRullen, R. (2015). Selective perceptual phase entrainment to speech rhythm in the absence of spectral energy fluctuations. Journal of Neuroscience, 35(5), 1954-1964. URL pmid: 25653354
[74]	Zoefel, B., & VanRullen, R. (2016). EEG oscillations entrain their phase to high-level features of speech sound. Neuroimage, 124, 16-23. doi: 10.1016/j.neuroimage.2015.08.054 URL pmid: 26341026