Processing of hierarchical structures in language and music

doi:10.3724/SP.J.1042.2019.02043

Abstract

Abstract:

In language and music development, smaller elements combine into larger units, which eventually form hierarchical structures. Previous studies have found that listeners can segment continuous stream of speech and music into hierarchies and represent them in a hierarchical way. On the basis of perception, listeners can also integrate incoming language and music events into hierarchical structures to form a coherent understanding and ultimately complete the communication smoothly. Future studies should analyze the role of boundary clues in hierarchical structure perception, examine the influencing factors of integration processes at different time scales, and further explore the relationship between language and music hierarchical structure processing.

Key words: language, music, hierarchical structure, perception, integration

CLC Number:

B842

ZHANG Jingjing, YANG Yufang. Processing of hierarchical structures in language and music[J]. Advances in Psychological Science, 2019, 27(12): 2043-2051.

References 54

[1]	马谐, 杨玉芳, 张秋月 . ( 2016). 音乐句法的加工. 科学通报, 61( 10), 1099-1111.
[2]	叶铮, 周晓林 . ( 2006). 音乐之脑. 心理科学进展, 14( 5), 641-647.
[3]	张晶晶, 杨玉芳 . ( 2017). 音乐句法加工的影响因素. 心理科学进展, 25( 11), 1823-1830.
[4]	周临舒, 蒋存梅, 杨玉芳 . ( 2012). 音乐和语言句法认知的比较. 科学通报, 57( 28-29), 2674-2685.
[5]	Berwick, R. C., Friederici, A. D., Chomsky, N., & Bolhuis, J. J . ( 2013). Evolution, brain, and the nature of language. Trends in Cognitive Sciences, 17( 2), 89-98.
[6]	Cutler, A., &Clifton, C. (1999). Comprehending spoken language: A blueprint of the listener. The Neurocognition of Language, 123-166.
[7]	Ding, N., Melloni, L., Zhang, H., Tian, X., & Poeppel, D . ( 2016). Cortical tracking of hierarchical linguistic structures in connected speech. Nature Neuroscience, 19( 1), 158-164.
[8]	Ding, N., & Simon, J. Z . ( 2012). Emergence of neural encoding of auditory objects while listening to competing speakers. Proceedings of the National Academy of Sciences, 109( 29), 11854-11859.
[9]	Doelling, K. B., &Poeppel, D. (2015). Cortical entrainment to music and its modulation by expertise. Proceedings of the National Academy of Sciences, 112( 45), E6233-E6242.
[10]	Egidi, G., & Caramazza, A. (2016). Integration processes compared: Cortical differences for consistency evaluation and passive comprehension in local and global coherence. Journal of Cognitive Neuroscience, 28( 10), 1568-1583.
[11]	Farbood, M. M., Heeger, D. J., Marcus, G., Hasson, U., & Lerner, Y. L . ( 2015). The neural processing of hierarchical structure in music and speech at different timescales. Frontiers in Neuroscience, 9, 157.
[12]	Glushko, A., Steinhauer, K., DePriest, J., & Koelsch, S . ( 2016). Neurophysiological correlates of musical and prosodic phrasing: Shared processing mechanisms and effects of musical expertise. PloS One, 11( 5), e0155300.
[13]	Guo, S., & Koelsch, S. (2015). The effects of supervised learning on event-related potential correlates of music- syntactic processing. Brain Research, 1626, 232-246.
[14]	Guo, S., & Koelsch, S. (2016). Effects of veridical expectations on syntax processing in music: Event-related potential evidence. Scientific Reports, 6, 19064.
[15]	Hagoort, P., & van Berkum, J. J. A, . ( 2007). Beyond the sentence given. Philosophical Transactions of the Royal Society B: Biological Sciences, 362( 1481), 801-811.
[16]	Hwang, H., & Steinhauer, K. (2011). Phrase length matters: The interplay between implicit prosody and syntax in Korean “garden path” sentences. Journal of Cognitive Neuroscience, 23( 11), 3555-3575.
[17]	Inubushi, T., &Sakai, K. L . ( 2013). Functional and anatomical correlates of word-, sentence-, and discourse-level integration in sign language. Frontiers in Human Neuroscience, 7, 681.
[18]	Jentschke, S., Friederici, A. D., & Koelsch, S . ( 2014). Neural correlates of music-syntactic processing in two-year old children. Developmental Cognitive Neuroscience, 9, 200-208.
[19]	Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY, US: Cambridge university press.
[20]	Knösche, T. R., Neuhaus, C., Haueisen, J., Alter, K., Maess, B., Witte, O. W., & Friederici, A. D . ( 2005). Perception of phrase structure in music. Human Brain Mapping, 24( 4), 259-273.
[21]	Koelsch, S., Gunter, T., Friederici, A. D., & Schröger, E . ( 2000). Brain indices of music processing: “Nonmusicians” are musical. Journal of Cognitive Neuroscience, 12( 3), 520-541.
[22]	Koelsch, S., Rohrmeier, M., Torrecuso, R., & Jentschke, S . ( 2013). Processing of hierarchical syntactic structure in music. Proceedings of the National Academy of Sciences, 110( 38), 15443-15448.
[23]	Koelsch, S., Vuust, P., & Friston, K . ( 2018). Predictive processes and the peculiar case of music. Trends in Cognitive Sciences, 23( 1), 63-77.
[24]	Lagrois, M.-É., Peretz, I., & Zendel, B. R . ( 2018). Neurophysiological and behavioral differences between older and younger adults when processing violations of tonal structure in music. Frontiers in Neuroscience, 12, 54.
[25]	Lerdahl, F., & Jackendoff, R. S . ( 1983). A generative theory of tonal music. Cambridge, MA: MIT press.
[26]	Lerner, Y., Honey, C. J., Silbert, L. J., & Hasson, U . ( 2011). Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. Journal of Neuroscience, 31( 8), 2906-2915.
[27]	Li, W., & Yang, Y. (2009). Perception of prosodic hierarchical boundaries in Mandarin Chinese sentences. Neuroscience, 158( 4), 1416-1425.
[28]	Li, W., & Yang, Y. (2010). Perception of Chinese poem and its electrophysiological effects. Neuroscience, 168( 3), 757-768.
[29]	Luo, H., & Poeppel, D. (2007). Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex. Neuron, 54( 6), 1001-1010.
[30]	Ma, X., Ding, N., Tao, Y., & Yang, Y. F . ( 2018 a). Differences in neurocognitive mechanisms underlying the processing of center-embedded and non-embedded musical structures. Frontiers in Human Neuroscience, 12.
[31]	Ma, X., Ding, N., Tao, Y., & Yang, Y. F . ( 2018 b). Syntactic complexity and musical proficiency modulate neural processing of non-native music. Neuropsychologia, 121, 164-174.
[32]	Maess, B., Koelsch, S., Gunter, T. C., & Friederici, A. D . ( 2001). Musical syntax is processed in Broca's area: An MEG study. Nature Neuroscience, 4( 5), 540-545.
[33]	Nan, Y., Knösche, T. R., & Friederici, A. D . ( 2009). Non- musicians’ perception of phrase boundaries in music: A cross- cultural ERP study. Biological Psychology, 82( 1), 70-81.
[34]	Nan, Y., Knösche, T. R., Zysset, S., & Friederici, A. D . ( 2008). Cross‐cultural music phrase processing: An fMRI study. Human Brain Mapping, 29( 3), 312-328.
[35]	Neuhaus, C., Knösche, T. R., & Friederici, A. D . ( 2006). Effects of musical expertise and boundary markers on phrase perception in music. Journal of Cognitive Neuroscience, 18( 3), 472-493.
[36]	Nozaradan, S., Peretz, I., Missal, M., & Mouraux, A . ( 2011). Tagging the neuronal entrainment to beat and meter. Journal of Neuroscience, 31( 28), 10234-10240.
[37]	Pannekamp, A., Toepel, U., Alter, K., Hahne, A., & Friederici, A. D . ( 2005). Prosody-driven sentence processing: An event- related brain potential study. Journal of Cognitive Neuroscience, 17( 3), 407-421.
[38]	Patel, A. D . ( 2010). Music, language, and the brain. New York, NY: Oxford university press.
[39]	Patel, A. D., Gibson, E., Ratner, J., Besson, M., & Holcomb, P. J . ( 1998). Processing syntactic relations in language and music: An event-related potential study. Journal of Cognitive Neuroscience, 10( 6), 717-733.
[40]	Patel, A. D., & Morgan, E. (2017). Exploring cognitive relations between prediction in language and music. Cognitive Science, 41( s2), 303-320.
[41]	Pauker, E., Itzhak, I., Baum, S. R., & Steinhauer, K . ( 2011). Effects of cooperating and conflicting prosody in spoken English garden path sentences: ERP evidence for the boundary deletion hypothesis. Journal of Cognitive Neuroscience, 23( 10), 2731-2751.
[42]	Peelle, J. E., Gross, J., & Davis, M. H . ( 2012). Phase-locked responses to speech in human auditory cortex are enhanced during comprehension. Cerebral Cortex, 23( 6), 1378-1387.
[43]	Poulin-Charronnat, B., Bigand, E., & Koelsch, S . ( 2006). Processing of musical syntax tonic versus subdominant: An event-related potential study. Journal of Cognitive Neuroscience, 18( 9), 1545-1554.
[44]	Silva, S., Branco, P., Barbosa, F., Marques-Teixeira, J., Petersson, K. M., & Castro, S. L . ( 2014). Musical phrase boundaries, wrap-up and the closure positive shift. Brain Research, 1585, 99-107.
[45]	Steinhauer, K., Alter, K., & Friederici, A. D . ( 1999). Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nature Neuroscience, 2( 2), 191-196.
[46]	Tillmann, B., & Bigand, E. (2004). The relative importance of local and global structures in music perception. The Journal of Aesthetics and Art Criticism, 62( 2), 211-222.
[47]	van Berkum, J. J. A., Hagoort, P., & Brown, C. M . ( 1999). Semantic integration in sentences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience, 11( 6), 657-671.
[48]	van Berkum, J. J. A., Zwitserlood, P., Hagoort, P., & Brown, C. M . ( 2003). When and how do listeners relate a sentence to the wider discourse? Evidence from the N400 effect. Cognitive Brain Research, 17( 3), 701-718.
[49]	van Petten, C., & Bloom, P. (1999). Speech boundaries, syntax and the brain. Nature Neuroscience, 2( 2), 103-104.
[50]	Zendel, B. R., Lagrois, M.-É., Robitaille, N., & Peretz, I . ( 2015). Attending to pitch information inhibits processing of pitch information: The curious case of amusia. Journal of Neuroscience, 35( 9), 3815-3824.
[51]	Zhang, J. J., Jiang, C. M., Zhou, L. S., & Yang, Y. F . ( 2016). Perception of hierarchical boundaries in music and its modulation by expertise. Neuropsychologia, 91, 490-498.
[52]	Zhang, J. J., Zhou, X. F., Chang, R. H., & Yang, Y. F . ( 2018). Effects of global and local contexts on chord processing: An ERP study. Neuropsychologia, 109, 149-154.
[53]	Zhang, Y., Jiang, X. M., Saalbach, H., & Zhou, X. L . ( 2011). Multiple constraints on semantic integration in a hierarchical structure: ERP evidence from German. Brain Research, 1410, 89-100.
[54]	Zhou, X. L., Jiang, X. M., Ye, Z., Zhang, Y. X., Lou, K. Y., & Zhan, W. D . ( 2010). Semantic integration processes at different levels of syntactic hierarchy during sentence comprehension: An ERP study. Neuropsychologia, 48( 6), 1551-1562.