Please wait a minute...
Acta Psychologica Sinica    2018, Vol. 50 Issue (12) : 1346-1355     DOI: 10.3724/SP.J.1041.2018.01346
Reports of Empirical Studies |
Effects of lyrics on the processing of musical emotion: Behavioural and ERP study
ZHANG Weixia1,WANG Wanqi2,ZHOU Linshu3,JIANG Cunmei3()
1 Department of Psychology, College of Education, Shanghai Normal University, Shanghai 200234, China
2 School of Finance and Business, Shanghai Normal University, Shanghai 200234, China
3 Music College, Shanghai Normal University, Shanghai 200234, China
Download: PDF(1002 KB)   HTML Review File (1 KB) 
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks     Supporting Info
Guide   
Abstract  

Music and language are unique to the human beings. It has been suggested that music and language have a common origin as an emotional protolanguage. The development of socialisation has resulted in the development of language into a symbolic communication system with explicit semantics. By contrast, music has become an important means of emotional expression. However, whether language with explicit semantics affects the emotional processing of music remains uncertain. Given that songs contain melody and lyrics, previous behavioural studies have focused on songs to analyse the influence of lyrics on the processing of musical emotion. However, several studies have also shown the influence of lyrics, although such findings are relatively contradictory.

Thus, the current study used behavioural and electrophysiological measurements to investigate the impact of lyrics on the processing of musical emotion. Experiment 1 analysed whether the emotional connotations in music with and without lyrics could be perceived by listeners at the behavioural level. Experiment 2 further investigated whether there are different neural responses to emotions conveyed by melodies with and without lyrics.

A cross-modal affective priming paradigm was used in Experiments 1 and 2, in which musical excerpts served as the prime and emotional faces as target. To avoidthe impact of familiarity, 120 musical stimuli were selected from European opera. Each was sung by a vocalist with and without lyrics, thereby resulting in 240 musical stimuli in two versions as potential prime stimuli. A total of 160 facial expressions affectively congruent or incongruent with the preceding musical stimuli were selected as potential target stimuli. Three pre-tests were conducted to ensure the validity of the stimuli. Eventually, 60 musical stimuli for each music version were selected as the prime stimuli, whilst 120 images were used as the target stimuli, thereby resulting in 240 music-image pairs. To ensure that each stimulus appears only once for each participant, two lists were prepared using a Latin square design. Each prime and target was presented in either the congruent or incongruent condition within each list. Thus, each list comprised 120 trials, with 30 trials in each condition. During the experiment, the two lists were equally distributed across the participants. A total of 40 healthy adults participated in Experiment 1. They were asked to judge as quickly and accurately as possible whether the emotion of the target was happy or sad. The accuracy and reaction time were collected. Meanwhile, 20 healthy adults participated in Experiment 2. They were required to judge whether the emotion between music and image was congruent or incongruent whilst their EEG waveforms were recorded. ERPs were analysed and compared between conditions at the time windows of 250~450 ms and 500~700 ms after the onset of the target.

The Experiment 1 results showed that when faces were primed by music either with or without lyrics, the participants responded faster and more accurately under affectively congruent condition compared with affectively incongruent condition. This finding indicated that the emotional connotations in music with and without lyrics could both be perceived. The ERP results in Experiment 2 showed that distinct neural mechanisms were activated by music with and without lyrics. Specifically, when faces were primed by music without lyrics, a larger N400 was elicited in response to affectively incongruent pairs than to affectively congruent pairs at the time window of 250~450 ms. However, when faces were primed by music with lyrics, a more positive LPC was observed in response to the affectively incongruent pairs than to the affectively congruent pairs at 500~700 ms. This finding confirms the results of Experiment 1, thereby suggesting that the emotion conveyed by music with and without lyrics could be perceived by the listeners. Moreover, the emotional processing between music with and without lyrics differs in the time course of neural processing. That is, the emotional processing of music with lyrics lagged behind that of music without lyrics.

In conclusion, the present results suggest that the neural processing of emotional connotations in music without lyrics preceded that of music with lyrics, although the emotional connotations conveyed by music with and without lyrics could both be perceived. These findings also supported theory of musical philosophy, which suggests that music without lyrics can express emotion more immediately and more directly than music with lyrics owing to the lack of “translation” from the propositional system. On the other hand, considering that lyrics influenced the time course of emotional processing in music with lyrics, our results also provide evidence that the emotional processing of music and language may share neural resources to some extent.

Keywords musical emotion      language      lyrics      N400      LPC     
ZTFLH:  B842  
Issue Date: 30 October 2018
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
ZHANG Weixia
WANG Wanqi
ZHOU Linshu
JIANG Cunmei
Cite this article:   
ZHANG Weixia,WANG Wanqi,ZHOU Linshu, et al. Effects of lyrics on the processing of musical emotion: Behavioural and ERP study[J]. Acta Psychologica Sinica, 2018, 50(12): 1346-1355.
URL:  
http://journal.psych.ac.cn/xlxb/EN/10.3724/SP.J.1041.2018.01346     OR     http://journal.psych.ac.cn/xlxb/EN/Y2018/V50/I12/1346
  
  
  
  
1 Ali S.O., &Peynircio?lu , Z. F . ( 2006). Songs and emotions: Are lyrics and melodies equal partners? Psychology of Music, 34( 4), 511-534.
url: http://psycnet.apa.org/record/2006-11720-006
2 Behrens G.A., &Green , S. B . ( 1993). The ability to identify emotional content of solo improvisations performed vocally and on three different instruments. Psychology of Music, 21( 1), 20-33.
url: http://psycnet.apa.org/record/1993-36915-001
3 Brattico E., Alluri V., Bogert B., Jacobsen T., Vartiainen N., Nieminen S., & Tervaniemi M . ( 2011). A functional MRI study of happy and sad emotions in music with and without lyrics. Frontiers in Psychology, 2, 308.
pmid: 3227856 url: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3227856/
4 Brown C., &Hagoort P. , ( 1993). The processing nature of the N400: Evidence from masked priming. Journal of Cognitive Neuroscience, 5( 1), 34-44.
pmid: 23972118 url: http://med.wanfangdata.com.cn/Paper/Detail?id=PeriodicalPaper_PM23972118
5 Daltrozzo J. &Sch?n D. , ( 2009). Conceptual processing in music as revealed by N400 effects on words and musical targets. Journal of Cognitive Neuroscience, 21( 10), 1882-1892.
pmid: 18823240 url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM18823240
6 Darwin C. ( 1871). The descent of man and selection in relation to sex. London, UK: John Murray.
7 Erickson D.A . ( 2005). Language, ineffability and paradox in music philosophy (Unpublished doctorial dissertation).Simon Fraser University.
8 Franco F., Chew M., & Swaine J. S . ( 2017). Preschoolers’ attribution of affect to music: A comparison between vocal and instrumental performance. Psychology of Music, 45( 1), 131-149.
9 Galizio M., &Hendrick , C. ( 1972). Effect of musical accompaniment on attitude: The guitar as a prop for persuasion. Journal of Applied Social Psychology, 2( 4), 350-359.
url: http://onlinelibrary.wiley.com/doi/10.1111/j.1559-1816.1972.tb01286.x/full
10 Goerlich K. S., Witteman J., Aleman A., & Martens S . ( 2011). Hearing feelings: Affective categorization of music and speech in alexithymia, an ERP study. Plos One, 6( 5), e19501.
11 Goerlich K. S., Witteman J., Schiller N. O., Van -Heuven V. J., Aleman A., & Martens S . ( 2012). The nature of affective priming in music and speech. Journal of Cognitive Neuroscience, 24( 8), 1725-1741.
pmid: 22360592 url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM22360592
12 Gong X., Huang Y-X., Wang Y., & Luo Y-J . ( 2011). Revision of the Chinese facial affective picture system. Chinese Mental Health Journal, 25( 1), 40-46.
url: http://en.cnki.com.cn/article_en/cjfdtotal-zxws201101012.htm
13 [ 龚栩, 黄宇霞, 王妍, 罗跃嘉 . ( 2011). 中国面孔表情图片系统的修订. 中国心理卫生杂志, 25( 1), 40-46.]
url: http://med.wanfangdata.com.cn/Paper/Detail?id=PeriodicalPaper_zgxlwszz201101011
14 Hailstone J. C., Omar R., Henley S. M. D., Frost C., Kenward M. G., & Warren J. D . ( 2009). It's not what you play, it's how you play it: Timbre affects perception of emotion in music. Quarterly Journal of Experimental Psychology, 62( 11), 2141-2155.
pmid: 2683716 url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM19391047
15 Herring D. R., Taylor J. H., White K. R., & Crites S. L . ( 2011). Electrophysiological responses to evaluative priming: The LPP is sensitive to incongruity. Emotion, 11( 4), 794-806.
pmid: 21517156 url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM21517156
16 Hu X., Downie J. S., & Ehmann A. F . ( 2009, October). Lyric text mining in music mood classification. Paper presented at the International Society for Music Information Retrieval Conference, Kobe, Japan.
17 Ibá?ez A., Manes F., Escobar J., Trujillo N., Andreucci P., & Hurtado E . ( 2010). Gesture influences the processing of figurative language in non-native speakers: ERP evidence. Neuroscience Letters, 471( 1), 48-52.
pmid: 20079804 url: http://www.sciencedirect.com/science/article/pii/S030439401000025X
18 Imbir K. K., Spustek T., & ?ygierewicz J . ( 2016). Effects of valence and origin of emotions in word processing evidenced by event related potential correlates in a lexical decision task. Frontiers in Psychology, 7, 271.
pmid: 26973569 url: http://pubmedcentralcanada.ca/pmcc/articles/PMC4773610/
19 Jackendoff R.. , ( 2009). Parallels and nonparallels between language and music. Music Perception, 26( 3), 195-204.
url: http://www.jstor.org/stable/10.1525/mp.2009.26.3.195
20 Jankélévitch V. , & Abbate, C. ( 2003) . Music and the ineffable. Princeton, NJ: Princeton University Press.
21 Jarymowicz M.T., &Imbir , K. K . ( 2015). Toward a human emotions taxonomy (based on their automatic vs. reflective origin). Emotion Review, 7( 2), 183-188.
url: http://psycnet.apa.org/record/2015-12411-017
22 Juottonen K., Revonsuo A., & Lang H . ( 1996). Dissimilar age influences on two ERP waveforms (LPC and N400) reflecting semantic context effect. Cognitive Brain Research, 4( 2), 99-107.
pmid: 8883923 url: http://www.sciencedirect.com/science/article/pii/0926641096000225
23 Kamiyama K. S., Abla D., Iwanaga K., & Okanoya K . ( 2013). Interaction between musical emotion and facial expression as measured by event-related potentials. Neuropsychologia, 51( 3), 500-505.
pmid: 23220447 url: http://www.sciencedirect.com/science/article/pii/S0028393212005088
24 Koelsch S., Kasper E., Sammler D., Schulze K., Gunter T., & Friederici A. D . ( 2004). Music, language and meaning: Brain signatures of semantic processing. Nature Neuroscience, 7( 3), 302-307.
pmid: 14983184 url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_JJ028967606
25 Kutas M., &Federmeier , K. D . ( 2000). Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Sciences, 4( 12), 463-470.
pmid: 11115760 url: http://www.sciencedirect.com/science/article/pii/S1364661300015606
26 Kutas M., &Federmeier , K. D . ( 2011). Thirty years and counting: Finding meaning in the N400 component of the event related brain potential (ERP). Annual Review of Psychology, 62( 1), 621-647.
27 Kutas M., &Hillyard , S. A . ( 1980). Reading between the lines: Event-related brain potentials during natural sentence processing. Brain and Language, 11( 2), 354-373.
pmid: 7470854 url: http://www.sciencedirect.com/science/article/pii/0093934X80901339
28 Laurier C., Grivolla J., & Herrera P . ( 2008, May). Multimodal music mood classification using audio and lyrics. Paper presented at the Seventh International Conference on Machine Learning and Applications, Copenhagen, Denmark.
29 Levman, B. G . ( 1992). The genesis of music and language. Ethnomusicology, 36( 2), 147-170.
url: http://www.jstor.org/stable/851912
30 Mithen, S. J . ( 2006). The singing Neanderthals: The origins of music, language, mind and body. Cambridge Archaeological Journal, 59( 2), 54.
31 Mori K., &Iwanaga M. , ( 2013). Pleasure generated by sadness: Effect of sad lyrics on the emotions induced by happy music. Psychology of Music, 42( 5), 643-652.
32 Morton J. BJ..,& Trehub S. E., (2007). Children's judgements of emotion in song. Psychology of Music, 35( 4), 629-639.
url: http://psycnet.apa.org/record/2007-15293-004
33 Perlovsky L. ( 2010). Musical emotions: Functions, origins, evolution. Physics of Life Reviews, 7( 1), 2-27.
pmid: 20374916 url: http://www.sciencedirect.com/science/article/pii/S1571064509000438
34 Rohaut B.., &Naccache L. ,( 2017). Disentangling conscious from unconscious cognitive processing with event-related EEG potentials. Revue Neurologique, 173( 7-8), 521-528.
pmid: 28843414 url: http://www.ncbi.nlm.nih.gov/pubmed/28843414
35 Serafine M. L., Davidson J., Crowder R. G., & Repp B. H . ( 1986). On the nature of melody-text integration in memory for songs. Journal of Memory and Language, 25( 2), 123-135.
url: http://www.sciencedirect.com/science/article/pii/0749596X86900252
36 Schirmer A., Kotz S. A., & Friederici A. D . ( 2002). Sex differentiates the role of emotional prosody during word processing. Cognitive Brain Research, 14( 2), 228-233.
pmid: 12067695 url: http://www.sciencedirect.com/science/article/pii/S0926641002001088
37 Sousou S.D . ( 1997). Effects of melody and lyrics on mood and memory. Perceptual & Motor Skills, 85( 1), 31-40.
pmid: 9293553 url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM9293553
38 Steinbeis N.., &Koelsch , S. ( 2011). Affective priming effects of musical sounds on the processing of word meaning. Journal of Cognitive Neuroscience, 23( 3), 604-621.
pmid: 19925192 url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM19925192
39 Stratton V.N., &Zalanowski , A. H . ( 1994). Affective impact of music vs. lyrics. Empirical Studies of the Arts, 12( 2), 173-184.
url: http://psycnet.apa.org/psycinfo/1995-12404-001
40 Thompson W. F., Marin M. M., & Stewart L . ( 2012). Reduced sensitivity to emotional prosody in congenital amusia rekindles the musical protolanguage hypothesis. Proceedings of the National Academy of Sciences, 109( 46), 19027-19032.
url: http://med.wanfangdata.com.cn/Paper/Detail/PeriodicalPaper_PM23112175
41 Wang Y., &Qin Z. , (2016). Affective priming by simple geometric shapes: Evidence from event-related brain potentials. Frontiers in Psychology, 7, 917.
pmid: 4911398 url: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4911398/
42 Werheid K., Alpay G., Jentzsch I., & Sommer W . ( 2005). Priming emotional facial expressions as evidenced by event-related brain potentials. International Journal of Psychophysiology, 55( 2), 209-219.
pmid: 15649552 url: http://www.sciencedirect.com/science/article/pii/S0167876004001643
43 Yu R. Y. ( 2000). An introduction to modern western music philosophy. Changsha, China: Hunan Education Press.
44 [ 于润洋 . ( 2000). 现代西方音乐哲学导论. 长沙: 湖南教育出版社.]
45 Zhang Q., Kong L., & Jiang Y . ( 2012). The interaction of arousal and valence in affective priming: Behavioral and electrophysiological evidence. Brain Research, 1474, 60-72.
pmid: 3694405 url: http://www.sciencedirect.com/science/article/pii/S0006899312011705
46 Zhang Q., Li X. H., Gold B. T., & Jiang Y . ( 2010). Neural correlates of cross-domain affective priming. Brain Research, 1329, 142-151.
pmid: 2857548 url: http://www.sciencedirect.com/science/article/pii/S0006899310005652
47 Zhang Q. , & Wang, C. Z.( 1992) . Foundation in musical aesthetics. Beijing, China: People's Education Press.
48 [ 张前, 王次炤 . ( 1992). 音乐美学基础. 北京: 人民音乐出版社.]
49 Zhang Q., Lawson A., Guo C. Y., & Jiang Y . ( 2006). Electrophysiological correlates of visual affective priming. Brain Research Bulletin, 71( 1-3), 316-323.
pmid: 1783676 url: http://www.sciencedirect.com/science/article/pii/S0361923006002802
[1] ZHANG Jijia, CHEN Xuqian, YOU Ning, WANG Bin.  On how conceptual connections influence the category perception effect of colors: Another evidence of connections between language and cognition[J]. Acta Psychologica Sinica, 2018, 50(4): 390-399.
[2] BAI Xuejun; GAO Xiaolei; GAO Lei; WANG Yongsheng. An eye movement study on the perceptual span in reading Tibetan language[J]. Acta Psychologica Sinica, 2017, 49(5): 569-576.
[3] ZHAO Qingbai; KE Wei; TONG Biao; ZHOU Zhijin; ZHOU Zongkui. Creative processing of internet language: Novel N400 and LPC[J]. Acta Psychologica Sinica, 2017, 49(2): 143-154.
[4] LIU Shixiong, BI Xiaopei, HE Kaibin.  The impact of internet language copy on consumers’ attention and perceptions of the advertisement[J]. Acta Psychologica Sinica, 2017, 49(12): 1590-1603.
[5] LIU Cong; JIAO Lu; SUN Xun; WANG Ruiming. Immediate effect of language switch on non-proficient bilinguals’ cognitive control components[J]. Acta Psychologica Sinica, 2016, 48(5): 472-481.
[6] LI Heng; CAO Yu. Impact of bilingual experiences on language inhibition ability: Evidence from English-Chinese unimodal and English-American sign language bimodal bilinguals[J]. Acta Psychologica Sinica, 2016, 48(4): 343-351.
[7] WANG Tao; FENG Wenting; WEI Hua; ZHOU Nan. Loneliness makes me fall in love with you: The influence of product display on lonely individual’s product preference[J]. Acta Psychologica Sinica, 2016, 48(4): 398-409.
[8] DUAN Jinyun; GU Xiaohua; SUN Luying. The influence of explicit self-esteem, implicit self-esteem and the discrepancies between them on advice taking[J]. Acta Psychologica Sinica, 2016, 48(4): 371-384.
[9] . An elaboration on the underlying mechanisms through which consumers’ desire for product touch forms: A cognitive experiential perspective[J]. Acta Psychologica Sinica, 2016, 48(4): 410-422.
[10] MENG Yingfang; LIN Wuji; LIN Jingyuan; CAI Chaoqun. The phonological or semantic activation of non-target language in an immediate cross-language switching paradigm[J]. Acta Psychologica Sinica, 2016, 48(2): 121-129.
[11] LI Li; ZHANG Yang; LI Xuan; GUO Hongting; WU Limei; WANG Ruiming. Cross-language repetition priming effect of semantic access for trilinguals[J]. Acta Psychologica Sinica, 2016, 48(11): 1401-1409.
[12] CHEN Suiqing; ZHANG Jijia; LI Yanxia; ZHANG Huixia. Chinese Sign Language Lexical Recognition: Two Network Systems and Interaction Effect Among Sign Language Words[J]. Acta Psychologica Sinica, 2015, 47(7): 878-889.
[13] LI Cuihong; HE Xu; GUO Chunyan. The Storage Mechanism of Multi-feature Objects in Visual Working Memory[J]. Acta Psychologica Sinica, 2015, 47(6): 734-745.
[14] YANG Yaping; WANG Pei; YIN Zhihui; CHEN Qingwei; FENG Xiaying. The Pattern and Neural Correlates of Unintentional Stereotype Activation[J]. Acta Psychologica Sinica, 2015, 47(4): 488-502.
[15] GAO Xuemei; WENG Lei; ZHOU Qun; ZHAO Cai; LI Fang. Dose Violent Offenders Have Lower Capacity of Empathy for Pain: Evidence from ERPs[J]. Acta Psychologica Sinica, 2015, 47(4): 478-487.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech