Please wait a minute...
Advances in Psychological Science    2015, Vol. 23 Issue (8) : 1361-1370     DOI: 10.3724/SP.J.1042.2015.01361
Regular Articles |
The Cognitive and Neural Mechanism of Text and Picture Processing in Multimedia Learning
LI Songqing; ZHAO Qingbai; ZHOU Zhijin; ZHANG Yi
(Key Laboratory of Adolescent Cyberpsychology and Behavior (CCNU), Ministry of Education; Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan 430079, China)
Download: PDF(976 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    

According to the cognitive theory of multimedia learning and the integrated theory of text and picture comprehension, text and picture are processed through distinct channels at the early stage, and then integrated into a whole. The theories are supported by empirical researches in neuroscience. Brain imaging studies have shown that the processing of text and picture is different in the semantic pre-process, but shares the same semantic neural system in information integration later. Event-related potential studies have found that only picture evokes a N300 in the early stage, while both picture and text induce the N400 effect later. Although existing studies have provided clear evidence that text and picture share the same semantic system, further research is needed to explore how the information is integrated in this system.

Keywords multimedia learning      semantic processing      text and picture integration      brain imaging      event- related potential     
Corresponding Authors: ZHAO Qingbai, E-mail:   
Issue Date: 15 August 2015
E-mail this article
E-mail Alert
Articles by authors
LI Songqing
ZHAO Qingbai
ZHOU Zhijin
Cite this article:   
LI Songqing,ZHAO Qingbai,ZHOU Zhijin, et al. The Cognitive and Neural Mechanism of Text and Picture Processing in Multimedia Learning[J]. Advances in Psychological Science, 2015, 23(8): 1361-1370.
URL:     OR
[1] Shijing CHENG, Wenguang HE. The acquisition, development and aging of semantic cognition and related neural mechanism[J]. Advances in Psychological Science, 2020, 28(7): 1156-1163.
[2] Qinghua HE, Dandan LI. Brain maturation and improvement in intertemporal choice in Chinese children and adolescents[J]. Advances in Psychological Science, 2020, 28(3): 381-389.
[3] Yanqing WANG, Fuxing WANG, Heping XIE, Jiaxue CHEN, Wenjing LI, Xiangen HU. A picture is worth a thousand words: Self-generation drawing for multimedia learning[J]. Advances in Psychological Science, 2019, 27(4): 623-635.
[4] YANG Jianfeng, DANG Min, ZHANG Rui, WANG Xiaojuan.  The neural circuit of semantic processing and its dynamic cooperation with the neural circuit of phonological processing in reading Chinese characters[J]. Advances in Psychological Science, 2018, 26(3): 381-390.
[5] Jiaxue CHEN, Heping XIE, Fuxing WANG, Li ZHOU, Wenjing LI. Do induced positive emotions facilitate multimedia learning?[J]. Advances in Psychological Science, 2018, 26(10): 1818-1830.
[6] Haishuo XIA, Qingwen DING, Yan ZHUANG, Antao CHEN. The brain mechanisms of the physical exercise enhancing cognitive function[J]. Advances in Psychological Science, 2018, 26(10): 1857-1868.
[7] TONG Wei, CHEN Taolin, HUANG Xiaoqi, GONG Qiyong, FANG Xiaoyi.  Episodic and background craving: Distinguishing pathway of caving effect on nicotine addition[J]. Advances in Psychological Science, 2017, 25(11): 1932-1941.
[8] WANG Fuxing; LI Wenjing; XIE Heping; LIU Huashan. Is pedagogical agent in multimedia learning good for learning? A meta-analysis[J]. Advances in Psychological Science, 2017, 25(1): 12-28.
[9] WANG Fuxing; XIE Heping; LI Hui. Visual text or narration? Meta-analysis of the modality effect in multimedia learning[J]. Advances in Psychological Science, 2016, 24(3): 335-350.
[10] CAI Lin;ZHANG Yaxu. EEG Time-Frequency Analysis of Syntactic and Semantic Processing in Sentence Comprehension[J]. Advances in Psychological Science, 2014, 22(7): 1112-1121.
[11] WANG Fuxing;DUAN Zhaohui;ZHOU Zongkui. Attention Guidance in Multimedia Learning: The Role of Cueing[J]. Advances in Psychological Science, 2013, 21(8): 1430-1440.
[12] HAN Shi-Hui;ZHANG Yi-Fan. A Cultural Neuroscience Approach to Self-Concept Representation[J]. , 2012, 20(5): 633-640.
[13] DING Guo-Sheng;LI Yan-Yan. The Role of Early Sign-language Experience in Shaping the Structure and Function of the Deaf Brain[J]. , 2012, 20(3): 328-337.
[14] SHEN Wang-Bing;LIU Chang;CHEN Jing–Jing. Neural Basis of Creativity: Evidence From Structural and Functional Imaging[J]. , 2010, 18(09): 1420-1429.
[15] WANG Nai-Yi; LUO Yue-Jia; DONG Qi. Neural Correlates of Aesthetics[J]. , 2010, 18(01): 19-27.
Full text



Copyright © Advances in Psychological Science
Support by Beijing Magtech