Please wait a minute...
Advances in Psychological Science    2015, Vol. 23 Issue (8) : 1361-1370     DOI: 10.3724/SP.J.1042.2015.01361
Regular Articles |
The Cognitive and Neural Mechanism of Text and Picture Processing in Multimedia Learning
LI Songqing; ZHAO Qingbai; ZHOU Zhijin; ZHANG Yi
(Key Laboratory of Adolescent Cyberpsychology and Behavior (CCNU), Ministry of Education; Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan 430079, China)
Download: PDF(976 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    

According to the cognitive theory of multimedia learning and the integrated theory of text and picture comprehension, text and picture are processed through distinct channels at the early stage, and then integrated into a whole. The theories are supported by empirical researches in neuroscience. Brain imaging studies have shown that the processing of text and picture is different in the semantic pre-process, but shares the same semantic neural system in information integration later. Event-related potential studies have found that only picture evokes a N300 in the early stage, while both picture and text induce the N400 effect later. Although existing studies have provided clear evidence that text and picture share the same semantic system, further research is needed to explore how the information is integrated in this system.

Keywords multimedia learning      semantic processing      text and picture integration      brain imaging      event- related potential     
Corresponding Authors: ZHAO Qingbai, E-mail:   
Issue Date: 15 August 2015
E-mail this article
E-mail Alert
Articles by authors
LI Songqing
ZHAO Qingbai
ZHOU Zhijin
Cite this article:   
LI Songqing,ZHAO Qingbai,ZHOU Zhijin, et al. The Cognitive and Neural Mechanism of Text and Picture Processing in Multimedia Learning[J]. Advances in Psychological Science, 2015, 23(8): 1361-1370.
URL:     OR
[1] WANG Fuxing; LI Wenjing; XIE Heping; LIU Huashan. Is pedagogical agent in multimedia learning good for learning? A meta-analysis[J]. Advances in Psychological Science, 2017, 25(1): 12-28.
[2] WANG Fuxing; XIE Heping; LI Hui. Visual text or narration? Meta-analysis of the modality effect in multimedia learning[J]. Advances in Psychological Science, 2016, 24(3): 335-350.
[3] CAI Lin;ZHANG Yaxu. EEG Time-Frequency Analysis of Syntactic and Semantic Processing in Sentence Comprehension[J]. Advances in Psychological Science, 2014, 22(7): 1112-1121.
[4] WANG Fuxing;DUAN Zhaohui;ZHOU Zongkui. Attention Guidance in Multimedia Learning: The Role of Cueing[J]. Advances in Psychological Science, 2013, 21(8): 1430-1440.
[5] HAN Shi-Hui;ZHANG Yi-Fan. A Cultural Neuroscience Approach to Self-Concept Representation[J]. , 2012, 20(5): 633-640.
[6] DING Guo-Sheng;LI Yan-Yan. The Role of Early Sign-language Experience in Shaping the Structure and Function of the Deaf Brain[J]. , 2012, 20(3): 328-337.
[7] SHEN Wang-Bing;LIU Chang;CHEN Jing–Jing. Neural Basis of Creativity: Evidence From Structural and Functional Imaging[J]. , 2010, 18(09): 1420-1429.
[8] WANG Nai-Yi; LUO Yue-Jia; DONG Qi. Neural Correlates of Aesthetics[J]. , 2010, 18(01): 19-27.
[9] ZHANG Ye; ZHANG Qing-Lin. Analysis on Cognitive Function of Recognition Potential[J]. , 2010, 18(01): 28-33.
[10] LIU Cong-Hui;WANG Yong-Mei;YU Guo-Liang;WANG Yong-Jun. Related Theories and Exploration on Dynamic Model of Empathy[J]. , 2009, 17(05): 964-972.
[11] ZENG Hong-Ling;LIU Si-Yun. Sentence Comprehension in Discourse Context: Evidence from the N400[J]. , 2009, 17(02): 314-320.
[12] FU Yan;WANG Da-Hua. Cognitive Aging and the Brain: HAROLD Model Debate[J]. , 2009, 17(01): 86-91.
[13] CUI Fang;NAN Yun;LUO Yue-Jia. A Review of Cognitive Neuroscience Studies on Empathy[J]. , 2008, 16(02): 250-254.
[14] Yang Shuang,Liu Xiangping,Wang Bin. Cognitive Processing of the Children with Reading Comprehension Difficulties[J]. , 2006, 14(03): 368-374.
[15] Gao Bing;Cao Hui;Cao Pin. The Brain Mechanism of Syntactic Processing[J]. , 2006, 14(01): 32-39.
Full text



Copyright © Advances in Psychological Science
Support by Beijing Magtech