ISSN 1671-3710
CN 11-4766/R

Advances in Psychological Science ›› 2015, Vol. 23 ›› Issue (8): 1361-1370.doi: 10.3724/SP.J.1042.2015.01361

• Regular Articles • Previous Articles     Next Articles

The Cognitive and Neural Mechanism of Text and Picture Processing in Multimedia Learning

LI Songqing; ZHAO Qingbai; ZHOU Zhijin; ZHANG Yi   

  1. (Key Laboratory of Adolescent Cyberpsychology and Behavior (CCNU), Ministry of Education; Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan 430079, China)
  • Received:2014-12-29 Online:2015-08-15 Published:2015-08-15
  • Contact: ZHAO Qingbai, E-mail:


According to the cognitive theory of multimedia learning and the integrated theory of text and picture comprehension, text and picture are processed through distinct channels at the early stage, and then integrated into a whole. The theories are supported by empirical researches in neuroscience. Brain imaging studies have shown that the processing of text and picture is different in the semantic pre-process, but shares the same semantic neural system in information integration later. Event-related potential studies have found that only picture evokes a N300 in the early stage, while both picture and text induce the N400 effect later. Although existing studies have provided clear evidence that text and picture share the same semantic system, further research is needed to explore how the information is integrated in this system.

Key words: multimedia learning, semantic processing, text and picture integration, brain imaging, event- related potential