Communicative gesture cognition theory

doi:10.3724/SP.J.1042.2019.00499

Abstract

Abstract:

Gesture is an important nonverbal medium of communication. Gesture can not only assist language communication but also has independent communication attribute. As a nonverbal medium co occurring with language media, gesture communication can help to reduce communication cognitive load. This paper mainly summarizes and reviews the theory of communicative gesture based on the expression relationship between gesture and language, the theory of communicative gesture activation, and the cognitive saving theory of communicative gesture. The future research should explore the balance between naturalness and control rigor of the experimental situation, the relationship between gesture and other nonverbal factors, and the practical significance of gesture cognition research.

Key words: communication, gesture, cognition

CLC Number:

B842

ZHANG Hengchao. Communicative gesture cognition theory[J]. Advances in Psychological Science, 2019, 27(3): 499-507.

References 58

[1]	张恒超 . ( 2013). 参照性交流中的“听者设计”. 心理发展与教育, 29( 5), 552-560.
[2]	张恒超 . ( 2017). 共享因素对参照性交流双方学习的影响. 心理学报, 49( 2), 197-205. doi: 10.3724/SP.J.1041.2017.00197 URL
[3]	张恒超 . ( 2018). 交流语言认知特征. 心理科学进展, 26( 2), 270-282.
[4]	Alibali M. W., Kita S., & Young A. J . ( 2000). Gesture and the process of speech production: We think, therefore, we gesture. Language and Cognitive Processes, 15( 6), 593-613. doi: 10.1080/016909600750040571 URL
[5]	Alibali M. W., Spencer R. C., Knox L., & Kita S . ( 2011). Spontaneous gestures influence strategy choices in problem solving. Psychological Science, 22( 9), 1138-1144. doi: 10.1177/0956797611417722 URL pmid: 21813800
[6]	Arbib M. A., Gasser B., & Barrès V . ( 2014). Language is handy but is it embodied?. Neuropsychologia, 55, 57-70. doi: 10.1016/j.neuropsychologia.2013.11.004 URL pmid: 24252354
[7]	Arnold J. E., Kahn J. M., & Pancani G. C . ( 2012). Audience design affects acoustic reduction via production facilitation. Psychonomic Bulletin & Review, 19( 3), 505-512. doi: 10.3758/s13423-012-0233-y URL pmid: 22419403
[8]	Beaudoin-Ryan L. & Goldin-Meadow S., ( 2014). Teaching moral reasoning through gesture. Developmental Science, 17( 6), 984-990. doi: 10.1111/desc.12180 URL pmid: 24754707
[9]	Berezan O., Yoo M., & Christodoulidou N . ( 2016). The impact of communication channels on communication style and information quality for hotel loyalty programs. Journal of Hospitality and Tourism Technology, 7( 1), 100-116. doi: 10.1108/JHTT-08-2015-0031 URL
[10]	Bock K., & Cutting J.C . ( 1992). Regulating mental energy: Performance units in language production. Journal of Memory and Language, 31( 1), 99-127. doi: 10.1016/0749-596X(92)90007-K URL
[11]	Brentari D. & Goldin-Meadow S., ( 2017). Language Emergence. Annual Review of Linguistics, 3( 1), 363-388. doi: 10.1146/annurev-linguistics-011415-040743 URL
[12]	Broaders S.C., & Goldin-Meadow S. ,( 2010). Truth is at hand: How gesture adds information during investigative interviews. Psychological Science, 21( 5), 623-628. doi: 10.1177/0956797610366082 URL
[13]	Broaders S. C., Cook S. W., Mitchell Z., & Goldin-Meadow S . ( 2007). Making children gesture brings out implicit knowledge and leads to learning. Journal of Experimental Psychology: General, 136( 4), 539-550 doi: 10.1037/0096-3445.136.4.539 URL pmid: 17999569
[14]	Brooks N. & Goldin-Meadow S., ( 2015). Moving to learn: How guiding the hands can set the stage for learning. Cognitive Science A Multidisciplinary Journal, 40( 7), 1831-1849. doi: 10.1111/cogs.12292 URL
[15]	Cook S. W., Duffy R. G., & Fenn K. M . ( 2013). Consolidation and transfer of learning after observing hand gesture. Child development, 84( 6), 1863-1871. doi: 10.1111/cdev.12097 URL pmid: 23551027
[16]	de Marco D., de Stefani E., & Gentilucci M . ( 2015). Gesture and word analysis: The same or different processes?. NeuroImage, 117, 375-385. doi: 10.1016/j.neuroimage.2015.05.080 URL pmid: 26044859
[17]	Drijvers L. & Özyürek A., ( 2017). Visual context enhanced: The joint contribution of iconic gestures and visible speech to degraded speech comprehension. Journal of Speech, Language, and Hearing Research, 60( 1), 212-222. doi: 10.1044/2016_JSLHR-H-16-0101 URL pmid: 27960196
[18]	Duran N.D., & Dale R. , ( 2014). Perspective-taking in dialogue as self-organization under social constraints. New Ideas in Psychology, 32, 131-146. doi: 10.1016/j.newideapsych.2013.03.004 URL
[19]	Edelman S. ( 2017). Language and other complex behaviors: Unifying characteristics, computational models, neural mechanisms. Language Sciences, 62, 91-123. doi: 10.1016/j.langsci.2017.04.003 URL
[20]	Glenberg A.M., & Gallese V. , ( 2012). Action-based language: A theory of language acquisition, comprehension, and production. Cortex, 48( 7), 905-922. doi: 10.1016/j.cortex.2011.04.010 URL pmid: 21601842
[21]	Goldin-Meadow S. ( 2015). From action to abstraction: Gesture as a mechanism of change. Developmental Review, 38, 167-184. doi: 10.1016/j.dr.2015.07.007 URL pmid: 4672635
[22]	Goldin-Meadow S., Cook S. W., & Mitchell Z. A . ( 2009). Gesturing gives children new ideas about math. Psychological Science, 20( 3), 267-272. doi: 10.1111/j.1467-9280.2009.02297.x URL pmid: 2750886
[23]	Goldin-Meadow S., Nusbaum H., Kelly S. D., & Wagner S. M . ( 2001). Explaining math: Gesturing lightens the load. Psychological Science, 12( 6), 516-522. doi: 10.1111/1467-9280.00395 URL pmid: 11760141
[24]	Graziano M. & Gullberg M., ( 2013). Gesture production and speech fluency in competent speakers and language learners. In Tilburg Gesture Research Meeting (TiGeR) 2013. Tilburg University.
[25]	Grenoble L. A., Martinovic´ M. & Baglini R.. ,( 2014) . Verbal gestures in Wolof. In R. Kramer, L. Zsiga, & O. Boyer (Eds.), Selected proceedings of the 44th annual conference on African linguistics. Somerville, MA: Cascadilla Press.
[26]	Hadar U. & Butterworth B., ( 1997). Iconic gestures, imagery, and word retrieval in speech. Semiotica, 115( 1-2), 147-172.
[27]	Hostetter A.B . ( 2011). When do gestures communicate? A meta-analysis. Psychological bulletin, 137( 2), 297-315. doi: 10.1037/a0022128 URL pmid: 21355631
[28]	Hostetter A.B . ( 2014). Action attenuates the effect of visibility on gesture rates. Cognitive Science, 38( 7), 1468-1481. doi: 10.1111/cogs.12113 URL pmid: 24889881
[29]	Hostetter A.B., & Alibali, M. W . ( 2010). Language, gesture, action! A test of the Gesture as Simulated Action framework. Journal of Memory and Language, 63( 2), 245-257. doi: 10.1016/j.jml.2010.04.003 URL
[30]	Hostetter A. B., Alibali M. W., & Kita S . ( 2007). I see it in my hands’ eye: Representational gestures reflect conceptual demands. Language and Cognitive Processes, 22( 3), 313-336. doi: 10.1080/01690960600632812 URL
[31]	Hu F. T., Ginns P., & Bobis J . ( 2015). Getting the point: Tracing worked examples enhances learning. Learning & Instruction, 35, 85-93. doi: 10.1016/j.learninstruc.2014.10.002 URL
[32]	Iverson J.M., & Goldin-Meadow S. ,( 1998). Why people gesture when they speak. Nature, 396( 6708), 228. doi: 10.1038/24300 URL pmid: 9834030
[33]	Kang S., Tversky B., & Black J. B . ( 2015). Coordinating gesture, word, and diagram: Explanations for experts and novices. Spatial Cognition & Computation, 15( 1), 1-26. doi: 10.1080/13875868.2014.958837 URL
[34]	Kelly S.D., & Church R.B . ( 1998). A comparison between children's and adults' ability to detect conceptual information conveyed through representational gestures. Child Development, 69( 1), 85-93. doi: 10.1111/j.1467-8624.1998.tb06135.x URL pmid: 9499559
[35]	Kita S. & Özyürek A., ( 2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and language, 48( 1), 16-32. doi: 10.1016/S0749-596X(02)00505-3 URL
[36]	Koppensteiner M., Stephan P & Jäschke J. P. M. ., ( 2016). Moving speeches: Dominance, trustworthiness and competence in body motion. Personality and Individual Differences, 94, 101-106. doi: 10.1016/j.paid.2016.01.013 URL
[37]	Krauss R. M., Chen Y. S. & Gottesman R.. ,( 2000) . Lexical gestures and lexical access: A process model In D McNeill (Eds) Language and gesture (pp. 261-283). Cambridge, UK: Cambridge University Press.
[38]	Krauss R.M., & Weinheimer S. , ( 1964). Changes in reference phrases as a function of frequency of usage in social interaction: A preliminary study. Psychonomic Science, 1, 113-114. doi: 10.3758/BF03342817 URL
[39]	Louwerse M.M . ( 2011). Symbol interdependency in symbolic and embodied cognition. Topics in Cognitive Science, 3( 2), 273-302. doi: 10.1111/j.1756-8765.2010.01106.x URL pmid: 25164297
[40]	Matovic D., Koch A. S., & Forgas J. P . ( 2014). Can negative mood improve language understanding? Affective influences on the ability to detect ambiguous communication. Journal of Experimental Social Psychology, 52, 44-49. doi: 10.1016/j.jesp.2013.12.003 URL
[41]	McNeill D & Duncan S. ., ( 2000) . Growth points in thinking-for-speaking. In D. McNeill. (Eds) Language and gesture (pp. 141-161). Cambridge,UK: Cambridge University Press.
[42]	Morsella E., & Krauss R.M . ( 2004). The role of gestures in spatial working memory and speech. The American Journal of Psychology, 117( 3), 411-424. doi: 10.2307/4149008 URL pmid: 15457809
[43]	Nicoladis E., Pika S., Yin H. U. I., & Marentette P . ( 2007). Gesture use in story recall by Chinese-English bilinguals. Applied Psycholinguistics, 28( 4), 721-735. doi: 10.1017/S0142716407070385 URL
[44]	Novack M. A., Congdon E. L., Hemanilopez N., & Goldin-Meadow S . ( 2014). From action to abstraction: Using the hands to learn math. Psychological Science, 25( 4), 903-910. doi: 10.1177/0956797613518351 URL
[45]	Novack M.A., & Goldin-Meadow S. , ( 2016). Gesture as representational action: A paper about function. Psychonomic Psychonomic Bulletin & Review, 24( 3), 652-665. doi: 10.3758/s13423-016-1145-z URL pmid: 27604493
[46]	Novack M. & Goldin-Meadow S., ( 2015). Learning from gesture: How our hands change our minds. Educational Psychology Review, 27( 3), 405-412. doi: 10.1007/s10648-015-9325-3 URL pmid: 4562024
[47]	Novack M. A., Goldin-Meadow S., & Woodward A. L . ( 2015). Learning from gesture: How early does it happen?. Cognition, 142, 138-147. doi: 10.1016/j.cognition.2015.05.018 URL pmid: 4500665
[48]	Perniss P., Özyürek A., & Morgan G . ( 2015). The Influence of the visual modality on language structure and conventionalization: Insights from sign language and gesture. Topics in Cognitive Science, 7( 1), 2-11. doi: 10.1111/tops.12127 URL pmid: 25565249
[49]	Pine K. J., Bird H., & Kirk E . ( 2007). The effects of prohibiting gestures on children's lexical retrieval ability. Developmental Science, 10( 6), 747-754. doi: 10.1111/j.1467-7687.2007.00610.x URL pmid: 17973791
[50]	Ping R.M., & Goldin-Meadow S. , ( 2008). Hands in the air: Using ungrounded iconic gestures to teach children conservation of quantity. Developmental Psychology, 44( 5), 1277-1287. doi: 10.1037/0012-1649.44.5.1277 URL pmid: 18793062
[51]	Ping R. M., Goldin-Meadow S., & Beilock S. L . ( 2014). Understanding gesture: Is the listener’s motor system involved?. Journal of Experimental Psychology: General, 143( 1), 195-204. doi: 10.1037/a0032246 URL pmid: 23565671
[52]	Post L. S., Gog T. V., Paas F., & Zwaan R. A . ( 2013). Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning. Computers in Human Behavior, 29( 4), 1450-1455. doi: 10.1016/j.chb.2013.01.005 URL
[53]	Rauscher F. H., Krauss R. M., & Chen Y. S . ( 1996). Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science, 7( 4), 226-231. doi: 10.1111/j.1467-9280.1996.tb00364.x URL
[54]	Sacchi S., Riva P., & Aceto A . ( 2016). Myopic about climate change: Cognitive style, psychological distance, and environmentalism. Journal of Experimental Social Psychology, 65, 68-73. doi: 10.1016/j.jesp.2016.03.006 URL
[55]	Sassenberg U. & van Der Meer E. , ( 2010). Do we really gesturemorewhen it is more difficult? Cognitive Science, 34( 4), 643-664. doi: 10.1111/j.1551-6709.2010.01101.x URL pmid: 21564228
[56]	Sharma V. & Droch B., ( 2015). Gesture-controlled user interfaces. Journal of Information Sciences and Computing Technologies, 2( 1), 133-135.
[57]	Trofatter C., Kontra C., Beilock S., & Goldin-Meadow S . ( 2015). Gesturing has a larger impact on problem-solving than action, even when action is accompanied by words. Language, Cognition and Neuroscience, 30( 3), 251-260. doi: 10.1080/23273798.2014.905692 URL pmid: 4318567
[58]	Weinberg A., Fukawa-Conolly T., & Wiesner E . ( 2015). Characterizing instructor gestures in a lecture in a proof-based mathematics class. Educational Studies in Mathematics, 90( 3), 233-258. doi: 10.1007/s10649-015-9623-1 URL