手势认知功能研究的新视角：“空间化”手势假设

doi:10.3724/SP.J.1042.2020.00426

摘要/Abstract

摘要：

人类在说话或思考的时候常常伴随着手势。手势是在认知加工或交流过程中自动产生的, 具有表征性, 同时, 手势能够影响人类的认知加工。尽管研究者对手势的概念界定各有侧重, 但普遍认为手势不同于直接行动, 具有认知功能。手势认知功能的代表性理论模型有词汇索引模型、信息打包假设、表象保持理论、语义特殊性假设和嵌入/延展观。根据手势认知功能研究中主要自变量的不同, 可以把手势认知功能分成三种不同的研究范式, 即允许-限制手势的研究范式、侧重手势模式改变的研究范式、侧重情境改变的研究范式。今后值得关注的研究方向除了深入探讨手势认知功能的神经机制、加强对手势认知功能的干预研究外, 提出了建立更具解释力的手势认知功能的理论模型——“空间化”手势假设。

关键词: 手势, 认知功能, 词汇索引, 信息打包, 具身认知

Abstract:

Human beings often make gestures when talking or thinking. Gestures are generated automatically during cognitive processing or communication, and as representations, they in turn affect human cognitive processing. Although researchers have different emphasis on the concept of gestures, it is generally believed that gestures are different from direct actions and have internal and external cognitive functions. Seminal theoretical models include Lexical Index Model, Information Packing Hypothesis, Image Maintenance Theory, Semantic Specificity Hypothesis and Embodied/Extended Perspective. According to the main independent variables, the research paradigms fall into three categories, namely, allow-and-limit gestures, the change of gesture patterns, and changes of situation. In addition to in-depth study of the neural mechanism of cognitive functions of gestures and their intervention research, this paper proposes a theoretical model with more explanatory power - the "Spatializing" Gesture Hypothesis.

Key words: gestures, cognitive function, lexical index, information package, embodied cognition

于文华, 鲁忠义. (2020). 手势认知功能研究的新视角：“空间化”手势假设. 心理科学进展 , 28(3), 426-433.

YU Wenhua, LU Zhongyi. (2020). A new perspective on the cognitive function of gestures: The “spatializing” gesture hypothesis. Advances in Psychological Science, 28(3), 426-433.

参考文献 56

[1]	张盼, 鲁忠义 . (2013). 句子理解中颜色信息的心理表征. 心理学报, 45(4), 406-415.
[2]	Alibali M. W., Spencer R. C., Knox L., & Kita S . (2011). Spontaneous gestures influence strategy choices in problem solving. Psychological Science, 22(9), 1138-1144.
[3]	Arbib M. A . (2005). From monkey-like action recognition to human language: an evolutionary framework for neurolinguistics. Behavioral and Brain Sciences, 28(2), 105-124.
[4]	Beaudoin-Ryan L., & Goldin-Meadow S . (2014). Teaching moral reasoning through gesture. Developmental Science, 17(6), 984-990.
[5]	Buccino G., Binkofski F., Fink G. R., Fadiga L., Fogassi L., & Gallese V., ... Freund H. J . (2001). Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study. European Journal of Neuroscience, 13(2), 400-404.
[6]	Chu M. Y., & Kita S . (2008). Spontaneous gestures during mental rotation tasks: Insights into the microdevelopment of the motor strategy. Journal of Experimental Psychology: General, 137(4), 706-723.
[7]	Chu M. Y., & Kita S . (2011). The nature of gestures' beneficial role in spatial problem solving. Journal of Experimental Psychology General, 140(1), 102-116.
[8]	Chu M. Y., & Kita S . (2015). Co-thought and co-speech gestures are generated by the same action generation process. Journal of Experimental Psychology Learning Memory & Cognition, 42(2), 257-270.
[9]	Clark A . (2008). Supersizing the Mind: Embodiment, Action, and Cognitive Extension New York, NY: Oxford University Press.
[10]	Connell L . (2007). Representing object colour in language comprehension. Cognition, 102(3), 476-485.
[11]	Cook S. W., & Tanenhaus M. K . (2009). Embodied communication: speakers' gestures affect listeners' actions. Cognition, 113(1), 98-104.
[12]	Gentner D . (1982). Why nouns are learned before verbs: Linguistic relativity versus natural partitioning. technical report no. 257. Child Language, 2, 86.
[13]	Glenberg A. M., & Gallese V . (2012). Action-based Language: A theory of language acquisition, comprehension, and production. Cortex, 48(7), 905-922.
[14]	Glenberg A. M., & Kaschak M. P . (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558-565.
[15]	Goldin-meadow S . (2000). Beyond words: the importance of gesture to researchers and learners. Child Development, 71(1), 231-239.
[16]	Goldin-Meadow S., Nusbaum H., Kelly S. D., & Wagner S . (2001). Explaining math: gesturing lightens the load. Psychological Science, 12(6), 516-522.
[17]	Goldin-Meadow S., & Wagner S. M . (2005). How our hands help us learn. Trends in Cognitive Sciences, 9(5), 234-241.
[18]	Goodrich W., & Hudson Kam, C. L . (2009). Co-speech gesture as input in verb learning. Developmental Science, 12(1), 81-87.
[19]	Grade S., Badets A., & Pesenti M . (2016). Influence of finger and mouth action observation on random number generation: an instance of embodied cognition for abstract concepts. Psychological Research, 81(3), 538-548.
[20]	Hasegawa D., Shirakawa S., Shioiri N., Hanawa T., Sakuta H., & Ohara K . (2015). The effect of metaphoric gestures on schematic understanding of instruction performed by a pedagogical conversational agent. Lecture Notes in Computer Science, 361-371.
[21]	Hostetter A. B., & Alibali M. W . (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15(3), 495-514.
[22]	Hostetter A. B., & Alibali M. W . (2010). Language, gesture, action! A test of the gesture as simulated action framework. Journal of Memory & Language, 63(2), 245-257.
[23]	Kang S., Tversky B., & Black J. B . (2015). Coordinating gesture, word, and diagram: Explanations for experts and novices. Spatial Cognition & Computation, 15(1), 1-26.
[24]	Kaup B., Yaxley R. H., Madden C. J., Zwaan R. A., & Lüdtke J . (2007). Experiential simulations of negated text information. Quarterly Journal of Experimental Psychology, 60(7), 976-990.
[25]	Kirk E., & Lewis C . (2017). Gesture facilitates children's creative thinking. Psychological Science, 28(2), 225-232.
[26]	Kita S., Alibali M. W., & Chu M. Y . (2017). How do gestures influence thinking and speaking? The gesture-for- conceptualization hypothesis. Psychological Review, 124(3), 245-266.
[27]	Kita S., & Davies T. S . (2009). Competing conceptual representations trigger co-speech representational gestures. Language & Cognitive Processes, 24(5), 761-775.
[28]	Kita S., de Condappa O., & Mohr C . (2007). Metaphor explanation attenuates the right-hand preference for depictive co-speech gestures that imitate actions. Brain & Language, 101(3), 185-197.
[29]	Krauss R. M., Chen Y., & Gottesman R. F . (2000). Lexical gestures and lexical access: A process model In D. McNeill (Ed.), Language and gesture (pp.261-283). Britain: Cambridge University Press.
[30]	Leung A. K.-Y., Kim S., Polman E., Ong L. S., Qiu L., Goncalo J. A., & Sanchez-Burks J . (2012). Embodied metaphors and creative "acts". Psychological Science, 23(5), 502-509.
[31]	Lhommet M., & Marsella S . (2014). Metaphoric gestures: Towards grounded mental spaces. 14th International Conference on Intelligent Virtual Agents Conference. Berlin, Germany: Springer International Publishing.
[32]	Loughlin V . (2016). Zdravko radman, the hand: an organ of the mind, what the manual tells the mental. Phenomenology & the Cognitive Sciences, 15(2), 291-296.
[33]	Mcgregor K. K., Rohlfing K. J., Bean A., & Marschner E . (2009). Gesture as a support for word learning: The case of under. Journal of Child Language, 36(4), 807-828.
[34]	Mcneill D . (1985). So you think gestures are nonverbal?. Psychological Review, 92(3), 350-371.
[35]	Mcneill D . (1992). Hand and mind: What gestures reveal about thought Chicago: University of Chicago Press.
[36]	Mol L., & Kita S . (2012). Gesture structure affects syntactic structure in speech. Cognitive Science Society, 34(34), 761-766.
[37]	Mumford K. H., & Kita S . (2013). Children use gesture to interpret novel verb meanings. Child Development, 85(3), 1181-1189.
[38]	Novack M. A., Congdon E. L., Hemani-Lopez N., & Goldin- Meadow S . (2014). From action to abstraction: using the hands to learn math. Psychological Science, 25(4), 903-910.
[39]	Novack M. A., & Goldin-Meadow S . (2016). Gesture as representational action: A paper about function. Psychonomic Bulletin & Review, 24(3), 1-14.
[40]	Novack M. A., Wakefield E. M., & Goldin-Meadow S . (2016). What makes a movement a gesture?. Cognition, 146, 339-348.
[41]	Özçalışkan S., Lucero C., & Goldin-meadow S . (2016). Does language shape silent gesture?. Cognition, 148, 10-18.
[42]	Pine K. J., Gurney D. J., & Fletcher B . (2010). The semantic specificity hypothesis: when gestures do not depend upon the presence of a listener. Journal of Nonverbal Behavior, 34(3), 169-178.
[43]	Pouw W. T. J. L., de Nooijer J. A., van Gog T., Zwaan R. A., & Paas F . (2014). Toward a more embedded/extended perspective on the cognitive function of gestures. Frontiers in Psychology, 5(5), 1-14.
[44]	Raos V., Umilta M-A., Murata A., Fogassi L., & Gallese V . (2006). Functional properties of grasping-related neurons in the ventral premotor area F5 of the macaque monkey. Journal of Neurophysiology, 95(2), 709-729.
[45]	Rimé B., Schiaratura L., Hupet M., & Ghysselinckx A., . (1984). Effects of relative immobilization on the speaker’s nonverbal behavior and on the dialogue imagery level. Motivation & Emotion, 8(4), 311-325.
[46]	Rizzolatti G., Camarda R., Fogassi M., Gentilucci M., Luppino G., Matelli M . (1988). Functional organization of inferior area 6 in the macaque monkey: II. Area F5 and the control of distal movements. Experimental Brain Research, 71(3), 491-507.
[47]	Romano D., Marini F., & Maravita A . (2017). Standard body-space relationships: Fingers hold spatial information. Cognition, 165, 105-112.
[48]	Sixtus E., Fischer M. H., & Lindemann O . (2017). Finger posing primes number comprehension. Cognitive Processing, 18(3), 237-248.
[49]	So W. C., Ching T. H.-W., Lim P. E., Cheng X. Q., & Lp K. Y . (2014). Producing gestures facilitates route learning. Plos One, 9(11), e112543.
[50]	Straube B., Green A., Bromberger B., & Kircher T . (2011). The differentiation of iconic and metaphoric gestures: Common and unique integration processes. Human Brain Mapping, 32(4), 520-533.
[51]	Tucker M., & Ellis R . (2001). The potentiation of grasp types during visual object categorization. Visual Cognition, 8(6), 769-800.
[52]	Umilta M. A., Brochier T., Spinks R. L., Lemon R. N . (2007). Simultaneous recording of macaque premotor and primary motor cortex neuronal populations reveals different functional contributions to visuomotor grasp. Journal of Neurophysiology, 98(1), 488-501
[53]	Wagner S. M., Nusbaum H., & Goldin-meadow S . (2004). Probing the mental representation of gesture: Is handwaving spatial?. Journal of Memory & Language, 50(4), 395-407
[54]	Wesp R., Hesse J., Keutmann D., & Wheaton K . (2001). Gestures maintain spatial imagery. American Journal of Psychology, 114(4), 591-600.
[55]	Wheeler M . (2013). Is cognition embedded or extended? The case of gestures. In Z. Radman (Ed.), The hand, an organ of the mind: What the manual tells the mental (pp. 269-301). Cambridge, Britain: MIT Press.
[56]	Zwaan R. A., & Taylor L. J . (2006). Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology: General, 135(1), 1-11.