动、静态视觉信息在真实世界视觉搜索中的作用

doi:10.3724/SP.J.1042.2020.01219

摘要/Abstract

摘要：

真实环境中的视觉搜索是人和动物赖以生存的重要能力。目前的视觉搜索研究多使用静态的观察者和静止的二维搜索对象, 侧重于探究注意在搜索中的作用; 现有的视觉搜索理论模型主要概括了影响搜索的自上而下的注意因素, 而将自下而上影响因素简单归结为影像显著性, 然而在真实环境中, 观察者或搜索对象是可以运动的, 搜索时可利用的视觉信息包括动态光流和静态影像结构信息。已有的视觉识别研究发现这两种信息相结合可以使观察者准确持久地识别场景、事件和三维结构。在现有视觉搜索理论模型中引入两种视觉信息可以较好还原真实环境中的搜索任务。我们提出研究构想和实验方案,探究利用动、静态视觉信息的视觉搜索过程, 从而完善现有的视觉搜索模型。我们认为充分利用环境信息可以提高搜索效率, 且在视觉搜索训练和智能搜索设计等方面有重要的应用价值。

关键词: 视觉搜索, 光流, 生物运动, 生态知觉理论

Abstract:

Visual search is a ubiquitous task and a critical skill for men and animals. Existing studies on visual search mainly focus on attentional guidance and the top-down cognitive influences on search effectiveness. The bottom-up influence on visual search is, rather crudely, simplified as objects’ image saliency. However, when searching in real world, where the observer and/or objects move, both static image information (the saliency of which has been considered in existing search models) and dynamic optic flow information are available. Optic flow is generated by the relative motions between an observer and world objects. So by detecting flow patterns, observers get to know the kinematic properties of events (which is defined as objects in motion) and hence perceive the physical properties of constituent objects, such as the mass, size and frictional coefficient etc.. These physical properties distinguish objects and allow the observer to search for a particular one. We integrate dynamical perceptual information (i.e. optic flow) into existing search models and in two studies, we test how combined dynamical and static perceptional information affect visual search for three-dimensional objects and for moving people, when the observer is stationary or moving. Furthermore, we attempt to develop a training protocol that improves search effectiveness in real world. Findings from this project will bring forth new theories for understanding visual search in real world, and have direct applications on personnel training and intelligent search designs.

Key words: visual search, optical flow, biological motion, ecological theory of perception

中图分类号:

B842

潘静, 张慧远, 陈东濠, 徐宏格. (2020). 动、静态视觉信息在真实世界视觉搜索中的作用. 心理科学进展 , 28(8), 1219-1231.

PAN Jing, ZHANG Huiyuan, CHEN Donghao, XU Hongge. (2020). Visual search in real world: The role of dynamic and static optical information. Advances in Psychological Science, 28(8), 1219-1231.

图/表 3

图1 理论模型框架。我们认为视觉信息对真实环境里的视觉搜索起到自下而上的影响, 尤其是光流以及光流和影像结构的交互。本研究补充了光流信息在视觉搜索中的自下而上的作用, 但是运动产生的视觉、运动觉及本体觉信息可能以其它方式对视觉搜索造成自上而下的影响, 有待后续研究。图中实线框为现有理论模型, 主要基于Wolfe的引导搜索模型。虚线框内为本文提出理论构想的关键词。

图2 生态光学理论总结。Gibson认为观察者利用环境光里的视觉信息完成知觉任务。环境光里包括静态影像结构信息和动态光流信息。

图3 本项目的技术路线总图。其中分三个研究, 分别对应静态三维物体搜索、事件搜索以及视觉搜索训练。

参考文献 71

[1]	Bahle, B., Matsukura, M., & Hollingworth, A. (2018). Contrasting gist-based and template-based guidance during real-world visual search. Journal of Experimental Psychology: Human Perception and Performance, 44(3), 367-386. doi: 10.1037/xhp0000468 URL pmid: 28795834
[2]	Bingham, G. P., Schmidt, R. C., & Rosenblum, L. D. (1995). Dynamics and the orientation of kinematic forms in visual event recognition. Journal of Experimental Psychology: Human Perception and Performance, 21(6), 1473-1493. doi: 10.1037//0096-1523.21.6.1473 URL pmid: 7490589
[3]	Broadbent, D. E. (1991). A word before leaving. In D. E. Meyer & S. Kornblum (Eds.), Attention and performance XIV (pp. 863-879). Cambridge, MA: Bradford Books/MIT Press.
[4]	Brooks, D. I., Rasmussen, I. P., & Hollingworth, A. (2010). The nesting of search contexts within natural scenes: Evidence from contextual cuing. Journal of Experimental Psychology: Human Perception and Performance, 36(6), 1406-1418. doi: 10.1037/a0019257 URL pmid: 20731525
[5]	Cavanagh, P., Labianca, A. T., & Thornton, I. M. (2001). Attention-based visual routines: Sprites. Cognition, 80(1-2), 47-60. URL pmid: 11245839
[6]	De Vries, J. P., Hooge, I. T. C., Wertheim, A. H., & Verstraten, F. A. J. (2013). Background, an important factor in visual search. Vision Research, 86, 128-138. doi: 10.1016/j.visres.2013.04.010 URL
[7]	Ding, X., Yin, J., Shui, R., Zhou, J., & Shen, M. (2017). Backward-walking biological motion orients attention to moving away instead of moving toward. Psychonomic Bulletin & Review, 24(2), 447-452. doi: 10.3758/s13423-016-1083-9 URL pmid: 27368634
[8]	Domini, F., Vuong, Q. C., & Caudek, C. (2002). Temporal integration in structure from motion. Journal of Experimental Psychology: Human Perception and Performance, 28(4), 816-838. URL pmid: 12190252
[9]	Drew, T., Boettcher, S. E. P., & Wolfe, J. M. (2016). Searching while loaded: Visual working memory does not interfere with hybrid search efficiency but hybrid search uses working memory capacity. Psychonomic Bulletin & Review, 23(1), 201-212. URL pmid: 26055755
[10]	Drew, T., Boettcher, S. E. P., & Wolfe, J. M. (2017). One visual search, many memory searches: An eye-tracking investigation of hybrid search. Journal of Vision, 17(11), 5. doi: 10.1167/17.11.5 URL pmid: 28892812
[11]	Duncan, J. S., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96(3), 433-458. doi: 10.1037/0033-295x.96.3.433 URL pmid: 2756067
[12]	Ehinger, K. A., & Wolfe, J. M. (2016). When is it time to move to the next map? Optimal foraging in guided visual search. Attention, Perception, & Psychophysics, 78(7), 2135-2151.
[13]	Eriksen, C. W., & Schultz, D. W. (1979). Information processing in visual search: A continuous flow conception and experimental results. Perception & Psychophysics, 25(4), 249-263. doi: 10.3758/bf03198804 URL pmid: 461085
[14]	Foulsham, T., Chapman, C. S., Nasiopoulos, E., & Kingstone, A. (2014). Top-down and bottom-up aspects of active search in a real-world environment. Canadian Journal of Experimental Psychology, 68(1), 8-19. doi: 10.1037/cep0000004 URL pmid: 24219246
[15]	Foulsham, T., & Underwood, G. (2009). Does conspicuity enhance distraction? Saliency and eye landing position when searching for objects. The Quarterly Journal of Experimental Psychology, 62(6), 1088-1098. doi: 10.1080/17470210802602433 URL pmid: 19142829
[16]	Gibson, J. J. (1958). Visually controlled locomotion and visual orientation in animals. British Journal of Psychology, 49(3), 182-194. URL pmid: 13572790
[17]	Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc. (Original work published 1979).
[18]	Gold, J. M., Tadin, D., Cook, S. C., & Blake, R. (2008). The efficiency of biological motion perception. Attention Perception & Psychophysics, 70(1), 88-95.
[19]	Henderson, J. M., & Hayes, T. R. (2017). Meaning-based guidance of attention in scenes as revealed by meaning maps. Nature Human Behaviour, 1(10), 743-747. doi: 10.1038/s41562-017-0208-0 URL pmid: 31024101
[20]	Henderson, J. M., Malcolm, G. L., & Schandl, C. (2009). Searching in the dark: Cognitive relevance drives attention in real-world scenes. Psychonomic Bulletin & Review, 16(5), 850-856. URL pmid: 19815788
[21]	Hickey, C., Chelazzi, L., Theeuwes, J., & Geng, J. J. (2014). Reward-priming of location in visual search. PLoS ONE, 9(7), e103372. URL pmid: 25080218
[22]	Hirai, M., & Hiraki, K. (2006). Visual search for biological motion: An event-related potential study. Neuroscience Letters, 403(3), 299-304. URL pmid: 16716511
[23]	Kamkar, S., Moghaddam, H. A., & Lashgari, R. (2018). Early visual processing of feature saliency tasks: A review of psychophysical experiments. Frontiers in Systems Neuroscience, 12, 54. doi: 10.3389/fnsys.2018.00054 URL pmid: 30416433
[24]	Kingstone, A., Smilek, D., & Eastwood, J. D. (2008). Cognitive ethology: A new approach for studying human cognition. British Journal of Psychology, 99, 317-340. URL pmid: 17977481
[25]	Kingstone, A., Smilek, D., Ristic, J., Friesen, C. K., & Eastwood, J. D. (2003). Attention, researchers! It is time to take a look at the real world. Current Directions in Psychological Science, 12(5), 176-180.
[26]	Koch, C., & Ullman, S. (1987). Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiology, 4(2), 115-141. URL pmid: 4030421
[27]	Koehler, K., Guo, F., Zhang, S., & Eckstein, M. P. (2014). What do saliency models predict. Journal of Vision, 14(3), 14-14. doi: 10.1167/14.3.14 URL pmid: 24618107
[28]	Kristjánsson, Á., Jóhannesson, Ó. I., & Thornton, I. M. (2014). Common attentional constraints in visual foraging. Plos One, 9(6), e100752. URL pmid: 24964082
[29]	Lee, Y. L., Lind, M., Bingham, N. & Bingham, G. P. (2012). Object recognition using metric shape. Vision Research, 69, 23-31. doi: 10.1016/j.visres.2012.07.013 URL pmid: 22884632
[30]	Lu, H., Tjan, B. S., & Liu, Z. (2017). Human efficiency in detecting and discriminating biological motion. Journal of Vision, 17(6), 4-4. URL pmid: 28593248
[31]	Mayer, K. M., Riddell, H., & Lappe, M. (2019). Concurrent processing of optic flow and biological motion. Journal of Experimental Psychology: General, 148(11), 1938-1952.
[32]	Mayer, K. M., Vuong, Q. C., & Thornton, I. M. (2015). Do People “Pop Out”?. PLOS ONE, 10(10), e0139618. doi: 10.1371/journal.pone.0139618 URL pmid: 26441221
[33]	Muchisky, M. M.. & Bingham, G. P. (2002). Trajectory forms as a source of information about events. Attention Perception & Psychophysics, 64(1), 15-31.
[34]	Nakayama, K., & Martini, P. (2011). Situating visual search. Vision Research, 51(13), 1526-1537. URL pmid: 20837042
[35]	Ort, E., Fahrenfort, J. J., & Olivers, C. N. L. (2017). Lack of free choice reveals the cost of having to search for more than one object. Psychological Science, 28(8), 1137-1147. URL pmid: 28661761
[36]	Pan, J. S., Bingham, N., & Bingham, G. P. (2013). Embodied memory: Effective and stable perception by combining optic flow and image structure. Journal of Experimental Psychology: Human Perception and Performance, 39(6), 1638-1651.
[37]	Pan, J. S., Bingham, N., & Bingham, G. P. (2017). Embodied memory allows accurate and stable perception of hidden objects despite orientation change. Journal of Experimental Psychology: Human Perception and Performance, 43(7), 1343-1358. doi: 10.1037/xhp0000392 URL pmid: 28301185
[38]	Pan, J. S., Bingham, N., Chen, C., & Bingham, G. P. (2017). Breaking camouflage and detecting targets require optic flow and image structure information. Applied Optics, 56(22), 6410-6418. URL pmid: 29047842
[39]	Pan, J. S., Li, J., Chen, Z., Mangiaracina, E. A., Connell, C. S., Wu, H., ... Hassan, S. E. (2017). Motion-generated optical information allows event perception despite blurry vision in AMD and amblyopic patients. Journal of Vision, 17(12), 13-13.
[40]	Ruddle, R. A., & Lessels, S. (2006). For efficient navigational search, humans require full physical movement, but not a rich visual scene. Psychological Science, 17(6), 460-465. doi: 10.1111/j.1467-9280.2006.01728.x URL pmid: 16771793
[41]	Runeson, S., & Frykholm, G. (1983). Kinematic specification of dynamics as an informational basis for person-and-action perception: Expectation, gender recognition, and deceptive intention. Journal of Experimental Psychology: General, 112(4), 585-615. doi: 10.1037/0096-3445.112.4.585 URL
[42]	Seidl-Rathkopf, K. N., Turk-Browne, N. B., & Kastner, S. (2015). Automatic guidance of attention during real-world visual search. Attention Perception & Psychophysics, 77(6), 1881-1895.
[43]	Smith, A. D., Hood, B. M., & Gilchrist, I. D. (2008). Visual search and foraging compared in a large-scale search task . Cognitive Processing, 9(2), 121-126. URL pmid: 18188627
[44]	Smith, A. D., Hood, B. M., & Gilchrist, I. D. (2010). Probabilistic cuing in large-scale environmental search . Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(3), 605-618. URL pmid: 20438260
[45]	Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011). Eye guidance in natural vision: Reinterpreting salience. Journal of Vision, 11(5), 5-5. doi: 10.1167/11.5.5 URL pmid: 21622729
[46]	Theeuwes, J., Kramer, A. F., & Belopolsky, A. V. (2004). Attentional set interacts with perceptual load in visual search. Psychonomic Bulletin & Review, 11(4), 697-702. URL pmid: 15581120
[47]	Todd, J. T., Tittle, J. S., & Norman, J. F. (1995). Distortions of three-dimensional space in the perceptual analysis of motion and stereo. Perception, 24(1), 75-86. URL pmid: 7617420
[48]	Torralba, A., Oliva, A., Castelhano, M. S., & Henderson, J. M. (2006). Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychological Review, 113(4), 766-786. URL pmid: 17014302
[49]	Treisman, A. (1982). Perceptual grouping and attention in visual search for features and for objects. Journal of Experimental Psychology, 8(2), 194-214. doi: 10.1037//0096-1523.8.2.194 URL pmid: 6461717
[50]	Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97-136. URL pmid: 7351125
[51]	Treisman, A., & Gormican, S. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Review, 95(1), 15-48. URL pmid: 3353475
[52]	Treisman, A., Sykes, M., & Gelade, G. (1977). Selective attention and stimulus integration. Attention and Performance VI, 333.
[53]	Van Boxtel, J. J., & Lu, H. (2011). Visual search by action category. Journal of Vision, 11(7), 19-19. URL pmid: 21709212
[54]	Vo, M. L., & Wolfe, J. M. (2012). When does repeated search in scenes involve memory? Looking at versus looking for objects in scenes. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 23-41. URL pmid: 21688939
[55]	Võ, M. L. H., & Wolfe, J. M. (2015). The role of memory for visual search in scenes. Annals of the New York Academy of Sciences, 1339(1), 72-81. doi: 10.1111/nyas.2015.1339.issue-1 URL
[56]	Wang, L., Zhang, K., He, S., & Jiang, Y. (2010). Searching for life motion signals: Visual search asymmetry in local but not global biological-motion processing. Psychological Science, 21(8), 1083-1089. doi: 10.1177/0956797610376072 URL
[57]	Wickelgren, E. A. & Bingham, G. P. (2004). Perspective distortion of trajectory forms and perceptual constancy in visual event identification. Attention Perception & Psychophysics, 66, 629-641.
[58]	Wickelgren, E. A., & Bingham, G.P. (2008). Trajectory forms as information for visual event recognition: 3D perspectives on path shape and speed profile. Attention Perception & Psychophysics, 70(2), 266-278.
[59]	Wolfe, J. M. (1994). Guided Search 2.0: A revised model of visual search. Psychonomic Bulletin &. Review, 1, 202-238 doi: 10.3758/BF03200774 URL pmid: 24203471
[60]	Wolfe, J. M. (2003). Moving towards solutions to some enduring controversies in visual search. Trends in Cognitive Sciences, 7(2), 70-76. URL pmid: 12584025
[61]	Wolfe, J. M., Boettcher, S. E. P., Josephs, E. L., Cunningham, C. A., & Drew, T. (2015). You look familiar, but I don’t care: Lure rejection in hybrid visual and memory search is not based on familiarity. Journal of Experimental Psychology: Human Perception and Performance, 41(6), 1576-1587. doi: 10.1037/xhp0000096 URL pmid: 26191615
[62]	Wolfe, J. M., Cain, M. S., & Aizenman, A. M. (2019). Guidance and selection history in hybrid foraging visual search. Attention, Perception, & Psychophysics, 81(3), 637-653.
[63]	Wolfe, J. M., Cain, M. S., Ehinger, K. A., & Drew, T. (2015). Guided Search 5.0: Meeting the challenge of hybrid search and multiple-target foraging. Journal of Vision, 15(12), 1106-1106.
[64]	Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989). Guided search: An alternative to the feature integration model for visual search. Journal of Experimental Psychology: Human Perception and Performance, 15(3), 419-433. URL pmid: 2527952
[65]	Wolfe, J. M., & Gancarz, G. (1997). Guided Search 3.0. In Basic and clinical applications of vision science (pp. 189-192). Dordrecht: Springer.
[66]	Wolfe, J. M., & Gray, W. (2007). Guided search 4.0. Integrated models of cognitive systems, 99-119.
[67]	Wolfe, J. M., & Horowitz, T. S. (2017). Five factors that guide attention in visual search. Nature Human Behaviour, 1(3), 0058.
[68]	Wolfe, J. M., & van Wert, M. J. (2010). Varying target prevalence reveals two dissociable decision criteria in visual search. Current Biology, 20(2), 121-124. URL pmid: 20079642
[69]	Woodman, G. F., & Chun, M. M. (2006). The role of working memory and long-term memory in visual search. Visual Cognition, 14(4-8), 808-830.
[70]	Wu, C., Wick, F. A., & Pomplun, M. (2014). Guidance of visual attention by semantic information in real-world scenes. Frontiers in Psychology, 5, 54-54. doi: 10.3389/fpsyg.2014.00054 URL pmid: 24567724
[71]	Wu, H., Wang, X. M., & Pan, J. S. (2019). Perceiving blurry scenes with translational optic flow, rotational optic flow or combined optic flow. Vision Research, 158, 49-57. URL pmid: 30796993