Temporal dynamics of eye movements and attentional modulation in perceptual judgements of structure-from-motion (SFM)

doi:10.3724/SP.J.1041.2021.00337

Abstract

Abstract:

By using depth visual cues, the eye movements of 22 participants were recorded during perceiving the 3D SFM (structure from motion) to explore the effect of attention on SFM perception and its time course. The results showed that the judgement of cue stimulus was faster and more certain (higher percentage) than that of an ambiguous stimulus. The directions of eye movement and microsaccade were consistent with that of perceptual judgement, respectively. Microsaccade frequency (rate), peak velocity, and amplitude also showed the facilitating effect of depth cue. The results suggest that SFM can be divided into two stages: velocity calculation and construction of the three-dimensional structure. The modulation of attention to SFM perception mainly occurs in the construction of the three-dimensional structure stage, and attention orients to the selected object from 150 ms. After staying about 200 ms, attention will shift to the perception of global motion direction from local motion vector flow.

Key words: structure-from-motion (SFM), attention, microsaccade, time course, perceptual grouping

DING Jinhong, WANG Yamin, JIANG Yang. (2021). Temporal dynamics of eye movements and attentional modulation in perceptual judgements of structure-from-motion (SFM). Acta Psychologica Sinica, 53(4), 337-348.

Figures/Tables 5

Figure 1. Stimulus and possible percepts (a) and experimental procedure (b).

Figure 2. The percentage of judgement (a) and reaction time (b) to different stimuli.
Note. * stands for p < 0.05; ** stands for p < 0.01; the error bar is standard error

Figure 3. The eye position and its change with the time of exposure to different stimuli. (a) the spatial distribution of eye position; (b) the temporal course of eye position changing with stimuli.
Note. In (a), 0 point is the center of the stimulus, which is the center of the screen. The values for “Horizontal” and “Vertical” are greater than 0 to the right and below the center position; less than 0 is to the left and above.

Figure 4. Comparison of microsaccade parameters under different conditions. (a) peak velocity; (b) amplitude; (c) duration; (d) frequency.
Note. * stands for p < 0.05; ** stands for p < 0.01; the error bar is standard error.

Figure 5. Comparison of the time course of microsaccade frequency in different directions. (a) the microsaccade frequency to stimulus of CW and AMB when making cw judgement; (b) the frequency with CCW-ccw and AMB-ccw; (c) the frequency under the cued condition (CW-cw and CCW-ccw); (d) the frequency of microsaccade under conditions of AMB-cw and AMB-ccw.
Note. The shadow part was the significant time window; * stands for p < 0.05; ** stands for p < 0.01.

References 83

[1]	Alais D., Apthorp D., Karmann A., & Cass J. (2011). Temporal integration of movement: The time-course of motion streaks revealed by masking. PLoS ONE, 6(12), e28675. doi: https://doi.org/10.1371/journal.pone.0028675 doi: 10.1371/journal.pone.0028675 URL pmid: 22205961
[2]	Andersen R. A. & Bradley D. C. (1998). Perception of three-dimensional structure from motion. Trends in Cognitive Sciences, 2(6), 222-228. doi: 10.1016/s1364-6613(98)01181-4 URL pmid: 21227176
[3]	Andersen S. K., & Muller M. M. (2010). Behavioral performance follows the time course of neural facilitation and suppression during cued shifts of feature-selective attention. Proceedings of the National Academy of Sciences, 107(31), 13878-13882.
[4]	Aydın M., Herzog M. H., & Öğmen H. (2011). Attention modulates spatio-temporal grouping. Vision Research, 51(4), 435-446. URL pmid: 21266181
[5]	Bartlett L. K., Graf E. W., Hedger N., & Adams W. J. (2019). Motion adaptation and attention: A critical review and meta-analysis. Neuroscience & Biobehavioral Reviews, 96, 290-301. doi: 10.1016/j.neubiorev.2018.10.010 URL pmid: 30355521
[6]	Bonneh Y. S., Adini Y., & Polat U. (2015). Contrast sensitivity revealed by microsaccades. Journal of Vision, 15(9), 1-12. doi: 10.1167/15.9.1 URL pmid: 26131592
[7]	Born R. T., & Pack C. C. (2002). Integration of motion signals for smooth pursuit eye movements. Annals of the New York Academy of Sciences, 956(1), 453-455. doi: 10.1111/nyas.2002.956.issue-1 URL
[8]	Burr D., & Thompson P. (2011). Motion psychophysics: 1985-2010. Vision Research, 51(13), 1431-1456. doi: 10.1016/j.visres.2011.02.008 URL
[9]	Cai L. T., Yuan A. E., & Backus B. T. (2019). Binocular global motion perception is improved by dichoptic segregation when stimuli have high contrast and high speed. Journal of Vision, 19(13), 1-17. doi: 10.1167/19.13.1 URL pmid: 31675057
[10]	Caplovitz G. P., & Tse P. U. (2007). Rotating dotted ellipses: Motion perception driven by grouped figural rather than local dot motion signals. Vision Research, 47(15), 1979-1991. doi: 10.1016/j.visres.2006.12.022 URL pmid: 17548102
[11]	Cavanagh P. (1992). Attention-based motion perception. Science, 257(5076), 1563-1565. doi: 10.1126/science.1523411 URL pmid: 1523411
[12]	Cavanagh P., Hunt A. R., Afraz A., & Rolfs M. (2010). Visual stability based on remapping of attention pointers. Trends in Cognitive Science, 14(4), 147-153. doi: 10.1016/j.tics.2010.01.007 URL
[13]	Chung C. Y. L., & Khuu S. K. (2014). The processing of coherent global form and motion patterns without visual awareness. Frontiers in Psychology, 5. doi: https://doi.org/10.3389/fpsyg.2014.00195. doi: 10.3389/fpsyg.2010.00005 URL pmid: 21833188
[14]	Cohen J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates.
[15]	Conci M., Töllner T., Leszczynski M., & Müller H. J. (2011). The time-course of global and local attentional guidance in Kanizsa- figure detection. Neuropsychologia, 49(9), 2456-2464. doi: 10.1016/j.neuropsychologia.2011.04.023 URL pmid: 21549722
[16]	Deubel H. (2008). The time course of presaccadic attention shifts. Psychological Research, 72(6), 630-640. doi: 10.1007/s00426-008-0165-3 URL
[17]	Di Stasi L. L., Catenad A., Cañasc J. J., Macknike S. L., & Martinez-Conde S. (2013). Saccadic velocity as an arousal index in naturalistic tasks. Neuroscience and Biobehavioral Reviews, 37(5), 968-975. doi: 10.1016/j.neubiorev.2013.03.011 URL pmid: 23541685
[18]	Dombrowe I. C., Olivers C. N. L., & Donk M. (2010). The time course of color and luminance-based salience effects. Frontiers in Psychology, 1. doi: https://doi.org/10.3389/fpsyg.2010.00189 URL pmid: 21833184
[19]	Egeth H. E., & Yantis S. (1997). Visual attention: Control, representation, and time course. Annual Review of Psychology, 48, 269-297. doi: 10.1146/annurev.psych.48.1.269 URL pmid: 9046562
[20]	Engbert R., & Kliegl R. (2003). Microsaccades uncover the orientation of covert attention. Vision Research, 43(9), 1035-1045. URL pmid: 12676246
[21]	Fernandez J. M., & Farell B. (2006). A reversed structure-from-motion effect for simultaneously viewed stereo-surfaces. Vision Research, 46(8-9), 1230-1241. doi: 10.1016/j.visres.2005.10.026 URL pmid: 16356526
[22]	Festman Y., & Braun J. (2012). Feature-based attention spreads preferentially in an object-specific manner. Vision Research, 54(1), 31-38. doi: 10.1016/j.visres.2011.12.003 URL
[23]	Forschack N., Andersen S. K., & Müller M. M. (2017). Global enhancement but local suppression in feature-based attention. Journal of Cognitive Neuroscience, 29(4), 619-627. URL pmid: 27897668
[24]	Franconeri S., & Handy T. (2007). Rapid shifts of attention between two objects during spatial relationship judgements. Journal of Vision, 7(9), 582-582a. doi: https://doi.org/10.1167/7.9.582 doi: 10.1167/7.9.582 URL
[25]	Grosbras M. H., Laird A. R., & Paus T. (2005). Cortical regions involved in eye movements, shifts of attention, and gaze perception. Human Brain Mapping, 25(1), 140-154. doi: 10.1002/hbm.20145 URL pmid: 15846814
[26]	Gumming B. G., & Parker A. J. (1994). Binocular mechanisms for detecting motion-in-depth. Vision Research, 34(4), 483-495. doi: 10.1016/0042-6989(94)90162-7 URL pmid: 8303832
[27]	Hafed Z. M., & Clark J. J. (2002). Microsaccades as an overt measure of covert attention shifts. Vision Research, 42(22), 2533-2545. doi: 10.1016/s0042-6989(02)00263-8 URL pmid: 12445847
[28]	Haith M. M. (1966). The response of the human newborn to visual movement. Journal of Experimental Child Psychology, 3(3), 235-243. URL pmid: 5945067
[29]	Hedges J. H., Gartshteyn Y., Kohn A., Rust N. C., Shadlen M. N., Newsome W. T., & Movshon J. A. (2011). Dissociation of neuronal and psychophysical responses to local and global motion. Current Biology, 21(23), 2023-2028. doi: 10.1016/j.cub.2011.10.049 URL pmid: 22153156
[30]	Hermens F., & Walker R. (2010). What determines the direction of microsaccades? Journal of Eye Movement Research, 3(4), 1-20.
[31]	Hirshkowitz A., Biondi M., & Wilcox T. (2017). Cortical responses to shape-from-motion stimuli in the infant. Neurophoton, 5(1), 011014. doi: https://doi.org/10.1117/1.nph.5.1.011014
[32]	Horwitz G. D., & Albright T. D. (2003). Short-latency fixational saccades induced by luminance increments. Journal of Neurophysiology, 90(2), 1333-1339. doi: 10.1152/jn.00146.2003 URL pmid: 12904512
[33]	Hubel D. H., & Wiesel T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology, 160(1), 106-154. doi: 10.1113/jphysiol.1962.sp006837 URL
[34]	Ishii T., Motoyoshi I., & Kamachi M. G. (2013). Removal of attention facilitates global motion detection. The Japanese Journal of Psychonomic Science, 32(1), 135-136.
[35]	Jiang Y., Boehler C. N., Nönnig N., Düzel E., Hopf J. M., Heinze H. J., & Schoenfeld M. A. (2008). Binding 3-D object perception in the human visual cortex. Journal of Cognition Neuroscience, 20(4), 553-562.
[36]	Jiang Y., Pantle A. J., & Mark L. S. (1998). Visual inertia of rotating 3-D objects. Perception & Psychophysics, 60(2), 275-286. doi: 10.3758/bf03206036 URL pmid: 9529911
[37]	Johnson S. P., Davidow J., Hall-Haro C., & Frank M. C. (2008). Development of perceptual completion originates in information acquisition. Developmental Psychology, 44(5), 1214-1224. doi: 10.1037/a0013215 URL pmid: 18793055
[38]	Kaneko H., Itakura S., & Inagami M. (2011). Relationship between the frequency of microsaccade and attentional state. i-Perception, 2(4), 332-332.
[39]	Kasai T., & Takeya R. (2012). Time course of spatial and feature selective attention for partly-occluded objects. Neuropsychologia, 50(9), 2281-2289. URL pmid: 22683447
[40]	Krueger E., Schneider A., Sawyer B., Chavaillaz A., Sonderegger A., Groner R., & Hancock P. (2019). Microsaccades distinguish looking from seeing. Journal of Eye Movement Research, 12(6). doi: https://doi.org/10.16910/jemr.12.6.2
[41]	Kuldkepp N., Kreegipuu K., Raidvee A., Näätänen R., & Allik J. (2013). Unattended and attended visual change detection of motion as indexed by event-related potentials and its behavioral correlates. Frontiers in Human Neuroscience, 7, 476. doi: https://doi.org/10.3389/fnhum.2013.00476 doi: 10.3389/fnhum.2013.00476 URL pmid: 23966932
[42]	Lamberty K., Gobbelé R., Schoth F., Buchner H., & Waberski T. D. (2008). The temporal pattern of motion in depth perception derived from ERPs in humans. Neuroscience Letters, 439(2), 198-202. doi: 10.1016/j.neulet.2008.04.101 URL pmid: 18514406
[43]	Laubrock J., Engbert R., & Kliegl R. (2008). Fixational eye movements predict the perceived direction of ambiguous apparent motion. Journal of Vision, 8(14), 1-17. doi: 10.1167/8.14.10 URL pmid: 19146311
[44]	Lin Y., & Tadin D. (2019). Motion perception: Slow development of center-surround suppression. Current Biology, 29(18), R878-R880. doi: 10.1016/j.cub.2019.07.079 URL pmid: 31550474
[45]	Martínez G. A. R., & Parra H. C. (2018). Bistable perception: Neural bases and usefulness in psychological research. International Journal of Psychological Research, 11(2), 63-76. URL pmid: 32612780
[46]	Meyberg S., Sinn P., Engbert R., & Sommer W. (2017). Revising the link between microsaccades and the spatial cueing of voluntary attention. Vision Research, 133, 47-60. doi: 10.1016/j.visres.2017.01.001 URL pmid: 28163059
[47]	Meyberg S., Sommer W., & Dimigen O. (2017). How microsaccades relate to lateralized ERP components of spatial attention: A co-registration study. Neuropsychologia, 99, 64-80. doi: 10.1016/j.neuropsychologia.2017.02.023 URL pmid: 28254651
[48]	Motoyoshi I., Ishii T., & Kamachi M. G. (2015). Limited attention facilitates coherent motion processing. Journal of Vision, 15(13), 1. doi: https://doi.org/10.1167/15.13.1 URL pmid: 26327254
[49]	Nishida S., Kawabe T., Sawayama M., & Fukiage T. (2018). Motion perception: From detection to interpretation. Annual Review of Vision Science, 4(20), 501-523. doi: 10.1146/annurev-vision-091517-034328 URL
[50]	Otero-Millan J., Castro J. L. A., Macknik S. L., & Martinez-Conde S. (2014). Unsupervised clustering method to detect microsaccades. Journal of Vision, 14(2), 1-17. doi: 10.1167/14.2.1 URL pmid: 24492596
[51]	Papathomas T. V., Gorea A., & Julesz B. (1991). Two carriers for motion perception: Color and luminance. Vision Research, 31(11), 1883-1891. doi: 10.1016/0042-6989(91)90183-6 URL pmid: 1771772
[52]	Park W. J., & Tadin D. (2018). Motion perception. In The Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience: Sensation, Perception and Attention, 4th Edition, J. Serences, ed. (Wiley), pp. 415-488.
[53]	Peterson M. S., & Kramer A. F. (2001). Attentional guidance of the eyes by contextual information and abrupt onsets. Perception & Psychophysics, 63, 1239-1249. doi: 10.3758/bf03194537 URL pmid: 11766947
[54]	Pomerantz J. R. (1970). Eye movements affect the perception of apparent (beta) movement. Psychological Science, 19(4), 193-194.
[55]	Ramachandran V. S., & Anstis S. M. (1983). Perceptual organization in moving patterns. Nature, 304, 529-531. doi: 10.1038/304529a0 URL pmid: 6877373
[56]	Ramirez-Moreno D. F., Schwartz O., & Ramirez-Villegas J. F. (2013). A saliency-based bottom-up visual attention model for dynamic scenes analysis. Biological Cybernetics, 107(2), 141-160. URL pmid: 23314730
[57]	Rider A. T., Nishida S., & Johnston A. (2016). Multiple-stage ambiguity in motion perception reveals global computation of local motion directions. Journal of Vision, 16(15), 7, 1-11. doi: 10.1167/16.15.1 URL pmid: 27918785
[58]	Rock I., Halper F., DiVita J., & Wheeler D. (1987). Eye movement as a cue to figure motion in anorthoscopic perception. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 344-352. URL pmid: 2958583
[59]	Rolfs M., Engbert R., & Kliegl R. (2004). Microsaccade orientation supports attentional enhancement opposite a peripheral cue: Commentary on Tse, Sheinberg, and Logothetis (2003). Psychological Science, 15(10), 705-707. doi: 10.1111/j.0956-7976.2004.00744.x URL pmid: 15447643
[60]	Ryan A. E., Keane B., & Wallis G. (2019). Microsaccades and covert attention: Evidence from a continuous, divided attention task. Journal of Eye Movement Research, 12(6). doi: https://doi.org/10.16910/jemr.12.6.6
[61]	Sara G., Tony P., Roberto B., Kerstin H., & Mariagrazia B. (2017). The effect of luminance condition on form, form-from-motion and motion perception. Frontiers in Cognitive Psychology, 2(2), 65-72.
[62]	Schmitt C., Klingenhoefer S., & Bremmer F. (2018). Preattentive and Predictive Processing of Visual Motion. Scientific Reports, 8, 12399. doi: https://doi.org/10.1038/s41598-018-30832-9 URL pmid: 30120337
[63]	Schütz A. C., Braun D. I., & Gegenfurtner K. R. (2011). Eye movements and perception: A selective review. Journal of Vision, 11(5), 9, 1-30. doi: 10.1167/11.5.1 URL pmid: 21536727
[64]	Scocchia L., Valsecchi M., & Triesch J. (2014). Top-down influences on ambiguous perception: The role of stable and transient states of the observer. Frontiers in Human Neuroscience, 8. doi: https://doi.org/10.3389/fnhum.2014.00979 URL pmid: 18958209
[65]	Simion F., Regolin L., & Bulf H. (2008). A predisposition for biological motion in the newborn baby. Proceedings of the National Academy of Sciences of the United States of America, 105(2), 809-813.
[66]	Smith K. C., & Abrams R. A. (2018). Motion onset really does capture attention. Attention, Perception & Psychophysics, 80(7), 1775-1784.
[67]	Stelmach L. B., Herdman C. M., & McNeil K. R. (1994). Attentional modulation of visual processes in motion perception. Journal of Experimental Psychology: Human Perception and Performance, 20(1), 108-121. doi: 10.1037/0096-1523.20.1.108 URL
[68]	Stone L. S., & Thompson P. (1992). Human speed perception is contrast dependent. Vision Research, 32(8), 1535-1549. URL pmid: 1455726
[69]	Stonkute S., Braun J., & Pastukhov A. (2012). The role of attention in ambiguous reversals of structure-from-motion. PLoS ONE, 7(5), e37734. doi: https://doi.org/10.1371/journal.pone.0037734 doi: 10.1371/journal.pone.0037734 URL pmid: 22629450
[70]	Stoppel C. M., Boehler C. N., Strumpf H., Krebs R. M., Heinze H. J., Hopf J. M., & Schoenfeld M. A. (2012). Spatiotemporal dynamics of feature-based attention spread: Evidence from combined electroencephalographic and magnetoencephalographic recordings. Journal of Neuroscience, 32(28), 9671-9676. doi: 10.1523/JNEUROSCI.0439-12.2012 URL pmid: 22787052
[71]	Thompson P. (1982). Perceived rate of movement depends on contrast. Vision Research, 22(3), 377-380. doi: 10.1016/0042-6989(82)90153-5 URL pmid: 7090191
[72]	Treisman A., & Gelade G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97-136. doi: 10.1016/0010-0285(80)90005-5 URL pmid: 7351125
[73]	Treisman A., & Gormican S. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Review, 95(1), 15-48. URL pmid: 3353475
[74]	Treue S., Husain M., & Andersen R. A. (1991). Human perception of structure from motion. Vision Research, 31(1), 59-75. doi: 10.1016/0042-6989(91)90074-f URL pmid: 2006555
[75]	Tsal Y., & Kolbet L. (1985). Disambiguating ambiguous figures by selective attention. The Quarterly Journal of Experimental Psychology Section A, 37(1), 25-37. doi: 10.1080/14640748508400950 URL
[76]	van Rullen R., Reddy L., & Koch C. (2005). Attention-driven discrete sampling of motion perception. Proceedings of the National Academy of Sciences of the United States of America, 102(14), 5291-5296.
[77]	van Zoest W., Donk M., & van der Stigchel S. (2012). Stimulus salience and the time-course of saccade trajectory deviations. Journal of Vision, 12(8), 16-16. doi: 10.1167/12.8.16 URL pmid: 22923727
[78]	van Zoest W., Heimler B., & Pavani F. (2016). The oculomotor salience of flicker, apparent motion and continuous motion in saccade trajectories. Experimental Brain Research, 235(1), 181-191. doi: 10.1007/s00221-016-4779-1 URL pmid: 27683004
[79]	Wallach H., & O'Connell D. N. (1953). The kinetic depth effect. Journal of Experimental Psychology, 45(4), 205-217. doi: 10.1037/h0056880 URL pmid: 13052853
[80]	Wang L., Yang X., Shi J., & Jiang Y. (2014). The feet have it: Local biological motion cues trigger reflexive attentional orienting in the brain. NeuroImage, 84, 217-224. doi: 10.1016/j.neuroimage.2013.08.041 URL
[81]	Ward R., Duncan J., & Shapiro K. (1996). The slow time-course of visual attention. Cognitive Psychology, 30(1), 79-109. doi: 10.1006/cogp.1996.0003 URL pmid: 8660782
[82]	Wasmuht D. F., Parker A. J., & Krug K. (2019). Interneuronal correlations at longer time scales predict decision signals for bistable structure-from-motion perception. Scientific Reports, 9, 11449. doi: https://doi.org/10.1038/s41598-019-47786-1 doi: 10.1038/s41598-019-47786-1 URL pmid: 31391489
[83]	Zhang Y., Li A., Han Y., Zhang S., & Zhang M. (2016). The effect of microsaccade types on attention. Journal of Sichuan Normal University (Social Sciences Edition), 43(6), 29-37.