ISSN 1671-3710
CN 11-4766/R
主办:中国科学院心理研究所
出版:科学出版社

Advances in Psychological Science ›› 2023, Vol. 31 ›› Issue (suppl.): 174-174.

Previous Articles     Next Articles

MRGazerII: Camera-free Decoding Eye Movements from Functional Magnetic Resonance Imaging

Rongjie Hua, Jie Lianga, Yiwen Dingb, Shuang Jianb, Xiuwen Wua, Yanming Wanga, Zhen Liangb, Bensheng Qiua, Xiaoxiao Wanga   

  1. aCenter for Biomedical Imaging, University of Science and Technology of China, Hefei, China;
    bAnhui Medical University, Hefei, China
  • Online:2023-08-26 Published:2023-09-08

Abstract: PURPOSE: A raw fMRI-based end-to-end deep learning model, MRGazerII, was proposed to recognize eye-movement state at the temporal interval of tens of millisecond.
METHODS: The movie-watching fMRI data we used were from the Human Connectome Project (HCP) 7T release, in which each subject watched four movies with their eye movements recorded. The binary morphology method was then used to segment the eye domains and the intermediate 6-layer slices of the eye balls were extracted and fed into the deep neural network for the eye movement prediction. A series of ResNet-CBAM, Transformer encoder and fully connected layer were assembled to give slice-level prediction. The dataset was categorized into training and validating group at cross-subject level.
RESULTS: The proposed model achieves an accuracy of 0.48 in the classification of eye movements with f1-scores of 0.56, 0.51 and 0.34 for the fixation, blink and saccade respectively. Correlation analysis show that the prediction of fixation (averaged r = 0.28) and blink (averaged r = 0.46) were highly correlated with the records of the eye tracker.
CONCLUSIONS: The proposed MRGazerII, a camera-free eye tracking method, is able to report typical eye movements at temporal interval of tens of milliseconds and would be helpful to future fMRI & eye-movement analysis.

Key words: fMRI, deep learning, eye movement