Test-retest reliability of EEG: A comparison across multiple resting-state and task-state experiments

doi:10.3724/SP.J.1041.2023.01587

Abstract

Abstract:

Owing to its advantages in time resolution, electroencephalography (EEG) provides an important basis for studying the dynamic cognitive process of the human brain. To explore the electrophysiological mechanism of psychological processes, scalp EEG must have good test-retest reliability. Most studies explore the reliability of the resting-state EEG (rsEEG) or event-related potentials (ERP), lacking a comprehensive comparison of multiple states. We comprehensively compared the test-retest reliability of the two rsEEG with eyes-open (EO) and eyes-close (EC) states, and the ERPs of PVT and oddball tasks, from frequency, time, and spatial domains to identify more widely applicable indicators.

A total of 42 healthy adults (age range = 18-26 years old; mean = 19.5 ± 1.4 years old; 14 males) underwent all three EEG recording sessions, including the present (Session 1), 90 mins later (Session 2), and one month later (Session 3). During each EEG recording session, all the participants completed the same five states including two resting states (eyes-open, eyes-closed, each with 5 minutes) and two task states (PVT and oddball task) (Figure 1). Intra-class correlation coefficients (ICC) were computed to assess the test-retest reproducibility of the five states.

This study compares and analyses the test-retest reliability of two resting-state and three task-state EEG from the perspectives of time, frequency, and spatial domains. Results revealed the following: (1) The test-retest reliability of rsEEG was generally better than that of ERP (Figure 2). (2) For rsEEG, the test-retest reliability of the EC resting-state was higher than that of the EO, with the ICC median value of approximately 0.6 (Table 1). Furthermore, the test-retest reliability of the alpha band was the highest in all frequency bands. (3) For the two task-states ERP (Figure 2), the overall ICC of the PVT paradigm was higher than that of the oddball paradigm, and the test-retest reliability was highest at about 200 ms after the stimulus onsets (Figure 3). (4) In the spatial domain, the test-retest reliability is higher in the central region than in the peripheral region (Figure 4/5, Table 2), which may be related to the signal-to-noise ratio (SNR).

Our research involves multiple resting-state and task-state experiments. Based on the characteristics of frequency, time, and space domains, we comprehensively compared the optimal retest characteristics of multiple EEG and suggest the possible reasons. Some suggestions for the selection of appropriate experimental paradigms and indicators for the follow-up study of EEG test-retest reliability are provided and guide the application of EEG in the basic and clinical fields.

Key words: EEG, event-related potential, test-retest reliability

QIN Huiyi, DING Lihong, DUAN Wei, LEI Xu. (2023). Test-retest reliability of EEG: A comparison across multiple resting-state and task-state experiments. Acta Psychologica Sinica, 55(10), 1587-1596.

Figures/Tables 7

Figure 1. Spectrum (A) and waveform (B) of rsEEG and ERP in three sessions. The electrode positions are all Cz, with a solid line representing the mean and a light shadow representing the standard error. (A) The spectrum of two conditions in rsEEG, the horizontal axis represents the frequency, the vertical axis represents the absolute power value, the purple line represents the rsEEG with EO, and the green line represents the rsEEG with EC. (B) The waveform of three conditions in ERP, with the horizontal axis representing time, the vertical axis representing amplitude, the black line representing PVT, the yellow line representing Oddball-D, and the blue line representing Oddball-S.
Note. EO: eyes-open; EC: eyes-close; PVT: psychomotor vigilance task; Oddball-D: deviant stimulus of oddball task; Oddball-S: Standard stimulus of oddball task.

Figure 2. The distribution of ICC values in power (A) and amplitude (B) under 5 conditions, including all electrodes, frequency points (rsEEG), and time points (ERPs). The horizontal axis represents the condition, and the vertical axis represents the ICC.The black reference line represents the mean value, while the red reference line represents the median value. (A) Distribution of ICC under two conditions of rsEEG. (B) The distribution of ICC values under three conditions of ERP.
Note. EO: eyes-open; EC:eyes-close; PVT: psychomotor vigilance task; Oddball-D: deviant stimulus of oddball task; Oddball-S: Standard stimulus of oddball task.

Table 1 Overall distribution of ICC under five conditions

Statistical indicators	Experimental condition
Statistical indicators	EO	EC	PVT	Oddball-D	Oddball-S
Mean ± Standard Deviation	0.54 ± 0.16	0.55 ± 0.19	0.50 ± 0.13	0.33 ± 0.13	0.33 ± 0.16
Maximum value	0.91	0.94	0.82	0.68	0.80
Median value	0.53	0.56	0.51	0.35	0.35
Difference test	EC > EO, t = -2.71, p = 0.007		PVT > Oddball-D / Oddball-S, F = 753.16, p < 0.001

Figure 3. The variation of ICC of all electrodes under 5 conditions. The solid line represents the mean, and the light shadow represents the standard error. (A) The horizontal axis represents the frequency, the vertical axis represents the ICC, the purple line represents the rsEEG with EO, and the green line represents the rsEEG with EC. (B) The changes in ICC under three conditions of ERPe. the horizontal axis represents time, the vertical axis represents ICC, the black line represents PVT, the yellow line represents Oddball-D, and the blue line represents Oddball-S.
Note. EO: eyes-open; EC: eyes-close; PVT: psychomotor vigilance task; Oddball-D: deviant stimulus of oddball task; Oddball-S: Standard stimulus of oddball task.

Figure 4. The distribution of ICC corresponding to the alpha frequency band of rsEEG and the P2 component of ERP in the scalp
Note. EO: eyes-open; EC: eyes-close; PVT: psychomotor vigilance task; Oddball-D: deviant stimulus of oddball task; Oddball-S: Standard stimulus of oddball task.

Figure 5. The average ICC values of 2 regions of interest (central and peripheral area) under 5 conditions (EO, EC, PVT, Oddball-D, Oddball-S). The horizontal axis represents the condition and the vertical axis represents the ICC value. The blue bars represent the central area and the orange bars represent the peripheral area.
Note. The insertion plot shows the central and peripheral area of the electrode, with the central area in blue and the peripheral area in orange.

Table 2 ICC of two regions of interest(ROI) and their independent sample t-test results under 5 conditions

Statistical indicators	EO		EC		PVT		Oddball-D		Oddball-S
Statistical indicators	Central area	Peripheral area	Central area	Peripheral area	Central area	Peripheral area	Central area	Peripheral area	Central area	Peripheral area
Average Value of ICC	0.81	0.76	0.82	0.79	0.69	0.59	0.46	0.38	0.52	0.50
Variance of ICC	0.0007	0.0021	0.0010	0.0017	0.0026	0.0222	0.0072	0.0100	0.0106	0.0127
t	5.42		3.10		3.52		3.26		0.44
p	<0.001		0.003		0.001		0.002		0.661

[1]	ZHANG Wenyun, ZHUO Shiwei, ZHENG Qianqian, GUAN Yinglin, PENG Weiwei. Autistic traits influence pain empathy: The mediation role of pain-related negative emotion and cognition [J]. Acta Psychologica Sinica, 2023, 55(9): 1501-1517.
[2]	XIE Hui, LIN Xuanyi, HU Wanrou, HU Xiaoqing. Emotion regulation promotes forgetting of negative social feedback: Behavioral and EEG evidence [J]. Acta Psychologica Sinica, 2023, 55(6): 905-919.
[3]	LI Mei, LI Jin, ZHANG Guanfei, ZHONG Yiping, LI Hong. Influence of social distance and promise levels on trust decisions: An ERPs study [J]. Acta Psychologica Sinica, 2023, 55(11): 1859-1871.
[4]	SUN Fang, SONG Wei, WEN Xiaotong, LI Huanhuan, OUYANG Lisheng, WEI Shijie. Efficacy of suicide ideation classification based on pain avoidance and the EEG characteristics under self-referential punishment [J]. Acta Psychologica Sinica, 2022, 54(9): 1031-1047.
[5]	ZHAN Youlong, XIAO Xiao, TAN Qianbao, LI Jin, ZHONG Yiping. Influence of reputational concern and social distance on moral decision-making under the harmful dilemma: Evidence from behavioral and ERPs study [J]. Acta Psychologica Sinica, 2022, 54(6): 613-627.
[6]	LI Jianhua, XIE Jiajia, ZHUANG Jin-Ying. An effect of menstrual cycle phase on episodic memory [J]. Acta Psychologica Sinica, 2022, 54(5): 466-480.
[7]	FAN Wei, REN Mengmeng, ZHANG Wenjie, ZHONG Yiping. The impact of feedback on self-deception: Evidence from ERP [J]. Acta Psychologica Sinica, 2022, 54(5): 481-496.
[8]	LUO Wenbo, QI Zhengyang. The influence of concreteness on emotional nouns valence processing: An ERP study [J]. Acta Psychologica Sinica, 2022, 54(2): 111-121.
[9]	YANG Jimei, CHAI Jieyu, QIU Tianlong, QUAN Xiaoshan, ZHENG Maoping. Relationship between empathy and emotion recognition in Chinese national music: An event-related potential study evidence [J]. Acta Psychologica Sinica, 2022, 54(10): 1181-1192.
[10]	CHEN Jie, WU Ke, SHI Yupeng, AI Xiaoqing. The relationship between dispositional self-construal and empathy for ingroup and outgroup members’ pain: evidence from ERPs [J]. Acta Psychologica Sinica, 2021, 53(6): 629-638.
[11]	LIU Jie, LI Jinqi, SHEN Chaoran, HU Xiaohui, ZHAO Tinghao, GUAN Qing, LUO Yuejia. The neural mechanism of approximate number processing for mathematical anxious individuals: An EEG study [J]. Acta Psychologica Sinica, 2020, 52(8): 958-970.
[12]	LI Jin, SUN Yu, YANG Zilu, ZHONG Yiping. Social value orientation modulates the processing of social rewards for self： Evidence from ERPs study [J]. Acta Psychologica Sinica, 2020, 52(6): 786-800.
[13]	SUN Yan, LV Jiaojiao, LAN Fan, ZHANG Lina. Emotion regulation strategy of self-focused and situation-focused reappraisal and their impact on subsequent cognitive control [J]. Acta Psychologica Sinica, 2020, 52(12): 1393-1406.
[14]	LU Wei, HUANG Erqi, YUAN Jinxia. The effects of music mode and lamplight color on human emotion based on quantitative EEG [J]. Acta Psychologica Sinica, 2018, 50(8): 880-891.
[15]	TIAN Lumei, YUAN Jingchi, LI Yongmei. Effects of peer presence and self-esteem on adolescent risk-taking behavior: Evidence from an ERP study [J]. Acta Psychologica Sinica, 2018, 50(1): 47-57.