Abstract:

Episodic memory consists of item memory and associative memory. Individual cognitive resources are typically allocated to more valuable information during encoding through metamemory, leading to competitive processing of item and associative information. Reward prediction error (RPE), defined as the difference between reward results and reward expectations, has two properties: valence (positive or negative) and salience (degree of difference). To examine the impact of reward prediction error valence and salience on item and associative memory, and how reward prediction error influences memory based on metamemory, three experiments were conducted.

In Experiment 1, the effect of RPE on memory performance and metamemory monitoring in retrieval was investigated. The reward results included 1, 4 and 7, and their proportions in high and low value pictures were 2:3:5 and 5:3:2 respectively. RPE is generated by the difference between the reward results and the value prediction given by the subject, including −6, −3, 0, 3 and 6, and its salience is calculated as unsigned RPE (URPE). Number of subjects for analysis is 34, and the process is shown in Figure 1. In the learning stage, participants were presented with indoor and outdoor scene pictures. They were asked to predict the score of each picture and then received feedback on the actual score. Through this reinforcement learning process, participants had to find out which type of pictures is more valuable, and 30% of the scores were accumulated into the total score. To induce the effect of reward motivation on memory, participants were introduced to the opportunity to choose between two pictures and receive the value of the selected picture, although the actual program did not include a decision-making stage. After the learning stage, participants were tested on item and reward associative memory.

On the basis of taking reward result as another independent variable, take RPE and URPE as the independent variables respectively to carry out the generalized mixed linear model (GMLM) on the hit rate of item and associative memory. The results are shown in Table 1. The performance of associative memory is higher in positive valence and low salience of RPE. On the basis of taking the memory result in each trial as another independent variable, RPE and URPE were taken as the independent variables respectively to carry out the mixed linear model (MLM) of item and associative memory JOCs. The results are shown in Table 2. When RPE valence is positive, the associative memory JOCs are higher in correct trials and lower in wrong trials, indicating the promotion of RPE positive valence to the metamemory monitoring at retrieval.

Experiment 2 recorded the eye-movement data (the change of pupil diameter and the fixation time of pictures and scores during the reward feedback display screen) of the subjects, to investigate the effect of RPE on the metamemory control during memory encoding. Number of subjects for behavior and eye movement analysis are 23 and 20. The process is shown in Figure 2. In order to avoid the unsignificant results of item memory in Experiment 1, the presentation time of each screen in the learning stage is adjusted.

The results of the GMLM on the hit rates of item and associative memory are shown in Table 3. Contrary to the associative memory performance, positive valence and high salience of RPE improved item memory performance. MLM was carried out on the picture fixation time, score fixation time, mean and maximum change of pupil diameter (Table 4). With the decrease of RPE salience, the subjects' fixation time on pictures was shortened, and the fixation time on scores was prolonged, which was consistent with the effect of salience on memory performance, revealing the allocation of cognitive resources through metamemory control based on RPE salience. The positive valence and low salience of RPE promote the change of pupil diameter. Because the performance of associative memory is also higher in these conditions, this may indicate that individuals have taken measures to increase the degree of encoding effort in order to promote associative memory.

In Experiment 3, the reward results were set as 1, 3, 5 and 7 to increase the RPE levels, in order to verify the stability of the influence of RPE valence and salience on item and associative memory when the overlap between the RPE valence and reward result size is reduced. 27 subjects were included in analysis, and the process was similar to Experiment 2, except for the value feedback time increased to 5 s. The results of GMLM of memory performance are shown in Table 5. The effects of RPE on item and associative memory performance are the same as those of Experiment 2, which shows that the effects of RPE valence and salience on item and associative memory are stable.

To sum up, the results above suggest that the effects of RPE on item and associative memory are distinct. During the encoding stage, individuals use the valence and salience of reward prediction error as cues to allocate cognitive resources differently in item and associative memory encoding through metamemory control. In the retrieval stage, positive valence of reward prediction error enhances the metamemory monitoring level of associative memory retrieval.

Keywords reward prediction error, associative memory, eye movements, episodic memory, metamemory