The effect of reward prediction error on temporal order and source memory

doi:10.3724/SP.J.1041.2023.01049

Abstract

Abstract:

Although previous research has demonstrated that event boundaries enhanced source memory at boundaries and reduced temporal order memory across boundaries, there is little research on whether there is a mnemonic trade-off between temporal order and source memory and how intrinsic, socially meaningful change as event boundaries affects memory. The present study explored the effects of RPE event boundaries on temporal order and source memory by reward prediction errors (RPE) as event boundaries in two behavioural experiments and one ERP experiment. The results showed that RPE event boundaries enhanced source memory at the boundaries, and high RPE event boundaries produced mnemonic trade-off effect; compared to within-event/non-boundary conditions, a larger N400 amplitude was induced by across-event/ boundary conditions, and activation of temporal order memory was focused on anterior region, activation of source memory was focused on posterior region. The present study showed that the strength of segmentation of event boundaries is an important factor in mnemonic trade-off effects, and that the N400 component may be an important indicator of the integration and segmentation of episodic memory by event boundaries.

Key words: reward prediction error, event boundaries, temporal order memory, source memory, mnemonic trade-off effect

ZHANG Hongchi, CHENG Xuan, MAO Weibin. (2023). The effect of reward prediction error on temporal order and source memory. Acta Psychologica Sinica, 55(7), 1049-1062.

Figures/Tables 10

Figure 1. Experiental material examples

Figure 2. Experimental procedure (learning phase on the left, test phase on the right)

Table 1 Accuracy of temporal order and source memory across-events/ boundary and within-events/non-boundary(M ± SD)

Test Type	Across-Event/ Boundary	Within-Event/ Non-Boundary
Temporal Order Memory	0.56 ± 0.09	0.59 ± 0.09
Source Memory	0.73 ± 0.10	0.63 ± 0.13

Figure 3. Accuracy of temporal order and source memory across- event/boundary and within-event/non-boundary
Note: Error bar is standard error,*p < 0.05，**p < 0.01，***p < 0.001.

Figure 4. Experimental procedure for the learning phase

Table 2 Accuracy of temporal order and source memory across-event/ boundary and within-event/non-boundary for different RPE strength (M ± SD)

RPE Strength	Test Type	Across-Event/ Boundary	Within-Event/ Non-Boundary
High Strength	Temporal Order Memory	0.52 ± 0.13	0.62 ± 0.12
High Strength	Source Memory	0.71 ± 0.15	0.65 ± 0.15
Low Strength	Temporal Order Memory	0.58 ± 0.11	0.61 ± 0.10
Low Strength	Source Memory	0.71 ± 0.14	0.72 ± 0.14

Figure 5. Accuracy of temporal order and source memory across- event/boundary and within-event/non-boundary for different RPE strength

Table 3 Accuracy of temporal order and source memory across-events/ boundary and within-events/non-boundary (M ± SD)

Test Type	Across-Event/ Boundary	Within-Event/ Non-Boundary
Temporal Order Memory	0.58 ± 0.07	0.62 ± 0.08
Source Memory	0.72 ± 0.09	0.68 ± 0.09

Figure 6. Accuracy of temporal order and source memory across- event/boundary and within-event/non-boundary.

Figure 7. N400 and P600 waveforms under different conditions in temporal order and source memory (the figure above corresponds to temporal order memory and the figure below corresponds to source memory)

References 53

[1]	Bai, L., Ma, H., Huang, Y. X., & Luo, Y. J. (2005). The development of native Chinese affective picture system-A pretest in 46 college students. Chinese Mental Health Journal, 19(11), 719-722.
[2]	Ben-Yakov, A., Smith, V., & Henson, R. (2022). The limited reach of surprise: Evidence against effects of surprise on memory for preceding elements of an event. Psychonomic Bulletin & Review, 29(3), 1053-1064. doi: 10.3758/s13423-021-01954-5
[3]	Bladon, J. H., Sheehan, D. J., de Freitas, C. S., & Howard, M. W. (2019). In a temporally segmented experience hippocampal neurons represent temporally drifting context but not discrete segments. Journal of Neuroscience, 39(35), 6936-6952. doi: 10.1523/JNEUROSCI.1420-18.2019 pmid: 31253754
[4]	Braun, E. K., Wimmer, G. E., & Shohamy, D. (2018). Retroactive and graded prioritization of memory by reward. Nature Communications, 9(1), 1-12. doi: 10.1038/s41467-017-02088-w
[5]	Carr, M. F., Jadhav, S. P., & Frank, L. M. (2011). Hippocampal replay in the awake state: A potential substrate for memory consolidation and retrieval. Nature Neuroscience, 14(2), 147-153. doi: 10.1038/nn.2732 pmid: 21270783
[6]	Clewett, D., & Davachi, L. (2017). The ebb and flow of experience determines the temporal structure of memory. Current Opinion in Behavioral Sciences, 17, 186-193. doi: 10.1016/j.cobeha.2017.08.013 pmid: 29276730
[7]	Clewett, D., DuBrow, S., & Davachi, L. (2019). Transcending time in the brain: How event memories are constructed from experience. Hippocampus, 29(3), 162-183. doi: 10.1002/hipo.23074 pmid: 30734391
[8]	Clewett, D., Gasser, C., & Davachi, L. (2020). Pupil-linked arousal signals track the temporal organization of events in memory. Nature Communications, 11(1), 4007.
[9]	Davachi, L., & DuBrow, S. (2015). How the hippocampus preserves order: The role of prediction and context. Trends in Cognitive Sciences, 19(2), 92-99. doi: 10.1016/j.tics.2014.12.004 pmid: 25600586
[10]	Davis, E. E., Chemnitz, E., Collins, T. K., Geerligs, L., & Campbell, K. L. (2021). Looking the same, but remembering differently: Preserved eye-movement synchrony with age during movie watching. Psychology & Aging, 36(5), 604-615.
[11]	Delogu, F., Drenhaus, H., & Crocker, M. W. (2018). On the predictability of event boundaries in discourse: An ERP investigation. Memory & Cognition, 46(2), 315-325. doi: 10.3758/s13421-017-0766-4 URL
[12]	DeLong, K. A., Urbach, T. P., & Kutas, M. (2005). Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience, 8(8), 1117-1121. pmid: 16007080
[13]	Donaldson, D. I., Wheeler, M. E., & Petersen, S. E. (2010). Remember the source: Dissociating frontal and parietal contributions to episodic memory. Journal of Cognitive Neuroscience, 22(2), 377-391. doi: 10.1162/jocn.2009.21242 pmid: 19400677
[14]	DuBrow, S., & Davachi, L. (2013). The influence of context boundaries on memory for the sequential order of events. Journal of Experimental Psychology: General, 142(4), 1277-1286. doi: 10.1037/a0034024 URL
[15]	Dunsmoor, J. E., Kroes, M. C., Moscatelli, C. M., Evans, M. D., Davachi, L., & Phelps, E. A. (2018). Event segmentation protects emotional memories from competing experiences encoded close in time. Nature Human Behaviour, 2(4), 291-299. doi: 10.1038/s41562-018-0317-4 pmid: 30221203
[16]	Ezzyat, Y., & Davachi, L. (2011). What constitutes an episode in episodic memory?. Psychological Science, 22(2), 243-252. doi: 10.1177/0956797610393742 pmid: 21178116
[17]	Ezzyat, Y., & Davachi, L. (2014). Similarity breeds proximity: Pattern similarity within and across contexts is related to later mnemonic judgments of temporal proximity. Neuron, 81(5), 1179-1189. doi: S0896-6273(14)00073-7 pmid: 24607235
[18]	Ezzyat, Y., & Davachi, L. (2021). Neural evidence for representational persistence within events. Journal of Neuroscience, 41(37), 7909-7920. doi: 10.1523/JNEUROSCI.0073-21.2021 pmid: 34330773
[19]	Federmeier, K. D., & Kutas, M. (1999). A rose by any other name: Long-term memory structure and sentence processing. Journal of Memory and Language, 41(4), 469-495. doi: 10.1006/jmla.1999.2660 URL
[20]	Gurguryan, L., Dutemple, E., & Sheldon, S. (2020). Conceptual similarity alters the impact of context shifts on temporal memory. Memory, 29(1), 11-20. doi: 10.1080/09658211.2020.1841240 URL
[21]	Hayama, H. R., Vilberg, K. L., & Rugg, M. D. (2012). Overlap between the neural correlates of cued recall and source memory: Evidence for a generic recollection network?. Journal of Cognitive Neuroscience, 24(5), 1127-1137. doi: 10.1162/jocn_a_00202 pmid: 22288393
[22]	Heusser, A. C., Ezzyat, Y., Shiff, I., & Davachi, L. (2018). Perceptual boundaries cause mnemonic trade-offs between local boundary processing and across-trial associative binding. Journal of Experimental Psychology: Learning Memory & Cognition, 44(7), 1075-1090.
[23]	Horner, A. J., Bisby, J. A., Wang, A., Bogus, K., & Burgess, N. (2016). The role of spatial boundaries in shaping long-term event representations. Cognition, 154, 151-164. doi: S0010-0277(16)30128-7 pmid: 27295330
[24]	Howard, M. W., Fotedar, M. S., Datey, A. V., & Hasselmo, M. E. (2005). The temporal context model in spatial navigation and relational learning: Toward a common explanation of medial temporal lobe function across domains. Psychological Review, 112(1), 75-116. doi: 10.1037/0033-295X.112.1.75 pmid: 15631589
[25]	Jang, A. I., Nassar, M. R., Dillon, D. G., & Frank, M. J. (2019). Positive reward prediction errors during decision-making strengthen memory encoding. Nature Human Behaviour, 3(7), 719-732. doi: 10.1038/s41562-019-0597-3 pmid: 31061490
[26]	King, D. R., & Miller, M. B. (2014). Lateral posterior parietal activity during source memory judgments of perceived and imagined events. Neuropsychologia, 53, 122-136. doi: 10.1016/j.neuropsychologia.2013.11.006 pmid: 24269856
[27]	Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential(ERP). Annual Review of Psychology, 62(1), 621-647. doi: 10.1146/psych.2011.62.issue-1 URL
[28]	Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207(4427), 203-205. doi: 10.1126/science.7350657 pmid: 7350657
[29]	Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307(5947), 161-163. doi: 10.1038/307161a0
[30]	Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. University of Florida, Gainesville. In.: Tech Rep A-8
[31]	Lin, J., Pype, A., Murray, S., & Boynton, G. (2010). Encoding of a scene into memory is enhanced at behaviorally relevant points in time. Journal of Vision, 10(7), 754-754. doi: 10.1167/10.7.754 URL
[32]	Lisman, J., Grace, A. A., & Duzel, E. (2011). A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends in Neurosciences, 34(10), 536-547. doi: 10.1016/j.tins.2011.07.006 pmid: 21851992
[33]	Milner, B., Corsi, P., & Leonard, G. (1991). Frontal-lobe contribution to recency judgements. Neuropsychologia, 29(6), 601-618. pmid: 1944864
[34]	Newtson, D., Engquist, G. A., & Bois, J. (1977). The objective basis of behavior units. Journal of Personality and Social Psychology, 35(12), 847-862. doi: 10.1037/0022-3514.35.12.847 URL
[35]	O’Neill, J., Pleydell-Bouverie, B., Dupret, D., & Csicsvari, J. (2010). Play it again: Reactivation of waking experience and memory. Trends in Neurosciences, 33(5), 220-229. doi: 10.1016/j.tins.2010.01.006 pmid: 20207025
[36]	Polyn, S. M., Norman, K. A., & Kahana, M. J. (2009). A context maintenance and retrieval model of organizational processes in free recall. Psychological Review, 116(1), 129-156. doi: 10.1037/a0014420 pmid: 19159151
[37]	Pouthas, V., Garnero, L., Ferrandez, A. M., & Renault, B. (2000). ERPs and PET analysis of time perception: Spatial and temporal brain mapping during visual discrimination tasks. Human Brain Mapping, 10(2), 49-60. pmid: 10864229
[38]	Radvansky, G. A. (2012). Across the event horizon. Current Directions in Psychological Science, 21(4), 269-272. doi: 10.1177/0963721412451274 URL
[39]	Radvansky, G. A., & Zacks, J. M. (2014). Event Cognition. New York: Oxford University Press.
[40]	Radvansky, G. A., & Zacks, J. M. (2017). Event boundaries in memory and cognition. Current Opinion in Behavioral Sciences, 17, 133-140. doi: 10.1016/j.cobeha.2017.08.006 pmid: 29270446
[41]	Rouhani, N., Norman, K. A., Niv, Y., & Bornstein, A. M. (2020). Reward prediction errors create event boundaries in memory. Cognition, 203, 104269. doi: 10.1016/j.cognition.2020.104269 URL
[42]	Roumis, D. K., & Frank, L. M. (2015). Hippocampal sharp-wave ripples in waking and sleeping states. Current Opinion in Neurobiology, 35, 6-12. doi: 10.1016/j.conb.2015.05.001 pmid: 26011627
[43]	Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593-1599. doi: 10.1126/science.275.5306.1593 pmid: 9054347
[44]	Siefke, B. M., Smith, T. A., & Sederberg, P. B. (2019). A context-change account of temporal distinctiveness. Memory & Cognition, 47(6), 1158-1172. doi: 10.3758/s13421-019-00925-5
[45]	Sitnikova, T., Kuperberg, G., & Holcomb, P. J. (2003). Semantic integration in videos of real-world events: An electrophysiological investigation. Psychophysiology, 40(1), 160-164. pmid: 12751813
[46]	Tulving, E. (1983). Elements of episodic memory. London: Oxford University Press.
[47]	Tulving, E. (2002). Episodic memory: From mind to brain. Annual Review of Psychology, 53(1), 1-25. doi: 10.1146/psych.2002.53.issue-1 URL
[48]	van Berkum, J. J., Zwitserlood, P., Hagoort, P., & Brown, C. M. (2003). When and how do listeners relate a sentence to the wider discourse? Evidence from the N400 effect. Cognitive Brain Research, 17(3), 701-718. pmid: 14561457
[49]	van de Ven, V., Jäckels, M., & de Weerd, P. (2022). Time changes: Timing contexts support event segmentation in associative memory. Psychonomic Bulletin & Review, 29(2), 568-580. doi: 10.3758/s13423-021-02000-0
[50]	West, W. C., & Holcomb, P. J. (2002). Event-related potentials during discourse-level semantic integration of complex pictures. Cognitive Brain Research, 13(3), 363-375. pmid: 11919001
[51]	Zacks, J. M., Speer, N. K., & Reynolds, J. R. (2009). Segmentation in reading and film comprehension. Journal of Experimental Psychology: General, 138(2), 307-327. doi: 10.1037/a0015305 URL
[52]	Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007). Event perception: A mind-brain perspective. Psychological Bulletin, 133(2), 273-293. pmid: 17338600
[53]	Zacks, J. M., Tversky, B., & Iyer, G. (2001). Perceiving, remembering, and communicating structure in events. Journal of Experimental Psychology: General, 130(1), 29-58. doi: 10.1037/0096-3445.130.1.29 URL