反馈负波及其近10年理论解释

doi:10.3724/SP.J.1042.2018.01642

摘要/Abstract

摘要：

个体决策后的反馈对随后的结果监控和行为调整起着至关重要的作用。事件相关脑电位研究发现, 反馈负波(feedback-related negativity, FRN)是与决策后反馈加工过程紧密相关的脑电成分。近10年来关于FRN的理论解释, 在最初经典的强化学习理论和情绪动机假说的基础上又提出了反应-结果的预测模型、奖赏正波理论以及积极情绪启动模型。未来的研究应该立足于大样本, 采用互补的研究手段和多样的分析技术来探讨FRN的心理意义; 同时考虑将FRN作为奖赏加工的脑电指标, 研究社会互动情境下的人类行为。

关键词: 反馈负波, 强化学习, 前扣带回, 奖赏正波, 奖赏预测误差

Abstract:

External feedback plays a vital role in ongoing outcome monitoring and future behavioral adjustments. Previous event-related potentials (ERPs) studies have consistently associated the feedback- related negativity (FRN) component with feedback processing after decision making. Until now, several theoretical interpretations of FRN were proposed and continuously updated in the last decade, including the classical reinforcement learning theory, affective-motivational hypothesis, reward positivity (RewP) theory, Predicted response-outcome model (PRO model) and Positive affective model. Whereas different models emphasized on different aspects, no unified theory has yet been proposed to integrate all of the existing experimental evidences. Based on these literature reviews, we argued that big-sample data, multiple and complementary technologies are highly necessary in future FRN studies. Moreover, we proposed that FRN should be considered as an electrophysiological index of reward system to investigate human behaviors in complex social interaction contexts.

Key words: feedback-related negativity, reinforcement learning, anterior cingulate cortex, reward positivity, reward prediction error

中图分类号:

B845

李丹阳, 李鹏, 李红. (2018). 反馈负波及其近10年理论解释. 心理科学进展 , 26(9), 1642-1650.

LI Danyang, LI Peng, LI Hong. (2018). The updated theories of feedback-related negativity in the last decade. Advances in Psychological Science, 26(9), 1642-1650.

参考文献

1	李鹏, 李红 . ( 2008). 反馈负波及其理论解释. 心理科学进展, 16( 5), 705-711.
2	Alexander, W. H., & Brown, J. W. ( 2011). Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14( 10), 1338-1344.
3	Angus D. J., Kemkes K., Schutter D. J. L. G., & Harmon- Jones E. ( 2015). Anger is associated with reward-related electrocortical activity: Evidence from the reward positivity. Psychophysiology, 52(10), 1271-1280.
4	Baker, T. E., & Holroyd, C. B. ( 2011). Dissociated roles of the anterior cingulate cortex in reward and conflict processing as revealed by the feedback error-related negativity and N200. Biological Psychology, 87( 1), 25-34.
5	Balconi M., Finocchiaro R., & Canavesio Y. ( 2015). Reward sensitivity (behavioral activation system), cognitive, and metacognitive control in gambling behavior: Evidences from behavioral, feedback-related negativity, and p300 effect. The Journal of Neuropsychiatry and Clinical Neurosciences, 27( 3), 219-227.
6	Becker M. P. I., Nitsch A. M., Miltner W. H., & Straube T. ( 2014). A single-trial estimation of the feedback-related negativity and its relation to bold responses in a time- estimation task. Journal of Neuroscience, 34( 8), 3005-3012.
7	Bellebaum, C., & Daum, I. ( 2008). Learning-related changes in reward expectancy are reflected in the feedback-related negativity. European Journal of Neuroscience, 27( 7), 1823-1835.
8	Bernat E. M., Nelson L. D., & Baskin-Sommers A. R. ( 2015). Time-frequency theta and delta measures index separable components of feedback processing in a gambling task. Psychophysiology, 52( 5), 626-637.
9	Beyer F., Sidarus N., Bonicalzi S., & Haggard P. ( 2017). Beyond self-serving bias: Diffusion of responsibility reduces sense of agency and outcome monitoring. Social Cognitive & Affective Neuroscience, 12( 1), 138-145.
10	Bismark A. W., Hajcak G., Whitworth N. M., & Allen J. J. B.( 2013). The role of outcome expectations in the generation of the feedback-related negativity. Psychophysiology, 50( 2), 125-133.
11	Bress, J. N., & Hajcak, G. ( 2013). Self-report and behavioral measures of reward sensitivity predict the feedback negativity. Psychophysiology, 50( 7), 610-616.
12	Carlson J. M., Foti D., Mujica-Parodi L. R., Harmon-Jones E., & Hajcak G. ( 2011). Ventral striatal and medial prefrontal BOLD activation is correlated with reward- related electrocortical activity: A combined ERP and FMRI study. NeuroImage, 57( 4), 1608-1616.
13	Chen J., Wu Y., Tong G. Y., Guan X. M., & Zhou X. L. ( 2012). ERP correlates of social conformity in a line judgment task. BMC Neuroscience, 13( 1), 43.
14	Cohen M. X., Wilmes K. A., & van de Vijver, I. ( 2011). Cortical electrophysiological network dynamics of feedback learning. Trends in Cognitive Sciences, 15( 12), 558-566.
15	Doñamayor N., Dinani J., Römisch M., Ye Z., & Münte T. F. ( 2014). Performance monitoring during associative learning and its relation to obsessive-compulsive characteristics. Biological Psychology, 102, 73-87.
16	Ernst, B., & Steinhauser, M. ( 2015). Effects of invalid feedback on learning and feedback-related brain activity in decision-making. Brain & Cognition, 99, 78-86.
17	Ferdinand N. K., Mecklinger A., Kray J., & Gehring W. J. ( 2012). The processing of unexpected positive response outcomes in the mediofrontal cortex. Journal of Neuroscience, 32( 35), 12087-12092.
18	Foti D., Weinberg A., Dien J., & Hajcak G. ( 2011). Event-related potential activity in the basal ganglia differentiates rewards from nonrewards: Temporospatial principal components analysis and source localization of the feedback negativity. Human Brain Mapping, 32( 12), 2207-2216.
19	Gehring W. J., Goss B., Coles M. G. H., Meyer D. E., & Donchin E. ( 1993). A neural system for error detection and compensation. Psychological Science, 4( 6), 385-390.
20	Gehring, W. J., & Willoughby, A. R. ( 2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science, 295( 5563), 2279-2282.
21	Gu R. L., Feng X., Broster L. S., Yuan L., Xu P. F., & Luo Y. J. ( 2017). Valence and magnitude ambiguity in feedback processing. Brain & Behavior, 7( 5), e00672.
22	Gu R. L., Huang Y. X., & Luo Y. J. ( 2010). Anxiety and feedback negativity. Psychophysiology, 47( 5), 961-967.
23	Gu R. L., Ge Y., Jiang Y., & Luo Y. J. ( 2010). Anxiety and outcome evaluation: The good, the bad and the ambiguous. Biological Psychology, 85( 2), 200-206.
24	Hewig J., Kretschmer N., Trippe R. H., Hecht H., Coles M. G. H., Holroyd C. B., & Miltner, W. H. R. ( 2010). Hypersensitivity to reward in problem gamblers. Biological Psychiatry, 67( 8), 781-783.
25	Heydari, S., & Holroyd, C. B. ( 2016). Reward positivity: Reward prediction error or salience prediction error?. Psychophysiology, 53( 8), 1185-1192.
26	Holroyd, C. B., & Coles, M. G. ( 2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109( 4), 679-709.
27	Holroyd, C. B., & Krigolson, O. E. ( 2007). Reward prediction error signals associated with a modified time estimation task. Psychophysiology, 44( 6), 913-917.
28	Holroyd C. B., Krigolson O. E., & Lee S. ( 2011). Reward positivity elicited by predictive cues. Neuroreport, 22( 5), 249-252.
29	Holroyd C. B., Pakzad-Vaezi K. L., & Krigolson O. E. ( 2008). The feedback correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45( 5), 688-697.
30	Holroyd, C. B., & Umemoto, A. ( 2016). The research domain criteria framework: The case for anterior cingulate cortex. Neuroscience & Biobehavioral Reviews, 71, 418-443.
31	Holroyd, C. B., & Yeung, A. N. ( 2012). Motivation of extended behaviors by anterior cingulate cortex. Trends in Cognitive Sciences, 16( 2), 122-128.
32	Hu X. M., Xu Z. H., & Mai X. Q. ( 2017). Social value orientation modulates the processing of outcome evaluation involving others. Social Cognitive and Affective Neuroscience, 12( 11), 1730-1739.
33	Kimura, K., & Katayama, J. ( 2016). Cooperative context is a determinant of the social influence on outcome evaluation: An electrophysiological study. International Journal of Psychophysiology, 100, 28-35.
34	Kobza S., Thoma P., Daum I., & Bellebaum C. ( 2011). The feedback-related negativity is modulated by feedback probability in observational learning. Behavioural Brain Research, 225( 2), 396-404.
35	Leng, Y., & Zhou, X. L. ( 2010). Modulation of the brain activity in outcome evaluation by interpersonal relationship: An ERP study. Neuropsychologia, 48( 2), 448-455.
36	Li P., Han C. H., Lei Y., Holroyd C. B., & Li H. ( 2011). Responsibility modulates neural mechanisms of outcome processing: An ERP study. Psychophysiology, 48(8), 1129-1133.
37	Li P., Jia S. W., Feng T. Y., Liu Q., Suo T., & Li H. ( 2010). The influence of the diffusion of responsibility effect on outcome evaluations: Electrophysiological evidence from an ERP study. NeuroImage, 52( 4), 1727-1733.
38	Luft, C. D. B. ( 2014). Learning from feedback: The neural mechanisms of feedback processing facilitating better performance. Behavioural Brain Research, 261( 6), 356-368.
39	Ma Q. G., Shen Q., Xu Q., Li D. D., Shu L. C., & Weber B. ( 2011). Empathic responses to others' gains and losses: An electrophysiological investigation. NeuroImage, 54( 3), 2472-2480.
40	Maia, T. V., & Frank, M. J. ( 2011). From reinforcement learning models to psychiatric and neurological disorders. Nature Neuroscience, 14( 2), 154-162.
41	Marco-Pallares J., Cucurell D., Münte T. F., Strien N., & Rodriguez-Fornells A. ( 2011). On the number of trials needed for a stable feedback-related negativity. Psychophysiology, 48( 6), 852-860.
42	Mensen A., Poryazova R., Huegli G., Baumann C. R., Schwartz S., & Khatami R. ( 2015). The roles of dopamine and hypocretin in reward: A electroencephalographic study. PLoS One, 10( 11), e0142432.
43	Miltner W. H. R., Braun C. H., & Coles, M. G. H. ( 1997). Event-related brain potentials following incorrect feedback in a time-estimation task: Evidence for a "generic" neural system for error detection. Journal of Cognitive Neuroscience, 9( 6), 788-798.
44	Mushtaq F., Stoet G., Bland A. R., & Schaefer A. ( 2013). Relative changes from prior reward contingencies can constrain brain correlates of outcome monitoring. PLoS One, 8( 6), e66350.
45	Mushtaq F., Wilkie R. M., Mon-Williams M. A., & Schaefer A. ( 2016). Randomised prior feedback modulates neural signals of outcome monitoring. NeuroImage, 125, 868-879.
46	Nieuwenhuis S., Aston-Jones G., & Cohen J. D. ( 2005). Decision making, the P3, and the locus coeruleus- norepinephrine system. Psychological Bulletin, 131, 510-532.
47	Nieuwenhuis S., Holroyd C. B., Mol N., & Coles, M. G. H. ( 2004). Reinforcement-related brain potentials from medial frontal cortex: Origins and functional significance. Neuroscience & Biobehavioral Reviews, 28( 4), 441-448.
48	Oliveira F. T. P., McDonald J. J., & Goodman D. ( 2007). Performance monitoring in the anterior cingulate is not all error related: Expectancy deviation and the representation of action-outcome associations. Journal of Cognitive Neuroscience, 19( 12), 1994-2004.
49	Padrón I., Fernández-Rey J., Acuña C., & Pardo-Vazquez J. L. ( 2016). Representing the consequences of our actions trial by trial: Complex and flexible encoding of feedback valence and magnitude. Neuroscience, 333, 264-276.
50	Sambrook, T. D., & Goslin, J. ( 2015). A neural reward prediction error revealed by a meta-analysis of ERPs using great grand averages. Psychological Bulletin, 141( 1), 213-235.
51	Sambrook, T. D., & Goslin, J. ( 2016). Principal components analysis of reward prediction errors in a reinforcement learning task. NeuroImage, 124, 276-286.
52	San Martín R., Manes F., Hurtado E., Isla P., & Ibañez A. ( 2010). Size and probability of rewards modulate the feedback error-related negativity associated with wins but not losses in a monetarily rewarded gambling task. NeuroImage, 51( 3), 1194-1204.
53	Schaefer A., Buratto L. G., Goto N., & Brotherhood E. V. ( 2016). The feedback-related negativity and the p300 brain potential are sensitive to price expectation violations in a virtual shopping task. PLoS One, 11( 9), e0163150.
54	Schönberg T., Daw N. D., Joel D., & O'Doherty J. P. ( 2007). Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. Journal of Neuroscience, 27( 47), 12860-12867.
55	Schulreich, S. ( 2016). Altered performance monitoring in psychopathy: A review of studies on action selection, error, and feedback processing. Current Behavioral Neuroscience Reports, 3( 1), 19-27.
56	Shahnazian, D., & Holroyd, C. B. ( 2017). Distributed representations of action sequences in anterior cingulate cortex: A recurrent neural network approach. Psychonomic Bulletin & Review, doi: 10.3758/s13423-017-1280-1
57	Shestakova A., Rieskamp J., Tugin S., Ossadtchi A., Krutitskaya J., & Klucharev V. ( 2013). Electrophysiological precursors of social conformity. Social Cognitive & Affective Neuroscience, 8( 7), 756-763.
58	Takács á., Kóbor A., Janacsek K., Honbolygó F., Csépe V., & Németh D. ( 2015). High trait anxiety is associated with attenuated feedback-related negativity in risky decision making. Neuroscience Letters, 600, 188-192.
59	Talmi D., Atkinson R., & El-Deredy W. ( 2013). The feedback-related negativity signals salience prediction errors, not reward prediction errors. Journal of Neuroscience, 33( 19), 8264-8269.
60	Thoma P., Norra C., Juckel G., Suchan B., & Bellebaum C. ( 2015). Performance monitoring and empathy during active and observational learning in patients with major depression. Biological Psychology, 109, 222-231.
61	Umemoto A., Hajihosseini A., Yates M. E., & Holroyd C. B. ( 2017). Reward-based contextual learning supported by anterior cingulate cortex. Cognitive, Affective, & Behavioral Neuroscience, 17( 3), 642-651.
62	Von Borries, A. K. L., Verkes, R. J., Bulten B. H., Cools R., & de Bruijn, E. R. A. ( 2013). Feedback-related negativity codes outcome valence, but not outcome expectancy, during reversal learning. Cognitive, Affective, & Behavioral Neuroscience, 13( 4), 737-746.
63	Walsh, M. M., & Anderson, J. R. ( 2012). Learning from experience: Event-related potential correlates of reward processing, neural adaptation, and behavioral choice. Neuroscience & Biobehavioral Reviews, 36( 8), 1870-1884.
64	Wu Y., Zhang D. X., Elieson B., & Zhou X. L. ( 2012). Brain potentials in outcome evaluation: When social comparison takes effect. International Journal of Psychophysiology, 85( 2), 145-152.
65	Yeung, N. ( 2004). Relating cognitive and affective theories of the error-related negativity. In Errors, conflicts, and the brain. Current opinions on performance monitoring( pp. 63-70). Leipzig.
66	Zheng Y., Li Q., Wang K., Wu H. Y., & Liu X. ( 2015). Contextual valence modulates the neural dynamics of risk processing. Psychophysiology, 52( 7), 895-904.
67	Zheng Y., Li Q., Zhang Y. Y., Li Q., Shen H. J., Gao Q. H., & Zhou S. Y. ( 2017). Reward processing in gain versus loss context: An ERP study. Psychophysiology, 54( 7), 1040-1053.

[1]	陈念劬. 从好奇发生到满足：好奇反馈循环的神经生理机制[J]. 心理科学进展, 2023, 31(2): 256-273.
[2]	陈幸明, 付彤, 刘昌, 张宾, 伏云发, 李恩泽, ZHANG Jian, 陈盛强, 党彩萍. 工作记忆训练诱发的神经可塑性——基于系列fMRI实验的脑区分布递减时空模型[J]. 心理科学进展, 2022, 30(2): 255-274.
[3]	黎穗卿, 陈新玲, 翟瑜竹, 张怡洁, 章植鑫, 封春亮. 人际互动中社会学习的计算神经机制[J]. 心理科学进展, 2021, 29(4): 677-696.
[4]	秦浩方, 黄蓉, 贾世伟. 反馈相关负波：一种抑郁症的生物标记物[J]. 心理科学进展, 2021, 29(3): 404-413.
[5]	褚昕宇, 王泽军, 肖焕禹. 身体活动的双系统理论:一种强化学习的视角[J]. 心理科学进展, 2020, 28(8): 1337-1350.
[6]	陈乐乐, 黄蓉, 贾世伟. 反馈相关负波与成瘾[J]. 心理科学进展, 2020, 28(6): 959-968.
[7]	杜玮玮, 宋婷, 李富洪. 双价效应及其认知机制[J]. 心理科学进展, 2018, 26(11): 1969-1975.
[8]	杨紫嫣;罗宇;古若雷;刘云芝;蔡华俭. 自尊的认知神经机制[J]. 心理科学进展, 2017, 25(5): 788-798.
[9]	赵春黎. 社会从众的生物学基础[J]. 心理科学进展, 2015, 23(11): 1956-1965.
[10]	聂其阳;罗劲. “啊哈！”和“哈哈！”：顿悟与幽默的脑认知成分比较[J]. 心理科学进展, 2012, 20(2): 219-227.
[11]	李旭;胡金生. 脑神经回路预警系统：研究社会公正的新视角[J]. 心理科学进展, 2010, 18(7): 1175-1181.
[12]	刘春雷;张庆林. 错误加工的神经机制[J]. 心理科学进展, 2009, 17(2): 341-348.
[13]	李鹏;李红. 反馈负波及其理论解释[J]. 心理科学进展, 2008, 16(5): 705-711.
[14]	蔡厚德. 强迫症的脑功能障碍[J]. 心理科学进展, 2006, 14(3): 401-407.
[15]	朱湘如;刘昌. 前扣带回功能的冲突监测理论[J]. 心理科学进展, 2005, 13(6): 767-773.