Sleep is known to benefit consolidation of memories, especially those of motivational relevance. Yet, it remains largely unknown the extent to which sleep influences reward-associated behavior, in particular, whether and how sleep modulates reward evaluation that critically underlies value-based decisions. Here, we show that neural processing during sleep can selectively bias preferences in simple economic choices when the sleeper is stimulated by covert, reward-associated cues. Specifically, presenting the spoken name of a familiar, valued snack item during midday nap significantly improves the preference for that item relative to items not externally cued. The cueing-specific preference enhancement is sleep-dependent and can be predicted by cue-induced neurophysiological signals at the subject and item level. Computational modeling further suggests that sleep cueing accelerates evidence accumulation for cued options during the post-sleep choice process in a manner consistent with the preference shift. These findings suggest that neurocognitive processing during sleep contributes to the fine-tuning of subjective preferences in a flexible, selective manner.

Neuroscientists are beginning to advance explanations of social behavior in terms of underlying brain mechanisms. Two distinct networks of brain regions have come to the fore. The first involves brain regions that are concerned with learning about reward and reinforcement. These same reward-related brain areas also mediate preferences that are social in nature even when no direct reward is expected. The second network focuses on regions active when a person must make estimates of another person's intentions. However, it has been difficult to determine the precise roles of individual brain regions within these networks or how activities in the two networks relate to one another. Some recent studies of reward-guided behavior have described brain activity in terms of formal mathematical models; these models can be extended to describe mechanisms that underlie complex social exchange. Such a mathematical formalism defines explicit mechanistic hypotheses about internal computations underlying regional brain activity, provides a framework in which to relate different types of activity and understand their contributions to behavior, and prescribes strategies for performing experiments under strong control.

Our decisions are guided by outcomes that are associated with decisions made in the past. However, the amount of influence each past outcome has on our next decision remains unclear. To ensure optimal decision-making, the weight given to decision outcomes should reflect their salience in predicting future outcomes, and this salience should be modulated by the volatility of the reward environment. We show that human subjects assess volatility in an optimal manner and adjust decision-making accordingly. This optimal estimate of volatility is reflected in the fMRI signal in the anterior cingulate cortex (ACC) when each trial outcome is observed. When a new piece of information is witnessed, activity levels reflect its salience for predicting future outcomes. Furthermore, variations in this ACC signal across the population predict variations in subject learning rates. Our results provide a formal account of how we weigh our different experiences in guiding our future actions.

Disproportionate reactions to unexpected stimuli in the environment are a cardinal symptom of posttraumatic stress disorder (PTSD). Here, we test whether these heightened responses are associated with disruptions in distinct components of reinforcement learning. Specifically, using functional neuroimaging, a loss-learning task, and a computational model-based approach, we assessed the mechanistic hypothesis that overreactions to stimuli in PTSD arise from anomalous gating of attention during learning (i.e., associability). Behavioral choices of combat-deployed veterans with and without PTSD were fit to a reinforcement learning model, generating trial-by-trial prediction errors (signaling unexpected outcomes) and associability values (signaling attention allocation to the unexpected outcomes). Neural substrates of associability value and behavioral parameter estimates of associability updating, but not prediction error, increased with PTSD during loss learning. Moreover, the interaction of PTSD severity with neural markers of associability value predicted behavioral choices. These results indicate that increased attention-based learning may underlie aspects of PTSD and suggest potential neuromechanistic treatment targets.

Visual cognition is thought to rely heavily on contextual expectations. Accordingly, previous studies have revealed distinct neural signatures for expected versus unexpected stimuli in visual cortex. However, it is presently unknown how the brain combines multiple concurrent stimulus expectations such as those we have for different features of a familiar object. To understand how an unexpected object feature affects the simultaneous processing of other expected feature(s), we combined human fMRI with a task that independently manipulated expectations for color and motion features of moving-dot stimuli. Behavioral data and neural signals from visual cortex were then interrogated to adjudicate between three possible ways in which prediction error (surprise) in the processing of one feature might affect the concurrent processing of another, expected feature: (1) feature processing may be independent; (2) surprise might

Why do people judge hypocrites, who condemn immoral behaviors that they in fact engage in, so negatively? We propose that hypocrites are disliked because their condemnation sends a false signal about their personal conduct, deceptively suggesting that they behave morally. We show that verbal condemnation signals moral goodness (Study 1) and does so even more convincingly than directly stating that one behaves morally (Study 2). We then demonstrate that people judge hypocrites negatively-even more negatively than people who directly make false statements about their morality (Study 3). Finally, we show that

Social decision-making is increasingly studied with neurocomputational modeling. Here we discuss how this approach allows researchers to better understand and predict behavior in social settings. Using examples from the study of resource distributions and social learning, we illustrate how this methodology provides a flexible way to quantify social values and beliefs, identify specific motives and cognitive processes underlying social choice and learning, and arbitrate between competing theories of social behavior. We also critically discuss open questions and potential problems associated with this methodology.

People make numerous decisions every day including perceptual decisions such as walking through a crowd, decisions over primary rewards such as what to eat, and social decisions that require balancing own and others' benefits. The unifying principles behind choices in various domains are, however, still not well understood. Mathematical models that describe choice behavior in specific contexts have provided important insights into the computations that may underlie decision making in the brain. However, a critical and largely unanswered question is whether these models generalize from one choice context to another. Here we show that a model adapted from the perceptual decision-making domain and estimated on choices over food rewards accurately predicts choices and reaction times in four independent sets of subjects making social decisions. The robustness of the model across domains provides behavioral evidence for a common decision-making process in perceptual, primary reward, and social decision making.

The diffusion model (Ratcliff, Psychol Rev 85(2):59-108, 1978) is a stochastic model that is applied to response time (RT) data from binary decision tasks. The model is often used to disentangle different cognitive processes. The validity of the diffusion model parameters has, however, rarely been examined. Only few experimental paradigms have been analyzed with those being restricted to fast response time paradigms. This is attributable to a recommendation stated repeatedly in the diffusion model literature to restrict applications to fast RT paradigms (more specifically, to tasks with mean RTs below 1.5 s per trial). We conducted experimental validation studies in which we challenged the necessity of this restriction. We used a binary task that features RTs of several seconds per trial and experimentally examined the convergent and discriminant validity of the four main diffusion model parameters. More precisely, in three experiments, we selectively manipulated these parameters, using a difficulty manipulation (drift rate), speed-accuracy instructions (threshold separation), a more complex motoric task (non-decision time), and an asymmetric payoff matrix (starting point). The results were similar to the findings from experimental validation studies based on fast RT paradigms. Thus, our experiments support the validity of the parameters of the diffusion model and speak in favor of an extension of the model to paradigms based on slower RTs.

Humans exhibit considerable variation in how they value their own interest relative to the interests of others. Deciphering the neural codes representing potential rewards for self and others is crucial for understanding social decision-making. Here we integrate computational modeling with functional magnetic resonance imaging to investigate the neural representation of social value and the modulation by oxytocin, a nine-amino acid neuropeptide, in participants evaluating monetary allocations to self and other (self-other allocations). We found that an individual's preferred self-other allocation serves as a reference point for computing the value of potential self-other allocations. In more prosocial participants, amygdala activity encoded a social-value-distance signal; that is, the value dissimilarity between potential and preferred allocations. Intranasal oxytocin administration amplified this amygdala representation and increased prosocial behavior in more individualistic participants but not in more prosocial ones. Our results reveal a neurocomputational mechanism underlying social-value representations and suggest that oxytocin may promote prosociality by modulating social-value representations in the amygdala.

A standard view in neuroeconomics is that to make a choice, an agent first assigns subjective values to available options, and then compares them to select the best. In choice tasks, these cardinal values are typically inferred from the preference expressed by subjects between options presented in pairs. Alternatively, cardinal values can be directly elicited by asking subjects to place a cursor on an analog scale (rating task) or to exert a force on a power grip (effort task). These tasks can vary in many respects: they can notably be more or less costly and consequential. Here, we compared the value functions elicited by choice, rating and effort tasks on options composed of two monetary amounts: one for the subject (gain) and one for a charity (donation). Bayesian model selection showed that despite important differences between the three tasks, they all elicited a same value function, with similar weighting of gain and donation, but variable concavity. Moreover, value functions elicited by the different tasks could predict choices with equivalent accuracy. Our finding therefore suggests that comparable value functions can account for various motivated behaviors, beyond economic choice. Nevertheless, we report slight differences in the computational efficiency of parameter estimation that may guide the design of future studies.

Behavioral neuroscientists have shown that the neuropeptide oxytocin (OT) plays a key role in social attachment and affiliation in nonhuman mammals. Inspired by this initial research, many social scientists proceeded to examine the associations of OT with trust in humans over the past decade. To conduct this work, they have (a) examined the effects of exogenous OT increase caused by intranasal administration on trusting behavior, (b) correlated individual difference measures of OT plasma levels with measures of trust, and (c) searched for genetic polymorphisms of the OT receptor gene that might be associated with trust. We discuss the different methods used by OT behavioral researchers and review evidence that links OT to trust in humans. Unfortunately, the simplest promising finding associating intranasal OT with higher trust has not replicated well. Moreover, the plasma OT evidence is flawed by how OT is measured in peripheral bodily fluids. Finally, in recent large-sample studies, researchers failed to find consistent associations of specific OT-related genetic polymorphisms and trust. We conclude that the cumulative evidence does not provide robust convergent evidence that human trust is reliably associated with OT (or caused by it). We end with constructive ideas for improving the robustness and rigor of OT research.

Natural selection is conventionally assumed to favour the strong and selfish who maximize their own resources at the expense of others. But many biological systems, and especially human societies, are organized around altruistic, cooperative interactions. How can natural selection promote unselfish behaviour? Various mechanisms have been proposed, and a rich analysis of indirect reciprocity has recently emerged: I help you and somebody else helps me. The evolution of cooperation by indirect reciprocity leads to reputation building, morality judgement and complex social interactions with ever-increasing cognitive demands.

In the past decade the field of cognitive sciences has seen an exponential growth in the number of computational modeling studies. Previous work has indicated why and how candidate models of cognition should be compared by trading off their ability to predict the observed data as a function of their complexity. However, the importance of falsifying candidate models in light of the observed data has been largely underestimated, leading to important drawbacks and unjustified conclusions. We argue here that the simulation of candidate models is necessary to falsify models and therefore support the specific claims about cognitive function made by the vast majority of model-based studies. We propose practical guidelines for future research that combine model comparison and falsification.

Humans not only value extrinsic monetary rewards but also their own morality and their image in the eyes of others. Yet violating moral norms is frequent, especially when people know that they are not under scrutiny. When moral values and monetary payoffs are at odds, how does the brain weigh the benefits and costs of moral and monetary payoffs? Here, using a neurocomputational model of decision value (DV) and functional (f)MRI, we investigated whether different brain systems are engaged when deciding whether to earn money by contributing to a

The diffusion decision model allows detailed explanations of behavior in two-choice discrimination tasks. In this article, the model is reviewed to show how it translates behavioral data-accuracy, mean response times, and response time distributions-into components of cognitive processing. Three experiments are used to illustrate experimental manipulations of three components: stimulus difficulty affects the quality of information on which a decision is based; instructions emphasizing either speed or accuracy affect the criterial amounts of information that a subject requires before initiating a response; and the relative proportions of the two stimuli affect biases in drift rate and starting point. The experiments also illustrate the strong constraints that ensure the model is empirically testable and potentially falsifiable. The broad range of applications of the model is also reviewed, including research in the domains of aging and neurophysiology.

There is growing interest in diffusion models to represent the cognitive and neural processes of speeded decision making. Sequential-sampling models like the diffusion model have a long history in psychology. They view decision making as a process of noisy accumulation of evidence from a stimulus. The standard model assumes that evidence accumulates at a constant rate during the second or two it takes to make a decision. This process can be linked to the behaviors of populations of neurons and to theories of optimality. Diffusion models have been used successfully in a range of cognitive tasks and as psychometric tools in clinical research to examine individual differences. In this review, we relate the models to both earlier and more recent research in psychology.

The effects of aging on decision time were examined in a brightness discrimination experiment with young and older subjects (ages, 60-75 years). Results showed that older subjects were slightly slower than young subjects but just as accurate. Ratcliff's (1978) diffusion model was fit to the data, and it provided a good account of response times, their distributions, and response accuracy. There was a 50-msec slowing of the nondecision components of response time for older subjects relative to young subjects, but response criteria settings and rates of accumulation of evidence from stimuli were roughly equal for the two groups. These results are contrasted with those obtained from letter discrimination and signal-detection-like tasks.

Egalitarian motives form a powerful force in promoting prosocial behavior and enabling large-scale cooperation in the human species [1]. At the neural level, there is substantial, albeit correlational, evidence suggesting a link between dopamine and such behavior [2, 3]. However, important questions remain about the specific role of dopamine in setting or modulating behavioral sensitivity to prosocial concerns. Here, using a combination of pharmacological tools and economic games, we provide critical evidence for a causal involvement of dopamine in human egalitarian tendencies. Specifically, using the brain penetrant catechol-O-methyl transferase (COMT) inhibitor tolcapone [4, 5], we investigated the causal relationship between dopaminergic mechanisms and two prosocial concerns at the core of a number of widely used economic games: (1) the extent to which individuals directly value the material payoffs of others, i.e., generosity, and (2) the extent to which they are averse to differences between their own payoffs and those of others, i.e., inequity. We found that dopaminergic augmentation via COMT inhibition increased egalitarian tendencies in participants who played an extended version of the dictator game [6]. Strikingly, computational modeling of choice behavior [7] revealed that tolcapone exerted selective effects on inequity aversion, and not on other computational components such as the extent to which individuals directly value the material payoffs of others. Together, these data shed light on the causal relationship between neurochemical systems and human prosocial behavior and have potential implications for our understanding of the complex array of social impairments accompanying neuropsychiatric disorders involving dopaminergic dysregulation.

Do moral disagreements regarding specific issues (e.g., patriotism, chastity) reflect deep cognitive differences (i.e., distinct cognitive mechanisms) between liberals and conservatives? Dyadic morality suggests that the answer is

The nature of harm-and therefore moral judgment-may be misunderstood. Rather than an objective matter of reason, we argue that harm should be redefined as an intuitively perceived continuum. This redefinition provides a new understanding of moral content and mechanism-the constructionist Theory of Dyadic Morality (TDM). TDM suggests that acts are condemned proportional to three elements: norm violations, negative affect, and-importantly-perceived harm. This harm is dyadic, involving an intentional agent causing damage to a vulnerable patient (A-->P). TDM predicts causal links both from harm to immorality (dyadic comparison) and from immorality to harm (dyadic completion). Together, these two processes make the

Many important moral decisions, particularly at the policy level, require the evaluation of choices involving outcomes of variable magnitude and probability. Many economic decisions involve the same problem. It is not known whether and to what extent these structurally isomorphic decisions rely on common neural mechanisms. Subjects undergoing fMRI evaluated the moral acceptability of sacrificing a single life to save a larger group of variable size and probability of dying without action. Paralleling research on economic decision making, the ventromedial prefrontal cortex and ventral striatum were specifically sensitive to the

A decade's research highlights a critical dissociation between automatic and controlled influences on moral judgment, which is subserved by distinct neural structures. Specifically, negative automatic emotional responses to prototypically harmful actions (e.g., pushing someone off of a footbridge) compete with controlled responses favoring the best consequences (e.g., saving five lives instead of one). It is unknown how such competitions are resolved to yield

Individuals exposed to community violence are more likely to engage in antisocial behavior, resulting in a dramatic increase in contact with justice and social service systems. Theoretical accounts suggest that disruptions in learning underlie the link between exposure to violence and maladaptive behaviors. However, empirical evidence specifying these processes is sparse. Here, in a sample of incarcerated males, we investigated how exposure to violence affects the ability to learn about the harmfulness of others and use this information to adaptively modulate trust behavior. Exposure to violence does not impact the ability to accurately develop beliefs about agents' harm preferences and predict their choices. However, exposure to violence disrupts the ability to form moral impressions that dissociate between agents with distinguishable harm preferences, and subsequently, the ability to adjust trust behavior towards different agents. These findings reveal a process that may explain the association between exposure to violence and maladaptive behavior.

People form moral impressions rapidly, effortlessly and from a remarkably young age(1-5). Putatively 'bad' agents command more attention and are identified more quickly and accurately than benign or friendly agents(5-12). Such vigilance is adaptive, but can also be costly in environments where people sometimes make mistakes, because incorrectly attributing bad character to good people damages existing relationships and discourages forming new relationships(13-16). The ability to accurately infer the moral character of others is critical for healthy social functioning, but the computational processes that support this ability are not well understood. Here, we show that moral inference is explained by an asymmetric Bayesian updating mechanism in which beliefs about the morality of bad agents are more uncertain (and therefore more volatile) than beliefs about the morality of good agents. This asymmetry seems to be a property of learning about immoral agents in general, as we also find greater uncertainty for beliefs about the non-moral traits of bad agents. Our model and data reveal a cognitive mechanism that permits flexible updating of beliefs about potentially threatening others, a mechanism that could facilitate forgiveness when initial bad impressions turn out to be inaccurate. Our findings suggest that negative moral impressions destabilize beliefs about others, promoting cognitive flexibility in the service of cooperative but cautious behaviour.

Mask-dependent cuing effects, like those previously found in yes-no detection, were found in a task in which observers judged the orientations of orthogonally-oriented Gabor patches presented at cued or uncued locations. Attentional cues enhanced sensitivity for masked, but not unmasked, stimuli. Responses were faster to cued than to uncued stimuli, irrespective of masking. The distributions of response times and accuracy were well described by a diffusion process model of decision making. Mask-dependent cuing was explained by an orienting model in which: (a) decisions are based on stable stimulus representations in visual short term memory that determine the rate of evidence accumulation in the diffusion process; (b) inattention delays the entry of stimuli into short term memory, and (c) masks limit the visual persistence of stimuli.

A fundamental challenge in social cognition is how humans learn another person's values to predict their decision-making behavior. This form of learning is often assumed to require simulation of the other by direct recruitment of one's own valuation process to model the other's process. However, the cognitive and neural mechanism of simulation learning is not known. Using behavior, modeling, and fMRI, we show that simulation involves two learning signals in a hierarchical arrangement. A simulated-other's reward prediction error processed in ventromedial prefrontal cortex mediated simulation by direct recruitment, being identical for valuation of the self and simulated-other. However, direct recruitment was insufficient for learning, and also required observation of the other's choices to generate a simulated-other's action prediction error encoded in dorsomedial/dorsolateral prefrontal cortex. These findings show that simulation uses a core prefrontal circuit for modeling the other's valuation to generate prediction and an adjunct circuit for tracking behavioral variation to refine prediction.

In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem. Our computational fMRI analysis revealed that anterior dorsomedial prefrontal cortex encoded inferences about action-values within the value space of the agent as opposed to that of the observer, demonstrating that inverse RL is an abstract cognitive process divorceable from the values and concerns of the observer him/herself.

The effects of aging on accuracy and response time were examined in a letter discrimination experiment with young and older subjects. Results showed that older subjects (ages 60-75) were generally slower and less accurate than young subjects. R. Ratcliff's (1978) diffusion model was fit to the data, and it provided a good account of response times, their distributions, and response accuracy. The results produce similar age effects on the nondecision components of response time (about 50 ms slowing) and the response criteria (more conservative settings) to those from R. Ratcliff, A. Thapar, and G. McKoon (2001), but also show a reduced rate of accumulation of evidence for older subjects. The model-based approach has the advantage of allowing the separation of aging effects on different components of processing.

Both normative theories of ethics in philosophy and contemporary models of moral judgment in psychology have focused almost exclusively on the permissibility of acts, in particular whether acts should be judged on the basis of their material outcomes (consequentialist ethics) or on the basis of rules, duties, and obligations (deontological ethics). However, a longstanding third perspective on morality, virtue ethics, may offer a richer descriptive account of a wide range of lay moral judgments. Building on this ethical tradition, we offer a person-centered account of moral judgment, which focuses on individuals as the unit of analysis for moral evaluations rather than on acts. Because social perceivers are fundamentally motivated to acquire information about the moral character of others, features of an act that seem most informative of character often hold more weight than either the consequences of the act or whether a moral rule has been broken. This approach, we argue, can account for numerous empirical findings that are either not predicted by current theories of moral psychology or are simply categorized as biases or irrational quirks in the way individuals make moral judgments.

Computational modelling has been used to address: (1) the variety of symptoms observed in schizophrenia using abstract models of behavior (e.g. Bayesian models - top-down descriptive models of psychopathology); (2) the causes of these symptoms using biologically realistic models involving abnormal neuromodulation and/or receptor imbalance (e.g. connectionist and neural networks - bottom-up realistic models of neural processes). These different levels of analysis have been used to answer different questions (i.e. understanding behavioral vs. neurobiological anomalies) about the nature of the disorder. As such, these computational studies have mostly supported diverging hypotheses of schizophrenia's pathophysiology, resulting in a literature that is not always expanding coherently. Some of these hypotheses are however ripe for revision using novel empirical evidence. Here we present a review that first synthesizes the literature of computational modelling for schizophrenia and psychotic symptoms into categories supporting the dopamine, glutamate, GABA, dysconnection and Bayesian inference hypotheses respectively. Secondly, we compare model predictions against the accumulated empirical evidence and finally we identify specific hypotheses that have been left relatively under-investigated.

The diffusion model (Ratcliff, 1978) allows for the statistical separation of different components of a speeded binary decision process (decision threshold, bias, information uptake, and motor response). These components are represented by different parameters of the model. Two experiments were conducted to test the interpretational validity of the parameters. Using a color discrimination task, we investigated whether experimental manipulations of specific aspects of the decision process had specific effects on the corresponding parameters in a diffusion model data analysis (see Ratcliff, 2002; Ratcliff & Rouder, 1998; Ratcliff, Thapar, & McKoon, 2001, 2003). In support of the model, we found that (1) decision thresholds were higher when we induced accuracy motivation, (2) drift rates (i.e., information uptake) were lower when stimuli were harder to discriminate, (3) the motor components were increased when a more difficult form of response was required, and (4) the process was biased toward rewarded responses.

Damage to orbitofrontal cortex (OFC) produces an unusual pattern of deficits. Patients have intact cognitive abilities but are impaired in making everyday decisions. Here we review anatomical, neuropsychological, and neurophysiological evidence to determine the neuronal mechanisms that might underlie these impairments. We suggest that OFC plays a key role in processing reward: It integrates multiple sources of information regarding the reward outcome to derive a value signal. In effect, OFC calculates how rewarding a reward is. This value signal can then be held in working memory where it can be used by lateral prefrontal cortex to plan and organize behavior toward obtaining the outcome, and by medial prefrontal cortex to evaluate the overall action in terms of its success and the effort that was required. Thus, acting together, these prefrontal areas can ensure that our behavior is most efficiently directed towards satisfying our needs.

Two experiments tested participants' attributions for others' immoral behaviors when conducted for more versus less money. We hypothesized and found that observers would blame wrongdoers more when seeing a transgression enacted for little rather than a lot of money, and that this would be evident in observers' hand-washing behavior. Experiment 1 used a cognitive dissonance paradigm. Participants (N = 160) observed a confederate lie in exchange for either a relatively large or a small monetary payment. Participants blamed the liar more in the small (versus large) money condition. Participants (N = 184) in Experiment 2 saw images of someone knocking over another to obtain a small, medium, or large monetary sum. In the small (versus large) money condition, participants blamed the perpetrator (money) more. Hence, participants assigned less blame to moral wrong-doers, if the latter enacted their deed to obtain relatively large sums of money. Small amounts of money accentuate the immorality of others' transgressions. (C) 2013 Society for Consumer Psychology. Published by Elsevier Inc.

Humans face a fundamental challenge of how to balance selfish interests against moral considerations. Such trade-offs are implicit in moral decisions about what to do; judgments of whether an action is morally right or wrong; and inferences about the moral character of others. To date, these three dimensions of moral cognition-decision-making, judgment, and inference-have been studied largely independently, using very different experimental paradigms. However, important aspects of moral cognition occur at the intersection of multiple dimensions; for instance, moral hypocrisy can be conceived as a disconnect between moral decisions and moral judgments. Here we describe the advantages of investigating these three dimensions of moral cognition within a single computational framework. A core component of this framework is harm aversion, a moral sentiment defined as a distaste for harming others. The framework integrates economic utility models of harm aversion with Bayesian reinforcement learning models describing beliefs about others' harm aversion. We show how this framework can provide novel insights into the mechanisms of moral decision-making, judgment, and inference.

Enforcement of social norms by impartial bystanders in the human species reveals a possibly unique capacity to sense and to enforce norms from a third party perspective. Such behavior, however, cannot be accounted by current computational models based on an egocentric notion of norms. Here, using a combination of model-based fMRI and third party punishment games, we show that brain regions previously implicated in egocentric norm enforcement critically extend to the important case of norm enforcement by unaffected third parties. Specifically, we found that responses in the ACC and insula cortex were positively associated with detection of distributional inequity, while those in the anterior DLPFC were associated with assessment of intentionality to the violator. Moreover, during sanction decisions, the subjective value of sanctions modulated activity in both vmPFC and rTPJ. These results shed light on the neurocomputational underpinnings of third party punishment and evolutionary origin of human norm enforcement.

Substantial correlational evidence suggests that prefrontal regions are critical to honest and dishonest behavior, but causal evidence specifying the nature of this involvement remains absent. We found that lesions of the human dorsolateral prefrontal cortex (DLPFC) decreased the effect of honesty concerns on behavior in economic games that pit honesty motives against self-interest, but did not affect decisions when honesty concerns were absent. These results point to a causal role for DLPFC in honest behavior.

Reinforcement learning (RL) is a powerful concept underlying forms of associative learning governed by the use of a scalar reward signal, with learning taking place if expectations are violated. RL may be assessed using model-based and model-free approaches. Model-based reinforcement learning involves the amygdala, the hippocampus, and the orbitofrontal cortex (OFC). The model-free system involves the pedunculopontine-tegmental nucleus (PPTgN), the ventral tegmental area (VTA) and the ventral striatum (VS). Based on the functional connectivity of VS, model-free and model based RL systems center on the VS that by integrating model-free signals (received as reward prediction error) and model-based reward related input computes value. Using the concept of reinforcement learning agent we propose that the VS serves as the value function component of the RL agent. Regarding the model utilized for model-based computations we turned to the proactive brain concept, which offers an ubiquitous function for the default network based on its great functional overlap with contextual associative areas. Hence, by means of the default network the brain continuously organizes its environment into context frames enabling the formulation of analogy-based association that are turned into predictions of what to expect. The OFC integrates reward-related information into context frames upon computing reward expectation by compiling stimulus-reward and context-reward information offered by the amygdala and hippocampus, respectively. Furthermore we suggest that the integration of model-based expectations regarding reward into the value signal is further supported by the efferent of the OFC that reach structures canonical for model-free learning (e.g., the PPTgN, VTA, and VS).