首页 期刊介绍 编 委 会 投稿指南 期刊订阅 联系我们 English

## 心理表征的可视化途径：基于噪音的反向相关图像分类技术

,1, 2

1 东北师范大学心理学院, 长春 130024

2 长春理工大学社会学系, 长春 130022

## Visualization of mental representation: Noise-based reverse correlation image classification technology

,1, 2

1 School of psychology, Northeast Normal University, Changchun 130024, China

2 Department of Sociology, Changchun University of Science and Technology, Changchun 130022, China

 基金资助: 教育部人文社会科学研究青年基金.  16YJC190009中央高校基本科研业务费专项资金资助资助.  2412017QD030

Abstract

Studies of the mental representation of images in social psychology have encountered difficulty in accurately portraying psychological activity. Over the past decade, reverse correlation image classification has emerged as a new psychophysical method that assumes there is a relationship between an observer’s response and visual noise, and that the response is based on the observer’s social judgment criteria, and are not random. Performing a sufficient number of weight calculations on the corresponding noise patterns of the observer’s reaction allows us to visualize the intrinsic evaluation characteristics of the observer. The use of reverse correlation image classification technology has achieved some results in the areas of trait research, ethnicity, and intergroup bias. In the future, however, it is necessary to solve the problems of excessive experimental trials, separation of mixed noise, and subjects’ performance, in order to achieve more realistic mental representations.

Keywords： face ; mental representation ; reverse correlation image classification technology

HOU Chun-Na, LIU Zhi-Jun. (2019). Visualization of mental representation: Noise-based reverse correlation image classification technology. Advances in Psychological Science, 27(3), 465-474

## 1 RCIC的起源与数理基础

RCIC发展自反向相关技术, 后者在过去40年中被广泛应用于视觉和听觉的知觉过程研究。反向相关技术本质上遵从信号检测理论, 但是与通过被试甄别刺激信号的标准实验程序相反。传统的信号检测范式中, 反应依赖于对刺激属性的有意义操作, 两者间的关系是由固定刺激与反应的属性所决定的, 即同样刺激可诱发出不同反应。但在反向相关技术中, 其假定由于不同刺激的推断模式导致了不同的反应, 因而刺激不再固定不变而是随机变化的, 由被试自行决定信号是否存在刺激中。这种技术通过对产生虚惊的噪音场的收集和组合, 以推断该过程中用于执行特定任务的策略。其中, 术语“反向(reverse)”即是指刺激和反应之间统计学关系的反转之意。RCIC能够通过刺激噪音参数的随机变化及对刺激判断中每个刺激噪音模式相对权重的估计而生成直观的表征内容视觉图像代码。

RCIC的数理基础可追溯至数学中的相关。在视觉研究中, 依照视觉刺激信号从视网膜的输出神经元经由外侧膝状核(lateral geniculate nucleus), 传递至初级视觉皮层(primary visual cortex)的过程, 刺激与神经反应两种信号之间随时间滞后的相关关系, 可用积分公式表现为：

$Corr[g,h](t)=\int_{-\infty }^{\infty }{dt'g(t')h(t+t')}$

$Corr{{(g,h)}_{j}}=\sum\limits_{i=-\infty }^{\infty }{{{g}_{i}}{{h}_{i+j}}}$

${{y}_{i}}\approx \sum\limits_{j}{{{h}_{j}}{{x}_{i-j}}}$

${{h}_{k}}=\frac{C_{k}^{xy}}{C_{0}^{xx}}$(注：这里的$C_{k-j}^{xx}=C_{0}^{xx}{{\delta }_{kj}}$)

### 2.2 正弦随机噪音研究范式

Mangini和Biederman (2004)采用随机变化的正弦噪音替代高斯白噪音, 相比之下正弦曲线更具优势。这是因为：首先, 正弦曲线更接近早期皮层视觉区域的优选刺激。其次, 由于噪声谱(noise spectrum)类似于信号分布, 观察者更可能将噪声解释为刺激变化。虽然这样的功率谱(power spectrum)可以通过对白噪声进行滤波来实现, 但是正弦噪声的功率谱比白噪声要少四倍的参数。更为重要的是, 在少量的实验次数中(例如少于1000次), 正弦噪声收敛要比白噪声估计的更为准确, 这无疑有益于降低实验次数, 减轻被试的负担。

### 图3

Dotsch等人(2008)将实验范式区分为生成随机噪音刺激、执行双图片迫选任务、产生分类图像及执行像素聚类分析四个组成部分。学者认为此种范式将面孔刺激生成过程与特质形成过程相分离, 能够从机制上客观揭示出面孔表征直接导致的特质推断过程(Ratner, Dotsch, Wigboldus, van Knippenberg, & Amodio, 2014)。目前, 已经广泛应用于社会知觉的相关研究中(Karremans, Dotsch, & Corneille, 2011; Paulus, Rohr, Dotsch, & Wentura, 2016)。

### 3.1 方法原理优势

RCIC虽然受到特定实验范式中所使用判断和刺激类型的限制, 但它对反映社会认知的刺激信息所实行的可视化手段却是无约束的。此种方法通过要求被试对叠加随机噪音的基本图像进行判断, 由于随机噪音在像素水平上扭曲了基本图像(如面孔), 因此形成的刺激变化要较传统心理物理法受到的约束更小。这种方法的最终目的是为了计算出分类图像, 从而将驱动社会判断的心理特征可视化。需要注意的是RCIC的可视化效果与眼动追踪研究有着根本的不同。眼动追踪技术识别的只是呈现完整图像时, 被试视网膜中央凹所形成的部分图像; 而RCIC识别则为分类提供潜在信息的整个刺激可视区域。

### 3.2 生态效度优势

RCIC另一个优势是生态效度较高。当信号特征未知或想要考察分类表征时, 由于没有任何先验假定图像的偏差, 使得这种范式不易受社会期望或需求特征的影响。RCIC也继而成为挖掘高级心理过程中自发使用信息的一种理想方法。

## 5 技术局限与前景展望

RCIC的目的是建立可视化的心理表征内容, 虽然这种技术具有较高的生态效度, 但其只能够提供真实心理表征的近似值。这是因为由该技术生成的分类图像除了包含真实心理意象外, 还有叠加的噪音混杂其中, 同时被试的表现也会影响分类图像的生成, 如何将这些混杂的因素分离, 从而得到真实的心理表征仍然是值得继续深入探究的问题之一。此外对于基本图像的设置也存在不同的声音, 有学者通过面孔平均技术所得到的合成面孔作为基础面孔能够降低单独面孔间的差异细节, 且具有与单独面孔更多的相似性而被认为更具代表性(Bijvank, 2014), 其他学者则认为RCIC技术基于计算机做出的三维面孔作为刺激材料可能更好(Chen, Garrod, Schyns, & Jack, 2017)。

## 参考文献 原文顺序 文献年度倒序 文中引用次数倒序 被引期刊影响因子

Adler R.J., & Hasofer A.M . ( 1976).

Level crossings for random fields

The Annals of Probability, 4( 1), 1-12.

For an $n$-dimensional random field $X(\mathbf{t})$ we define the excursion set $A$ of $X(\mathbf{t})$ by $A = \{\mathbf{t} \in \mathbf{I}_0: X(\mathbf{t}) \geqq u\}$, where $I_0$ is the unit cube in $R^n.$ It is shown that the natural generalisation of the number of upcrossings of a one-dimensional stochastic process to random fields is via the characteristic of the set $A$ introduced by Hadwiger (1959). This characteristic is related to the number of connected components of $A$. A formula is obtained for the mean value of this characteristic when $n = 2, 3$. This mean value is calculated explicitly when $X(\mathbf{t})$ is a homogeneous Gaussian field satisfying certain regularity conditions.

Bijvank M. ( 2014).

Periodic review inventory systems with a service level criterion

Journal of the Operational Research Society, 65( 12), 1853-1863.

Full-cost inventory models are mostly studied in the literature, whereas service level constraints are more common to be observed in practical settings. In this paper, we consider periodic review inventory systems with service level restrictions. The control of such inventory systems is limited to ( s , S )-type policies in the literature. To the best of our knowledge, we are the first authors to compare such policies with optimal replenishment policies, and illustrate an average cost difference of 0.64%. This justifies the use of these popular ( s , S ) policies in practice. Furthermore, we propose a new one-dimensional search procedure that is bounded to set the reorder level s and order-up-to level S , whereas the solution space is unbounded and two dimensional. Our heuristic procedure is guaranteed to satisfy the service level constraint and numerical experiments illustrate that it results in an average cost deviation of 1 2% compared with the best ( s , S ) policy. Consequently, it significantly outperforms all existing procedures from literature, both in service and costs.

Brinkman L., Todorov A., & Dotsch R . ( 2017).

Visualising mental representations: A primer on noise-based reverse correlation in social psychology

European Review of Social Psychology, 28( 1), 333-361.

In recent years, social psychologists have adopted the psychophysical method of reverse correlation. Reverse correlation is a data-driven method in which judgments of randomly varying stimuli are used to reconstruct participants' internal representation subtending those judgments. Here, we review the method and its findings within the context of social psychology, discussing its promises, achievements, and validity. Our review suggests that, even in these early stages, the technique has proven to be an invaluable tool in the investigation of social perception, such as perception of race, gender, personality traits, and internal states such as emotions. Integrating the way the technique has been used in social psychology, we suggest that reverse correlation primarily enables researchers to do two things: visualise what visual features are diagnostic for particular social judgments and reveal top down biases on social perception. Given these functions, we argue that reverse correlation can be best understood as visualising priors for social perception within a predictive coding framework.

Brown-Iannuzzi J. L., Dotsch R., Cooley E., & Payne B. K . ( 2017).

The relationship between mental representations of welfare recipients and attitudes toward welfare

Psychological Science, 28( 1), 92-103.

Chen C., Garrod O., Schyns P., & Jack R . ( 2017).

Mapping dynamic conversational facial expressions across cultures

Journal of Vision, 17( 10), 834-834.

Clark C. M., Gosselin F., & Goghari V. M . ( 2013).

Aberrant patterns of visual facial information usage in schizophrenia

Journal of Abnormal Psychology, 122( 2), 513-519.

URL     PMID:23713505

Deficits in facial emotion perception have been linked to poorer functional outcome in schizophrenia. However, the relationship between abnormal emotion perception and functional outcome remains poorly understood. To better understand the nature of facial emotion perception deficits in schizophrenia, we used the Bubbles Facial Emotion Perception Task to identify differences in usage of visual facial information in schizophrenia patients (n = 20) and controls (n = 20), when differentiating between angry and neutral facial expressions. As hypothesized, schizophrenia patients required more facial information than controls to accurately differentiate between angry and neutral facial expressions, and they relied on different facial features and spatial frequencies to differentiate these facial expressions. Specifically, schizophrenia patients underutilized the eye regions, overutilized the nose and mouth regions, and virtually ignored information presented at the lowest levels of spatial frequency. In addition, a post hoc one-tailed t test revealed a positive relationship of moderate strength between the degree of divergence from "normal" visual facial information usage in the eye region and lower overall social functioning. These findings provide direct support for aberrant patterns of visual facial information usage in schizophrenia in differentiating between socially salient emotional states.

Dotsch R. & Todorov A., ( 2012).

Reverse correlating social face perception

Social Psychological and Personality Science, 3( 5), 562-571.

ABSTRACT Reverse correlation (RC) techniques provide a data-driven approach to model internal representations in an unconstrained way. Here, we used this approach to model social perception of faces. In the RC task, participants repeatedly selected from two face images—created by superimposing randomly generated noise masks on the same face—the face that looked most trustworthy (or, in other conditions: untrustworthy, dominant, or submissive). We calculated classification images (CIs) by averaging all selected images. Trait judgments of independent participants, as well as objective metrics, showed that the CIs visualized the intended traits well. Furthermore, tests of pixel clusters showed that diagnostic information resided mostly in mouth, eye, eyebrow, and hair regions. The current work shows that RC provides an excellent tool to extract psychologically meaningful images that map onto social perception.

Dotsch R., Wigboldus D. H. J., Langner O., & van Knippenberg A . ( 2008).

Ethnic out-group faces are biased in the prejudiced mind

Psychological Science, 19( 10), 978-980.

URL     PMID:19000205

Prejudice biases cognition, affect, and behavior toward ethnic out-groups. We propose that prejudice also biases the way people conceptualize the facial appearance of out-group members. Popular belief holds that people's personality traits are reflected in their facial features. We hypothesized that prejudiced people also have more negatively stereotyped mental representations of faces of people in the out-group. To test this hypothesis, we conducted two studies involving the category of Moroccans, a highly stigmatized immigrant group in The Netherlands. The results of the first study suggested that highly prejudiced people have biased mental representations of Moroccan faces. We ran a second study using more trials in the image construction phase, but with an otherwise identical design, to enhance the quality of individual participants' classification images. This allowed us to replicate the findings of the first study on an individual rather than subgroup level.

Dunham Y., Srinivasan M., Dotsch R., & Barner D . ( 2014).

Religion insulates ingroup evaluations: The development of intergroup attitudes in India

Developmental Science, 17( 2), 311-319.

URL     PMID:24205988

Research on the development of implicit intergroup attitudes has placed heavy emphasis on race, leaving open how social categories that are prominent in other cultures might operate. We investigate two of India's primary means of social distinction, caste and religion, and explore the development of implicit and explicit attitudes towards these groups in minority-status Muslim children and majority-status Hindu children, the latter drawn from various positions in the Hindu caste system. Results from two tests of implicit attitudes find that caste attitudes parallel previous findings for race: higher-caste children as well as lower-caste children have robust high-caste preferences. However, results for religion were strikingly different: both lower-status Muslim children and higher-status Hindu children show strong implicit ingroup preferences. We suggest that religion may play a protective role in insulating children from the internalization of stigma.

Dotsch R.., Wigboldus D.H, & Van K.A . ( 2011).

Biased allocation of faces to social categories

Journal of Personality and Social Psychology, 100(6), 999-1014.

URL     PMID:21443368

Three studies show that social categorization is biased at the level of category allocation. In all studies, participants categorized faces. In Studies 1 and 2, participants overallocated faces with criminal features-a stereotypical negative trait-to the stigmatized Moroccan category, especially if they were prejudiced. On the contrary, the stereotype-irrelevant negative trait stupid did not lead to overallocation to the Moroccan category. In Study 3, using the stigmatized category homosexual, the previously used negative trait criminal-irrelevant to the homosexual stereotype-did not lead to overallocation, but the stereotype-relevant positive trait femininity did. These results demonstrate that normative fit is higher for faces with stereotype-relevant features regardless of valence. Moreover, individual differences in implicit prejudice predicted the extent to which stereotype-relevant traits elicited overallocation: Whereas more negatively prejudiced people showed greater overallocation of faces associated with negative stereotype-relevant traits, they showed less overallocation of faces associated with positive stereotype-relevant traits. These results support our normative fit hypothesis: In general, normative fit is better for faces with stereotypical features. Moreover, normative fit is enhanced for prejudiced individuals when these features are evaluatively congruent. Social categorization thus may be biased in itself.

Fiske S.T . ( 2015).

Intergroup biases: A focus on stereotype content

Current Opinion in Behavioral Sciences, 3, 45-50.

URL     PMID:4955357

Impressions of others, including societal groups, systematically array along two dimensions, warmth (trustworthiness/friendliness) and competence. Social structures of competition and status respectively predict these usually orthogonal dimensions. Prejudiced emotions (pride, pity, contempt, and envy) target each quadrant, and distinct discriminatory behavioral tendencies result. The Stereotype Content Model (SCM) patterns generalize across time (2oth century), culture (every populated continent), level of analysis (targets from individuals to subtypes to groups to nations), and measures (from neural to self-report to societal indicators). Future directions include individual differences in endorsement of these cultural stereotypes and how perceivers view combinations across the SCM space.

Gosselin F., Bacon B. A., & Mamassian P . ( 2004).

Internal surface representations approximated by reverse correlation

Vision research, 44( 21), 2515-2520.

URL     PMID:15358086

We presented two na?¨ve observers with 20,000 random-dot stereograms. On each trial, the observers had to indicate the presence or absence of a complex 3D pattern (a large ‘+’ sign in relief). However, unbeknownst to them, the stereograms did not contain any signal, but only disparity noise. Responses and verbal reports indicate that the observers ‘saw’ the suggested 3D surface configuration in roughly half the trials even though structured local low-level signal was never presented. Using reverse correlation, we derived an approximation of the internal surface-based representations, or templates, that best accounted for the observers’ responses. These templates were shown to be spatially well defined and temporally stable. We propose that the 3D surface-based representations that we derived are the first approximations and depictions of the intermediary process that allows the visual system to successfully link degraded, bottom–up signal and high-level, top–down object recognition.

Gunaydin G., & Delong J.E . ( 2015).

Reverse correlating love: Highly passionate women idealize their partner's facial appearance

Plos One, 10( 3), e0121094.

URL     PMID:4373891

A defining feature of passionate love is idealization valuating romantic partners in an overly favorable light. Although passionate love can be expected to color how favorably individuals represent their partner in their mind, little is known about how passionate love is linked with visual representations of the partner. Using reverse correlation techniques for the first time to study partner representations, the present study investigated whether women who are passionately in love represent their partner facial appearance more favorably than individuals who are less passionately in love. In a within-participants design, heterosexual women completed two forced-choice classification tasks, one for their romantic partner and one for a male acquaintance, and a measure of passionate love. In each classification task, participants saw two faces superimposed with noise and selected the face that most resembled their partner (or an acquaintance). Classification images for each of high passion and low passion groups were calculated by averaging across noise patterns selected as resembling the partner or the acquaintance and superimposing the averaged noise on an average male face. A separate group of women evaluated the classification images on attractiveness, trustworthiness, and competence. Results showed that women who feel high (vs. low) passionate love toward their partner tend to represent his face as more attractive and trustworthy, even when controlling for familiarity effects using the acquaintance representation. Using an innovative method to study partner representations, these findings extend our understanding of cognitive processes in romantic relationships.

Imhoff R., Woelki J., Hanke S., & Dotsch R . ( 2013).

Warmth and competence in your face! Visual encoding of stereotype content

Frontiers in Psychology, 4( 386), 1-8.

URL     PMID:23825468

Previous research suggests that stereotypes about a group's warmth bias our visual representation of group members. Based on the stereotype content model (SCM) the current research explored whether the second big dimension of social perception, competence, is also reflected in visual stereotypes. To test this, participants created typical faces for groups either high in warmth and low in competence (male nursery teachers) or vice versa (managers) in a reverse correlation image classification task, which allows for the visualization of stereotypes without any a priori assumptions about relevant dimensions. In support of the independent encoding of both SCM dimensions hypotheses-blind raters judged the resulting visualizations of nursery teachers as warmer but less competent than the resulting image for managers, even when statistically controlling for judgments on one dimension. People thus seem to use facial cues indicating both relevant dimensions to make sense of social groups in a parsimonious, non-verbal and spontaneous manner.

Jack R. E., Caldara R., & Schyns P. G . ( 2012).

Internal representations reveal cultural diversity in expectations of facial expressions of emotion

Journal of Experimental Psychology: General, 141( 1), 19-25.

URL     PMID:21517206

Facial expressions have long been considered the "universal language of emotion." Yet consistent cultural differences in the recognition of facial expressions contradict such notions (e.g., R. E. Jack, C. Blais, C. Scheepers, P. G. Schyns, & R. Caldara, 2009). Rather, culture--as an intricate system of social concepts and beliefs--could generate different expectations (i.e., internal representations) of facial expression signals. To investigate, they used a powerful psychophysical technique (reverse correlation) to estimate the observer-specific internal representations of the 6 basic facial expressions of emotion (i.e., happy, surprise, fear, disgust, anger, and sad) in two culturally distinct groups (i.e., Western Caucasian [WC] and East Asian [EA]). Using complementary statistical image analyses, cultural specificity was directly revealed in these representations. Specifically, whereas WC internal representations predominantly featured the eyebrows and mouth, EA internal representations showed a preference for expressive information in the eye region. Closer inspection of the EA observer preference revealed a surprising feature: changes of gaze direction, shown primarily among the EA group. For the first time, it is revealed directly that culture can finely shape the internal representations of common facial expressions of emotion, challenging notions of a biologically hardwired "universal language of emotion."

Johnson K. L., Iida M., & Tassinary L. G . ( 2012).

Person (mis)perception: functionally biased sex categorization of bodies

Proceedings of the Royal Society B Biological Sciences, 279( 1749), 4982-4989.

URL     PMID:3497247

Social perception is among the most important tasks that occur in daily life, and perceivers readily appreciate the social affordances of others. Here, we demonstrate that sex categorizations are functionally biased towards a male percept. Perceivers judged body shapes that varied in waist-to-hip ratio to be men if they were not, in reality, exclusive to women, and male categorizations occurred more quickly than female categorizations (studies 1 and 4). This pattern was corroborated when participants identified the average body shapes of men and women (study 2) and when we assessed participants' cognitive representations (study 3). Moreover, these tendencies were modulated by emotion context (study 4). Thus, male categorizations occurred readily and rapidly, demonstrating a pronounced categorization bias and temporal advantage for male judgements.

Karremans J.C., Dotsch R., & Corneille O . ( 2011).

Romantic relationship status biases memory of faces of attractive opposite-sex others: Evidence from a reverse-correlation paradigm

Cognition, 121( 3), 422-426.

URL     PMID:21903209

Previous research has demonstrated that, presumably as a way to protect one’s current romantic relationship, individuals involved in a heterosexual romantic relationship tend to give lower attractiveness ratings to attractive opposite-sex others as compared to uninvolved individuals (i.e., the derogation effect). The present study importantly extends this research by examining whether romantic relationship status actually biases memory for the facial appearance of attractive (vs. unattractive) mates. To address this issue, we used a reverse-correlation technique ( Mangini & Biederman, 2004), originally developed to get a visual approximation of an individual’s internal representation of a target category or person. In line with the derogation effect, results demonstrated that romantically involved (vs. uninvolved) individuals indeed held a less attractive memory of a previously encountered attractive mate’s face. Interestingly, they also held a more attractive memory of an unattractive mate’s face as compared to uninvolved individuals. This latter finding may suggest that romantically involved (as compared to uninvolved) individuals differentiate opposite-sex others along the attractiveness dimension less.

Krosch A.R., & Amodio D.M . ( 2014).

Economic scarcity alters the perception of race

Proceedings of the National Academy of Sciences of the United States of America, 111( 25), 9079-9084.

URL     PMID:24927595

Abstract When the economy declines, racial minorities are hit the hardest. Although existing explanations for this effect focus on institutional causes, recent psychological findings suggest that scarcity may also alter perceptions of race in ways that exacerbate discrimination. We tested the hypothesis that economic resource scarcity causes decision makers to perceive African Americans as "Blacker" and that this visual distortion elicits disparities in the allocation of resources. Studies 1 and 2 demonstrated that scarcity altered perceptions of race, lowering subjects' psychophysical threshold for seeing a mixed-race face as "Black" as opposed to "White." In studies 3 and 4, scarcity led subjects to visualize African American faces as darker and more "stereotypically Black," compared with a control condition. When presented to na ve subjects, face representations produced under scarcity elicited smaller allocations than control-condition representations. Together, these findings introduce a novel perceptual account for the proliferation of racial disparities under economic scarcity.

Lick D. J., Carpinella C. M., Preciado M. A., Spunt R. P., & Johnson K. L . ( 2013).

Reverse-correlating mental representations of sex-typed bodies: The effect of number of trials on image quality

Frontiers in Psychology, 4( 2), 476-484.

URL     PMID:3727110

Sex categorization is a critical process in social perception. While psychologists have long theorized that perceivers have distinct mental representations of men and women that help them to achieve efficient sex categorizations, researchers have only recently begun using reverse-correlation to visualize the content of these mental representations. The present research addresses two issues concerning this relatively new methodological tool. First, previous studies of reverse-correlation have focused almost exclusively on perceivers' mental representations of faces. Our study demonstrates that this technique can also be used to visualize mental representations of sex-typed bodies. Second, most studies of reverse-correlation have employed a relatively large number of trials (1000+) to capture perceivers' mental representations of a given category. Our study demonstrated that, at least for sex-typed representations of bodies, high quality reverse-correlation images can be obtained with as few as 100 trials. Overall, our findings enhance knowledge of reverse-correlation methodology in general and sex categorization in particular, providing new information for researchers interested in using this technique to understand the complex processes underlying social perception.

Mangini M.C., & Biederman I. , ( 2004).

Making the ineffable explicit: Estimating the information employed for face classifications

Cognitive Science, 28( 2), 209-226.

When we look at a face, we readily perceive that person's gender, expression, identity, age, and attractiveness. Perceivers as well as scientists have hitherto had little success in articulating just what information we are employing to achieve these subjectively immediate and effortless classifications. We describe here a method that estimates that information. Observers classified faces in high levels of visual noise as male or female (in a gender task), happy/unhappy (in an expression task), or Tom Cruise/John Travolta (in an individuation task). They were unaware that the underlying face (which was midway between each of the classes) was identical throughout a task, with only the noise rendering it more like one category value or the other. The difference between the average of noise patterns for each classification decision provided a linear estimate of the information mediating these classifications. When the noise was combined with the underlying face, the resultant images appeared to be excellent prototypes of their respective classes. Other methods of estimating the information employed in complex classification have relied on judgments of exemplars of a class or tests of experimenter-defined hypotheses about the class information. Our method allows an estimate, however subtle, of what is in the subject's (rather than the experimenter's) head.

Martin-Malivel J., Mangini M. C., Fagot J., & Biederman I . ( 2010).

Do humans and baboons use the same information when categorizing human and baboon faces?

Psychological Science, 17( 7), 599-607.

URL     PMID:16866746

ABSTRACT What information is used for sorting pictures of complex stimuli into categories? We applied a reverse correlation method to reveal the visual features mediating categorization in humans and baboons. Two baboons and 6 humans were trained to sort, by species, pictures of human and baboon faces on which random visual noise was superimposed. On ambiguous probe trials, a human-baboon morph was presented, eliciting "human" responses on some trials and "baboon" responses on others. The difference between the noise patterns that induced the two responses made explicit the information mediating the classification. Unlike the humans, the baboons based their categorization on information that closely matched that used by a theoretical observer responding solely on the basis of the pixel similarities between the probe and training images. We show that the classification-image technique and principal components analysis provide a method to make explicit the differences in the information mediating categorization in humans and animals.

Nunnari F. & Heloir A., ( 2017).

Generating virtual characters from personality traits via reverse correlation and linear programming

Conference on Autonomous Agents and Multiagent Systems, 1661-1663.

Oosterhof N.N., & Todorov A. , ( 2008).

The functional basis of face evaluation

Proceedings of the National Academy of Sciences, 105( 32), 11087-11092.

URL     PMID:18685089

People automatically evaluate faces on multiple trait dimensions, and these evaluations predict important social outcomes, ranging from electoral success to sentencing decisions. Based on behavioral studies and computer modeling, we develop a 2D model of face evaluation. First, using a principal components analysis of trait judgments of emotionally neutral faces, we identify two orthogonal dimensions, valence and dominance, that are sufficient to describe face evaluation and show that these dimensions can be approximated by judgments of trustworthiness and dominance. Second, using a data-driven statistical model for face representation, we build and validate models for representing face trustworthiness and face dominance. Third, using these models, we show that, whereas valence evaluation is more sensitive to features resembling expressions signaling whether the person should be avoided or approached, dominance evaluation is more sensitive to features signaling physical strength/weakness. Fourth, we show that important social judgments, such as threat, can be reproduced as a function of the two orthogonal dimensions of valence and dominance. The findings suggest that face evaluation involves an overgeneralization of adaptive mechanisms for inferring harmful intentions and the ability to cause harm and can account for rapid, yet not necessarily accurate, judgments from faces.

Paulus A., Rohr M., Dotsch R., & Wentura D . ( 2016).

Positive feeling, negative meaning: Visualizing the mental representations of in-group and out-group smiles

PloS one, 11( 3), e0151230.

URL     PMID:4786158

Even though smiles are seen as universal facial expressions, research shows that there exist various kinds of smiles (i.e., affiliative smiles, dominant smiles). Accordingly, we suggest that there also exist various mental representations of smiles. Which representation is employed in cognition may depend on social factors, such as the smiling person group membership: Since in-group members are typically seen as more benevolent than out-group members, in-group smiles should be associated with more benevolent social meaning than those conveyed by out-group members. We visualized in-group and out-group smiles with reverse correlation image classification. These visualizations indicated that mental representations of in-group smiles indeed express more benevolent social meaning than those of out-group smiles. The affective meaning of these visualized smiles was not influenced by group membership. Importantly, the effect occurred even though participants were not instructed to attend to the nature of the smile, pointing to an automatic association between group membership and intention.

Ponsot E., Arias P., & Aucouturier J. J . ( 2018).

Uncovering mental representations of smiled speech using reverse correlation

Journal of the Acoustical Society of America, 143( 1), 19-24.

URL     PMID:29390775

Abstract Which spectral cues underlie the perceptual processing of smiles in speech? Here, the question was addressed using reverse-correlation in the case of the isolated vowel [a]. Listeners were presented with hundreds of pairs of utterances with randomly manipulated spectral characteristics and were asked to indicate, in each pair, which was the most smiling. The analyses revealed that they relied on robust spectral representations that specifically encoded vowel's formants. These findings demonstrate the causal role played by formants in the perception of smile. Overall, this paper suggests a general method to estimate the spectral bases of high-level (e.g., emotional/social/paralinguistic) speech representations.

Ratner K. G., Dotsch R., Wigboldus D. H., van Knippenberg A., & Amodio D. M . ( 2014).

Visualizing minimal ingroup and outgroup faces: implications for impressions, attitudes, and behavior

Journal of Personality and Social Psychology, 106( 6), 897-911.

URL     PMID:24841095

More than 40 years of research have shown that people favor members of their ingroup in their impressions, attitudes, and behaviors. Here, we propose that people also form different mental images of minimal ingroup and outgroup members, and we test the hypothesis that differences in these mental images contribute to the well-established biases that arise from minimal group categorization. In Study 1, participants were assigned to 1 of 2 groups using a classic minimal group paradigm. Next, a reverse correlation image classification procedure was used to create visual renderings of ingroup and outgroup face representations. Subsequently, a 2nd sample naive to the face generation stage rated these faces on a series of trait dimensions. The results indicated that the ingroup face was significantly more likely than the outgroup face to elicit favorable impressions (e.g., trusting, caring, intelligent, attractive). Extending this finding, Study 2 revealed that ingroup face representations elicited more favorable implicitly measured attitudes than did outgroup representations, and Study 3 showed that ingroup faces were trusted more than outgroup faces during an economic game. Finally, Study 4 demonstrated that facial physiognomy associated with trustworthiness more closely resembled the facial structure of the average ingroup than outgroup face representation. Together, these studies suggest that minimal group distinctions can elicit different mental representations, and that this visual bias is sufficient to elicit ingroup favoritism in impressions, attitudes and behaviors.

Saegusa C., Yamaoka M., & Watanabe K . ( 2015).

Seeing faces in noise: Exploring machine and human face detection processes by the reverse correlation method

Paper presented at the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Siem Reap, Cambodia.

Todorov A., Dotsch R., Wigboldus D. H. J., & Said C. P . ( 2011).

Data-driven methods for modeling social perception

Social and Personality Psychology Compass, 5( 10), 775-791.

How do we model the complexity of social perception? A major methodological problem is that the space of possible variables driving social perceptions is infinitely large, thus posing an insurmountable hurdle for conventional approaches. Here, we describe a set of data-driven methods whose objective is to identify quantitative relationships between high-dimensional variables (e.g., visual images) and behaviors (e.g., perceptual decisions) with as little bias as possible. We focus on social perception of faces, although the methods could be applied to other visual and nonvisual categories. We review two reverse correlation approaches: (a) psychophysical methods based on judgments of images altered with randomly generated noise, where the analysis relates the random variations of the images to judgments; and (b) methods based on judgments of randomly generated faces from a statistical, multidimensional face space model, where the analysis relates the dimensions of the face model to judgments.

Todorov A., Olivola C. Y., Dotsch R., & Mende-Siedlecki P . ( 2015).

Social attributions from faces: determinants, consequences, accuracy, and functional significance

Annual Review of Psychology, 66( 1), 519-545.

URL     PMID:25196277

Since the early twentieth century, psychologists have known that there is consensus in attributing social and personality characteristics from facial appearance. Recent studies have shown that surprisingly little time and effort are needed to arrive at this consensus. Here we review recent research on social attributions from faces. Section I outlines data-driven methods capable of identifying the perceptual basis of consensus in social attributions from faces (e.g., What makes a face look threatening?). Section II describes nonperceptual determinants of social attributions (e.g., person knowledge and incidental associations). Section III discusses evidence that attributions from faces predict important social outcomes in diverse domains (e.g., investment decisions and leader selection). In Section IV, we argue that the diagnostic validity of these attributions has been greatly overstated in the literature. In the final section, we offer an account of the functional significance of these attributions.

van Driel, S. D . ( 2017).

Prediction of Self Perception based on Dominance and Trustworthiness by using Reverse Correlation

( Unpublished master’s thesis). Utrecht University, Netherlands.

Van Rijsbergen N., Jaworska K., Rousselet G. A., & Schyns P. G . ( 2014).

With age comes representational wisdom in social signals

Current Biology, 24( 23), 2792-2796.

URL     PMID:4251953

Old people have more-faithful mental representations of facial age, whereas young people have an underdeveloped mental representation that divides the age spectrum into young (like them) and old (everyone else). van Rijsbergen et al. demonstrate that the older mind depicts socially relevant information more accurately than its younger counterpart.

Young A.I . ( 2014).

Seeing scary: Predicting variation in the scariness of the mental representations of spiders

( Unpublished doctoral dissertation). The Ohio State University, Ohio State.