Facial Movements Strategically Camouflage Involuntary Social Signals of Face Morphology

Abstract

Animals use social camouflage as a tool of deceit to increase the likelihood of survival and reproduction. We tested whether humans can also strategically deploy transient facial movements to camouflage the default social traits conveyed by the phenotypic morphology of their faces. We used the responses of 12 observers to create models of the dynamic facial signals of dominance, trustworthiness, and attractiveness. We applied these dynamic models to facial morphologies differing on perceived dominance, trustworthiness, and attractiveness to create a set of dynamic faces; new observers rated each dynamic face according to the three social traits. We found that specific facial movements camouflage the social appearance of a face by modulating the features of phenotypic morphology. A comparison of these facial expressions with those similarly derived for facial emotions showed that social-trait expressions, rather than being simple one-to-one overgeneralizations of emotional expressions, are a distinct set of signals composed of movements from different emotions. Our generative face models represent novel psychophysical laws for social sciences; these laws predict the perception of social traits on the basis of dynamic face identities.

Keywords

social cognition facial expressions face perception

In human social interaction, the face is a central tool of communication because it provides a rich source of social information. Although some signals (e.g., facial expressions of emotion) can be voluntarily deployed strategically to negotiate social situations, other signals (e.g., those indicating social traits such as dominance, trustworthiness, and attractiveness) are transmitted involuntarily by the phenotypic morphology of the face (Oosterhof & Todorov, 2008). This involuntary signaling has significant consequences for individuals (e.g., mate choice—Little, Burriss, Jones, DeBruine, & Caldwell, 2008; occupational opportunities—Johnson, Podratz, Dipboye, & Gibbons, 2010; sentencing decisions—Blair, Judd, & Chapleau, 2004) and groups (e.g., voting preferences—Todorov, Mandisodza, Goren, & Hall, 2005). However, humans are highly adaptive social beings; like other social animals, humans can camouflage these involuntary morphology-based signals to optimize success within their ecological niche.

Here, we address three main questions about such human social-camouflage strategies. First, can dynamic facial signals camouflage the involuntary social signals transmitted by static facial morphology? Second, if they can, which specific facial movements camouflage which facial morphologies, and how efficiently do they do so? Finally, how do dynamic camouflaging signals relate to other socially relevant dynamic facial signals, such as facial expressions of emotion?

To address these questions, in Experiment 1, using a reverse-correlation approach, we modeled the dynamic facial signals of three basic social traits—dominance, trustworthiness, and attractiveness (Oosterhof & Todorov, 2008; Sutherland et al., 2013). Hereafter, we refer to these dynamic models as dynamic social masks.

In Experiment 2, we examined the camouflaging capabilities of the dynamic social masks (i.e., whether they could override the involuntary social signals transmitted by static facial morphology). Specifically, in a first phase, we determined the perceived social traits of static facial morphology by asking observers to rate a new set of three-dimensional (3-D) static faces for dominance, trustworthiness, and attractiveness. In a second phase, we parametrically applied the dynamic social masks to each of the new face identities and obtained social-trait ratings for each combined facial animation using a new set of observers.

In Experiment 3, we examined the relationship between two dynamic systems of facial signaling, emotion and social traits, by comparing the action units (AUs) of dynamic social and emotional signals using the same reverse-correlation approach as in Experiment 1.

Our data revealed that dynamic social masks successfully camouflaged the involuntary social signals transmitted by face morphology, such that the dynamic social masks drove social perception, modulating the effects of underlying facial morphology. Comparison with dynamic models of facial expressions of emotion showed that dynamic social masks constitute a unique set of signals that differ from the six classic facial expressions of emotion.

Experiment 1

To model the dynamic social masks of dominance, trustworthiness, and attractiveness, we used reverse correlation (Ahumada & Lovell, 1971), the Facial Action Coding System (FACS; Ekman & Friesen, 1978), a random generator of photo-realistic facial movements (the four-dimensional generative face grammar, or 4-D GFG; Yu, Garrod, & Schyns, 2012), and subjective perceptual judgments.

Observers

Twelve White observers participated (6 men, 6 women; mean age = 21.0 years, SD = 3.0 years). We recruited observers with minimal experience of other cultures (as assessed by a screening questionnaire; see Methodological Details in the Supplemental Material available online) and normal or corrected-to-normal vision. All observers gave written informed consent and were paid £6 per hour for their participation. The study was approved by the College of Science and Engineering ethics committee at the University of Glasgow.

Stimuli

Figure 1 illustrates the stimulus generation and task procedure. On each experimental trial, a photo-realistic 4-D GFG randomly selected a subset of facial muscle actions (AUs) from a set of 42 in the following way. The number of AUs was drawn from a binomial distribution (n = 5, p = .6). Then the identities of AUs were assigned randomly. For each randomly selected AU, the GFG selected random values for each of six temporal parameters from a uniform distribution. These temporal parameters (onset latency, peak latency, offset latency, peak amplitude, acceleration, and deceleration) characterize each AU’s activation curve (see the illustrative color-coded AU curves in Fig. 1). We then applied the random facial animation to one of eight neutral-expression face identities (all White; 4 female, 4 male; mean age = 23.0 years, SD = 4.1 years; we used the same procedure as in Yu et al., 2012; see also the Methodological Details in the Supplemental Material). These animations are illustrated as four snapshots in Figure 1 (see also Video S1 in the Supplemental Material, which demonstrates a random facial animation). Each facial animation comprised 30 image frames, presented at 24 frames per second, for a total duration of 1.25 s.

Procedure

On each experimental trial, observers viewed a randomly generated facial animation and rated the animation for intensity of an indicated trait (dominance, trustworthiness, or attractiveness), using a 5-point scale (1 = very low, 5 = very high). Figure 1 illustrates a dominance trial that elicited a “medium” response.

Each facial animation appeared on a black background displayed on a 17- or 19-in. flat-panel monitor (refresh rate = 60 Hz; image size = 1,280 × 1,024 pixels). Each animation appeared in the central visual field and remained visible until the participant responded. A chin rest ensured a constant viewing distance. Images subtended 14.8° (vertical) by 9.4° (horizontal) of visual angle, which reflects a typical viewing distance for social interaction (Hall, 1966).

Each observer rated facial animations, presented in random order, in three blocks, one per social trait. There were 1,200 animations per block, and the block order was counterbalanced across observers.

Results

For each observer, we reverse-correlated the facial movements (dynamic AUs and temporal parameters) associated with the perception of each social trait and produced separate models for the strongest and weakest perceived intensities (i.e., polarities) of that trait. Thus, for each observer, we created a total of six dynamic social masks (three social traits × two trait polarities), for a total of 72 masks. Each model formed a 42-dimensional binary vector (one dimension per AU). The vector coded AUs that were significantly correlated (p < .05) with the social trait at a given polarity.

Figure 2 depicts the social masks with the strongest and weakest intensities for each trait and list the significant AUs that were combined to produce each mask. The color-coded heat maps show the movement magnitude of the 3-D vertices that make up each dynamic social mask (for illustrations of the dynamic social masks of each trait applied to a common face, see Videos S2–S7 in the Supplemental Material). Note that the models presented in Figure 2 share some AUs. These commonalities between our dynamic masks that convey social traits are in some cases the same as the commonalities between static facial morphologies that convey these social traits (Oosterhof & Todorov, 2008): Trustworthiness and attractiveness, which are highly correlated for static facial morphologies, are also highly correlated in the AUs of our dynamic models (for high intensities of trustworthiness and attractiveness, r = .84, p < .05; for low intensities of trustworthiness and attractiveness, r = .51, p < .05). Dominance and trustworthiness are negatively correlated in static facial morphologies and also in our dynamic models (for high-intensity dominance and low-intensity trustworthiness, r = .60, p < .05). For details of these correlations, see Table S1 in Methodological Details in the Supplemental Material.

Experiment 2

To test the social-camouflage capabilities of the dynamic social masks, we proceeded in two phases. First, we obtained a new set of 3-D static face identities using the same procedure as in Experiment 1 (following the method of Yu et al., 2012). We instructed a new set of observers to rate each according to perceived dominance, trustworthiness, and attractiveness. For these judgments, observers could use only the information conveyed by the morphology and texture of the faces. Second, to test social camouflage, we parametrically applied the dynamic social masks derived in Experiment 1 to the independently rated face identities and asked a second set of observers to rate the intensity of each studied trait in each animation. We describe these two phases in more detail in the next sections.

Social-trait rating of static face identities

Observers

We recruited 12 White observers (6 female, 6 male; mean age = 21.7 years, SD = 2.9 years) using the same criteria and ethical approval as in Experiment 1.

Stimuli

We obtained a new set of 3-D faces with neutral expressions and naturally varying morphologies (25 female and 25 male White models; mean age = 23.3 years, SD = 4.0 years) according to the method of Yu et al. (2012).

Procedure

Observers viewed the stimuli under the same conditions as in Experiment 1. They rated the 50 static face identities according to perceived intensity of dominance, trustworthiness, and attractiveness, using the same 5-point scale. Trials were divided into six blocks according to the gender of the face and social trait to be rated. Each face was presented three times per block, for a total of 75 trials per block and 450 trials across the six blocks.

Validation of the social masks

Observers

We recruited 12 White observers (6 women, 6 men; mean age = 20.5 years, SD = 1.8 years) using the same criteria and ethical approval as in Experiment 1.

Stimuli

For this phase, we derived 11 levels of trait intensity from the dynamic social masks by interpolating the temporal parameters of the models that were computed in Experiment 1. We then produced a facial animation for each combination of static facial identity (25 women, 25 men), level of perceived trait intensity, and social trait. Thus, we created 550 animations per social trait. Each animation comprised 30 frames, presented at 24 frames per second, for a total duration of 1.25 s.

Procedure

Observers viewed the stimuli under the same conditions as in Experiment 1. They rated the trait intensity of each animation on a 5-point scale (1 = very low, 5 = very high). Trials were divided into six blocks according to gender of the face and social trait to be rated. Each animation was presented once. Thus, there were 275 trials per block, for a total of 1,650 trials across the six blocks.

Model of dynamic social camouflaging

Our aim was to understand whether the dynamic social masks modulated the social perception of dominance, trustworthiness, and attractiveness transmitted involuntarily by static facial morphology. To address this question, we computed a model of social-trait perception as follows.

For each social trait and observer, we calculated independent z scores for the observer’s trait ratings across trials. We then averaged the resulting z scores across observers and fitted (on the basis of visual inspection of the scatter plot) a new function of perceived trait intensity described in the following equation:¹

f (x, y) = a + b x + \frac{d}{1 + e^{- c y}},

where x = morphology, y = dynamics, and f(x,y) = perceived trait intensity.

Technically, Equation 1 contains both linear and nonlinear (logistic) components. The linear component models the contribution of default static face morphology to perception; a is the offset in perception at the origin (i.e., mean static facial morphology with no dynamic social masking), and b is the (linear) slope of perceived intensity as a function of static facial morphology only. The logistic component models the effect of dynamic social masking on perception; c represents the slope of the logistic function at the origin, and d represents the asymptotic dynamic range of perceived intensity for any static facial morphology. Note that d quantifies the social-camouflaging capability of each dynamic social mask, and larger values of d reflect greater social camouflaging of the considered trait.

Results

Figure 3 represents, as surfaces, the effect of dynamic camouflaging on the perception of dominance, trustworthiness, and attractiveness. For easier visualization, changes in surface height are also represented as changes in color. Consider the results for dominance. The white dot at the center of the surface represents the locus of socially neutral face morphology with no dynamic social masking (i.e., the origin). Along the surface gradient (white arrows), perception of dominance is modulated (in a sigmoidal manner) by changing the intensity of the dynamic social mask (i.e., very low to very high social camouflage). Note that this logistic-function-shaped perception gradient applies to all static facial morphologies, ranging from highly dominant (+1.5 SD on the static-morphology axis) to highly submissive (−1.5 SD) morphologies. Note also that even the most submissive face (−1.5 SD) is transformed into a dominant face by social camouflaging (black dashed line) and reaches the same level of dominance as the most dominant static facial morphology (black dot on the surface). Figure 3 shows that similar sigmoidal perception surfaces were found for both trustworthiness and attractiveness, with one interesting caveat: For attractiveness, the sigmoidal surface is flatter overall (smaller d parameter); in other words, facial attractiveness is more difficult to mask than are facial dominance and trustworthiness (see Table 1 for parameter values for all three social traits).

Table 1.

Parametric Fit of the Models of Social-Trait Perception Derived in Experiment 2

	Social trait
Parameter	Dominance	Trustworthiness	Attractiveness
a	−0.67 [−0.70, −0.63]	−0.50 [−0.54, −0.47]	−0.46 [−0.50, −0.43]
b	0.41 [0.38, 0.45]	0.34 [0.30, 0.37]	0.39 [0.35, 0.43]
c	3.15 [2.03, 4.27]	3.92 [1.44, 6.40]	2.84 [1.59, 4.10]
d	1.33 [1.28, 1.39]	1.20 [1.15, 1.25]	0.93 [0.87, 0.99]
Adjusted R²	.86	.85	.76

Note: The parameters a through d refer to the terms in Equation 1. The table presents estimated values for these parameters, with 95% confidence intervals, for each social trait.

In summary, Experiment 2 shows that specific facial movements—dynamic social masks—provide effective social camouflage and even override the social signals conveyed by the phenotypic morphology of the face.

Experiment 3

Experiment 2 established that dynamic social masks can provide social camouflage, but an interesting question remains: What is the origin of these social masks? One account suggests that dynamic social masks could be an overgeneralization of dynamic facial expressions of emotions (Engell, Todorov, & Haxby, 2010; Knuston, 1996; Montepare & Dobish, 2003; Oosterhof & Todorov, 2008; Said, Sebe, & Todorov, 2009; Zebrowitz & Montepare, 2008). In Experiment 3, we examined the relationship between the dynamic systems for facial signaling of emotions and social traits. To do so, we compared the AUs of dynamic social and emotional signals using the same reverse-correlation approach used in Experiment 1.

Observers

To model the dynamic facial signals associated with the perception of each of the six classic emotion categories (“happiness,” “surprise,” “fear,” “disgust,” “anger,” and “sadness”; see Jack, Garrod, Yu, Caldara, & Schyns, 2012; Yu et al., 2012), we recruited 63 observers (32 women, 31 men; mean age = 21.6 years, SD = 1.7 years) using the same criteria and ethical approval as in Experiment 1.

Stimuli

We generated random facial animations using the same procedure as in Experiment 1.

Procedure

We used exactly the same procedure as in Experiment 1, except that in this experiment, observers categorized the stimuli according to the six classic emotions—“happiness,” “surprise,” “fear,” “disgust,” “anger,” and “sadness,” plus “don’t know.” Observers also rated the intensity of the perceived emotion in each stimulus using a 5-point scale (1 = very weak, 5 = very strong). Each observer completed 2,400 trials conducted over 12 blocks of 200 trials each (for additional procedural details, see Jack et al., 2012).

Results

To derive the dynamic models for each emotion category, we used a reverse-correlation procedure similar to that in Experiment 1 to created 63 dynamic models for each of the six emotions, for a total of 378 models. To compare the dynamic models of social traits and emotions, we proceeded as follows. First, for each emotion (n = 6) and each social trait polarity (n = 6: 3 traits × 2 polarities), we computed a distinct 42-dimensional proportion vector. Each dimension on the vector coded the degree to which a given AU was present across observers. For example, if AU12 was present in 59 of the 63 happiness models, then the proportion 59/63 would represent this AU in the 42-dimensional proportion vector for happiness. We then cross-correlated (Pearson’s r) the resulting 42-dimensional proportion vectors between each of the six social traits and six emotions. Figure 4 shows the statistical significance of the 36 resulting correlations. As the figure demonstrates, dynamic social masks do not correspond to single emotion signals but are instead composed of AUs found across different emotion categories. For example, signals of high trustworthiness correlate positively with signals of happiness and surprise, and they correlate negatively with signals of disgust and anger.

Comparison of AU patterns for the social-trait and emotion models revealed that the facial movements used to signal a given social trait are associated with several (not just one) emotion categories (for an analogue with a computational model, see Said et al., 2009). In addition, dynamic social masks correlated (positively and negatively) in systematic and in unique manners with the emotion models.

General Discussion

In primates, facial expressions are used as signals to inform group members of transitory emotional states and also to display long-term social intentions and capacities. For example, high-status rhesus monkeys use specific facial movements to signal their rank to subordinates in the dominance hierarchy (de Waal & Luttrell, 1985). We have shown that specific human facial movements, applicable in the context of agonistic and mating rituals, predictably modulate perception of basic social traits. Furthermore, we have shown that the modeled facial movements modulate the default perception of social traits in face morphology. Our results reveal an intriguing source of inequality among humans in a particular social niche. Of the three traits studied, attractiveness is the most influential for reproduction and the most difficult to camouflage. Humans are thus condemned to bear the social consequences of the inherited attractiveness of their faces. By contrast, social camouflage of dominance and trustworthiness is probably commonplace in everyday interactions. Casting directors are probably aware that not all social traits are equal. An attractive character will require an actor with attractive morphology; however, social camouflage can help an actor fake a dominant or trustworthy character.

The current study brings the rigor of parametric psychophysics (typically confined to simple visual stimuli varying on few dimensions) to complex questions in social sciences (typically involving a greater number of dimensions) and offers new methods to derive predictive laws regarding the dynamics of social perception.

Footnotes

Declaration of Conflicting Interests

The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.

Funding

This work was supported by Open Research Area Grant ES/K00607X/1.

Supplemental Material

Additional supporting information may be found at

Notes

References

Ahumada

Lovell

(1971). Stimulus features in signal detection. Journal of the Acoustical Society of America, 49, 1751–1756.

Blair

I. V.

Judd

C. M.

Chapleau

K. M.

(2004). The influence of Afrocentric facial features in criminal sentencing. Psychological Science, 15, 674–679.

de Waal

F. B. M.

Luttrell

(1985). The formal hierarchy of rhesus monkeys: An investigation of the bared teeth display. American Journal of Primatology, 9, 73–85.

Ekman

Friesen

W. V.

(1978). Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.

Engell

A. D.

Todorov

Haxby

J. V.

(2010). Common neural mechanisms for the evaluation of facial trustworthiness and emotional expressions as revealed by behavioral adaptation. Perception, 39, 931–941.

Hall

(1966). The hidden dimension. Garden City, NY: Doubleday.

Jack

R. E.

Garrod

O. G. B.

Caldara

Schyns

P. G.

(2012). Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences, USA, 109, 7241–7244.

Johnson

S. K.

Podratz

K. E.

Dipboye

R. L.

Gibbons

(2010). Physical attractiveness biases in ratings of employment suitability: Tracking down the “beauty is beastly” effect. The Journal of Social Psychology, 150, 301–318.

Knutson

(1996). Facial expressions of emotions influence interpersonal trait inferences. Journal of Nonverbal Behavior, 20, 165–182.

10.

Little

A. C.

Burriss

R. P.

Jones

B. C.

DeBruine

L. M.

Caldwell

C. A.

(2008). Social influence in human face preference: Men and women are influenced more for long-term than short-term attractiveness decisions. Evolution & Human Behavior, 29, 140–146.

11.

Mandel

Betensky

R. A.

(2008). Simultaneous confidence intervals based on the percentile bootstrap approach. Computational Statistics & Data Analysis, 52, 2158–2165.

12.

Montepare

J. M.

Dobish

(2003). The contribution of emotion perceptions and their overgeneralizations to trait impressions. Journal of Nonverbal Behavior, 27, 237–254.

13.

Oosterhof

N. N.

Todorov

(2008). The functional basis of face evaluation. Proceedings of the National Academy of Sciences, USA, 105, 11087–11092.

14.

Said

C. P.

Sebe

Todorov

(2009). Structural resemblance to emotional expressions predicts evaluation of emotionally neutral faces. Emotion, 9, 260–264.

15.

Sutherland

C. A. M.

Oldmeadow

J. A.

Santos

I. M.

Towler

Burt

D. M.

Young

A. W.

(2013). Social inferences from faces: Ambient images generate a three-dimensional model. Cognition, 127, 105–118.

16.

Todorov

Mandisodza

A. N.

Goren

Hall

C. A.

(2005). Inferences of competence from faces predict election outcomes. Science, 308, 1623–1626.

17.

Garrod

O. G. B.

Schyns

P. G.

(2012). Perception-driven facial expression synthesis. Computer & Graphics, 36, 152–162.

18.

Zebrowitz

L. A.

Montepare

J. M.

(2008). Social psychological face perception: Why appearance matters. Social & Personality Psychology Compass, 2, 1497–1517.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.54 MB