Abstract
Facial expressions of emotion have important communicative functions. It is likely that mask-wearing during pandemics disrupts these functions, especially for expressions defined by activity in the lower half of the face. We tested this by asking participants to rate both Duchenne smiles (DSs; defined by the mouth and eyes) and non-Duchenne or “social” smiles (SSs; defined by the mouth alone), within masked and unmasked target faces. As hypothesized, masked SSs were rated much lower in “a pleasant social smile” and much higher in “a merely neutral expression,” compared with unmasked SSs. Essentially, masked SSs became nonsmiles. Masked DSs were still rated as very happy and pleasant, although significantly less so than unmasked DSs. Masked DSs and SSs were both rated as displaying more disgust than the unmasked versions.
An important channel of communication between human beings involves facial emotional expressions, defined by particular contractions and positions of the muscles under the skin of the face. First described by Charles Darwin (1872), facial expressions were thought to be part of evolved bodily systems which serve important adaptive functions for the individual, including expressive, communicative, and persuasive functions. Although there is still debate concerning the innate versus learned status of the production and recognition of facial expressions, most emotion researchers would agree that there is an important biological and involuntary basis to most emotions, which is expressed in the face.
A case in point is the Duchenne smile (DS), first identified by Guillaume Duchenne in 1862 (Duchenne, 1990). DSs are defined by activity in both the mouth (the zygomatic major muscle, turning up the corners of the mouth) and the eyes (the orbicularis oculi muscle, crinkling the eyes and upper cheeks; Ekman & Davidson, 1993). DSs have been conceptualized as “honest signals” of both momentary positive emotion and of chronic positive affect (Sheldon et al., 2021, p. 654), because they are difficult to fake. Typically, DSs only occur in the presence of felt positive emotion, via the subcortical pathway that causes the upper cheeks to crease. Lacking such felt emotion, what is expressed instead is the non-Duchenne or “social smile” (SS), in which communicators raise the corners of their mouths, without involving the eyes. Perceivers can easily tell the differences between DSs and SSs (Krumhuber & Manstead, 2009).
DSs are typically perceived in a positive way (Gunnery & Hall, 2014; Soussignan, 2002), and DSs cause others to trust the smiler, to feel closer to the smiler (Bogodistov & Dost, 2017), and to cooperate with the smiler (Owren & Bachorowski, 2001). Thus, it is often to a communicator’s advantage to smile genuinely while interacting with others, to advance their social goals (Ekman & Davidson, 1993). Smiles can help convince listeners that the communicator cares about them and is friends with them.
However, facial emotions do not always express actually felt emotions; they may also be posed for various social purposes. Such strategic emotional expressions are more cortically driven and are often responsive to display rules and norms within a particular situation or culture (Matsumoto, 1990, 1991). Also, as emotional productions rather than spontaneous emanations, strategic emotion displays may not always be convincing and effective. Communicators may sometimes fail to convince observers they are in a particular emotional state, if communicators do not actually feel the emotion being displayed. One example concerns polite SSs, which do not convey strong positive affect from the smiler to the perceiver.
Although the meaning of a DS is relatively straightforward, signaling felt positive emotion, the meaning of an SS is much more ambiguous (Prkachin & Silverman, 2002), because SSs have many potential functions (Niedenthal et al., 2010). In addition to perhaps being a failed attempt to express happiness, an SS can also represent an attempt to convey politeness, acquiescence, or harmlessness (Papa & Bonanno, 2008), or to meet social demands (Ekman & Friesen, 1982), or to achieve social power (Hecht & LaFrance, 1998). Moreover, SSs can also be used to hide the smiler’s true feelings, feelings such as aggression or disgust (Morse & Afifi, 2015), embarrassment (Kraut & Johnston, 1979), or uncertainty (Labarre, 1947). Thus, from a feelings-as-information perspective (Schwarz, 2011), SSs are much more difficult to interpret than DSs.
Mask Effects
Masking may make this problem worse. During the COVID-19 pandemic, people around the world have become accustomed to wearing face masks in the presence of strangers. Of course, the purpose of masks is to prevent the emission and intake of potentially virus-laden respiratory droplets. However, masks may also have important unintended effects, given that they partially obscure the wearer’s facial muscle activity. This suggests that masks should make it more difficult to read others’ facial expressions. This should be especially true for facial expressions that are defined by the lower half of the face, which is most obscured by masking.
Recent literature looking into the effects of face masking on emotion recognition of facial expressions gives strong support for the contention that face masking impairs recognition of several facial expressions (Calbi et al., 2021; Carbon, 2020; Gori et al., 2021; Ruba & Pollak, 2020). However, these studies have used facial expression databases that contain images of “happy” individuals as a single category among other expressions such as fear, anger, sadness, and so on. These images of happy individuals generally display activation of the muscles surrounding the eyes, thus expressing true DSs. No previous mask research specifically tested how perceptions of DSs and SSs might differ with face masking.
We tested these ideas by comparing observers’ perceptions of both DSs and SSs, presented first in a masked and then in an unmasked format. We assembled 16 photos in which four different people (two males and two females, all Caucasian) displayed either a DS or an SS, either with the full face or with a black mask photo-shopped over the lower half of the face. We asked our participants to rate four emotions within each of the 16 target photos (64 ratings in all): (a) a genuinely happy smile, (b) a pleasant social smile, (c) disgust, and (d) a merely neutral expression. The first two emotions were chosen to conform to the theoretical definitions of DS and non-DS smiles, allowing us to evaluate whether masks affect peoples’ accuracy in identifying the two types of smiles. The neutral emotion allowed us to evaluate whether masks strip emotional expressions of their emotionality. The disgust emotion allowed us to evaluate (in one way) whether masks add a negative tone to emotion perceptions and also represent a second emotion with eye activation.
Our primary study hypothesis was that perceptions of SSs would be more strongly affected by masking, compared with perceptions of DSs. This is because masking obscures the only indicator of the SS; the upturning of the corners of the mouth via contraction of the zygomatic major muscles. Specifically, we expected that the presence of a mask would greatly suppress the perception of “pleasant social smiling” in the SSs, compared with DSs. By similar reasoning, we also expected that masking would increase the perception of “facial neutrality” in SSs, compared with DSs. Finding support for these two interaction hypotheses would imply that social smilers may be much less effective in their communicative goals, when their faces are masked. This may have important negative implications for communication, negotiation, and relationship functioning.
What about perceptions of genuine happiness in the smiler? We hypothesized a main effect such that DSs are rated as happier than SSs, because the orbicularis oculi muscle, the main signaler of felt positive emotion, is active and visible within both masked and unmasked DSs. We also hypothesized a main effect such that masked images would be perceived as less happy than unmasked images, because the masked images are missing zygomatic major information. We ventured no predictions about interactions between the two factors.
We also made no specific predictions for the fourth perceived emotion, disgust. We included disgust because it is a second expression in which the top half of the face is active (nose and brows), as with the DS. And, as mentioned earlier, it also allows evaluation of whether masks add negative tone to facial perceptions. For similar reasons as with the DS, that the main information for the emotion remains visible despite masking, we did not expect perceived disgust to be strongly affected by the presence versus absence of a mask.
Methods
Participants
The data were collected late in the fall of 2020, from 282 psychology students at the University of Missouri, who took an online survey near the end of the semester, in exchange for research or extra credit. Participants were 75.6% Caucasian, 11% African American, 5.4% Asian American, 4.2% Hispanic, and 3.8% of individuals responded “other,” with a median age of 20 years. We included all students who supplied complete data in the survey, and used no stop rules while collecting the data, beyond the arrival of the end of the semester.
Procedure
As an introduction to the task, participants read: “In this task, you will be shown a set of faces. We’ll ask you to make four different ratings of each face. Please go by your first impression of each face.” Then they were shown the eight masked faces in a random order, making four ratings of each face. We presented the masked faces first because if the unmasked faces were presented first, participants might later remember the missing parts of the expressions, when rating the masked faces. After rating the eight masked faces, participants then rated the eight unmasked faces, also in random order.
Stimuli
Our eight source images came from a picture set originally created by Manera et al. (2011), for a study of individual differences in the ability to recognize DSs versus SSs. We drew from the subset of these images used by Bogodistov and Dost (2017), in their study of the effects of DSs upon felt psychological distance. As noted earlier, our source pictures were of two young men and two young women, all Caucasian, each photographed twice: either while displaying a DS (with activity in both the orbicularis oculi muscle, at the eyes, and the zygomatic major muscle, at the mouth) or while displaying an SS (with activity only in the zygomatic major muscle). The pictures were grayscale and were approximately 100 kilobytes in size (see Bogodistov & Dost, 2017, for examples of these images). We created eight further images by photo-shopping a completely opaque black covering around the lower contours of each face, reaching up to cover the bottom of the nose.
Measures
The emotion ratings were prefaced with the question, “To what extent is this person displaying the following emotions?” As noted earlier, the four emotions were “a genuinely happy smile,” “a pleasant social smile,” “a look of disgust,” and “a merely neutral expression.” A 1 (not at all) to 7 (very much) scale was provided.
We computed 16 composite variables by averaging across the four faces, for each within-subject condition. For example, the ratings of “genuinely happy smile” for the four masked faces displaying DSs were averaged into a single variable, as were the four ratings of social smile, of disgust, and of a neutral expression. Then, the same was done for the unmasked faces displaying DSs. Then, the process was repeated for the faces displaying masked and unmasked SSs. Alpha reliability coefficients ranged between .70 and .87 for these 16 variables, indicating that participants’ ratings converged well across the four human exemplars of each expression.
Results
Table 1 contains the means and standard deviations for the 16 emotion variables. To test our hypotheses, we conducted four different mixed effects general linear models estimated with maximum likelihood (one for each of the four emotion variables) using SPSS 25 and the “MIXED” command. We first ran unconditional models to assess fit and intraclass correlation coefficient (ICC). Then, we ran the conditional models with smile-type (DS. vs. SS), appearance (masked vs. unmasked), and the interaction of smile-type and appearance as level 1 fixed factors, and participants’ mean score across all images on the respective target emotion as a random factor (e.g., mean happiness ratings for each individual participant were set as a random factor when the model specified happiness as the dependent variable). This had the effect of removing mean differences associated with participant-raters, such that only within-subject variability in intercepts remained. In all models, smile-type and appearance were both dummy-coded to have DSs and unmasked appearances as the respective reference groups.
Means and standard deviations for the perceived emotion variables.
Note. DS = Duchenne smile displayed; SS = non-Duchenne smile displayed.
Hypothesis Tests
Pleasant Social Smile
We ran the unconditional model and calculated an ICC of .86, indicating that 86% of the total variance in social smiles ratings is associated between-participant social smile ratings. We then ran the full conditional model. The conditional model exhibited a better fit (Akaike information criterion [AIC] = 3,804.34) than the unconditional model (AIC = 5,361.33). As hypothesized, there was a significant interaction between the two factors, F(1, 849.58) = 333.83, p < .001, based on the rated pleasantness of the SS dropping from 4.95 in the unmasked condition to 2.44 in the masked condition, while the pleasantness of the DS was unchanged in the masked condition, estimate = −2.57 (.14), 95% confidence interval [CI] [−2.85, −2.30]. There was also main effects of smile-type, F(1, 848.26) = 304.82, p < .001, with DS being rated higher than SS, estimate = 2.51 (.10), 95% CI [2.32, 2.71], and of appearance, F(1, 849.58) = 299.81, p < .001, with unmasked appearances being rated higher than masked appearances, estimate = 2.51 (.10), 95% CI [2.31, 2.70].
Neutral Expression
We first ran the unconditional model and calculated an ICC of .68, indicating that 68% of the total variance in neutral expression ratings is associated between-participant neutral expression ratings. We then ran the full conditional model. The conditional model exhibited a better fit (AIC = 3,577.86) than the unconditional model (AIC = 5,288.35). As hypothesized, there was a significant interaction between the two factors, F(1, 846.83) = 269.46, p < .001, based on the rated neutrality of the SS increasing from 2.30 in the unmasked condition to 5.10 in the masked condition, while the rated neutrality of the DS only increased from 1.54 to 2.25, estimate = 2.07 (.13), 95% CI [1.82, 2.341]. In this analysis there were highly significant main effects of both the smile-type factor, F(1, 846.83) = 819.23, p < .001, and the masked versus unmasked factor, F(1, 849.38) = 775.12, p < .001, in that both masked, estimate = −2.79 (.09), 95% CI [−2.97, −2.62], and social smile, estimate = −2.84 (.09), 95% CI [−3.01, −2.66], faces were viewed as more neutral.
Genuine Happiness
We first ran the unconditional model and calculated an ICC of .78, indicating that 78% of the total variance in genuine happiness ratings is associated between-participant genuine happiness ratings. We then ran the full conditional model. The conditional model exhibited a better fit (AIC = 3,504.98) than the unconditional model (AIC = 5,601.58). There was no interaction between the two experimental factors, F(1, 848.18) = 0.61, p = .437, in the analysis. As expected, there was a significant main effect of smile-type, F(1, 850.52) = 2,405.97, p < .001, with DSs being perceived as significantly more happy than SSs, estimate = 2.87 (.08), 95% CI [2.70, 3.03]. There was also a significant main effect of appearance, F(1, 850.53) = 297.64, p < .001, with unmasked appearances being perceived as significantly more happy than masked appearances, estimate = 0.98 (.08), 95% CI [0.81, 1.14]. Notably, rated happiness of the masked DSs was still far above the midpoint of 3.5 on the 7-point scale (M = 4.77), suggesting that “the glow still shows” even when a DS is masked.
Disgust
We first ran the unconditional model and calculated an ICC of .85, indicating that 85% of the total variance in disgust ratings is associated between-participant disgust ratings. We then ran the full conditional model. The conditional model exhibited a better fit (AIC = 2,691.76) than the unconditional model (AIC = 3,566.83). There was a main effect of the masked versus unmasked condition, F(1,844.95) = 98.76, p < .001, such that rated disgust was higher in both masked conditions, estimate = −0.60 (.05), 95% CI [−0.71, −0.50]. However, there was also a significant interaction, F(1,844.20) = 34.65, p < .001, such that the increase was larger in the SS condition (+ .61) than in the DS condition (+ .16) in the masked condition, estimate = 0.45 (.08), 95% CI [0.30, 0.60]. In addition, there was a main effect of smile-type, F(1, 844.20) = 331.07, p < .001, such that perceived disgust was on average greater in the SS conditions, estimate = −0.92 (.05), 95% CI [−1.02, −0.81].
Discussion
This research asked a very timely question: How do facemasks affect our ability to communicate emotions to others? We focused on the distinction between DSs and SSs because smiles are a very important type of nonverbal communication, and because DSs and SSs are neatly distinguished vis-à-vis mask-wearing, given that masks obscure only the lower portion of the face, which is the only active part of the face in an SS.
We found a clear pattern of effects in support of our hypotheses. Masking strongly undermines peoples’ perceptions of pleasantness regarding SSs and also causes SSs to become much more neutral in appearance. Masking also reduces the already low-level of happiness perceived in SSs and increases the level of disgust perceived in SSs. In contrast, masked DSs were still effective at signaling strong positive emotion, remaining well above the midpoint of the scale. Masking also made DSs appear slightly more disgusted and slightly more neutral and did not affect pleasantness ratings of DSs.
The social smile is a widely used tool in human social life, helping to grease the rails of communication and reduce conflict (Niedenthal et al., 2010). These findings suggest that masks limit the effectiveness of this tool, turning our SSs into merely neutral faces, or even making us look disgusted. It would doubtless be helpful for people to display more DSs in compensation, but unfortunately, DSs are largely nonamenable to conscious control (Sheldon et al., 2021) and may be especially difficult to draw forth given the stresses and restricted living conditions imposed by the pandemic. Our findings also suggest that transparent masks may perhaps provide an effective remedy. In support of this idea, Marini et al. (2021) found that participant’s recognition of emotional expressions did not differ significantly between images of nonmasked individuals and individuals wearing transparent masks.
This study has several limitations. First, this was a predominantly Caucasian sample, taken from a single university in the U.S. Midwest. Also, the target faces were all Caucasian. Although there is reason to expect that results would generalize to other samples and other target faces, of different cultures and ethnicities, this remains to be established. Second, we only asked for four emotion ratings, corresponding to the two types of smile, a neutral expression, and a disgusted expression. It would be useful to conduct a more thorough and differentiated assessment of participants’ emotion perceptions. Third, we only used static facial images as targets: There is emerging consensus in the emotion literature that there is much value in studying temporal segments of expression, rather than single moments of expression (Barrett et al., 2019). Despite these limitations, we hope that our data make a useful contribution to the literature and will spark further research.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported within the framework of a subsidy by the Russian Academic Excellence Project “5–100.”
