Tracking a leader’s humility and its emotions from body,face and voice

1. Introduction

The field of persuasive computing generally tries to understand how to design emotional and dominant technologies with the aim of influencing people’s attitudes and beliefs [11]. In order to this, technologies are generally designed to be competent and flawless, thus reducing the potential active role of the User, and finally possibly “imposing” a given decision instead of favouring free deliberation. But what if, on the contrary, the User could interact with a “humble” machine? Could a “softer” influence, not stressing the qualities of the Source, leave the Target the chance of a more sensible reflection and more mature decision?

In view of testing the different possible features and effects of a Dominant vs. a Humble source of influence, this study aims at proposing a model of humble communication, at singling out the characterizing features of its multimodal communication [19], and at providing tools for its automatic detection and hints for its simulation in persuasive machines.

To investigate such issue, this paper focuses on the field of political communication and specifically on the verbal and body behaviors of humble leaders. Studies in the psychology of political communication have stressed the role of dominance in the self-presentation of politicians, while implicitly excluding the very hypothesis that a political leader can be or present himself as humble.

Yet, in some cases – for instance, in particular historical situations, ideological milieus, or for people with particular personality traits [3] – a humble leadership may be more effective than utterly dominant behavior, thus making it a worth endeavor not only its in depth analysis but also its possible implementation in affective or persuasive devices.

Section 2 overviews previous work on humble leadership, Section 3 presents a definition of humility in terms or a socio-cognitive model of leadership, social influence and multimodal communication, drawn from an empirical study on laypeople’s view of humility and humble leaders; Section 4 analyses the multimodal communication or four humble leaders, focusing on their facial expression, body multimodal behavior, and prosodic features, ordered in Section 5 where it is explained a first tentative list of humble leaders’ body cues. Finally Section 6 presents a third study on humble leaders’ emotions extracted by an automatic detection method.

2. Humility in social psychology

The concept of humility, so widely promoted in religious literature, has rarely been approached and coherently described in psychological research, in favour of the opposite notion of dominance as a common expression of a person’s power [21].

Tangney [27] defines the psychological notion of humility as a state/disposition to “forget the self”, that makes reference to a highest level of self-awareness, in which the humble person is highly aware of her strengths and limits. In this sense, the psychological notion of humility, unlike the common-sense one [10], is not a negative feature simply linked to an under-estimation of one’s own personal value.

In the psychology of organization humble leadership presupposes 1) acknowledgement of personal faults, mistakes and limits, 2) openness to new, even contradictory ideas, 3) the tendency to give voice and merits to “employees” [17].

A humble leadership, as results from studies in organizational psychology [16,17], tends to have positive effects on employees when they have a positive evaluation of the leader, for example thanks to the humble leader’s “voice behaviors (i.e. proactive and constructive suggestions)”. On the other hand, in political communication it can be amisinterpreted [4,6] and favour negative effects.

This work aims at defining the notion of “humble leadership” in the political field and at describing what characterizes the multimodal communication of “humble” political leaders. To this goal, first a survey has been conducted to explore the common-sense view of “humility” and of “humble leader”, as well as the assessment of humble political leaders (Study 1); then a qualitative analysis of the “humble” leaders’ communication was carried on, finding out the characterizing features of humble leadership in their discourse and their body communication (Study 2). Finally, an extensive analysis was carried out of videos of humble political leaders, and their emotions were extracted by means of face and voice detection tools (Study 3).

3. Study 1. A survey to define humility and humble communication

To explore the notions of humility and of humble leader, two approaches can be applied. On the one hand, a conceptual analysis can be drawn, in a top-down fashion, from a general model of social relations between humans; on the other hand, the common-sense definition of these notions can be investigated through empirical research, trying to discover how people in real life classify others as humble, and why they do so. This latter approach was first taken in our research, in order to finally come up with an empirically grounded conceptual definition of humility and humble leaders.

To find out how people define humility and how they describe “humble leaders”, a survey was conducted on 62 participants (balanced for gender; 52% women, mean age 24.1). 30 questions, phrased on the basis of a previous focus group, asked participants: to describe a hypothetically humble behaviour, of both themselves and others; to provide a definition of humility; to name possible humble political leaders; and to find videos on Youtube witnessing their humble communicative behavior.

Here we only overview the participants’ definition of humility, then starting with their choice of political leaders as representative of humble communication, and with their respective Youtube videos, we finally focus on their multimodal analysis.

According to our participants, humility is characterized by the following features:

first, it is

a stance (that is, a way of being and behaving with others [13] of H, the humble person), which entails:

This definition of humility from a cognitive perspective shows how the main goal of the humble person H is to be “like others”, not more nor less; and to pursue this goal H does not show nor make appeal to one’s own power, superiority in terms of status, knowledge, merits, contributions, virtues and capabilities. From a communicative point of view, H considers important “not to put oneself first” but rather attributes a positive value to a more large dimension of belonging (i.e., others, group, organizations, party) and focuses on the problem rather than on the person who did something. Such horizontal perspective of the humble person, focused on others and on the group, leads to emphasize elements of similarity, familiarity and informality.

4. Humility inside and outside

Similarly to a previous definition of charisma [7,20], the property of humility of a person can be viewed as a set of internal features of a person that is manifested by a corresponding set of external cues: some traits or behaviours that cause other people to view that person as humble.

So far, the conceptual analysis of humility resulting from our survey only gives us information on what are the internal features of a humble person; which though, just because internal, need to be displayed, manifested externally, in order to be sensed, perceived by other people. So our next research question is: what are the external cues to humility? What are the verbal and body traits and behaviours that make us see a person as endowed with a humble stance? What are the cues to empathy, familiarity, realism, altruism, equality? Could we generally find a correspondence between a specific behaviour and a specific humility feature?

4.1. Study 2. The multimodal communication of humble leaders

This issue was addressed by our survey by asking participants to name some examples of humble leaders, with the aim of analysing their multimodal communication to find out the characterizing aspects of humble politicians. Actually, almost 17% of our participants stated they could not choose a politician when talking of humility because they thought this is self-contradictory. 17 participants (39%) indicated foreign politicians: Obama (11.25%), Mujica (3.7%) and three others (Harold V, the Norwegian King; the French leader Melenchon, and Martin Luther King); 34% of participants chose Italian politicians from the past like Aldo Moro (7%), Enrico Berlinguer (7%), Marco Pannella (5%), and Veltroni; only 27% named current Italian politicians chosen from Beppe Grillo’s anti-establishment movement “Five Stars Movement” (Virginia Raggi, Di Battista, Di Maio, 21% in total); finally, former prime ministers and presidents were named: Enrico Letta, Sergio Mattarella, Giorgio Napolitano, Sandro Pertini. In order to perform the multimodal analysis of verbal and bodily signals the most frequently named were chosen: Obama and Mujica as current leaders, and Berlinguer and Moro as past time politicians.

For each of them we selected a video from a real TV program: three interviews for the former three, and the “farewell speech” of Obama after his second presidential term; then each video was subject to a qualitative analysis of the Leader’s multimodal communication. Our goal in analysing humble communication was to find out if and how the features of humility mentioned by our participants are expressed by the multimodal communication of humble politicians, and in particular of the four leaders considered as most representative of humble leadership.

Therefore we examined the fragments of their communication in terms of two types of analysis: one especially suited for full body multimodal behaviour, and one focused on facial expression. The multimodal body communication of the four videos of humble leaders above was first analysed in terms of the annotation scheme of Table 1. Two independent judges separately analysed the same videos; then at the end of the annotation the two judges discussed about critical passages by reaching a good agreement.

In the scheme, the timeline of the behaviours taken into exam is represented vertically: at each second the behaviour or trait under analysis is analysed across three rows. For each group of three rows, the first one contains a description of the analysed signal in terms of the parameters and values of Table 2. In the second row we write the literal meaning we attribute to that signal, and possibly, preceded by an arrow, its indirect meaning [19], since, according to the principles adopted in previous analyses [5,25], each signal may have, beyond its literal meaning, a further meaning to be inferred by the Addressee.

OPEN IN VIEWER

Table 2

Description of parameters

Modality	Parameter	Values
Voice	Voice	High, medium, low
Body	Trunk position	Erected, close, distancing
	Body movement	Forward, backward
Head	Movement direction	Default, nod, shake, canting, toss to interlocutor, forward, backward, leftward, rightward
Gaze	Direction	Oblique, gazing down, gazing upward, downward, upward
	Eyebrows	BROW LOWERED AU 4, BROW RAISE AU 2, INNER BROW RAISE AU 1, LID TIGHTEN AU 23
	Eyelids	Default, raising, frown, asymmetrical, AU 43
Mouth	Lips	Default, half open, closed, LIP CORNER DEPRESSOR AU 15, LIP SUCK AU 28, LIPS PRESS AU 24, LIF STRETCH AU 20, UPPER LIP RAISE AU 5, SMILE AU 12,
	Lip comers	Default, upward, downward, retracted, MOUTH STRETCH AU 27, SMILE AU 12, DIMPLER AU 14, CHIN RAISE AU 17, SMIRK
Nose	Nose	Nose wrinkle, cheek raise – AU 9, 6
Clothing		Simple, formal, informal, sophisticated

In the third row we write the feature that forms the core of the meaning conveyed: a mental ingredient that may or may not be included within the “humility” features listed above. The signals are distinguished according to their productive modality, that is, the body organs by which they are produced, represented in the columns [19]. So, while column 1 specifies the time in the video and the Sender of the signal under analysis, the subsequent columns contain the modalities taken into account: Verbal (Col. 2), Voice (3), Body (4), Head (5) Gaze (6), Mouth (7), Nose (8), Gesture (9), and Clothing (10).

Within each modality, Table 1 distinguishes some specific parameters: e.g., for the body (col. 4) we take trunk position and body movement into account, for head (5), movement and direction, and so on (see Table 2). This is because sometimes a body organ, or even some aspect of its behaviour or traits (like position, direction, movement), may by itself convey a single piece of meaning that when combined with other behaviours or aspects of behaviour makes up a complex message; and even, sometimes the meaning conveyed by one part or aspect of the signal relevantly combines with an aspect of another. For example (see Table 1), in the multimodal communication by Uruguay president José Alberto Mujica a global meaning resulting in the features of CARE, WORRY FOR OTHERS is conveyed, at time 2.30, by body movement (Mujica moves his trunk forward and backward, swaying), head position (head canting) and head movement (he turns toward Interlocutor), and gaze direction (he stares at Interlocutor fixedly from down upward).

Beside analysing body and head movements, all along the fragments from the humble leaders’ faces we manually annotated the AUs (action units; [9]) by using the annotation scheme described below. Further, we detected their facial expressions by means of I-Motion, the automatic detection system based on Affectiva’s Affdex technology to capture emotional reactions via facial expressions.

The two analyses in terms of full-body multimodality and of facial expression allows us to describe political leaders’ humble communication in detail.

4.1.1. Enrico Berlinguer

Among the Italian politicians of the past mentioned by participants as humble leaders and analysed in our study, we start from Enrico Berlinguer, the secretary of the Italian Communist Party from 1972 to 1984. He is known for having raised the moral question regarding power management on the part of the Italian political parties. Berlinguer is still a very popular politician among Italian leftists, he was respected by opponents and loved by his party militants, so much so that at his funeral in Rome more than one million people were present.

In an interview on April 23th, 1983, http://www.la-storiasiamonoi.rai.it/video/faccia-a-faccia-con-enrico-berlinguer/1575/default.aspx, Berlinguer says at minute 3.18:

(Non posso definire i berlingueriani, perché nego che esiste una categoria di berlingueriani)

(I cannot define “Berlinguerian”, because I deny that there is such a category as the “Berlinguerians”)

By his low tone of voice and a very slow speech rate, Berlinguer diminishes the fact that there might exist a category with his name; an element of humble communication in this case is the verbal act of underestimating and belittling one’s own merits, by attributing them to a larger category (the entire party, the community) [2]; in this fragment Berlinguer displays a facial expression of slight embarrassment (eyes slightly averted from the Interlocutor) and modesty (coyness AU 6, 7, 12, 25, 26, small smile, head slightly turned down: [14]). See Fig. 1.

OPEN IN VIEWER

Fig. 1.

Berlinguer.

At minute 4.48 Berlinguer says:

Esiste un’evoluzione perché le situazioni cambiano e la politica di partito si adegua. L’importante è che tutti i cambiamenti siano adeguati alla situazione e che siano condotti in modo tale che tutti comprendano la necessità e l’opportunità).

(There is an evolution because situations change and the party’s politics adapts to them. The important thing is for all changes to be appropriate to the situation and conducted in such a way that everyone understands the need and the opportunity of doing so).

The multimodal analysis of the fragment shows that Berlinguer quietly explains how a party’s policy addresses change in such a way as to let everyone understand the need thereof, and at the same time he gesturally emphasizes the word “tutti” (all) by opening his arms wide to point out how important it is to achieve an ecumenical level of attention.

As to facial expression, in this fragment (Fig. 2) Berlinguer displays an almost sad, but serious gaze (AU 9 + 1 + 4 ; nose wrinkle, inner brow raised), staring out into space, a cue to cognitive effort and attention to the words he is choosing. This conveys a meaning of attention and care towards the Interlocutor (he searches the right words because the other person is important for him).

OPEN IN VIEWER

Fig. 2.

Berlinguer.

Later, Berlinguer moves forward towards the Interviewer, and starting from a position of head canting, he makes nods to approve and integrate the journalist’s perspective [24]; he looks at him straight in his eyes, conveying that he is ”going toward” the interlocutor and trying to get to a common point, a sign of conciliation and mediation of viewpoints. See Fig. 3.

OPEN IN VIEWER

Fig. 3.

(a), (b) Berlinguer.

4.1.2. Aldo Moro

Aldo Moro was a politician, a jurist and a university professor during the sixties until 1978, when he was kidnapped and killed by the “Red Brigades”, an extreme left group of terrorists; he was five times the Prime Minister and the chairman of the Christian Democrats, and he was very popular also because he promoted a conciliatory policy towards the Italian Communist Party.

At minute 4.07. Moro says:

“Non mi pento di aver trovato naturale un incontro con tutti i parlamentari.”

(I do not regret to have found a natural accordance with all parliament member)

The multimodal analysis evidences Moro’s flat pitch and very slow rhythm, both conveying on the one hand the need to be understood and on the other the need to choose the right words to use, given the importance of the other’s point of view; attention to the other is marked by the cognitive effort and seriousness expressed by the facial expression of sadness/concern (knitted brows, eyes slightly tighten, lip corner depressed, AU 1, 4, 6, 15) and by the gaze downward, raised to the Interlocutor only in case of emphasis. At the same time, Moro’s hands bent before chin, and swaying toward the interlocutor, express a state of concentration and sometimes the resolution of an internal reasoning finally communicated externally. See Fig. 4.

OPEN IN VIEWER

Fig. 4.

Moro.

4.1.3. José Alberto Mujica

The first fragment is from President of Uruguay, José Alberto Mujica, who by his very life style has demonstrated a humble way to be president. He lives on a small farm in Rincón del Cerro, the suburbs of Montevideo, and also during his tenure he refused to live in the presidential palace. In reference to the small amount of salary he kept for himself (about 800 EUR), which made him deserve the nickname ”the poorest President in the world”, Mujica said in an interview to the Colombian newspaper El Tiempo that this amount of money was sufficient to him, given that many farmers in his country live on less than this.

In 2015, Josè Mujica recorded an interview For “Human”, a collection of stories to answer the film director Yann Arthus-Bertrand’s question on “what means to be human”.

https://www.youtube.com/watch?v=4GX6a2WEA1Q

At minute 2.14 he says

Esto no es una apologìa de la povresa, es una apologìa de la sobriedad.

“This is not an apology of poverty but one of sobriety”

The multimodal analysis of this fragment, performed by means of the annotation scheme (Table 1), describes Mujica, dressed with a simple shirt (col. 10) – a signal of informality – leaning forward as if to communicate a desire to get closer, to create familiarity with the interlocutor (col. 4); his head (col. 5) lowered and directed straight to the camera but, like gaze direction, from down upward, thus positioning him in a down-up position, conveys the meaning that he does not judge or command over the potential listener (non-superiority). His gaze (col. 6), by raised and frowning eyebrows (AU 1, 4, 15) expresses worry, hence concern about the interlocutor’s future (empathy), while nose wrinkles (col. 8) and AU 7, lid tightener, AU 23, lip tightener – the Action Units of anger [16] communicate the importance of the subject by raising attention (Fig. 5).

OPEN IN VIEWER

Fig. 5.

Mujica.

OPEN IN VIEWER

Fig. 6.

Mujica.

At 2.39 Mujica says:

Inventamos una montaña de consumo superfluo. …Y lo que estamos comprando es el tiempo de la vida! Porqué quando yo compro algo, o tu, no lo compra con plata, lo compra con el tiempo de vida que use para otener ese plata. Pero con esta diferencia: la unica cosa che no se puede comprar es la vida.

(We invent a mountain of unnecessary consumption. …What we’re buying is the time of life! Because when I or you buy something, you do not buy it with money, but with the life time it takes to earn that money. But with a difference: the only thing you cannot buy is life).

In this fragment Mujica’s speech has a simple phrase structure, he suggests and gives advice with a slow rhythm and a low intensity of voice, whispering phrases as if talking to a family member, thus expressing familiarity. But he emphasizes “la vida” (life) with louder voice and also looking directly toward the interlocutor while lowering his head and with a facial expression of anger (frown, raised eyebrows, nose wrinkles), which is also another way to display empathy for the possible negative consequences of not being sober. See Fig. 6.

4.1.4. Barack Obama

The most quoted politician in our survey as “humble” politician was Barack Obama, and the speech most frequently chosen by participant was the “farewell speech” issued on January 10th, 2017.

(https://www.youtube.com/watch?v=XcCrH6rWG-0&t=95s). Obama is cited as humble mostly for the moment in which, while thanking his wife, he is blatantly moved: in so doing he complies with two rules of humble communication. The first is “do not show your own power, merits, or successes”, but rather attribute them to another person through thanksgiving. The second is linked to the idea that to express one’s emotions, but also to be seen while trying to retain emotional expression describes a politician in all his humanness. Here Obama expresses emotions as if he were in his family and, therefore, he does not hold back his tears.

At 3:39 Obama names his wife: “Michelle Obama, southern girl (…)”, he looks down at her, stops talking, trying to hold back the emotion and then swallows up tears, he shakes his head and bites his lips.

Then he goes on (3:51): ”over the past 25 years you have been not only my wife or the mother of my children you have been my best friend” and again presses his lips to create the one hand dimpler (dimpler + lip corner depressor AU 15 + lip press AU 24 and lip suck AU 28), he tries to speak again but lowers his head and raises his eyes bright. See Fig. 7.

OPEN IN VIEWER

Fig. 7.

Obama.

So far we have described in detail the analysis of humble communication in four specific leaders. But to be able to detect a humble stance in leaders – or even in people – on a general basis, a further step of investigation would be to extract some general cues to the internal features of humility. To find out systematic correspondences between all humility features and their respective bodily displays is of course a long and tricky endeavour; yet on the basis of our analysis we can propose at least some hypotheses about this: let us overview some, focusing on the expressive and communicative behaviour of a leader’s head, hands, and trunk.

From our analysis of the body behaviours in the fragments analyzed, we can draw a tentative list of correspondences between head, hands, and trunk movements and positions with the features of humility or, to the contrary, with opposite features of superiority or distance (see Table 3).

a. Head

As we have seen in Section 4.1.1, a relevant cue to humility is head position and movement; and the importance of such features of head, and consequently of chin, as a clear display of a person’s dominant or submissive stance are witnessed by previous literature. For instance, head and chin upward are a typical cue to the emotion of pride [29], hence to a boasting or dominant person [21,23]. On the other hand, head leaning aside – head canting – for its ethological origins [8] typically reveals a non-aggressive, when not submissive, attitude [1]. Therefore a bent head generally expresses a submissive, but sometimes possibly a humble attitude, as opposed to backward tilt that conveys pride or arrogance. But when your head is leaning not downward but forward, this – as we will see for trunk – sometimes means a need for help from the other (a request), but other times a caring attitude towards the other. The head canting that we have seen in Berlinguer displays a listening attitude and good disposition to the other. In other only seemingly opposite cases, it conveys incredulity, scepticism, disproval; in this case it is an ironic way of listening and conveys the opposite just thanks to the inversion of meaning of irony.

b. Chin

Of course, chin moves jointly with head; but its movements or positions may underline its meanings. Chin downward on chest is a cue to sadness or shame, but sometimes to humility, whereas chin upward (like in Mussolini’s typical pose, [20]) makes part of the body signals of pride and imperiousness. But chin forward again signals request or good disposition toward the other.

c. Trunk

For trunk, the most typical position of humility is forward: it is a cue to a listening attitude when the other is speaking, as displayed by Berlinguer while listening to the interviewer’s questions; but it conveys care and even worry for the audience, and even a sense of urgence to change, in Mujica. What makes a difference in the two cases is the tension of the two positions: relaxed the former, tense the latter. The trunk positions that are, to the contrary, clear cues to arrogance, are the strutting trunk, typical of a sense of superiority, that is straight and tense, and the position taken as one is sitting slouched, where trunk is backward but relaxed: in this case the ostentatious relaxation conveys carelessness about the other, thus meaning: I am not at all concerned about you, I am superior to you [21].

d. Shoulders

As for shoulders, a backward position is generally a cue to arrogance or distance, keeping them horizontal shows self-confidence, when closed and bent forward they are quite typical of humility, while what we may call “bottle shoulders” express sadness or depression [28], but may also be a cue to humility.

e. Gesture amplitude

Among the parameters of gesture expressivity [12], amplitude may be a reliable cue to humility: while an assertive person tends to invade the other’s territory with ample gestures, gestures within shoulders width reveal the feature of leaving the other free and not imposing one’s will: a feature characterizing humble persons.

To the multimodal analysis of the humble politicians we added a third observation specifically based on an extensive detection of emotions extracted by face and voice.

For the face we used I-Motion, which is based on a facial expression algorithm that automatically identifies 6 Basic Emotions, Valence, Excitement, 15 Facial Expressions, 33 Facial Landmarks, Interocular Distance, and Head Pose (www.imotion.com). The 23 Action Units – and the six basic emotions – based on Ekman’s studies [9] were automatically detected, by extracting the relative frequencies. These data were obtained by means of an internal I-Motions procedure named ‘post-processing’, that counts every AU with percentage above 50% of probability of being that AU. Three additional parameters were also automatically detected by I-Motion: attention (head orientation toward screen), valence of emotions (positive, vs. negative, AU 4, 9, 10) and engagement (maximum intensity of all actions).

To carry out the detection of emotions from voice we used OpenVokaturi, an open-source version of the Vokaturi library distributed under the General Public License (GPL). This library provides the analysis of the basic emotions. The acknowledged cross-validated accuracy of 76.1% on the five built-in emotions is based on voice cues like “spectral balance”, that is higher than average for Anger, Happiness and Fear, and lower than average for Boredom and Sadness [15].

The studies presented were aimed, as a first step, at identifying the semantic elements of humility, secondly, investigating its multimodal communication in the political domain and finally observing emotional expressions of humble stance by face and voice. As results from our survey, humility entails acknowledgement of one’s own limits and flaws, a feeling of equality, and a goal of not showing superior to others, to the point of hiding one’s own advantages or merits. In political communication, the humble politician expresses his/her thoughts by treating the interlocutor in a familiar and empathic way. From multimodal analysis it emerges how anger is often expressed by humble politicians, as detected from Action Units such as AU 7. Yet this in their case can be viewed as a way to express their empathy, given that the anger is addressed to injustice for the people. Such interpretation can be supported by the fact that the second most expressed emotion is sadness, and both emotions result in an expression of worry and concentration about the topic at hand. Also from the descriptive results of the automatic detection of emotions by face and voice, humble politicians seem to express negative emotions more than positive ones, in particular anger. Furthermore, what emerges from the AU detection is the higher recurrence of some cues like brow furrow (AU 4) and lid tighten (AU 23) that are generally associated to anger but might also represent a state of worry and concentration [26] of the humble politician. The state of worry and concentration is also confirmed by other cues like lip press and [18], since while talking the humble politician typically tries to ‘measure his words’ as a form of empathy and care toward the interlocutor or also a form of self regulation of emotions. Future analysis will further deepen these emotional aspects, by also looking at other secondary emotions [22], and by comparing humble politicians with dominant ones, thus enlarging the numerosity of videos to analyse.

Beside the genuinely theoretical goals of studying humility, its everyday notion and its multimodal expression in politicians, this paper tries to move towards the automatic detection of humility and its simulation in artificial systems. One step in this direction is the search for the relevant parameters and the specific values that cue humble leaders’ multimodal communication, to be continued in the future.