Political orientation in media treatment of police violence: Evidence from modal adjectives

Abstract

The goal of this paper is to unveil possible correlations between the political orientation of newspapers and their treatment of police violence. We consider three different news publications with diverging political orientations, namely Jacobin (Left), Breitbart (Right), and The New York Times (Center). We performed a corpus study that relies on two different categorizations: a new ontology for police violence situations, identifying a set of recurrent themes, and the use and distributions of modal adjectives across these themes as revealing stances toward them. Modal adjectives are highly polysemous, and our analysis distinguishes between the epistemic readings as relating to factual truth, and evaluative readings as relating to norms. Our study shows that left and right leaning journals share similar uses of themes, but differ in their use of modal adjectives. These results could suggest that political orientation is responsible for differences that lie in the stance that the newspapers adopt.

Keywords

Corpus modality moral adjectives police violence political orientation

Introduction

On May 25, 2020, George Floyd died of asphyxia after police officer Derek Chauvin kneeled on his neck for almost 10 minutes. Bystanders filmed the altercation between the two men, repeatedly asking Chauvin to stop. The video went viral online, and a wave of protests started during the summer 2020, in the United States and in the rest of the world. The George Floyd case became an emblematic symbol of police violence in the United States, and the media took a great interest in both the trial and the protests. Police violence is here understood as the unnecessary and illegal use of force from police officers against civilians. It can take several forms (asphyxiation, shootings, etc.) and can, in the most extreme cases, lead to death. The question of systematic racism and racial biases in police intervention has been researched (Ross, 2015), and became especially central after the George Floyd case, as discourses surrounding the case and protests dominated the media in 2020. Research studies have shown that social media discourse surrounding the case focused on the injustice of the excessive use of force (Péloquin et al., 2022) and on the fact that the George Floyd case opened a broad discourse on systematic racism in the social media space (Thelwall and Thelwall, 2021). Fewer studies have been conducted on the treatment of the case by newspapers, which mostly focused on the different opinions expressed about the protests, using for example foucauldian discourse analysis (Hammond et al., 2021). Hammond et al.’s goal was to analyze the way the police violence victim was portrayed in the media, and concluded that the news sources were drawing attention to the crimes of both the victim and the protesters to portray them as violent agents. None of these research studies took the political orientation of the newspaper into account, though research has been carried out on the link between biases toward a controversial topic and political orientation of newspapers (see i.a. Caffarel and Rechniewski, 2009; Tavassoli et al., 2019).

In an attempt to understand how different news sources handled the evaluation of the information surrounding the George Floyd case, we created a corpus of US American news articles. Articles were gathered from news sources associated with three different political orientations: Jacobin (left), Breitbart (right), and The New York Times (center). We assumed the widely used and stereotypical left- right-center trichotomy for this study (see i.a. Emler, 2003; Jost et al., 2009). Our goal in this study was to unravel the correlation between the political orientation of newspapers and their use of the modal vocabulary and of moral vocabulary more specifically.

To see the relevance of modal vocabulary, consider (1) and (2). In these examples, the speaker is expressing his attitude toward a situation:

(1) Marc’s answer to question B is wrong.

(2) Marc shouldn’t have acted like this with Marie, it was wrong.

In both cases, we gain insight about the speaker’s stance on a topic, and even though the same word, wrong, is used, those two tokens of wrong are very different: the former has an epistemic reading, relating to knowledge and truth, while the second had a deontic reading, related to norms. While the basic epistemic/deontic contrast is correct and useful, it is too coarse-grained to allow for a detailed investigation of texts about police violence. Consider the following corpus examples, both again involving wrong:

(3) ‘What that cop did was wrong, but I’m scared now’, Brown said. (Breitbart, 2020)

(4) ‘Any aggressive act toward a peaceful protester sends exactly the wrong message’, Mr. de Blasio said at a news conference, referring to videos that appeared to show violent acts by the police. (Feuer and Paybarah, 2020)

Both of these are broadly deontic, but they involve different types of norms. (3) is based on the speaker’s understanding of morality, and it is used to set up a contrast between morality and the speaker’s sense of safety. This sense of wrong is based on a higher, transcendent norm. In (4) in contrast, wrong is not based on this type of moral norm, but rather emphasizes that attacking protesters divides the community, and so goes against a more mundane social norm. Building on previous studies (Portner, 2009, and references therein), we use a classification of modal senses that refines both the epistemic and deontic categories. Within epistemics, we can distinguish objective and subjective uses (Kratzer, 1981; Lyons, 1977; Nuyts, 2001; Giannakidou and Mari 2021). The following examples from Kratzer (1981) exemplify this distinction:

(5) Es ist wahrscheinlich, daß das Schiff sinkt.

‘It is probable that the boat will sink’.

(6) Wahrscheinlich sinkt das Schiff. issues.

‘Probably, the boat will sink’.

Example (5) makes a statement that purports to be based on consideration of some objective reasons, due to the presence of the adjective wahrscheinlich. Just like English probable, the adjectival modal relies on consideration of circumstances and factual evidence (Portner and Rubinstein, 2012); the adverbial counterpart in (6) more easily enhances an opinion or a subjective evaluation (for extended discussion on subjective adverbs, see Ernst, 2009). On the deontic side, we first note that many lexical items with prominent deontic senses have related meaning that are not related to norms. Kratzer (1981) points out that modal auxiliaries in German and English are often ambiguous among norm-oriented (deontic), goal-oriented (teleological), and desire-oriented (bouletic) readings. We see the same polysemy in modal adjectives like good.

(7) Her kind actions were good. (deontic)

(8) That path is a good way to get to town. (teleological)

(9) The cake is good. (bouletic)

Portner (2009) labels this broader class that includes deontic, teleological, and bouletic readings ‘Priority modality’. Within the norm-oriented deontic category, we can distinguish sub-types according to the source of the norm involved. We propose focusing on a distinction between conventional readings, where the norm is established within a community, and moral readings, where the norm is portrayed as coming from a higher source. This difference can be expressed through lexical choice, with correct based on a conventional norm, and just on a moral norm. For the purposes of our analysis, not all of the categories laid out above are equally important. As mentioned above, we study a corpus our journalistic prose, and we focus on a set of modal adjectives: good, bad, just, unjust, fair, unfair, right, wrong, correct. In the corpus, we find examples of these words expressing objective epistemic modality and the two sub-types of deontic modality, moral and conventional. Thus, we can summarize our assumptions about modal readings as in Table 1, where the boldfaced categories are the ones that will play a role in subsequent discussion. We use italics for specific lexical entries.

Table 1.

Modal readings.

Category	Epistemic		Deontic		Other priority
Subcategory	Objective	Subjective	Moral	Conventional
Basis	knowledge	opinion	transcendent	social norm	goals/desires
Example	certain	plausible	just	correct	delightful

Analyzing the link between uses of modal readings and political orientation could both provide a more precise understanding of the treatment of the George Floyd case in the US American media, and allow us to critically re-think the stereotypical left-right-center trichotomy.

Our two main hypotheses for this study are:

H1. The three newspapers will address different themes. In particular, because they are the most different in the left-right-center trichotomy, Jacobin (left) and Breitbart (right) will differ the most.

The first hypothesis targets the themes treated. It is based on research by Karjus and Cuskley (2024), who showed that the Left and the Right don’t focus on the same themes on social media, with the Right addressing more controversial issues. Karjus and Cuskley (2024) specifically found that conservative social media users tended to discuss themes that trigger opposing views of topics such as religion, political figures and political parties more than left-leaning users.

Our second hypothesis targets the use of modal adjectives:

H2. The New York Times (center) will more frequently use adjectives in an epistemic sense than to Jacobin and Breitbart, both of which will use more moral and conventional readings.

The hypothesis is doubly motivated. First, Frimer et al. (2019) have found that the political extremes use more subjective emotional language; we hypothesize that this will translate into a greater use of moral and conventional readings of modal adjectives. And second, based on discussion by Stanley (2015), we assume that the center aims at presenting itself as more ‘objective’ than the extremes, and, as such, will use more objective, epistemic language.

To test our hypotheses, we have elaborated an ontology of police violence, described in Section 5, which organizes different aspects of a situation of police violence as well as the main actors involved. With this categorization at hand, we have performed two types of annotation:

A thematic annotation, based on our ontological framework of police violence.

A modal annotation of each sentence containing a modal adjective, in order to finely analyze the link between modal reading and treatment of the George Floyd case in the media.

The main results of our study are the following: Breitbart (right) and Jacobin (left) are more similar than expected in the themes they use contra H1. Surprisingly, The New York Times (center) stands out by including different themes from the two other opposed publications. In terms of the readings of modal adjectives, Breitbart (right) and Jacobin (left) differ in the way they use the modal readings of the adjectives: Jacobin used more epistemics than Breitbart and The New York Times (center), and Breitbart used more moral readings. The New York Times doesn’t show any strong tendency. Thus, neither of the two hypotheses was confirmed, revealing that the empirical study does not support common assumptions about political ideology. Putting the two results together, we find that the left- and right-leaning newspapers tend to discuss similar themes, but to treat them in opposite ways: Jacobin was more ‘objective’ (epistemic) and Breitbart more ‘evaluative’ (moral).

The paper is organized as follows: in Section 2 we discuss previous works on corpus studies of police violence, the literature on political orientation and the relevant literature on modal adjectives. In Section 3, we present a motivated taxonomy of polysemous adjective usages. Section 4 presents our corpus, Section sets out 5 our Thematic Categories, and Section 6 discusses the results of our double annotation and the distribution of modal adjectives across newspapers with different political orientations. Section 7 concludes.

Related work

Corpus studies on police violence

Police violence has been the subject of several corpus studies in the last few years, with a definite focus on the George Floyd case (see i.a. Dong and Wu, 2022; Giorgi et al., 2022; Péloquin et al., 2022; Priniski et al., 2021). Tweets were the main material that was analyzed, often using computational techniques like Latent Dirichlet Allocation (LDA) to group words that appear in the same contexts (see Giorgi et al., 2022), and sentiment analysis techniques like the FrameAxis method (see Priniski et al., 2021).

Priniski et al. (2021)’s article, for example, automatically associated categories of the Moral Foundation Theory (Graham et al., 2013) with tweets by researching words linked to each category. Each category of the Moral Foundation Theory, like Authority or Fairness, contains a ‘vice’ and ‘virtue’ counterpart, and allows for an understanding of a biased context. For example, it was determined through this study that the words associated with Authority as a vice (like violence or riot) were used at a higher frequency than words associated with Authority as a virtue (like obedience). This revealed a sentiment of negativity surrounding police institutions and authority figures of the state in general. This study was mostly qualitative, but quantitative research was also conducted on the tweets related to the George Floyd case (see Nguyen et al., 2021).

The quantitative studies showed a decrease of the negative tweets mentioning Black people after the George Floyd case over the few weeks following the case. Racism was also mostly discussed negatively during these same few weeks. The study concluded that Twitter discourse on police violence following the George Floyd case involved more discourses surrounding systemic racism. While these studies brought forward interesting insights about the judgments made in relation to the George Floyd case, no study has, to our knowledge, analyzed news articles’ subjectivity and bias through their use of polysemous adjectives, as a key indication of stances adopted by the writers.

Disentangling political orientations

We assume in this paper the stereotypical left-right-center trichotomy of political orientations. We will describe the Center as the moderate view, the absence of leaning toward one extreme or the other. Indeed, ‘Left’ and ‘Right’ have been considered as political extremes, and studies have led to contradictory conclusions concerning how they relate to one another: extremists from the Left or the Right are either described as sharing the constitutive quality of authoritarianism (McClosky and Chong, 1985) or not sharing it (Stone and Smith, 1993).

As for the first position, Rokeach (1960) defines communism and fascism as ideologies having different contents, but similar structures. McClosky and Chong (1985) define the Left and the Right as being very similar in their political style and how they reach political goals. Both are, according to McCloskey and Chong, likely to use cruelty and manipulation to reach said goals.

Studies that present them as different emphasize that stereotypical Left characteristics include the defense of minorities, the support of democracy and free speech, while stereotypical Right characteristics include the defense of the state and the respect for traditions. The extreme of the Left is defined by Stone and Smith (1993), following Rokeach (1960), as communism. They describe it as the ideology according to which all humans are equal. The extreme of the Right is defined as fascism, which, among others, emphasizes the superiority of one group over another. Unlike Rokeach (1960) who claimed that communism and fascism were different in content but not in structure, Stone and Smith (1993) argue that the Left and the Right differ in terms of structure. In particular, based psychological research by Parrott and Brown (1972), Stone and Smith argue the Right is associated with a distinctive, authoritarian and dogmatic structure. Stone and Smith (1993) argue that the Left’s structure has not been properly defined yet, and that the focus of the literature on authoritarianism has favored studies on fascists and conservatives, rather than on the differences between political extremes.

Regardless of the stance taken on the similarities or differences between the Left and the Right, it is worth noting that they are suitable concepts to understand political dynamics and opinions (see Jost et al., 2009, i.a.). Arguing for the use of political orientations as a tool does not imply they should be understood as perfect opposites on a political spectrum. Indeed, the Left and the Right are not merely motivated by contradictory values, but also focus on different themes as whole. We use corpus analyses to better understand the similarities and differences between the extremes, and in particular to unveil a possible focus on different aspects of the events.

Adjectives and biases

Our study isn’t the first to utilize predicates in order to better understand biases (see among others, Bonyadi, 2011; Recasens et al., 2013; Sadia and Ghani, 2018). Biases can be expressed, among other ways, through grammar, metaphors, and the use of a certain vocabulary. Recasens et al. (2013)’s study extracted Wikipedia edits considered ‘too biased’ to be published and investigated what type of words was used. They found that subjective intensifiers like evaluative adjectives are often used for ‘framing bias’, revealing a specific stance toward a topic (Entman, 2007). Our study seeks to take advantage of the polysemy of modal adjectives and the types of norms that they can express, in order to unravel the perspectives that different political orientations adopt in communicating about police violence.

Evaluative adjectives, a class that includes deontics, have been discussed in the literature, especially for their subjective components (i.a. Bylinina, 2017; Goddard et al., 2019; Silk, 2021; Umbach, 2016). Theoreticians have argued that deontic adjectives are similar to aesthetic adjectives (e.g. ugly) and predicates of personal taste (PPTs) like tasty and fun (Lasersohn, 2005; Stephenson, 2007). However, recent work focusing on the distributions under attitude verbs of evaluative adjectives has showed that the evaluativity of deontics is not similar to that of PPTs (see Soria-Ruiz, 2019; Soria-Ruiz and Faroldi, 2022; Soria-Ruiz and Franzén, 2023; Stojanovic, 2019; Stojanovic and McNally, 2023). In particular, an important difference has been discovered by Stojanovic and McNally relating to the embedding of the adjectives under attitude verbs consider and find.

According to Kennedy and Willer (2022), find and consider differ in that consider is used in sentences involving a mental attitude, while find (10-b) relates to bodily senses and experience.

(10) a. I consider this homework well-written.

b. I find the soup tasty.

PPTs are typically used with find rather than with consider; in contrast, deontic predicates tend to be used with consider rather than find, and are thus not identical to PPTs (Kennedy and Willer, 2022). Moral predicates being more felicitous with consider than with find implies that the attitude held by speakers using a deontic reading is more of a mental, intellectual one.

Stojanovic and McNally (2023) do not study the polysemy of the adjectives and do not investigate the context of sentences containing moral predicates. Our work incorporates their important finding that deontics are not akin to subjectives and pursues the new goal of investigating the polysemy of the epistemic and deontic adjectives in the precise context of newspaper treatment of police violence.

Focusing on deontic adjectives and their specific relation to the expressions of norms, Travis and Cacoullos (2023) identify three possible sources of obligation: personal choice, hierarchical norms and general circumstances. Their description focuses on types of individual/group relations, and does not address the question of the degree of importance certain norms have inside a community. Portner and Rubinstein (2016) focus instead on the degree of importance of norms by investigating the commonalities between deontic modals and extreme/non-extreme adjectives. So-called extreme adjectives like brilliant and huge describe entities that have a property to a very high degree; they differ from non-extreme adjectives on the same scales like smart and big. Portner and Rubinstein (2016) show that some deontic adjectives are extreme, and moreover that strong necessity deontic modals like must could be analyzed as extreme in comparison their weak necessity counterpart, like should. The deontic strength of the necessity has a clear grammatical counterpart and is reflected in the distributional properties of the adjectives.

Scales of morality have been discussed in the meta-ethics literature as well. For example, Turiel (1983) and Smetana (1993) distinguish highly ranked moral norms, based on a vague authority but generalizable, and conventions based on a known authority but not generalizable outside of the context.

(11) Gender inequality is not right.

MORAL READING

(12) Marie’s behavior wasn’t right.

CONVENTIONAL READING

We build on the idea that different adjectives bear different deontic forces, revealing norms of different natures that can be ranked on a scale. In the following section we provide our working typology and examples extracted from our corpus.

Modal adjectives: Annotation schema

Based on our corpus study, we propose to distinguish a moral and a conventional reading, with the first revealing a necessity of a higher order, relying on a transcendental authority and the second conveying norms that are socially accepted within a community sharing the same assumptions.

Table 2 illustrates our working typology.

Table 2.

Working typology of the adjectives.

Category	Epistemic	Deontic
Category	Objective	Modal	Conventional
Basis	Knowledge	Transcendent norm	Social norm

In what follows, we describe our findings concerning the range of readings that are possible with a number of adjectives that we studied; the results of this discussion are summarized in Table 3.

Table 3.

Polysemous adjectives and their readings.

	Epistemic	Moral	Conventional
Good		✓	✓
Bad		✓	✓
Just		✓
Unjust		✓
Right	✓	✓	✓
Wrong	✓	✓	✓
Correct	✓	✓	✓
Incorrect	✓	✓	✓
Fair		✓	✓
Unfair		✓	✓

Adjectives that have an epistemic moral and conventional reading: right, wrong, correct, incorrect.

right:

(13) a. ‘New York’s newest protesters are right: it’s time to defund police’. Wong (2020)

EPISTEMIC

b. ‘They’ve been killing black folks for years, and we’ve been saying that it wasn’t right’. Rodriguez-Sanchez and Taylor (2020)

MORAL

c. ‘I have seen firsthand how the right kind of policing can serve our communities and make people’s lives better’. Hughes (2020)

CONVENTIONAL

wrong:

(14) a. He falsely claimed ‘80% of the rioters in Minneapolis last night were from out of state’, seemingly referring to a statement about the home states of the arrestees initially made by the Mayor of St Paul but almost instantly proven wrong. Gibbons (2020)

EPISTEMIC

b. ‘What that cop did was wrong, but I’m scared now’, Brown said. Breitbart (2020)

MORAL

c. ‘Any aggressive act toward a peaceful protester sends exactly the wrong message’, Mr. de Blasio said at a news conference, referring to videos that appeared to show violent acts by the police. (Feuer and Paybarah 2020)

CONVENTIONAL

correct:

(15) a. ‘But if even the moderate projections are correct, the downturn we are in may make building those capacities necessary to avoid a profound loss’. McCarthy (2020)

EPISTEMIC

b. ‘that’s one of the most fascinating pieces of your article for me, because obviously abolitionism was the morally correct position to take, but you need more than moral correctness to build a political coalition that can actually win’. Karp (2020)

MORAL

c. ‘So, I think the president is sending a strong and a correct message that we will not tolerate it under any circumstances’, he added.¹

CONVENTIONAL

incorrect:

(16) a. ‘It is incorrect to classify the revolt of the negro as simply a racial conflict of black against white, or as a purely American problem’. Darder and Fletcher (2020)

EPISTEMIC

b. His actions are morally incorrect.²

MORAL

c. Marie’s behavior was incorrect.

CONVENTIONAL

Adjectives that have moral and conventional readings: fair, unfair, good, bad, fair

fair:

(17) a. ‘Utla was an early signer onto this year’s ballot measure, back when it didn’t have a new name yet, when we were just calling it make it fair’. Myart-Cruz (2020)

MORAL

b. ‘People have been trying for a long time in California to tax corporations and make them pay their fair share’. Myart-Cruz (2020)

CONVENTIONAL

unfair:

(18) a. ‘This refusal to acknowledge how unfair life is based on skin color needs to stop now’. Network (2020)

MORAL

b. ‘These are unfair business practices in which idols are made to sign extremely long contracts (an average of seven years, but up to thirteen years) with stringent clauses regarding their behavior’. Pegolo (2020)

CONVENTIONAL

good:

(19) a. ‘Biden’s “most cops are good” comment rankles some progressives’. Epstein and Eligo (2020)

MORAL

b. ‘There’s a good chance the bad man in the white house will go’. Marcetic (2020)

CONVENTIONAL

bad:

(20) a. ‘Racism is bad’. Delingpole (2020)

MORAL

b. ‘A local resident said things are so bad at night, there’s a bullet hole in the windshield of her car’. Nolte (2020)

CONVENTIONAL

Adjectives that only have a moral reading: just, unjust.

just:

(21) We are hoping for a just verdict in this case.

MORAL

unjust:

(22) ‘First, they needed the belief that the system was unjust’. Fox Piven (2020)

MORAL

Corpus elaboration

No dataset existed, to our knowledge, containing specifically articles published about the George Floyd case. We collected our own corpus by gathering articles published in Jacobin, Breitbart, and The New York Times, between May 2020 and August 2020. This period of time was chosen because most articles related to the case were published within those 4 months. Indeed, the summer 2020 saw the BLM (Black Lives Matter) protests, which led to many relevant articles. The corpus contains a total of 294 articles (12,513 sentences). Written material was chosen for various reasons, including that no transcription existed for most speeches available online.

Our corpus does not distinguish between different types of articles published in the three papers. We are interested in the overall presentation of events by the three publications, whether they do it through pieces that one might categorize as ‘news’ or ‘opinion’. Note that Breitbart and Jacobin do not even make an explicit distinction between news and opinion pieces, and we could not impose such a distinction in a non-circular way. In view of the differences in the organization of the three newspapers and the lack of comparable categorizations of their articles, we have decided to include all relevant articles, without trying focus on only ‘news’ or ‘opinion’.

We performed an annotation of modal adjectives with the goal of investigating:

The link between modal readings of adjectives and theme of the sentence containing them.

The relation between the readings of modal adjectives and the political orientation of the newspaper.

Each sentence containing one of the modal adjectives listed above was annotated for one of the three senses (epistemic, moral, conventional). There were in total 377 modal adjectives in the corpus. We also considered the polarity of the sentence to study whether, aside from which readings were associated with each thematic category, the modal use reflected a positive or a negative stance taken by the journal.

Thematic categories

To achieve the thematic annotation, we developed a basic ontology of police violence, summarized in Figure 1. To our knowledge this the first proposed ontology of events of police violence. This ontology was developed through the reading of 50% of the articles about the George Floyd case published in Jacobin, Breitbart and The New York Times between May 2020 and August 2020. Recurrent actors were noted, and this process allowed us to distinguish among 11 categories, counting the ‘Other’ category: Police, Justice, Government, General (political) movement, Political party, Social medias, Traditional medias, Celebrities, Protests, Family, and Other.

Figure 1.

Ontology of police violence.

What follows is a short rendering of the guidelines for the thematic annotation, with an example taken from the corpus, for each thematic category. Participants were asked to focus on what the sentence was about, not who was uttering it.

Police: sentences related to the policemen involved or of the police institution as a whole, statements from the lawyers of the policemen involved and of police unions.

Especially in the same week it was revealed that two police officers took selfies with the dead bodies of black women. Bankole (2020)

Justice: sentences related to judges, justice decisions, trials, testimonies from witnesses, autopsy results.

He surrendered in the morning to the Queens district attorney’s office and was released without bail after a video arraignment later in the day in criminal court. Southall (2020)

Government: sentences related to the government and elected officials, changes in state policing.

In 2015, President Obama used Camden, New Jersey as a prop to announce the findings of the President’s Taskforce on 21st Century Policing, a package of procedural reforms to address the post-Ferguson crisis of police legitimacy. McQuade (2020)

General political movement: sentences related to vague political movements, like ‘the Left’, or ‘populists’ and their political stances. Also includes vague mentions of communities coming together, outside of protests.

Activists and community leaders I interviewed in 2016 suspected these events were intentional acts to trigger a crime spike, grab headlines, and create the conditions to impose further changes in the city. McQuade (2020)

Political party: sentences related to a specific political party like the Republicans or the Democrats, or one of their elected officials, mentions of their political stances.

But while all legislation aimed at tackling racial discrimination has been passed by the Labour Party, the party has a checkered history on race and racism. Bankole (2020)

Social media: sentences related to social medias such as Twitter or Facebook, and their uses.

Facebook is OK with encouraging polarization. Leonhardt (2020)

Traditional media: sentences related to traditional medias such as TV, radio, newspaper, and their uses, mentions of journalists.

The sentiment was perhaps best summed up by Sunny Hostin during an episode of ABC’s The View on Wednesday. Alic (2020)

Celebrities: sentences related to any type of celebrity, whether they support BLM or not.

Kyle Larson was suspended from the sport after saying the n-word over what he thought was a private channel but was broadcast to all the participants in a race. Rabin-Havt (2020)

Family: sentences related to the victim’s family or the victim themselves, victim’s family statements, and statements from the lawyer of the family. Also includes direct descriptions of the victim himself, as well as tributes.

(about George Floyd) He grew up in Houston, in a black neighborhood south of downtown known as the Third Ward, and was raised in a house with his siblings and two cousins, Shareeduh Tate and Tera Brown. Times (2020)

Protests: sentences related to any type of protesters or protests, whether they support BLM or not, descriptions of protest violence and discourse.

Protesters tore down a statue of Christopher Columbus in St. Paul, Minn. Wilkerson (2020)

Other: all sentences that can’t be related to any of these categories.

The erasure of Leon Trotsky from official Soviet pictures under Stalinism was another form of damnatio memoriae, and was inspiration for George Orwell’s 1984. Traverso (2020)

The annotation guidelines focused on the disambiguation of less straightforward categories, like General political movement, Government, or Political party. In cases multiple annotations were plausible, for example when an individual was connected to different aspects of the case, participants were told to focus on the broad context of the article. For example, (23) corresponds to the category Justice, not Police, because, while the policeman involved in Floyd’s murder is mentioned, the main topic of the sentence is the trial.

(23) The trial of Mr. Chauvin, charged in the death of George Floyd, will resume on Monday.

Aside from those guidelines, participants were encouraged to email the research team if they couldn’t find an answer to their questions.

Results

Results related to the distributions of the thematic categories (H1)

The annotation of the thematic categories allows us to determine to what extent each aspect of the George Floyd case was discussed by each paper. We expected the left- oriented journal, Jacobin, to be quite different from the right-oriented journal, Breitbart, due to their opposite political orientations (H1). However, what we found is that these two papers were more similar to each other than to the center newspaper, The New York Times. Before comparing the newspapers and in order to ensure that there is no correlation between the newspapers in a way that could impact the analysis of their comparison, we previously performed a Pearson’s chi-squared test (χ²). Pearson’s chi-squared test, also called test of independence, is used to measure the (in-)dependence of two variables. The test returned an overall p-value is consistently above 0.99999, supporting the independence hypothesis. This guarantees that the statistical results can be analyzed independently for each newspaper and that the comparison is sound. As we can see in Figure 2, the themes used the most in the corpus are Police, Government, General Political Movement, and Protests.

Figure 2.

Distribution of the thematic categories in the corpus.

As Figure 3 shows, Jacobin mostly dealt with the themes General Political movement (26.42%), Police (21.54%), and Protests (16.56%). Breitbart wrote mainly about the themes Police (19.51%), Protests (17.08%), and Government (15.47%). Finally, The New York Times mostly produced articles concerning the themes Protests (30.45%) and Police (27.39%). Indeed, for all three newspapers, the four most frequently used themes are the same, in particular Protests and Police, indicating a shared interest in the main protagonists of the confrontation, Protestors and Policers, independently of political orientation (see Figure 3).

Figure 3.

Comparison of the distribution of the four most frequent categories in Jacobin, The New York Times, and Breitbart.

The Euclidean distance between the distribution of themes highlights a stronger thematic proximity between Jacobin and Breitbart: Jacobin and Breitbart display an average Euclidean distance of 3.5%, while the average distance between The New York Times and Breitbart is 7.8% and between The New York Times and Jacobin is 10.4%. These measures indicate that Jacobin’s and Breitbart’s thematic focus are significantly more similar to each other than they are to The New York Times, grounding the observation that political orientation plays no apparent role in the choice of the themes.

The main difference in distribution between Jacobin and Breitbart lies in their treatment of General Political Movement, with Jacobin using this category much more than the other publications. General Political Movement was used to categorize mentions of communities and vague groups (like the ‘Left’). This could be a sign that Breitbart had a lower tendency to mention other movements than the protests, whereas Jacobin opened the article’s themes to broader movements and communities while discussing the George Floyd case. Jacobin and Breitbart, however, have similar distributions of Police, Protests,and Government.

The New York Times stands out when it comes to treatment of these themes. The comparison between The New York Times and Breitbart in Figure 3 shows that while both Police and Protests are highly frequent in both papers, there are important differences, especially for the Protests category. The New York Times uses it at 30.45%, while Breitbart uses it just over half as much, at 17.08%. This difference also emerges when comparing The New York Times to Jacobin; the latter only uses the Protests category at 16.56%. On average, The New York Times treats the Police and Protests categories more than Jacobin and Breitbart, but discusses Government and General Political Movement less than the other two newspapers.

In spite of these differences, the initial observation that there is an overlap in the interest toward Police and Protests stands. It follows that thematic categories by themselves aren’t sufficient to fully differentiate the newspapers in their treatment of the George Floyd case. As revealed by the Euclidean distances, Breitbart and Jacobin in particular show important similarities, invalidating H1.

Results related to the use of modal categories (H2)

Next, we will look at the readings of modal adjectives in each newspaper with the goal of uncovering any correlations between political orientation and the use of the adjectives. We in particular aim to determine whether H2 is correct in expecting that centrist newspaper adopt a more ‘objective’ stance by using the adjectives with an epistemic interpretation. As a reminder, we are working with 10 adjectives: good, bad, just, unjust, fair, unfair, right, wrong, correct, and incorrect, annotated for the 3 readings: epistemic, moral, and conventional.

As Figure 4 shows, the moral reading was found much more than the others, with 57.14% of use throughout all journals. The epistemic reading was used much less than the moral and conventional reading, with only 10.61% of use. The overall percentage for the conventional reading was of 31.43%. The New York Times percentages are very similar to the average percentage of the entire corpus. However, that is not the case for Jacobin and Breitbart: Jacobin uses the moral reading at 50%, which is less than the average, while Breitbart uses it more than the average, with 63.64%. Jacobin uses the epistemic reading more with 15.38% while Breitbart uses it less with 5.19%. The use of a higher percentage of epistemics and a lower percentage of moral readings seems to point to a less subjective stance from Jacobin, thus disconfirming H2.

Figure 4.

Distribution of the modal readings in Breitbart, Jacobin, and The New York Times.

As we saw earlier, there are fewer adjectives with a potential epistemic reading than adjectives with a potential moral or conventional reading. Only right, wrong, correct, and incorrect have a potential epistemic reading, 4 out of the 10 adjectives we are focusing on. In the overall corpus, 20% of the uses of these four modal adjectives was epistemic. This could in part explain the lower overall amount of epistemic readings in Figure 4. However, at the journal level, we continue observe that Jacobin privileges the epistemic reading for these adjectives at 28.33%, against 18.03% for The New York Times and 11.11% for Breitbart. These results still display a tendency for Jacobin to use more epistemic readings for these adjectives. These results show the importance of focusing on readings at the scale of each journal instead of overall results.

Modal reading and thematic categories

To understand the potential link between context and modal reading, we combined the results of the two annotations. The sentences containing modal adjectives mostly discussed Police, General Political Movement, and Protests, just like the rest of the corpus. However, an analysis of the reading of each adjective provides additional information. As seen in Figure 5, the theme General Political Movement was largely discussed using epistemics, whereas the theme Police was mostly discussed using adjectives with a moral or conventional reading.

Figure 5.

Distribution of the modal readings in the thematic categories across all journals.

This result is made more nuanced by the fact that different newspapers favored different readings for specific categories. Jacobin discusses Police using adjectives with a conventional reading at a higher frequency than the other journals (see Figure 6), while The New York Times uses both conventional and epistemic readings. Breitbart discusses Police using the moral reading at higher frequency than Jacobin (see Figure 7), with 75% against 62.5%. Breitbart showcases no epistemic readings for General Political Movement, while Jacobin uses epistemics 25% of the time for this category, which is higher than Breitbart and The New York Times (Figure 8).

Figure 6.

Distribution of the modal readings in the four most used thematic categories in Jacobin.

Figure 7.

Distribution of the modal readings in the thematic categories in Breitbart.

Figure 8.

Distribution of the modal readings in the thematic categories in The New York Times.

Polarity and thematic annotation

The polarity of the sentences containing modal adjectives was also annotated. Our goal was to make sure the overall valence associated with the category was correct, and did not merely depend on the valence of the adjective. This part of the analysis focuses on the four most discussed categories, since the others were too infrequent for us to draw conclusions. As a reminder, those categories are Police, Protests, General Political Movement and Government.

As we can see in Figures 9 and 10, investigating the polarity of the sentences allowed us to differentiate further between Breitbart and Jacobin. Jacobin is indeed a lot more negative when discussing Police than Breitbart is. Jacobin is however more positive in contexts of General Political Movement than Breitbart. The New York Times has notably 50% of sentences that have a positive valence, and 50% of sentences that have a negative one. Unlike Jacobin and Breitbart, The New York Times is more evenly distributed per category, and there are less disparities between a positive and negative use of a modal than with Jacobin and Breitbart (Figure 11).

Figure 9.

Distribution of the polarity of sentences containing modal adjectives, in the four most used thematic categories, in Breitbart.

Figure 10.

Distribution of the polarity of sentences containing modal adjectives, in the four most used thematic categories, in Jacobin.

Figure 11.

Distribution of the polarity of sentences containing modal adjectives, in the four most used thematic categories, in The New York Times.

Conclusion

This paper has laid out a unified way of classifying the readings of polysemic modal adjectives, taking into account epistemic, moral, and conventional readings. We went over the classic deontic/epistemic categorization, and proposed a finer distinction based on whether the epistemic reading was objective or subjective, and on whether the norm for a deontic reading was based on transcendent moral principles or social conventions. We performed two annotation studies, a thematic one to understand what recurring theme was present in the articles, and a modal one, to finely analyze the different readings of modal adjectives. Contrary to our hypothesis, the two journals at the opposite end of the political scale, Jacobin and Breitbart, did not use completely different themes to discuss the case; rather, they were similar in their thematic foci. The analysis of the modal readings gave us a more nuanced view of the attitudes at play in the treatments of the case. Jacobin used on average more epistemics than Breitbart and The New York Times, while Breitbart used more moral readings. Epistemics being related to truth in the actual world, their greater use in Jacobin implies more objectivity. In using more moral and conventional readings, Breitbart is providing an evaluation of the situation, based on norms it subscribes to. These findings seem to imply that the stereotypical left-right-center trichotomy is not a sufficient categorization to understand divergence of opinion in a controversial topic such as police violence.

To conclude, we would like to emphasize that this is the first study of its kind, and we hope replications will be attempted on different corpora. Working with the same ontology of police violence on other cases could also allow us to see if there is a real tendency for news publications of different political orientation to emphasize one aspect of the case or another. This study also provided an initial demonstration of the importance of studying modality when investigating political orientations, and further work of this kind could enrich the research on political biases (Table 4).

Table 4.

Summary of the association between newspapers and thematic categories, modal categories and polarity.

	Thematic categories	Modal categories	Polarity	Conclusion
Jacobin	• Uses General Political Movement more than other journals• Is very similar to Breitbart	• Uses epistemics more than the NYT and Breitbart• Uses overall less morals	• Is more negative toward the Police than the NYT and Breitbart• Is positive toward General Political Movement, but negative toward Protests, like the NYT	• Leans toward objectivity
The New York Times	• Uses Police and Protests more than the other journals• Is not similar to any other journal	• No tendency emerges	• Is positive toward General Political Movement, but negative toward Protests, like Jacobin	• No tendency emerges
Breitbart	• Is very similar to Breitbart, except for the use of General Political Movement	• Uses more moral readings than Jacobin and the NYT	• No theme is treated extremely positively or extremely negatively in comparison to other journals	• Leans toward evaluativity

Footnotes

Acknowledgements

The annotation study performed in this article was approved the IRB of Georgetown University. Alda Mari gratefully acknowledges ANR -17-EURE-0017 FrontCog.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Isidora Stojanovic (same institution) Andres Soria-Ruiz (previously same institution)

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Alda Mari

Paul Portner

Notes

Author biographies

Tess Feyen is a Ph.D. Candidate in Linguistics at the Ecole Normale Supérieure/PSL and former visiting scholar at Georgetown University for 2021-2024. Her research interests are at the intersection between semantics, political sciences and philosophy of language.

Alda Mari is Directrice de Recherches at CNRS. She is a formal semanticist with a background in ancient Greek and Sanskrit. She has extensively published on modality, mood and attitudes, in both a theoretical and computational perspective. She is the co-author of Truth and Veridicality in Grammar and Thought (University of Chicago Press), and of Modal Sentences (Cambridge University Press).

Paul Portner, Professor of Linguistics at Georgetown University, has published on a wide range of topics in semantics and its interfaces with syntax and pragmatics. In particular, he has worked on imperatives and exclamatives, tense and aspect, allocutivity, modality, and verbal mood. He is the author of two major books, Modality (2009) and Mood (2018), published by Oxford University Press.

Valentin Tinarrage holds a Master’s degree in Natural Language Processing from Université Paris Nanterre. Previously a Research Engineer at Institut Jean Nicod (CNRS), he is currently a Digital Project Manager at the French Ministry of Culture.

References

Alic

(2020) Klobuchar previously declined to prosecute officer involved in George Floyd’s Death. Breitbart.

Bankole

(2020) Unconscious bias training is not the answer. Jacobin.

Bonyadi

(2011) Linguistic manifestations of modality in newspaper editorials. International Journal of Linguistics 3(1): E30.

Breitbart (2020) National guard activated to respond to Minneapolis violence. Breitbart.

Bylinina

(2017) Judge-dependence in degree constructions. Journal of Semantics 34(2): 291–331.

Caffarel

Rechniewski

(2009) A systemic functional approach to analysing and interpreting ideology: An illustration from French editorials. Revista Alicantina de Estudios Ingleses 22: 27–43.

Darder

Fletcher

(2020) Black lives matter is part of a global struggle against oppression. Jacobin.

Delingpole

(2020) Delingpole: Black lives matter and Antifa in London: Where are the grown ups? Breitbart.

Dong

(2022) Reaching and engaging people: Analyzing tweeting practices of large U.S. police departments pre- and post- the killing of George Floyd. PLoS One 17(7): e0269288.

10.

Emler

(2003) Morality and political orientations: An analysis of their relationship. European Review of Social Psychology 13(1): 259–291.

11.

Entman

(2007) Framing bias: Media in the distribution of power. Journal of Communication 57(1): 163–173.

12.

Epstein

Eligo

(2020) Biden’s ‘most cops are good’ comment rankles some progressives. The New York Times.

13.

Ernst

(2009) Speaker-oriented adverbs. Natural Language & Linguistic Theory 27: 497–544.

14.

Feuer

Paybarah

(2020) Thousands protest in N.Y.C., clashing with police across all 5 boroughs. The New York Times. Available at: https://www.nytimes.com/2020/05/30/nyregion/protests-nyc-george-floyd.html (accessed 20 December 2022).

15.

Fox Piven

(2020) Frances Fox Piven on why protesters must “defend their ability to exercise disruptive power. Jacobin.

16.

Frimer

Brandt

Melton

, et al. (2019) Extremists on the left and right use angry, negative language. Personality and Social Psychology Bulletin 45(8): 1216–1231.

17.

Giannakidou

Mari

(2021) Truth and Veridicality in Grammar and Tought. Mood, Modality, and Propositonal Attitudes. Chicago: University of Chicago Press.

18.

Gibbons

(2020) Donald Trump’s “Antifa” hysteria is absurd. But it’s also very dangerous. Jacobin.

19.

Giorgi

Guntuku

Himelein-Wachowiak

, et al. (2022) Twitter Corpus of the #BlackLivesMatter movement and counter protests: 2013 to 2021. Proceedings of the International AAAI Conference on Web and Social Media 16(1): 1228–1235.

20.

Goddard

Taboada

Trnavac

(2019) The semantics of evaluational adjectives: Perspectives from natural semantic metalanguage and appraisal. Functions of Language 26(3): 308–342.

21.

Graham

Haidt

Koleva

, et al. (2013) Chapter two - moral foundations theory: The pragmatic validity of moral pluralism. Advances in Experimental Social Psychology 47: 55–130.

22.

Hammond

Ncube

Fido

(2021) A foucauldian discourse analysis of the construction of people of colour (POC) as criminals in UK and US print media following the Black Lives Matter protests of May 2020. Race and Crime in Print Media. PsyArXiv Preprints. DOI: 10.31234/osf.io/3yp45.

23.

Hughes

(2020) I’m a Black police officer. Here’s how to change the system. The New York Times.

24.

Jost

Federico

Napier

(2009) Political ideology: Its structure, functions, and elective affinities. Annual Review of Psychology 60: 307–337.

25.

Karjus

Cuskley

(2024) Evolving linguistic divergence on polarizing social media. Humanities and Social Sciences Communications 11(1): 1–14.

26.

Karp

(2020) How the antislavery movement ignited a political revolution. Jacobin.

27.

Kennedy

Willer

(2022) Familiarity inferences, subjective attitudes and counterstance contingency: Towards a pragmatic theory of subjective meaning. Linguistics and Philosophy 45(6): 1395–1445. https://doi.org/10.1007/s10988-022-09358-x

28.

Kratzer

(1981) The notional category of modality. In: Eikmeyer

H-J

Rieser

(eds) Words, Worlds, and Contexts. Berlin: de Gruyter, pp.38–74.

29.

Lasersohn

(2005) Context dependence, disagreement, and predicates of personal taste. Linguistics and Philosophy 28(6): 643–686.

30.

Leonhardt

(2020) The persistence of police killings. The New York Times.

31.

Lyons

(1977) Semantics. Cambridge: Cambridge University Press.

32.

McCarthy

(2020) To recover from this crisis, we need to defund wall street. Jacobin.

33.

McClosky

Chong

(1985) Similarities and differences between left-wing and right-wing radicals. British Journal of Political Science 15(3): 329–363.

34.

McQuade

(2020) The “Camden model” is not a model. It’s an obstacle to real change. The New York Times.

35.

Marcetic

(2020) American authoritarianism runs deeper than Trump. Jacobin.

36.

Myart-Cruz

(2020) The union has to be about social justice. Jacobin.

37.

Network

(2020) What students are saying about the George Floyd Protests. The New York Times.

38.

Nguyen

Criss

Michaels

, et al. (2021) Progress and push-back: How the killings of Ahmaud Arbery, Breonna Taylor, and George Floyd impacted public discourse on race and racism on Twitter. SSM - Population Health 15: 100922.

39.

Nolte

(2020) Nolte: NYT says ‘sacred’ site of George Floyd’s death spiraling into Chaos—‘Constant Gunshots Every Night.’ Breitbart.

40.

Nuyts

(2001) Subjectivity as an evidential dimension in epistemic modal expressions. Journal of Pragmatics 33: 383–400.

41.

Parrott

Brown

(1972) Political bias in the Rokeach Dogmatism Scale. Psychological Reports 30(3): 805–806.

42.

Pegolo

(2020) How K-Pop’s record labels exploit its “idols”. Jacobin.

43.

Péloquin

Fortin

Paquette

(2022) Examining negative online social reaction to police use of force: The George Floyd and Jacob Blake events. Canadian Journal of Criminology and Criminal Justice 64(1): 53–81.

44.

Portner

(2009) Modality. Oxford: Oxford University Press.

45.

Portner

Rubinstein

(2012) Mood and contextual commitment. In: Chereches

(ed.) Semantics and Linguistic Theory. pp.461–487. Chicago: University of Chicago Press.

46.

Portner

Rubinstein

(2016) Extreme and non-extreme deontic modals. In: Charlow

Chrisman

(eds) Deontic Modality. Oxford: Oxford University Press, pp.256–282.

47.

Priniski

Mokhberian

Harandizadeh

, et al. (2021). Mapping moral valence of tweets following the killing of George Floyd. arXiv.

48.

Rabin-Havt

(2020) Black lives matter protests are even reaching NASCAR. Jacobin. Available at: https://jacobin.com/2020/06/black-lives-matter-nascar-blm-kirk-price-bubba-wallace (accessed 20 December 2022).

49.

Recasens

Danescu-Niculescu-Mizil

Jurafsky

(2013) Linguistic models for analyzing and detecting biased language. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ed. Schuetze

Fung

Poesio

), Sofia, Bulgaria, August 2013, pp.1650–1659. Association for Computational Linguistics. Canadian Center of Science and Education (CSCE).

50.

Rodriguez-Sanchez

Taylor

(2020) Police are not designed to solve the problems people are facing. Jacobin.

51.

Rokeach

(1960) The Open and Closed Mind. New York, NY: Basic Books.

52.

Ross

(2015) A multi-level Bayesian analysis of racial bias in police shootings at the county-level in the United States, 2011–2014. PLoS One 10(11): e0141854.

53.

Sadia

Ghani

(2018) Modality in editorials of Pakistani English newspapers: A corpus based study. International Journal of English Linguistics 9(1). DOI: 10.5539/ijel.v9n1p144.

54.

Silk

(2021) Evaluational adjectives. Philosophy and Phenomenological Research 102(1): 127–161.

55.

Smetana

(1993) Understanding of social rules. In: Bennett

(ed.) The Development of Social Cognition: The Child as Psychologist. New York, NY: Guilford Press, pp. 111–141.

56.

Soria-Ruiz

(2019) The place of value in natural language: A non-factualist approach to evaluative terms and judgment. These de doctorat. Paris Sciences et Lettres (ComUE), Paris.

57.

Soria-Ruiz

Faroldi

FLG

(2022) Moral adjectives, judge-dependency and holistic multidimensionality. Inquiry 65(7): 887–916.

58.

Soria-Ruiz

Franzén

(2023) Moral and Moorean incoherencies. Ergo an Open Access Journal of Philosophy 10(0). DOI: 10.5539/ijel.v9n1p144.

59.

Southall

(2020) N.Y.P.D. officer charged with using illegal chokehold. The New York Times.

60.

Stanley

(2015) How Propaganda Works, 1st edn. Princeton, NJ: Princeton University Press.

61.

Stephenson

(2007) Judge dependence, epistemic modals, and predicates of personal taste. Linguistics and Philosophy 30: 487–525.

62.

Stojanovic

(2019) Disagreements about Taste vs. Disagreements about Moral Issues. American Philosophical Quarterly 56(1): 29–41.

63.

Stojanovic

McNally

(2023) Are moral predicates subjective? A Corpus study. In: Plou

(ed.) Experimental Philosophy of Language: Perspectives, Methods and Prospects. Cham: Springer, pp.103–120.

64.

Stone

Smith

(1993) Authoritarianism: Left and right. In: Stone

Lederer

Christie

(eds) Strength and Weakness: The Authoritarian Personality Today. New York, NY: Springer, pp.144–156.

65.

Tavassoli

Jalilifar

White

(2019) British newspapers’ stance towards the Syrian refugee crisis: An appraisal model study. Discourse & Society 30(1): 64–84.

66.

Thelwall

(2021) Twitter during COVID-19: George Floyd opening a space to address systematic and institutionalized racism? SSRN.

67.

Times

TNY

(2020) Explosive anger hits twin cities, and the nation. The New York Times.

68.

Traverso

(2020) Tearing down statues doesn’t erase history, it makes us see it more clearly. Jacobin.

69.

Travis

Cacoullos

(2023) Form and function covariation: Obligation modals in Australian English. Language Variation and Change 35: 1–27.

70.

Turiel

(1983) The Development of Social Knowledge: Morality and Convention. Cambridge: Cambridge University Press.

71.

Umbach

(2016) Evaluative propositions and subjective judgments. In: Meier

van Wijnbergen-Huitink

(eds) Subjective Meaning Alternatives to Relativism. Berlin: De Gruyter, pp.127–168.

72.

Wilkerson

(2020) America’s enduring caste system. The New York Times.

73.

Wong

(2020) Leftists have been calling to ‘defund the police’ for over a decade. Breitbart.