Governing Hate: Facebook and Digital Racism

Governance and Regulation

The shift toward regulating social media spaces has generated a set of problems, the first of which concerns the definition of problematic content. Facebook, and other platforms, must first define what constitutes acceptable and not acceptable content, and then seek its removal. One of the ways platforms approach this issue is through mobilizing the construct of hate speech and developing policies around it. The second problem concerns the implementation of relevant policies and the efficiency of the implementation. In short, the platforms have to develop a relevant apparatus to address hate speech. This apparatus must be placed within the broader workings of the platforms, and their overall operations as for-profit enterprises. In other words, defining, regulating and implementing regulations and policies on hate speech have to be understood in the broader context of digital platforms as intermediating between various users, and various levels of users (businesses, individuals, organizations, groups, and so on), operating across national borders, and relying on a particular revenue model.

At this level, research has highlighted two main issues concerning hate speech: firstly, that any regulation is essentially voluntary; and secondly, that the implementation of this regulation has led to the creation of a new role, that of content moderator. Given that social media platforms are almost exclusively US based corporations it is not too surprising that they are subjected to very limited regulation in terms of the contents they host. In the US context they are subjected to the so-called “safe harbor” regulation: Section 230 of U.S. telecommunication law. According to Gillespie (2018) this law has two parts: the first ensures that platforms are not liable for the contents they host, because they are not publishers; the second is that if they decide to moderate contents, they do not lose their safe harbor status, while, moreover, they are not required to meet any standards of effective monitoring or moderation. In short, whatever steps they take toward addressing the issue of hate speech are entirely voluntary.

In the EU, the focus of regulation is on illegal contents, which refer to “any information which is not in compliance with Union law or the law of a Member State concerned” (European Commission 2018, 10). In 2016, the European Commission along with the main digital platforms prepared and signed a voluntary Code of Conduct. The working assumption is that EU countries do not need new legislation for illegal online hate speech, but the enforcement of the existing one by digital platforms. One of the direct results of the Code of Conduct is the increase in the number of human moderators, who review the contents reported. Content moderation not only constitutes an entirely new job, but as Gillespie (2018) argues, it is central to the work platforms do. Roberts (2019) reports that there are currently over 100,000 people worldwide employed as content moderators, mostly outsourced workers in precarious contracts. Despite a shift toward algorithmic moderation, human moderators are still an important element in the regulation of contents.

But moderation is only one part of a larger model of platform governance. How is it nested within other processes, and how do all these processes fit together? Discussions of platform governance understand it in its broadest dimension as involving the various “layers of governance relationships structuring interactions between key parties” (Gorwa 2019, 2). Gorwa (2019) primarily focuses on the “external” processes by which state actors, platforms and civil society groups are asymmetrically involved in developing rules for platform governance. From this perspective, the ways in which hate speech is approached in platforms is seen as the outcome of negotiations between supranational bodies such as the European Commission, the platforms, and civil society groups. This arrangement is closely reflected in the Code of Conduct. Other research on internal governance mechanisms, for example, by Van Dijck (2013), has looked at the terms of service or regulations that social media impose on their users. However, this perspective does not consider the ideological parameters of governing systems, in the sense of a set of discourses that permeate these key actors, and which then structure the relationships between them.

Governing in Foucault’s (1982) understanding refers to structuring “the possible field of action of others” (p. 790). As Foucault (2007) noted, we cannot understand governing systems without understanding the political rationality that underpins them. This governmental rationality in turn guides and directs those governed in specific ways; it socializes them into engaging in certain behaviors and not others, it confers certain rights and responsibilities, and inevitably involves rewards and punishments. Foucault (2007) found that government must show a purpose that is beyond the act of government itself; it must have a certain telos or vision. In these terms, governing consists of all the actions aimed to take people there and accomplish this vision. Both the goal of government and the “methodology” for achieving it are characterized by specific logics which then constitute its rationality. In modern states in what is generally understood as the “West,” the dominant governmental rationality is that of liberalism, and its telos or vision is that of individual freedom, understood as removal of coercion and barriers to action (c.f. Berlin 1959). Overall, this rationality is characterized by minimal state intervention and a social space that is governed through free interactions of individuals, exemplified by the market. The market, where forces of production are free to meet and exchange products and ideas, is therefore placed at the center of liberal and neoliberal governance.

Following therefore a Foucauldian understanding of government and its rationality, this article focuses on the ideological structures of Facebook’s internal governance mechanisms surrounding hate speech and through this digital racism as it occurs on Facebook. Using this framework, we examine the ways in which hate speech is understood and acted upon with a view to outline the implications for the targets of hate and more broadly for users of Facebook. In understanding the structures of governing hateful contents, we seek to understand how Facebook approaches the question of racism, how it defines and regulates it. Since Facebook has 1.69 billion users in 2020, its definitions and regulations of (racist) hate speech are in many instances the prevailing ones within which users are socialized: for many users, in other words, racism is what Facebook defines as racist hate speech, what it allows and disallows from its platform. Understanding its governance systems and the ideology that drives them, enables us to understand one of the most influential approaches to defining and regulating racism.

In short, through an analysis of Facebook’s system of governance of hate speech and racist contents, this article seeks to understand some of the ideological and techno-material conditions that give rise to what Matamoros-Fernández (2017) calls platformed racism: the kind of racism that emerges at the intersection of platform affordances, algorithms, policies, business models, and user engagement.

Governing Documents and Policies

We begin the analysis with a discussion of the Facebook Principles as the document that effectively explains Facebook’s identity, its DNA, as it was known colloquially within Facebook. In November 2019, these were simplified and condensed into four principles: give people voice, build connection and community; serve everyone; keep people safe; and protect privacy. Here we discuss the Facebook Principles which were, until November 2019, the key text that determined everything else: “This Statement of Rights and Responsibilities: “Statement,” “Terms,” or “SRR” is derived from the Facebook Principles, and it is our terms of service that govern our relationship with users and other people who interact with Facebook” (Facebook, last revision January 2018, link no longer available). Figure 1 below shows a capture of the Principles archived on October 23, 2019 by the WayBack Machine:

OPEN IN VIEWER

Figure 1.

Facebook principles, October 2019.

All principles are important and contribute to the formation of the subsequent more detailed Terms of Service and Community Standards. They all reveal the main ideological principles behind Facebook. In this context, we understand ideology as “a special form of social cognition shared by social groups” (Van Dijk 2001, 12), which then the basis of a belief system and guide actions. The general premises of the ideology that permeates Facebook Principles are spelled out in an essay written almost ten years before Facebook was even created but are still valid. Barbrook and Cameron (1996) described this as the Californian Ideology. Its main concept is that of libertarian individualism empowered by technology, in the context of free market and minimal state involvement. The articles on freedom to connect, free flow of information and “Fundamental Equality” all speak to the libertarian and individualist elements of the Principles, while the ownership of information and open platforms speak to market liberalism. At a meta level, the very need for Facebook to develop these principles and terms of service, rather than implement those that governments require them to, alludes to the minimal state involvement in all this.

While all articles work in synergy, we consider Article 4 on “Fundamental Equality” as the most relevant to hate speech:

Every person—whether individual, advertiser, developer, organization or other entity—should have representation and access to distribution and information within the Facebook Service, regardless of the person’s primary activity. There should be a single set of principles, rights, and responsibilities that should apply to all people using the Facebook Service. [Facebook Principles, emphasis added]

This directly addresses the question of fair and equal treatment among all Facebook users. Because of Article 4 on “Fundamental Equality,” all mechanisms that deal with users and their profiles treat them in the same manner, whether they are a page of a celebrity or a politician with millions of followers or a personal profile with a handful of friends. Since therefore there are no criteria by which to differentiate among users, Facebook applies an arithmetic understanding of equality among units. Each profile or page is for Facebook a unit that operates under the same rules as other units. This equality loses ethical consistency to acquire arithmetic consistency and is reductionist in terms of the substance of identity, since it does not consider any differences: all identities are reduced to equivalent units. In practical terms, the page/profile of an account that posts materials attacking others and the page/profile of an account that supports others are treated in the exact same manner. This kind of arithmetic equality reinforces the post-racial idea of color-blindness. Facebook, under the argument that it is not “in the business of assessing which group has been disadvantaged or oppressed” as we saw in the opening quote, does not distinguish between users in any way. But this color-blind approach is, in fact, as Goldberg (2002) has argued, blind to people of color.

This understanding of “Fundamental Equality” filters through the remaining instruments of governing Facebook. The Terms of Service ² is the next step down. These begin with numbered headings, where under the first one titled “The Services we Provide,” there are nine bullet points. Fifth in row, after “Help you discover content, products and services that may interest you” and before “Use and develop advanced technologies to provide safe and functional services for everyone” we find the relevant point on hateful contents: “Combat harmful conduct, and protect and support our community”. The text below reads:

People will only build community on Facebook if they feel safe. We employ dedicated teams around the world and develop advanced technical systems to detect misuse of our Products, harmful conduct towards others and situations where we may be able to help support or protect our community. If we learn of content or conduct like this, we will take appropriate action—for example, offering help, removing content, blocking access to certain features, disabling an account or contacting law enforcement. [Facebook, Terms of Service, emphasis added]

There is no direct reference to hate speech but general references to safety, misuse and harmful contents. Of special significance here is the condition “if we learn,” as it alludes to the effort that must go into detection. Until recently, the labor of activating the detection and control apparatus relied exclusively on the targets of hate speech and their allies. However, as we shall see later, Facebook has now shifted toward automated moderation, proactively removing contents that breach its policies. The steps listed afterwards are in order of escalation where we see that “disabling an account” is the penultimate step taken before law enforcement is contacted. That this is considered one of the final steps is a testament to the importance of “Fundamental Equality” and the idea that everyone should have representation. It is only at a deeper level, however, when moving into the Community Standards, that we encounter the first direct reference to hate speech. It is listed under the third category, Objectionable Content—the two preceding ones are Violence and Criminal Behavior and Safety. Overall, it is number 13, somewhere in the middle of the 25 numbered clauses. The clause reads:

Policy rationale

We do not allow hate speech on Facebook because it creates an environment of intimidation and exclusion, and in some cases, may promote real-world violence.

We define hate speech as a direct attack on people based on what we call protected characteristics—race, ethnicity, national origin, religious affiliation, sexual orientation, caste, sex, gender, gender identity and serious disease or disability. We also provide some protections for immigration status. We define “attack” as violent or dehumanising speech, statements of inferiority, or calls for exclusion or segregation. ³

These are the lines along which Facebook governs hate speech. It is neither a priority nor neglected but treated as part and parcel of a generic category of harmful contents. The policy toward harmful contents generally is justified based on safety, while the policy toward hate speech specifically is justified based on intimidation, exclusion, and the potential for violence. “Fundamental Equality” is manifested in the description of the protected characteristics, which are stripped of any historical understanding and particularities. It is noteworthy that there are no references to racism, but only a reference to race as part of a set of protected categories. Race blindness is evident here as all races are seen as equivalent and equally protected. While of course this is right in principle, in practice it fails to recognize the history of systematic oppression of racialized people. The implications for racist hate speech and user behaviors around this are clearer when we look at the way in which Facebook operationalizes and enforces these policies.

The Operational Dimension: Enforcing Policies

While the above analysis focused on the perspective of Facebook, we now shift to the perspective of the user to understand how these policies are enforced. Every single piece of content, be it a post or a comment below a post, offers users a certain number of possible actions. To report a post, users must go through the sequence of actions detailed in Figure 2. First, users must click on the three dots and then choose one of the options shown on Figure 2. If users are unhappy with the post or comment, the first option is to hide it, then temporarily stop seeing posts from the page or profile, then unfollow the page/profile. Reporting the post appears as the last option.

OPEN IN VIEWER

Figure 2.

Sequence of actions in reporting.

This sequence, which contains no less than five steps, indicates two things: firstly, that users are tasked with applying their own judgment to make the decision to report a piece of content and determine the basis on which this report is justified; secondly, the relative reluctance of Facebook to prioritize content removal. In the first instance, therefore, users are asked to self-regulate their newsfeeds through hiding, snoozing or unfollowing the offending profile. This self-regulation and the sequence of events socializes users in a way that prioritizes individual solutions, as it is the easiest step to follow.

This sequence of action is a direct application of the Community Standards with the addition of the in-between step of information about the process that speaks to transparency. It is important here to note the link between the global enforcement of Community Standards and Article 4 of the Facebook Principles on “Fundamental Equality” along with the list of protected characteristics devoid of any historical reference. According to our informant from Facebook, this is because otherwise the policies could not be operational, that is, acted upon:

If any individual reports any piece of content and there is an attack on any of the protected categories it doesn’t matter who is perpetrating it, it will be removed [. . .] we would become too subjective if we start thinking which group is a minority, which minority deserves more protection [. . .]. We would have to be thinking about the history of the groups and how do you do that? It would create an unbalanced playfield, and to create a balanced field is very important to us. (Interview with Aoibhin Kelleher)

It is of no consequence therefore if a piece of content is against historically oppressed groups or against those who were responsible for the oppression. There is no room for any differential treatment, as this would be against both the idea of “Fundamental Equality” and against the procedural enforcement of the policies. This is considered by Facebook a fairer approach, as is evident in responses to reports. Figure 3 shows both a response when contents are not taken down, and a response when contents are removed.

OPEN IN VIEWER

Figure 3.

Responses to reporting contents.

In the first response, which is about contents that were not removed, Facebook makes clear that the Community Standards are the same in order to keep the process fair. This is both an ethical and an operational decision taken by Facebook that is wholly compatible with a liberal governmental rationality and its emphasis on individuals and individual freedom and consequent disregard of the tenacity of social groups and forces of history.

In the last few years, Facebook is increasingly using automated systems to detect and remove contents. In its Transparency Report of 2019, Facebook notes that AI systems are now accurate enough to remove contents automatically. Specifically, these tools work by giving flagged contents a score according to their similarity to contents previously removed by human moderators because they violated the Community Standards (Facebook 2019). In 2019, 80.2% of all hate speech contents removed were detected and deleted using these AI systems. Other contents which are flagged as potentially violating the policy but did not receive a score high enough for automatic deletion are sent for review by human moderators. While Facebook has still not developed a metric to measure the prevalence of hate speech on its platform, it has reviewed and acted upon 7 million pieces of content in the three months of August to September 2019, an increase of almost 50% from the previous quarter, when it took action on 4.4 million pieces of contents. This increase is due to the improvement of its AI systems. As Facebook does not differentiate among types of harmful contents, there is no information as to how many of these posts were removed for racist hate speech.

This shift to automatic detection and deletion falls squarely within the parameters of the operation of Facebook as a technology and innovation company. It further addresses the problems associated with human moderation, and the negative publicity Facebook has been receiving for exposing moderators to high levels of toxicity. Facebook has placed considerable emphasis and investment on these systems, as Zuckerberg has indicated in his Blueprint for Content Governance:

“The single most important improvement in enforcing our policies is using artificial intelligence to proactively report potentially problematic content to our team of reviewers, and in some cases to take action on the content automatically as well.” (Zuckerberg 2018, unpaginated)

In this turn toward automation, Facebook reveals two elements: firstly, it is expanding its editing system, where it decides what can and cannot be seen by the user—until recently this system was composed only by Facebook’s newsfeed algorithm. Secondly, hate speech contents are not deleted but stored and used to train machine learning tools. While Gillespie (2018) has noted his scepticism as to the accuracy of the AI systems of Facebook, because of the fluidity of culture and language and the adaptability of users, the continuous enrichment of Facebook’s hate speech database ensures that the systems are constantly trained on current materials. In this sense, the process of removing data from newsfeeds is a productive process.

Additionally, Facebook must no longer choose between keeping problematic contents online or removing them. It had long indicated that borderline cases were proving very difficult to deal with. These are contents that do not fall clearly within the hate speech definition, but which are not too far from it either. While they do not qualify for removal Facebook does not want them proliferating, because borderline contents may “degrade the quality of our services” (Zuckerberg 2018, unpaginated). According to Zuckerberg (op. cit., 2018), Facebook’s internal research suggests that borderline contents attract more engagement both from people who like the contents and from those who do not. Facebook has therefore trained its AI systems to detect these contents and distribute them less—a variant of the practice known as shadow banning, that is, not showing certain contents in newsfeeds but not removing them. Of significance here is not only that this is Facebook’s preferred solution as opposed to moving the lines of what is acceptable. It is also that Facebook has developed this approach as a means of re-socializing users. Zuckerberg (2018) refers to this as the “incentive problem”: if borderline contents get more engagement, their creators are incentivized to produce more. By removing the incentive through reducing the distribution of these contents, Facebook believes that it will “create a healthier, less polarized discourse” (Zuckerberg, 2018). In this manner, it seeks to train not only AI systems, but also users, by rewarding with engagement only certain kinds of contents.

We have already noted that hate speech is not seen as any different from other types of problematic contents, such as nudity and unauthorized sales. For Facebook, the issue of racist and other forms of hate speech is an operational problem to be resolved, not a question of social justice. Rather than looking at racist hate speech as part and parcel of racialized social structures and following an approach that specifically addresses the needs and specificities of groups that have been historically disadvantaged and oppressed, Facebook considers it part of a broader category of equivalent problematic contents. In these terms, Facebook applies a post racial understanding of race and racism, which is essentially a denial of the existing reality of racism. Moreover, in doing so Facebook strengthens the application of this frame as users in all parts of the world are conditioned by and socialized with this. Because it is not concerned with the socio-historical parameters of hate speech against racialized people, Facebook has defined it as involving three types of contents that vary in severity but which can be clearly made “operational,” ready to be employed across all contents by human and automated systems: do contents contain dehumanizing language; statements of inferiority; calls for exclusion? If the answer is affirmative, they are removed, if not they remain, if they are borderline, they will be suppressed. All remaining effort is put on enforcement. This is a point that Facebook has made time and again: there is no need to expand the definition, but rather to enforce policies:

We build specific systems to address each type of harmful content [. . .] This is a massive investment. We now have over 35,000 people working on security, and our security budget today is greater than the entire revenue of our company at the time of our IPO earlier this decade. All of this work is about enforcing our existing policies, not broadening our definition of what is dangerous. If we do this well, we should be able to stop a lot of harm while fighting back against putting additional restrictions on speech. (Zuckerberg 2019)

Enforcement of policies through automated systems, evidently a key process for Facebook, is not only oriented toward cleaning newsfeeds and promoting a “healthier discourse”; it is further a means by which more data is collected, stored and processed. The productivity of the process is central to rendering this the preferred solution. While human moderation can remove and store contents, these are not as useful as when they are used to train AI systems. In the checks and balances involved in multinational corporations the usefulness of any process beyond its immediate application is key to adding value and contributing to efficiency.

This analysis showed that the liberal individualist framework of “fundamental equality” found at the center of Facebook’s content policies is key to understanding the platform’s approach to problematic contents. Because users are individualized, de-historicized and understood as lone units, there are no distinctions made based on race, historical oppression and systemic discrimination. Facebook refuses to become involved and take a side against racism in an explicit and unequivocal manner. Rather, it employs a post racial, color-blind approach, in which all races are equivalent, and racist hate speech is just another type of problematic contents, such as spamming and nudity. These issues, that clearly belong to very different orders, are dealt with in a very similar manner. Moderators and automated systems are used to remove or downgrade them. The treatment of these contents further socializes users through a system of rewards (visibility of their contents) and punishment (removal or downgrading). In this manner, users develop adaptive responses to these content policies, leading to the kinds of flexible racism identified by among others, Matamoros-Fernández (2017) and Lentin (2016, 2020). At the same time, the ideological frames of liberal individualism and the associated race-blind approach to racism travel and spread across the world where Facebook is used.