Countering Hate Speech on Facebook: The Case of the Roma Minority in Slovakia

Abstract

This article explores hate speech against the Roma in Slovakia on Facebook between April 2016 and January 2017 and the impact of fact-checking and personal experience strategies in countering hate speech through a quasi-experimental research design. It examines how the Roma were constructed and how discussion participants reacted to our pro-Roma interventions. The research sample consisted of 60 Facebook discussions (with more than 7,500 comments) on Roma-related topics posted by the profiles of various members of the Slovak Parliament and the most popular online news media outlets. Qualitative content analysis revealed that the Roma in Facebook discussions were constructed primarily in a negative sense, as asocial criminals misusing welfare benefits. This study demonstrated that Facebook discussion participants presenting anti-Roma attitudes did not use any research evidence to support their constructions. It also demonstrated that pro-Roma comments encouraged other participants with a pro-Roma attitude to become involved.

Keywords

hate speech Facebook social media Roma counterspeech strategies

Hate speech on social media is a widespread phenomenon that affects a large portion of Internet users. Findings from a cross-sectional study (Keipi, Näsi, Oksanen, & Räsänen, 2016) conducted in the United Kingdom, United States, Finland, and Germany show that, on average, 42% of 15- to 30-year-olds are exposed to hateful material online, mostly on social media such as Facebook, YouTube, and Twitter. Similarly, according to a study conducted by Velšic (2016), 69% of young Slovaks between 18 and 39 experienced hate speech online, mainly on social media or on Internet discussion forums.

Hate speech can be openly or covertly spread online by politicians or other public figures, which can make the language more acceptable to the general public. Politicians can even use hateful ideas feeding on irrational fears based on prejudices on social media as an electoral strategy to win the popular vote and the presidential or parliamentary seats. For instance, Donald Trump’s remarks on women, Muslims, Blacks, ethnic groups, and people with disabilities included sexist, racist, or xenophobic insults suggesting inferiority or a set of negative characteristics of these groups (Ott, 2017). The leader of the Slovak far-right political party, Kotleba People’s Party, Our Slovakia (Ľudová strana—Naše Slovensko), which was elected to the Slovak Parliament in 2016, Marián Kotleba and his sympathizers regularly openly claim or covertly suggest that the holocaust did not happen or that the Roma minority is inferior, misuses the social system, and is too lazy to work (Petková, 2016). The repetition and high frequency of occurrence of hate speech in society creates a broad societal acceptance of the idea that some groups or minorities really are inferior and should be treated accordingly (Nielsen, 2002; Tirrell, 2012). In this respect, social media, such as Facebook and Twitter, play a crucial role as communication channels.

The concept of hate speech, both off-line and online, includes different characteristics associated with abusive content. Hate speech does not present only a verbal conduct but, according to Simpson (2013), also other symbolic and communicative actions that purposefully deliver a message of inferiority or intense antipathy toward a member (or members) of particular social group on the basis of this membership. Therefore, the concept of hate speech that is used in this article refers to different forms of communication, both verbal and nonverbal. Regarding online hate speech, Hawdon, Oksanen, and Räsänen (2017) argue that the core of online hate expresses hatred toward a collective and in this respect it targets characteristics of a collective, for example, race, ethnicity, gender, religion, sexual orientation, and others. They further claim that the orientation of hatred toward a collective and its features rather than an individual is what differs hate speech from cyberbullying (Hawdon, Oksanen, & Räsänen, 2017). Therefore, hate speech includes various slurs and epithets and extremists’ religious and political statements and displays of hate symbols (e.g., swastikas, burning crosses, or numbers symbolizing hate ideologies; Simpson, 2013) with the purpose of vilifying certain social groups.

Hate speech on social media in particular can include hateful posts, comments, or memes (images or videos), posted publicly or in closed groups in different social media platforms such as Facebook or Twitter or private messages sent to the inboxes of targeted individuals or groups. Foxman and Wolf (2013) point out that hate speech online can be easily spread. Hence, virtually any person with access to the Internet is potentially able to create, publish, and make viral any hateful material affecting large numbers of people in a very short time period.

Importantly, hate speech may lead to a detrimental impact on the targeted groups or individuals in different ways. Hate speech has been criminalized in various countries, and much work has been undertaken to prove the harmful impact scientifically. Researchers in various fields (e.g., medicine, psychology, social science, and political sciences) have delved into describing the harms. Research has proven that hate speech causes health and psychological harms (e.g., Boeckmann & Liew, 2002; Delgado & Stefancic, 2004; Maitra, 2012; Nielsen, 2002), poses a threat to democracy in terms of silencing minorities and suppressing their political participation in democratic processes (Maitra, 2012; West, 2012; Young, 2000), contributes to advocacy for exclusionary policies (Yong, 2011), and may incite violence (Dharmapala & McAdams, 2005; Fyfe, 2017).

The question about why particular social groups are generally more often targeted by hate speech than others remains a conundrum. In this article, we identify with the social constructionist interpretation of this conundrum (Burr, 2015; Gergen & Gergen, 2003; Surdu & Kovats, 2015). Social constructionists argue that a specific kind of person as well as the idea of that person are socially constructed. By constructing an idea or category of a person (such as the Roma, migrants, LGBTI, or persons with disabilities), the individual himself or herself is constructed as a particular kind of person. Hence, through practices and interactions with things, other people, and institutions, the ideas about particular groups of persons may have a tangible impact on their lives (Hacking, 1999). Social groups, which are perceived as different to one’s own, for example, based on their skin color, sexual orientation, or mother tongue, become more often targeted with accusations that all members of these social groups are born with some particular, less valuable, and/or threatening characteristics and behaviors. Dirtiness, smell, criminal and aggressive behavior, and promiscuity are just a few examples of characteristics and behaviors that are often ascribed to these social groups as something inherent, unchangeable, and essential to all its members (Castañeda, 2014; De Cecco & Elia, 2013; Kováčová, Miškolci, & Rigová, 2017; Stăiculescu & Gherasim, 2013). The processes of creating and reiterating these social constructions occur in all possible social environments—through mainstream media, social media, artistic or scientific artifacts, or just everyday common interactions with work colleagues, family members, friends, civil servants, and others. The omnipresence of these messages about what certain people are like and how they behave makes these characteristics and behaviors appear true, taken-for-granted, and unquestionable (Foucault, 1977, 1979).

Nevertheless, not everyone who is exposed to hate speech online experiences harm. According to Hawdon et al. (2017), both those who intentionally and accidentally encounter hate speech may not be victimized and, thus, experience its negative consequences. Regarding this, not all social groups are evenly exposed to hate speech and are likely to be victimized. Certain attributes and patterns of behavior increase the likelihood of being exposed to hate speech. For example, Costello, Hawdon, Ratliff, and Grantham (2016) demonstrate that Internet users with higher education and a greater proclivity toward risk-taking are more likely to be exposed to hateful material, while those with a greater level of trust in the federal government, African Americans, and people with immigrant background are less exposed to negative material, although most hateful material is oriented against race and ethnicity. Costello et al. (2016) further explain that members of minorities and stigmatized groups intentionally avoid cyber environments with hateful content. According to Räsänen et al. (2016), young people who produce hate material themselves, who intentionally seek hate material, who worry about being victimized, and who experience victimization off-line are more likely to be victimized by hate speech. The likelihood of being targeted by hate speech online is also associated with routines just like using social networking services and visiting hostile environments (Costello, Hawdon, & Ratliff, 2016). Additionally, the occurrence of hate incidents both online and off-line is closely related to exceptional events such as terrorist attacks (Burnap & Williams, 2015; Byers & Jones, 2007). In these cases, hate speech online is directed toward a social group, which a perpetrator or a group of perpetrators identify with (Burnap & Williams, 2015). All these findings serve to predict who might be the most vulnerable to hate speech online and when these incidents tend to occur.

Scholars and activists in the field of hate speech, particularly online hate speech, suggest a variety of strategies and tools to reduce its prevalence on the Internet. These strategies may be categorized based on different environments, particularly (1) educational systems, (2) cultural events and community activities, (3) mass media, and (4) social media and Internet discussion forums. These strategies include educational and training activities, developing critical thinking skills to help unmask prejudices and hoaxes in children, youth, or journalists; organizing large media campaigns; and becoming actively involved and closing down various profiles on Facebook (Banks, 2010; Citron & Norton, 2011; Durrheim, Greener, & Whitehead, 2015; Foxman & Wolf, 2013; Molnar, 2012; Sponholz, 2016; Titley, Keen, & Foldi, 2012).

Nonprofit organizations, public administration institutions, mass media, politicians, public figures, and Internet service providers, any individual social media user may join endeavors to counter hate speech, as well. Besides engaging in reporting, flagging, and down-voting hateful comments or up-voting positive ones, users may intervene and actively interact with those who spread hate speech. According to Atchison (2000), intervention in the online world represents social control, which means that Internet users impose informal norms (as well as punishments) on others in case somebody violates the rules of the domain. For instance, they may show that certain behavior is not acceptable. Therefore, each Internet user can impose informal rules and at the same time be subject to them. As Atchison (2000) further states, social control is especially important in the absence of formal authority or control which would exercise power and respond to unacceptable behavior.

There are two main strategies for countering hate speech. First, the user may counter hate speech with factual information that refutes prejudice and myths (Foxman & Wolf, 2013; Kriplean, Bonnar, Borning, Kinney, & Gill, 2014). Second, the user, when engaging in online discussions, may use personal experience to refute stereotypes and hate. One can use personal experiences of interacting with minority members to highlight the fact that the negative characteristics ascribed to the targeted group by society are actually not exclusive to this group but also appear among members of the majority (Hatefree Culture, 2017).

Nevertheless, engaging in counterspeech to tackle hate online brings several difficulties to the intervenors. Research conducted by Costello, Hawdon, and Ratliff (2016) shows that those who enact social control in the online world are more likely to be exposed to hateful material and targeted by hate. Thus, confronting users who spread hateful posts is associated with being victimized by hate speech, since intervening users are more exposed to attackers and hateful material.

Despite the aforementioned research, our knowledge of different aspects of hate speech on social media platforms is currently still rather limited. One explanation for this is that social media are, from a historical perspective, a recent phenomenon. While the topic of hate speech against people of color, women, and, recently, even migrants and Muslims receives some scientific attention, hate speech against other groups such as the Roma is still rather unexplored, especially in the context of social media. Existing research on social media also mostly covers English-speaking discussions and deals with less widespread languages only marginally (e.g., Hrdina, 2016). Moreover, there is a lack of knowledge about the effectiveness of strategies and tools to counter and reduce hate speech online. The aim of this research article is to fill this gap by exploring anti-Roma hate speech on Facebook in the Slovak language context in particular. It explores the impact of the two selected strategies to reduce hate speech on Facebook, that is, fact-checking and personal experience strategies.

This research project focuses on the social group of the Roma because in the Slovak sociopolitical context in various polls (e.g., Vašečka, 2009) that measure attitudes toward minorities, the Roma are consistently perceived negatively. In 2013, an opinion poll with a representative sample of the general public in Slovakia demonstrated that almost two thirds of all respondents felt strong aversion against the Roma. These respondents selected the most extreme option on the scale of social distance, that is, they expressed their wish to have the Roma “deported” outside Slovakia (Macháček, 2013). The Roma live in various countries of the European Union (EU), primarily in Central and Southeastern Europe. In Slovakia, the latest estimate of the Roma population is 402,741 inhabitants (Mušinka, Škobla, Hurrle, Matlovičová, & Kling, 2014), which is approximately 7.44% of the entire population of Slovakia. Thus, the Roma represents the second largest minority after the Hungarians. The EU Agency for Fundamental Rights (FRA, 2016) pointed out that in the EU, 80% of Roma live below the at-risk-of-poverty threshold of their country of citizenship; every third Roma lives in a household without tap water access; and 1 in 10 in a household without electricity. Specifically, in Slovakia, 87% of Roma live below the at-risk-of-poverty threshold. Their access is significantly constrained in the areas of education (ethnic segregation, low educational results, and early school leaving), labor market (low employment rate), and health (FRA, 2016).

This research project primarily aims to uncover how discussion participants from Slovakia on Facebook perceived (or constructed) the Roma and the practices and policies related to the Roma. It also investigates how presenting a positive construction of the Roma or, to put it more precisely, how contributing with pro-Roma comments on Facebook, impacted discussions. Translating these broad aims into research questions, this article attempts to address the following questions:

Research Question 1: How are the Roma constructed on Facebook?

Research Question 2: How are the current policies and practices toward the Roma constructed on Facebook?

Research Question 3: How are the desired future policies and practices toward the Roma constructed on Facebook?

Research Question 4: How does the proportion of anti-Roma and pro-Roma comments change if we purposefully enter discussions with pro-Roma comments?

While the first three questions are primarily of qualitative nature, the last one is quantitative. Our hypothesis was formulated in line with the last research question:

Research Question 5: If we intervene into Facebook discussions using various pro-Roma comments, it motivates other participants to join the discussion in order to present more pro-Roma comments. In other words, the proportion of pro-Roma comments is higher in the intervention group than in the control group (while our pro-Roma comments are not included in the calculation).

Method

To answer the abovementioned research questions and test the hypothesis, we used a quasi-experimental research design (Creswell, 2009; Patten & Newhart, 2017) that compares two sets of Facebook discussions. The first set included 30 Facebook posts that related to the topic of the Roma and were posted by one of several high-level politicians in Slovakia or by the most popular mainstream news media (MEDIANSK, 2015) and the discussions under these posts. In each of these 30 discussions, we contributed at least 10 comments while using two argumentative strategies: fact-checking and/or personal experience (see Online Appendix with examples of these interventions). We designated this group as the “intervention group.” Additionally, we also identified and analyzed 30 Facebook discussions from similar sources, which we did not contribute to. We designated this latter set as the “control group.” All 60 discussions occurred on Facebook in the period between April 2016 and January 2017; thus, we started intervening in April 2016, and due to the very low frequency of Facebook posts related explicitly to Roma, it took us 10 months to collect all 60 discussions.

Regarding the intervention process, each intervening comment was slightly or significantly different depending on the content of the hateful comment that preceded it. For instance, if the hateful comments included misinformation about the level of social benefits in Euro, we intervened to point out and correct the misleading information. If they included prejudiced statements about Roma parents not being interested in the education of their children, we intervened accordingly to counter this prejudice. We commented in two possible ways, either using the personal experience strategy or using the fact-checking strategy. When using the former, we intervened by claiming that we personally know a great number of Roma parents who are particularly attentive to their children’s educational results and aspirations. If using the latter, we intervened by pointing out a particular research study relevant to the prejudice, and, in simple words, we summarized its findings and provided a link to the study. If one hateful comment included more than one piece of misinformation or prejudiced statement, we addressed the one that seemed to be stressed the most. It was feasible to intervene using the personal experience strategy with practically any kind of hateful comment. However, it was not always possible to use the fact-checking strategy if no research existed on a particular topic (e.g., Roma are criminals, Roma are asocial) or when discussants proposed extremist policies (e.g., deportation and killing). Overall, we attempted to use both strategies relatively evenly.

In order to protect ourselves from personal attacks from other discussion participants, we intervened with the use of a pseudonymous Facebook profile. The profile represented a fictitious young, White, male student of the Faculty of Education, Comenius University in Bratislava. There was no picture published of the fictitious person.

We chose to intervene below Facebook posts from various high-level politicians or mainstream news media due to the research findings of Hrdina (2016) who discovered that the most common hate speech on Facebook is not produced by extremist groups or extremist personalities but primarily emerges below posts that promote or share “neutral” articles related to the targeted social group that are published in various mainstream news media. The Facebook profiles of relevant news media also publish posts that promote these articles, and various high-level politicians merely share these articles while commenting on them with their original posts, which may include hate speech.

Table 1 shows the sources of posts included in the intervention and control group. As is clear from Table 1, news media sources slightly outnumbered the Facebook discussions that were initiated by the politicians.

Table 1.

Sources of Facebook Discussions Studied.

Source	Control Group (No. of Posts)	Intervention Group (No. of Posts)
Aktuality (online news media)	9	15
Topky (online news media)	8	2
Boris Kollár (Member of Parliament for We Are Family—Boris Kollár)	5	5
Milan Krajniak (Member of Parliament for We Are Family—Boris Kollár)	3	2
Lucia Nicholsonová (Member of Parliament for Freedom and Solidarity)	2	0
Alojz Hlina (former Member of Parliament, Chairman of Christian Democratic Movement)	2	0
Milan Uhrík (Member of Parliament for Kotleba—People’s Party Our Slovakia)	1	0
SME (print and online news media)	0	3
RTVS (TV, radio, and online media)	0	2
Lucia Žitňanská (Minister of Justice of the Slovak Republic, Most-Híd)	0	1
Total	30	30

Note. Kováčová, Miškolci, and Rigová (2017).

Starting in April 2016, we undertook the daily monitoring of the Facebook profiles of all members of the parliament of the Slovak Republic and the most popular online mainstream news media. Since the frequency of Facebook posts that mentioned the Roma or Roma-related policies turned out to be extremely low, we intervened in every single Facebook discussion that we noticed as long as at least one researcher was available for the task. Limitations on our availability meant that sometimes we missed a relevant Facebook discussion. After intervening in 30 Facebook discussions in January 2017, these missed discussions within the intervention period were identified using ZoomSphere (a social media management tool) and were instead incorporated into the control group.

To make sense of the large amount of data that were gathered, we utilized qualitative content analysis (Lacy, Watson, Riffe, & Lovejoy, 2015). In line with this research method and with the aforementioned research questions, we inductively created a number of categories that either explicitly or implicitly had a similar meaning. The creation of these categories was also partly informed by previous research about the construction of Roma identity in mainstream media, policies, or the general public (Kováčová et al., 2017; New, 2012; Stăiculescu & Gherasim, 2013; Surdu & Kovats, 2015; Vermeersch, 2012).

Since three researchers participated in conducting this study, we put into place procedures to better synchronize the intervention and analysis process (Lacy et al., 2015; Long & Johnson, 2000). First, all three researchers discussed in great detail the intervention strategies, agreed on how to intervene, and mutually checked various comments before they were posted. Second, at the very beginning of the coding process, (February 2017) to enhance its reliability, we randomly selected four samples of Facebook discussions that were included in the intervention group, and all three researchers independently coded the text after a prior discussion about the potential codes. We then compared results, discussed the differences, and agreed on the codes. The same process was repeated in March 2017, with data from both the intervention and the control groups. Third, during the entire analysis period (February 2017–May 2017), all three researchers discussed cases whenever a new potential code emerged from the data, while any other ad hoc coding dilemmas were discussed among all researchers as they arose.

With regard to the quantitative aspect of the study, and testing our hypothesis in particular, the first and third authors coded the data. To be more specific, all 7,586 comments in the 60 Facebook discussions included in this study were coded as “pro-Roma,” “anti-Roma,” “mixed,” or “irrelevant.” The first author coded all comments within the 30 discussions of the control group (3,559 comments), and the third author coded all comments within the 30 discussions of the intervention group (4,027 comments). To check for the interrater reliability, we randomly selected three discussions out of the control group (480 comments) and three discussions out of the intervention group (512 comments), which were coded by both coders. From this subset of coded comments (992 in total), which were coded as nominal data, we calculated the interrater reliability of the two coders while using the Krippendorff’s α coefficient, and it was at the level of .779. In the instances in which the coders did not match, final codes were used in the first author’s codes for the control group and the third author’s codes for the intervention group.

Results

Negative Constructions of the Roma

In addressing our first three research questions (i.e., the construction of the Roma and constructions of current and future practices/policies toward the Roma), we did not code the 7,586 individual comments but merely the emergence of particular constructions in the individual discussions. Therefore, the construction of, for example, “Roma as asocial”¹ appeared a maximum 60 times in the entire sample and not 7,586 times, as we counted the emergence of one construction per one discussion. The main rationale for this is that the Facebook discussions that were studied differed in both their length and the involvement of particular discussion participants. There is a qualitative difference in the value of comments if they are authored merely by one participant versus those authored by many participants. For instance, in two discussions, both with 55 comments, the former may have 30 comments authored only by one participant claiming that “Roma are asocial,” while in the latter discussion, the construction of “Roma as asocial” may be presented by 30 different participants. This type of domination occurred in several Facebook discussions. When dealing with a sample as large as 7,586 comments, it would be extremely demanding, if possible at all, to connect the particular participants with particular comments.

First of all, we distinguished negative and positive comments related to the Roma. Predictably, the former dominated in terms of their occurrence. With regard to the types of anti-Roma comments (Figure 1), no participants were observed to use a fact-checking strategy. They either used a personal experience strategy or merely presented their ideas as claims of truth or as something taken-for-granted or unquestionable. The emergence of various types of anti-Roma comments was very similar in both the intervention and the control groups; hence, Figure 1 presents the results for the entire sample of the 60 Facebook discussions.

Figure 1.

Types of anti-Roma comments in both the control and intervention group. Source. Kováčová, Miškolci, and Rigová (2017).

When focusing on our first research question, about how the Roma are constructed on Facebook, Figure 1 shows the various negative descriptive characteristics or behaviors attributed to the Roma, such as being:

asocial (55 of 60);

unwilling to work (48 of 60);

criminals (43 of 60);

likened to animals (e.g., rats and pigs), insects (e.g., cockroaches), or things (34 of 60);

too fertile and having too many children (32 of 60);

unwilling to study (29 of 60); and

destroyers of their dwellings (22 of 60).

Each of these categories was operationalized as merely aggregating descriptions of the characteristics or behaviors of Roma. For instance, if the discussion participants spoke of the Roma as “lazy,” “unwilling,” or “not wanting” to work, all these comments were categorized under a descriptive category of “unwilling to work.” The same process was applied for all the other descriptive categories.

When the discussion participants presented these characteristics or behaviors of the Roma, they did so as if these characteristics were something essential and inherent to all or the vast majority of the Roma; that the Roma were unchangeably “nonadjustable” to the majority. They claimed for instance “The Gypsies will never improve, I do not believe it.” or “We may engage the Roma into any programs for common people but we will not manage to change them anyway.”

In some cases, even Roma participants entered the discussions and disclosed their Roma identity, while at the same time, they presented some hateful comments toward other Roma. For instance, “I am a Roma and I hate those good-for-nothings. They are given free apartments and they demolish them.” In line with this comment, anti-Roma commentators sometimes admitted that there are two kinds of Roma: the “adjustable” and the “nonadjustable” ones. They, however, frequently stated that the former constituted a very negligible minority or exception.

Negative Constructions of Current Practices and Policies Toward the Roma

Regarding the second research question, about the current practices and policies toward the Roma, one of the most significant and surprising findings of this study was that the occurrence of the negative construction of “Roma are privileged” was as high as the construction of “Roma are asocial” (Figure 1). In other words, the occurrence of this construction of what the Roma are like was as frequent as one concerned with what current policies and practices toward the Roma are; both emerged in 55 discussions of 60. Within this construction, the participants presented the idea that the Roma received everything for free (public transport, social benefits, meals at school, medication, and even housing) and that they were privileged even in criminal matters, whereby police ignored crime committed by the Roma in order to avoid being accused of racism. The most frequently mentioned policy was that “Roma receive too high social benefits” (49 discussions of 60).

Participants usually denied the existence of discrimination against the Roma. If they did acknowledge that such a phenomenon occurred from time to time, they justified it as something understandable considering “what the Roma are like or behave like.” For instance, one participant defended employment discrimination against the Roma: “And who would give them a job when they only misuse the system and are not interested in working?”

The participants ignored the fact that Roma exclusion is influenced by a number of cultural, social, political, and psychological factors, specifically at the international, state, regional, or municipal level, and that the impact of state policies is not straightforward and direct (Bacchi, 2009; Ball, 1994, 2012). The participants oversimplified social “reality” without demonstrating an awareness that a social change toward greater inclusion of any marginalized social group is a long-term process that requires concentrated effort at all policy levels and encompasses all spheres of social life, for example, employment, education, housing, health, culture, and so on (McMichael, 2011). For example, one participant expressed her or his view as: “Those who want, really work.”

While failing to see these social phenomena as complex matters, if not totally blaming the particular marginalized social group, the participants tended to blame the politicians or nongovernmental organizations as merely misusing financial resources for their self-interests. For instance, one participant blamed the international policies of the EU: “What would Brussels have said, which supports garbage and in Slovakia a bunch of people live off the Gypsies.” He or she also used the label “Gypsy” (Cigán), which is a very widespread derogatory term signifying the Roma.

Negative Constructions of Future Practices and Policies Toward the Roma

Regarding the third research question, participants with a negative view proposed either paternalistic or extremist approaches as the most desirable practices and policies toward the Roma for the future. These constructions are related to the oversimplistic perception of social phenomena. In the paternalistic comments, which emerged in 27 of 60 monitored discussions, participants positioned themselves in the superior position of knowing what is best for the Roma and proposed a sanction-based approach instead of an empowerment-based approach. For instance, one discussant proposed: “It is necessary to harshly limit social benefits, and to connect them to works for towns or municipalities. It is necessary to regulate allowances for children, since some of them made businesses out of these.” Another discussant claimed: “The Roma should not receive any money. Just as a child cannot deal with a knife, so I don’t hand it to him/her, because he/she would hurt him/herself - this [the Roma] is a similar case.” In the former statement, the discussant proposes “harsh” policies toward the Roma such as forcing them to work for municipalities in order to deserve their social benefits, while in the latter, the Roma are likened to children, which implies they should be strictly directed as such. Both of these statements can be interpreted as paternalistic in the sense that they portray the Roma as subordinates or dependents who should be considerably stripped of their autonomy and liberty.

In another discussion, participants did not shy away from proposing extremist policies and practices, such as: “Authorization to kill, please.” or “Let’s push them into the wagons without windows, so they stamp on each other and send them to the concentration camps and release the gas!!!” The category of extreme policy and practice proposals was operationalized to include any calls for instigating physical violence (beating up, punching, kicking, etc.), killing (shooting, gas poisoning, etc.), and forced deportation abroad or to concentration labor camps. These extremist proposals emerged in more than half of all the monitored Facebook discussions (37 discussions of 60). The frequency and severity of these comments within particular discussions varied in accordance to the content of the Facebook post that initiated the discussion. The highest number of extremist comments occurred under a post describing attack on a tourist by a “Roma gang.”

In terms of the operationalization of these categories in relation to practices and policies toward the Roma, these categories differ from the previously mentioned ones in the level of their descriptiveness and abstractness. In comparison to the constructions of the characteristics and behaviors of the Roma (e.g., Roma as asocial, criminals, and unwilling to work), which were rather descriptive categories, the constructions of policies and practices toward the Roma (e.g., Roma as privileged or extremist and paternalistic policy proposals), there were more analytical and abstract categories. In the former case, the discussion participants even used the key terms (e.g., “criminals” and “asocial”), which we used as the coding categories. Nonetheless, the terms “paternalistic” or “extremist” were never used by discussants, but we used them as analytical categories (see Figure 1).

Positive Construction of the Roma and Practices and Policies Toward the Roma

When considering the comments, which defended the Roma or portrayed them in a more positive light, we merely focused on the 30 Facebook discussions of the control group. The discussions within the intervention group were significantly influenced by our positive comments; hence, we were interested in the types of positive constructions of the Roma, which emerged without our interference. While speaking about the pro-Roma comments within the control group, it should be noted that these were almost always reactionary to the anti-Roma comments. Therefore, reactions to the anti-Roma or hateful comments were most commonly categorized into the “personal attacks” group (15 discussions out of 30; Figure 2), which were aimed against individual authors of hate speech without using any arguments. If pro-Roma participants did argue against any prejudices, for example, unwillingness to work; privileges of the Roma; or receiving unfairly high, special, or extra social benefits, the discussion participants used the personal experience strategy in all cases except for only two discussions, in which fact-checking was used.

Figure 2.

Types of pro-Roma comments in the control group. Source. Kováčová, Miškolci, and Rigová (2017).

As an answer to the first research question, regarding how the Roma are constructed, we found that when participants displayed solely positive portrayals of the Roma, they constructed an image of them as hard workers or as common people who were no more asocial or criminal than the majority population. With regard to the second question, regarding the current practices and policies toward the Roma, these were perceived to either enable or support discrimination against the Roma (“Roma are not privileged,” “Roma receive low social benefits,” “Inappropriate policies”). In addressing the third question of what the future policies and practices toward the Roma should look like, the discussion participants merely referred to morals and ethics (“Moral preaching”) without proposing any specific policies.

Hypothesis Testing

Across all 60 Facebook discussions, there were 7,586 comments. The intervention group was comprised of 4,027 comments, of which 362 were our intervening comments arguing in favor of the Roma. The control group consisted of 3,559 comments. Each of these comments was coded with four possible codes:

pro-Roma—Arguing in favor of the Roma and their inclusion, though some of them also included prejudicial or aggressive comments against the majority population.

anti-Roma—Arguing based on prejudices against the Roma.

mixed—Arguing in some aspects in favor of the Roma and their inclusion, but, at the same time, demonstrating various prejudices in other aspects.

irrelevant—Not mentioning the topic of the Roma.

As evidenced in Table 2, irrelevant comments were the most frequent comments in all discussions researched. In these comments, discussion participants often talked generally about the corruption of politicians, about other social problems, or merely contributed with uninterpretable emojis and stickers. Anti-Roma comments were the second most frequent comments in both the control and the intervention group and, in both groups, they vastly outnumbered the pro-Roma comments. Nonetheless, there was one statistically significant difference between the two groups. After excluding the 362 comments that constituted our intervention, the proportion of pro-Roma comments was higher in the intervention group (7.3% with 293 comments) than in the control group (4.5% with 160 comments). This difference is statistically significant at the probability level of .0117. From this result, we can infer that our hypothesis was confirmed: If we enter Facebook discussions with pro-Roma comments, it motivates other followers of those particular Facebook profiles to join the discussion arguing in favor of the Roma as well. This hypothesis was confirmed despite the fact that in our comments we might have used the exact arguments other pro-Roma discussion participants would have liked to use.

Table 2.

Proportion of Pro-Roma and Anti-Roma Comments.

Intervention Group						Control Group
Our Comments	Pro-Roma (Without Ours)	Anti-Roma	Mixed	Irrelevant	Total	Pro- Roma	Anti- Roma	Mixed	Irrelevant	Total
362	293	1,203	78	2,091	4,027	160	1,581	99	1,719	3,559
9%	7.3%	29.9%	1.95%	51.9%	100%	4.5%	44.4%	2.8%	48.3%	100%

Note. Kováčová, Miškolci, and Rigová (2017).

Discussion Participants’ Reactions to the Pro-Roma Intervention

We anticipated that our intervention could have caused four possible reactions:

continuation of hateful comments within the thread;

partially acknowledging the argument from our intervention;

completely acknowledging the argument from our intervention; and

no more comments posted at all within the thread.

The first reaction can be interpreted as an unsuccessful intervention, and it occurred most frequently (Table 3). The second reaction can be interpreted as partially successful, but it occurred rather rarely. The third reaction, the most desirable successful intervention, never occurred in our experiment. The fourth reaction was the second most frequent reaction, and it is the most problematic to interpret. On the one hand, it could be viewed as a success, since it stopped the hate speech and the participant might have been persuaded by our arguments. On the other hand, not reacting could have been caused by many other factors, while the participant might have kept her or his prejudicial attitude.

Table 3.

Frequency of Various Reactions to Our Interventions.

Type of Reaction	Number of Reactions
Continuation of hate speech	225
Partially acknowledging	12
Completely acknowledging	0
No more comments	112
Total	349

Note. Kováčová, Miškolci, and Rigová (2017).

With regard to the first type of reactions, when we intervened with a fact-checking strategy, the participants usually either undermined the research presented or the credibility of the researchers: “I do not accept the research studies as an argument! To deal with something and someone who perhaps not even met a Gypsy.” or “Wiseacres who only sit in offices and know everything, huh? It is useless to comment on this when somebody is directed by statistics and not reality.”

When we intervened with a personal experience strategy, the participants framed the positive experiences with the Roma as exceptions, constituting a very marginal portion of the population: “All these arguments that they want to work but cannot…. How poor they are! 95% of the Gypsies are parasites. I’m afraid 95% of the whites would only hardly be [parasites]” or “Of course there are exceptions, but only a few.” Although the majority of our interventions did not stop the hate speech, the fact that our interventions either motivated other pro-Roma participants to join the discussion or motivated anti-Roma discussions not to continue in their hate speech provides some evidence that it is worthwhile to intervene on Facebook.

Discussion

A crucial aspect of the usefulness of counter hate speech (e.g., increasing the number of pro-inclusive or constructive comments) is based on the assumption that speaking up serves as an open expression that not all participants agree with hateful content. According to Foxman and Wolf (2013), counterspeech may convince other participants that there are other opinions and people who refuse bigotry and hate. This may encourage others to engage in discussions and to counter hatred as well. Regarding this, according to Costello, Hawdon, and Cross (2017), there are several factors that affect the likelihood of enacting social control, particularly having strong off-line and online social bonds and support systems among family members and friends, prior experience with victimization, high self-esteem, and high aversion toward hateful material. The already mentioned collective efficacy also belongs to this set of factors, which means that when individuals see other users intervening online on behalf of the victimized, they are more likely to intervene themselves (Costello, Hawdon, & Cross, 2017). Our research study confirmed this phenomenon even with regard to a particular social group of Roma in a Slovak-speaking Facebook environment.

Nevertheless, counterspeech strategies, especially fact-checking, have their limitations. The fact-checking strategy is based on the assumption that when participants are exposed to information and, particularly scientific data, they will accordingly reconsider their opinions. As we have shown, reasoning and providing people with reliable scientific data does not always result in that. The data show that people tend to overestimate their knowledge of common social phenomena (Fernbach, Rogers, Fox, & Sloman, 2013; Lawson, 2006), and as a result, they refuse scientific evidence because they are convinced that they already have enough information. According to Fernbach, Rogers, Fox, and Sloman (2013), people are unreasonably confident in their knowledge and understanding of public policies on very complex issues such as climate change or poverty, but they may actually know very little and, as a result, have extreme views on these issues. Such high confidence in understanding of social exclusion was also evident in the examined Facebook discussions, where the participants presented their anti-Roma views and even proposed extreme policy solutions such as the withdrawal of financial allowances.

This, however, does not inevitably mean that the strategy of fact-checking is entirely ineffective. Fernbach et al. (2013) claim that this “illusion of understanding” may be addressed by a mechanistic explanation and that an attempt to explain causal processes behind policies may even lead to moderated opinions. The phenomenon of mixed comments (Table 2) and partial acknowledgment of our arguments in the intervening comments (Table 3) illustrate this phenomenon. Discussion participants could be persuaded with solid arguments about some particular prejudices (e.g., the level of social benefits marginalized Roma receive, or their willingness to work), but they presented several other prejudices, which would all need to be mechanistically addressed. Just as removing one layer of an onion still leaves several more layers in place, a similar phenomenon happens with respect to the Roma: Deconstructing one particular prejudice does not bring about a complete change of attitude. The process seemingly requires a mechanistic and exhaustive explanation.

Not only scientific evidence but also emotions spread through Facebook may influence the content of other users’ posts and comments. According to Kramer, Guillory, and Hancock (2014), who conducted an experiment testing emotional contagion on Facebook, reducing positive expressions posted in the News Feed section leads to producing fewer positive posts and more negative posts and vice versa. Kramer et al. (2014) argue that emotional contagion may change other Facebook users’ emotions and occur between Internet users in the social network platforms outside in-person interaction. This finding also suggests that online interaction between people matters in different ways and may substantially change their behavior on social networking platforms.

Nonetheless, to explore the effectiveness of particular strategies to counter hate speech, online is just one aspect of the whole hate speech reduction agenda. Technological aspects of social media platforms as such appear to play a crucial role in determining the frequency of hate speech. Hrdina (2016) points out that the design of Facebook, which is currently based on suggesting content the user is likely to agree with, may lead to the suggestion of pages with hateful material to people who have already expressed a negative opinion, particularly with respect to certain sensitive topics such as migration. As a result, the user becomes frequently exposed to hateful material which may confirm his or her opinions, potentially leading to their fostering of radical views (Liao & Fu, 2013). The same is valid for YouTube recommender system (O’Callaghan, Greene, Conway, Carthy, & Cunningham, 2015). In other words, the structural design of Facebook or YouTube itself may contribute to creating communities of users who are poorly confronted with different opinions and information. Hence, there is some space for social media developers to consider more strategies to combat hate speech besides shutting down particular profiles, such as redesigning the way it essentially operates.

This study confirmed that the constructions of the Roma in social media are similar to the ones presented in mass media and by the general public. To be precise, mass media commonly portrays the Roma as deviants and criminals who are noisy and dirty and who voluntarily remain dependent on social benefits (Castañeda, 2014; Stăiculescu & Gherasim, 2013). Similar constructions are presented by the general public when asked in opinion polls (Macháček, 2013) or in focus groups and interviews (Kováčová et al., 2017; Walsch & Krieg, 2007). Our research study, however, introduced an original finding in shedding some light on the frequencies of these constructions and particularly in showing that Facebook discussion participants presented their frustration about the Roma being privileged just as often as they viewed them as asocial. Such a finding reveals a need to combat hate and prejudice by focusing primarily on explaining why it is reasonable and desirable to affirmatively support and empower the marginalized social groups and that this does not have to be perceived as unjust (Young, 2001).

Despite applying a quasi-experimental research design, the peculiarities of Facebook as a platform must be acknowledged here, since they present some noteworthy limitations upon the validity of this research. In particular:

there were significant differences between the social composition of the “befriended” discussion participants and the followers of the particular source profiles;

there were significant differences in the content of the Facebook posts; while news media posts usually used “neutral” language, various populist politicians such as Boris Kollár or Milan Krajniak demonstrated hate and prejudice in their posts;

there were significant differences in the visibility (number of “shares” and “likes”) and the length of discussions included in the sample pool (the number of comments under each post varied from 35 to 523 comments);

some discussion participants or administrators of the relevant Facebook profiles deleted various comments from the discussions, which distorted discussion in a particular thread;

many discussion participants evidently did not read the comments left by other participants but reacted merely to the main post; hence, they repeated each other’s points extensively within the same discussion;

some very active discussion participants dominated discussions by posting a very high number of comments and replies; and

at times when we did not intervene immediately, the author of the comment that our intervention addressed may have no longer been online, and thus may have become disengaged.

Nonetheless, due to the very low frequency of posts related to the Roma, we did not have the luxury of choosing and restricting our comparison to discussions by the same profiles of a similar length and of similar content. Hence, these limitations were unavoidable.

While bearing these limitations in mind, in exploring the effectiveness of countering hate speech in social media, this research study proved to be innovative in a number of aspects. It demonstrated that discussion participants in this sample who presented hateful attitudes never used, and usually did not respect, research evidence or hard data. It also showed that intervening in Facebook discussions in order to defend the marginalized group has some potential in motivating other participants to support this defense. It also has the potential to stop the participant from presenting any further hate, or at least to partially acknowledge the prejudicial nature of some of their opinions. Nonetheless, much more research needs to be done on the effectiveness of particular counterspeech strategies with regard to various social groups in diverse social and political contexts.

Conclusion

This research study focused on two areas: how the Roma are constructed on Facebook, and how discussion participants react to pro-Roma interventions. It demonstrated that the Roma were primarily constructed in a negative way as asocial; as criminals; or as unwilling to work, study, or look after their dwellings. The discussion participants presented opinions that the Roma are generally privileged, receiving many services (such as school meals, transportation, medicine, accommodation, and social benefits) for free and presented this as a form of injustice to the majority population. They propose to solve this injustice through various paternalistic or extremist policy measures. Furthermore, the Facebook discussion participants demonstrated a very limited and oversimplistic knowledge of social and political complexities in relation to ethnic-based discrimination and exclusion.

When we attempted to counter hate speech against the Roma on Facebook, most often the participants continued in their hate. Nonetheless, something important was happening alongside this: Other participants started to join the discussion in order to defend the Roma as well and in a greater capacity than they would do without substantial intervention. Moreover, commenting in favor of the Roma made a substantial portion of discussion participants stop presenting any more hate. Such findings leave us with hope that it is worthwhile to counter the prejudices and hate as one part of a larger drive toward a greater social equality.

Supplemental Material

Supplemental_Material - Countering Hate Speech on Facebook: The Case of the Roma Minority in Slovakia

Supplemental_Material for Countering Hate Speech on Facebook: The Case of the Roma Minority in Slovakia by Jozef Miškolci, Lucia Kováčová and Edita Rigová in Social Science Computer Review

Footnotes

Authors’ Note

Data are available from the corresponding author upon request at jozef.miskolci@fedu.uniba.sk.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Supported by a grant from the Foundation Open Society Institute in cooperation with the Think Tank Fund of the Open Society Foundations.

Supplemental Material

The supplemental material is available in the online version of the article.

Note

References

Atchison

C. H

. (2000). Emerging styles of social control on the Internet: Justice denied. Critical Criminology, 9, 85–99.

Bacchi

C. L.

(2009). Analysing policy: What’s the problem represented to be? Sydney, Australia: Pearson Education.

Ball

S. J.

(1994). Education reform: A critical and post-structural approach. Buckingham, England: Open University Press.

Ball

S. J.

(2012). Politics and policy making in education. London, England: Routledge.

Banks

(2010). Regulating hate speech online. International Review of Law, Computers & Technology, 24, 233–239.

Boeckmann

R. J.

Liew

(2002). Hate speech: Asian American students’ justice judgments and psychological responses. Journal of Social Issues, 58, 363–381.

Burnap

Williams

M. L.

(2015). Cyber hate speech on Twitter. An application of machine classification and statistical modelling for policy and decision making. Policy and Internet, 7, 223–242.

Burr

(2015). Social constructionism (3rd ed.). London, England: Routledge.

Byers

B. D.

Jones

J. A.

(2007). The impact of the terrorist attacks of 9/11 on anti-Islamic hate crime. Journal of Ethnicity in Criminal Justice, 5, 43–56.

10.

Castañeda

(2014). European mobilities or poverty migration? Discourses on Roma in Germany. International Migration, 53, 87–99.

11.

Citron

D. K.

Norton

(2011). Intermediaries and hate speech: Fostering digital citizenship for our information age. Boston University Law Review, 91, 1435–1484.

12.

Costello

Hawdon

Cross

(2017). Virtually standing up or standing by? Correlates of enacting social control online. International Journal of Criminology and Sociology, 6, 16–28.

13.

Costello

Hawdon

Ratliff

(2016). Confronting online extremism: The effect of self-help, collective efficacy, and guardianship on being a target for hate speech. Social Science Computer Review, 35, 587–605.

14.

Costello

Hawdon

Ratliff

Grantham

(2016). Who views online extremism? Individual attributes leading to exposure. Computers in Human Behavior, 63, 311–320.

15.

Creswell

(2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage.

16.

De Cecco

J. P.

Elia

J. P.

(2013). If you seduce a straight person, can you make them gay? Issues in biological essentialism versus social constructionism in gay and lesbian identities. New York, NY: Routledge.

17.

Delgado

Stefancic

(2004). Understanding words that wound. Boulder, CO: Westview Press.

18.

Dharmapala

McAdams

R. H.

(2005). Words that kill? An economic model of the influence of speech on behavior (with Particular Reference to Hate Speech). The Journal of Legal Studies, 34, 93–136.

19.

Durrheim

Greener

Whitehead

K. A.

(2015). Race trouble: Attending to race and racism in online interaction. British Journal of Social Psychology, 54, 84–99.

20.

Fernbach

P. M.

Rogers

Fox

C. R.

Sloman

S. A.

(2013). Political extremism is supported by and Illusion of Understanding. Psychological Science, 24, 939–946.

21.

Foucault

(1977). The archaeology of knowledge. London, England: Tavistock.

22.

Foucault

(1979). Discipline and punish: The birth of the prison. New York, NY: Vintage.

23.

Foxman

A. H.

Wolf

. (2013). Viral hate: Containing its spread on the Internet. New York, NY: Palgrave Macmillan.

24.

FRA (European Union Agency for Fundamental Rights). (2016). EU-MIDIS II: Second European Union minorities and discrimination survey: Roma—selected findings. Luxembourg: Publications Office of the European Union.

25.

Fyfe

. (2017). Tracking hate speech acts as incitement to genocide in international criminal law. Leiden Journal of International Law, 30, 523–548.

26.

Gergen

K. J.

(2003). Social construction: A reader. London, England: Sage.

27.

Hacking

(1999). The social construction of what? Cambridge, MA: Harvard University Press.

28.

Hatefree Culture. (2017). Jak reagovat na nenávist na internetu [How to react to the hate on Internet]. Praha, Slovakia: Agentura pro sociální začleňování. Retrieved July 30, 2017 from the World Wide Web https://www.hatefree.cz/jak-reagovat-na-nenavist-na-internetu

29.

Hawdon

Oksanen

Räsänen

(2017). Exposure to online hate in four nations: A cross-national consideration. Deviant Behavior, 38, 254–266.

30.

Hrdina

(2016). Identity, activism and hatred: Hate speech against migrants on Facebook in the Czech Republic in 2015. Naše společnosť, 14, 38–47.

31.

Keipi

Näsi

Oksanen

Räsänen

(2016). Online hate and hateful content: Cross-national perspectives. New York, NY: Routledge.

32.

Kováčová

Miškolci

Rigová

(2017). Diskutuj bez nenávisti. Stratégie na znižovanie nenávistných prejavov v online priestore [In Slovak] [Discuss without hate: Strategies to counter hate speech online]. Bratislava, Slovakia: Slovak Governance Institute.

33.

Kramer

A. D. I.

Guillory

J. E.

Hancock

J. T.

(2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111, 8788–8790.

34.

Kriplean

Bonnar

Borning

Kinney

Gill

. (2014). Integrating on-demand fact-checking with public dialogue. Proceedings of the 17th Association of Computing Machinery conference on Computer supported cooperative work & social computing. February 15–19, New York.

35.

Lacy

Watson

B. R.

Riffe

Lovejoy

(2015). Issues and best practices in content analysis. Journalism & Mass Communication Quarterly, 92, 791–811.

36.

Lawson

(2006). The science cycology: Failures to understand how everyday objects work. Memory and Cognition, 34, 1667–1675.

37.

Liao

Q. V.

W. T

. (2013, 04 27–05 2). Beyond the filter bubble: Interactive effects of perceived threat and topic involvement on selective exposure to information. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Retrieved July 30, 2017, from the World Wide Web https://dl.acm.org/purchase.cfm?id=2481326&CFID=953875486&CFTOKEN=97626206

38.

Long

Johnson

(2000). Rigour, reliability and validity in qualitative research. Clinical Effectiveness in Nursing, 4, 30–37.

39.

Macháček

. (2013). Vzťah slovenských občanov a mládeže k pravicovému extrémizmu [The relationship of the Slovak citizens and youth towards right-wing extremism]. In Štefančík

Macháček

Poláková

Spálová

Kluknavská

Bieliková

Šrámek

Ľ.

(Eds.), Pravicový extrémizmus a mládež na Slovensku [Right-wing extremism and youth in Slovakia] (pp. 37–91). Brno, Czech Republic: Tribun EU.

40.

Maitra

(2012). Subordinating speech. In Maitra

McGowan

M. K.

(Eds.), Speech and harm: Controversies over free speech (pp. 94–120). Oxford, England: Oxford University Press.

41.

McMichael

(2011). Development and social change: A global perspective (5th ed.). Thousand Oaks, CA: Sage.

42.

MEDIANSK. (2015). Kontinuálny multiklientsky prieskum spotrebného správania, mediálnej konzumácie a životného štýlu [Continual multi-client survey on consumer behaviour, media consumption and life style]. Retrieved July 30, 2017, from the World Wide Web http://www.median.sk/sk/mml-tgi/

43.

Molnar

(2012). Responding to “hate speech” with art education and the imminent danger test. In Herz

Molnar

(Eds.), The content and context of hate speech: Rethinking regulation and responses (pp. 183–197). New York, NY: Cambridge University Press.

44.

Mušinka

Škobla

Hurrle

Matlovičová

Kling

(2014). Atlas rómskych komunít na Slovensku 2013 [In Slovak] [The atlas of Roma communities in Slovakia 2013]. Bratislava, Slovakia: UNDP.

45.

New

(2012). Stigma and Roma education policy reform in Slovakia. European Education, 43, 45–61.

46.

Nielsen

L. B.

(2002). Subtle, pervasive, harmful: Racist and sexist remarks in public as hate speech. Journal of Social Issues, 58, 265–280.

47.

O’Callaghan

Greene

Conway

Carthy

Cunningham

(2015). Down the (White) rabbit hole: The extreme right and online recommender system. Social Science Computer Review, 33, 459–478.

48.

Ott

B. L.

(2017). The age of Twitter: Donald J. Trump and the politics of debasement. Critical Studies in Media Communication, 34, 59–68.

49.

Patten

M. L.

Newhart

(2017). Understanding research methods: An overview of the essentials (10th ed.). Thousand Oaks, CA: Routledge.

50.

Petková

. (2016, 03 3). Čo prinesú fašisti do parlamentu? [What will fascists bring to the Parliament]. Trend. Retrieved July 30, 2017, from the World Wide Web https://www.etrend.sk/trend-archiv/rok-2016/cislo-10/co-prinesu-fasisti-do-parlamentu.html

51.

Räsänen

Hawdon

Holkeri

Keipi

Näsi

Oksanen

(2016). Targets of online hate: Examining determinants of victimization among young Finnish Facebook users. Violence and Victims, 31, 708–726.

52.

Simpson

R. M.

(2013). Dignity, harm, and hate speech. Law and Philosophy, 32, 701–728.

53.

Sponholz

(2016). Islamophobic hate speech—What is the point of counter speech. Journal of Muslim Minority Affairs, 36, 502–522.

54.

Stăiculescu

A. R.

Gherasim

(2013). Roma representation in the media. Contemporary Readings in Law and Social Justice, 5, 947–955.

55.

Surdu

Kovats

(2015). Roma identity as an expert-political construction. Social Inclusion, 3, 5–18.

56.

Tirrell

(2012). Genocidal language games. In Maitra

McGowan

M. K.

(Eds.), Speech and harm: Controversies of free speech (pp. 222–238). Oxford, England: Oxford University Press.

57.

Titley

Keen

Foldi

. (2012). Starting points for combating hate speech online: Three studies about online hate speech and ways to address it. Council of Europe, British Institute of Human Rights. Retrieved July 30, 2017, from the World Wide Web https://rm.coe.int/1680665ba7

58.

Vašečka

(2009). Postoje verejnosti k cudzincom a zahraničnej migrácii v Slovenskej republike [In Slovak] [The attitudes of the public towards foreigners and foreign migration in the Slovak Republic]. Bratislava, Slovakia: International Organization for Migration.

59.

Velšic

(2016). Mladí ľudia v kyberpriestore: Šance a riziká pre demokraciu [In Slovak] [Young people in cyberspace—Chances and risks for democracy]. Bratislava, Slovakia: Inštitút pre verejné otázky.

60.

Vermeersch

(2012). Reframing the Roma: EU initiatives and the politics of reinterpretation. Journal of Ethnic and Migration Studies, 38, 1195–1212.

61.

Walsh

Krieg

(2007). Roma identity: Contrasting constructions. Canadian Ethnic Studies, 39, 1/2, 169–186.

62.

West

(2012). Words that silence? Freedom of expression and racist hate speech. In Maitra

McGowan

M. K.

(Eds.), Speech and harm: Controversies over free speech (pp. 222–248). Oxford, England: Oxford University Press.

63.

Yong

(2011). Does freedom of speech include hate speech? Res Publica, 17, 385–403.

64.

Young

I. M.

(2000). Inclusion and democracy. Oxford, England: Oxford University Press.

65.

Young

I. M.

(2001). Equality of whom? Social groups and judgments of injustice. The Journal of Political Philosophy, 9, 1–18.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB