Counter-narratives for the prevention of violent radicalisation: A systematic review of targeted interventions

Counter-narrative interventions may affect some risk factors related to violent radicalisation, but there is no effect on intent to act violently

Counter-narratives may affect certain risk factors for violent radicalisation, including realistic perceptions of threat, in-group favouritism and out-group hostility. However, the effects are inconsistent across outcomes, failing to target symbolic threat perceptions, implicit bias or intent to act violently.

While the findings from this review support the feasibility of the concept more broadly, they also highlight the care and complexity needed to design and implement effective counter-narratives in the context of violent radicalisation.

The review in brief

Narratives which reduce complex, real-world phenomena to simplistic, violence-promoting propaganda can activate the necessary mechanisms for violent radicalisation to occur. To stop this from happening, researchers and counterterrorism practitioners have turned to counter-narratives; targeted interventions that challenge the instrumentality of violence as put forth in dominant narratives.

This review summarises the available evidence on this approach, looking at whether counter-narratives are effective at preventing violent radicalisation across a spectrum of contexts, including right-wing, ethnic and religious extremism

What studies are included?

This review includes studies that evaluate the effects of counter-narrative interventions in individuals exposed to a dominant narrative which, if not countered, may promote a violent extremist belief system. The outcomes targeted by the intervention include the intent to act violently, as well as “risk factors” for violent radicalisation.

Nineteen studies met the inclusion criteria. These studies span the period 2000–2018 and mainly include study populations of University and high school students. Although the studies represent a range of geographical locations, the majority were conducted in North America.

Twelve of the studies are moderate-high quality randomised controlled trials and the remainder are quasiexperimental studies.

What is the aim of this review?

This Campbell systematic review examines the effects of counter-narrative interventions on primary and secondary outcomes relating to violent radicalisation. The review summarises evidence from 19 independent studies, including 12 randomised controlled trials. The majority of the included studies are from North America.

The problem, condition, or issue

Since the earliest days of researching extreme violence, academics and counter-terrorism practitioners have increasingly sought to better understand the process(es) by which an individual comes to perpetrate an act of terror. In recent years, these efforts have grown into a multi-disciplinary pursuit, embracing methodologies, as well as theoretical insights, from psychology (Horgan, 2005), psychiatry (Melle, 2013), political science (Bjorgo, 2005), anthropology (Atran, 2006), sociology (Turk, 2004) and communication science (Archetti, 2013; Braddock, 2014). However, despite these contributions, conceptually, the phenomenon has been poorly defined, leading to extensive academic debate (see Schmid, 2004; Schuurman, 2018) with terms such as “extremism” and “violent extremism” emerging, somewhat, as a tonic for this ambiguity.

While support for “extreme” politics carries with it some negative connotations (interpreted as indicative of dwindling support for democratic values, see Knigge, 1998), the term itself is not synonymous with violence. Instead, it is best understood as a belief system existing at the poles of society's central tendency. From here, familiar belief systems such as “far-right”, “far left” and “single-issue

In an attempt to reduce the likelihood of individuals engaging in terrorism, research efforts have moved towards understanding what happens before an individual reaches such a climactic point. This shift has sparked investigation into various precipitating factors, such as the onset of insurgency (O'Neill, 2005), conflict (Newman, 2006) and even the increased variability of global temperatures (Fjelde & von Uexkull, 2012; Miles-Novelo & Anderson, 2019; Price & Elu, 2016) that may serve as “triggers” for the political, sociological and, ultimately, psychological changes that cultivate a violent extremist mentality. It has been argued, however, that this mentality cannot be explained by precipitating factors alone (Kruglanski, Bélanger, & Gunaratna, 2019) and is dependent, instead, on the activation of certain, psychological mechanisms that trigger a cognitive shift; the process of violent radicalisation.

Description of the intervention

In their review of strategies to stop violent radicalisation from happening, Briggs and Feve (2013) proposed a strategy of challenging such narratives, by deconstructing, discrediting and “demystifying” (p. 6) the themes they purport. This deconstruction falls under the umbrella term “counter-narrative”. By discrediting their respective dominant narratives, counter-narrative strategies are intended to present individuals with alternative social constructions to those presented by the dominant narrator. In counter-terrorism, it follows a logic of prevention; by treating the risk of violent radicalisation through narratives upstream, incidence and prevalence of violent extremism and terrorism downstream will be reduced.

This idea is not novel. The approach has been explored in relation to challenging inaccurate historical narratives (“counter-factual” narratives, see Mordhorst, 2008), as well as persistent, socially constructed hegemonic narratives relating autism (autism as neurodiversity, rather than disease, see Broderick & Ne'eman, 2008), infertility (maintaining a narrative of femininity in narratives of failed IVF; Bell, 2004), and disability (challenging dominant societal scripts that reduces disability, fostering exclusion, see Harter, Scott, Novak, Leeman, & Morris, 2006, p. 12). Here, the counter-narrative is described as “counter-storying”, designed to offer a narrative identity which resists those emerging from dominant discourse(s) (Ingamells, 2016, p. 58). The approach has also been applied to violence prevention more broadly. In 2004, in response to increased homicide rates, the World Health Organization recommended the implementation of media campaigns as a way of changing “attitudes, behaviours, and social norms” (p. 16) with regard to violence, leading to the development of numerous violence prevention interventions rooted in the concept of the counter-narrative.

How the intervention might work

In terms of the mechanics of this idea, there have been a number of proposals for how the intervention may work. These have come from both researcher and practitioner spheres. In 2013, The Quilliam Foundation published a practical guide (Hussain et al., 2014) to countering violent extremism online and advised governments to create counter-extremist content that challenges the various political or theological arguments put forth in dominant narratives. In terms of evidence, this approach (i.e., the technique of contradicting, or directly countering, an argument or narrative with the intention of refuting it, see Wheeler, Briñol, & Hermann, 2007, p. 151) has seen mixed empirical success. In early research on stereotyping, Brock (1967) found that pointing out inconsistencies (or discrepancies) in a persuasive appeal reduced belief change in certain cases through more sophisticated information processing. However, Taber and Lodge (2006) found that participants who were offered pro and con arguments for topical issues such as affirmative action and gun control uncritically accepted arguments which supported their own, baseline attitudes and counter-argued ones to the contrary. In other words, for participants with existing, baseline attitudes, the approach worked differently, and was not effective. It has been suggested that this occurs when a strategy focuses exclusively on the content of the argument, ignoring, according to Schwarz, Sanna, Skurnik and Yoon (2007), “the metacognitive experiences that are part and parcel of the reasoning process” (p. 128).

For this reason, some have highlighted the impracticality of crafting counter-narratives that are both initiated, and shaped, by an adversary. A counter-narrative comprised of counter-arguments inevitably ends up becoming an “information contest” (Reed, Ingram, & Whittaker, 2017, p. 44). Irrespective of the correctness of the information (which introduces moral ambiguity), simply correcting misinformation is not likely to “dislodge the feeling” (Kahneman and Frederick, 2005, p. 278) that what one believes (or what one has learned) is correct. The same has been suggested of contradictory evidence to dispel erroneous beliefs. Attempts to do so often increase later acceptance of the prior beliefs, as observed in Allport and Lepkin's pioneering research on rumours (1945).

Subsequently, others in the field have suggested that logical arguments will pale in comparison to emotionally evocative counter-narratives. In fact, such strategies have been described as “more important than evidence” (Radicalisation Awareness Network, 2015, p. 6). In line with these guidelines, attempts have been made to create and edit ISIS defector videos (i.e., first-person critique of the organisation and its tenets) to their most “damaging, denouncing and derisive content” (McDowell-Smith et al., 2017, p. 55). The intention here is to add to the speaker's emotionality and, presumably, the target's capacity for identification (see Cohen, 2001), reducing the appeal of dominant, pro-ISIS narratives. In terms of theory and evidence, much understanding of persuasive processes (and narrative persuasion, in particular) have been informed by resistance-based theories such as “Reactance Theory” (Brehm, 1966), “Cognitive Dissonance Theory” (Aronson, Turner, & Carlsmith, 1963; Festinger, 1957), and dual process models of cognition such as the “Elaboration Likelihood Model” (ELM; Petty & Cacioppo, 1986; Petty & Wegener, 1999), and Green and Brock's (2002) “Transportation-Imaginary Model”. These models view narrative persuasion as a process occurring through a peripheral, rather than central route. The latter suppresses resistance and counter-arguing, allowing for a narrative to successfully persuade. However, the application of these theories to countering existing, dominant narratives through the mechanics described above has not been extensively examined.

In 2008, the United Nations published a report of different approaches to countering violent extremist content online. Among the strategies cited were those that implemented “alternative views” (p. 16), or an alternative account of events. Conceptually different to alternative narratives (which, by definition, do not directly undercut the logic of violent extremism), this approach involves presenting the same story from a different perspective. Informed by the ELM, this approach has seen some success in the context of counter-stereotypical information, and early research on stereotyping. According to the ELM, individuals process information through two channels; the first, also known as “system one”, is quick, intuitive and requires very little “cognitive effort” (Dhami & Thomson, 2012, p. 219); the second, also referred to as “analysis” (Kahneman, 2003; Lamond & Thompson, 2000), or “reflective judgement” (Kitchener & King, 1990) is a slower, more careful and consistent form of information processing (Hamm, 1988). Vasiljevic and Crisp (2013) found that exposing participants to contradictory information about another social group encouraged more systematic, “system two” processing of information, resulting in lower hostility. Similarly, Power et al. (1996) found that introducing counter-stereotypical information about African-Americans to Caucasian Americans reduced their attribution of blame towards a target.

Finally, the creation of one's own counter-arguments could be applied to the design of counter-narratives. This is well-documented in “Inoculation Theory”, which posits that exposing individuals to weakened arguments can inoculate (i.e., protect) them against stronger arguments of the same nature (McGuire, 1961a, 1961b). The theory follows the same rationale as viral inoculation, whereby a weaker form of a virus is introduced to the body to encourage the creation of antibodies, reducing susceptibility to an active viral infection. However, instead of developing antibodies, the individual develops counter-arguments to reduce their susceptibility to persuasion. In a meta-analysis of 54 cases, inoculation was found to be an effective form of creating resistance to persuasive messages when compared to matched controls (Banas & Rains, 2010). Successful attempts have also been made to experimentally manipulate inoculation techniques in the context of violent radicalisation-prevention (Braddock, 2019).

However, despite suggestions and theoretical insights from proximal research areas, specific theoretical frameworks to inform the design of counter-narratives have yet to be developed.

Why it is important to do the review

In 2017, the United Nations Security Council adopted Resolution 2354. Tasked with preventing violent radicalisation through narratives, this Resolution seeks to achieve multiple strategic communication objectives, amongst which includes the development of effective counter-narratives. Following this, the European Commission appointed the International Centre for Counter-Terrorism (ICCT) to identify and report on the state of knowledge in regards to this elusive strategy. Despite an overwhelming volume of guides, reports and individual studies on the topic, however, Reed and colleagues (2017) described the counter-narrative as conceptually “underdeveloped” and lacking a “thorough grounding in empirical research” (p. 8). As such, calls have been made for more stringent methodological designs in terms of counter-narrative evaluation, such as baseline measures and control groups. Notes have also been made on the need for a stronger theoretical focus in order to develop a more thorough understanding of the behavioural and attitudinal bases of messaging efforts. Ultimately, there appears to be a “glaring gap” (Reed, 2018, p. 1) in counter-narrative research, despite its stature in policy.

While there does not yet exist a large-scale synthesis of counter-narrative interventions in the context of violent radicalisation, syntheses of similar approaches have been conducted in other research fields. For example, Stice and Shaw (2004) provided meta-analytic evidence on the use of a proximal approach called dissonance-based interventions (DBI) which encourage individuals to adopt a way of thinking that contradicts their current way of thinking (e.g., challenging social constructions of “beauty” or “thinness”). Furthermore, Chan, Jones, Hall-Jamieson and Albarracin (2017) provided meta-analytic evidence on the factors underlying effective counter-arguing or “debunking” of misinformation (e.g., conspiracy theories or “fake news”). Although efforts have been made to synthesise current governmental and nongovernmental strategies to counter violent extremist narratives (Briggs et al., 2013) and provide a “horizon scan” (Ferguson, 2016) of the research landscape, to date, there has been no synthesis of the effectiveness of counter-narrative interventions for the prevention of violent radicalisation (Schmid, 2014). This review seeks to address this.

The review will contribute to existing theory and evidence on counter-narrative interventions, allowing researchers and practitioners to better understand message construction as well as the psychological fulcra for change-targeted. Most importantly, the review will offer evidence on the effectiveness of the approach in reducing outcomes related to violent radicalisation. In doing so, the review may help those tasked with preventing violent radicalisation to effectively counter harmful, violence-promoting messages masquerading as innocuous stories.

Title registration and review protocol

This review followed an explicit protocol with methodological guidance provided by the Campbell Collaboration. The title was registered in The Campbell Collaboration Library of Systematic Reviews in September 2017. The protocol was published in September 2018 (Carthy, Doody, O'Hora and Sarma, 2018).

Criteria for considering studies for this review

See Appendix C for coding categories according to the inclusion and exclusion criteria.

Search methods for identification of studies

Potentially relevant literature was identified through a five-stage search strategy, which comprised:

Stage 1: Scoping exercise.

The first three stages were conducted simultaneously at the start of the review process (August–September 2016). The final two stages were conducted once duplicates were removed from Stages 1 and 2 (October 2016). Stages 2–3, and 5 were replicated in May 2019 due to the surge in the publication of relevant literature between 2016 and 2019 (see Figure B1 (Appendix B) for review timeline).

Data collection and analysis

Description of studies

As mentioned, this review included two separate searches. The initial search was conducted in August 2016 (date range: 2000–2016) and an updated search was conducted almost three years later (in May 2019) to account for the rise in experimental literature in the area of counter-narratives (date range: 2016–2018). The two searches are first described independently before the final, included studies are pooled and described collectively for the remainder of the review. See Figure 1 (overleaf) for the flow of studies through the search and screening process, across both searches.

OPEN IN VIEWER

Figure 1

PRISMA (the flow charts have been adapted from Moher et al. (2009). Additional sections have been added to demonstrate the contribution from nondigital sources, as well as the role of the second coder in the screening process) flow chart of searches

Risk of bias in included studies

As mentioned, two independent coders (S.C. and K.C.) used the EPOC data collection checklist to assess risk of bias. This checklist allows the coders to rate study designs against nine study criteria, indicating a high, medium or low risk of bias on that particular item. Cohen's Kappa (κ) (Cohen, 1960) was calculated for testing inter-rater reliability on a total of 171 risk of bias items (i.e., 9 items across both randomised and nonrandomised studies), obtaining κ = 0.72 (i.e., substantial agreement between coders). Table 3 (overleaf) summarises the risk of bias for the 12 randomised studies included in this review.

The majority exhibited low risk of bias for randomisation to conditions (92%), treatment of incomplete data (92%), contamination (100%), and selective outcome reporting (92%).

Some studies presented an unclear risk on certain domains. The majority of studies did not measure outcomes at baseline (75%) increasing the risk of sampling bias. It was not clear if deception was used in certain studies and, given the nature of the research, knowledge of condition (i.e., allocation) presented an unclear risk for 8% of the studies. The study that exhibited a high risk of bias in the “other” domain was Ramasubramanian and Oliver (2007) whose use of a counter-arguments exercise prior to the measurement of outcomes may have primed participants, or created a detection or social desirability bias (Fisher, 1993).

In assessing the nonrandomised studies (n = 7), the studies naturally had a high risk of bias for maturation (change over time) and were, therefore, all regarded as exhibiting a high overall risk of bias. As well as maturation, participants' knowledge of the true nature of the study posed a high risk for 86% of the nonrandomised studies. For example, Alhabash and Wise (2012, 2015)

Synthesis of results

Raw data could not be obtained for one measure of explicit bias (Kendrick and Fullerton, 2004), one measure of realistic threat (Ramasubramanian and Oliver, 2007) or for the singular outcome measured in one randomised control trial (Garagozov, 2013). As such, the findings for these outcomes are presented narratively throughout the following sections, where relevant.

Overall impact of counter-narrative interventions

The present analysis incorporates 29 effect sizes across 18 studies (many studies measured more than once, conceptually different outcome). This includes 11 randomised control trials (Banas & Richards 2017; Bilewicz & Jaworska, 2013; Bruneau et al., 2017; Cernat, 2001; Cohen et al., 2015; Gonsalkorale et al., 2010; Ramasubramanian & Oliver, 2007; Riles et al., 2018; Saleem et al., 2015), two interrupted time-series, and five single group pre-/posttest designs (Alhabash & Wise, 2012, 2015; Čehajić-Clancy & Bilewicz, 2017; Frischlich et al., 2018; Kendrick & Fullerton, 2004). The studies represent a total sample of 2627 (M _age

The mean age of participants was calculated from the following studies: Bilewicz & Jaworska, (2013, M _age = 16.84), Cohen et al. (2015, M _age = 25.24), Banas and Richards (2017, M _age = 19.71), Riles et al. (2017, M _age = 19.47), Alhabash and Wise (2012, M _age = 20; 2015, M _age = 20.80), Bruneau et al. (2017, M _age = 33.65; 2017b, M _age = 34.95; 2017c, M _age = 34.3) and Čehajić-Clancy and Bilewicz, (2017, M _age = 20.09; 2017b, M _age = 20). The standard deviation was only provided by five studies so could not be confidently determined.

 = 24.1, 57% female

Calculated from the following studies: Gonsalkorale et al. (2010, 78% female), Cernat (2001, 33% female), Alhabash and Wise (2012, 74% female), Bilewicz & Jaworska, (2013, 69.1% female), Cohen et al. (2015, 75% female), Saleem et al. (2015, 49% female), Banas and Richards (2017, 43% female), Bruneau et al. (2017, 47% female; 2017b, 50.15% female; 2017c, 43.35% female), Riles et al. (2018, 60% female), and Čehajić-Clancy and Bilewicz, (2017, 61% female).

) participants; 1,789 participants were allocated to either a counter-narrative or control condition. The remainder (n = 838) participated in nonrandomised, within-subjects designs, and served as their own controls. Randomised and nonrandomised studies are analysed separately; it may be useful to consult Table 1 (Theory and Techniques) for the remaining results sections.

Randomised studies

See Figure 2. On average, when all proximal outcomes (i.e., risk factors for violent radicalisation) were pooled to represent each randomised control trial (n = 11) the difference between those who did, and those who did not, receive a counter-narrative intervention was significant, representing a small effect size. Under a random effects model, the standardised mean reduction in risk factor(s) for violent radicalisation was SMD = −0.38; (95% CI, −0.52 to −0.23, p = .000). Unsurprisingly, given that the interventions were informed by different theoretical frameworks, measuring conceptually varied outcomes (for example, both implicit and explicit measures), there was modest between-study heterogeneity (χ ² = 20.42 [p = .03], I ² = 51%, τ ² = 0.03).

OPEN IN VIEWER

Figure 2

Forest plot of counter-narrative intervention effects on all risk factors for violent radicalisation compared to a control group

Subgroup analysis was conducted to determine if the variation among studies could be different depending on the theory and techniques used. Outcome differences were tested by the presence or absence of four theorised key components: counter-stereotypical exemplars, persuasion, alternative accounts and inoculation. Table 4 presents meta-analysis statistics separately by technique (i.e., levels of the moderator) (Hunter & Schmidt, 2004, p. 402). The use of persuasive techniques was not found to be effective (d = 0.08), while inoculation showed promising effects (d = −0.57). However, both effect sizes represented single study samples, limiting the generalisability of both findings based on theory or technique(s). The between-group differences were not significant and, as such, no further analyses were conducted.

OPEN IN VIEWER

Table 4

Separate variance estimates for each group

	k	d	95% CI	τ 2	p
Technique(s)
Counter-stereotypical exemplars	6	−0.40*	−0.70, −0.11	0.08	−.19
Persuasion	1	0.08	−0.39, 0.55	–	–
Alternative account(s)	3	−0.39*	−0.53, −0.26	−0.00	−.67
Inoculation	1	−0.57*	−0.87, −0.28	–	–

p < .05.

Nonrandomised studies

See Figure 3. For studies that used single group pre-/posttest designs (n = 7), a separate analysis was conducted. Pooling all measured outcomes to represent each study, the effect of the intervention over time was not significant. Under a random effects model, the standardised mean reduction was −0.05; (95% CI, −0.15 to 0.04; p = 0.27). In this model, there was little between-study heterogeneity (χ ² = 4.37 [p = 0.63], I ² = 0%, τ ² = 0.00), suggesting that the finding was consistent across studies. Four studies used persuasive techniques (Alhabash & Wise, 2012, 2015; Frischlich et al., 2018), Kendrick and Fullerton (2004) did not specify their theory or techniques but used counter-stereotypical information while Čehajić-Clancy and Bilewicz, (2017) delivered an eclectic intervention that included moral exemplars.

OPEN IN VIEWER

Figure 3

Forest plot of counter-narrative intervention effects on all risk factors for violent radicalisation postintervention

Given the between-study heterogeneity among the RCTs, as well as the disparate findings between randomised and nonrandomised studies (the former being superior, but the latter, nonetheless, homogenous and of interest), separate analyses were subsequently conducted for each risk factor (perceived group threat and in-group favouritism/out-group hostility), including subgroups for each subcategory risk factor where possible. Randomised and nonrandomised designs were, again, analysed separately. It may be useful to consult Table 2 for the following subsections.

Perceived group threat

The present analysis incorporates effect sizes measuring participants' perception of threat from their adversarial group. Across both randomised and nonrandomised studies, this represents a total sample of 2,046 participants; 1,662 participants were allocated to either a counter-narrative or control condition (Banas and Richards 2017; Bilewicz & Jaworska, 2013; Cernat, 2001; Cohen et al., 2015; Riles et al., 2018; Saleem et al., 2015). The remainder (n = 384) participated in single group pre-/posttest designs (Alhabash & Wise, 2012, 2015; Čehajić-Clancy and Bilewicz, 2017). In many cases, the studies measured a combination of “symbolic” threat perceptions (e.g., perceived differences in motives, values or standards between the in-group and the out-group) and “realistic” threat perceptions (e.g., perceived threats to one's safety or existence).

Randomised studies

This analysis incorporates 10 effect sizes from nine randomised studies. As shown in the forest plot in Figure 4, the intervention effect for randomised studies which measured both symbolic and realistic threat (i.e., perceived group threat) was not significant. The mean reduction was, SMD = −0.33 (95% CI, −0.82 to −0.16; p = .18). Although there was significant between subject heterogeneity (χ ² = 208.42 [p = .000], I ² = 96%, τ ² = 0.58), the test for subgroup differences (between symbolic threat outcomes and realistic threat outcomes) was not significant (p = .24).

OPEN IN VIEWER

Figure 4

Forest plot of counter-narrative intervention effects on all perceived group threat (both symbolic and realistic) compared to a control group

Looking at the subgroups, the intervention effect for randomised studies targeting symbolic threat not significant, SMD = 0.34 (95% CI, −1.16 to 1.85; p = .66). However, there was significant between-study heterogeneity (χ ² = 64.85 [p = .000], I ² = 97%, τ ² = 1.71), likely explained by Riles et al. whose intervention saw a significant increase in symbolic threat (i.e., in the wrong direction, SMD = 1.68). Their effect for realistic threat was also significant, but in the intended direction (i.e., a reduction).

On average, realistic threat decreased by SMD = −0.60 (95% CI, −1.05 to −0.15; p = 0.01), with, again, significant between subject heterogeneity (χ ² = 208.42 [p = .000], I ² = 96%, τ ² = 0.58).

Nonrandomised studies

For the nonrandomised studies, only Kendrick and Fullerton (2004) measured realistic threat, with the remainder measuring symbolic threat (as shown in the forest plot in Figure 5). When pooled, the intervention effect was not significant, SMD = −0.09 (95% CI −0.27 to 0.08; p = .28), with minimal between-study heterogeneity (χ ² = 5.65 [p = .23], I ² = 29%, τ ² = 0.01). Across both symbolic and realistic threat, this finding was heterogeneous, with no significant subgroup differences (p = .77). Thus, in within-groups samples, the counter-narrative interventions do not appear to reduce perceived group threat.

OPEN IN VIEWER

Figure 5

Forest plot of posttest changes following a counter-narrative intervention on symbolic threat

In-group favouritism and/or out-group hostility

The current analysis incorporates 11 effect sizes representing in-group favouritism and/or out-group hostility. This includes seven randomised control trials (Bilewicz & Jaworska, 2013; Bruneau et al., 2017; Cernat, 2001; Gonsalkorale et al., 2010; Ramasubramanian & Oliver, 2007), and four single group pre-/posttest designs (Alhabash & Wise, 2012, 2015; Čehajić-Clancy & Bilewicz, 2017). The studies represent a total sample of 1,420 participants (demographic descriptive statistics were not consistently provided); 1,151 participants were allocated to either a counter-narrative or control condition. The remainder (n = 269) participated in nonrandomised, before-and-after studies (i.e., no control group). Again, randomised and nonrandomised studies are analysed separately.

Randomised studies

As shown in the forest plot in Figure 6, the intervention effect for randomised studies which measured in-group favouritism and out-group hostility was significant, SMD = −0.39 (95% CI, −0.52 to −0.25; p = .000), with minimal between-subject heterogeneity (χ ² = 6.86 [p = .33], I ² = 13%, τ ² = 0.00). In other words, those in the counter-narrative condition showed a decrease in the overall risk factor compared to a control group, with a small-medium effect; this was consistent across the subgroups. The effect of the intervention on out-group hostility also showed a small effect, SMD = −0.36 (95% CI, −0.48 to −0.24; p = .000).

OPEN IN VIEWER

Figure 6

Forest plot of counter-narrative intervention effects on in-group favouritism and out-group hostility compared to a control group

Data for in-group-favouritism was only available for one randomised study, and the effect was significant with a very large effect size, SMD = −0.90 (95% CI, −1.49 to −0.30; p = .003). In terms of implicit bias, Garagozov (2013) narratively reported that their “Common Suffering” counter-narrative was the most effective at reducing implicit bias, while “Blame the Russians” was the least; the latter findings are to be interpreted with caution as no effect sizes were provided.

Nonrandomised studies

In the studies that used single group pre-/posttest designs and measured out-group hostility, the implicit (e.g., the AMP) and explicit measures (e.g., negative evaluations of the out-group) of this outcome were observed separately. As shown in the forest plot in Figure 7, there were significant subgroup differences (p = .03).

OPEN IN VIEWER

Figure 7

Forest plot of posttest changes following a counter-narrative intervention on out-group hostility (both implicit and explicit)

With two separate samples, Čehajić et al. (2017) measured out-group hostility on explicit measures, and the effect was significant, SMD = −0.25 (95% CI, −0.47 to −0.03; p = .003), with minimal between-study heterogeneity (χ ² = 0.00 [p = .99], I ² = 0%, τ ² = 0.00). Conversely, the intervention effect for Alhabash and Wise (2012, 2015) who measured out-group hostility using implicit measures was not significant, SMD = 0.13 (95% CI −0.14 to 0.40; p = .34), with, again, minimal between-study heterogeneity (χ ² = 0.36 [p = .55], I ² = 0%, τ ² = 0.00). This suggests that, while the counter-narrative seems to be effective (pre- to posttest) at reducing bias on an explicit level, this is not the case on an implicit level. Data could not be obtained for Kendrick and Fullerton (2004) measure of explicit bias (“attitudes towards the US people”), but it is reported that the effects of the counter-narrative intervention did not reduce this bias to a level of significance.

Sensitivity analysis

Two meta-analyses required sensitivity analysis. The first related to the overall impact of counter-narrative interventions (Section 5.3.3.1; Figure 2). This required more arbitrary decision-making and therefore, sensitivity analysis was conducted to determine the impact of certain decisions. With multiple study outcomes, the authors chose to average all the scores for outcomes categorised as risk factors for violent radicalisation in the original analysis, creating a single, standardised mean difference for each study. However, in line with The Campbell Collaboration policies and procedures, an alternative approach can be used whereby outcomes are chosen using specific decision criteria (e.g., reliability, validity or relevance).

Using this approach, a sensitivity analysis was conducted in which the most reliable outcome (reported as Cronbach's α) was chosen. In cases where no reliability analysis was provided, the most relevant outcome was chosen. These criteria were applied to the relevant studies individually. No notable differences were observed (see Table A8, Appendix A). When all these changes were applied together (Table 5, previous page), although there was a moderate increase in effect size (−0.39 to −0.58), the between-effect difference was not statistically significant (χ ² = 0.82, df = 1; p = .37).

The second sensitivity analysis was concerned with risk of bias with the nonrandomised studies; in assessing the risk of bias for these studies (n = 7), all were regarded as high risk. However, it was noted that two studies (Frischlich et al., 2018) posed particularly high risk for carryover effects, and performance biases. As such, the second analysis was re-run, excluding these studies (see Table 6). The removal of these studies resulted in a stronger effect size, but this remained nonsignificant. The between-effect difference was, also, not significant.

Publication bias analysis

Publication bias was assessed with all the randomised and nonrandomised studies used throughout the analysis. The effects were heterogeneous, (χ ² = 30.30 [p = .004], I ² = 57%, τ ² = 0.04) so a Baujat plot was produced to explore possible contributors to heterogeneity. As shown in Figure 8 (below), Alhabash and Wise (2015) was the leading contributor.

OPEN IN VIEWER

Figure 8

Funnel (left) and Baujat (right) plots

The publication bias analysis was not repeated for the remaining meta-analyses. The authors make the point that the categorisation of measured outcomes onto secondary outcomes (see Table A7, Appendix A) was unavoidably biased, and testing for different reporting biases within the individual analyses would reflect this, regardless of the true risk of bias.

To avoid any potential biasing of the meta-analysis due to multiple studies

Summary of main results

The objective of this review was to provide a synthesis of the effectiveness of targeted counter-narrative interventions in reducing the risk of violent radicalisation by asking the question: what is the impact of targeted counter-narrative interventions on violent radicalisation (primary outcomes) and/or risk factors for violent radicalisation (secondary outcomes)?

Overall completeness and applicability of evidence

Several international actors such as the ICCT, the Institute for Strategic Dialogue (ISD), and the Radicalisation Awareness Network (RAN) have commented on the need for evaluation in the design of counter-narratives (Saltman, Dow, & Bjornsgaard, 2016), resulting in numerous counter-narrative initiatives and strategies. While this review offers a comprehensive analysis of the effectiveness of certain, targeted counter-narrative interventions on reducing propensity towards violent radicalisation, it must be acknowledged that the scope and span of counter-narrative interventions likely extends beyond the 19 studies included in this review. This is the case for two reasons.

First, while many counter-narrative strategies appeared in the initial searches (e.g., Frennett et al., 2015; Macnair & Frank, 2017), the majority did not meet the inclusion criterion for outcomes related to violent radicalisation. Instead, the evaluative components of many of these campaigns were more reflective of feasibility, rather than effectiveness. Metrics such as likes, comments, “bounce-and-exit rates”, or shares (see Denaux & Rollo, 2019) may tell us about the counter-narrative campaign from a practical perspective, but they are not empirically supported risk factors for violent radicalisation, or components of a radical belief. As such, they cannot indicate if the target of the campaign has a reduced risk of transitioning into violent extremism. It is for these reasons that many published, potentially informative counter-narrative campaigns could not be included in the synthesis, despite their relevance to the area at large.

Second, it may be the case that certain, counter-narrative strategies were not identified through the search strategy to begin with. This is not necessarily a critique of the strategy itself. Instead, it refers to what Sageman terms the “stagnation” (2014, p. 565) of terrorism research, with regards to government-funded projects not being made available to academics, creating an “unbridgeable gap between academic and the intelligence community” (p. 573).

Quality of the evidence

The 19 studies were assessed according to the GRADE approach for evaluating quality of evidence. Randomised control trials were first graded as “high”, and downgraded accordingly depending on the severity of the study limitations. Nonrandomised studies were first graded as “low” quality, and upgraded or downgraded accordingly, using the same criteria. Case series, interrupted-time-series or uncontrolled longitudinal designs were graded as “very low” quality, and ungraded if necessary.

The results of the quality analysis are provided in Table D2 (Appendix D). Just over half of the studies (58%) were rated as moderate (n = 6) to high (n = 5) quality, with one randomised study of low quality. Randomised studies downgraded from high to moderate were generally characterised by the following limitations: using outcome measures with poor reliability, small (or unequal) sample sizes, potentially uncredible control groups (e.g., a control group that did “nothing”, rather than an active control), and the potential for crossover effects. For example, the randomised study (Cernat, 2001) which was double-downgraded to “low” shared the above limitations and, additionally, the intervention was poorly informed (i.e., not guided by a specific theoretical framework).

The remaining studies were all nonrandomised and categorised as “low” (n = 2) or “very low” (n = 5). Similarly, studies were generally downgraded for using outcome measures with poor reliability. The risk of crossover, or practice, effects for these studies was, also, logically higher. One nonrandomised study was double-downgraded as, alongside the limitations outlined above, the intervention not informed by a specific theoretical framework, and it used single-item measures. Studies were upgraded for design strengths such as the use of deception, reliable or validated outcome measures, and large effect sizes.

Violent radicalisation and, in particular, the evaluation of interventions to prevent it, is a challenging area to conduct high quality research. Quality standards such as those described above require that researchers deliver theoretically informed interventions and measure empirically supported outcomes using validated, reliable measures. However, the normal challenges encountered at various stages of the study design process are much thornier in this area. Although research into the process(es) of violent radicalisation is ever-expanding, leading to the identification of specific risk factors, robust theories, and novel ways of testing them, the field is nonetheless in the early stages of theory-building, mid-“leap” between exploratory and explanatory phases (Silke, 2001, p. 2). Ultimately, this an area which does not, yet, have an explanatory understanding of its central problem (i.e., violent radicalisation as a process leading to the perpetration of violent extremism or terrorism), or how to measure it. For this reason, stringent quality appraisal, although insightful, may be premature. Nonetheless, efforts to reduce sampling bias (and unequal sample sizes), utilise measures with sufficient construct validity and, finally, introduce credible, comparable control conditions would help ameliorate a number of biases.

Limitations and potential biases in the review process

There are several limitations that could affect the results of the present review. First, the literature base was limited, as is to be expected with research of this nature. For this reason, the target populations in the studies had varying, dominant narratives, ranging from entrenched ideas about conflicts, to prejudicial leanings towards Muslims. This rendered it difficult to determine the effects of the intervention(s) on different, dominant narratives, as well as embeddedness of the dominant narrative(s) to begin with. The sparse literature base also leads to another limitation in the review; a lack of comparable, valid outcomes. Although the outcomes in this review all measured outcomes related to violent radicalisation, and followed explicit protocol in defining acceptable outcome measures, they were conceptually broad and, despite every effort to preserve each outcome's original construct, the process of conceptually mapping the outcomes on to risk factors for violent radicalisation was, nonetheless, subject to bias.

Implications for practice and policy

The findings from this review have implications for those seeking to prevent violent radicalisation into terrorism by challenging dominant, violence-promoting narratives. The findings from this review illustrate the complexity of violent-radicalisation in terms of secondary outcomes or risk factors. While the use of counter-narrative interventions may decrease some risk factors, their effects on others are unpredictable. The evidence from this review, from a combination of randomised and nonrandomised studies, indicates that counter-narratives can effectively target perceptions of realistic threat but, conversely, are likely to be ineffective at targeting symbolic threat, as measured by adverse stereotypes, reconciliatory beliefs, or social stigma. The verdict as to which risk factor is more predictive of violent radicalisation is beyond the scope of this review. However, the authors propose that potential counter-narrative interventions reflect the complexity of their prospective outcomes, and consider that an ineffective counter-narrative, beyond having no effect, has the potential to have exacerbating effects.

The second policy implication relates to the use of specific techniques in counter-narrative design. As mentioned, several counter-narrative guidelines have been published by varying counter-terrorism actors, and are freely available to the public.

Another technique which arises frequently in counter-narrative guidelines is the use of persuasion. In 2013, the RAN, in collaboration with the ISD, published a detailed report on counter-narratives, with recommendations for designing successful counter-narrative campaigns using variety of techniques, including emotions, professional-looking productions, and satire. Emotions, they report, are more important than evidence as facts and statistics can be dismissed while emotional appeals have “greater power” (p. 6); satire, they report, has historically played an effective role in undermining extremists such as the Ku Klux Klan while high quality productions “critical to legitimacy and appeal” (p. 6). The results from this review do not support this approach.

In this review, interventions which employed persuasive techniques, such as identification and transportation, demonstrated no effect at targeting perceived realistic threat (d = 0.08) or out-group hostility (d = 0.00–0.18). In general, persuasive communication is a precarious methodology for manipulating attitudes or behaviour on contentious outcomes, such as threat or hostility. Although such techniques are well established in creating new attitudes (“response-making”; see Berlo, 1960), as it has been suggested that counter-narratives should be aimed at individuals “further along the path to radicalisation” (Briggs et al., 2013), there are several difficulties which may arise from the use of persuasive techniques. If an individual wants to maintain psychological consistency with their baseline attitudes (Wegener, Petty, Smoak, & Fabrigar, 2004), is not motivated to cognitively restructure (Festinger, 1957), or simply does not wish to engage with the appeal (Briñol, Rucker, Tormala, & Petty, 2004) such attempts will likely result in a “boomerang effect” (attitude change in the unintended direction; see Byrne & Hart, 2009), or no change at all.

It is well evidenced that persuasive techniques are used to entice vulnerable individuals into supporting, or perpetrating, acts of violence, rendering them fundamental components of terrorist communication (Braddock et al., 2016; Jowett et al., 2018). While persuasive communication certainly encompasses a spectrum of techniques (many of which do not function as a product of manipulation but, rather, of cognition, and how human beings process complex information), the logic of relying upon these methods in attempts to counter their effectiveness is counter-intuitive. If the counter-narrative is to become an evidence-based tool for countering violent extremism, it should not need to employ the same techniques of those whom it intends to discredit.

Implications for Research

The reviewers make a number of recommendations for future research on counter-narratives, specifically for violent radicalisation. These are broadly discussed under two central themes.