Cognitive and behavioral radicalization: A systematic review of the putative risk and protective factors

What is this review about?

Radicalization entails the development of attitudes supportive of the use of violence in the name of a cause, and for a small number (<1%) of radicalized individuals, the carrying out of such violence.

Risk and protective factors, which increase or decrease the likelihood of these radicalization outcomes, are used in risk assessment and counter-radicalization interventions. However, in practice, the selection of factors is often not evidence-based. As a result, policies and practices are unlikely to be as effective as they could be, and can even increase stigmatization of certain communities, thereby increasing the risk of radicalization.

This systematic review supports the development of more evidence-based approaches by identifying the relative magnitude of the effects for a large number of factors.

This Campbell systematic review examines putative risk and protective factors (correlates) of radical attitudes, intentions, and behaviours (including terrorism) in democratic countries. The review examines 101 factors, derived from over 1300 effect sizes extracted from 127 studies.

What do the findings of this review mean?

Some of the factors most central to risk assessment and counter-radicalization interventions actually have relatively small relationships with radicalization outcomes. Conversely, factors known to be associated with ordinary criminal outcomes have the largest relationships. These findings suggest the need for moving towards weighted risk assessment instruments, and alternative interventions.

Additionally, findings of differences in the magnitude of the effects for different factors according to regional context suggest that risk assessment and interventions may be tailored to local contexts.

How up-to-date is this review?

The review authors searched for studies up to March 2020.

The risk-protective factor approach

In line with what is known about radicalization, the criminological risk-protective factor framework maintains that no specific or single risk factor can or will cause violent behaviors. Rather, it is the accumulative and interactive weight of present risk factors, either in the absence or outweighing of protective factors, that increases or decreases the likelihood or risk of offending (Farrington et al., 2016). One of the most significant risk factors for criminal behaviors is criminal attitudes (or cognitions). However, only a small number of individuals who hold criminal attitudes will go on to engage in criminal behaviors (Kim & Hunter, 1993; Pogarsky, 2004). Nevertheless, the majority of those who engage in criminal behaviors are likely to hold criminal attitudes. In recognition of this fact, while most of the research on risk factors has focused on behavioral outcomes, scholars have increasingly been interested in examining risk and protective factors for antecedent cognitive outcomes, including both criminal attitudes and intentions (e.g., Mazerolle & Piquero, 1997; Walters, 2017; Willits, 2015, 2019).

Many of the same risk factors and protective factors tend to predict each of the outcomes of criminal attitudes, intentions, and behaviors. For example, anger and exposure to violence have been found to predict criminal attitudes (Baron et al., 2001), and intentions (e.g., Mazerolle & Piquero, 1997). Similarly, deviant peers (differential associations) and low self-control have been found to predict criminal intentions (Skrzypiec, 2017). Willits (2015) found that anger, low self-control, prior violence and peer violence were all risk factors for violent intentions. Each of these factors figure prominently among risk factors for criminal offending behaviors as well (Hawkins et al., 2000; Walters, 2015).

The differential balance of risk and protective factors that present themselves in individuals, may serve to explain why only a small number of those who hold criminal attitudes or intentions will engage in criminal or criminal analogous behaviors, and most would not (Farrington et al., 2016; Folk et al., 2018; Walters, 2018). In other words, risk and protective factors mediate the relationship between criminal cognitions and behaviors. Some traditional criminogenic factors have been found to be moderators for this relationship, including; thrill-seeking and low self-control (Brezina, 2010; Bulten et al., 2009; Skrzypie, 2017; Walters, 2016), deviant peers and criminal history (Boduszek et al., 2011, 2013).

Risk and protective factors for radicalization

Understandings about how risk and protective factors affect criminal attitudes and behaviors, and the relationship between these distinct but interrelated outcomes, have important implications for understanding the distinctiveness between the cognitive and behavioral outcomes of radicalization, but also the nature of their relationship (Hafez & Mullins, 2015; Khalil, 2014; Neumann, 2013; Carpenter et al., 2009). The literature indicates that there are significant overlaps between criminals and terrorists, who also tend to come from similar segments of the population (Clarke & Newman, 2006). Additionally, criminality is a common feature of terrorists' backgrounds. As such, traditional criminogenic and criminotrophic factors may have relevance for radical attitudes, intentions and behaviors as well (Lösel et al., 2018; Wolfowicz et al., 2020). However, there are also a number of theoretical perspectives that emphasize different factors that are thought to be more specific to radicalization. And while differences certainly exist, it is believed that risk and protective factors for radicalization are relevant for a spectrum of radicalizing ideologies, including Left-wing, Right-wing, and religious ideologies. While it would be both impossible and undesirable to attempt to review all of the factors noted in the literature (Monahan, 2012, 2016), below are some of the most widely discussed factors pertaining to the domains of sociodemographics, attitudinal and experiential factors, psychological/personality trait related factors, and criminogenic factors.

Sociodemographic factors

Almost all perspectives on radicalization place some degree of emphasis on sociodemographic background characteristics. While there are certainly exceptions, young, single males are seen as the most susceptible to radicalization and indeed make up the largest number of radical offenders (Meloy & Gill, 2016). However, there are mixed perspectives concerning other factors. For example, poverty and low socioeconomic status have traditionally been viewed as risk factors for radicalization (Atwood, 2003). However, the evidence is actually quite mixed (Victoroff, 2005). While some studies have found a high prevalence of poor socioeconomic standing among terrorists in both the US and Europe (Bakker, 2011; Handler, 1990; Ljujic et al., 2020), other have found that many terrorist offenders come from the middle and even upper-middle classes (Berrebi, 2007; Russell & Miller, 1977; Sageman, 2008). Additionally, those of the poorest settings appear the least likely to be involved in terrorism (Berrebi, 2007; Lee, 2011). Similarly, while some studies examining radical attitudes have also found correlations between low socioeconomic status and support or justification of terrorism (e.g., Pedersen et al., 2017), others have found the relationship to point in the opposite direction (e.g., Berger, 2016; Bhui et al., 2016).

There is also mixed evidence with regard to other sociodemographic factors such as education and employment (Victoroff, 2005). While many have assumed that low education breeds radicalization, studies have found that terrorist offenders in the West often have some degree of post-secondary education, or were even current students at the time of their attacks (Carlton, 1979; Gambetta & Hertog, 2017; Russell & Miller, 1977; Weinberg & Eubank, 1988). Similarly, while some studies have found an association between lower education and support for terrorism, other have found that the relationship exists in the opposite direction, and that full-time students are more likely to express these types of attitudes (Bhui et al., 2014a; Krueger & Maleckova, 2002; Schbley, 2003). Indeed, the university has long been considered as a 'hot-spot' for radicalization going back to at least the 1960's (Brown & Saeed, 2015; Glees & Pope, 2005; Saeed & Johnson, 2016). According to the NYPD's original four-stage model of radicalization, those with higher education are considered at higher risk, and this factor is most relevant to the preradicalization stage (Silber & Bhatt, 2007).

Unemployment has traditionally been viewed as a key risk factor for radicalization (Sageman, 2004), and has been found to be more prevalent among violent radicals than the general population (Altunbas & Thornton, 2011; Ljujic et al., 2020) and nonviolent radicals (LaFree et al., 2018). However, Krueger (2008) found no statistically significant differences between a samples of terrorists and the general population in being unemployed (and not currently in education). In a study comparing Palestinian terrorists with the general population, it was found that 90% of terrorists were in full-time employment, compared to only 60% of the general population (Berrebi, 2007). Similarly, with regard to radical attitudes, while some studies indicate a positive correlation with employment (e.g., Bhui et al., 2016), others have found a negative correlation (e.g., Acevedo & Chaudhary, 2015).

Experiential and attitudinal factors

Psychological/personality trait factors

Criminogenic factors

The current state of the literature

Despite the growth of radicalization and terrorism research in recent years, empirical studies still only account for a small percentage of the knowledge (Christmann, 2012; Schuurman, 2020). There is therefore little concrete information upon which policies and interventions can be developed, and they are therefore unlikely to have the desired impact (Davis, 2014). While such policies and strategies aim to tackle risk factors, there is often mixed and contradictory evidence about what the risk factors are, and their relative importance (Hafez & Mullins, 2015). The lack of systematic investigation has led policy makers to develop policies and strategies that are not evidence based (Neumann & Kleinmann, 2013; Victoroff, 2005). Only “(g)reater analytical depth may eventually reconcile contradictory claims” (Wikström & Bouhana, 2016, p. 183).

While some systematic reviews have been conducted in the broader topic of radicalization, the evidence that they have synthesized with regard to risk and protective factors can generally be considered to be mixed. For example, the review prepared by the International Centre for the Prevention of Crime (ICPC 2015) identified more than half of the 32 risk factors examined as being characterized by “mixed evidence.”Gilloway et al. (2015) similarly described the evidence concerning most of the 15 risk factors examined as mixed. These findings highlight the limitations of reviews with inclusion criteria that are too broad, narrative reviews, or “vote counting” approaches. It continues to be the case that there is a lack of systematic synthesis and reconciliation of data on *risk factors for radicalization. The literature therefore continues to be highly contradictory (Allen et al., 2015; Bondokji et al., 2017; Victoroff, 2005). Even when there exists a relative consensus as to which factors represent risk, it remains unknown as to what sort of relative weight should be assigned to them (Hafez & Mullins, 2015; Haggerty & Bucerius, 2018; Rahimi & Graumans, 2016; Richards, 2003).

The most promising approach for dealing with such inconsistencies, is systematic reviews employing meta-analytic techniques. Meta-analysis advances parsimony, helps to settle inconsistencies, and importantly, it can provide data concerning the relative effect sizes of different individual-level factors. Meta-analysis therefore also provides for the possibility of a degree of reconciliation between divergent findings and debates (Pratt & Cullen, 2005). Meta-analysis also adds an additional benefit to the study of risk factors for multiple outcomes, namely inter-related cognitive and behavioral outcomes of the same phenomenon. It can quantify variations in magnitudes of the effects of the same risk factors across the different outcomes (Ribeiro et al., 2016). This can potentially provide important indications as to which factors may be relevant for the development between cognitive and behavioral outcomes.

Types of studies

This review sought to extract only quantitative studies and excluded qualitative studies, including studies that are purely theoretical, provide theoretical models, literature reviews, opinion pieces, and those studies based on basic descriptive statistics. Given the nature and objectives of the review, we sought studies employing case-control, single-sample longitudinal, and single-sample cross-sectional designs. Experimental designs that also reported on cross-sectional or longitudinal correlations were also considered eligible for inclusion. Studies were considered to be eligible for inclusion irrespective of the language in which they were written or their publication status (see “Search methods“).

In order for a study to have been included, its design must have been one that provided for the possibility of calculating an effect size. This means that in order for a study to have been eligible it must have included either a direct comparison or control group not displaying the outcome of interest, or a single sample in which there was variation on the dependent variable (Higginson et al., 2014).

For radical behaviors, studies were excluded when they compared terrorists or other radical offenders with samples of ordinary criminal offenders (e.g., Liem et al., 2018; Lyons & Harbinson, 1986). This decision was made on account of the great similarity that exists between terrorists and ordinary criminals. While important, we believe that effect sizes from such studies do not represent risk and protective factors for radicalization per se. Rather, they represent the study of similarities and differences between terrorists and ordinary criminals, which we view as being a separate topic and set of outcomes. Similarly, studies that compared terrorists of different ideologies (e.g., religious vs. right-wing) (e.g., Smith & Morgan, 1994) or types (e.g., lone wolf vs. organizational), or studies examining macro-level predictors of the occurrence of terrorism events (e.g., Piazza, 2006) were excluded as the nature of their comparison groups mean that they are examining outcomes that are divergent from those that are the focus of the current study.

Types of participants

The review included studies in which the unit of analysis were individuals. The review set no limitations on the types of individuals contained in a sample in order for a study to be included. That is, the review set no limitations based on age, gender, race, religion, or the type of ideological strain being investigated (e.g., right-wing, left-wing, religious etc.).

Types of factors

The review included all individual-level factors for which a positive or negative association with the outcome of interest could be identified.

The literature discusses many classifications of factors, such as:

Social, economic and psychological factors

There are many other classifications and this review includes factors from all such categories. We note the review discusses factors in the context of four domains:

Sociodemographic/background characteristic factors

The review excludes all factors that are not measured at the individual level, for example:

Meso-level factors: Community level deprivation, population density, and so forth

Additionally, the review did not include experimental manipulations as factors since combining these with nonmanipulated versions of the same factors would lead to methodological inconsistencies.

Types of outcome measures

Duration of follow-up

As the review expected to collect data primarily from cross-sectional studies, no limitations were placed on the duration of follow-up for a longitudinal study to be included.

Types of settings

It is well known that differences in political, socioeconomic and cultural contexts affect the types of factors that contribute to radicalization in different countries (Brockhoff et al., 2016). Studies that have examined support for suicide bombings for example, have found significant differences in the direction and magnitude of the effects for risk and protective factors between western and non-western countries (e.g., Zhirkov et al., 2014). The issue of heterogeneity across contexts provides for the methodological justification for systematic reviews to limit their focus to particular types of countries, such as high and low-medium income countries, or democratic and nondemocratic countries (Higginson et al., 2018; Litmanovitz & Montgomery, 2016; Murray et al., 2018; Shenderovich et al., 2016).

Given that there is no established norm as to whether systematic reviews should distinguish countries by income level or the system of government, in this review the approaches are combined. High-income countries are considered to be countries who are member states of the Organisation for Economic Co-operation and Development (OECD), and democratic states are those states ranked as being full or partial democracies by the Democracy Index. In cross-matching countries from these two sources, all OECD countries, with the exception of Turkey and Columbia are considered countries eligible for inclusion. In order for a study to be included, its sample must originate from one of the eligible countries, and data had to have been collected in a year in which the country was eligible. This means, for example, that studies from Columbia prior to 2020 would be excluded, since Columbia only became a member of the OECD in 2020.

Additionally, in cases where studies' samples were drawn from multiple countries, and included participants originating from ineligible countries, the study would be included if at least 50% of the sample originated from eligible countries (Table 1).

Electronic searches

Electronic searches were performed across a large number of electronic databases, organizational databases, and specialty journals (Table 2). While these searches were performed in English, many items written in other languages are indexed in these databases as well. Given that these items are indexed in English, it was possible for them to be screened at the first and second stages in English. Where necessary, items in other languages were sent for translation.

Searches were ongoing from December 2019 to April 2020.

In addition to the above, searches were conducted in the databases of the following organizations: The National Consortium for the Study of Terrorism and Responses to Terrorism (START), National Institute of Justice (NIJ), NCJRS (National Criminal Justice Reference Center), The Crime Prevention Council, Sweden (BRÅ).

Furthermore, searches were conducted in specialty journals, including: Perspective on Terrorism, and Journal of Deradicalization.

Extensive supplementary searches were carried out on Google scholar on an ongoing basis in order to identify the most up to date studies which had yet to be indexed in the electronic database.

In order to ensure a more comprehensive review, supplementary searches were also carried out in German and Dutch, the local languages of countries known for producing research relevant to the topic of the review. German sources were searched for in two databases, namely the German National Library and GESIS. Dutch sources were searched for in PiCarta. These databases do not allow for the use of the Boolean search strings used in the English language databases. As such, they were searched for three key terms: Terrorism, Radicalization, and Extremism.

Searching other resources

In addition to the electronic searches, we contacted a number of leading researchers to try to identify missing studies. When we contact them we provided them with the review's topic and inclusion criteria, as well as a list of the included studies that we had collected by the time and inquired as to whether they were aware of any additional studies that would potentially meet the inclusion criteria that we had not identified. We also reviewed the bibliographies of other systematic reviews.

Selection of studies

Data extraction and management

Data extraction was carried out by a team of five researchers. A shared Microsoft Excel spreadsheet was created which included multiple sheets. The first sheet was used for filling out the study-level characteristics of included studies:

Study name

A separate workbook was then created for each of the outcomes examined in the review (titled: radical attitudes, radical intentions, and radical behaviors); in each of these workbooks a separate sheet was created for each factor and labeled. For each effect size, a separate row was created that included the following information:

Study name

Where the main study documents failed to include any of the above information, searches were conducted to identify supplementary materials. Where supplementary materials could not be identified, or where identified supplementary materials also failed to provide the missing information, the authors of the relevant documents were contacted directly by the project team (See “Missing data” below).

Assessment of risk of bias in included studies

Risk of bias was assessed by using the coding fields contained in the extraction tool (see Supplementary materials). The factors assessed were:

Did the study report its sample in replicable detail?

Calculation of effect sizes

All effect sizes were calculated as r and subsequently transformed to Fisher's Z in order to approximate a normal sampling distribution and achieve a more stable variance across different values (Borenstein et al., 2009; Rosenthal, 1984).

In line with previous research, the current review gave preference to bivariate correlations, which provide for a consistent and uncontaminated measurement of effect metrics for the same factors between studies (Hanushek & Jackson, 1977; Hunter & Schmidt, 2004; Pratt et al., 2014). Bivariate effect sizes were derived from zero-order correlation matrices, but were also calculated from summary statistics such as t tests, F tests, χ ² tests, and ANOVAs, and other classical hypothesis tests. In other cases, bivariate effect sizes were calculated from descriptive data such as means and standard deviations, frequencies, and binary proportions. In all such cases, we used Lipsey and Wilson's (2001), “Practical Meta-Analysis Effect Size Calculator” available through the Campbell Collaboration website. The effect sizes were calculated as Cohen's d and subsequently converted to r using the formula:

r = d 4 + d 2

However, as anticipated, some studies did not provide sufficient information to calculate a bivariate effect size; these studies only provided the results of a range of different types of regression models together with basic sample level descriptive data. In such cases, we attempted to identify sources for the missing data (see Section 4.3.9), including contacting the authors (Aloe & Thompson, 2013). Where we were unable to acquire the missing data, we standardized the partial effect sizes derived from the regression models to be used as supplementary effect sizes (e.g., Najaka et al., 2001; Wong et al., 2010). While not ideal, this approach is preferable to conducting multiple separate meta-analyses for each risk factor split by effect size measurement type, which would entail losing important data (Borenstein et al., 2009). In case in which there were more than two bivariate effect sizes and two standardized partial effect sizes in a single analysis, meta-regression and moderator analysis (see Section 4.3.10) were used to identify whether combining these effect sizes had any effect on the pooled estimates (Aloe et al., 2016).

With regard to the standardization of partial effects sizes, there are no standard conventions (Aloe & Thompson, 2013). As such, we adopted a number of widely accepted methods for each of the different types of measures that we encountered, as described below. 1.

For linear regression models where the independent variable (IV) and dependent variable (DV) are both continuous, r was calculated as

r = S D x B S D y

2.

In situations in which SD were not reported, thereby precluding using the above approach, and also in situations in which the IV was dichotomous and the DV was continuous, and in situations in which the IV was ordinal or continuous and the DV was dichotomous, r was calculated based on the t ratio (B/SE) using the following formula:

r = t / t 2 + n − 2

3.

In instances in which both the IV and the DV were dichotomous measures, and only B was reported, we first calculated Cohen's d and then subsequently converted d to r as per the above. In such situations, Cohen's d was calculated as:

d = B 3 π

Calculation of standard errors

For all effect sizes derived from bivariate sources, the standard error was calculated as the square root of the variance of the z transformed correlation, which is calculated as 1/N – 3 (N = sample size).

With regard to effect sizes standardized from regression models, standard errors were calculated based on a rescaling of the model-based standard error, which is calculated as:

se r = r ⁎ SE B B

In situations in which the SE was not reported, it was calculated from reported confidence intervals in order to enable the above calculation to be made. In the case of 95% confidence intervals, the SE was calculated as:

SE = CI upper − CI lower 1.96

Independence of effect sizes

When conducting meta-analysis, it is important to reduce or eliminate potential dependence among effect sizes so that dependent effect sizes are not over-weighted (Lipsey & Wilson, 2001). In this study, common issues affecting dependence were absent, since no single study (or sample within a study containing multiple samples) contributed more than a single effect size for a given factor, and each factor was examined in a separate analysis (Pinquart, 2017; Van den Noortgate et al., 2015).

A second issue relating to dependence concerns the possibility that multiple studies may be based on the same dataset and report on the same factors. We carefully scrutinized the data by searching for repeated instances of sample size, mean age, and proportion of males in the studies. Whenever it was identified that more than one study was reporting on the same factor derived from an overlapping dataset, an internal meta-analysis was performed and the pooled estimate was used as the input for these studies (Higginson et al., 2018).

Dealing with missing data

Whenever a study had missing data the following actions were taken in order: 1.

Search for supplementary materials

Missing data was successfully retrieved from online supplementary materials for 21 studies, from the re-processing of original data for 6 studies, and directly from authors for eight studies.

Assessment of heterogeneity

Heterogeneity was assessed using Cochran's Q (and its associated χ ² p value) as well as the I ² statistic. I ² scores of >75 indicate high heterogeneity, whereas, >50 moderate, >25 low, <25 very low. When I ²  = 0 it indicates an absence of heterogeneity.

Assessment of reporting biases

Given the large number of factors that are often contained in field-wide reviews it was expected that it would be impractical to present funnel plots for each factor analysed. As such, publication bias was assessed using two methods that are based on the funnel plot, the Trim-and-Fill method (Duval & Tweedie, 2000a, 2000b) and Egger's regression test (Egger et al., 1997). In Egger's regression test, the standardized effect sizes are regressed on their precisions. This is the equivalent of a weighted regression of the effect sizes on their standard errors, in which weighting is on the inverse variance. In the absence of publication bias, the intercept is expected to be zero (Rothstein et al., 2005). A statistically significant intercept above zero indicates the presence of publication bias. In the trim-and-fill method, asymmetric funnel plots (where smaller studies are skewed to one side of the bottom of the plot) are identified as indicators of publication bias. The method employs an iterative approach to removing the studies causing the asymmetry, estimating the number of studies missing due to this bias and their effect sizes, and then re-computes a new, adjusted effect size (Rothstein, 2008). As both methods suffer from limitations (Sterne et al., 2000), they have often been used complimentarily, including in risk factor related research (e.g., Assink et al., 2015; Vazsonyi et al., 2017).

Data synthesis

Meta-analysis was performed using Biostat's Comprehensive Meta-Analysis (CMA) software (Borenstein et al., 2009). Random effects models were used in order to account for the anticipated heterogeneity of the studies, which is common when dealing with correlational data from observational studies, and also in studies dealing with violence related cognitions and behaviors. Effect sizes pertaining to risk and protective factors are known to be more heterogeneous than those for interventions, in part because they deal with heterogeneous populations and segments of the different populations. CMA's Random effects models calculate estimates based on the inverse variance, which is derived from the sample size associated with each effect size. CMA V3 utilizes the Method of Moments random effects estimator for τ ², the between-study variance (DerSimonian & Laird, 1986).

We categorized each factor based on a careful assessment of the measurement constructs contained in the studies and the outcome measure, and combined effect sizes that measured the same construct, or conceptually similar constructs for the same outcome. For the most part, studies were quite clear as to the nature of the factors, with detailed descriptions of the factors' name, construct, and measurement in the methodology sections. In order to provide for the most meaningful analysis, we tried to avoid combining factors that were conceptually related but distinct. For example, while Law legitimacy and Law abidance are closely related, in analysing the items used to measure these constructs it was determined that they were conceptually distinct. In other cases, such as Authoritarianism and Fundamentalism, a close inspection of the items used to measure these factors across different studies revealed that they often only differed in how they were labeled, with labeling differing between studies analysing different ideological strains (e.g., Right-wing, Islamist, or nonspecific/mixed ideologies). The literature also indicates that these factors are nearly indistinguishable, and common scales were used in most of the studies reporting on these factors (Altemeyer & Hunsberger, 1992, 2004, 2005). Supporting Information Appendix A includes a list of the factors included in the review with examples of the common constructs used in studies.

A separate meta-analysis was conducted for each of the identified factors. We present the results of the meta-analyses as r correlations with 95% confidence intervals in a series of rank-ordered tables, with each row representing a separate analysis for a single factor.

Subgroup analysis and investigation of heterogeneity

Only a few studies included data that would have enabled any sort of meaningful subgroup analysis (e.g., based on gender, ethnicity, or religion). Given the limited number of these studies, such analysis would have only been possible for a small number of factors. Given that we were able to conduct an extensive array of univariate meta-regression analyses to assess the impact of study-level characteristics on heterogeneity, these subgroup analyses were not conducted.

The protocol predefined that meta-regression analyses would be used to assess the effects of the following factors on pooled estimates:

Region/country from which the sample was derived (EU, United States, and Other)

While we had hoped to examine individual countries, few analyses provided a sufficient number of studies to enable this type of inquiry. In order to provide consistency across all analyses, we grouped the countries by region. Given that the majority of the studies were from the EU, this region was used as the reference category. With regard to ideological strain, given that the majority of the studies examined nonspecific or mixed samples, this was used as the reference category. For moderator analyses between-group heterogeneity was assessed using Cochran's Q.

Sensitivity analysis

Sensitivity analysis was performed for each factor that included 3 or more effect sizes using the “leave-one-out” method. This approach uses an iterative method in which the meta-analysis is repeated k times (k = the number of effect sizes in the analysis) and a different study is excluded at each iteration. We inspected the results to identify whether a single effect size had a significant influence on heterogeneity (Viechtbauer & Cheung, 2010). This was assessed by examining whether the removal of any single study caused heterogeneity to be reduced by at least one level, with the levels being set at; very low (I ²  < 25), low (I ²  > 25), moderate (I ²  > 50), and high (I ²  > 75). We reported on factors for which the removal of a single study led to a significant reduction in heterogeneity.

Results of the search

The results of the systematic searches and screening process are displayed below in Figure 5. The primary English-language searches resulted in the identification of over 22,000 items. An additional 2925 items were retrieved from a combination of German and Dutch language databases, as well as items sent to the research team by authors. These additional items also included 90 items that were retrieved from Google Scholar by the research team and were added to the first sample screening. The 28 items received directly from authors who had been contacted were screened separately.

English studies were screened by two of the senior researchers together with trained research assistants. The German studies were screened by a native German speaking research assistant and the Dutch studies were screened by a native Dutch speaking research assistant. A single study in Spanish was provided and translated, which enabled its screening in English. Three studies in French, including one which was sent to us by one of the experts we had contacted were screened by the main author who had sufficient French language skills to assess the items.

During the initial screening it was evident based on titles and abstracts that the overwhelming majority of the items did not relate to the field or topic of interest. The removal of these studies resulted in the exclusion of 19,998 items, with 2160 items moving on to the second screening stage. During secondary screening, the titles and abstracts were more carefully scrutinized. While many of the items dealt with topics related to “terrorism,” “extremism,” or “radicalism,” it was clear that for these and many other items, they were not assessing radicalization or radicalization analogous outcomes. A large number of items clearly stated in their titles or abstracts that the study was “qualitative,” or based on “case studies,” and it was evident that they did not meet the methodological inclusion criteria. A number of other studies which appeared to meet the inclusion criteria for the topic, clearly stated in their titles or abstracts that the study was conducted in a country outside of the eligible country list.

The final screening stage involved a 'full-text' reading of each item by the first author and the research assistants. Another senior researcher read half of the items. The methodology sections were reviewed first in order to identify the nature of the data and sample and outcome measures. While many of the studies met between 1 and 4 of the criteria for inclusion, only 127 studies met all inclusion criteria and were subsequently submitted for inclusion in the review (Figure 5).

OPEN IN VIEWER

Figure 5

Flowchart of search and screening process

Included studies

A total of 127 studies were screened as being eligible for inclusion. The full references for these studies appear in the reference section. A description of the included studies can be found below (Table 3). The included studies were published between 2003 and 2020. Of these, 50% were published from 2018 to 2020. A total of 42 of the included items included more than one study, or reported on more than one individual sample. This meant that the total number of individual studies/samples derived from the included items was 206.

Excluded studies

In the full-text screening stage, almost half of the studies were excluded for a variety of reasons. A total of 86 of the excluded studies pertained to the topic and were conducted in eligible countries. However, 58 of them were found to have examined ineligible outcomes with measures of radicalization that fell outside of the inclusion criteria. Further, 76 of the excluded studies did not meet the inclusion criteria pertaining to methodology. This was generally due to the lack of variation on the dependent variable (e.g., samples made up exclusively of terrorists).

Additionally, as described above, some of the studies that were included reported on multiple individual studies. In a few cases, one or more of these individual studies failed to meet the inclusion criteria.

Replicability

In 42 cases a sample's mean age was not reported and we were unable to identify it from other sources. Similarly, for N = 18 cases, the proportion of males in a sample was not reported and we were unable to identify it from other sources. Additionally, in 24 cases, the year of data collection for a sample was not reported and we were unable to identify it from other sources.

There were 53 samples which were identifiable as overlapping with other samples from included items. There were an additional nine cases in which we suspected that a sample was overlapping with one from another study, however we were unable to confirm this due to missing descriptive data.

Inclusion criteria and sampling method

Most of the studies provided no specific criteria for inclusion or exclusion. Rather, most studies sought to provide random or representative samples (see Table S1). In terms of sampling methods, most studies appear to have used appropriate sampling methods. While most studies reported the sampling method, 19 studies did not. However, for nine of these studies, it was possible to deduce the method based on other descriptions within the study. For 10 of the samples it was not possible to deduce the method with any level of confidence. For studies that reported the sampling method, or for which it was possible to deduce the methods, a variety of methods were found, including: Accidental (N = 4), Convenience (N = 17), Convenience/Snowball (N = 1), Disproportionate stratified random (N = 4), Probability (N = 2), Proportional quota (N = 3), Purposive nonrandom (N = 3), Purposive random (N = 59), Quota (N = 7), Random (N = 66), Representative (N = 22), Snowball (N = 7), and Stratified random (N = 2).

For the most part, included items and their samples were based off of original data (N = 168). An additional 3 studies combined original and secondary data, in the case of case-control studies for example. This meant that 35 studies used secondary data.

Use of validated measures

While 104 of the samples used validated measures of “radicalization,” 102 of the samples used nonvalidated measures. However, among those that did not use validated measured were a large number of studies that used single item measurements assessing support for or justification of terrorism, and suicide bombings specifically. While imperfect, such measures are considered to be strong proxies for Radical attitudes in the absence of more validated measures (Schmid, 2017). Additionally, a number of the samples with nonvalidated outcome measures examined Radical intentions, and used measures that have been widely used in the literature. With regard to independent variables, the majority of the samples used validated measured (N = 171) and only a minority of studies used nonvalidated measured (N = 35) items. However, even in cases of nonvalidated measures, single and multiple survey items were found to be close to the constructs captured by more validated measures.

Most studies described independent and dependent variables well enough to facilitate replication. Many studies included supplementary materials.

Overlapping data

As noted above, we identified overlapping data used in 54/206 samples, and overlapping data was suspected between an additional 9 samples. While overlapping data was more common in studies using secondary data, overlapping data in studies utilizing original data was also identified. Whilst some of these studies were explicit about the use of data in prior studies, and even cited these studies, for other studies the overlaps were identified by the research team by comparing sample characteristics. For some studies we continued to suspect overlapping data, although differences in a study level characteristic (such as a sample's mean age) meant that we were unable to confirm these suspicions.

For the most part, there was little overlap in the factors analysed in these studies. For studies using secondary data, the most common overlaps were for the factors of Gender, Age and Education. For studies using original data, there were only a small number of cases of reporting of effect sizes for the same factor. In all such cases we pooled the studies in an internal meta-analysis and used the pooled estimate as the input.

For studies with suspected overlaps, we took the same approach (carrying out an internal meta- analysis and used the pooled estimate as the supplementary effect sizes) and assessed whether this had any effect on the estimates. As the largest change in effect size for any of the six factors for which this approach was used was.01, and there were no significant effects on heterogeneity, we present the results from the analyses that included the independent effect sizes. In all cases there were no more than 2 effect sizes from samples suspected of overlap.

Reporting on nonsignificant findings

A number of the included studies reported that some factors were found not to be statistically significant, but did not report any information from which an effect size could have been calculated. While a number of studies noted that they controlled for key sociodemographic variables, these were not reported in a number of studies. Missing data was successfully retrieved from online supplementary materials for 21 studies, from the re-processing of original data for 6 studies, and directly from authors for eight studies.

For 10 factors from two studies it was still not possible to obtain the missing information needed for the calculation of effect sizes which the documents merely reported as not having been statistically significant (Moskalenko & McCauley, 2009; Pfundmair et al., 2019). In one case a minimum p value was reported and effect sizes were estimated from this. In the other case we took the standard approach in which we ran the analysis without the missing study, and then compared it with an analysis in which the study's effect size was imputed as zero. While this approach may lead to underestimation of the magnitude of the estimate (Durlak & Lipsey, 1991), it is considered to be preferable over exclusion of the effect size and the possible overestimation of the magnitude of the estimate (Rosenthal, 1995).

Analysis of factors as described

For some studies, while sociodemographic and other factors were reported as having been included in analyses, their effects were not reported. For some studies, additional effects were identifiable in supplementary materials, whereas for others such materials were not available. In some cases, we were able to ascertain the data directly from the authors, or from accessing the publicly available data used in the studies (k = 6).

Nature of the data

While the majority of studies provided sufficient data for the calculation of bivariate relationships between individual factors and at least one of the outcomes of interest, some studies only provided data for calculating partial effect sizes. As with observational studies more generally, it is necessary to consider the potential for confounding. While the correlations appear to operate in the theorized direction, the data does not provide the ability to derive causal inferences. Caution must therefore be taken in interpreting the results for the purposes of policy. While correlations may be useful for the purposes of risk assessment, they do not provide sufficient evidence that changes in these factors will lead to changes in radicalization outcomes, as in the case of interventions (Murray et al., 2009).

With the exception of the longitudinal studies, most studies did not examine factors before the onset of the outcome. Even in the case of longitudinal studies, which primarily used ordinal or continuous measures of Radical attitudes or intentions, the time and variation between Time 1 and Time 2 measures was small, indicating that they too were not measuring factors before the onset of the outcome. However, for some factors, survey items were constructed retrospectively, seemingly to a period that is likely to have preceded the onset of the outcome, such as in the case of abuse or exposure to violence in adolescence. However, for case-control studies assessing radical behaviors, many of the factors were measured as having occurred or having been present prior to the onset of the outcome.

Radical attitudes

The outcome of radical attitudes included studies whose outcomes assessed justification or support for the use of violent radical behaviors, of groups engaged in such behaviors, or of specific events of radical behaviors. The analysis was based on 838 effect sizes pooled across 100 factors, made up of 29 protective factors derived from 234 effect sizes, and 71 risk factors derived from 604 effect sizes. The identified risk factors span all of the domains of 1) Sociodemographic and background factors, 2) Psychological and personality trait factors, 3) Attitudinal and subjective belief related factors, and 4) Experiential factors, and 5) “Traditional criminogenic and criminotrophic factors.” The results are arranged in a color coded rank-order according to the size of the estimates for the different factors. The general findings indicate that the smallest estimates are related to sociodemographic and background characteristics, whereas the largest estimates tend to be associated with the domains of psychological and personality trait factors and traditional criminogenic factors (Table 5).

The protective factors for radical attitudes had estimates ranging from -0.00 to -0.19, with a single outlier estimate of -0.55. The estimates for Neuroticism, Extraversion, Student, Political Efficacy, Procedural Justice, as well as Public and Group Self-Esteem were not statistically significant (p > .10). The estimate for Openness was only marginally significant (p < .10). Among the statistically significant factors, sociodemographic characteristics made up the bulk of the tier of very small estimates (r = .00-.10), with factors such as: Children, Socioeconomic status, Marital status, Age, Education and parental education level. Other subjective attitude and experiential factors also featured in this tier, namely general trust, school performance, and out-group friendships. The tier of factors with small estimates (r = .10-.30) was made up of a combination of psychological and social factors, namely: Parental Involvement, Social Support, Conscientiousness, Parental Control, School and Teacher Bonding, Agreeableness, Political Satisfaction, Personal Self-Esteem, Institutional Trust, and Life Satisfaction. A single outlier estimate of r = -.55 was found for Law Abidance.

The risk factors for radical attitudes had estimates ranging from r = .00 -.43. The estimates for 13 factors were statistically nonsignificant, namely: Depression, Immigrant Status, Political Participations, Fear of Crime, Need-for-Closure, Life Events, Moved House, Adjusted Personality Disorder/Narcissistic Personality Disorder (APD), Religiosity, Self-Efficacy, Psychopathy, Social Dominance Orientation (SDO). Four more factors were found to be only marginally significant (p < .10), namely; Anxiety, Alcohol Use, Drug Use, and Negative Affect. Among factors with very small (r = .00-.10) but statistically significant estimates, were a combination of sociodemographic, psychological, attitudinal, and experiential factors, namely: Physical health, belief in a battle between the West Vs. Islam, Unemployment, Welfare Recipient, Frequency of Attendance at Places of Worship, Time spent online, Uncertainty, In-Group Identity, Perceived Injustice, Experiences of Discrimination, Teacher Mistreatment, Experiences of Violence, Aggression, and Experiences of Family Violence. The tier consisting of small estimates (r = .100–.300) featured factors from the entire spectrum of factor domains, namely: Gender (male), Posting of political/radical content online, Individual relative deprivation, Exposure to violent media, Personal strains, Victim of parental abuse, Anger/Hate, Quest for significance, Prayer frequency, Identity fusion, Perceived discrimination, Political grievances, Segregationist attitudes, Collective relative deprivation, Societal disconnectedness, Deviant peers, Anti-democratic attitudes, Anomia, Juvenile Delinquency, Willingness to Self-Sacrifice, Low Integration, Low Legitimacy, Post-Traumatic Stress Disorder/Trauma (PTSD), Positive Affect, Authoritarianism/Fundamentalism, Low Self-Control, Exposure to Radical Media and Criminal History. The tier consisting of moderate sized estimates (r = .30–.50) was made up of 11 factors, namely: Police Contact, Thrill-Seeking/Risk-Taking, Similar Peers, Symbolic Threat, Moral Neutralizations, In-Group Superiority, Realistic Threat, Political Extremism (Right-Left), Low Life-Attachment, Machoism and Dehumanization.

Radical intentions

As opposed to radical attitudes, the outcome of radical intentions included studies which assessed individuals' willingness or intentions to engage in violent radical behaviors, or to participate in the activities of groups already engaging in such behaviors. For this outcome, the analysis was based on 338 effect sizes that were pooled across 45 factors, made up of 8 protective factors derived from 79 effect sizes, and 37 risk factors derived from 259 effect sizes (Table 6).

The protective factors for radical intentions had estimates ranging from r =  -.03 to -.22. The estimates for Education, Socioeconomic status (SES) and Outgroup friendships were not statistically significant (p > .10). Age had the smallest of the statistically significant estimates, followed by three factors from the “Big Five” set of personality traits, namely Agreeableness, Conscientiousness and Openness (the other traits, Extraversion and Neuroticism were risk factors). The largest estimate was found for Immigrant status, where being an immigrant had a negative association with radical intentions.

The risk factors for radical intentions had estimates that ranged from r = .05-.52. Statistically nonsignificant estimates were found for Unemployment, Social-Dominance Orientation (SDO), External political efficacy, and Perceived discrimination. Very small (r = .00-.10) but statistically significant estimates were found for: Uncertainty, Neuroticism, Experiencing discrimination, and Adjusted Personality Disorder/Narcissism. The next tier, consisting of small estimates (r = .10-.30) included: Student, Gender (Males), Quest for significance, Extraversion, Individual relative deprivation, Religious convert, Positive affect, Harmonious passion, and Low integration, Dark-triad personality traits, Power Distance Orientation (PDO), In-group connectedness, Personal self-esteem, Anomie, In-group identity, Realistic threat, Perceived injustice and Symbolic threat. The tier consisting of moderate sized estimates (r = .30-.50) included: Past activism, Collective relative deprivation, Moral neutralizations, Perceived discrimination, In-group superiority, Anger, Commitment to a cause, Activist intentions, Negative affect, Radical attitudes. Large estimates (r > 50 >) were found for Obsessive Passion and Identity Fusion.

Radical behaviors

The outcome of radical behaviors included studies assessing involvement in violent radical behaviors, including illegal and violent subterroristic behaviors motivated by a radical ideology, and behaviors that can be classified as terrorism. The analysis was based on 137 effect sizes pooled across 33 risk and protective factors, made up of 7 protective factors derived from 39 effect sizes, and 26 risk factors derived from 98 effect sizes (Table 7).

The protective factors had estimates ranging from r =  -.03 to -.22. Statistically nonsignificant estimates were found for Marital status and Education. Statistically significant but very small estimates were found for Parental involvement, Age and School Bonding, and small estimates for Law Legitimacy and Law Abidance.

The estimates for the risk factors for radical behaviors ranged from r = .01-.63 and covered all of the primary factor domains. Statistically nonsignificant estimates were found for Religious Upbringing, Religious Convert, Immigrant Status, Relationship Problems, Anger, and Current Military Service. Only three of the very small (r = .00-.10) factors had statistically significant estimates, namely Bullying Victim, Welfare Recipient, and Parental Abuse. The tier consisting of small estimates (r = .10-.30) included: Low Integration, Experiencing Violence, Personal Injustice, Mental Health, Radical Family, Authoritarianism/Fundamentalism, Unemployment, Thrill-Seeking/Risk-Taking, Anger and Low Self Control. The tier consisting of moderate estimates (r = .30-.50) included: Deviant/Radical Peers, Radical Attitudes, Online Contact with Extremists, Past Military Service, Criminal History, Recent Job Loss, and Gender (Male). A large estimate (r > .50) was found Previous Incarcerations.

Heterogeneity

As per the results displayed in Tables 5–7, there was a wide range of heterogeneity across the factors analysed. For radical attitudes, heterogeneity was high (I ²  > 75) for 67 factors, moderate (I ²  > 50) for 16 factors, low (I ²  > 25) for 1 factor, very low (I ²  < 25) for 2 factors, and absent (I ²  = 0) for 11 factors. For radical intentions, heterogeneity was found to be high (I ²  > 75) for 25 factors, moderate (I ²  > 50) for 6 factors, low (I ²  > 25) for 3 factors, very low (I ²  < 25) for 4 factors and absent (I ²  = 0) for 6 factors. For radical behaviors, heterogeneity was high (I ²  > 75) for 19 factors, moderate (I ²  > 50) for 6 factors, low (I ²  > 25) for 2 factors and absent for 6 factors.

While high heterogeneity is common in meta-analyses of observational studies, and studies examining deviant outcomes, results for factors displaying high heterogeneity must be interpreted with a degree of caution. Estimates in the presence of high heterogeneity may not be accurate reflections of the true estimate. We therefore set out to investigate potential sources of heterogeneity using two methods, meta-regression and moderator analysis.

Meta-regression analyses were conducted for three key study-level characteristics, namely the year of data collection, the average age of studies' samples, and the proportion of males in studies' samples. For each risk factor, these study level variables were regressed individually (univariate analysis). The analysis was only conducted on risk factors which had a minimum of 6 effect sizes for which information on the study level variable was available. For this reason, the number of effect sizes (k) in Table 8 may be different for each variable as they pertain to a specific risk factor. As the results demonstrate, one or more of the study-level characteristics analysed was found to have a significant impact on the results for a number of factors.