Abstract
BACKGROUND
The Problem
The involvement of young people in gangs and gang crime is not only an issue in western nations, but also across low- and middle-income countries. Research demonstrates the existence of youth gangs in Africa, Asia, Central and South America, with much of the evidence coming from Latin American nations (Decker & Pyrooz, 2010; Gatti, Haymoz & Schadee, 2011). Although official and academic estimates of gang membership differ, estimates put the number of gang members in Central America at up to 200,000 (UNODC, 2007), and research suggests that over 85,000 people are members of gangs in El Salvador, Guatemala and Honduras (Seelke, 2013). In South Africa, it is estimated that there are up to 100,000 members in Western Cape alone (Reckson & Becker, cited in Decker & Pyrooz, 2010). Gang activities - and particularly those of youth gangs - contribute significantly to the violent crime problem in low- and middle-income countries. The cost of violence in Latin America is estimated at approximately 14.2 per cent of GDP - almost three times the proportion of GDP reported in industrialised countries (Seelke, 2013). Gang violence makes up a significant proportion of this cost: the annual cost of violent crime in El Salvador is reported at US$ 1.7 billion, with gang violence accounting for 60 per cent (Seelke, 2013).
Gang violence undermines social cohesion in communities, creating fear amongst residents (see Lane & Meeker, 2003; Seelke, 2013; Washington Office of Latin America [WOLA], 2006) and results in people avoiding certain areas of neighbourhoods known to be gang areas. George Tita and his colleagues explain that these places develop an appearance of visible disorder as non-gang activity in the neighbourhood is abandoned (Tita, Cohen, & Engberg, 2005). Youth gangs are also increasingly associated with trafficking in drugs, arms and humans (Organization of American States [OAS], 2007).
Gang violence and crime can occur between gangs and non-gang individuals, as well as between or within gangs. Violence may be used to defend or expand gang turf, recruit new members, keep members from leaving, exclude or remove undesired members, exercise revenge or seek redress for actual or perceived wrongs, enhance perceptions of power and invincibility, gain respect or dominance over others, and enforce the gang rules (Pacheco, 2010). Although there are significant negative repercussions in the life course for members of youth gangs (Cruz, 2007; Davies & MacPherson, 2011; OAS, 2007; WOLA, 2006), for many young people who lack other opportunities, gangs offer a sense of belonging and purpose (Howell, 2012; Tobin, 2008).
Researchers often contest a uniform definition of a youth gang, as it varies by time and place (Howell, Egley, & O'Donnell, n.d.). Notwithstanding these debates, the literature typically describes a gang as: comprising between 15 to 100 members, generally aged 12 to 24; having members that share an identity linked to name, symbols, colours or physical or economic territory; having members and outsiders that view the group as a gang; having some permanence and degree of organisation; and involvement in an elevated level of criminal activity (Decker & Curry, 2003; see also Esbensen, Winfree, He, & Taylor, 2001; Howell et al., n.d.; Huff, 1993; Miller, 1992; Rodgers, 1999; Spergel, 1995; Theriot & Parker, 2008). There have been significant efforts amongst academics and policy makers to reach agreement on the definition of a youth gang. The “Eurogang Working Group” (see The Eurogang Project, 2012) consensus definition is as follows: “A street gang (or troublesome youth group corresponding to a street gang elsewhere) is any durable, street-oriented youth group whose involvement in illegal activity is part of its group identity” (Weerman et. al., 2009, p.20). A youth gang is differentiated from an adult gang if the majority of the gang members are aged between 12 and 25 (Weerman et. al., 2009).
The General Secretariat of the Organization of American States (OAS) describes the social function that the gang plays for its members as a means to overcome “extreme poverty, exclusion, and a lack of opportunities” (OAS, 2007, p.5). The OAS further elaborates on the role of the gang using a rights-based approach: “Youth gangs represent a spontaneous effort by children and young people to create, where it does not exist, an urban space in society that is adapted to their needs, where they can exercise the rights that their families, government, and communities do not offer them. Arising out of extreme poverty, exclusion, and a lack of opportunities, gangs try to gain their rights and meet their needs by organizing themselves without supervision and developing their own rules, and by securing for themselves a territory and a set of symbols that gives meaning to their membership in the group. This endeavor to exercise their citizenship is, in many cases, a violation of their own and others' rights, and frequently generates violence and crime in a vicious circle that perpetuates their original exclusion. This is why they cannot reverse the situation that they were born into. Since it is primarily a male phenomenon, female gang members suffer more intensively from gender discrimination and the inequalities inherent in the dominant culture.” (OAS, 2007, p.5)
Youth gang violence is a problem that is widespread throughout the developing world. Not all youth gangs are involved in crime or violence; however it is understood that gangs evolve along a continuum towards criminality and violence, from youth gangs that engage in non-criminal activities to youth gangs actively involved in serious violent behaviour (OAS, 2007). Gang types have been described on a continuum “from weakly organized playgroups to more clearly organized supergangs” (Tobin, 2008, p.62).
It is well established that gang-involved youth commit more crime than non-gang-involved youth, and violence has been described as central to gang membership (Klein & Maxson, 2006). Overall, however, the offending of gang members tends to be generalist, rather than specialising in violent crime (Klein & Maxson, 2006). In order to reduce the prevalence of youth gang violence, it is important not only to target the violence directly but also to target the process of young people joining youth gangs.
The Interventions
Responses to the problem of youth gang violence in low- and middle-income countries can be grouped into one of two categories: suppression or prevention. Suppression approaches aim to combat gang violence in a reactive way that attempts to stop the criminal behaviour reoccurring, generally using legislative or policing resources. By contrast, prevention programs focus on capacity building and social prevention and are designed to work proactively to stop gang crime before it occurs, either by preventing youth from joining gangs (primary and secondary prevention) or by rehabilitating gang members outside of the criminal justice system (tertiary prevention) (Esbensen, 2000; Van Der Merwe & Dawes, 2007). Whilst acknowledging the many suppression strategies that are enacted to combat youth gang violence, this review will focus on interventions that use primary, secondary or tertiary prevention strategies.
Primary prevention strategies are applied most broadly to the entire population who are potentially able to join gangs (Esbensen, 2000); in this case, all young people. Primary prevention programs include general community and school based programs to enhance the life skills and resilience of adolescents. An example of a primary prevention program is the Gang Resistance Education and Training (GREAT) program, a school-based curriculum run by law enforcement officers that uses elements of cognitive-behavioural training, social skills development and conflict resolution to improve young people's resistance to gang membership (Esbensen & Osgood, 1999). This program was developed in North America, and has been delivered in Belize, Guatemala, El Salvador, Honduras, Nicaragua, Costa Rica, and Panama (GREAT, 2013).
Secondary prevention strategies target those individuals who are identified as being at higher risk of joining gangs 1 (Esbensen, 2000). Many of these programs provide a mix of education, therapeutic services, and recreational opportunities. An example of a program that has a secondary prevention component is the Por Mi Barrio Outreach Centres, a program implemented in Central America by the United States Agency for International Development (USAID) that focuses on creating a safe space for youth to engage in recreational activities (USAID, 2010a). Further examples of secondary prevention programs supported by USAID that provide skills training for at-risk youth include: the Educatodos program in Honduras, which provides basic education for at-risk youth; the Civil Rights and Values for Youth program in Honduras, that focuses on participatory citizenship and problem solving skills for at-risk youth; and the Community Empowerment and Transformation project (COMET) in Jamaica, that provides micro-entrepreneurship opportunities for at-risk youth (USAID, 2010b). In South Africa, examples of secondary prevention interventions include the Usiko program, funded by NGOs, businesses and communities, which uses ‘rites of passage’ programs for young offenders and at-risk youth, and the Chrysalis Academy, funded by the West Cape Department of Community Safety, an intensive program that provides training and support for a five-year period with the aim of transforming at-risk youth into community leaders (Ward & Cooper, 2012).
Tertiary prevention strategies target youth who have already become involved in gangs or criminal behaviour (Esbensen, 2000). Tertiary prevention programs are designed to reintegrate ex-gang members into society pro-socially, by focusing on rehabilitation and education. An example of a tertiary prevention program is the Medellin program in Colombia, which provides at-risk youth with access to long-term employment programs through state and private institutions on the proviso that gang members withdraw from their gang (Cooper & Ward, 2008). Tertiary prevention programs in South African prisons include the Reintegration and Diversion for Youth (READY) program, the Tough Enough Program, and the Destinations Program (Ward & Cooper, 2012). Tertiary programs can also include negotiations and gang truces, as these strategies aim to engage with current gang members to reduce the levels of violence occurring within or between gangs, even if they do not result in the participants completely disengaging from a gang framework.
How the Intervention Might Work
The predictors of gang membership are routinely categorised across five domains: individual, peer, family, school and community (Decker et al., 2013; Hawkins et al., 2000; Howell, 2012; Howell & Egley, 2005; Katz & Fox, 2010; Klein & Maxson, 2006; O'Brien et al., 2013; Tobin, 2008). Research in high-income countries demonstrates that the predictors of gang involvement cut across all five domains, that youth with multiple risk factors have a proportionately higher risk of gang involvement, and that those youth with risk factors in multiple domains have further increased likelihood of gang involvement (Decker et al., 2013; Howell & Egley, 2005). Preventive interventions seek to target these predictors in order to disrupt the developmental pathway to gang membership.
Building on Thornberry and colleagues' developmental framework of gang membership (Thornberry et al., 2003), Howell and Egley (2005) propose a developmental perspective that incorporates predictors from early childhood through to adolescence. The model is illustrated in Figure 1, and can be viewed as a ‘life-cycle’ approach to gang prevention.
Logic model of predictors of gang membership (Source: Howell & Egley, 2005)
The logic model of gang membership (Howell & Egley, 2005) begins with preschool factors, where it is theorised that structural disadvantage and lack of social capital at the community level, combined with family factors such as low human capital, family conflict and poor parenting, and child level risk factors such as aggressive and impulsive temperament, can lead to conduct disorders at the pre-school stage. It is suggested that these aggressive and disruptive behaviours can lead to rejection by pro-social peers, which increases the likelihood of early delinquent behaviour and decreased school performance. In later childhood, it is argued that peer factors become even more important. Early rejection by pro-social peers may increase the likelihood of association with aggressive or delinquent peers, and therefore the likelihood of further delinquent behaviour and the weakening of social bonds. School level factors such as poor grades, low-quality schooling or school policies such as suspension or expulsion, are also theorised to increase the likelihood of gang membership due to the weakening of school-student bonds and the potential for increased time without adult supervision.
In early adolescence, it is argued that the influence of community level predictors increases. Community factors such as high crime rates, drug use, and concentrated disadvantage may lead to decreased informal social control and decreased community attachment. This may lead to negative life stressors, delinquency, and the perception that gang membership offers benefits to the young person. Negative family characteristics (both structural and social process factors) may continue to affect young people by decreasing family bonds, increasing delinquency and reducing school performance. School risk factors such as poor academic performance, low aspirations, negative labelling by teachers and feeling unsafe at school may reduce attachment and increase the risk of gang membership. Delinquent beliefs and delinquent peers in early adolescence, and individual predictors including substance use, delinquency and life stressors such as violent victimisation may further increase the likelihood of delinquency and violence, a key precursor of youth gang membership.
Gang membership is theorised to be a culmination of interrelated structural and process factors. The model suggests that individual, community and structural family characteristics influence early pro-social behaviours and pro-social bonds. In an interactive feedback relationship, the model suggests that antisocial behaviours decrease pro-social friendships and in turn increase the impact of negative peer attachments and the risk of delinquent behaviours. These social and structural factors, in combination with negative life events, negative school experiences and a lack of school attachment, may increase the attractiveness of gang membership, not only for the most desperate in a community, but also for more ‘ambitious’ youth who see gangs as providing a positive alternative pathway.
Interventions to prevent youth gang membership can act on any of the five domains of risk factors, and at any of the developmental stages. The logic of preventive interventions is that they disrupt the developmental pathway to gang formation across any of the risk domains of individual, peer, family, school and community. There is no standard approach to preventive interventions, and as such, there is considerable variety in the programs implemented. Scholars suggest, however, that due to the cumulative and interactive impact of risk factors, interventions that address risk factors across multiple domains are likely to be the most successful (O'Brien et al., 2013; Klein & Maxson, 2006; Esbensen et al., 2009). Interventions can target all youth (primary prevention), at-risk youth (secondary prevention) or youth who are already gang-involved (tertiary prevention). The success or otherwise of preventive interventions can be measured both by the direct outcome of gang membership, and by the impact on gang-related crime, and we argue that the monitoring and evaluation of gang prevention programs using such outcomes is extremely important for the ongoing development of successful strategies. Figure 2 represents the relationship between categories of youth targeted by interventions and the outcomes and impacts that can be used as measurements of intervention effectiveness.
Relationship between interventions, outcomes and impacts
Why it is Important to do this Review
Two systematic reviews previously published in the Campbell library consider gang involvement for children and young people (Fisher, Montgomery, & Gardner, 2008a, 2008b), focusing on cognitive-behavioural and opportunities provision interventions to prevent gang involvement - interventions predominantly utilised in high-income nations. These reviews were essentially empty reviews as they did not identify any studies that met all of their inclusion criteria. Another review of comprehensive interventions designed to reduce gang-related crime was conducted by the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre, 2009). This also focused on high-income countries, and found that there was a small positive but not statistically significant effect of comprehensive intervention in reducing gang crime.
We propose that there are clear differences in the application and success of gang prevention programs between those implemented in high income (predominantly western) nations, and those implemented in low- and middle-income nations. We suggest that the motivations for joining and remaining with a gang will differ across regions for a variety of reasons, primarily because many low- and middle-income countries experience - or have experienced - some form of war or conflict (for example, Colombia, Nicaragua and South Africa). Post-conflict societies can provide fertile ground for gang formation and gang violence. In some post conflict nations, people live within an existing culture of violence, experiencing a low sense of citizen security and distrust of authorities alongside poor economic outlooks and easy access to firearms and drugs (Cruz, 2007; Davies & MacPherson, 2011).
Given the different antecedents, motivations, and social, economic and political conditions that give rise to gang formation and gang violence, a review on interventions aimed at combating youth gang formation and violence in countries classified as low- and middle-income by the World Bank will address some of the identified gaps in the research literature (World Bank, 2013).
This review aims to inform not only the academic literature on the effectiveness of preventive interventions, but also to provide a valuable resource for both policy makers and practitioners to assist in selecting the most appropriate interventions for implementation. Preventive gang interventions in low- and middle-income countries are funded and implemented by NGOs, government agencies, international aid agencies, and community organisations. This systematic review has been funded by the United States Agency for International Development (USAID), with the aim of informing best practice in youth gang interventions. USAID supports a variety of preventive anti-gang programs in Latin America and the Caribbean, including both primary and secondary prevention programs, and argues that evaluation is important to improve programs and build support for crime prevention programs (USAID, 2010b).
OBJECTIVES
There are two key objectives to this review. The first objective is to review the evidence on the effectiveness of interventions designed to prevent youth involvement in gangs and gang crime in low- and middle-income countries. This objective has two parts: to summarise the overall effectiveness of interventions, and to examine variability in effectiveness across different interventions and populations. The second objective of the review is to identify the reasons why preventive interventions to reduce youth involvement in gangs and gang crime may fail or succeed in low- and middle-income countries.
METHODOLOGY
CRITERIA FOR INCLUDING STUDIES IN THE REVIEW
Characteristics of the studies relevant to the objectives of the review
To be included in the review, a study must either evaluate the impact of preventive gang interventions using an appropriate quantitative methodology (Objective 1) or evaluate the reasons for success or failure of preventive gang interventions using either a quantitative or qualitative methodology (Objective 2). The review is conducted alongside a broader project on conduct problems and crime in low- and middle-income countries (Murray et al., 2013) and utilises the broad set of studies identified in that project, with further refinement during screening to ensure that the studies are relevant to preventive gang interventions.
Types of participants (population)
This review focuses on preventive interventions aimed at reducing youth involvement in gangs and gang violence. Whilst research suggests the majority of gang members are 12 to 24 years of age (Howell et al., n.d.; Huff, 1993; Rodgers, 1999; Seelke, 2013), we acknowledge that the definitions of youth vary by country, and that a strict age cut-off may not be appropriate. We will therefore extend the age range to include studies where the participants are aged between 10 and 29, in part because formal definitions of youth vary across countries, and in part to ensure that the age range is broad enough to ensure that tertiary prevention programs targeting current and ex gang members are not excluded.
We acknowledge that there is no consensus definition of a youth gang; therefore we take a broad approach and include any intervention where (1) the target group meets the Eurogang definition of youth gangs, “a street gang (or troublesome youth group corresponding to a street gang elsewhere) is any durable, street-oriented youth group whose involvement in illegal activity is part of its group identity” (Weerman et. al., 2009, p.20), (2) the target group is identified by the authors as members of a youth gang or equivalent (for example, pandilla, maras etc), or (3) involvement in youth gangs is a measured outcome of the study. We exclude groups described as organised crime gangs, terrorist gangs and piracy gangs.
This review is focused on interventions to reduce youth gang membership in low- and middle-income countries; therefore, we will only include studies that take countries that have been classified by the World Bank as low- and middle-income countries for at least 50 per cent of the time since 1987, when recordings of country classifications start (World Bank, 2013).
Types of interventions
Interventions must adopt a preventive approach, implemented at either primary, secondary, or tertiary stages of prevention, as described in the Interventions section of the Background (above). There are a very wide rang e of activities that fall under the banner of preventive interventions; however, in general, preventive interventions focus on capacity building or social prevention to prevent or reduce gang membership or gang violence.
We take a broad approach to inclusion, based on the stated intent of the intervention to reduce or prevent gang membership or gang crime, and we exclude interventions that achieve this aim purely by the use of suppression strategies and tactics such as increased law enforcement or focused legislation. Interventions included in this review must use a preventive approach and either explicitly aim to (1) reduce participation in youth gangs, or (2) to reduce youth involvement in gang crime.
Types of outcome measures
Studies included to address the objective of assessing the effects of preventive interventions to reduce youth gang membership (Objective 1) may include a number of outcomes. These include the change in youth gang participation and the change in the negative consequences of youth gang activities, including levels of crime and violence.
We will include all outcomes related to individual or aggregate measures of youth participation in gangs and/or gang crime. These outcomes may include: individual measures of arrests, reoffending, or gang membership; self-reported, peer-reported or officially-reported crime; geographically aggregated measures of youth gang participation, youth gang arrests and/or youth gang violence; and perceptions of youth gang participation and/or youth gang violence. We will analyse these outcomes separately at the synthesis stage. In particular, we will ensure that individual and geographically aggregated outcomes are analysed separately.
Other issues
To address the objective of identifying reasons for implementation success or failure (Objective 2), we will include a broader range of studies that assess the reasons for success or failure of preventive gang interventions as outlined above. From these studies we will include any research based findings relating to implementation. Examples of types of findings include those relating to political support, funding, training, the presence of international aid, community participation, education component, social support components, and the socio-political context of the implementation of the intervention.
Types of study designs
To address the two objectives of this review, we will use two different, but potentially overlapping, sets of studies. The analysis for Objective 1 (intervention effectiveness) will use experimental and quasi-experimental counterfactual evidence, whilst the analysis for Objective 2 (reasons for intervention success or failure) will include relevant studies from the corpus for objective 1, as well as a further set of qualitative or descriptive quantitative studies and process evaluation studies. The study designs for the two objectives are listed in detail below.
Study designs for Objective 1: Intervention effectiveness
To be included in the synthesis of intervention effectiveness, studies must use an experimental or quasi-experimental evaluation design with a valid comparison group as defined below. We will include the following experimental and quasi-experimental study designs, all of which provide a counterfactual analysis: randomised control trials regression discontinuity designs quasi-experimental, cross-sectional, cohort or panel designs that use multiple regression analysis and control for some combination of pre-intervention control variables listed below matched control group designs (with or without baseline measurement) unmatched control pre- and post-test designs, and time-series designs (at least 25 pre- and 25 post-intervention observations).
Studies that use valid comparison (control) groups are those that use randomly assigned control groups, propensity score matched control groups, or statistically matched control groups. Appropriate matching variables include: baseline measures of crime, delinquency, aggression or gang membership, or pre-intervention socio-demographic characteristics such as some combination of age, gender, ethnicity, socio-economic status, and education. We will also include designs that use non-matched control groups, if the study also takes a pre-intervention baseline measure of the outcome, thereby allowing difference-in-difference analysis.
The quasi-experimental designs we have included can be used to provide causal inference, albeit weaker inference than that which is provided by RCTs, as they provide a counterfactual by attempting to control for selection bias. This can be done in a number of different ways, such as: simulating randomisation of the treatment and control groups (regression discontinuity), matching the characteristics of the treatment and control groups (matched control), statistically accounting for differences between the treatment and control groups (multiple regression analysis), or providing a difference-in-difference analysis (short interrupted time series, unmatched control with pre-test). We do recognise that including a wide range of quasi-experimental study designs may lead to an increased risk of bias introduced into the analysis. We will conduct meta-analysis separately for randomised and non-randomised research designs, and will conduct moderator analysis on study design to assess whether including these studies changes the estimate of effect size.
We will include studies that measure the outcome at either the individual level or an aggregate level of geography such as the community; however, we will synthesise the results separately for different levels of analysis.
To be eligible for inclusion in a meta-analysis, the study must report an effect size, or provide sufficient detail such that an effect size can be calculated.
Eligible comparison conditions
We will include studies where the control group receives no intervention, placement on a wait-list or “business as usual”. We will also include studies that compare two treatments without reference to a no-intervention, wait-list or business as usual control group. We will conduct meta-analysis separately for studies that compare two active treatments.
Study designs for Objective 2: Reasons for intervention success or failure
To be included in the synthesis of factors influencing implementation success, studies are not required to use experimental or quasi-experimental designs. These studies need not be linked to the studies of intervention effectiveness, and will form an additional corpus of literature in which the authors have identified mechanisms, activities, people or resources that influence the success of the intervention implementation.
In order to capture the broadest range of evidence that speaks to the reasons for success or failure, we will include (1) qualitative or descriptive quantitative studies and (2) process evaluations and other types of implementation evaluations. These studies may use qualitative rather than experimental or quasi-experimental designs; for example, key informant interviews or focus groups.
We will only include studies that empirically assess the intervention using either a quantitative or qualitative methodology, and report on the sampling strategy, data collection, and the type of analysis. We will exclude descriptive papers and opinion pieces where an analysis of primary data was not conducted. Studies rated as low quality on the CASP checklist will be excluded from the review.
Exclusion criteria
We will exclude studies from countries that have not been categorised as low- or middle-income by the World Bank for at least 50% of the time since 1987.
SEARCH METHODS FOR IDENTIFICATION OF RELEVANT STUDIES
The search for eligible studies is conducted as part of a broader project that is systematically reviewing literature on conduct problems and crime in low- and middle-income countries (Murray et al., 2013). The search strategy will include published and unpublished literature with no date constraints. We will also not place any language restrictions on the eligibility of documents; however our search of published literature will be conducted in English and we will search grey literature in seven languages: English, French, Chinese, Arabic, Russian, Spanish and Portuguese. The geographic location of studies will be limited to countries located in a LMIC, defined according to the World Bank 2 as low- or middle-income at least 50 per cent of the time since 1987, when the recordings start 3 . The countries and regions currently classified by the World Bank as low- and middle-income are shown in Table 1.
Countries classified as “low- and middle-income” and their corresponding region (World Bank, 2013)
Search terms
This systematic review is conducted as part of a larger project focusing on conduct problems and crime in low- and middle-income countries (Murray et al., 2013) and alongside a systematic review on predictors of youth gang membership in low- and middle-income countries (Higginson et al., 2013). The search terms are broad enough to capture a corpus of studies for the present systematic review as well as for the predictors review (Higginson et al., 2013), and further refinement will occur at the abstract and title screening stage for each review. For the present review, the studies located in the search will be examined to determine whether they are eligible to address Objective 1 or Objective 2, or both, as studies may address questions of impact effectiveness as well as reasons for intervention success or failure (see Full text eligibility screening for further details).
The search strategy was developed using the Cochrane Collaboration's Effective Practice and Organisation of Care Group search strategy for low- and middle-income countries, combined with selected MeSH/DeCS terms and free text terms relating to conduct problems, crime and violence. To maximise sensitivity, no methodological filters were used. The full search strategy is listed in Appendix A.
Search locations
We will search a wide range of electronic academic databases, international organisation databases, the websites of NGOs and other organisations. The search locations are listed in Table 2. All locations will be searched electronically.
Search locations used in the English language systematic search (hosting platforms in parentheses)
Table 3 shows the locations to be searched in languages other than English. Due to the nature of database interfaces, the searches in these databases will be less systematic. The outcome search terms will be used and, where possible, the search terms for child and youth age groups. The non-English language searches will be conducted by a team of six researchers (four native speakers and two speaking the search language fluently).
Search locations used in the non-English language systematic search
Gang-specific search terms for first step of title and abstract screening
We will conduct citation searches and undertake citation harvesting from the references of included studies. We will contact members of the Advisory Group as well as other prominent scholars in the field to locate further studies that may not yet be published or located in our search. Any new literature of interest will be obtained and assessed for eligibility.
DATA COLLECTION AND ANALYSIS
Selection of Studies
Title and abstract screening
As the wider search strategy includes a broad array of studies on youth in low- and middle-income countries, the first step will be to search within the results for terms specific to gangs. We will export the full search results from EndNote to Access and search for any occurrence of the gang-specific terms that appear in Table 4. The group of studies that contain these terms will be considered potentially eligible and will be imported into SysReview, a Microsoft Access database designed for screening and coding of documents for systematic reviews.
A team of trained research assistants will use a set of inclusion criteria to assess, on the basis of titles and abstracts, whether the studies returned from the systematic search are potentially eligible for inclusion in the systematic review. After training to ensure that each reviewer is adopting the same approach to screening, each document will be screened by only one reviewer. The training will include a comprehensive briefing by the review manager, including reading and discussion of the protocol, followed by each reviewer independently screening a set of 20 studies. The results of the initial screening of the training corpus will then be mediated by the review manager, in consultation with the full review team. Further blocks of 20 studies will be reviewed independently by each member of the review team, and mediated by the review manager. Once the review team reaches an agreement rate of above 95 per cent, the subsequent screening of each document will be conducted by only one reviewer. Any issues or questions that arise during coding will be discussed amongst the review team and the review manager, and the review manager will randomly check screening decisions to ensure consistency.
The title and abstract screening inclusion criteria are: all participants are 10-29 years old the study is located in a LMIC, defined according to the World Bank as low- or middle-income at least 50 per cent of the time since 1987, when the recordings start the document reports on youth gangs
Documents will be excluded if the answer to any one of the criteria is unambiguously “No”, and will be classified as potentially eligible otherwise. We will err on the side of inclusivity and only exclude studies where it is clear that these criteria are not met.
Full text eligibility screening
The full text document will be located for all studies screened as potentially eligible at the title and abstract stage, and attached to SysReview. If dissertations are located that are potentially eligible for inclusion we will contact the author or their institution for a copy of the document. In order to narrow down the results of the initial search to the subset of studies that specifically focus on preventive interventions in youth gangs, different criteria are included at the full text eligibility screening stage.
The team of research assistants will be trained on full text eligibility screening and will each screen a corpus of 20 eligible studies independently. All screening conducted during training will be double checked by the review manager to ensure accuracy and consistency of information capture. Screening discrepancies at the training stage will be resolved by discussion between reviewers, in consultation with the review manager if required. Once training is completed, each document will be screened by one research assistant only.
The full text eligibility screening criteria consists of nine screening questions, the answers to which will determine whether a study is eligible for the meta-analysis (Objective 1) or the thematic synthesis (Objective 2), or both. It is important to note that a study may be eligible for both the meta-analysis and the thematic synthesis. The process and the screening criteria are shown in the flowchart in Figure 3
Screening criteria for meta-analysis and thematic synthesis
Data Extraction
Trained research assistants will use the SysReview database, along with a detailed coding companion document, to code in detail the documents that are eligible for inclusion in the meta-analysis. The coding fields are shown in Appendix B, including information on study information, sample characteristics, risk of bias, outcomes reported, and effect size data.
The team of research assistants will be trained on coding and will each code a corpus of 10 eligible studies independently. All coding conducted during training will be double checked by the review manager to ensure accuracy and consistency of information capture. For the final coding, all coding and effect size data will be checked by a second reviewer who is not blinded to the initial coding. Coding discrepancies will be resolved by discussion between reviewers, in consultation with the review manager if required. For data from between-groups studies, relevant data will be input into Comprehensive Meta-Analysis software (Borenstein, Hedges, Higgins & Rothstein, 2005) to calculate standardised effect sizes and their standard errors.
Assessment of methodological quality and risk of bias
We will separately assess the risk of bias of experimental and quasi-experimental studies included in the analysis for Objective 1, and of quantitative and qualitative studies included in the analysis for Objective 2. For studies included in the analysis for Objective 1, we will use the IDCG Risk of Bias tool (see Appendix C for details). For the studies included in the analysis for Objective 2, we will use a modified version of the Critical Appraisal Skills Programme (CASP) Qualitative Research Checklist 31.05.13, adapted to deal with descriptive quantitative studies and process evaluations (see Appendix D for details). The risk of bias of each study will be assessed by one reviewer, and all studies will be double checked by the review manager, who will not be blind to the assessment. Coding discrepancies will be resolved by discussion between reviewers, in consultation with the review manager.
For the synthesis of effectiveness (Objective 1), we will not exclude studies on the basis of risk of bias, but will conduct moderator analysis to determine whether inclusion of studies with higher risk of bias impacts on the summary effect size, using the number of Yes answers as a continuous moderator variable. When assessing risk of bias we will not allocate a score or index, as extreme failure in one area can be more serious than minor breaches across multiple arenas. We will present the results of the assessments of risk of bias in a “traffic light” format (see de Vibe et al., 2012).
For the synthesis of reasons for success or failure of intervention implementation (Objective 2) we will not include studies where the quality is rated as low. For the purposes of this review, a study will be rated as low quality if the answer to all of the following items is ‘No’ or ‘Can't tell‘: Is the research design appropriate to answer the research question? Was the sampling strategy appropriate to the aims of the research? Were the analyses sufficiently rigorous?
STATISTICAL PROCEDURES AND CONVENTIONS
Statistical procedures and conventions for Objective 1: Synthesis of intervention effectiveness
Measures of treatment effect
For continuous outcomes we will use Hedges' g as the measure of effect size, as it includes an adjustment for estimator bias in smaller samples (Borenstein, 2009). If binary outcomes are found, we will calculate a log odds ratio as the measure of effect size.
We will input all effect size data into Comprehensive Meta-Analysis software (Borenstein et al., 2005) to allow the calculation of standardised effect sizes and their standard errors, and the conversion between effect size types, to ensure that a common metric is used. Should an outcome be measured across different studies using binary data in some studies and continuous data in others, we will convert all effect sizes and their variances for this outcome to a common metric. For example, log odds ratios will be converted first to Cohen's d and then to Hedges' g, and the meta-analysis will be conducted on all outcomes using Hedges' g as the effect size of choice. Following Borenstein and colleagues (2009), we argue that this approach, whilst imperfect, is preferable to conducting two separate meta-analyses.
Some studies may use an interrupted time-series design with observations at multiple time points before and after the implementation of an intervention in an area and some may use comparison groups in addition to multiple time points. For studies that collect data at multiple time points, we assume an underlying uniform distribution for violent crime, and a step function for the effect of the intervention on the outcome. We will therefore calculate an average effect size for the time points before the intervention, and an average effect size for the time points after the intervention, and compare the two. We recognise that there are many other ways to deal with this type of time series data; however, given the research questions and the likely nature of the intervention effect, we believe that this method is the most defensible and parsimonious. We will synthesise the results of time-series studies separately from other experimental and quasi-experimental designs, as time series designs standardise for variability over time rather than variability over units, resulting in a different scaling (D. Wilson, personal communication, September 20, 2013).
Unit of analysis
The standardised coding sheet contains fields to code both the unit of treatment and the unit of analysis. We will also assess each study for unit of analysis error, as part of the IDCG risk of bias tool. If a study is assessed as suffering from unit of analysis error, we will correct for the standard error and confidence intervals of the studies, using the formula SEcorrected = SEuncorrected ∗ ICC), where m is the number of units in each cluster, if the intra-class correlation (ICC) can be obtained or estimated.
Missing data
We will use reported statistics such as t, F, p, or z-values to convert to effect sizes if effect size data is not reported. If data required to compute effect sizes is missing, we will attempt to contact the authors of the studies.
Method of synthesis
If the systematic search results in at least three studies that provide effect sizes for a conceptually equivalent outcome we will use meta-analysis to synthesise the results for each equivalent outcome reported. We will use a random-effects model and inverse variance weighting to combine study results, given the likely heterogeneity in the interventions and populations studied. We will conduct all analyses using Comprehensive Meta-Analysis software (Borenstein et al., 2005).
We will only combine results of evaluations if the outcomes are conceptually equivalent. For example, if studies report on self-reported gang membership and officially reported gang-related crime, we will conduct two separate meta-analyses - one for gang membership and one for gang-related crime outcome - as we do not consider that these two outcomes are conceptually equivalent.
The following is a preliminary list of outcomes that we will synthesise separately: Individual (self-reported, peer-reported or officially-reported) measures of: total crime violent crime /assault/aggravated assault/robbery property crime/break and enter/ vandalism/theft Geographically aggregated measure of: total crime violent crime/assault/aggravated assault/robbery property crime/break and enter/ vandalism/theft Individual (self-reported, peer-reported or officially-reported) measures of: arrests reoffending gang membership Geographically aggregated measures of: arrests reoffending gang membership
We will conduct separate meta-analyses for outcomes measured at different levels of analysis (for example, individual, municipality, country). We will present the results of the meta-analysis in forest plots, including 95 per cent confidence intervals for individual studies and the overall effect.
We will conduct meta-analyses separately for randomised and non-randomised study designs, as well as for designs that compare two active treatments. If statistical meta-analysis is not possible due to a small numbers of effect sizes in each category, we will present the effect sizes from each study in a forest plot without providing an overall summary of effect sizes.
Assessment and investigation of heterogeneity
We will test for heterogeneity using I2, and Q statistics, following Borenstein et al. (2009). We will also estimate and report the between studies variance component (τ2).
We will code a range of study-level moderators that we expect would have an impact on the effect size. If there is sufficient information available, we will test the effect of key variables on the heterogeneity of the intervention impact, using moderator analysis for categorical predictors and meta-regression for continuous predictors. We will use a random effects model with inverse variance weighting for all moderator analyses. We anticipate that we will perform moderator analysis on target population (for example, school based samples, gender specific, age specific, gang members), geographic region (for example, school district, city, state), the type of preventive strategy used (for example, primary, secondary, tertiary, combined), source of data (for example, official data, self-reported, peer-reported, family-reported, practitioner-reported, other), and study design (experimental, quasi-experimental). We will distinguish in the final review between a priori planned analyses (those listed in the protocol) and post hoc analyses identified only during the analytic stage.
Criteria for determination of independent findings
There are two issues of independence that may occur in this review. The first is that documents may report multiple outcomes for one study. Documents will be allowed to contribute multiple effect sizes to the syntheses, but only one effect size for each outcome. If a study reports multiple effect sizes for one outcome, for example across multiple intervention sites within the one study, the mean effect size for that outcome will be calculated.
The second issue of independence is that multiple documents may evaluate the same intervention using the same data. Each intervention may only contribute one effect size for each outcome; therefore, if multiple related studies are identified, we will assess all sources in order to select an effect size. This assessment will be based on the completeness of the data and the risk of bias assessment of the studies, and all decisions will be reported in the final review.
Sensitivity analysis
We will conduct subgroup analyses in order to assess the impact of risk of bias assessments and study design. Using moderator analysis for categorical variables, and meta-regression for continuous variables, we will perform sensitivity analysis on the effect of risk of bias, publication status, publication year, and geographic level of analysis. We will use a random effects model with inverse variance weighting for all sensitivity analyses. We will distinguish in the final review between a priori planned analyses (those listed in the protocol) and post hoc analyses identified only during the analytic stage.
Assessment of publication bias
We will test and adjust for publication bias using funnel plots and trim-and-fill analysis as suggested in Rothstein, Sutton, and Borenstein (2005).
Treatment of qualitative research
We will not use qualitative research to evaluate the effectiveness of preventive youth gang interventions, but we will include qualitative research to address objective 2, as outlined below.
Methods of synthesis for Objective 2: Reasons for intervention success or failure
To address the second objective of the review and assess the reasons for the success or failure of preventive youth gang interventions, we will conduct a thematic synthesis of evidence on the reasons for success or failure of the implementation of preventive youth gang interventions. The aim of this is to supplement the findings of the synthesis of effectiveness. The synthesis will specifically focus on practical, policy-focused implications from the literature.
Method of synthesis
We will use the method of thematic synthesis outlined by Thomas and Harden (2008). A review team member with expertise in the analysis of qualitative data will use Leximancer 4 and NVivo 10 text analytic software (Leximancer Pty Ltd, 2012; QSR International, 2012) to identify and code the key themes in the included studies. The eligible studies will be initially categorised according to the type of intervention that is reported. One reviewer will read the full text of all eligible studies and record any barriers or facilitators of implementation that are identified by the study authors using NVivo software. In an iterative process, the extracted data will then be tabulated and each study re-examined in light of the collated list to ensure full data capture. The corpus of studies will be analysed using Leximancer software to identify any key overarching themes that can be identified using data mining. The facilitators and barriers will be mapped onto key themes. Each study will be classified by intervention type and the frequency of each key theme will be tabulated across intervention types. The identified factors will be examined both within intervention groups and across intervention groups to examine questions of generalisability.
The synthesis will be organised in two parts. The first part will be a descriptive analysis. The studies will be grouped according to intervention type, and each section will include a summary of study characteristics, textual descriptions of the studies, and the authors' conclusions about barriers and facilitators of implementation success. The descriptive analysis in the first part of the review will also include the development of logic models for those interventions with sufficient data to allow a robust model to be constructed. The second part of the review will contain a thematic summary. The results will be summarised according to key identified themes, and this section will contain an analysis of any barriers and facilitators of intervention success cut across the various interventions, and the extent to which the identified factors can be generalised. Tables of summaries of findings will be presented in the final review.
REFERENCES
SOURCES OF SUPPORT
Support for this study will be provided by the Institute for Social Sciences Research, the University of Queensland, and the ARC Centre of Excellence in Policing and Security.
This review is externally funded by USAID through 3ie (International Initiative for Impact Evaluation, Inc.) (SR/1117). The views expressed in this article are not necessarily those of USAID or 3ie or its members.
Funding for the broader database searching (Murray et al., 2013) was provided by the Wellcome Trust [089963/Z/09/Z].
DECLARATIONS OF INTEREST
None of the authors have any known conflicts of interest.
REVIEW AUTHORS
ROLES AND RESPONSIBLIITIES
Content: Angela Higginson, Joseph Murray, Lorraine Mazerolle, Laura Bedford, Kathryn Benier Systematic review methods: Angela Higginson, Joseph Murray, Yulia Shenderovich Statistical analysis: Angela Higginson Information retrieval: Yulia Shenderovich, Kathryn Benier, Laura Bedford
PRELIMINARY TIMEFRAME
PLANS FOR UPDATING THE REVIEW
The authors plan to update the review every five years.
AUTHORS' RESPONSIBILITIES
By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.
A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.
You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.
PUBLICATION IN THE CAMPBELL LIBRARY
The support of the Campbell Collaboration and the relevant Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review and subsequent updates in the Campbell Library. Concurrent publication in other journals is encouraged. However, a Campbell systematic review should be published either before, or at the same time as, its publication in other journals. Authors should not publish Campbell reviews in journals before they are ready for publication in the Campbell Library. Authors should remember to include a statement mentioning the published Campbell review in any non-Campbell publications of the review.
I understand the commitment required to undertake a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:
Form completed by: Angela Higginson
Date: 15 July 2014.
Footnotes
APPENDIX A
APPENDIX B
APPENDIX C
APPENDIX D
1
We will hereafter refer to the subset of youth who are at higher risk of joining gangs as “at-risk youth”.
3
This approach ensures that we include countries which have consistently been ranked as LMIC. For the vast majority of countries there has been very little change in status over the last few decades, therefore rather than cross-referencing countries against categorisations in the year the study was conducted, it is more efficient to establish the list of countries that meet 50% criteria. All excluded countries had either been consistently ranked as high-income or had moved from upper-middle-income to high-income during this period.
4
The tool has been developed by Jorge Hombrados and Hugh Waddington, drawing on existing tools, in particular EPOC (n.d.), Higgins and Green (2011) and Coalition for Evidence-Based Policy (2010). Thanks to Richard Palmer-Jones, Maren Duvendack and Phil Davies for comments on previous drafts.
5
If a quasi-randomized assignment approach is used (e.g. alphabetical order), you must be sure that the process truly generates groupings equivalent to random assignment, to score “Yes” on this criteria. In order to assess the validity of the quasi-randomization process, the most important aspect is whether the assignment process might generate a correlation between participation status and other factors (e.g. gender, socio-economic status) determining outcomes; you may consider covariate balance in determining this (see question 2).
6
If the research has serious concerns with the validity of the randomisation process or the group equivalence completely fails, we recommend to assess the risk of bias of the study using the relevant questions for the appropriate methods of analysis (cross-sectional regressions, difference-in-difference, etc) rather than the RCTs questions.
7
If the research has serious concerns with the validity of the assignment process or the group equivalence completely fails, we recommend to assess the risk of bias of the study using the relevant questions for the appropriate methods of analysis (cross-sectional regressions, difference-in-difference, etc) rather than the RDDs questions.
8
Accounting for and matching on all relevant characteristics is usually only feasible when the programme allocation rule is known and there are no errors of targeting. It is unlikely that studies not based on randomisation or regression discontinuity can score “YES” on this criterion.
9
There are different ways in which covariates can be taken into account. Differences across groups in observable characteristics can be taken into account as covariates in the framework of a regression analysis or can be assessed by testing equality of means between groups. Differences in unobservable characteristics can be taken into account through the use of instrumental variables (see also question 1.d) or proxy variables in the framework of a regression analysis, or using a fixed effects or difference-in-differences model if the only characteristics which are unobserved are time-invariant.
10
Please note that when a), b) or f) score no or large differences in baseline characteristics, we suggest assessing risk of bias considering other study design (Diff-in-Diff, cross-sectional regression, instrumental variables)
11
Even in the context of RCTs, when randomisation is successful and carried out over sufficiently large assignment units, it is possible that small differences between groups remain for some covariates. In these cases, study authors should use appropriate multivariate methods to correcting for these differences.
12
Knowing allocation rules for the programme - or even whether the non-participants were individuals that refused to participate in the programme, as opposed to individuals that were not given the opportunity to participate in the programme - can help in the assessment of whether the covariates accounted for in the regression capture all the relevant characteristics that explain differences between treatment and comparison.
13
Matching strategies are sometimes complemented with difference-in-difference regression estimation methods. This combination approach is superior since it only uses in the estimation the common support region of the sample size, reducing the likelihood of existence of time-variant unobservables differences across groups affecting outcome of interest and removing biases arising from time-invariant unobservable characteristics.
14
The Hausman test explores endogeneity in the framework of regression by comparing whether the OLS and the IV approaches yield significantly different estimations. However, it plays a different role in the different methods of analysis. While in the OLS regression framework the Hausman test mainly explores endogeneity and therefore is related with the validity of the method, in IV approaches it explores whether the author has chosen the best available strategy for addressing causal attribution (since in the absence of endogeneity OLS yields more precise estimators) and therefore is more related with analysis reporting bias.
15
If the instrument is the random assignment of the treatment, the reviewer should also assess the quality and success of the randomisation procedure in part a).
16
An instrument is exogenous when it only affects the outcome of interest through affecting participation in the programme. Although when more than one instrument is available, statistical tests provide guidance on exogeneity (see background document), the assessment of exogeneity should be in any case done qualitatively. Indeed, complete exogeneity of the instrument is only feasible using randomised assignment in the context of an RCT with imperfect compliance, or an instrument identified in the context of a natural experiment.
17
Contamination, that is differential receipt of other interventions affecting outcome of interest in the control or comparison group, is potentially an important threat to the correct interpretation of study results and should be addressed via PICO and study coding.
18
‘Common methods’ refers to the use of the most credible method of analysis to address attribution given the data available.
19
A comprehensive assessment of the existence of data mining' is not feasible particularly in quasi-experimental designs where most studies do not have protocols and replication seems the only possible mechanism to examine rigorously the existence of data mining.
20
All interventions may create expectations (placebo effects), which might confound causal mechanisms. In social interventions, which usually require behaviour change from participants, expectations may form an important component of the intervention, so that isolating expectation effects from other mechanisms may be less relevant.
21
Standard errors may be inflated in parametric approaches if the intervention does not have a homogeneous effect across the whole sample population, and the authors fail to conduct appropriate sub-group analyses.