Protocol for a systematic review: Crop targeting's impact as a drug control strategy

Logic Model

Figure 1 (below) provides a Logic Model to help guide the review. The logic model presents a simplified, graphical interpretation of the logic of intervention effectiveness. The boxes and arrows represent interventions, outcomes and relationships that exist in the overall logic of crop targeting as a drug control strategy. As this logic model shows, the primary goal of the interventions is to reduce the supply of illegal drugs. By implementing crop targeting initiatives such as eradication and substitution, various direct (proximate) outcomes are produced. For the purpose of this review we choose to code a range of direct and indirect outcomes. Direct outcomes could include: seizure rates, production rates, cultivation, yield, net farm income, and the number of eradicated hectares. As demonstrated in the far right box, indirect outcomes could include: violence, economic, harm, or unintended outcomes such as displacement of drug cultivation.

OPEN IN VIEWER

1

Logic model of interventions involving crop targeting as a strategy for drug control

Displacement and Diffusion of Crime Control Benefits

Our review will also record information, where possible, on displacement, spill-over or unintended consequences of any crop targeting intervention that seeks to control the supply of illegal drugs. In this review, unintended consequences are defined as consequences/outcomes that were not planned for in a plan of action (Chouvy, 2012). Unintended consequences for eradication could include an increase in poverty and cultivation, or environmental pollution. A recent systematic review of interventions that measured macro-level displacement of crime examined international displacement of important drug and precursor laboratories (see Weisburd, Telep, Teichman, Gill, & Vitter, 2011). This research showed that the U.S. national drug control policy may have been responsible for pushing drug laboratories such as methamphetamine labs across country borders (Weisburd et al., 2011). They found, for example, that “improved controls in Canada and further tightening of controls in the U.S. have led to a shift of production across the border to Mexico” (Weisburd et al., 2011). This finding demonstrates the importance of examining the potential side-effects of crop targeting initiatives as means of dealing with the control of illegal substances.

In summary, our review will focus on crop targeting efforts to control illegal drug supply at the wholesale level of the illicit drug supply chain. Even with the re-positioning of the U.S. drug policy toward a combination of prevention, education, and smarter use of law-enforcement resources (Kerlikowske, 2011), drug policies throughout the world continue to include, and rely upon, law enforcement and supply-side reduction strategies. Our systematic review seeks to provide policy makers with the research evidence to help guide a smarter use of scarce law enforcement resources aimed at the wholesale level of efforts to control the supply of illegal drugs.

Types of participants

The units of analysis will be any geographic place (e.g. farm, village, province, state, region, country or countries) that is the subject of crop targeting interventions. In order to obtain an accurate global overview of the effectiveness of crop targeting as a strategy for drug control, there will be no geographic limitations for inclusion.

Types of interventions

To be eligible for inclusion in the review, studies must have examined the impact and/or effectiveness of a crop targeting strategy aimed at controlling the wholesale supply of illicit, plant-based drugs. For the purpose of our systematic review, crop targeting interventions are defined as interventions aimed at reducing the supply of drugs through destroying or suppressing the cultivation of illegal drug crops. We will include interventions that involve crop targeting and explicitly state that the initiative, program, policy, or legislation are aimed at managing, reducing, removing, curtailing, stopping or eradicating illicit plant based drug crops.

The drugs that will be considered in crop targeting interventions are all drugs that are illegally grown and cultivated according to international agreements and local (national) laws. As outlined in the World Drug Report 2011, such illegal drugs include cannabis, coca (base or paste), opiate and poppy straw (United Nations Office on Drugs and Crime, 2011). The drugs must be plant-based, thus any illicit drugs that are chemical based will be excluded from the review e.g. amphetamine, methamphetamine. We will not include interventions that seek to eradicate precursor chemicals (non-plant based) used to create illicit drugs unless the strategy relates to crop targeting as a drug-control strategy.

We will only include crop targeting activities that focus on reducing cultivation at the wholesale level of drug activity by suppressing or reducing the farming of illicit drug crops, and will exclude activities targeting the cultivation of illicit drug plants purely for personal use. Wholesale level crop targeting interventions that seek to reduce the wholesale supply of illegal drugs and disrupt the supply chain might include, for example, eradication of crops (Andersson, 2010), substitution of illicit crops (Carpenter, 2005), alternative livelihoods, alternative development (Berg, 2002), and crop monitoring (Tian et al., 2011).

An example of a well-known crop targeting initiative is the Cannabis Crop Eradication Programme, a New Zealand crop targeting initiative, involving a series of police operations designed to reduce the cultivation of cannabis by destroying the supply at the source using fixed wing aircraft and helicopters (Wilkins, Bhatta, & Casswell, 2002). A second example of a wholesale level crop targeting initiative is Plan Colombia, a Colombian and U.S. effort to reduce the supply of illegal drugs entering the U.S. market (Veillette, 2005). With United States support through the State Department's Office of Interregional Aviation, the initiative involved aerial eradication through the spraying of coca and poppy crops with a glyphosate herbicide mixture (Veillette, 2005).

Alternative development interventions include crop targeting activities that assume that “drug problems are closely linked to development problems and that effective development policy measures can bring about a sustainable reduction in drug cultivation” (Berg, 2002, p. 1). Berg argues that alternative development policies aim to create economic and social conditions in which households can achieve an acceptable standard of living without having to resort to drug cultivation. In Thailand, for example, alternative development projects have been used to target opium production (see Berg, 2002).

Types of outcome measures

Interventions which deal with some outcome measure of drug production, prevalence and availability of the drug on the illicit market will be included in the review, including: production, cultivation, yield, net farm income, market availability and number of eradicated hectares. We will differentiate between effects measured at the local level (such as farm gate prices) and at the global level (such as worldwide estimates of drug consumption) in our analyses.

As figure 1 shows, we will include a number of indirect outcome measures in our review, for example, crime rates, measures of improvement in democracy or the security of the country, economic outcomes, violence outcomes and harm outcomes. Moreover, we will include any unintended outcomes in our review. All six types of displacement and diffusion (spatial, target, temporal, tactical, perpetrator, and type of crime) will be coded (Windle & Farrell, 2012).

Outcome measures relating to harm reduction or demand reduction will be included in our review. We will include any study that reports a harm outcome (as an indirect outcome) resulting from a crop targeting intervention.

We recognise that as the review has no geographic restriction, the mechanisms of measurement for the outcomes may vary considerably. For example, we have identified studies that measure cultivation in two distinctly different fashions. In one set of studies, conducted in Colombia, coca cultivation is measured as a continuous variable: the number of square kilometres under coca cultivation in an area (see for example, Dion & Russler, 2008). In another set of studies, conducted in Afghanistan, opium poppy cultivation is measured as a dichotomous variable: whether a village expected to plant opium poppies that year (see for example, UNODC, 2012). Both of these studies rely on data collected by UNODC; the data on number of square kilometres cultivated is gathered through aerial surveys, whilst the data on village cultivation relies on the responses of key informants, which we would argue are intrinsically less reliable. We suggest that there may be a perceived incentive for village chiefs who have received alternative development assistance to report that their village will not subsequently cultivate illicit drug crops, if doing so is seen as increasing the likelihood of continued assistance. We will code each study for the possibility of introduced bias and conduct sensitivity analysis with study quality as a moderator variable. We anticipate that other outcomes such as consumption, harm, and farm income will likewise be measured in a variety of ways, and pose similar challenges.

Types of study designs

Because we anticipate a limited pool of relevant studies in our global search, we will include in our review both studies that measure the causal impact of crop targeting on outcomes and studies that measure the association between crop targeting and outcomes. We will, however, synthesize the effect sizes from the different sets of studies separately. We describe the range of studies included in our review below.

First, experimental designs and certain quasi-experimental designs can be used to evaluate the causal impact of introducing an intervention. We will include randomised control trials (RCT) where crop targeting interventions are assessed against a control group with random assignment to the intervention and control conditions. We will also include a subset of quasi-experimental designs which we refer to as “strong” quasi-experiments. Strong quasi-experiments can be used to provide causal inference, albeit weaker inference than that which is provided by RCTs, as they provide a counterfactual by attempting to control for selection bias. This can be done in a number of different ways, such as: controlling the assignment of cases to treatment and control groups (regression discontinuity), matching the characteristics of the treatment and control groups (matched control), statistically accounting for differences between the treatment and control groups (multiple regression design), or providing a difference-in-difference analysis (parallel cohorts with pre-test and post-test). We will include the following quasi-experimental designs in our synthesis of the causal impact of crop targeting interventions:

regression discontinuity designs,

For those quasi-experimental designs that use matched control groups, we will accept both propensity score matched controls and statistically matched controls. We will also accept non-matched control groups, provided that a pre-intervention measure is recorded.

A second set of study designs that can provide good evidence for causal impact, but which will be synthesised separately, are long interrupted time-series designs without control groups (at least 25 pre and 25 post observations (Glass, 1997)). We will not synthesise the results from time-series designs together with other causal study designs, as the effect size from a time-series design has a different meaning than a numerically equivalent effect size from other quasi-experimental studies. We will synthesise the effect sizes from time-series designs separately.

The third set of study designs include correlational studies. Unlike the study designs listed above, correlational and “weak” quasi-experimental designs cannot be used to demonstrate causality, and can only be used to demonstrate the magnitude of the relationship between crop targeting and the outcome. In this group of designs we will include both raw unadjusted correlational designs, as well as the subset of quasi-experimental designs that we refer to as “weak” quasi-experiments. Weak quasi-experiments include unmatched control group designs without pre-intervention measures. The reason for the lack of evidence for causal inference from these types of studies depends on the study design. In raw unadjusted correlational designs where the variation in the level of the intervention is compared to the variation in the level of the outcome, there is no comparison group which can demonstrate that the observed relationship is not due to extraneous factors. For example, we have identified studies that examine the impact of crop targeting over time within one country, reporting correlation between the annual number of hectares under illicit drug cultivation and the prior-year number of hectares eradicated. These studies show the association between levels of intervention and outcomes, but this association cannot be considered causal, as it may be due to unmeasured factors. Similarly, unmatched control group designs cannot demonstrate the counterfactual without a baseline pre-intervention measurement to control for selection bias. These designs can demonstrate an association between the presence of an intervention and an outcome, but they cannot measure the causal impact of introducing the intervention. An example of a weak quasi-experimental design that occurs commonly within the crop targeting literature is a post-test only design with an unmatched control group. The results from this type of design are commonly represented as a two by two contingency table, where villages are divided into those that received the intervention in the previous year and those that did not (without statistically matching the groups on other variables), and the outcome is measured as the number of villages that cultivated illicit drug crops in the current year. We note that this design is the most likely to demonstrate selection bias, and will therefore conduct sensitivity analyses of the meta-analysis using study design as a moderator variable.

Finally, in addition to correlational and weak quasi-experiments, studies that address questions of causality are often used to address questions of association (Cooper, 2009). We therefore group the following types of research designs as those that can be used to address the magnitude of association between crop targeting and outcomes:

randomised control trials

Exclusion criteria

Since we are focusing primarily on the reduction of wholesale drug supply through crop targeting, any evaluation of interventions that are not directed at plant-based drugs and targeted toward activities to reduce/eliminate crops will not be included in the review.

We will also exclude all of the street-level drug law-enforcement interventions included in Mazerolle and colleagues' earlier review of “street-level drug law enforcement” (Mazerolle et al., 2007). Interventions such as community-wide policing, problem-oriented policing and hotspots policing will all be excluded unless the evaluation explicitly states that the intervention approach is aimed at the wholesale level of the market and used to target crop cultivation activities.

Settings and timeframe

We will include documents produced after 1 January 1980. We will not limit the country or region where the intervention was staged.

Search terms

A list of preliminary search terms was developed to cover four key categories: primary intervention; intervention; outcome; and substance. The primary intervention terms ensure that the intervention of interest is applied to crops. The intervention terms capture the variety of crop interventions that take place, and narrow the focus of the search onto interventions designed to destroy or target crops, or capture a law enforcement aspect of the document. The outcome intervention terms ensure that the intervention is measured by its effect on the market. The substance terms ensure that the document is focused specifically on illicit plant-based drugs. The search strategy combines the sets of search terms with a Boolean AND. 1.

Primary intervention

CROP or CROPS

The search term list was generated using Leximancer software (available from https://www.leximancer.com). Using a list of search terms derived from reading background literature, we ran a search on the Web of Knowledge and Scopus databases. We sorted the results according to relevance and selected the first 100 from each database (with abstracts, if available). These articles were fed into Leximancer, which generated a list of themes and concepts pertinent to the body of texts. From Leximancer, we took the top ten themes and concepts, as well as other concepts that we thought relevant, to arrive at a list of initial search terms. Certain broad concepts, such as “plant” and “drug”, were removed in order to make the search more topic focused. A comparison of the results from our initial keyword search in Web of Knowledge with the keywords formulated from the Leximancer keywords found that a small number of texts were missed by the Leximancer keyword search. Examination of these texts resulted in the addition of “alternative cultivation” to the list of keywords, and the use of wild card notation to “substitution” (substitut*). For a more detailed treatment of this technique, see Thompson and colleagues (2013). Through feedback from reviewers and the project advisory group, these keywords will be revised and modified accordingly.

Whilst we will not exclude documents based on language, we will only search using English terms due to budgetary constraints.

Search locations

The list below provides a starting list of databases and websites to be searched for the review:

Abdul Latif Jameel Poverty Action Lab (J-PAL) (www.povertyactionlab.org)

As with the preliminary list of search terms, the modification of this list of databases will occur in accordance with the feedback from the project advisory group. In order to maintain an accurate list of search terms, a pilot search will be conducted so that any modifications to the list can be made prior to commencing the systematic search. The exact search term combinations used to search the different locations are listed in Appendix E.

After the initial list of eligible documents is identified, we will search the reference lists of all eligible documents. Newly identified documents will go through the title and abstract screen, document retrieval and document coding stages. This iterative process will continue until no further new documents are identified.

Once we have completed the list of eligible studies it will be sent to the project Advisory Group to determine whether or not we missed any important sources. Furthermore, the authors of the included studies will be sent the list and asked for recommendations for further sources.

Once the search is completed, duplicate records will be removed, and the bibliographic details of each potentially eligible document will be exported to a Microsoft Access database at the University of Queensland.

Search language

Although we will conduct our search in English, we still anticipate that this search will yield studies written in languages other than English. Since this review is targeted at any state, province, region, country or countries that engage in a crop targeting intervention to reduce the supply of drugs, it is difficult to account for the array of languages which may or may not be encountered within the found studies. We assume that some of the evidence base for this review will be published in languages other than English. Therefore, we have budgeted for translation services to translate located documents into English from up to five predetermined languages. These languages will include Spanish, because of a large literature around crop targeting drug interventions in Latin American countries (see for example Dion & Russler, 2008; Tabares & Rosales, 2005; Reyes-Hernandez, 2010), and may include up to four other languages selected for optimum geographic coverage. The areas where the interventions have occurred will play a large role in determining these other languages. Such languages may include Chinese (Mandarin), Arabic, French, and Russian, as these are official languages of the United Nations and cover a wide geographic and social range.

Selection of studies

Study coding

A team of trained research assistants will code the documents using the “Crop targeting review coding sheet companion” (Appendix A). Each coder will first code a selection of 15 documents for eligibility and inter-coder agreement will again be assessed. Disagreements in the coding of the training corpus will be discussed between the coders and the review manager, as part of training to ensure coding consistency.

Documents will be read in detail and coded according to document eligibility, study information, intervention information, implementation success, quality, authors' conclusions, and outcomes. Each document may contain multiple studies which may in turn report on multiple outcomes. The coding database captures this nested data arrangement. Details of coding fields are contained in Appendix A.

As a further quality control measure, all documents which are coded as eligible will have their coding double-checked by the review manager.

Assessment of study quality

We will assess study quality using an adapted version of the Campbell Collaboration International Development Coordinating Group (IDCG) Risk of Bias tool (see Appendix D). We will not allocate a score or index, as extreme failure in one area of study quality can be more serious than minor breaches of quality across multiple arenas. Rather we will make a critical qualitative decision for each study as to whether there is a clear risk of bias such that the study quality is sufficiently low to warrant exclusion from the review. Any evaluations that are excluded on the basis of quality will be listed in the final review. We will present the results of the assessment of study quality in a “traffic light” format (see de Vibe et al., 2012).

Method of synthesis

If the search results in the identification of suitable data for meta-analysis, we will use meta-analysis to synthesize the results of the included evaluations. We will conduct a meta-analysis if we have at least two independent effect sizes that measure a conceptually equivalent outcome, measured at an equivalent unit of analysis. As discussed in “Study designs”, we will synthesize the effect sizes separately for experiments and strong quasi-experiments, long time series designs, and correlational and weak quasi-experimental designs, and we will conduct separate meta-analyses for conceptually different outcome measures, and for effects measured at different units of analysis. If a study reports multiple effect sizes for the one outcome, we will use the mean effect size for that outcome. We will use a random-effects model with inverse variance weighting to combine study results.

Our method of synthesis for results from regression models will depend upon how many studies we identify that use multiple regression analysis. Ideally, when interpreting regression studies, we are interested in the partial effect of the intervention, after controlling for a number of covariates. However, we recognize that there may be differences in the covariates used in regression analyses across studies. We will synthesise the results from the regression models separately to the results from experiments and quasi-experiments; however, if our search identifies a large number of regression studies, we will consider conducting a model-based meta-analysis. In this circumstance, we will contact the study authors to request the zero-order correlation matrices, and will then follow Becker (2009) in estimating the mean correlation matrix, R , and from this matrix use the key variables across studies to synthesise an overall random-effects multiple regression model.

We expect that a number of factors will introduce heterogeneity into the analysis, specifically the type of intervention strategy, crop type, and geographic location. We anticipate that, for example, forced eradication will be less successful in reducing crop cultivation than alternative development strategies, as the latter provide additional pathways for income support for farmers, whilst the former remove what may in many instances be the farmers' most successful cash crop. We also expect that there will be different impacts of a similar strategy in different geographic locations, due to the varying socio-political and economic environments. We will examine these variables as sources of heterogeneity in the intervention impact, using subgroup analysis (analogue to the ANOVA) for categorical outcomes and meta-regression for continuous variables. If the analysis shows significant heterogeneity of effects across locations, we will display this effect graphically in a series of maps to inform interpretability of the results.

We will present the results of the meta-analysis in forest plots, including 95 percent confidence intervals for individual studies and for the overall weighted mean effect estimate. We will test and adjust for publication bias using a range of approaches suggested in Rothstein, Sutton, and Borenstein (2005); depending on the data collected, this may include funnel plots and trim-and-fill analysis. We will conduct sensitivity analysis to test the effect of study design and evaluation quality on the results of the analyses. We expect that we would see a smaller effect for experimental study designs and more rigorous evaluations, as these are less likely to introduce bias to the effect size estimate. We will use Comprehensive Meta-Analysis software (Biostat, 2005) for calculations and production of figures.

Effect size metric and calculations

We will calculate a range of effect sizes, depending on the data available from each study, and then aim to convert these to a common metric where possible. For continuous outcomes we will calculate the standardized mean difference using Hedges' g as the measure of effect size, as it includes an adjustment for estimator bias in smaller samples (Borenstein, 2009). If binary outcomes are found we will calculate a log odds ratio as the measure of effect size. If we locate studies that report regression coefficients we will calculate the standardized mean difference from the regression coefficients if there are only a small number of such studies, but if a larger body of studies is identified, we will consider conducting a model-based meta-analysis as discussed above. For raw unadjusted correlations we will extract r as the effect size of choice. As a quality control measure, all effect size calculations will be double-checked by a second reviewer.

Some studies may use an interrupted time-series design with observations at multiple time points before and after the implementation of an intervention in an area and some may use comparison groups in addition to multiple time points. For studies that collect data at multiple time points, we assume an underlying uniform distribution for violent crime, and a step function for the effect of the intervention on the outcome. We will therefore calculate an average effect size for the time points before the intervention, and an average effect size for the time points after the intervention, and compare the two. We recognize that there are many other ways to deal with this type of time series data; however, given the research questions and the likely nature of the intervention effect, we believe that this method is the most defensible and parsimonious. We will synthesise the results of time-series studies separately from other experimental and quasi-experimental designs, as time series designs standardize for variability over time rather than variability over units, resulting in a different scaling (D. Wilson, personal communication, September 20, 2013).

We will use reported statistics such as t, F, p or z-values to convert to effect sizes if effect size data is not reported. If data required to compute effect sizes is missing, we will attempt to contact the authors of the studies to obtain the required data.

We will input all effect size data into Comprehensive Meta-Analysis software (BioStat, 2005) to allow the calculation of standardized effect sizes and their standard errors, and the conversion between effect size types, to ensure that a common metric is used. Should an outcome be measured across different studies using binary data in some studies and continuous data in others, we will convert all effect sizes and their variances for this outcome to a common metric. For example, correlation coefficients and log odds ratios will be converted first to Cohen's d and then to Hedges' g, and the meta-analysis will be conducted on all outcomes using Hedges' g as the effect size of choice. Following Borenstein (2009), we argue that this approach whilst imperfect is preferable to conducting two separate meta-analyses. If this approach is required, we will conduct a sensitivity analysis to compare the results with those obtained by conducting separate meta-analyses.

Treatment of qualitative research

This review will not use qualitative research.