PROTOCOL: Use of DNA Testing in Police Investigative Work for Increasing Suspect Identification,Arrest,Conviction,and Case Clearance

Types of interventions

The scope of the review will be limited to the use of DNA testing by police as part of their investigations of crime. We will not consider the use of DNA testing by criminal defendants or by prosecutors. Of particular interest is the routine or expanded application of DNA testing in cases that often do not make use of available DNA evidence. The primary basis for eligible studies requires some variation in the use of DNA in the investigation.

Types of studies

Given the limited amount of research in this area, we will include a broad range of study designs. All designs, however, must estimate the effect of DNA testing relative to an alternative or more limited type or application of analysis (i.e., varying degrees of DNA testing, but not necessarily effects sizes).

The ideal design type would randomly assign cases to either a DNA testing condition or a traditional investigative practices condition, and then assess the outcomes of both conditions from the same time-frame. We will consider any such designs that vary the degree of DNA testing used and examine one or more of the outcomes discussed above. The comparison condition does not need to represent the absence of DNA analysis, but simply a variation of DNA from the treatment condition.

We will also consider quasi-experimental designs in which there is a control group that is either matched to the DNA testing group, or identified as comparable. To include these less methodologically rigorous designs will require statistical justification of the suitability of the control groups identified.

Interrupted time-series designs will also be included in this review, or other regression-based analyses that estimate the impact of DNA testing on a relevant outcome, but these designs will be handled separately as they especially vulnerable to historical threats to validity. An essential feature of time-series designs is the multiple baseline estimates of the rate of interest (e.g., identification of a suspect). This allows for an assessment of both the natural change over time and change that may be associated with the start of an intervention, such as the use of DNA testing or some other change related to the use of DNA in police investigative work.

Basic pre-post designs with a single pre-DNA and a single post-DNA estimate will be included, but these designs will also be handled separately as they provide a weak basis for drawing causal inferences. Other quasi-experimental designs will be included, such as a design that contrasts the clearance rates for different police agencies (without statistically justifying the validity of the comparison), but these will be reported separately.

Types of outcome measures

This review will extend to all crime types. We recognize, however, that the utility of DNA testing is likely to vary substantially across crime type. The current trend toward increased use of DNA testing in burglary cases reflects that burglary is often a high volume crime, with offenders engaged in serial burglaries (Roman et al., 2008). The serial nature of this crime increases the likelihood that a suspect may be identified in an existing database and helps police connect crimes committed by the same individual. The added value of DNA testing may be less for other crimes. As such, we will examine the evidence separately by type of offense.

DNA testing may improve outcomes at several stages of the investigative process. It may facilitate the identification of suspects through the use of DNA databases, such as CODIS in the United States or the UK National Criminal Intelligence Database in the United Kingdom. DNA testing also may help eliminate suspects or identify one suspect among multiple suspects found through traditional police investigative methods. These processes may increase the likelihood of an arrest and a conviction, raising the number of cleared cases. Studies under review may examine the effectiveness of DNA testing as measured by one or more of these outcomes. As such, we will examine as part of our review the following aspects of effectiveness as outcomes: the rate at which suspects are identified, the arrest rate of a suspect, the conviction rate, length and speed of an investigation, the cost of the investigation, and the case clearance rate (i.e., how often cases are successfully solved).

The source of the data will generally be from official records or reports of some form. However, we will not restrict eligibility based on the source of the outcome data. All sources will be considered.

Search methods for identification of studies

We will consult with an information retrieval specialist on the selection of keywords to use and databases to search. Preliminary searches will be conducted with the terms “DNA” and “Police” or “Policing” or “Investigation”. The terms will be refined and expanded as we gain greater familiarity with the existing literature and how it is cataloged. In addition to searching the databases listed below (see electronic sources), we will also search the UK Home Office website for relevant publications. Google Scholar will be searched to help identify publications not already captured through the formal databases. Key individuals working in this field will be solicited for assistance in identifying relevant studies (e.g., Peter Neyroud at the National Policing Improvement Agency). We will also be searching for studies in languages other than English. If we identify any such studies, we will solicit assistance in translating these documents.

Search Strategy for the Identification of Relevant Studies

A search strategy for electronic databases was developed in order to reach an optimal balance between potentially relevant search results and the large number of results using similar terms, yet addressing issues other than those of interest in this project. This search strategy is broad, but avoids the combination of terms that would include the large body of technical research done on DNA, as well as studies exclusively addressing non-DNA forensic sciences.

Search Terms

Two categories of keywords were developed for this search. The first category addresses the technology of interest (DNA). The second category addresses the application of DNA testing in police investigative work, and includes terms such as policing, detective, arrest, etc. The intention of separating the terms in this manner was to include all the potentially relevant results, while simultaneously excluding the large bodies of literature on DNA from non-forensic disciplines. These two sets of keywords were combined with a Boolean AND. Unfortunately, the body of literature on the application of DNA to criminal investigations is much smaller than the literature on aspects of DNA addressed by other disciplines (i.e., the basic science of DNA testing). This has resulted in search results in the low hundreds for some databases.

1. Particular Technology of Interest

2. Application context

Electronic Sources

The search strategy described above was applied to the following databases, which cover both the more accessible sources as well as the grey literature.

Association of Chief Police Officers ACPO

Australian Research Council Centre of Excellence in Policing and Security (CEPS)

Assessment of risk of bias in included studies

The methodological quality of the studies will be assessed by coding the features contained in the attached coding forms. The types of information that will be considered include: (1) nature of assignment to conditions; (2) use of matching of cases or the use of statistical controls, such as regression analysis to adjust for potential selection bias in the case of non-random assignment to conditions; (3) representativeness of the sample of cases (e.g., census, random sample, convenience sample); (4) attrition of cases from the study: and, (5) replication of findings in multiple jurisdictions.

The methodological quality information will be reported in tabular form and presented along with the effect size information. If a meta-analysis is performed, effect sizes will be compared on the above dimensions to assess for potential bias.

Measures of treatment effect

The odds-ratio will be the effect size of choice for all outcomes of a dichotomous or binary nature. For example, an odds-ratio will be computed to represent the effect of DNA testing relative to an alternative on the proportion of cases cleared. In contrast, the standardized mean difference effect size will be used for outcomes measured on a continuous measure or outcomes that represent counts or rates. Standard methods of computing effect sizes will be used (see Lipsey and Wilson, 2001). In the case of quasi-experimental designs, preference will be given for effect sizes adjusted for baseline differences. For example, a coefficient for a treatment dummy variable from a logistic regression model can be converted into a treatment effect odds-ratio that is adjusted for the other variables in the model. Similarly, covariate adjusted means can be used in the computation of a standardized mean difference of effect sizes (Lipsey and Wilson, 2001). Preference will also be given to quasi-experimental designs that use a control/comparison condition that is assessed at the same time as the treatment condition.

Unit of analysis issues

The primary unit-of-analysis within the studies of interest will usually be a criminal case, such as a sexual offense or a burglary. Within studies, these cases will typically be aggregated across a police station or some other jurisdiction. Effects from studies that report results separately for distinct and independent policing units, such as units in different cities, will be coded separately and treated as statistically independent.

Multiple reports or manuscripts based on the same study or data will be treated as a single entity for purposes of this review. We will either select the most complete references if all of the relevant information in secondary reports (e.g., journal manuscript) is contained within the primary report (e.g., technical report). However, if the multiple reports each provide unique information (e.g., different outcomes or different jurisdictions), then all reports will be coded as part of this review.

Assessment of reporting biases

Reporting bias will be addressed in several ways. First, the search for and inclusion of unpublished works, such as technical reports, will help guard against reporting and publication bias. If possible, we will compare the results from published and unpublished studies as one estimate of potential bias. Second, we will list any outcomes (dependent measures) included in a study for which an effect size could not be computed. These outcomes are more likely to have negative or null results. A large number of such outcomes, particularly for primary outcome measures, raise concerns regarding reporting bias. We will only perform more formal statistical assessments of publication bias if sufficient data permit, such as the Duval and Tweedie trim-and-fill method and funnel plots (Duval & Tweedie, 2000).

Data synthesis

Once we have completed our search, we will make the determination of whether meta-analysis is the most appropriate method of analyzing the data. Our decision rule for performing meta-analysis will be as follows: two or more studies, each with a computable effect size of a common outcomes construct (potentially measured in different ways), and similar comparison condition. Instances that satisfy this decision rule will be meta-analyzed using standard methods (i.e., inverse-variance weighted, random effects model; see Lipsey and Wilson, 2001). We will also use meta-analysis to synthesize results across multiple sites within a single study if results are reported separately. If data permit, we will explore potential moderators of the effectiveness of DNA, such as crime type, nature of the comparison or counterfactual condition (e.g,. finger-print or other non-DNA methods).

In the absence of meta-analysis, synthesis of the findings across studies and inferences about the effectiveness of DNA use in policing will be based on the size and direction of the effects and the confidence intervals. Greater emphasis will be placed on the direction of effects and consistency of effects across similar studies. A single high quality evaluation with an effect size of a meaningful magnitude that is also statistically significant will be interpreted as evidence that DNA use can be effective, assuming there is not an equally strong study with a negative result (multiple studies of the same relationship would, however, be meta-analyzed). Statements about the generalizability of the findings will be limited to the range of conditions examined. Studies with weak internal validity (i.e., plausible bias in the estimate of the causal relationship) will be viewed cautiously. Positive findings of a meaningful magnitude will be interpreted as promising, but will also be viewed as providing limited evidence of the potential effectiveness of forensic DNA use in police investigations.