Potential Unintended Consequences of the Movement Toward Forensic Laboratory Independence

Abstract

The scholarly literature identifies two types of errors of justice: errors of due process and errors of impunity. Errors of due process involve failing to protect the innocent from becoming ensnared in the criminal justice process or imposing excessive sanctions on offenders. Errors of impunity involve failing to sanction, or imposing insufficient sanctions, on culpable offenders. The great challenge in designing criminal justice systems is balancing these two types of errors. We contend that the National Research Council’s recent recommendation to remove crime laboratories from law enforcement agencies in the United States focuses too heavily on avoiding one type of error while largely ignoring the other. We believe that heeding this recommendation without appropriate caution might produce an imbalance that generates serious unintended consequences. We draw on recent studies of how crime labs and law enforcement agencies process sexual assault kits and ballistic evidence to illustrate the potential unintended consequences of separating crime labs from law enforcement agencies.

Keywords

criminal investigation crime laboratories forensic science errors of justice

Introduction

In 2009, the National Research Council (NRC) released its Congressionally mandated report on the status of the forensic sciences in the United States. The report, authored by a committee comprised largely of eminent scientists, attorneys, and statisticians, issued 13 sweeping recommendations for strengthening the practice of forensic science in the United States. The Committee’s fourth recommendation called for the removal of public crime laboratories from law enforcement agencies:

To improve the scientific bases of forensic science examinations and to maximize independence from or autonomy within the law enforcement community, Congress should authorize and appropriate incentive funds to the National Institute of Forensic Science (NIFS) for allocation to state and local jurisdictions for the purpose of removing all public forensic laboratories and facilities from the administrative control of law enforcement agencies or prosecutors’ offices. (NRC, 2009, p. 24)

While we understand the motivation and reasoning behind this recommendation, our concern is that it may focus too heavily on reducing one type of error of justice at the expense of another type. In this article, we draw on social scientific research on two forensic science practices—processing sexual assault kits and processing ballistic evidence—in urging state and local governments to use caution before acting upon the Committee’s fourth recommendation.

A Framework for Thinking About Errors of Justice

Justice Blackstone (1769, chap. 27) once famously wrote: “It is better that ten guilty persons escape than that one innocent suffer.” The specific ratio in Blackstone’s formulation has inspired debate for more than two centuries, but its underlying principle serves as a core value throughout the western legal tradition. According to Forst (2010/2011, p. 104), inherent in Blackstone’s ratio are two fundamentally different types of errors in justice: errors of due process and errors of impunity. Errors of due process involve failing to protect the innocent from becoming ensnared in the criminal justice process or imposing excessive sanctions on offenders. Errors of impunity involve failing to sanction, or imposing insufficient sanctions, on culpable offenders. The great philosophical challenge in designing criminal justice systems and processes is calibrating the precise and consequential balance between these two types of errors.

Much of the research on errors of justice has been dominated by a concern with wrongful convictions, particularly for cases in which capital punishment may be imposed. The notion that the state might erroneously take someone’s life for a crime they did not commit is so offensive to fundamental conceptions of fairness that people become understandably indignant about the possibility. As Acker (2012–2013, p. 1631) notes, “wrongful convictions entail profound social costs.” The discovery of numerous innocent people on death row led several states to abolish or impose moratoria on executions, thanks largely to the work of state and national innocence commissions (Gould, 2008). The Innocence Project’s well-publicized work in exonerating the wrongly convicted helped to popularize the notion that wrongful conviction was more common than previously thought. Though wrongful convictions deserve close scrutiny, in Forst’s framework, they constitute only one part of the conceptual domain of errors of justice. Forst’s (2004) framework is useful for thinking more broadly about what constitutes an error of justice.

Forst derives his framework from the notion of Type I and Type II errors in statistical inference. In statistics, a Type I error occurs when we erroneously reject a true null hypothesis.¹ A Type I error is often referred to as a false positive because we infer a statistical effect when there really is not one. A Type II error occurs when we erroneously fail to reject a false null hypothesis. A Type II error is often referred to as a false negative, because we infer the absence of a statistical effect when there really is one. The probability of committing a Type I error is denoted as α, which is often referred to as the statistical significance level of a test. The probability of committing a Type II error is denoted as β. The quantity (1 − β) equals the statistical power of a test. A key observation is that there is an inverse relationship between Type I and Type II errors, or put differently, between α and β. As α increases, β decreases (thus increasing statistical power). Put differently, as the probability of committing a Type I error increases, the probability of committing a Type II error decreases and vice versa. As a result of this compensatory relationship, statistical hypothesis testing often requires analysts to make important decisions about how to balance statistical significance levels and statistical power.²

Forst (2004) argues that the logic of statistical hypothesis testing, which is designed to provide a systematic framework for controlling inferential errors, also provides a useful framework for thinking about errors of justice. Any time we erroneously sanction an innocent person, we are committing an error of due process, which is analogous to a Type I error. Any time we erroneously fail to sanction a guilty person, we are committing an error of impunity, which is analogous to a Type II error. In many (but not all) circumstances, when we create structures or policies to reduce the probability of errors of due process, we may be increasing the probability of errors of impunity (and vice versa). Table 1 summarizes the linkages between Type I and II errors and Forst’s two categories of errors of justice.

Table 1.

A Framework for Understanding Errors of Justice.

	Person is not guilty	Person is guilty
Person is sanctioned	Error of due process (false positive; Type I error)	Correct result (true positive)
Person is not sanctioned	Correct result (true negative)	Error of impunity (false negative; Type II error)

We contend that the NRC’s recommendations for the reform of forensic evidence processing in the United States may focus too heavily on avoiding errors of due process while largely ignoring the potential for errors of impunity. Forst (2010/2011, p. 104) notes that these two types of errors are often (but not always): “hydraulically linked, so that reducing one produces an increase in the other.” Both types of errors generate costly social harms. Failure to protect the innocent from arrest, prosecution, and punishment generates obvious injustices and reduces public support for, and the perceived legitimacy of, the criminal justice system. Failure to sanction culpable offenders, particularly violent offenders, generates obvious threats to public safety and increases fear of crime. Managing errors of justice responsibly means developing systems, policies, and processes that attend carefully to the balance between both types of errors (see Clark, 2012). Because the NRC’s fourth recommendation focuses almost entirely on errors of due process, largely ignoring errors of impunity, heeding this recommendation without appropriate caution may produce an imbalance that generates potentially serious unintended consequences. We draw on two recent examples to illustrate this point.³

Example 1: Processing Sexual Assault Kits

Among the most noteworthy controversies faced by police departments and crime laboratories over the past 5 years has been the discovery of hundreds of thousands of untested sexual assault kits in police custody (Pratt, Gaffney, Lovrich, & Johnson, 2006; Strom et al., 2009). The existence of these kits has been framed as an error of impunity that has denied justice to victims and has allowed perpetrators to escape accountability for their misdeeds. A sexual assault kit is used to collect evidence from victims following a sexual assault. Physical evidence collected with the kit includes intimate swabs, blood, urine, clothing, pubic hair combings, and other forms of evidence depending on the nature of the case. Written reports also accompany the kit, documenting victim injuries and victim activities before, during, and after the assault. A sexual assault kit is collected as part of a forensic medical exam performed by medical personnel, including doctors, nurses, and sometimes specially trained sexual assault nurse examiners (SANEs). Sexual assault kits are submitted to law enforcement agencies, which then may or may not request crime labs to conduct forensic testing on evidence in the kit. Sexual assault kits have potential value for criminal investigations because forensic testing results can aid in validating victim reports and identifying perpetrators, including serial offenders. Testing evidence in sexual assault kits also sends powerful, positive messages to victims and their families that the case is being taken seriously.

The discovery of large numbers of unsubmitted and untested sexual assault kits in the United States raised a sense of alarm among multiple stakeholder communities, including feminist groups, human rights advocates, and victims’ rights organizations. Data collected from 120 state and local crime laboratories and more than 1,600 police departments between 2002 and 2003 showed there were nearly 170,000 sexual assault kits that had never been submitted to a crime laboratory for analysis (Pratt et al., 2006). Similarly, a survey of approximately 2,300 police agencies discovered that in 18% of unsolved rape cases, forensic evidence existed but was never submitted for analysis (Strom et al., 2009). Following the release of these survey data, media reports began to document these issues in cities across the country (Keteyian, 2009; Strickler, 2010). In spite of this widespread attention, some jurisdictions continue to report the existence of unsubmitted sexual assault kits. Some states, including Ohio and Texas, have adopted policies to test evidence in older sexual assault kits.

A key aspect of this story is that many of these kits were never submitted to a crime lab for analysis, thus they were not counted as part of a backlog (Ritter, 2013). For evidence to be considered “backlogged” according to the National Institute of Justice (NIJ), it has to be submitted to a crime lab and remain untested for more than 30 days (Durose, 2008). Unsubmitted sexual assault kits have remained hidden and thus it is not clear to what extent property storage facilities, police agencies, and crime labs were aware of their numbers. Peterson, Johnson, Herz, Graziano, and Oehler (2012, p. i) characterized the situation in Los Angeles by noting that they “were not crime laboratory backlogs per se but were untested kits held in police property rooms in cold storage, where investigators and prosecutors had not requested the SAK [sexual assault kit] be tested.”

The discovery of these kits has been characterized by some as a failure of the criminal justice system and as an egregious error of impunity. Media accounts have implied that untested sexual assault kits amount to denied justice (see Hargitay, Tofte, & Burrhus-Clay, 2011; Keteyian, 2009; New York Times, 2013).⁴ Both crime labs and police departments have been accused of contributing to this problem. Police department property storage facilities often lack the capacity to track evidence carefully and systematically, and crime labs face daunting workloads that sometimes inhibit their ability to examine sexual assault kits in a timely manner (Strickler, 2010; Strom et al., 2009). The accumulation of large numbers of sexual assault kits is likely driven by several interrelated factors (Campbell, Fehler-Cabral, Shaw, Horsford, & Feeney, 2014), including the fact that case attrition is high for sexual assaults (see Spohn & Tellis, 2012). In some instances, investigators and prosecutors may not believe there is value in devoting limited resources to testing evidence when a case is unlikely to proceed through the system. In addition, crime labs may have encountered crises (see, e.g., Bromwich, 2006) or faced resource constraints that reduced the chances of evidence being submitted or analyzed.

In 2011, NIJ responded to this crisis by funding research on the processing of sexual assault kits in Houston (Texas) and Wayne County (Michigan) and to identify and evaluate potentially viable responses. The lessons learned from these studies suggest that closing the gap between crime labs and other criminal justice agencies is one of the appropriate solutions. The most popular and well-known responses to the problem of unsubmitted sexual assault kits are those that promote enhanced collaboration between crime labs and other partner agencies and organizations. Even in circumstances when labs may have contributed little to the problem of unsubmitted sexual assault kits, they will still be called upon to become part of the solution. One important aspect of the issue is that few entities, including labs and property storage facilities, were aware that large numbers of sexual assault kits had been collected, stored, and never submitted for analysis, highlighting the importance of good communication and accountability networks between labs and police agencies. A key lesson from the research to date is that the accumulation of unsubmitted kits did not result from the actions or inactions of a single agency and that effective responses would therefore require a coordinated effort. A project in Houston illustrates this point (Houstonsakresearch.org, 2013).

A diverse group of stakeholders in Houston is working together on the problem of unsubmitted sexual assault kits. The group includes representatives from the crime lab, the police, and the prosecutor’s office, as well as victim advocates, SANEs, and social scientists (Houstonsakresearch.org, 2013). In the process of preparing to move evidence into a new property storage facility that opened in 2009, the Houston Police Department discovered that some sexual assault kits in storage had never been tested. Personnel from the crime lab and the evidence storage facility conducted a comprehensive audit of evidence in storage and then prepared for the forensic testing that would follow. As thousands of untested sexual assault kits were being identified and prepared for testing by outsourced labs, the investigative units and the prosecutor’s office were engaged, so they could be prepared for the follow-up work that would be required.

The Houston experience confirmed that testing large numbers of previously untested pieces of evidence generates significant workloads for personnel downstream in the criminal justice process (see also Ritter, 2013). This idea is consistent with a more general body of research which suggests that policies, practices, and decisions in one part of the criminal justice system often have implications for other parts of the system (Bernard, Paoline, & Pare, 2005; Walker, 1992). The Houston project was supported by external grant funding that provided modest amounts of overtime pay for investigative personnel to address the added workload (see also Office of Violence against Women [OVW], 2010). Changes in crime lab practices, including decisions to test evidence in large numbers of “cold cases,” can be expected to have significant implications for downstream actors (see Strom et al., 2009, 4-3). If these downstream actors are not prepared for the increased workload, then systemic breakdowns may result, reducing the chance that victims receive the justice they deserve and that offenders are held accountable for their actions. These kinds of systemic or interagency issues impose costs on both the accused and the victim and undermine the legitimacy of the criminal justice system.

Closing the information gap between crime labs, police evidence storage facilities, investigative units, and prosecutors through improved information management systems is a recurring recommendation (Nelson, n.d.; OVW, 2010; Strom et al., 2009). Testing more than 10,000 previously untested sexual assault kits in New York City entailed “constant communication with other [than the Medical Examiner’s Office] parties involved with collecting and transporting evidence and investigating and prosecuting sexual assault” (OVW, 2010, p. 19). In New Orleans, multiple agencies collaborated not only to test evidence in sexual assault kits but also to ensure that information was shared effectively. The project relied on new software known as CHOP (CODIS Hit Project) to ensure that CODIS hits resulting from testing sexual assault kits were communicated to the law enforcement agencies and individuals who could use this information in investigations (Nelson, n.d.). In addition, CHOP can be useful for more strategic problem analysis in the criminal justice system: “CHOP tracks CODIS hits so that all stakeholders can locate information and deal with bottlenecks and delays.” The implication is that crime labs, property storage facilities, police investigative divisions, and prosecutors can work collaboratively to identify and overcome problems with the management of evidence and test results.

An anecdote from Houston illustrates the potential value to be gained from collaborative problem solving. As part of the Houston project, we learned from crime lab personnel and prosecutors that a growing workload for the crime lab stems from prosecutors’ requests for additional evidence testing before cases go to trial. Lab personnel reported that prosecutors want all pieces of available evidence in a case to be tested even when the additional testing is not likely to generate new or meaningful information. Prosecutors confirmed that they request additional testing before trial because they expect jurors will want to know that all evidence was examined. Prosecutors recognize that this practice burdens the crime lab and that additional testing will not likely be probative. The demands placed on the crime lab will continue to grow if prosecutors continue to submit these types of testing requests and, with trial dates looming, these requests may interfere with protocols for prioritizing evidence testing. Lab workloads will also grow if comprehensive sexual assault kit testing mandates, like those adopted in Illinois and Texas, proliferate. A collaborative working relationship between justice system agencies, in conjunction with an effective information management system, may allow crime labs to determine the source of increasing work demands, provide evidence of how those demands impact productivity, and then respond effectively in a coordinated manner. Before embracing the recommendation to establish independent and autonomous crime laboratories, state and local governments must be careful to develop structures that support these types of productive interagency partnerships.

Finally, errors of impunity may be minimized through cross-training rather than independent, discipline-specific training (OVW, 2010). Strom et al. (2009), in their national study of forensic evidence processing, found that many police agencies lack a sound understanding of the potential utility of forensic evidence for criminal investigations. They recommended training for police personnel “on the benefits and uses of forensic analysis” (Strom et al., 2009, 4-4). Cross-training for police investigators, crime scene technicians, prosecutors, and crime lab personnel may improve work quality and reduce errors of impunity. Cross-training is a common practice in the network of agencies and personnel who respond collaboratively to sexual assaults. Sexual Assault Response Teams (SARTs), for example, represent a multiagency, coordinated response to sexual assault (Office of Victims of Crime [OVC], 2011a). Crime lab personnel are recognized as important stakeholders who participate in these teams (OVC, 2011b). One innovative SART practice in West Virginia allows crime labs to provide rapid, electronic feedback to SANEs about the quality of the kit they collected. This feedback can be used for training purposes to improve evidence collection (OVC, 2011c).

The lessons learned about responding effectively to sexual assaults demonstrate the importance of communication and collaboration between justice system agencies for reducing the social costs associated with both errors of due process and errors of impunity. Strom and colleagues (2009, 4–3) conclude that “systematic solutions are necessary to create a more efficient system that promotes coordination and information sharing across law enforcement agencies, forensic laboratories, and prosecutors.” Establishing structures and processes that undermine coordination, cooperation, collaboration, or communication can generate unintended consequences. In the case of sexual assault, these unintended consequences may inhibit justice for victims and their families and allow perpetrators to escape detection, avoid accountability for their actions, and continue offending.⁵

Example 2: Processing Ballistic Evidence

Research on ballistic imaging practices reinforces the points highlighted by the research on processing sexual assault kits. One study, for instance, assessed the operation and effects of the National Integrated Ballistics Imaging Network (NIBIN) in the United States (King, Wells, Katz, Maguire, & Frank, 2013). NIBIN is a program coordinated by the Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) that provides technological hardware and a data sharing network to 150 local crime labs and police agencies (NRC, 2008; U.S. Department of Justice, 2005). The operation of NIBIN at the local level involves different organizations (law enforcement agencies, crime labs, and prosecutors’ offices) that engage in exchanges of physical evidence and information (King & Maguire, 2009). The organizational, geographic, and social gaps between each of these entities present significant opportunities for evidence or information to get misrouted, miscommunicated, delayed, or lost. We begin by briefly describing the process of using NIBIN at the local level. We then discuss two aspects of the process that could become problematic by further decoupling crime labs from law enforcement.

The NIBIN system allows local labs and police agencies to enter images of ballistic evidence (fired bullets and shell casings) into a database (Thompson, 2010). The NIBIN system searches for probable matches and returns a list of likely matches to the submitting lab or police agency. A firearm examiner or technician must then visually compare the images in the NIBIN system and then examine the actual pieces of evidence with a comparison microscope to confirm matches between evidence from different crime scenes or other sources. If the visual examination indicates a match, that image is recorded as a “hit” in the NIBIN system,⁶ and a hit report is produced by the lab and routed to the appropriate law enforcement agency.

We focus on two aspects of the process of using NIBIN as an analytical and investigative tool: (a) the flow of physical evidence and information about that evidence, and (b) opportunities for end users of crime lab information to provide routine feedback about the quality of the lab’s work product. Both aspects may be complicated by further decoupling labs from police agencies. Note that NIBIN is also useful for prosecutors, judges, and defense attorneys, but here we focus primarily on its utility for criminal investigations.

First, separating crime labs from law enforcement agencies might impede the quick and secure delivery of evidence between them. The flow of physical evidence from crime scenes to labs, and the subsequent flow of information about that evidence from labs to criminal investigators and prosecutors, relies on a network of loosely coupled organizations embedded in different environments. Physical evidence must be properly retrieved and documented at crime scenes and securely transported to labs for storage or analysis. The evidence must also be transported with accompanying documentation concerning the exact question (for investigators) that should be answered regarding the evidence. For example, should a gun be checked for DNA, or fingerprints, or for trigger pull measurements, or should hairs and fibers be retrieved from the barrel of the weapon, or should the serial number be checked for alteration or obliteration, or should the firearm be test fired? Other information might also be necessary. For example, investigators might need a specific analysis to speed up, or for additional analyses to be conducted to aid in their investigations. The flow of information between criminal investigators and crime lab analysts is a vital element of the criminal justice process that must be carefully protected. Constraining this information flow may be useful for reducing errors of due process (see Giannelli, 2010 for a summary), but it may also increase errors of impunity.

Any step in this process from crime scene to lab and back to law enforcement is prone to delays and breakdowns. We highlight just one possible bottleneck, the step where the firearms section of a crime laboratory requests that ballistics evidence be returned from a law enforcement agency for visual confirmation of a hit. Presently, more than 60% of NIBIN sites (crime labs) return ballistic evidence to the law enforcement agency or investigator for storage after an image of the evidence is input into the NIBIN system (King et al., 2013: table 8). Yet, when a probable hit is later discovered, the firearms section at the lab must request that the evidence be returned to the lab for visual analysis, a step that is required before a lab can confirm a hit and issue a hit report for the law enforcement agency. This process is likely quick and efficient in labs that are part of a law enforcement agency because it is handled internally within a single organization (unless the evidence came from a different law enforcement agency). But if labs become autonomous from law enforcement agencies, confirming a hit will require retrieving at least two pieces of evidence, in some instances from two different law enforcement agencies. As information and material traverses the boundaries between organizations, the likelihood of delays and other coordination issues increases.

In order for a system for requesting evidence from a police agency to work smoothly, there must be an effective accountability mechanism in place. In labs located within a police agency, that mechanism is embedded in the organizational structure of the agency itself and thus applies to both the lab and other parts of the police agency. For example, the head of the firearms section or the lab can quickly notify a supervisor who oversees property storage if there is a delay in returning evidence to the lab for visual confirmation. Accountability mechanisms may be more loosely coupled when labs are autonomous from law enforcement. Separate agencies—which have separate leaders, structures, policies, and practices—may find it more difficult to establish effective interagency accountability mechanisms. The gaps between agencies may increase the likelihood of communication breakdowns, processing delays, and other issues. As one crime lab administrator noted: “There are lots of manuals within agencies about how work is to be done, but we rarely find those types of agreements at an interagency level” (Research Triangle Institute [RTI], 2013).

Second, increasing the autonomy of crime labs may decrease the extent to which the end users of the lab’s information (primarily criminal investigators and prosecutors) are able to provide routine feedback about the lab’s performance, including the utility of the information they receive from the lab. For instance, we interviewed homicide investigators in a city that is thought to have one of the most progressive independent crime labs in the United States. During the interview, we asked the investigators about their use of ballistic evidence. They told us that unless there are exceptional circumstances, they often don’t bother asking for tests to be run on ballistic evidence because the crime lab is so nonresponsive to their requests. Another lab placed severe constraints on how much ballistic evidence a police agency could submit for analysis. The lab would then produce lengthy, intensive reports of the type that are very useful in the courtroom but take too long to produce to be of utility for criminal investigators. Moreover, the constraints on how much evidence could be submitted for processing meant that police could only submit evidence from a tiny fraction of the gun crimes they were investigating (King et al., 2013; Maguire, King, Matusiak, & Campbell, 2015).

In short, it may be easier for independent crime labs to forget who their end users are and how well the crime lab’s services meet the needs of these end users. One crime lab administrator highlighted this issue in noting: “one of the problems we have is that … we have producers at times who are not speaking to consumers of the product” (RTI, 2013). As lab analysis has the potential to be used most often by criminal investigators, we focus primarily on the feedback process from investigators to labs. When labs are embedded in police agencies, there is a greater likelihood that investigators will find opportunities to inform the lab about the utility of the reports or analyses the lab produces. Such routine feedback may be formal or informal and consist of a text message, e-mail, phone conversation, or a short chat in the hallway, for example. While removing labs physically and organizationally from police agencies may produce some benefits, it will also likely limit the frequency and quality of these feedback opportunities. Routinely soliciting and incorporating feedback from end users is a vital part of achieving quality control in service organizations (Jones & Sasser, 1995; Osborne & Gaebler, 1992).

Feedback from investigators could prove beneficial in making lab analyses more useful, especially when investigators find lab analyses to be irrelevant or too dated (Smith, 1976, pp. 14; King & Maguire, 2009). For example, a recent study of NIBIN (King et al., 2013) revealed that in 65 criminal cases where a NIBIN hit was identified, the hit report was not helpful to investigators. NIBIN hit reports had helped identify a suspect in only 9.7% of cases, led to an arrest in only 1.6% of cases, helped in charging a suspect or obtaining a plea in only 4.9% of cases, and helped with sentencing the suspect in only 1.7% of cases. Only rarely were hit reports useful for helping to identify a previously unknown suspect (King et al., 2013). It may be tempting to blame these problems on the shortcomings of ballistic evidence, but that would be a mistake. Instead, these problems are a simple function of the structures and processes that police agencies and crime laboratories had put in place to process ballistic evidence.

Investigators typically do not use hit reports during the course of their investigations because the reports arrive too late to be useful. For example, in our sample of 19 NIBIN sites, the median number of elapsed days between the most recent offense in a hit dyad (two firearms offenses linked by a common piece of ballistic evidence) and the confirmation of the hit was 101 days (King et al., 2013: p. 65, table 18), with 5 sites posting median times of greater than 250 days. These system inefficiencies are a breeding ground for errors of impunity (and in those instances where the evidence is exculpatory, for errors of due process as well). These issues were allowed to persist, in part, because most labs lack a mechanism to receive constructive feedback from investigators and other consumers of lab analyses. Removing labs from law enforcement agencies will not improve this situation unless alternative systems for soliciting and processing end-user feedback are put in place.

Other research also finds that lab analyses are often not completed during the course of an investigation (Eck, 1983; Horvath & Meesig, 1996; Peterson, Mihajlovic, & Gilliland, 1984; Peterson, Sommers, Baskin, & Johnson, 2010). It seems likely that increased communication between labs and law enforcement, or even a systematic feedback system where police are routinely surveyed about their opinions about lab analyses, would help labs improve their performance and tailor their outputs to be most helpful for end users. In the case of ballistic evidence processing, labs might adjust their processes if they learned that investigators found hit reports to be of limited utility. Isolating labs from law enforcement, however, will likely impede the flow of evidence and information between labs and end users and will limit opportunities for routine feedback from end users. In the end, without the benefit of increased communication, labs will continue to process ballistics evidence as if their efforts mattered, and investigators will continue to find lab reports dated and not useful for their investigations.

Discussion and Conclusion

The types of issues highlighted here are not unique to crime labs and police departments. At a more general level, these same types of issues are common in organizations and networks of many types, including those within the criminal justice system. Thus, it may be possible to derive insights about these issues from organization theory and criminal justice theory (e.g., Maguire & Duffee, 2015; Nohria & Eccles, 1992). A branch of organizational theory focuses on networks of organizations and the capacity of organizations within these networks to coordinate, cooperate, collaborate, and communicate effectively. The capacity of organizations to work together toward common goals is not a new issue—it is a fundamental challenge faced by organizations of many types. Theorists often attribute the success or failure of interorganizational relationships to the efforts of boundary spanners—people whose role involves serving as a bridge between their organization and external entities on which the organization relies to achieve the flow of some resource, whether informational, material, or financial (Blau, 1964; Cole, 1970; Maguire & King, 2011). Any credible arrangements for coordinating the work of separate organizations must pay careful attention to the role of boundary spanners.

The idea of networks of organizations is especially relevant in criminal justice, which has been depicted for many years as a system of sorts. Introductory criminal justice textbooks routinely depict the criminal justice system in a flowchart, with cases progressing through the network of agencies. For some critics, this network is a dysfunctional, inefficient, or otherwise imperfect system, but it is a system nonetheless because it involves routine exchanges between interdependent agencies (Snipes & Maguire, 2015). Many of these exchanges involve either people (as individuals are transferred from agency to another) or information about people. Thus, agencies within the system are often referred to by organizational theorists as case-processing or people-processing organizations to distinguish them from other types of organizations that may exchange materials or products (Bernard et al., 2005; Maguire, 2003). At its most basic level, criminal justice systems function when police arrest offenders, courts adjudicate offenders, and correctional agencies punish, incapacitate, or rehabilitate offenders. Crime laboratories play an important and unique role in that network of agencies by processing evidence about cases neutrally, with no less emphasis on exculpatory evidence than on incriminating evidence. Thus, crime labs contribute key information to the investigation and adjudication process. Viewing criminal justice from a network or systems perspective provides a useful framework for thinking about the likely effects of criminal justice reforms (Bernard et al., 2005; Maguire & Duffee, 2015; Walker, 1992). Viewing proposed reforms from a systems perspective is especially useful for anticipating unintended consequences.

Our principal concern in this article is that the NRC’s recommendation to remove crime labs from police departments pays insufficient attention to the idea that crime laboratories are part of a larger system. For instance, there is little evidence in the Council’s report that the committee considered the likely effects of its recommendation on law enforcement agencies. The committee’s membership didn’t include any law enforcement executives or researchers specializing in the study of crime control. Upon the release of the NRC’s report, the International Association of Chiefs of Police (IACP) quickly raised a red flag about the recommendation to decouple crime labs from police departments:

The IACP is very concerned that the recent Forensics Report from the National Academy of Sciences was developed without input from law enforcement practitioners … The IACP is strongly opposed to the removal of crime laboratories and other forensic services from law enforcement agencies … .The IACP strongly believes that all research and other initiatives that are designed to study/enhance the delivery of forensic sciences must include the participation of law enforcement practitioners. (Polski, 2009, p. 25)

The National District Attorneys Association also objected to the recommendation, noting that other less expensive and disruptive solutions could achieve the same effect.⁷ These criticisms were both predictable and preventable. A committee that included a wider range of stakeholders may have been more inclined to consider the broader systemic effects of its recommendations and to adopt a more inclusive focus on errors of justice rather than a narrower focus on preventing wrongful convictions. Wrongful convictions are a vital problem that deserves serious attention from our nation’s brightest minds. But reformers need to be careful that the solutions they develop in their quest to prevent this important problem don’t unintentionally set in motion other vexing problems. What is necessary is a careful balance between errors of due process and errors of impunity.

Achieving this balance is not a new problem, and it is not a problem that is unique to police agencies and crime laboratories. These issues are inherent in the design of the criminal law and the criminal justice system more generally. In a classic essay, Herbert Packer (1968, p. 153) once highlighted the tension between two abstract “value systems that compete for priority in the operation of the criminal process.” Packer called these two value systems as the Crime Control Model and the Due Process Model. The Crime Control Model places great value on the repression of crime. Under the Crime Control Model, “the failure to apprehend and convict” leads to a breakdown in public order and represents a failure of the criminal process. The Due Process Model places great value on procedural protections to ensure that the criminal process doesn’t unintentionally ensnare the innocent. Under the Due Process Model, the failure to protect innocent people from being mistakenly arrested or convicted represents a failure of the criminal process. Over time and place, the fundamental tension between these two value systems is evident in all criminal justice systems. When crime rises, people may be willing to trade some procedural protections for improved security. When procedural injustices make the headlines, people clamor for greater due process protections. Packer is careful to point out that the two models do not represent the converse of one another. Put differently, the choice between due process and crime control is not necessarily a zero-sum game.

There is substantial overlap between Packer’s two value systems and Forst’s two errors of justice. There is an obvious analogue between Forst’s conception of errors of due process and Packer’s Due Process Model. There is a slightly less obvious analogue between Forst’s conception of errors of impunity and Packer’s Crime Control Model. Of most importance for this essay is the linkage between value systems and errors in justice. The basic idea is that every community must make choices about its due process and crime control values and develop a set of structures, policies, and practices that seek to minimize errors of justice in a way that is consistent with these value choices. While due process and crime control are not polar opposites, it is not difficult to imagine crime control practices that harm due process or due process protections that harm crime control. The move toward laboratory independence is being pitched as a method for preventing wrongful convictions, a clear endorsement of due process ideals. Done carelessly, such a move might inadvertently obstruct crime control ideals and increase the risk of errors of impunity.

In addition to these fundamental values issues, the idea of how to structure forensic evidence processing also raises a host of seemingly banal but ultimately vital logistical issues. The challenge is how to ensure that the relevant information and material held by any one agency in the system is readily available to other agencies that need it. The transfer of information and material is not easy, even within an organization. For example, even simple transactions in which evidence needs to be transferred from one unit or organization to another (e.g., from the police property room to the crime lab) often present difficult logistical challenges. This problem is not confined to forensic evidence. For instance, police agencies have long faced the challenge that information transfer across units and divisions (and even from shift to shift) can be very difficult, both logistically and in a manner that preserves high court standards of legal integrity. Likewise, transferring information and intelligence about specific matters such as gangs, sex offenders, and terrorism has been challenging due to the well-known issue of “loose coupling” within law enforcement agencies (Crank & Langworthy, 1992; Katz & Webb, 2006).

If the routine transfer of information and materials within organizations is difficult, transfers between organizations are even more challenging. Numerous constraints exist, including weak accountability structures, formal and informal organizational boundaries, cultural and occupational divides, different incentive systems, political issues, and the varying demands imposed from the outside by powerful stakeholders. For instance, federally sponsored programs that bring multiple criminal justice agencies together to share information and work together to address specific crime problems are often found not to be sustainable for more than a brief period. Information transfer, whether within or between agencies, is an erstwhile challenge.

As jurisdictions begin to remove crime labs from police departments, they will wrestle with significant questions about the many meanings of “independence” and the implications of these different definitions for all involved organizations.⁸ Crime labs can be independent or autonomous in many ways, including budgets and resources, supervision and accountability mechanisms, and physical space. Decisions about how “independent” labs should be organized in any given local context will have a key influence on the potential for errors of justice and impunity. It is crucial for labs to establish structures and policies that minimize the potential for bias or errors and that “protect the integrity of the laboratory’s findings” (Giannelli, 2010, p. 261). At the same time, whatever structures and policies are put in place must also be carefully designed to facilitate appropriate collaboration between police, prosecutors, defense counsel, and the crime lab. The challenge is to promote fair, accurate, timely, and efficient uses of forensic evidence that minimize errors of impunity and due process.

Judge Harry Edwards, Co-Chair of the NRC’s Committee on Identifying the Needs of the Forensic Science Community, noted that “reliable forensic evidence increases the ability of law enforcement officials to identify those who commit crimes, and it protects innocent people from being convicted of crimes they didn’t commit.”⁹ We agree. However, the systems and processes that must be put in place to generate these reliable analyses are not self-evident. The question of how to organize a criminal justice system to achieve a proper balance between due process and crime control, and to minimize errors of due process and errors of impunity, is both a normative and an empirical one. It requires a thoughtful elaboration of values and a careful analysis of systems dynamics and outcomes. Absent such analysis, we urge caution before acting on the Committee’s recommendation to remove crime laboratories from law enforcement agencies. We speculate, based on a large body of theory and a more limited body of research, that acting on the recommendation prematurely may generate unintended and potentially harmful consequences.

Footnotes

Acknowledgments

The authors are grateful to Professor Brian Forst of American University and Executive Assistant Chief Tim Oettmeier of the Houston Police Department for providing comments on an earlier draft of the paper.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Edward Maguire is a professor in the School of Public Affairs at American University. His research focuses primarily on policing and violent crime. He has led a number of externally funded research projects to completion, including a national study of community policing in the United States, a 6-year study of a violent crime outbreak in Trinidad and Tobago, and a 4-year study of human trafficking in the Philippines. He is currently leading studies on protest policing in the United States, predictive policing in Uruguay, transnational crime in El Salvador and the United States, and policing, gangs, and violence in the Caribbean.

William R. King is an associate professor of criminal justice, and associate dean of Research and Program Development in the College of Criminal Justice at Sam Houston State University, in Texas. He earned his PhD in Criminal Justice from the University of Cincinnati in 1998. His research focuses on the structure of police organizations and forensic crime labs and the role of forensic analysis in criminal investigations. Between 2005 and 2009, he helped implement reforms to the forensic crime lab, national police service, and homicide bureau in Trinidad and Tobago.

William Wells is a professor in the Department of Criminal Justice and Criminology at Sam Houston State University. His research interests include police practices to prevent and reduce crime and police uses of forensic evidence. Dr. Wells is working on a National Institute of Justice-funded research project with the Houston Police Department that seeks to reduce the problem of unanalyzed sex assault kits. In 2014, he collaborated with the Houston Police Department on an experiment that tested different methods of administering photo spreads and lineups to witnesses.

Charles M. Katz is Watts Family Director of the Center for Violence Prevention and Community Safety and is a professor in the School of Criminology and Criminal Justice at Arizona State University. He received his PhD from the University of Nebraska at Omaha in 1997. His research interests include gangs and gang control, police organizational theory, and crime control policy.

References

Acker

(2013) The flipside injustice of wrongful convictions: When the guilty go free. Albany Law Review 76(3): 1629–1712. Original work published 2012.

Bernard

T. J.

Paoline

E. A.

Pare

P. P.

(2005) General systems theory and criminal justice. Journal of Criminal Justice 33: 203–211.

Blackstone

(1769) Commentaries on the laws of England, book the fourth, Oxford, England: Clarendon Press.

Blau

P. M.

(1964) Exchange and power in social life, New York, NY: John Wiley.

Bromwich

M. R.

(2006) Fourth report of the independent investigator for the Houston Police Department Crime Laboratory and Property Room, Washington, DC: Fried, Frank, Harris, Shriver, & Jacobson, LLPRetrieved from http://www.hpdlabinvestigation.org.

Campbell, R., Fehler-Cabral, G., Shaw, J., Horsford, S., & Feeney, H. (2014). The Detroit Sexual Assault Kit Action research project: Preliminary findings. Paper presented at Powerful Partnerships: 20 Years of the Violence against Women Act and the Path Ahead at the University of Delaware, April 24.

Clark

S. E.

(2012) Costs and benefits of eyewitness identification reform: Psychological science and public policy. Perspectives on Psychological Science 7(3): 238–259.

Cole

G. F.

(1970) The decision to prosecute. Law and Society Review 4: 313–343.

Crank

Langworthy

R. H.

(1992) An institutional perspective on policing. Journal of Criminal Law and Criminology 83: 338–363.

10.

Durose

(2008) Census of publicly funded forensic crime laboratories, 2005, Washington, DC: Bureau of Justice Statistics.

11.

Eck

J. E.

(1983) Solving crimes: The investigation of burglary and robbery, Washington, DC: Police Executive Research Forum.

12.

Forst

(2004) Errors of justice: Nature, sources, and remedies, New York, NY: Cambridge University Press.

13.

Forst

(2011) Managing miscarriages of justice from victimization to reintegration. Albany Law Review 74(3): 1209–1275. Original work published 2010.

14.

Giannelli

P. C.

(2010) Independent crime laboratories: The problem of motivational and cognitive bias. Utah Law Review 10(2): 247–266.

15.

Gould

J. B.

(2008) The Innocence Commission: Preventing wrongful convictions and restoring the criminal justice system, New York: New York University Press.

16.

Gould, J. B., Carrano, J., Leo, R., & Young, J. (2012). Predicting erroneous convictions: A social science approach to miscarriages of justice. Final report submitted to the National Institute of Justice.

17.

Hargitay

Tofte

Burrhus-Clay

(2011, May 15) It’s time to test rape kits. Houston Chronicle. Retrieved from http://www.chron.com/opinion/outlook/article/It-s-time-to-test-rape-kits-1692862.php.

18.

Horvath

Meesig

(1996) The criminal investigation process and the role of forensic evidence: A review of empirical findings. Journal of Forensic Science 41: 963–969.

19.

Houstonsakresearch.org. (2013). Houston sexual assault kit research: A multidisciplinary project sponsored by the national institute of justice [ONLINE]. Retrieved from http://www.houstonsakresearch.org.

20.

Jones

Sasser

W. E.

Jr. (1995) Why satisfied customers defect. Harvard Business Review 73. (November–December), 88–99.

21.

Katz

C. M.

Webb

V. J.

(2006) Policing gangs in America, New York, NY: Cambridge University Press.

22.

Keteyian, A. (2009, November 9). Rape in America: Justice denied. Retrieved from http://www.cbsnews.com/stories/2009/11/09/cbsnews_investigates/main5590118.shtml.

23.

King

W. R.

Maguire

E. R.

(2009) Assessing the performance of systems designed to process criminal forensic evidence. Forensic Science Policy and Management: An International Journal 1(3): 159–170.

24.

King, W. R., Wells, W. H., Katz, C. M., Maguire, E. R., & Frank, J. (2013). Opening the black box of NIBIN: A process and outcome evaluation of the use of NIBIN and its effects on criminal investigations. Final report submitted to the National Institute of Justice (October 2).

25.

Maguire

E. R.

(2003) Organizational structure in American police agencies, Albany, NY: SUNY Press.

26.

Maguire

E. R.

Duffee

D. E.

(2015) Criminal justice theory: Explaining the nature and behavior of criminal justice, 2nd ed. New York, NY: Routledge.

27.

Maguire

E. R.

King

W. R.

(2011) Federal-local coordination in homeland security. In: Forst

Greene

J. R.

Lynch

J. P.

(eds) Criminologists on terrorism and homeland security, New York, NY: Cambridge University Press, pp. 322–356.

28.

Maguire, E. R., King, W. R., Matusiak, M. C., & Campbell, B. (2015). Testing the effects of people, processes, and technology on ballistic evidence processing productivity (working paper). Washington, DC: American University.

29.

National Research Council (2008) Ballistic imaging, Washington, DC: National Academies Press.

30.

National Research Council (2009) Strengthening forensic science in the United States: A path forward, Washington, DC: National Research Council, Committee on Identifying the Needs of the Forensic Sciences Community.

31.

Nelson

M. S.

(n.d.) Analysis of untested sexual assault kits in New Orleans, Washington, DC: U.S. Department of Justice, National Institute of Justice.

32.

New York Times. (2013, January 20). Evidence of rape ignored. Retrieved from http://www.nytimes.com/2013/01/21/opinion/untested-rape-kits-delay-justice.html.

33.

Nohria

Eccles

R. G.

(1992) Networks and organizations: Structure, form, and action, Boston, MA: Harvard Business School Press.

34.

Office of Violence against Women (2010) Eliminating the rape kit backlog: A roundtable to explore a victim-centered approach, Washington, DC: U.S. Department of Justice, Office of Violence against Women.

35.

Office of Victims of Crime. (2011a). SART toolkit: resources for sexual assault response teams. Office of justice programs. Retrieved from http://ovc.ncjrs.gov/sartkit/index.html.

36.

Office of Victims of Crime. (2011b). SART toolkit: resources for sexual assault response teams. Office of justice programs. Retrieved from http://ovc.ncjrs.gov/sartkit/develop/team-forensic.html.

37.

Office of Victims of Crime. (2011c). SART toolkit: resources for sexual assault response teams. Office of justice programs. Retrieved from http://ovc.ncjrs.gov/sartkit/practices/crime-sexual-wv.html.

38.

Osborne, D., & Gaebler, T. (1992). Reinventing government: How the entrepreneurial spirit is transforming the public sector. Reading, MA: Addison-Wesley.

39.

Packer

H. L.

(1968) The limits of the criminal sanction, Stanford, CA: Stanford University Press.

40.

Peterson

Johnson

Herz

Graziano

Oehler

(2012) Sexual assault kit backlog study: Final report, Washington, DC: National Institute of Justice.

41.

Peterson

Sommers

Baskin

Johnson

(2010) The role and impact of forensic evidence in the criminal justice process, Washington, DC: National Institute of Justice.

42.

Peterson

J. L.

Mihajlovic

Gilliland

(1984) Forensic evidence and the police: The effects of scientific evidence on criminal investigations, Washington, DC: U.S. Government Printing Office.

43.

Polski

(2009) Forensic science: A critical concern for law enforcement agencies. The Police Chief LXXVI(9): 24–25.

44.

Pratt

Gaffney

Lovrich

Johnson

(2006) This isn’t CSI: Estimating the national backlog of forensic DNA cases and the barriers associated with case processing. Criminal Justice Policy Review 17: 32–47.

45.

Research Triangle Institute. (2013). Comments made by a crime lab administrator at the “Investigating Sexual Violence—Emerging Solutions and Best Practices” conference. Washington, DC, September 19.

46.

Ritter

(2013) Untested evidence in sexual assault cases: Using research to guide policy and practices. Sexual Assault Report 16: 33–43.

47.

Smith

W. C.

(1976) Overview of crime laboratory management. In: Fox

R. H.

Wynbrandt

F. H.

(eds) Crime laboratory management forum 1976, Rockville, MD: The Forensic Sciences Foundation Press, pp. 5–21.

48.

Snipes

J. B.

Maguire

E. R.

(2015) Foundations of criminal justice theory. In: Maguire

E. R.

Duffee

D. E.

(eds) Criminal justice theory: Explaining the nature and behavior of criminal justice, 2nd ed. New York, NY: Routledge, pp. 27–54.

49.

Spohn

Tellis

(2012) The criminal justice system’s response to sexual violence. Violence Against Women 18: 169–192.

50.

Strom

Ropero-Miller

Jones

Sikes

Pope

Horstmann

(2009) The 2007 survey of law enforcement forensic evidence processing, Washington, DC: National Institute of Justice.

51.

Strickler, L. (2010, March 4). Thousands of rape kits wait to be tested. CBS News. Retrieved from http://www.cbsnews.com/2100-500690_162-5605770.html.

52.

Thompson

(2010) Firearm identification in the forensic science laboratory, Alexandria, VA: The National District Attorney’s Association.

53.

U.S. Department of Justice, Office of the Inspector General, Audit Division (2005) The Bureau of Alcohol, Tobacco, Firearms and Explosives’ National Integrated Ballistic Information Network Program, Washington, DC: U.S. Department of Justice.

54.

Walker

(1992) Origins of the contemporary criminal justice paradigm: The American Bar Foundation survey, 1953-1969. Justice Quarterly 9(1): 47–76.