Abstract
Pretrial risk assessments are used in local pretrial settings to inform release and supervision decisions. Yet, there is little research on the implementation of pretrial risk assessments in practice. We examined the characteristics of pretrial decision-making matrices in a statewide sample of counties using the same risk assessment tool. We procured pretrial decision-making matrices from 29 Indiana counties that had received or were in the process of receiving pretrial certification (88.6% response rate). Using a structured coding protocol, we found matrices shared common features but differed in their structure and available release and supervision options. Matrices weighted index charge information more heavily than risk assessment information, suggesting risk assessments likely explain between 20% and 36% of variability in decisions given typical rates of judicial adherence. Local decision-makers should be aware that structuring matrices to weight charge information more heavily than risk assessment information may limit the risk management potential of these tools.
Keywords
The United States has high rates of jail incarceration (World Prison Brief, 2018), driven in large part by the detention of pretrial defendants (Zeng & Minton, 2021). Individuals who are arrested, detained, and released into the community pose a risk of returning to the criminal-legal system (Digard & Swavola, 2019; Lowenkamp et al., 2013). The use of cash bail has been identified as a primary driver of high rates of pretrial detention (Fellner, 2010) and other adverse effects on case processing (Dobbie et al., 2018; Gupta et al., 2016; Heaton et al., 2017). Inconsistent decision-making at the pretrial stage disproportionately harms individuals from underrepresented racial and ethnic groups (Arnold et al., 2018; Demuth, 2003; Demuth & Steffensmeier, 2004; Sacks et al., 2015). Together, pretrial release decisions have cumulative effects on subsequent case processing, resulting in long-term impacts on individuals’ legal system involvement (Campbell et al., 2020; Leslie & Pope, 2017; Oleson et al., 2016; Sacks et al., 2015) and community ties (Heaton et al., 2017).
In response to these trends—and with support from national initiatives—states have enacted several pretrial reform efforts to increase the likelihood that defendants will receive pretrial release (Casey & Elek, 2015; National Conference of State Legislatures [NCSL], 2020). Pretrial risk assessment tools are a primary pretrial reform strategy implemented to facilitate structured decision-making at the pretrial phase. In the last 5 years, half of the most populous jurisdictions in the United States have implemented pretrial assessment tools (Lattimore et al., 2020), with some smaller jurisdictions implementing tools as well (Pretrial Justice Institute, 2019). Pretrial risk assessments are tools that assess defendants for risk of committing a future crime or failing to appear in court while on pretrial release ((Desmarais & Lowder, 2019; Stevenson & Mayson, 2017). Specifically, pretrial risk assessment estimates represent the likelihood that individuals with characteristics similar to the assessed defendant experience the outcome of interest. These tools often include measures of age, marital status, education, employment, financial resources, and criminal history among other factors to determine an individual’s level of risk (Desmarais et al., 2020). Some tools such as the Federal Pretrial Risk Assessment (PTRA; Lowenkamp & Whetzel, 2009) and the Public Safety Assessment (Laura and John Arnold Foundation, 2014) additionally include items related to the index offense. In comparison with other risk assessment tools implemented in correctional settings, pretrial risk assessments are often geared toward facilitating risk-based decision-making rather than needs-based intervention. In research, pretrial risk assessments have shown evidence of predictive validity for pretrial misconduct outcomes (DeMichele et al., 2020; Lowder et al., 2020), including in meta-analytic reviews (Desmarais et al., 2020).
Research on Pretrial Risk Assessments
There is growing, but still limited, research examining the effects of pretrial risk assessment implementation on pretrial release and misconduct outcomes. A handful of studies have provided consistent evidence that use of pretrial risk assessments can contribute to higher rates of nonfinancial release (e.g., release on own recognizance) and lower rates of financial bond decisions (Cooprider, 2009; Copp et al., 2022; Lowder et al., 2021; Sloan et al., 2018; Stevenson, 2018); however, the magnitude of these effects and their stability over time varies considerably across studies. There is more limited evidence for pretrial misconduct outcomes. Some studies have shown null effects of pretrial risk assessments on rates of pretrial misconduct (Stevenson, 2018), while other studies have found that pretrial risk assessment implementation is associated with small increases in nonviolent rearrest rates (Lowder et al., 2021; Sloan et al., 2018). More broadly, other studies have questioned whether risk assessments at large can reduce recidivism when implemented in practice (Viljoen et al., 2018), noting that jurisdictions often face barriers to successful implementation of risk assessment tools (Viljoen et al., 2018) and the amount of evidence on the use of risk assessments in practice is lacking (Viljoen et al., 2021).
Together, these trends raise important questions regarding whether and how pretrial risk assessments can be implemented in practice to achieve risk management objectives. Scholars who study risk assessments have noted several barriers to understanding the effectiveness of risk assessments on pretrial misconduct outcomes. First, pretrial risk assessments vary considerably, both in factors measured (Desmarais et al., 2020) and in degree of transparency in their development (König & Krafft, 2021). Thus, we may not expect pretrial risk assessments to produce similar outcomes in practice depending on the strength of tool development and validation procedures. Second, from an implementation perspective, pretrial decision-makers vary in how they perceive pretrial risk assessment tools (DeMichele et al., 2019; Terranova et al., 2020) and are often skeptical of the accuracy of measured risk factors (Terranova et al., 2020). Both of these considerations limit the stability and degree of adherence to pretrial risk assessment recommendations by pretrial decision-makers. For example, some evidence suggests that although pretrial decision-makers may consider pretrial risk assessment information, this adherence may fade over time (Stevenson, 2018). Third, and finally, risk assessment scholars have argued that risk assessments, particularly in the pretrial context, are designed only to inform decisions (Desmarais & Lowder, 2019). Indeed, local jurisdictions consider risk assessment information together with other legal considerations, primarily the index charge severity.
Pretrial Decision-Making Matrices
Within these constraints, local jurisdictions have attempted to standardize consideration of risk assessment information through the development and adoption of structured guidelines. In pretrial settings, these tools are referred to as pretrial decision-making matrices. Most typically, these matrices include pretrial risk assessment information on one axis and charge information on the other axis; an example of such a matrix is provided in Figure 1. Jurisdictions have discretion in how they organize charge information across columns, both in the disaggregation of charge categories into smaller units (e.g., distinguishing between violent- and nonviolent misdemeanors vs. all misdemeanors) and in the types of offenses included in each category. Different coordinates across the two axes direct decision-makers to specific release and supervision recommendations, which becomes a product of a defendant’s risk and index charge level. The use of pretrial matrices is a component of best practice guidelines for the implementation of risk assessments in pretrial settings (e.g., see Advancing Pretrial Policy & Research [APPR], 2022a; Bureau of Justice Assistance [BJA], n.d.). However, to date, there has been fairly limited empirical evaluation of these tools.
Sample Pretrial Decision-Making Matrix
Adherence to Matrix Guidelines
In the pretrial context, most evidence on decision-making matrices focuses on the extent to which decision-makers adhere to such guidelines. For example, in a statewide investigation in Virginia, Danner and colleagues (2015) found nearly 80% of release decisions adhered to structured guidelines; however, these rates were lower when release and supervision decisions were considered together (59.1%). In another study, Lowder and colleagues (2021) found 73.8% of release decisions were adherent to structured guidelines in the year following pretrial risk assessment implementation and across three counties. Another multijurisdictional study found slightly lower rates of adherence to structured guidelines for supervision decisions (60%; Lowder & Foudray, 2021). A final study in a single jurisdiction found much lower rates of adherence to structured guidelines in a 2-year postimplementation period (25%–47%; Copp et al., 2022). Overall, existing evidence suggests rates of adherence to structured guidelines vary considerable across localities and decision-making contexts.
Several studies have also examined judicial adherence as a moderator of the effects of pretrial risk assessments on pretrial outcomes, including release decisions and pretrial misconduct outcomes. In one study, researchers randomized jurisdictions to receive training on a structured decision-making framework, the “Praxis,” or to use pretrial risk assessments without the use of the Praxis (Danner et al., 2015). In this study, training on structured guidelines was associated with an over 2 times greater likelihood of release with nonfinancial conditions. In addition, defendants whose judges received training on structured guidelines were slightly less likely to fail on supervision (Danner et al., 2015). Similarly, Lowder and colleagues (2021) found when judges adhered to structured guidelines, the effects of pretrial risk assessments on release decisions were even stronger and included reduction in the time in detention and likelihood of receiving any detention. The findings also provided some evidence that the higher rate of rearrest post-risk assessment implementation was driven primarily by release decisions that diverged from structured guidelines (Lowder et al., 2021). A final study found that judges frequently departed from structured guidelines, overwhelmingly in favor of more restrictive release conditions. These departures were also associated with higher rates of failure to appear and rearrest during the pretrial period (Copp et al., 2022).
The Present Study
Together, findings from the existing evidence base raise important questions about the impact of pretrial risk assessments as a component of pretrial reform. There is currently considerable scrutiny on risk assessments in the pretrial context, including whether they may exacerbate existing disparities in criminal-legal processing (Minow et al., 2019; Pretrial Justice Institute, 2020). However, two assumptions made in pretrial risk assessment research and theorizing are that pretrial risk assessments exclusively guide pretrial decisions or that adherence to structured guidelines necessarily implies absolute adherence to pretrial risk assessment recommendations. In reality, adherence to structured guidelines varies widely across settings. In addition, even in cases of strict adherence to guidelines, the way that guidelines are operationalized in pretrial decision-making matrices functionally sets an upper bound for the possible impact pretrial risk assessments can have in practice relative to other criteria (e.g., legal characteristics like charge level and type).
To our knowledge, however, there has yet to be an empirical examination of pretrial decision-making matrices, including how they structure and weight different decision-making criteria. Understanding how these tools weight pretrial risk assessment information relative to legal factors may help calibrate local stakeholders’ expectations of the potential impact of pretrial risk assessments to their relative importance as a decision-making criterion. To address this limitation, we requested, procured, and coded pretrial decision-making matrices across 29 counties in Indiana. Our specific aims were to examine (a) variation in the structure and characteristics of pretrial decision-making matrices, particularly those informed by the same pretrial risk assessment tool and statutory guidelines; (b) the average weight placed on pretrial risk assessment versus charge information and variability within these estimates; and (c) the expected contribution of pretrial risk assessments to pretrial release and supervision decision-making based on estimated weights and expected rates of judicial adherence to matrix guidelines.
Method
Study Context
In 2014, the State of Indiana was designated an Evidence-Based Decision-Making (EBDM) site by the National Institute of Corrections. As part of this effort, Indiana selected pretrial reform as a primary area for the development of evidence-based decision-making practices. In developing a pretrial reform strategy, the State adopted the Ohio Risk Assessment System (Latessa et al., 2009), renamed the Indiana Risk Assessment System (Latessa et al., 2013), and implemented the Indiana Risk Assessment System (IRAS)–Pretrial Assessment Tool (IRAS-PAT) into pretrial decision-making. Eleven counties were selected as pilot sites to implement the IRAS-PAT into decision-making, develop local pretrial services capacity, and expand pretrial supervision operations. Initial evaluation of these efforts showed IRAS-PAT assessments produced good levels of predictive validity (Lowder et al., 2020) and were associated with an increase in nonfinancial release rates (Lowder et al., 2021). The latter evaluation additionally suggested that adherence to structured guidelines (i.e., pretrial decision-making matrices) amplified the effects of pretrial risk assessments on release decisions. However, pretrial risk assessments—with or without adherence to structured guidelines—had little effect on community misconduct outcomes (Lowder et al., 2021), suggesting potential issues with risk principle adherence. More recently, Indiana has expanded pretrial reform efforts to other localities and developed a pretrial certification process to assist jurisdictions in implementing evidence-based pretrial reform strategies (Judicial Conference of Indiana, 2019). The implementation and use of pretrial risk assessments are a central component of pretrial certification efforts. At the time of this study, 35 counties had received full certification, received provisional pretrial certification, or were in the planning stages of pretrial certification.
Sampling
Between July 30 and August 31, 2021, we emailed pretrial coordinators from 35 counties in Indiana that had received full or provisional pretrial certification or were going through the process of pretrial certification. For each county, we requested copies of current pretrial release and pretrial supervision matrices for research purposes. We received replies from 31 counties (88.6% response rate). Four counties did not respond (11.4%). Of the 31 counties responding, two counties (6.5%) stated they were in the process of remodeling or approving their matrix and could not provide a copy at this time. Across the other 29 counties, we received a total of 33 matrices. There were 17 matrices that included both pretrial release and supervision decisions (51.5%), 12 matrices that included only supervision decisions (36.4%), and four matrices that included only release decisions (12.1%). Most counties (n = 25, 86.2%) had one matrix while four counties (13.8%) had separate matrices for release and supervision decisions. One matrix relied on a separate risk assessment tool, the Proxy (Bogue et al., 2006), to determine release decisions. To ensure consistency in application of risk assessment information, this matrix was removed from further analysis, resulting in a final sample size of 32 matrices across 29 counties.
Data Cleaning and Coding
One of the main distinguishing features of matrices was how charge information was organized across columns. Oftentimes, violent offenses were separated from nonviolent offenses, but not always in a similar way across matrices. For example, some counties deemed violent misdemeanors as less serious than nonviolent felonies. Other counties classified violent misdemeanors as more serious than nonviolent felonies. Some counties distinguished between violent misdemeanor, low-level violent felony, and high-level violent felonies. Other counties placed more emphasis on distinguishing between charge classification across columns or grouped specific offenses into a column. We applied several criteria to ensure consistent coding across matrices despite wide variation in charge organization. To be included in coding, matrices had to (a) include charge information on one axis and pretrial risk assessment information on the other axis, (b) have charge categories and pretrial risk assessment categories ordered in terms of increasing severity, and (c) contain offenses that were eligible for pretrial risk assessment-guided decision-making. Prior to coding, three researchers reviewed all matrices to ensure similar structure. For matrices that do not conform to these rules, we reordered columns or transposed the matrix to ensure consistency in coding. The Indiana constitution requires all offenses with the exception of murder or treason to be bailable offenses (Indiana Constitution, Article 1, §17); as a result, we excluded columns that included only murder/treason offenses across matrices because these offenses would not be eligible for pretrial risk assessment–guided decisions. These columns were not factored into the size of a matrix.
Each matrix was assigned a unique identifier, which reflected the county and matrix version (i.e., release and/or supervision matrix). We developed a structured coding questionnaire to quantify various components of each matrix: the overall structure, the number and type of release and supervision decisions, the characteristics of charge columns, and changes in release and supervision decisions across matrix rows and columns. Two researchers (E.M.L. and A.K.) coded 126 variables for each matrix, achieving a final interrater agreement of 88.4% across all matrices coded, which varied between 71.4% and 99.2% for individual matrices. All disagreements were resolved via consensus to produce a final coded data set.
Variables
Matrix Characteristics
We first coded several characteristics of the matrix itself. This included matrix type (release, supervision, release and supervision), which denoted the kind of supervision matrix. We also counted the number of rows (count), corresponding to the pretrial risk assessment information, and the number of columns (count), corresponding to the charge information. We measured the overall matrix size (count), denoted as the number of unique cells. Unique release or supervision options (count) measured the total number of unique options for release or supervision decisions. To distinguish between types of release or supervision options, we considered any additional condition to represent a unique release or supervision option. For example, if judicial review of a recommended decision was added, this constituted a unique option. We also measured whether bond (yes; no) or judicial review (yes; no) of a matrix-guided release or supervision decision was incorporated into any matrix decision.
Column Characteristics
Because pretrial risk assessment information was organized in rows in a similar way across matrices (i.e., all matrices used the IRAS-PAT to inform decision-making and most organized rows as Low, Moderate, and High risk), we focused on understanding characteristics of columns across matrices. Column characteristics reflected charge information and varied considerably across jurisdictions. First, we measured the level of charges in each column (misdemeanor; felony; both). Next, we measured charge types (violent; nonviolent; both) in each column. Finally, we measured the highest charge severity (1–9) and lowest charge severity (1–9) in each column. Charge severity was operationalized ordinally such that lower values corresponded to more serious charges, consistent with Indiana Criminal Code. Felony 1 charges are the most serious charges in Indiana, which were coded as 1. Felonies 2 to 6 were coded as 2 to 6. Misdemeanor Level “A” charges are the most serious misdemeanor charge and were coded as 7. Less serious misdemeanor charges, Misdemeanor “B” and “C,” were coded as 8 and 9, respectively.
Relative Risk and Charge Weighting
We developed two ratio measures of relative risk and charge weighting. The first variable was developed based on severity of change in the release and/or supervision decision. For this variable, we first identified the total number of unique release and/or supervision options for each matrix. For example, possible release options could be release on own recognizance (ROR), ROR and Supervision Level 1, ROR and Supervision Level 2, ROR and Supervision Level 3, or Detain pending judicial review (five options). Second, we ranked these options in terms of severity (i.e., least severe to most severe), which was inferred primarily by ordering in the matrix. Third, each matrix was coded to indicate the number of rows, the number of columns, the total number of release and/or supervision options, and the degree of change in release and/or supervision decision severity across each row and column in the matrix (e.g., for a given row, the difference in severity ranking from the first column to the last column). Fourth, we then calculated two separate variables to indicate the maximum possible change in decision severity across rows (1) and across all columns (2). For example, for maximum row movement, we multiplied the number of total rows by the number of release and/or supervision options minus 1. Fifth, we calculated two summed variables for both rows and columns that indicated the number of changes in severity across all rows or columns, respectively. Sixth, for rows and columns separately, we then divided the summed variable by the maximum possible change in decision severity, as defined above. In a final step, the resulting quotients of these calculations were divided to produce a final ratio, with the quotient for changes within columns as the numerator (indicating decision movement prioritizing risk) and the quotient for changes within the rows as the denominator (indicating decision movement prioritizing charge). These calculations are illustrated in Supplemental Figures 1S to 3S (available in the online version of this article).
The second variable was developed based on the number of changes in the release decision. We replicated the steps above for this calculation, but we measured change as any increase in release and/or supervision decision severity from one row or column to another. Consistent with this operationalization, we restricted the number of possible changes to the number of columns or rows minus 1. These calculations are represented visually in Supplemental Figures 4S to 6S (available in the online version of this article). Both final variables were ratio measures where 1 indicated equal risk and charge weighting. Ratios above 1 indicated more prioritization of risk information; ratios below 1 indicated more prioritization of charge information. Stata syntax used to produce these calculations is available in an accompanying Supplemental Appendix (available in the online version of this article).
Analytic Strategy
We conducted descriptive statistics to measure the frequency, central tendency, and dispersion of all study variables. Where relevant, we present results separately for matrices informing release versus supervision decisions. To facilitate interpretability of column-level information, we produced modified box plots to indicate the average maximum charge severity and average minimum charge severity for a given column. Box plot whiskers represent the full range of maximum or minimum charge severity across all columns. For ratio variables measuring risk and charge weighting, we computed values across all matrices, but averaged values within counties with more than one matrix to ensure no undue influence of any specific county to averaged ratios. All analyses were conducted in Stata 16.
Results
Matrix Characteristics
Most matrices had three rows (87.5%, n = 28), corresponding to Low (reflecting an IRAS-PAT score of 0–2), Moderate (IRAS-PAT score 3–5), and High-risk (IRAS-PAT score 6–9) classifications. A small proportion of matrices (12.5%, n = 4) divided risk scores into High (IRAS-PAT score 6–7) and Very High (IRAS-PAT score 8–9) classifications. Matrices had an average of 4.12 columns (SD = 1.31, range = 3–7). Most matrices had three columns (46.9%, n = 15) with fewer having four (18.7%, n = 6), five (15.6%, n = 5), six (12.5%, n = 4), or seven (6.2%, n = 2) columns. Accordingly, the average size of a matrix was approximately 12.91 (SD = 4.26, range = 9–21), corresponding to a 3 × 4 matrix size. Possible matrix sizes were 3 × 3 (46.9%, n = 15), 3 × 4 (9.4%, n = 3), 3 × 5 (12.5%, n = 4), 4 × 4 (9.4%, n = 3), 3 × 6 (12.5%, n = 4), 4 × 5 (3.1%, n = 1), or 3 × 7 (6.2%, n = 2). On average, counties allowed for an average of 4.81 (SD = 1.47, range = 3–9) unique release or supervision decisions across matrices. For matrices that were used to inform release decisions (n = 20), 30.0% (n = 6) incorporated bond as a release option and 65.0% (n = 13) explicitly incorporated judicial review into matrix decisions.
Column Characteristics
Because column characteristics were more variable than row characteristics (i.e., most matrices had three rows corresponding to Low, Moderate, and High risk levels), we provide descriptive information on the organization of charge information across matrices informing release and supervision decisions, respectively.
Release Decisions
Column characteristics for matrices informing any type of release decision (n = 20) are presented in Figures 2 and 3. We present these graphs using frequencies rather than percentages to accurately depict the distribution of matrices with each number of columns (e.g., there was only one matrix with seven columns). As shown in Figures 2 and 3, most jurisdictions organized nonviolent misdemeanor offenses into the first matrix column. Subsequent columns were more likely to reflect mixed misdemeanor and felony offenses or felony offenses alone. Column 4 and subsequent columns primarily captured felony-level offenses. Violent offenses were primarily included in Columns 3 to 7 (i.e., in most cases, the final column of the matrix).
Level of Charge by Pretrial Release Matrix Column
Violent Charge Inclusion by Pretrial Release Matrix Column
Table 1 and Figure 4 show the average range of offense levels within each column and variability around these estimates. As shown in Figure 4, the box represents the difference between the average maximum charge (upper bound of box) and the average minimum charge (lower bound of box) within each column. Whiskers represent the full range of possible charges across matrix columns. As shown, for Column 1 and Column 7, the full range of charges was equivalent to the average maximum and minimum charge. Column 3 showed the widest variability in charge severity and was most likely to include all levels of felony-level offenses. Notably, the range in allowable charge severity within each column generally decreased as charges became more serious.
Average Minimum and Maximum Charge Severity by Column for Pretrial Release Matrices
Note. Higher charge severity values reflect less serious charges. There were no matrices with more than seven columns.
Charge Severity by Pretrial Release Matrix Column
Supervision Decisions
Column characteristics for matrices informing supervision decisions (n = 28) are presented in Supplemental Figures 7S and 8S (available in the online version of this article). One matrix did not have sufficient charge information available to calculate charge severity for each column and was removed from this analysis. Overall trends were similar to those of pretrial release matrices. However, supervision matrices were more likely to separate misdemeanor offenses by severity in separate columns, relative to release matrices. As shown in Supplemental Table 1S and Supplemental Figure 9S (available in the online version of this article) (see Supplemental Appendix), there was slightly more variability in the average minimum and maximum charge across columns relative to pretrial release matrices.
Relative Risk and Charge Weighting
Across matrices and averages for counties with more than one matrix, the average ratio of risk weighting to charge weighting for the actual change in severity was 0.66 (SD = 0.45, range = 0.11–2.04). Half of matrices produced ratios at or below 0.53. The average ratio of risk to charge weighting when measured by the number of changes was 0.92 (SD = 0.38, range = 0.39–2.00); half of matrices produced ratios at or below 0.83. Together, these results (i.e., ratios below 1) suggest that counties weighted charge information more heavily than pretrial risk assessment information. The higher ratio for number of changes in decisions may reflect that agencies often accommodate pretrial risk assessment information by increasing the number of release or supervision options in a matrix that reflect incrementally restrictive conditions.
Discussion
We sought to advance knowledge on the use of pretrial risk assessments in practice by examining the characteristics of pretrial decision-making matrices in a statewide sample. We procured pretrial decision-making matrices from 29 counties across Indiana that had received or were in the process of receiving pretrial certification from the State. Broadly, our objectives were to examine the structure of these matrices and to translate this information into implications for pretrial risk assessment–informed decision-making. Overall, our findings suggested that pretrial decision-making matrices share common features but also differ in their structure and available release and supervision options, even in a single state using the same pretrial risk assessment tool. In addition, decision-making matrices were more likely to prioritize index charge information relative to pretrial risk assessment information. Translated into practice, our examination suggests pretrial risk assessments may have less influence on release and supervision decisions than previously thought. Below we discuss these findings in greater detail.
A primary finding from this study was that pretrial decision-making matrices are fairly diverse in their structure and content. We observed this pattern even among matrices drawn from a single state (Indiana) and incorporating the same pretrial risk assessment tool (IRAS-PAT). Notably, jurisdictions differed in whether they incorporated both release and supervision decisions into pretrial decision-making matrices. In addition, the degree to which all charges were eligible for pretrial risk assessment-informed decision-making varied across matrices. Finally, jurisdictions also varied in the extent to which they would differentiate between different charge types and levels across the matrix, indicated by the varying number of columns across matrices (from three to seven).
Our findings underscore the considerable discretion retained by local jurisdictions even when implementing the same tool into practice. Yet, there remains very limited research on local policy related to pretrial decision-making. Although previous studies have examined decision-makers’ perceptions of pretrial risk assessments and their implementation (DeMichele et al., 2019; Terranova et al., 2020) or discussed the potential ways that risk assessments can be implemented into criminal-legal decision-making (van Eijk, 2020), there have been few—if any—investigations of how the policy and practice of pretrial risk assessment implementation may differ across jurisdictions. These types of investigations can answer basic descriptive questions such as whether jurisdictions use structured guidelines to facilitate risk assessment–guided decision-making, where pretrial risk assessments are used in the pretrial decision-making process (e.g., pre- or post-release), and for what purpose pretrial risk assessments are used (e.g., to inform release decisions, supervision decisions, or assignment of other conditions). However, these investigations can also answer more inferential questions such as the association between how risk assessments are implemented into local practice and their effects on pretrial release decision-making and outcomes. The widespread use of risk assessments (Lattimore et al., 2020) and increasing data availability make such investigations increasingly feasible to conduct.
When we examined how matrices prioritized various decision-making criteria, we found that matrices were more likely to prioritize charge information relative to pretrial risk assessment information. However, the extent to which this was true depended on how we operationalized changes in decisions. For example, when we examined the full range of decisions and their severity of restrictiveness (i.e., for a matrix with seven possible release decisions, moving from the least restrictive decision to the most restrictive decision would be a magnitude change of 6), matrices prioritized pretrial risk assessment information to an even lesser degree than when we examined whether there was any change at all from a less restrictive to more restrictive decision. This finding suggests that jurisdictions incorporate pretrial risk assessment information into matrix-guided decisions through use of incrementally restrictive release or supervision options. For example, a common strategy we saw was to add “judicial review” of a recommended supervision or release decision as a way to differentiate a decision across risk levels. However, slightly varying conditions by risk is likely to dilute the impact of pretrial risk assessments as a risk management strategy, particularly if the matrix introduces more judicial discretion or if defendants with different risk levels ultimately receive the same decision.
There remains very limited research on implementation of risk assessments in practice (Lawson et al., 2021; Vincent & Viljoen, 2020). Yet, researchers have argued that implementation issues may temper the potential success of pretrial risk assessments in practice. Indeed, pretrial risk assessments show stronger effects on decision-making when judges adhere to their recommendations (Lowder et al., 2021) and lower predictive accuracy when judges deviate from recommendations (Cohen et al., 2020). Scholars have called for more rigorous research to understand the impact of risk assessments in practice, implementation outcomes (e.g., fidelity to established protocols) and their determinants, and the effectiveness of strategies to improve implementation success (Vincent & Viljoen, 2020). We would extend these recommendations to include more study of the local policy and implementation context surrounding use of pretrial risk assessments. Specifically, as illustrated by the present study, pretrial decision-making matrices provide one avenue through which researchers can operationalize the extent to which pretrial risk assessments—at least from a local policy perspective—are implemented into decision-making. Cross-jurisdictional studies could examine how differences in matrix weighting of risk and charge information moderate the effectiveness of pretrial risk assessments on key pretrial outcomes in practice. At a minimum, researchers can provide more consistent reporting of how pretrial risk assessments guide decisions at the local level (e.g., at what stage of decision-making, for which types of defendants and charges, and for which types of decisions). As research accumulates on the effectiveness of pretrial risk assessments in practice, these characteristics can serve as site-level moderators in future meta-analytic studies.
As a novel implication of this study, ratios can be extrapolated to estimate the maximum degree of influence that pretrial risk assessments would have on release decisions. We present an illustration of such estimates in Table 2. These estimates measure the degree of influence of risk assessments as a percentage of the total variability in decisions (assuming a maximum of 100%). We varied these estimates based on (a) the proportion of release or supervision decisions for which judicial decision-makers would adhere to structured guidelines and (b) matrix ratios. As shown, a ratio between 0.75 and 1 would result in pretrial risk assessments explaining approximately 42.9% to 50.0% of the variability in pretrial release and supervision decisions if judicial decision-makers were fully adherent to structured guidelines. At adherence rates more typically seen in practice (e.g., 50%–75%), the anticipated impact of pretrial risk assessments is much lower (21.4%–37.5%). For the average matrix ratios observed in this study, pretrial risk assessments would explain between 19.9% and 36.0% of the variability in release and supervision decisions. Notably, within the upper bound of matrix ratios observed in this study (i.e., 2), and assuming perfect adherence to structured guidelines, pretrial risk assessments would maximally explain 66.7% of the variability in pretrial decisions.
Projected Influence (Percentage) of Risk Assessments on Pretrial Release and Supervision Decisions
Note. Highlighted and boldfaced cells reflect average ratios observed in the present study.
Considering previously published adherence rates to structured pretrial guidelines, our findings suggest that only approximately one fourth to one third of pretrial release or supervision decisions would be risk-informed. Many pretrial decision-making matrices prioritized pretrial risk assessment information to an even lesser degree. Practically, these findings suggest that the effectiveness of pretrial risk assessments on defendant risk management is likely to be tapered. Indeed, prior research has questioned the risk reduction effect of pretrial risk assessments, showing that their implementation is associated with either negligible effects on pretrial arrest rates (Stevenson, 2018) or small increases (Lowder et al., 2021; Sloan et al., 2018). These findings are consistent with findings on effects of risk assessment implementation more broadly, which suggests risk assessments can reduce restrictive placements (i.e., promote release from detention) but show inconsistent or negligible effects on recidivism (Viljoen et al., 2019). Critically, in order for risk assessments to causally impact pretrial misconduct rates, such tools must inform differential intervention, supervision, or service receipt (Hart et al., 2017). Our findings raise broader questions regarding why pretrial risk assessments may not be achieving their intended goals and what can be done to strengthen the implementation of tools in practice to achieve these objectives.
Through pretrial reform efforts, many jurisdictions implemented pretrial risk assessments as an alternative to financial bail. Prior research has shown that implementation of pretrial risk assessments is associated with a reduction in financial bond decisions (Cooprider, 2009; Copp et al., 2022; Lowder et al., 2021; Sloan et al., 2018; Stevenson, 2018). This is true even in a state such as Indiana, where pretrial defendants retain a constitutional right to bail (Indiana Constitution, Article 1, §17). However, the effect of pretrial risk assessments on release decisions seems to be driven more by a policy shift to increase use of nonfinancial release options in lieu of money bail than an explicit and heavy emphasis on risk-informed decision-making. The policy shift accompanying pretrial risk assessment implementation likely explains why prior empirical findings largely show that pretrial risk assessment implementation is associated with greater rates of nonfinancial release but not a reduction in pretrial misconduct.
Implications
Our findings suggest several implications for future practice, policy, and research. First, our findings raise more fundamental questions about the purpose of pretrial risk assessments, particularly what agencies expect to gain from the use of such tools. In criminal-legal contexts, risk assessments may serve a multitude of goals—for the agency, for staff, and for individuals involved in the system (Vincent & Viljoen, 2020). These goals may include increasing staff accountability in decision-making, improving communication around decision-making, improving adherence to Risk-Need-Responsivity principles, improving consistency of decision-making, improving treatment or program outcomes, or reducing rates of violence and reoffending in the community (Vincent & Viljoen, 2020). Although there remains limited evidence that pretrial risk assessments achieve risk management objectives (Lowder et al., 2021; Sloan et al., 2018; Stevenson, 2018; Viljoen et al., 2018), risk assessment tools can facilitate other goals like addressing and intervening around criminogenic needs. Needs-based intervention could allow for more customized pretrial release conditions for individuals compared with typical pretrial monitoring. Although such conditions have been a recognized legal standard for decades (American Bar Association, 2007), other organizations have expressed concern about mandating engagement in services or treatment during the pretrial period (Pretrial Justice Institute, 2021).
Second, given the weight that such decision-making matrices place on pretrial risk assessments, use of such tools may not be expected to reduce rates of pretrial misconduct. Rather, these tools and their associated structured guidelines may serve as a framework to structure pretrial decision-making. Risk assessment scholars have noted that such instruments should be used to inform pretrial decision-making, not serve as a unilateral determinant of pretrial release and supervision decisions (Desmarais & Lowder, 2019). Our findings draw parallels to the sentencing context, where risk assessments function as an extralegal consideration within a limiting retributivism punishment framework (Monahan & Skeem, 2016). Researchers have argued that structured guidelines are a key component of the integration of risk assessments into criminal-legal practice (Garrett & Monahan, 2020). However, there seems to be a divide in research and practice regarding whether pretrial risk assessments should function as a mechanism to facilitate a more consistent and structured process or to inform risk-based decision-making. These goals are not necessarily mutually exclusive, but researchers, practitioners, and policymakers alike should be aware of these considerations and their potential implications for evaluation of these tools in practice.
Third, and relatedly, placing more weight on legal and extralegal factors not measured by a pretrial risk assessment tool (i.e., noncriminogenic factors) may undermine the ability of the tool to inform risk-based decision-making. It is likely politically impossible for a local jurisdiction to fully exclude index charge information from pretrial release and supervision decision-making. Indeed, many risk assessment tools such as the PTRA and PSAD identify characteristics of the index charge as criminogenic risk factors. However, when the index charge is not included in the calculation of a risk estimate, such as for the IRAS-PAT, jurisdictions should be cognizant of how heavy reliance on charge information may dilute the effectiveness of risk assessments on pretrial outcomes. One option is for jurisdictions to establish a “threshold” for offenses where risk assessments should have less influence—for example, violent offenses. For lower-level and nonviolent offenses, risk assessment information could guide decisions equally without consideration of the index charge. Ultimately, as others have argued, decisions about how risk assessments should be used to classify someone as “high risk” and the role of the index charge in determining pretrial release and supervision decisions should be a local policy consideration, not a research consideration (Berk et al., 2021; Solow-Niederman et al., 2019). Local jurisdictions may be more or less willing to provide less restrictive release or supervision options for defendants who are accused of committing certain crimes. However, these jurisdictions should be aware that the less emphasis they place on risk assessment, the less potential impact risk assessments will have as a community-level risk management strategy.
Finally, critics of pretrial risk assessments argue that such tools may exacerbate racial bias in pretrial decision-making (e.g., see Minow et al., 2019; Pretrial Justice Institute, 2020). To risk assessment scholars who study differential effects of risk assessments by race, our findings underscore the importance of understanding the local policy and practice context—which includes understanding the legal (and extralegal) drivers that bring individuals into the system at higher rates. For example, if racial disparities are particularly prevalent for drug-related arrests and the local jurisdiction does not allow pretrial risk assessment information to be considered for those types of offenses, then risk assessment information will have a more limited effect on status quo disparities. Conversely, jurisdictions that repeatedly incorporate legal factors such as the index charge in risk assessment scoring, in structured guidelines, and in judicial overrides of structured guidelines may temper the effectiveness and utility of risk assessments and potentially exacerbate disparities in criminal case processing outcomes (Kurlychek & Johnson, 2019). Identifying, studying, and intervening around these issues require knowledge of how pretrial risk assessments are implemented into local decision-making. Qualitative work is important in this field and underrepresented; there is a great need for more engagement with local jurisdictions around pretrial decision-making policy and practice. For example, researchers could conduct in-depth case file reviews, interviews with key stakeholders, and ethnographic observations of decision-making stages to identify whether local policies and practices knowingly or unknowingly propagate disparities across racial groups. Observations of critical case processing points and interviews with pretrial defendants could inform how the experience of pretrial processing may differ across racial groups. Finally, observations of decision-making stages could inform how decision-makers, including pretrial officers and judges, wield their discretion for defendants and the ways in which discretion may further or mitigate disparities.
Limitations and Future Directions
Our findings should be considered in light of several limitations, which may inform future research directions. First, we conducted this investigation in a single state (Indiana) and with a single risk assessment tool (IRAS-PAT). Our findings may have limited generalizability to other states and tools. However, the single focus of this study is also a strength in that we removed any sources of variability in structured decision-making matrices attributable to different state policies or tools. Even with the same pretrial risk assessment tool, there were stark differences in how risk assessment findings were applied. Nevertheless, replication in other jurisdictions is warranted. Second, our study was descriptive in nature and focused on exploring how pretrial risk assessments are incorporated into structured guidelines. Given the scope of the study, we did not link our findings to local data on pretrial defendants. As such, we had no data on how defendants would be distributed across risk and charge levels of the matrix. Our analysis assumed an equal distribution of defendants across all matrix cells. However, among typically sized matrices (i.e., those that were 3 × 3 or 3 × 4), we found no consistent evidence that pretrial risk assessment consideration was lower among more serious charge types. Finally, previous studies have examined rates of adherence to structured guidelines in the pretrial context (Copp et al., 2022; Danner et al., 2015; Lowder et al., 2021; Lowder & Foudray, 2021), and we used these estimates to project the potential influence of risk assessments on decisions. We do not know the exact rates of adherence to structured guidelines across all 29 counties that provided matrices. True estimates could be higher or lower, which was the primary reason why we extrapolated findings from 0% to 100% adherence.
Advancing the science and practice of pretrial risk assessments will require more comprehensive research on the use of such tools in practice. Future investigations should examine how the arrangement of—and adherence to—structured guidelines moderates the effectiveness of pretrial risk assessment tools. There exists some limited evidence that adherence to structured guidelines plays a role in the risk management potential of these tools (Lowder et al., 2021); however, few studies have examined the impact of judicial adherence to structured guidelines on defendant outcomes in pretrial settings. Studies with an eye to implementation will be essential to understanding whether and how risk assessment tools can facilitate the current goals of pretrial reform. Alternatively—in light of growing calls by researchers (Desmarais et al., 2021), funding agencies (Pilnik, 2017), and advocacy groups (APPR, 2022b) to improve assessment and intervention around criminogenic needs as well as risks in the pretrial period—these studies may prompt reconsideration of both the goals of the pretrial period and the broader purpose of risk assessments in this setting.
Conclusions
Overall, our findings showed that pretrial decision-making matrices placed more weight on charge information relative to pretrial risk assessment information. The degree to which this was true varied across local jurisdictions, highlighting the considerable discretion maintained by local jurisdictions when implementing pretrial risk assessment tools. Key stakeholders responsible for developing policies and procedures for local pretrial risk assessment use should be aware of the tradeoffs in prioritizing pretrial risk assessment information versus other legal and extralegal factors. Greater emphasis on non-risk assessment factors such as the index charge may reduce the ability of pretrial risk assessments to measurably impact key pretrial outcomes.
Supplemental Material
sj-docx-1-cjb-10.1177_00938548231177709 – Supplemental material for Pretrial Decision-Making Matrices: The Role of Risk and Charge Weighting in Risk Assessment–Guided Decisions
Supplemental material, sj-docx-1-cjb-10.1177_00938548231177709 for Pretrial Decision-Making Matrices: The Role of Risk and Charge Weighting in Risk Assessment–Guided Decisions by Evan Marie Lowder, Zainab Bakarr Kamara and Autumn Kent in Criminal Justice and Behavior
Footnotes
AUTHORS’ NOTE:
We have no conflicts of interest to disclose.
Supplemental Material
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
