Explaining legal inconsistency

Abstract

Judges, scholars, and commentators decry inconsistent areas of judicially created policy. This could hurt courts’ policy making efficacy, so why do judges allow it to happen? I show judicially-created policy can become inconsistent when judges explain rules in more abstract terms than they decide cases. To do so, I expand standard case-space models of judicial decision making to account for relationships between specific facts and broader doctrinal dimensions. This model of judicial decision making as a process of multi-step reasoning reveals that preference aggregation in such a context can lead to inconsistent collegial rules. I also outline a class of preference configurations on collegial courts (i.e., multi-member courts) in which this problem cannot arise. These results have implications for several areas of inquiry in judicial politics such as models of principal-agent relationships in judicial hierarchies and empirical research utilizing case facts as predictor variables.

Keywords

Judicial politics Judicial rule making Stare decisis Collegial decision making

A wide range of observers have noted particularly inconsistent rules being produced by courts across several areas of the law. For example, legal scholars complain the U.S. Supreme “Court’s numerous [federal] preemption cases follow no predictable jurisprudential or analytical pattern” (Dinh, 2000).¹ Political commentators criticize the Court’s “Establishment Clause decisions that have been, in the words of Alice in Wonderland, curiouser and curiouser,” and hope the Court will “leaven with clarity the confusion it has sown” (Will, 2019). Supreme Court Justice Clarence Thomas bemoans “an Establishment Clause jurisprudence in shambles,” claiming the Court’s “jurisprudence has confounded the lower courts and rendered the constitutionality of displays of religious imagery on government property anyone’s guess… ” (Utah Highway Patrol Assoc. v. American Atheists Inc., 565 U.S. 994 (2011) at 994, Thomas, J., dissenting).

Courts’ policies are implemented by others, from lower courts applying appellate court rules, to outside actors enforcing judicially created policies (Maltzman et al., 2000, 5). When courts’ rulings are unpredictable, and their rules are confusing, it impedes these actors’ ability to implement judicial policies. Moreover, inconsistency in legal doctrine reduces judicial legitimacy (Landa and Lax, 2009, 959). Why would courts create confusing policies that endanger judicial legitimacy and their efficacy as policymakers? Perhaps judges are free to act relatively unconstrained (e.g. Segal and Spaeth, 2002), and current court members simply prefer outcomes inconsistent with prior cases. Or perhaps courts’ decisions are well explained by pronounced rules, even when scholars and commentators believe an area of the law is in disarray (Segal, 1984). Maltzman et al. (2000) explain that bargaining over opinion content among justices may produce results inconsistent with what we might otherwise expect. However, none of these accounts explain why courts’ descriptions of their decision rules do not provide clear guidance for lower court judges and other policy enforcers.

I use a social choice theoretic model to show preference aggregation on collegial courts can result in inconsistent rules when judges communicate policy in terms of subjective criteria that depend on objective facts.² That is, judges often explain rules using a low number of abstract determinations that in turn are derived from specific facts of cases. I show this kind of multi-step reasoning in appellate review can result in inconsistent collegial rules.

For example, in Fourth Amendment search and seizure cases, the constitutionality of police conduct can depend on (1) the intrusiveness of the search or severity of the seizure, and (2) whether the police had the requisite level of suspicion (e.g. probable cause) required to support such conduct. The court must determine how intrusiveness and police suspicion translate into outcomes, and further use the specific facts of cases to determine the level of police suspicion: “As the Court recognizes, determinations of probable cause and reasonable suspicion involve a two-step process. First, a court must identify all of the relevant historical facts … and second, it must decide whether … those facts would give rise to a reasonable suspicion justifying a stop or probable cause to search” (Ornelas v. United States, 517 U.S. 690 (1996) at 700–701, Scalia, J., dissenting).

To make this even more concrete, consider the case Terry v. Ohio. In Terry, a police officer observed Terry and two compatriots suspiciously “casing” a store. Although he had no other information about the men, he believed a robbery was imminent, and “feared ‘they may have a gun”’, so he approached them, stopped them, and frisked them for weapons. He found weapons on Terry and one of the other men, and they were convicted of weapons charges. These concrete events that happened, and the evidence collected, are the specific facts of the case, or the “historical facts” as Justice Scalia puts it. While the Court did not find these facts amounted to probable cause, they said the evidence of criminal conduct amounted to “reasonable suspicion”. Again, though the Court did not find these facts constituted an arrest, the seizure of Terry did constitute an investigatory stop. These findings are the abstract determinations I mentioned above, which I will call doctrinal facts throughout the article. The Court announced investigatory stops may be justified by reasonable suspicion; in other words, the Court updated doctrine.

When courts engage in such multi-step reasoning, opportunity for inconsistency in the resulting collegial rules arises, even when all the judges possess well-behaved preferences. The problem arises because with multiple levels of judgment or preference aggregation, judges can agree on outcomes while disagreeing on the proper justification for that outcome, so that applying the reasoning relied on by a majority coalition in any one case can be inconsistent with collegial outcomes in other cases. This source of inconsistency in the law is understudied despite related results in the literature (e.g. Kornhauser, 1992; Landa and Lax, 2009) because models have left unexplored the interaction between disagreements over doctrine and disagreements over intermediate legal determinations, or doctrinal facts.³

This mechanism leading to doctrinal inconsistency raises implications for some areas of research in judicial politics. For example, there is a large literature that uses case facts as explanatory variables in empirical models of judicial behavior (e.g. Segal, 1984; Richards and Kritzer, 2002; Bartels and O’Geen, 2015; Epstein et al., 2018). Studies utilizing doctrinal facts may ignore that individual judges can have different determinations of their own on such doctrinal facts, while if only historical facts are used, important inconsistencies in the reasoning presented by courts can be obscured. There is also a large literature on principal-agent relationships in judicial hierarchies (e.g. Cameron et al., 1994; Westerland et al., 2010; Baker and Kornhauser, 2015). This paper raises an important question for future research of these relationships: the decision to engage in the multi-step reasoning studied here is itself a strategic decision. If the appellate court defers to trial court findings of doctrinal facts, this multi-step reasoning does not occur. (See also the appendix titled “Deference to Trial Court Findings” for discussion of situations in which appellate courts may even revisit findings on historical facts, another setting in which such multi-step reasoning can occur). For example, in the Ornelas decision quoted above, the Supreme Court resolved a circuit split over whether findings of probable cause should be reviewed de novo or with deference (in favor of de novo review), resulting in multi-step reasoning in Fourth Amendment cases. When will collegial appellate courts choose increased control over trial court agents, even with the risk of the type of doctrinal inconsistency studied here, rather than defer to agents’ findings?⁴

After a short survey of the substantive literature, I provide a brief overview of related models before detailing the setup of a model that allows for courts’ multi-step reasoning. I then show why inconsistency in the law can result when appellate courts communicate policy this way, as well as when they can safely do so while maintaining clear policy; I illustrate these results with a simple Fourth Amendment example.

1. Causes and Consequences of Inconsistency

If an appellate court’s “jurisprudence [confounds] the lower courts” and makes the proper decision in future cases “anyone’s guess” (Utah Highway Patrol Assoc., 565 U.S. at 994, Thomas, J., dissenting), the court will be less effective as a policy maker. Such inconsistency also raises normative concerns—crafting an inconsistent doctrine leaves citizens potentially less empowered to assert their rights (since they can’t tell when they apply). Nevertheless, legal scholars highlight time and time again various doctrines that have grown inconsistent, from death penalty jurisprudence (Robinson and Simon, 2006) to First Amendment jurisprudence (Post, 1995) to federalism jurisprudence (Drahozal, 2004).

Empirical work has well documented the effects of unclear doctrine on courts’ policy-making efficacy. Spriggs (1996) argues administrative agencies will be more likely to follow Supreme Court opinions that offer clearer guidance, and finds evidence that agencies more closely follow opinions that were more specific and explicit. Westerland et al. (2010) hypothesize that unclear signals from the U.S. Supreme Court will lead to lower compliance by the appellate courts, finding an increased number of concurrences indeed reliably correlated with lower compliance.

Empirical work has also uncovered some causes of inconsistency or complexity in judicial behavior. Collins (2008) finds individual justices’ choices are more variable in complex cases. Owens and Wedeking (2011) use text analysis methods to measure the cognitive complexity of court decisions,⁵ finding, for example, that some justices provide clearer guidance in their opinion than others on average, and that majority opinions are less clear than dissents, perhaps due to the bargaining entailed in crafting a binding precedent (1032–1033; Maltzman et al., 2000).

Related theoretical work includes the discovery of the “doctrinal paradox” (Kornhauser, 1992) and its extension (Landa and Lax, 2009),⁶ as well as work on rules vs. standards (e.g. Clark, 2016; Lax, 2012). The doctrinal paradox shows that outcomes depend on whether judges on collegial courts decide cases by majority vote over outcomes or by majority vote over intermediate determinations, such as whether police had probable cause. Interestingly, Kornhauser (1992, 447) explicitly envisions the cases as coming from a fact space that the judges must then map to these intermediate conclusions, but does not model how the judges make these intermediate determinations; accounting for this step in judicial reasoning is one of the principal technical contributions of this article.

However, Kornhauser (1992) assumes legal rules are fixed, while appellate courts themselves create legal rules. So Landa and Lax (2009) instead assume the intermediate conclusions are fixed, but allow each judge on a collegial court to have their own preferred legal rule. With this setup, the paradox that arises is that the rule implied for the court is different if the judges directly vote over rules or vote over outcomes in cases. Additionally, “it might not be possible to form the same type of rule for a court as a whole as any individual judge might have. That is, to the extent that individual rules are each representative of coherent legal philosophies, it may not be possible to construct a similarly principled collegial doctrine” (949). This captures a type of legal incoherence, and I build on these two models to additionally capture uncertainty, or the type of incoherent policy that renders the proper decision in a case “anyone’s guess” as Justice Thomas complained.

The rules vs. standards literature tackles a separate but related issue to the doctrinal inconsistency I study. These studies seek to explain when judges will issue specific policies and when they will use vague policy. For example, Staton and Vanberg (2008) shows courts may use vague rules to prevent observed noncompliance with rulings by ideologically divergent governments or to allow leeway to governments that are ideologically aligned with the court.

Most on point for the present article in this vein are Clark (2016) and Lax (2012). Clark studies the trade-off between an opinion that clearly disposes of cases closely related to the present case and an opinion that is less precise but has more impact on dissimilar cases. Clark finds judges will be more precise when the instant case is most representative of potential disputes and when they anticipate being able to issue additional clarifying rulings in the future. This analysis starts from the important point that judges generally cannot specify a complete mapping from cases to outcomes in a single opinion. The import of Proposition 2 below, detailing the general susceptibility of doctrine to inconsistency, involves this issue; inconsistency has real bite precisely when judges cannot perfectly map every potential future dispute to an outcome.

Lax (2012) considers the ability of an appellate court to promulgate a bright-line rule that depends only on an objective fact, or a standard based also on a subjective dimension such as severity of the weather. In this context, we may say bright-line rules are specific or precise, whereas standards based on a subjective dimension are less precise, either because the Court cannot perfectly observe the subjective dimension or because it is difficult to specify doctrinal requirements on that dimension. In the first case, standards are preferred despite their vagueness when the ability to observe the subjective dimension is relatively higher, or there is lower risk of ideologically opposed lower courts. In the second, standards can be attractive despite imprecision if the weight placed on the subjective dimension in the Court’s preferences is high enough, or if the cost of writing more precise opinions is low enough. This provides a nuanced account of incentives to rely on potentially vague doctrine, but again, does not wrestle with inconsistency in doctrine.

Evidence exists that courts’ policy-making efficacy depends on legal clarity, and normatively we may expect courts to consistently interpret legal rights. Empirical work has uncovered some correlates of lack of clarity in the law, and theoretical work has shown conditions under which judges may choose vagueness over precision and clarity. I extend models of case-based adjudication (Kornhauser, 1992) and rulemaking (Lax, 2007) to show an explanation for inconsistent doctrine embedded in legal reasoning: Judges generally engage in multiple steps of judgment aggregation, and this multi-step reasoning provides more opportunity for inconsistency in aggregation than previous models have accounted for.

2. Rule Making on Collegial Courts

I use a case space model to study rule making on collegial courts (Lax 2011). A case space model considers the set of all possible cases, or factual scenarios, a court could be presented with, and represents judicial policy as dividing that space into outcomes. That is, the set of possible cases is divided into two sets: the set of cases where plaintiffs win and the set of cases where defendants win; or, the set of cases where government activity is permissible, and the set where it is unconstitutional.

In a traditional case space model, the court is presented with a case $x \in X \subseteq R^{n}$ , the set of all possible cases the court could hear.⁷ Each judge $j$ then has a preferred rule mapping cases to outcomes $ρ_{j} : X \to {- 1, 1}$ .⁸ The dimensions of $X$ are interpreted as “whatever facts might be considered relevant to the judges” (Landa and Lax, 2009, 593). Often models consider these facts to be high-level doctrinal concerns, such as the intrusiveness of a police search (Clark and Carrubba, 2012), or sometimes specific “historical” facts, such as the speed at which a car is travelling (Lax, 2012).

I will use as a running example the constitutionality of a seizure of a person—an investigatory stop or an arrest—under the Fourth Amendment.⁹ The Fourth Amendment to the U.S. Constitution provides the “right of the people to be secure … against unreasonable searches and seizures, shall not be violated…” (U.S. Const. Amend. IV). However, courts “must evaluate the reasonableness of a particular search or seizure in light of the particular circumstances” (Terry v. Ohio, 392 U.S. 1 (1968) at 21). For example, while arrests require probable cause, investigatory stops are less intrusive seizures that require only “reasonable suspicion” (Terry).

So, we might think of the case space dimensions as the doctrinal concerns of the level of police suspicion and severity of the seizure; an example of a rule in such a space is depicted in Figure 1a. In this example, there are some seizures so severe they could never be found constitutional, some circumstances under which there is so little evidence of criminality that no seizure could be constitutional, but as long as the seizure is sufficiently not severe and the police have sufficient certainty that criminal conduct has occurred, the judge will find the seizure was constitutional.

Figure 1.

An example individual rule and ICR for Fourth Amendment police seizure cases. The case space is comprised of two dimensions: severity of the police seizure, where larger values indicate a more intrusive seizure, and inverse police suspicion, where larger values indicate less certainty that criminal conduct has occurred.

Judges on collegial courts decide cases by majority rule over dispositions. The implicit collegial rule, or ICR, is the mapping between cases and outcomes that results from these majority votes over outcomes (Lax, 2007, 595). In other words, the ICR represents “the law.”¹⁰ An example of a three judge panel’s individual preferences and the resulting ICR is depicted in Figure 1b. In this case the judges’ preferences aggregate to an ICR in which for the lowest range of police suspicion, no seizure is warranted, for a moderate range of police suspicion low levels of seizure are permissible, and at the highest range of police suspicion a much broader range of seizures are found constitutional.

3. Model

Case space dimensions that capture high level doctrinal concerns are generated from historical facts, as Justice Scalia discusses in the Ornelas exerpt quoted in the introduction. As Lax (2007) explains, “in equal protection cases … the dimensions might include (1) how ‘suspect’ the class invoked is … (2) how compelling the state interest is … and (3) how necessary the classification is …Ȯr, these dimensions could be broken down further” (594). While the technology of traditional case space models can be used to model decisions based on historical facts, doctrinal concerns, or both, it lacks the ability to model the relationship between doctrinal concerns and the dimension of historical facts they are derived from. Abstracting away from this relationship is useful for analyzing other aspects of judicial decision making. However, to understand why outside observers are confused by judicial doctrine, it will be useful to separately represent the high dimensional space of all possible historical facts and the lower dimensional doctrinal space, and the relationship between these spaces.

A legal case presented to a court can be uniquely identified by its historical facts, such as whether a person seized by the police was placed in handcuffs or not, or how long a person was detained. We will say there are $N$ potentially relevant dimensions of historical facts, so that $H \subseteq R^{N}$ is the set of all possible combinations of historical facts.

A set of judges $J$ (with $| J |$ odd) must decide cases presented to it from $H$ , and assign them one of two outcomes ${- 1, 1}$ . So, as in other case space models, we will discuss policy as a partition of cases into outcomes. However, judges (and the public they communicate policies to) do not think about policy by considering every possible combination of historical facts, even if they could. They think about and communicate policy in more abstract terms informed by the historical facts, such as the severity of a police seizure or the degree of police certainty of criminality that supports the seizure. So we also need to define a lower dimensional doctrinal space, $D \subseteq R^{n}$ , with $1 < n < N$ .¹¹ Then each judge $j$ has a preferred doctrine $δ_{j}$ mapping $D$ to ${- 1, 1}$ . A doctrine is monotonic if for any two points $d, d^{'} \in D$ , $d_{i} \geq d_{i}^{'} \forall i$ implies $δ (d) \geq δ (d^{'})$ . We will assume the judges (and other relevant actors such as the public or lower court judges attempting to comply with the collegial appellate court’s rulings) can “consistently label” the dimensions of $H$ and $D$ such that higher values of any $h_{k}$ or $d_{i}$ should lead to a weakly higher outcome, all else equal.

Unfortunately, as we will see, judges can disagree not only over doctrine, but how historical facts map onto doctrinal facts.¹² Not only could judges disagree whether a particular type of police seizure needs to be supported by probable cause or only by reasonable suspicion, but they could disagree about whether the historical facts support a finding of probable cause or not. So, we add the last moving part to the model: each judge $j$ maps historical facts on to $D$ ; I will call this mapping a “fact finding function” $f_{j} : H \to D$ .¹³ For convenience, for a case $h$ and a fact finding function $f_{j}$ , we will write $d_{i j}$ to mean the $i$ th element of $f_{j} (h)$ . A fact finding function is monotonic if for any two points $h, h^{'} \in H$ , $h_{k} \geq h_{k}^{'} \forall k$ implies $d_{i j} \geq d_{i j}^{'} \forall i$ . For the remainder of the article, I assume all $f_{j}$ and $δ_{j}$ are monotonic.

In sum, each judge’s preferred disposition is thus determined by $δ_{j} (f_{j} (h))$ ; the judge is presented with the historical facts, they determine how those facts relate to the doctrinal dimensions they find relevant, and thus how the case should be decided according to their preferred doctrine. Thus, a judge’s preferred rule, or mapping from unique cases to outcomes, is a pair $ρ_{j} = (f_{j}, δ_{j})$ . This process is depicted in Figure 2.

Figure 2.

Assigning outcomes by translating a fact space to a doctrine space. A judge $j$ is presented with a set of historical facts, a point in a pontentially high dimensional space $H$ . Cases in this issue area are discussed using broader doctrinal terms—the lower dimensional space $D$ . So, the judge uses the function $f_{j}$ to translate the case from a point in $H$ to a point in $D$ , the space in which she describes her preferred partition ( $δ_{j}$ ) of cases into $- 1$ outcomes and $1$ outcomes.

Judges decide cases by majority vote over outcomes. Similarly to Landa and Lax (2009), define an outcome set as specifying the outcome ( $- 1$ or $1$ ) with each case $h \in H$ , and the collegial outcome set as the outcome set formed by majority voting among $J$ over the outcome in each case $h$ . A consistent rule is a rule $ρ = (f, δ)$ such that $f$ is monotonic and $δ$ is monotonic in $f$ . The implicit collegial rule (ICR) is the rule $ρ_{m} = (f_{m}, δ_{m})$ constructed as follows: $f_{m}$ takes the (dimension by dimension) median value of the $f_{j}$ for every $j$ in the majority coalition for every $h \in H$ ; and $δ_{m}$ maps $D_{m}$ to ${- 1, 1}$ using the collegial outcome set. A summary of notation used is presented in Table 1.

Table 1.

Notation Used.

$j$	A judge on the collegial appellate court.
$J$	The set of judges on the collegial appellate court.
$H$	The set of all possible combinations of historical facts.
$h_{k}$	One of the $N$ dimensions of $H$ .
$D$	The set of all possible combinations of doctrinal determinations.
$d_{i}$	One of the $n$ dimensions of $D$ .
$f_{j}$	The mapping from historical facts to doctrinal dimensions as seen by judge $j$ .
$δ_{j}$	The mapping from doctrinal determinations to outcomes preferred by judge $j$ .
$ρ$	A pair $(f, δ)$ mapping $H$ to outcomes through $D$ such that the outcome in case $h$ is $δ (f (h))$ .
$ρ_{m}$	The implicit collegial rule $(f_{m}, δ_{m})$ , where $f_{m}$ takes the (dimension by dimension) median value of the $f_{j}$ for every $j$ in the majority coalition for every $h \in H$ and $δ_{m}$ maps $D_{m}$ to ${- 1, 1}$ using the collegial outcome set.

4. Inconsistency from Multi-step Reasoning

Let us start with the simplest case, where the judges happen to agree on doctrine; that is, $δ_{j}$ is the same for all $j$ .¹⁴ For example, suppose the judges agree that some seizures of a person are never justified, probable cause is needed to justify others, and that some seizures can be justified merely by reasonable suspicion, but that the judges disagree on the set of historical facts that support a finding of probable cause or reasonable suspicion.

Three types of doctrines in particular will be of interest, both because they are common types of legal doctrines and because of their aggregation properties. Call a doctrine $δ$ such that

δ (d) = {\begin{matrix} 1 & if d \cdot w \geq τ \\ - 1 & otherwise, \end{matrix}

where

τ

is a scalar threshold and

w

is a vector of weights on the dimensions of

D

, a balancing test.¹⁵ A doctrine

δ

such that

δ (d) = {\begin{matrix} 1 & if d_{i} \geq τ_{i} \forall i \\ - 1 & otherwise, \end{matrix}

where

τ

is a vector of thresholds of length

n

, shall be called a conjunctive test.¹⁶ Finally, define a disjunctive test as a doctrine

δ

such that

δ (d) = {\begin{matrix} 1 & if \exists i : d_{i} \geq τ_{i} \\ - 1 & otherwise, \end{matrix}

where

τ

is a vector of thresholds of length

n

Then we can state the following:

Proposition 1
If all $δ_{j} = δ^{}$ , and $δ^{}$ is a balancing test, then $ρ_{m}$ is a consistent rule. If $δ^{}$ is a conjunctive or disjunctive test, $ρ_{m}$ need not be a consistent rule.

Call the situation in the first sentence of Proposition 1 a “shared balancing test.” Then let $δ = {δ_{j}}$ (and similarly for $f$ ) and let $F (δ)$ be the set of combinations of monotonic fact finding functions for the judges such that $ρ_{m}$ is not consistent given $δ$ and $f$ . Now we will deal with the more general case where judges may disagree on doctrine and state a more ominous result, which is a more general form of the second sentence in Proposition 1:
Proposition 2
If $δ$ is not a shared balancing test, $F (δ)$ is nonempty.

The implications of Proposition 2 explain a structural reason embedded in our common law system for inconsistent doctrine. Because the judges are engaging in multi-step reasoning to determine case outcomes, in general the court’s opinions taken as a whole can be inconsistent in the sense that doctrine is not monotonic in the findings of legal facts. To understand why such monotonicity is crucial, consider a situation in which we have not observed the court’s rulings in all of (the infinite number of) the potential cases, nor has the court completely revealed in its opinions $ρ_{m}$ . (Of course, this is in fact the situation we find ourselves in at all times).¹⁷ Then what we can say about the law, or “the prophecies of what the courts will do in fact” (Holmes, 1897), becomes very limited. If $δ_{m}$ is guaranteed to be monotonic in $f_{m}$ , we could deduce outcomes in some regions of $D_{m}$ , and we will have some information about the set of fact finding functions that could be $f_{m}$ . However, if $δ_{m}$ is not* guaranteed to be monotonic in $f_{m}$ , much less could be said about the outcomes we should expect in cases not observed. Whereas Clark (2016) models the Court’s strategy for reducing the uncertainty lower courts (and perhaps other actors) have about outcomes in cases so far unobserved, this result reveals a source of uncertainty courts have no choice over.

Moreover, when the revealed outcomes show $δ_{m}$ to be non-monotonic in $f_{m}$ , the collegial doctrine is revealed to be “perverse” (Lax, 2007, 594; Landa and Lax, 2009, 952).¹⁸ In other words, a person observing two different cases may believe in case one, the police had probable cause and conducted a seizure of a person amounting to an arrest, and in the second case, the police arrested a suspect with more evidence of criminality than in the first case, but find the court rules the police conduct constitutional in the first case but unconstitutional in the second. The ICR may even assign different outcomes to cases at the same location in the doctrine space. For example, a person may view two different set of historical facts and determine that in both cases police had probable cause and conducted a seizure of a person amounting to an arrest, and therefore acted in accordance with the Fourth Amendment, but observe the court rule the actions as constitutional in one case and unconstitutional in the other.

Let us make this example concrete, with $H = [0, 1]^{4}$ , $D = [0, 1]^{2}$ , and the judges’ fact-finding functions and doctrines as given in Table 2.¹⁹ Each of the judges has monotonic doctrines (disjunctive tests) and monotonic fact-finding functions; these are depicted in panels (a)–(c) of Figure 3, which show how each judge would place every case in $D$ and which outcome they would choose for those cases if they were deciding cases unilaterally. However, the collegial rule is decidedly inconsistent, as depicted in panel (d), which shows the implicit collegial rule, or how the collegial fact-finding function $f_{m}$ would place every case in $D$ and which outcome is assigned under the collegial outcome set. Although difficult to depict, in the darkly shaded region where both outcomes occur, the density of cases receiving each outcome varies, and importantly sometimes in an alternating fashion. We see both types of problems mentioned in the previous paragraph: opposing outcomes occurring at the same point in $D$ , and violations of strict monotonicity as well.

Figure 3.
An example of an inconsistent doctrine. The doctrine space is comprised of two dimensions: severity of the police seizure, where larger values indicate a more intrusive seizure, and inverse police suspicion, where larger values indicate less certainty that criminal conduct has occurred. The judges all have preferred monotonic doctrines and monotonic fact-finding functions; the judges’ preferred rules are depicted in panels (a)–(c). However, the implicit collegial rule is inconsistent as depicted in panel (d). In each panel, the three cases from Table 3 are labeled with their identifying number.

Table 2.
Doctrines and fact-finding functions for the judges on the collegial court.

$j$ $f$ $δ$

1 $(0.5 h_{1} + 0.5 h_{2}, 0.5 h_{3} + 0.5 h_{4})$ $1 \Leftrightarrow d_{1} > 0.750 \lor d_{2} > 0.750$

2 $(0.6 h_{1} + 0.4 h_{2}, 0.4 h_{3} + 0.6 h_{4})$ $1 \Leftrightarrow d_{1} > 0.375 \lor d_{2} > 0.750$

3 $(0.4 h_{1} + 0.6 h_{2}, 0.6 h_{3} + 0.4 h_{4})$ $1 \Leftrightarrow d_{1} > 0.750 \lor d_{2} > 0.375$

Table 3.
Example cases showing inconsistency.

Case $h$ $j$ $d$ Outcome

1 $(0.70, 0.05, 0.75, 0.00)$ 1 $(0.375, 0.375)$ $1$

2 $(0.440, 0.300)$

3 $(0.310, 0.450)$

$m$ $(0.375, 0.375)$

2 $(0.15, 0.60, 0.15, 0.60)$ 1 $(0.375, 0.375)$ $- 1$

2 $(0.330, 0.420)$

3 $(0.420, 0.330)$

$m$ $(0.375, 0.375)$

3 $(0.15, 0.65, 0.40, 0.40)$ 1 $(0.400, 0.400)$ $- 1$

2 $(0.350, 0.400)$

3 $(0.450, 0.400)$

$m$ $(0.375, 0.400)$

We can highlight a few specific cases to make this easier to see. Consider the cases listed in Table 3; these cases are labeled with their number in Figure 3. In case 1, both judges 2 and 3 find the case satisfies one element of their disjunctive test (though different ones), so both vote for outcome $1$ , while in case 2, both judges 1 and 3 find the case satisfies neither element of their disjunctive test, and so both vote for outcome $- 1$ . So, while $f_{m}$ places both cases at $(0.375, 0375)$ , they receive opposing outcomes! This is so because while the judges’ individual preferences at both levels of aggregation are assumed to be very well behaved, the different levels of aggregation do not always agree with each other. Then, in case 3, judges 1 and 2 find the case satisfies neither element of their disjunctive test, and so both vote for outcome $- 1$ , resulting in a case at $f_{m} = (0.375, 0.4)$ having an outcome of $- 1$ even though a case at $f_{m} = (0.375, 0.375)$ has an outcome of $1$ .

Not only is the implicit collegial rule inconsistent, the collegial outcome set is also not monotonic in any of the judges’ projection of historical facts into doctrine space, as depicted in Figure 4. Table 4 singles out three more cases for consideration, all of which are labeled in the panels of Figure 4. Cases 4 and 5 cause inconsistency under both $f_{1}$ and $f_{3}$ . For judge 1, cases 4 and 5 occupy the same point in $D_{1}$ but receive opposite collegial outcomes. For judge 3, case 4 is more extreme on both doctrinal dimensions than case 5, but receives a -1 outcome where case 5 receives a 1 outcome; that is, this is a situation where in case 4, judge 3 considers that there is both less evidence of criminality and a more severe seizure than in case 5, but the court rules that the seizure in case 4 is constitutional whereas the seizure in case 5 is not. For judge 2, cases 5 and 6 reveal inconsistency in a similar manner to cases 4 and 5 for judge 3.

Figure 4.
An example of an inconsistent doctrine. The doctrine space is comprised of two dimensions: severity of the police seizure, where larger values indicate a more intrusive seizure, and inverse police suspicion, where larger values indicate less certainty that criminal conduct has occurred. The judges all have preferred monotonic doctrines and monotonic fact-finding functions, but the collegial outcome set is not monotonic in any of the judges’ projection of historical facts into doctrine space, or even the projection taking the dimension-by-dimension median placement of the majority coalition in every case.

Table 4.
Further example cases showing inconsistency.

Case $h$ $j$ $d$ Outcome

4 $(0, 0.875, 0.875, 0)$ 1 $(0.438, 0.438)$ $- 1$

2 $(0.350, 0.350)$

3 $(0.525, 0.525)$

$m$ $(0.394, 0.394)$

5 $(0.125, 0.75, 0.75, 0.125)$ 1 $(0.438, 0.438)$ $1$

2 $(0.375, 0.375)$

3 $(0.500, 0.500)$

$m$ $(0.438, 0.438)$

6 $(0.875, 0, 0, 0.875)$ 1 $(0.438, 0.438)$ $- 1$

2 $(0.525, 0.525)$

3 $(0.350, 0.350)$

$m$ $(0.394, 0.394)$

Of course, in this example, the structure of $H$ is relatively simple, and the $f_{j}$ appear easy enough to communicate. Even if lower court judges and members of the public have not had a chance to observe the full mapping from $H$ to $D_{m}$ to outcomes, assuming the judges had full knowledge of their preferences they could simply announce when deciding any case the association between $H$ and the collegial outcome set. However, it is important to note the differences between an easily understood toy example like this and the even worse situation we generally find ourselves in. Generally $H$ will be of a much higher dimensionality than $4$ ; consider our Fourth Amendment example, where it is relevant whether the police restrained the suspect, the duration of the seizure, the credibility of information the police are acting on, what the supect was doing at the time of the seizure, etc. (many of which could be further broken down into multiple historical fact dimensions, but were not for simplicity here). Moreover, the judges are unlikely to know even their own full mapping $f_{j}$ . For example, in the model of Callander and Clark (2017), the High Court does not know with certainty their preferred legal outcome in a particular set of factual circumstances until they observe such a case, a reasonable assumption in many contexts. Additionally, judges are even commonly presented with new historical factual dimensions that they have never considered before, and they often do not know how such facts affect where the judge will place the case in $D$ until they have occasion to consider it.

In other words, judges often cannot create general doctrinal statements in terms of $H$ ; they must communicate their general decision principles in terms of $D$ , and relate cases $h \in H$ to $D$ as they come. In this setting, I have shown a very troublesome result; policies generated and communicated using such multi-step reasoning are generally subject to doctrinal inconsistency.
5. Discussion

$j$	$f$	$δ$
1	$(0.5 h_{1} + 0.5 h_{2}, 0.5 h_{3} + 0.5 h_{4})$	$1 \Leftrightarrow d_{1} > 0.750 \lor d_{2} > 0.750$
2	$(0.6 h_{1} + 0.4 h_{2}, 0.4 h_{3} + 0.6 h_{4})$	$1 \Leftrightarrow d_{1} > 0.375 \lor d_{2} > 0.750$
3	$(0.4 h_{1} + 0.6 h_{2}, 0.6 h_{3} + 0.4 h_{4})$	$1 \Leftrightarrow d_{1} > 0.750 \lor d_{2} > 0.375$

Case	$h$	$j$	$d$	Outcome
1	$(0.70, 0.05, 0.75, 0.00)$	1	$(0.375, 0.375)$	$1$
		2	$(0.440, 0.300)$
		3	$(0.310, 0.450)$
		$m$	$(0.375, 0.375)$
2	$(0.15, 0.60, 0.15, 0.60)$	1	$(0.375, 0.375)$	$- 1$
		2	$(0.330, 0.420)$
		3	$(0.420, 0.330)$
		$m$	$(0.375, 0.375)$
3	$(0.15, 0.65, 0.40, 0.40)$	1	$(0.400, 0.400)$	$- 1$
		2	$(0.350, 0.400)$
		3	$(0.450, 0.400)$
		$m$	$(0.375, 0.400)$

Case	$h$	$j$	$d$	Outcome
4	$(0, 0.875, 0.875, 0)$	1	$(0.438, 0.438)$	$- 1$
		2	$(0.350, 0.350)$
		3	$(0.525, 0.525)$
		$m$	$(0.394, 0.394)$
5	$(0.125, 0.75, 0.75, 0.125)$	1	$(0.438, 0.438)$	$1$
		2	$(0.375, 0.375)$
		3	$(0.500, 0.500)$
		$m$	$(0.438, 0.438)$
6	$(0.875, 0, 0, 0.875)$	1	$(0.438, 0.438)$	$- 1$
		2	$(0.525, 0.525)$
		3	$(0.350, 0.350)$
		$m$	$(0.394, 0.394)$

Legal inconsistency is a problem, both for judges as policy makers, since agents and outside actors cannot follow or implement rules they do not understand, and for the public, who might normatively expect consistent application of legal rights. The prior literature offers explanations for inconsistency in individual judges’ choices and their preferences (e.g. Collins, 2008; Maltzman et al., 2000) or for lack of precision in doctrine (e.g. Clark, 2016; Fox and Vanberg, 2014; Lax, 2012; Staton and Vanberg, 2008). Some sources of inconsistency in doctrine have been highlighted by Lax (2007) and Landa and Lax (2008, 2009); by expanding on such case-space models to account for judges’ multi-step reasoning, I highlight a new source of legal inconsistency. When we allow for disagreement over both how historical facts should be aggregated to doctrinal dimensions and how the doctrine space should be partitioned into outcomes, the resulting judgment and preference aggregation among judges displays inconsistency even under strict assumptions about how well-behaved the individual judgments and preferences are. The general presence of the danger of this inconsistency explains why so often, courts’ doctrines become inconsistent (Drahozal, 2004; Post, 1995; Robinson and Simon, 2006; Will, 2019).

This model also raises implications for other areas of judicial politics research. Related to research on the principal-agent relationship between appellate courts and trial courts, when will collegial appellate courts defer to lower court agents’ placement of cases in dotrine space to avoid this source of inconsistency on doctrine, and when will they exert more control despite the danger of doctrinal inconsistency shown in this article? For empirical research that uses case facts as explanatory variables, care should be used to recognize that individual judges can have different determinations of their own on such doctrinal facts, and if only historical facts are used, important inconsistencies in the reasoning presented by courts may be obscured.

Footnotes

Acknowledgements

I would like to thank Randall Calvert, Keith Schnakenberg, Jim Spriggs, Lee Epstein, Morgan Hazelton, Jordan Carr Peterson, and the anonymous reviewers for their helpful comments. A previous version of this paper was presented at the 2019 Annual Meeting of the Midwest Political Science Association.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

Appendix

Notes

ORCID iD

JBrandon Duck-Mayr

References

Badawi

Baker

(2015) Appellate lawmaking in a judicial hierarchy. The Journal of Law & Economics 58(1): 139–172.

Baker

Kornhauser

(2015) A Theory of Judicial Deference. https://perma.cc/C4QG-LJ46. Unpublished manuscript.

Bartels

O’Geen

(2015) The nature of legal change on the U.S. Supreme court: Jurisprudential regimes theory and its alternatives. American Journal of Political Science 59(4): 880–895.

Callander

Clark

(2017) Precedent and doctrine in a complicated world. American Political Science Review 111(1): 184–203.

Cameron

Segal

Songer

(1994) Strategic auditing in a political hierarchy: An informational model of the supreme court’s certiorari decisions. American Political Science Review 94(1): 101–116.

Clark

(2016) Scope and precedent: Judicial rule-making under uncertainty. Journal of Theoretical Politics 28(3): 353–384.

Clark

Carrubba

(2012) A theory of opinion writing in a political hierarchy. Journal of Politics 74(2): 584–603.

Collins

Jr (2008) The consistency of judicial choice. Journal of Politics 70(3): 861–873.

Dinh

(2000) Appellate lawmaking in a judicial hierarchy. Georgetown Law Journal 88: 2085–2118.

10.

Drahozal

(2004) The Supremacy Clause: A Reference Guide to the United States Constitution.Westport, Connecticut: Praeger.

11.

Epstein

Parker

Segal

(2018) Do justices defend the speech they hate? an analysis of in-group bias on the U.S. Supreme court. Journal of Law and Courts 6: 237–261.

12.

Fox

Vanberg

(2014) Narrow versus borad judicial decisions. Journal of Theoretical Politics 26(3): 355–383.

13.

Hoffman

(2001) Corralling constitutional fact: De novo fact review in the federal appellate courts. Duke Law Journal 50: 1427–1466.

14.

Holmes

Jr (1897) The path of the law. Harvard Law Review 10: 457–478.

15.

Hübert R (2019) Getting their way: Bias and deference to trial courts. American Journal of Political Science 63: 706–718.

16.

Kornhauser

(1992) Modeling collegial courts ii. legal doctrine. Journal of Law, Economics, & Organization 8(3): 441–470.

17.

Landa

Lax

(2008) Disagreements on collegial courts: A case-space approach. Journal of Constitutional Law 10(2): 305–329.

18.

Landa

Lax

(2009) Legal doctrine on collegial courts. Journal of Politics 71(3): 946–963.

19.

Lax

(2007) Constructing legal rules on appellate courts. American Political Science Review 101(3): 591–604.

20.

Lax

(2012) Political constraints on legal doctrine: How hierarchy shapes the law. Journal of Politicse 74(3): 765–781.

21.

List

(2012) The theory of judgment aggregation: An introductory review. Synthese 187(1): 179–207.

22.

List

Pettit

(2002) Aggregating sets of judgments: An impossibility result. Economics and Philosophy 18: 89–110.

23.

Maltzman

Spriggs

JFII

Wahlbeck

(2002) Crafting Law on the Supreme Court: The Collegial Game. New York, NY: Cambridge University Press.

24.

Nehring

Puppe

(2006) Consistent judgement aggregation: The truth-functional case. Social Choice and Welfare 31(1): 41–57.

25.

Nehring

Puppe

(2010) Justifiable group choice. Journal of Economic Theory 145(2): 583–602.

26.

Owens

Wedeking

(2011) Justices and legal clarity: Analyzing the complexity of U.S. Supreme court opinions. Law & Society Review 45(4): 1027–1061.

27.

Post

(1995) Recuperating first amendment doctrine. Stanford Law Review 47(6): 1249–1281.

28.

Redish

Gohl

(2017) The wandering doctrine of constitutional fact. Arizona Law Review 59: 289–338.

29.

Richards

Kritzer

(2002) Jurisprudential regimes in supreme court decision making. American Political Science Review 96(2): 305–320.

30.

Robinson

Simon

(2006) Logical and consistent? an analysis of supreme court opinions regarding the death penalty. Justice Policy Journal 3(1): 1–59.

31.

Segal

(1984) Predicting supreme court cases probabilistically: The search and seizure cases, 1962-1981. American Political Science Review 78(4): 891–900.

32.

Segal

Spaeth

(2002) The Supreme Court and the Attitudinal Model Revisited. New York, NY: Cambridge University Press.

33.

Spriggs

JFII

(1996) The supreme court and federal administrative agencies: A resource-based theory and analysis of judicial impact. American Journal of Political Science 40(4): 1122–1151.

34.

Staton

Vanberg

(2008) The value of vagueness: Delegation, defiance, and judicial opinions. American Journal of Political Science 52(3): 504–519.

35.

Westerland

Segal

Epstein

, et al. (2010) Strategic defiance and compliance in the u.s. courts of appeals. American Journal of Political Science 54(4): 891–905.

36.

Will

(2019) The supreme court can undo past confusion with its ruling on this WWI memorial. https://www.sltrib.com/opinion/commentary/2019/02/24/george-f-will-supr.