Abstract
Different from more traditional policy-making avenues, the courts provide an antipolitical arena that does not require broad agreement from various constituents for policy enactment. Seeking to guide court decisions on these policy issues, individuals and organizations have filed amicus briefs that increasingly include social science to support their arguments. The Fisher v. University of Texas at Austin Supreme Court case presents an ideal example to study the use of social science evidence in amicus briefs to shape educational policy. Findings from this study identify differences in the use of social science research that suggest many ways in which our current understanding of the efforts of actors to shape educational policy via the highest court in the nation is incomplete. This study also highlights why developing this understanding could be extremely useful to both the creation of educational policy and the use of antipolitical approaches to change such policy.
Introduction
Although courts, especially the U.S. Supreme Court, serve as a mechanism to influence educational policy making in the United States (Dunn & West, 2008; Garces, 2013), this approach is often ignored in policy-making conversations. In the 21st century alone, the U.S. Supreme Court has issued several critical education-related opinions that have directly shaped educational policy nationally. Some of these cases, for example, include decisions that restructured voluntary K-12 school desegregation policies (Parents Involved in Community Schools v. Seattle School District No. 1, 2007), upheld a voter initiative in Michigan that banned affirmative action policy at public colleges and universities (Schuette v. Coalition to Defend Affirmative Action et al., 2014), affirmed the constitutionality of postsecondary race-conscious admissions policies (Fisher v. University of Texas I, 2013; Fisher v. University of Texas II, 2016; Gratz v. Bollinger, 2003; Grutter v. Bollinger, 2003), and clarified the rights of students with disabilities in schools (Endrew F. v. Douglas County School District, 2017). These recent instances provide firm foundation for the influence the nation’s highest court has not only on legal precedent but also on educational policy.
Seeking to guide the court’s decisions on an educational policy issue, individuals and organizations who are not the main parties in a case but who have an interest in its outcome can become involved as friends-of-the-court (i.e., amici curiae) by providing additional arguments and information relevant to the case via amicus curiae briefs. This avenue for engagement with the courts has become more and more important over time as illustrated by the fact that the number of amicus briefs submitted to the high court has steadily increased (Simard, 2008) as has the number of interest groups signing on to briefs (Box-Steffensmeier, Christenson, & Hitt, 2013; Collins, 2004). In addition, we are witnessing an increased use of social science to support the arguments amici make in these briefs (e.g., Roesch, Golding, Hans, & Reppucci, 1991; Rublin, 2011). Because “the most common method of introducing non-legal materials, such as social science evidence, to the court is through non-record evidence in amicus curiae briefs” (Simpson & Vasaly, 2015, p. 68), these briefs and their use of social science merit closer examination. This study, focusing on one court case, offers an opportunity to contribute a more systematic and nuanced understanding of the use of social science, which we define later in the article, in amicus briefs to influence the court’s opinion.
Although the study of the use of social science in policy making has examined administrative and legislative settings (see Asen, Gurke, Conners, Solomon, & Gumm, 2013; Davies, Nutley, & Smith, 2009; DeBray, Scott, Lubienski, & Jabbar, 2014; Nutley, Walter, & Davies, 2008; Weiss, 1987), much less focus has been placed on how research is used in the legal arena (Mertz, 2010; Weiss, 1987)—a field where key educational policy issues are regularly argued and decided. In particular, researchers who have studied the court’s use of social science included in amicus briefs tend to agree that such evidence has come to play an influential, albeit nuanced, role in judicial decision making particularly on questions of constitutional interpretation (Ancheta, 2006; Erickson & Simon, 1998; Frankenberg & Garces, 2008; Garces, 2013; Morgan & Pullin, 2010).
As educational policy continues to be influenced via the courts, and in light of the growing use of social science research by amici as policy actors, we must have a better understanding of the inclusion of social science research by those attempting to shape policy via the legal arena. As Justice Breyer (1998) has suggested, amicus briefs “help [the U.S. Supreme Court] to become more informed about the relevant science” and. . . this “education improves the quality of [the Court’s] decisions” (para. 11). With implications for amici submitting briefs and social scientists conducting research, this inquiry can lead to a better understanding of how these actors are attempting to leverage this policy entry point and, ultimately, the degree to which, as Justice Breyer indicated, helpful information is put before the court to inform deliberations.
Framing the Legal Context: Fisher v. University of Texas at Austin
We focus our study on the first round of amicus briefs submitted to the U.S. Supreme Court in Fisher v. University of Texas at Austin. Specifically, the court was asked to consider whether the Fifth Circuit decision was “consistent with ‘this Court’s decisions interpreting the Equal Protection Clause of the Fourteenth Amendment” (Fisher v. Texas (Fisher I), 2013, p. 2415). Also known as Fisher I, the court’s 2013 opinion remanded the case to the Fifth Circuit Court of Appeals for additional review using the strict scrutiny standard. The Circuit Court’s affirmation of its own original ruling, after the court’s remand in Fisher I, was appealed by Fisher to the Supreme Court, which agreed to hear the case a second time with a focus on whether the Circuit Court’s re-endorsement of UT’s race-conscious policy met constitutional requirements. In 2016, the court ruled in favor of UT (Fisher II).
Fisher I presents a rich avenue and ideal example to study the use of social science evidence in amicus briefs submitted to the U.S. Supreme Court for several reasons: (a) the case addresses the type of education-related policy matters (in this instance, affirmative action) that makes it to the Supreme Court, such as those involving the interpretation of a constitutional provision or amendment; (b) the case has broad and ongoing implications for educational policy in postsecondary admissions and other areas such as outreach, student support programs, and diversity-related policies (e.g., Garces & Jayakumar, 2015); and (c) a broad range and number of amici (e.g., AFL-CIO, American Educational Research Association [AERA], APA, Cato Institute, the Center for Individual Rights, NAACP-LDF, social scientists) were involved seeking to influence educational policy through this process.
We concentrate on Fisher I as a discrete case for two reasons. First, Fisher I was briefed by amici as if it would be decided on the merits, which means that all the social science amici found relevant to the constitutional issues in the case at the time was included in the briefs. Said differently, when the court agreed to hear the case again in Fisher II, amicus briefs were prompted to change their focus as a function of the modified questions before the court. We also focus on Fisher I because many of the briefs filed in Fisher II are from the same amici who, despite focusing on new developments since Fisher I, also use similar (and sometimes the same) citations as they did in Fisher I, leading to inconsistent duplication of cited sources. The slightly changed focus of the cited evidence by amici in Fisher II and duplication of cited sources across both Fisher I and Fisher II would have confounded the analysis. From a practical perspective, Fisher I also has the advantage of involving more briefs than Fisher II. Focusing on Fisher I, we extend previous work (Horn & Marin, 2014; Marin, Horn, Miksch, Garces, & Yun, 2018) to address the following research questions:
In light of the participation by a large number and wide range of individuals and organizations as amici that cite social science research in their briefs in Fisher I, we also ask,
These questions allow us to focus on what research is being used by which educational policy actors and in which contexts in this process.
Conceptualizing and Framing the Ways Actors Shape Policy via the Courts
Plank and Boyd’s (1994) notion of the “antipolitics” of education provided the underlying conceptualization for this study. Different from more traditional policy-making avenues, the courts provide an “antipolitical” arena that does not require broad agreement from various constituents for policy enactment (Plank & Boyd, 1994). They argue that educational matters “have polarized American society” (1994, p. 266) and have led to “a search for institutions better able to address divisive questions” (p. 266) outside of “democratic governance” (p. 263). As such, Plank and Boyd do not use the term to suggest an absence of politics; instead, they are referring to a style of political action common to reformers from all parts of the political spectrum, in which the advocates of reform seek to spare themselves the rigors and uncertainties of interest mobilization and coalition-building by shifting consideration of key issues from legislatures and school boards to institutions that are less “political” and more authoritative, such as courts and markets. (p. 264)
In education, the courts have served in the critical role of authoritative reformer (Superfine, 2010) over many decades on issues such as affirmative action and segregation, taking action on the basis of law and without the need for a broad consensus for those decisions. With regard to the use of antipolitical approaches, it is imperative that these mechanisms be closely examined and understood so that, ultimately, they can be better leveraged to affect policy. A key entry point into one such process is via the submission of an amicus brief to the court (see Marin et al., 2018, for additional discussion of Plank and Boyd’s [1994] work on antipolitics as it relates to the use of amicus briefs).
To further inform this study’s research design, data analysis, and interpretation of findings, we also employ a framework from the use of research literature that outlines “the kinds of factors that seem to shape the use of research” providing a framework “for examining what we know about when and why research might get used” (Nutley et al., 2008, p. 61). According to Nutley et al., “these findings. . . generally hold true across a range of different contexts” (p. 67). In this study, we focus on (a) the nature of the research being applied (i.e., social science research and its characteristics), (b) the characteristics of potential research users (i.e., the amici using the research), and (c) the context for research use (i.e., the supporting party) as these are the main factors we can examine with the data we employ for this analysis (i.e., amicus briefs). Understanding the nature of research incorporated into briefs allows us to importantly focus on what is being used. In our analysis, we seek to broadly capture some of these factors by examining the characteristics of social science research, specifically the method of analysis, whether it is a peer-reviewed publication, and the geographic context. We also consider the categories of amici and the side of the case they support to better understand which policy actors are using what research in which contexts.
Amicus Curiae Briefs and Social Science
The current study situates itself within an intersecting research base that has examined amicus briefs from a variety of angles. For example, scholars have documented an increase in submission of amicus briefs over time (e.g., Hansford & Johnson, 2014). One study found that in 10 years, from 1946 to 1955, amici filed approximately 531 briefs. From 1986 to 1995, however, 4,907 briefs were filed (Kearney & Merrill, 2000). In addition, research studying amicus briefs submitted to the U.S. Supreme Court has focused on issues of influence, generating mixed conclusions as to their effectiveness (e.g., Caldeira & Wright, 1988, 1990; Collins, 2004, 2007, 2008; Larsen, 2014; Songer & Sheehan, 1993; Wofford, 2015) as well as strategies for making them effective (e.g., Ennis, 1984; Grisso & Saks, 1991; Lynch, 2004; Melton, 1987; Morgan & Pullin, 2010; Zuber, Sommer, & Parent, 2015). For example, some studies find that these briefs have little or marginal influence (Collins, 2004; Songer & Sheehan, 1993; Welner, 2012). Others find they do have some leverage when they reinforce the arguments in the main parties’ briefs (Spriggs & Wahlbeck, 1997); they are filed by experienced litigators or by the solicitor general, who represents the United States before the Supreme Court (Kearney & Merrill, 2000); or when they are filed in support of the respondent (the party in the case defending the lower court’s decision that the Supreme Court is reviewing; Kearney & Merrill, 2000). This body of work documents a mixed influence of amicus briefs on judicial decision making, with specific factors that may enhance their effect.
Studies that examine the use of research by amici have focused on the amici themselves or on documenting the range of issues the research addresses. For example, research has examined why interest groups submit briefs, at what stage of the legal process they submit it, how much they participate, how often they are on the winning side, and how the heterogeneity and reputation of the interest groups influence the justices (e.g., Box-Steffensmeier et al., 2013; Caldeira & Wright, 1990; Collins, 2008; Segal, 1988). Other work discusses the research amici put forward in particular types of cases, including lesbian and gay civil rights cases (Bersoff & Ogden, 1991), the use of psychological research in sex discrimination cases (Fiske, Bersoff, Borgida, Deaux, & Heilman, 1991), or the social science data in cases on abortion, sex discrimination, and sexual harassment (Erickson & Simon, 1998).
Although some scholars have questioned the impact of social science research in court cases (e.g., Ryan, 2003; Williams & McDermott, 2014), amici nevertheless continue to file briefs in increasing numbers and with increasing amounts of cited social science research. This will likely beget greater usage, as uncontested social science research may provide an area of advantage. But, as scholars have suggested, the goals of these briefs go beyond simply influencing the decisions of the court (Collins, 2004; Simpson & Vasaly, 2015). Indeed, the increased use of amicus briefs suggests that amici are convinced of their broader value, which includes influencing public discourse on an issue, increasing an organization’s visibility, or satisfying current and attracting new members (Simpson & Vasaly, 2015).
Emerging research has begun to examine the types of extralegal sources, including social science research, that amici include in their briefs to support their arguments. In Fisher I, for example, amici cited to 20 unique types of extralegal sources in their briefs, including dictionaries, government reports, websites, and speeches (Marin et al., 2018). Among these 20 types, academic nonlaw journals (e.g., Educational Policy) represented the largest single proportion of extralegal sources submitted by amici in Fisher I (28% of total extralegal sources). Four additional types, law journal articles, nongovernment reports, books, and media, each accounted for more than 10% of the total distribution. The citations to academic nonlaw journal articles—one of the main venues for social science research—is consistent with findings and conclusions from other studies that suggest the use of social science research in the legal arena to affect educational policy has become increasingly relevant. Moreover, of all the citations to extralegal sources in the briefs, citations to academic nonlaw journals were the single largest contributor in briefs submitted by individual scholars, faculty, and administrators; educational organizations; colleges, universities, and schools; and civil rights organizations and legal foundations (43%, 39%, 22%, and 19%, respectively).
To our knowledge, no other studies have gone beyond these considerations to understand the characteristics of the social science research that has been cited in amicus briefs. A more fine-grained understanding of these characteristics and whether they vary by supporting side of the case, or category of amici, can shed important light on our understanding of actors’ use of social science research to shape educational policy via the legal arena. This inquiry helps illuminate how we might enhance the use of social science research in legal decisions affecting educational policy. This knowledge is necessary to better identify the nature of research being used by policy actors so that future work can focus on the rationale behind that use as well as the policy impact of using social science research.
Definition of Social Science Research
The definition of social science research is critical for informing our study’s design. Although there is no universally agreed upon definition of social science research, we developed our definition by referencing broadly accepted understandings in the research use literature. Within this work, “social” science research includes “research aimed at understanding the social world, as well as the interactions between this world and public policy” (Nutley et al., 2008, p. 21). Such research can be conducted across and within a wide range of disciplines (e.g., anthropology, economics, education, health, law, political science, social psychology, and sociology; Bastow, Dunleavy, & Tinkler, 2014) and can draw from numerous disciplinary perspectives. We focus on research that is social because, as Nutley et al. explain, social research “is often contentious, may be reliant on controversial theories, draws on multiple and sometimes disputed methods, and may lead to contestable and at times ambiguous findings” (2008, p. 21). This “complex and contested nature of social research thus poses particular challenges” worthy of investigation for how those who use social science interpret what findings constitute evidence or knowledge (Nutley et al., 2008, p. 21). In addition, as noted earlier, previous work has already documented its prevalence among the evidence cited by amici in Fisher I (Marin et al., 2018).
Our definition of what constitutes “research” is intentionally comprehensive to make sure we do not preclude definitions used by amici. This choice is based on previous studies that “suggest that policy makers and practitioners have broader definitions” of evidence and research (Tseng, 2012, p. 6). We, therefore, follow other scholars in the research use literature (Nutley et al., 2008) and include work that involves “any investigation towards increasing the sum of knowledge based on planned and systematic enquiry [sic]” (Nutley et al., 2008, p. 21). In the current study, social science research includes “any systematic process of critical investigation and evaluation, theory building, data collection, analysis and codification relevant to the social world” (Nutley et al., 2008, p. 21). To further delineate this definition for our analysis, we also draw on the submission guidelines and practice of amicus curiae briefs. Specifically, we focus on sources listed under “Other Authorities” within the “Table of Authorities” for the briefs, rather than sources listed under “Cases” or “Statutory Law.” These lists include the extralegal sources cited in the briefs. Considering this broad definition of social science research and utilizing the list of “Other Authorities,” we reviewed the 20 unique types of extralegal sources cited in Fisher I briefs (Marin et al., 2018) and identified six categories most likely to include social science research: government reports, nongovernment reports, law journal articles, academic nonlaw journal articles, books, and book chapters. We, therefore, group these types of sources together to represent this broader, inclusive construct of social science for our analysis (for analysis by individual extralegal sources, see Marin et al., 2018).
Method
Because the court agrees to hear very few cases overall, we study Fisher I as an example of a U.S. Supreme Court case that involved a wide range of policy actors (i.e., amici) attempting to influence a highly contested area of educational policy. As “amicus briefs are aimed at specific cases and issues before the Court” (Collins, 2004, p. 813), it is appropriate to focus on one particular case. This is a methodological standard often employed in the legal arena (see, for example, Bersoff, 1987; Denno, 2006; Fiske et al., 1991; Hoewe & Zeldes, 2012).
Data Source: Amicus Briefs
Our data source includes the 92 amicus briefs that were submitted to address the merits of the case in Fisher I. After reviewing each brief’s list of “Other Authorities,” which includes a list of items cited that are not case law or legal authorities, and the body of text in each brief, we found that 91 of the 92 briefs (99%) cited social science evidence: 16- briefs in support of Abigail Fisher, 73 briefs in support of UT Austin, and two briefs in support of neither side. The brief of the Mountain States Legal Foundation, submitted in support of Abigail Fisher, does not cite any social science evidence and, therefore, we do not include it in our data set. This large share of briefs citing social science evidence is consistent with research discussed earlier that documents the increasing use of social science evidence in briefs.
In a prior study, we categorized the extralegal sources cited in Fisher I briefs and listed in the “Other Authorities” section of the “Table of Authorities” into 20 unique format types (see Marin et al. (2018) for a description of the complete data set). In the present study, we focus on six of the 20 types included in the list of “Other Authorities” that are the most likely to include social science research: government reports (n = 109), nongovernment reports (n = 294), law journal articles (n = 312), academic nonlaw journal articles (n = 649), books (n = 253), and book chapters (n = 90). This represents a total of 1,707 unique citations and includes all the retrievable social science citations (98%) in the 91 briefs (see Marin et al. (2018) for a full description of the data). The remaining 2% of social science citations are for items that could not be retrieved and, therefore, were not coded for this analysis. The selected categories also represent six of the top seven types of sources by frequency, underscoring the importance of understanding their characteristics in more detail.
What Are the Characteristics (i.e., Method of Analysis, Peer-Reviewed Publication, and Geographic Context) of the Social Science Research That Amici Cited in Fisher I?
To answer the first research question, we coded the cited extralegal social science sources in a series of iterative steps. As these sources had already been sorted into one of the six format types (Marin et al., 2018), we began by developing categories that identified characteristics of the different formats. First, we considered the research design, which consists of the plan the researcher follows in conducting the study, specifically the method of analysis. For this category, we relied on the broad methodological frames established by the scholarly cannon as widely used. In particular, we employed a research classification framework described by McMillan (2012) that captured the features of the two major paradigms in educational research—qualitative and quantitative inquiry (the latter we subdivided into experimental and nonexperimental)—as well as analytic and mixed/multiple methods (the latter was guided by how author[s] of the cited works identify their study). See Table 1 for a summary of the methods of analysis codes with descriptions for each. We did not include categories of “Basic,” “Applied,” “Action,” or “Evaluation” Research because these are intended to describe the goal of the research, as opposed to method of analysis.
Methods of Analysis Codes (Alphabetically Listed).
Source. McMillan (2012).
In addition to method of analysis, we considered the publication source and whether that source is peer reviewed. Feuer, Towne, and Shavelson (2002) indicated, Peer review is the single most commonly used mechanism for nurturing a scientific culture. . . Ideally, peer review is both a process by which scientific work is assessed and funded and a product that provides a venue for the scientific culture of self-regulation. (p. 10)
Guided by this understanding, we adapted our definition of “peer review” to be responsive to the various processes and mechanisms used to regulate and assess research in publication venues used across the disciplines included in the study. It is the norm for law journal publications, for example, to be “selected and edited largely by students with some assistance from faculty” (McCormack, 2009, p. 61), whereas education journal publications are traditionally reviewed by other scholars. We considered work that has undergone either of these processes to have been “peer reviewed,” following disciplinary norms versus placing a disciplinary hierarchy on peer review. As such, our definition of peer review is necessarily more expansive than a definition that may be used within any one field. For this study, we determined whether work was peer reviewed based on the information, if any, provided by Cabell’s journal directory or the publication source itself.
We also included geographic context as a category because of the nature of the legal issue in Fisher I, which involved considerations of a race-conscious policy within a specific geographical context (Texas) that was relevant to the constitutional question in the case. This question required the court to determine whether UT Austin’s policy met the legal requirements of strict scrutiny. In the context of the case, this involved facts specific to UT Austin and Texas, such as the effectiveness of the Top Ten Percent Plan in generating racial and ethnic diversity at UT Austin. For these reasons, the geographical codes included a comprehensive range of contexts to classify the central or primary focus and/or analysis of the extralegal source. For example, we defined national studies as those with data aggregated at the national level. Single-state studies are those that examine only one state, while multistate studies include several states. Because of the lawsuit’s focus on an institution in Texas, we created a separate code for studies focused solely on Texas. In addition, we created separate codes for studies focused on corporations and those focused on institutions (related to one university or system or one K-12 school or district), and also had a code for international/global studies in which all of the contexts were located outside of the United States.
In addition to applying a priori codes within categories informed by the literature (i.e., method of analysis, peer review, and geographic context), we were also open to identifying additional categories and codes during the analysis. Each category of codes also included options to indicate “other,” “none,” or “I don’t know.” Ultimately, the identified codes within each category allowed us to gain a descriptive understanding of the nature of the social science research cited.
Teams of three researchers (teams of two for books) coded the cited extralegal sources. Team members were assigned items within a particular format type (e.g., government reports, law journal articles, etc.) for individual coding. Once all members of a team completed their coding separately, the coding results of all members were sent to the team for review. Teams were required to compare the coding results across team members for consistency, as well as to discuss any “I don’t know” or “none” codes. In instances where there was a discrepancy across the coding by team members, the team reviewed the item in question along with code definitions and came to an agreement on the final code. As per Bradley, Curry, and Devers (2007), “the result is a single, agreed upon application of the final codes” (p. 1764) and so inter-rater reliability does not apply. In rare instances when a team could not agree, a member of another coding team was consulted to determine final codes. When the coding of a format type was complete, we reviewed all of the remaining “other” and “none” codes. Because coders were required to include notes if they used the “other” or “none” codes, we were able to identify new codes, as warranted.
To What Extent Do the Characteristics of the Social Science Research Vary (a) by Categories of Amici and (b) by Supporting Side of the Case?
To understand to what extent the nature, or characteristics, of the social science research cited vary by supporting party or category of amici, we analyzed results in two different ways: (1) by supporting side in the case: (a) in support of Fisher (the petitioner), (b) in support of UT Austin (the respondent), or (c) in support of neither party; and (2) by category of amici (Marin et al. (2018). To determine the amici categories, we were guided by the characteristics suggested by the extant literature (Anderson, 2015; Simpson & Vasaly, 2015). Specifically, we reviewed the “Interest of Amici Curiae” section in each brief, which includes background information about the amici and their specific interest in the case, and used that information, to place each brief into one of the nine categories. Using the interest of amici section allowed us to consider the entire group of individuals/organizations involved in the brief, as numerous amici can be part of a brief. When briefs involved a range of amici, we chose the category based on the lead amicus (i.e., the individual/organization listed first in the title of the brief).
These amici categories included briefs submitted by (a) individual scholars, faculty, and administrators; (b) colleges, universities, and schools; (c) individuals with current or former government affiliations; (d) the U.S. government or state governments (i.e., governments); (e) individuals and alumni who are not faculty or scholars affiliated with a university (i.e., individuals and alumni); (f) civil rights organizations and legal foundations; (g) bar and law associations; (h) business groups and trade associations; and (i) educational organizations.
Limitations
This analysis provides a critical foundation for understanding policy actors’ use of social science research to influence educational policy through the legal arena. An important limitation is that the present study restricts the definition of “use” to citation and does not consider the manner in which the item is used within the brief. In other words, for this study we did not review the item within the text of each brief to determine its use related to the argument being made or even if the description of the study was accurate. Given the nature of the data, we also do not consider the perspective of the user (i.e., the writers of the brief)—outside of what we can infer from the category of amici—or the characteristics that stood out for them as they decided to cite the item. Also, because this is limited to a document analysis, we do not know which sources were considered but not included or why. These are important considerations that would provide a more nuanced understanding of the use of social science research to shape educational policy through the legal arena and will be discussed further in our section “Conclusions and Next Steps.”
Findings
What Are the Characteristics (i.e., Method of Analysis, Peer-Reviewed Publication, and Geographic Context) of the Social Science Research That Was Used by Amici in Fisher I?
In answering our first research question, with respect to method of analysis, 41% of the unique social science research cited in amicus curiae briefs are quantitative nonexperimental in nature, followed by more than one third (35%) that are analytic. Just over one in 10 studies (12%) utilized multiple/mixed methodology and 8% employed quantitative experimental designs. Only 2% of the cited studies used qualitative methods. In considering the relative presence of peer-reviewed publications, two categories—academic nonlegal and legal journals—had articulated disciplinary standards and classifications. The analyses identified that 99% of social science studies published in either of these venues were classified as peer reviewed. As this is a document analysis, we could not confirm whether the cited social science studies published in other formats (i.e., governmental and nongovernmental reports, books, and book chapters) were peer reviewed; therefore, we excluded peer review as a consideration in all subsequent analyses. With respect to geographic context, more than half (54%) of the social science research cited in amicus briefs has a national focus. Institutional and multi-institutional geographic contexts followed in their representation at 11% and 10%, respectively. Only 5% of the studies cited by amici have Texas as a focus, despite the case itself being grounded in that context.
To What Extent Do the Characteristics of Social Science Research Vary by Supporting Side of the Case?
Table 2 represents the nonunique distribution of social science by methodology and supporting side of the case. Of particular note, among briefs submitted on behalf of Fisher, almost a majority (49%) of cited works are analytic in nature and nearly one third (30%) are quantitative nonexperimental. Among briefs submitted in support of UT Austin, 33% of cited studies used analytical approaches and 41% applied quantitative nonexperimental techniques. Fifty-three percent of the social science cited by the two briefs in support of neither side are also quantitative nonexperimental, with only 9% of cited works categorized as analytic or quantitative experimental. However, in these same briefs 24% of the cited works use multiple/mixed methods. Examining the geographic contexts represented by the cited social science research (Table 3), when considering briefs by supporting party, all three groups of amicus briefs have a majority of works with a national context (52%, 54%, and 62%, respectively). To varying degrees across supporting party focus, that was distantly followed by multi-institutional and institutional contexts.
Proportions of Social Science Research by Method of Analysis a and by Supporting Party.
Percentages do not total to 100 due to rounding and to the presence, in some circumstances, of cited work that did not apply a coded methodological approach (coded as “none”).
Proportions of Social Science Research by Geographic Context a and by Supporting Party.
Percentages do not total to 100 due to rounding and to the presence, in some circumstances, of cited work that did not apply a coded methodological approach (coded as “none”).
To What Extent Do the Characteristics of the Social Science Research Vary by Categories of Amici?
Table 4 presents the proportions of research by methodology and amicus category. For five of the nine categories of amici (i.e., individual scholars, faculty, and administrators; colleges, universities, and schools; individuals with current or former government affiliations; business groups and trade associations; and educational organizations), quantitative nonexperimental methodology represented the greatest proportion of social science cited. For four categories of amici (i.e., governments; individuals and alumni; civil rights organizations and legal foundations; and bar and law associations), a majority of cited social science applied an analytic methodology. In addition, for seven of the nine categories of amici, multiple/mixed methods studies represented at least 10% of their cited social science research. In addition, while most categories of amici cited quantitative experimental studies sparingly (representing 5% or less of the total distribution), the share of experimental studies cited by both individual scholars, faculty, and administrators and educational organizations was far greater (18% and 10%, respectively). With respect to geographic context (Table 5), as was the case overall, the predominant focus for the work cited across all categories of amici is national, ranging from 37% (individuals and alumni) to 72% (individuals with current or former government affiliations and bar and law associations). There were, however, interesting differences. For example, an additional 37% of the social science cited by individuals and alumni has a Texas focus. By comparison, the same geographic context captured only 1% of the research cited by individuals with current or former government affiliations and bar and law associations.
Proportions of Social Science Research by Method of Analysis a and by Amicus Category.
Percentages do not necessarily total to 100 due to rounding and to the presence, in some circumstances, of cited work that did not apply a coded methodological approach (coded as “none”).
Proportions of Social Science Research by Geographic Context and by Amicus Category a .
Percentages do not total to 100 due to rounding and to the presence, in some circumstances, of cited work that did not apply a coded methodological approach (coded as “none”).
Understanding Implications for the Briefing Process, Educational Policy, and Future Research
As Plank and Boyd (1994) identified, the polarizing nature of key education issues has increasingly driven policy discussions into antipolitical venues. Our study focused on understanding patterns of social science research used by policy actors engaged in an antipolitical process, that is, an important venue to seek to guide educational policy: the courts. Within this arena, we focus on the briefing stage, a crucial point in the legal process as information that is presented in legal briefs can ultimately inform and influence the court’s deliberations on issues that have broad implications for educational policy. Our findings contribute to research broadly focused on the use of research by amici and an emerging body of work that examines the types of extralegal sources presented in amicus briefs. Furthermore, this study suggests ways to expand future research use literature by identifying needed work to answer questions regarding the often perceived “weak link between research evidence and policymaking” (see, for example, Lubienski, Scott, & DeBray, 2014, p. 131), pointing to important areas for additional investigation in efforts to improve these links.
Method of Analysis, Geographic Context, and Their Intersections
In this study, we sought to specifically understand the social science research included in amicus briefs filed in Fisher I using a framework offered by Nutley et al. (2008). First, we considered the nature of the research. This distribution of methods is interesting in that it appears to heavily favor quantitative nonexperimental analyses (41%); however, only 8% of the cited studies had an experimental design. This may be indicative of the relatively small number of experimental designs in social science research on the topics being argued in the briefs, a lack of understanding of those studies that do exist, limited access to such studies, or the perceived lack of their usefulness in front of the court. In addition, the relatively tiny share of qualitative studies cited (2%) compared with the preponderance of quantitative studies (49% combining experimental and nonexperimental studies), may suggest some of the same issues. The relative presence of multiple/mixed methodology studies cited (12%) advances the idea, however, that there may be room for researchers from varied traditions to partner to develop these types of studies. However, in discussing effective amicus practice, Simpson and Vasaly (2015) suggested “the amicus should cite the most reputable sources available, such as authoritative academic works” (p. 71). In following such guidelines, are these amici citing what they believe to be “the most reputable sources?” And is “reputable” determined by the extralegal source, the methodology, or the status of the author? Or is the key word in this instance “most reputable sources available”? Further exploration is required to understand the observed differences of the use of social science research from distinct methodological traditions as it ultimately raises questions as to whether these results are due to supply or demand (see Nutley et al., 2008), with “supply” referring to the possibility of studies that either do not exist or studies that cannot be accessed by amici.
With respect to geographic context, a majority of cited social science in amicus briefs in Fisher I have a national frame. This focus on national studies is interesting given the relatively larger weight the courts have often given to local data for specific court cases. However, this finding aligns with Simpson and Vasaly (2015) who suggested that the court “is often concerned about the unrecognized ramifications of its opinion” (p. 51). With this in mind, individuals and organizations that are not the direct parties in the case but have an interest in the outcome on a broader level would include social science research that addresses the national context so as to help inform the court about the implications of its decision for higher education and other institutions across the country. Another strategy in legal cases involves seeking to convince a justice who may not agree with your position to at least consider a narrower ruling because of the negative national consequences that a broader ruling may have. This approach by amici reflects the diversity of goals that these briefs can have (Simpson & Vasaly, 2015) and suggests that there are many ways to influence the courts and different types of research contexts that can be more effective for some efforts than others.
Together, these results suggest that social scientists and educational policy makers may need to think of their engagement with the courts through amicus briefs not only through the research they cite, but also through the development of a validity hierarchy developed with the framework of the court in mind. The results related to methodology and geographic context lay a necessary foundation to explore critical questions that examine underlying reasons for these differences in efforts to address Weiss’s (1979) call to “learn how to make [social sciences’] contribution more effective in the future” (p. 427). For example, local experimental designs may be especially effective in swaying the court about the specifics of a particular case because of their strong internal validity and the strong causal inferences that can be made about that context. However, when it comes to understanding the impact of a particular question on the nation at large, experimental designs lack the external validity necessary to suggest the local findings might apply more broadly and to different contexts. In addition, qualitative designs may be used to substantiate the causal stories (Stone, 1984, 2011) created in amicus briefs to explain why a particular action will lead to a specific outcome.
Amici by Supporting Side and by Amici Category
Prior work has found that the influence of amicus briefs can depend on the supporting side, such as those filed by respondents, or on the category of amici, such as those filed by the solicitor general who represents the federal government (Kearney & Merrill, 2000). It remains an open question as to whether the type of research filed by these parties may explain why these particular groups may have more influence with the courts; however, our analysis of the characteristics of the social science research that was cited by amici by supporting party provide some insights and revealed moderate differences in patterns of use in the method of analysis (Table 2), as well as geographic context (Table 3). For example, for sources cited in briefs filed in support of Fisher, UT Austin, and neither party, the vast majority applied either an analytic, quantitative nonexperimental, or multiple/mixed methodological approach (in combination, 89%, 85%, and 86%, respectively). In addition, a slightly greater percentage of studies submitted in briefs in support of UT Austin (the respondent) as well as in briefs in support of neither party addressed the Texas context (6% each) compared with those submitted in support of Abigail Fisher (the petitioner; 3%). This slightly larger percentage is not necessarily surprising given the legal issues in the case, which require UT Austin to defend the constitutionality of its policy by demonstrating that the institution considered context-specific approaches (i.e., the Top Ten Percent Plan) for attaining racial and ethnic diversity in the student body. This also suggests that the extralegal sources cited may be tied directly to the argument an amicus is trying to advance. Said differently, all extralegal sources do not have the same utility to amici which may not be readily apparent to the scholars producing the work. Building on our findings, connecting these differences to the relative influence of the particular briefs is clearly a direction this work could move, extending the work of Kearney and Merrill (2000), and yielding insights identifying additional research needed by amici.
The differences by category of amici reveal nuanced differences. With respect to method of analysis (Table 4), three categories of amici stand out as not including any research that employed qualitative methods of analysis: individuals with current or former government affiliations, individuals and alumni, and civil rights organizations and legal foundations. In addition, only the amici categories of individual scholars, faculty, and administrators and educational organizations relied with relative weight on quantitative experimental studies, often considered the “gold standard” in social science. The focus, or lack thereof, of particular methodologies used in cited studies suggests several possible explanations: they are perceived to be more (or less) important by these groups compared with others; they are relatively more (or less) readily understood and, therefore, able to be used by this category of amici; or that this category of amici simply has more (or less) access to this work. However, other categories (e.g., colleges and universities) would have at least equal, if not greater, access and understanding of these types of sources so this begs deeper consideration. Again, this has implications for the types of studies conducted by researchers, the degree to which they are thought to have relevance by various interested groups, and possibly issues of dissemination/accessibility.
The lack of citation to certain types of studies may also, as noted above, reflect the focus of amicus briefs; that is, these briefs are most effective when they supplement, rather than duplicate, the information or arguments provided in the main parties’ briefs (Simpson & Vasaly, 2015). It could also be that these amici did not believe that certain types of studies provided the support needed for their arguments. Some of these reasons, however, are not likely to hold true for amici in the colleges and universities category that certainly had access to social science research as well as individuals able to interpret it. These important findings point to the need to better understand the dynamic relationship between those citing the studies and the producers of the research, as well as the legal context, background, and interest of the specific amici in the case.
Differences in geographic context (Table 5) across amici categories track those we found across all the briefs: a predominant focus on studies that addressed a national context. It is not surprising, for example, that the largest percentage of those that included national-context studies involved those filed by individuals with current or former government affiliations (72%). These affiliations, for example, included briefs of U.S. Senators, members of Congress, and current and former civil rights officials. As we mentioned, given the legal issues in the case, we expected a potential focus of the social science research cited to include those focused in the Texas context. Here, the largest percentage of those (37%) was included in amicus briefs filed by individuals and alumni. These groups included alumni of UT Austin and Texas-based associations, such as the Council for Minority Affairs at the University of Houston, the Society of Hispanic Professional Engineers at the University of Texas at Austin, the University Leadership Institute, and Texas College Students for Diversity. Amici in this category, moreover, were all in support of UT Austin, the respondent in the case, which, as noted, was placed in the position of justifying the educational policy being challenged. In general, then, the social science research cited by amici aligns with and supports their arguments in context and toward the common goal of influencing the court.
In total, then, findings from our study suggest that the choice of which social science research to cite within a brief is not solely a function of the type of evidence available. The perspective of the amici, or the lawyer serving as Counsel of Record for the amici, may play a role in deciding what extralegal sources are cited. Is this a function of the amici’s orientation or culture or the focus of their briefs? Could there be more and different types of research cited if the social science research incorporated analytic frameworks that directly addressed legal questions at hand for many of these court cases (e.g., Garces, 2013)? Are there other factors or criteria of relevance to the amici that differ from those of social scientists? The information gained from these analyses provide insights into understanding these choices and whether the use of social science research was related to the demand side (amici deciding what to cite to support their arguments), the supply side (researchers’ choices and decisions when producing the scholarship, as well as research accessibility and interpretability), or a combination and dynamic relationship between the two.
Conclusions and Next Steps
Findings from this study reinforce and expand existing literature and provide a significant foundation for understanding the efforts of actors to shape educational policy via the highest court in the nation. In particular, we discovered differences in the use of social science research to support arguments at an important stage in the legal arena: brief development. These differences suggest many ways in which our current understanding is incomplete and why developing this understanding could be extremely useful to both the creation of educational policy and the use of antipolitical approaches to change such policy. Our findings indicate differences of research use among categories of amici that are potentially important to both the production and use of research. This has relevance for researchers and those using research—both of whom should be interested in improving the use of social science research in this antipolitical process. To address these differences, one option for the producers of extralegal sources would be to partner with those amici who might be more inclined to cite social science. Another option would be to more broadly distribute extralegal sources to increase the possibility of its use where it may prove fruitful. For example, almost 60 media source citations in amicus briefs (not included in these analyses) functionally served as proxies for research by virtue of the information provided in the articles.
Building on our findings, our future research will further explore the use of social science research in amicus briefs via a survey of the producers of the extralegal sources cited in Fisher and interviews with Counsels of Record (lawyers representing amici) who may decide which studies to cite, or not to cite, in their briefs. This will provide greater clarity as to the rationale behind the choices being made and a better understanding of the circumstances that limit or promote the use of social science research in the legal context such as connections or networks among researchers and research users and research interpretability and accessibility. Other studies will also address the degree to which educational policy actors are specifically creating social science research following legal frameworks to support their arguments in language that is readily understood, and applicable, to the questions and grounds on which the court must apply their decisions. In addition, future work by authors and others can explore the impact of research use on the court and decision makers’ understanding of the specific benefits or limitations of various types of research to this end. Subsequent studies could also broaden the scope of the research by studying other cases and/or other doctrinal areas. These next steps will be important to help facilitate the influence of social science research in the legal arena—an area that has important consequences for anyone working in policy.
Footnotes
Acknowledgements
We thank our graduate research assistants for their tremendous assistance with this research project and our Advisory Board for their insights and guidance.
Author’s Note
Liliana M. Garces is now affiliated with University of Texas at Austin, Austin Texas.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors gratefully acknowledge the support of the William T. Grant Foundation. The findings represent the perspectives of the authors alone.
