Quality Indicators for Mixed-Methods Research in Special Education

Abstract

Mixed-methods research (MMR) holds promise for investigating several empirical questions within special education, capitalizing on the strengths of quantitative and qualitative traditions. We present an overview of MMR in special education and quality indicators for conducting and reporting such studies. We look to the future for how MMR may advance our understanding of crucial issues related to diversity, equity, and inclusion. Finally, we discuss special considerations for MMR and the open science movement.

Mixed-methods research (MMR) is an approach to inquiry that has gained prominence since Exceptional Children's publication of quality indicators of special education research (Odom et al., 2005). Scholarly interest in MMR has resulted in the publication of three handbooks on the topic (i.e., Hesse-Biber & Johnson, 2015; Hitchcock & Onwuegbuzie, in press; Tashakkori & Teddlie, 2010), the 2007 launch of the journals Journal of Mixed Methods Research and International Journal of Multiple Research Approaches, and several highly cited texts (e.g., Creswell & Plano Clark, 2018; Greene, 2007; Teddlie & Tashakkori, 2009).

Despite the increased attention toward and use of MMR writ large, its uptake within the field of special education has yet to be well established in premier special education research outlets.

Onwuegbuzie and Corrigan (2018) conducted a meta–prevalence study in Exceptional Children from 2000 to 2015 and found that mixed-methods studies were published less often than quantitative or qualitative studies. Specially, 14.53% of Exceptional Children empirical studies used MMR, 17.73% used predominantly qualitative inquiry, and 67.73% of the articles described quantitative research. More recently, Corr et al. (2021) found that only 0.63% of published studies from 2007 to 2019 in 15 top-ranked special education journals used MMR. Moreover, the authors found that MMR was used for a limited number of research purposes, with most research integrating methods for the purpose of strengthening interpretations as opposed to other purposes, such as investigating potentially divergent findings or intersecting phenomena. Of particular concern, Corr et al. (2021) found that a significant portion of published special education MMR did not follow established reporting guidelines, which made it difficult to judge research quality or assess the defensibility of researcher decisions.

Conroy et al. (2022) conducted a focused review of research published in two top-tier emotional-and-behavior-disorder journals between 2000 and 2020 and found only three MMR studies. Like Corr et al. (2021), the authors found that it was difficult to judge the quality of the published MMR. Finally, Marsh et al. (2022) examined the extent to which MMR was used in juvenile justice research involving students with disabilities. The authors found zero published MMR studies between 2000 and 2019. Collectively, literature reviews suggest that MMR designs are underutilized in special education research, and the MMR that has been published may lack rigor or fail to meet reporting standards.

The dearth of published MMR studies in special education is perhaps not surprising for several reasons. Compared with monomethod designs (i.e., quantitative, qualitative), MMR is nascent and presently evolving. Perhaps this explains why MMR was not included in the 2005 Exceptional Children special issue on quality indicators. But time, alone, does not fully explain the present reality. Although MMR does not require a single person to be an expert in quantitative, qualitative, and MMR, it does require collective knowledge across these methodologies as they are applied to a specific research study. Special education's roots in medical models and behaviorism (Brownell et al., 2010) has resulted in researchers leveraging quantitative methods more often than qualitative or MMR (Corr et al., in press; Klingner & Boardman, 2011). This, in turn, has limited opportunities special education doctoral students have to access training in MMR. In an MMR study of doctoral students enrolled in special education programs at research-intensive institutions, Corr et al. (in press) found roughly 84% of participants indicated they had taken more quantitative methodology courses than qualitative or MMR. From qualitative interviews, participants noted the preponderance of quantitative courses was due to lack of available courses in qualitative and MMR and advisors with limited expertise in these methods. The tradition of privileging quantitative designs over others has long-term implications for the field in terms of researchers’ ability to conduct MMR and reviewers’ and editors’ ability to accurately judge it and provide feedback. Moreover, tenure and promotion standards in higher education often reward large quantities of publications; thus, faculty may elect to separate the qualitative and quantitative components and publish two separate studies as opposed to one MMR study. To speak to these concerns, the purpose of this article is to discuss quality indicators of MMR and argue its use within special education may hold promise for addressing crucial, yet unresolved, issues.

Foundational Tenets of MMR

To start, consider a definition of MMR offered by Johnson et al. (2007):

Mixed methods research is the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration. (p. 123)

We draw attention to an important element of this definition. MMR involves the combination of quantitative and qualitative approaches. The word combination is crucial in that MMR necessitates purposeful and intentional “combining,” “mixing,” or “integrating” as opposed to parallel application of qualitative and quantitative approaches (Fetters & Freshwater, 2015). Collecting and analyzing both quantitative and qualitative data separately without ever integrating the two is typically referred to as multiple-methods research. Multiple-methods research can include multiple quantitative methods or multiple qualitative methods. Multiple-methods research can also entail a qualitative component and a quantitative component that are not integrated (Anguera et al., 2018). This brings up an important point that more is not always better. MMR, though beneficial for several reasons, is not necessary, or even appropriate, in all instances. A research question that can be answered using a single method will likely require fewer resources and will still produce important scientific knowledge for the field— mixing methods, in this instance, would be ill-advised. When conducting any research, it is essential that researchers are guided by their specific research purpose and research question(s) and that they employ methods that are appropriate for the types of research questions they endeavor to answer (Cook & Cook, 2016; Newman & Houchins, 2018).

Scholars have argued MMR constitutes a third methodological paradigm following quantitative and qualitative traditions (Johnson & Onwuegbuzie, 2004). Many mixed-methods researchers adopt pragmatism as an underlying philosophical orientation (Tashakkori & Teddlie, 2010), which espouses pluralism (Johnson & Onwuegbuzie, 2004), practicality, and applied research that attempts to uncover “what works” in real-world practice (Creswell & Plano Clark, 2018, p. 36). Pragmatist mixed-methods researchers reject the idea that conflicting philosophical worldviews (see, e.g., Onwuegbuzie, 2012) associated with qualitative and quantitative methods render their combination untenable. Instead, they view MMR as a way to capitalize on the strengths of quantitative and qualitative methods to answer research questions that would be otherwise difficult when using one approach alone. In doing so, mixed-methods researchers can design investigations that fulfill multiple empirical purposes (see Supplemental Table 1) using core designs (See Table 1).

Table 1.

Example Mixed Methods Research Studies by Core Design.

Purpose	Description	Example published research	Additional example
Convergent	Results of the qualitative and quantitative data analyses are combined, often with the intent to compare them.	Scanlon et al. (2018) examined the effectiveness of alternative IEP dispute resolution practices. Various stakeholder participants completed surveys about how the teams addressed disagreements before participating in the dispute resolution program, their perspectives on the program, and how their knowledge and experiences changed after their participation. Survey items included closed-ended questions that were statistically analyzed and open-ended questions that were analyzed using a qualitative content coding analysis process. Research purpose/question(s): “. . . determine if the parties in past SpedEx cases are satisfied and whether there has been a positive impact on their subsequent team interactions” (p. 71)	Study of special educators’ use of AT. Participants complete a survey and individual interviews so researchers can better understand the issue and determine ways the data sets converge or diverge. Survey and interviews are first analyzed separately, and then findings are compared. Sample research question(s): What is the relationship between special educators’ use of AT and their perceptions of the facilitators and barriers towards using AT?
Explanatory sequential	Quantitative data collection and analysis are implemented first, followed by qualitative data collection and analysis. Qualitative data are used to explain the quantitative findings.	Garwood et al. (2018) explored the relationship between rural special educators’ classroom management and role stressors and their feelings of burnout and contributors to burnout. Researchers first administered an online survey, then conducted follow-up focus groups. Quantitative analyses of the survey informed development of focus group protocols, which allowed researchers to examine contributors to role stressors, classroom management struggles, and teaching satisfaction. Research purpose/question(s): “(1) What is the relationship between rural special education teachers’ self-reported classroom management efficacy and role stressors to their feelings of burnout? (2) To what factors do rural special education teachers attribute their feelings of burnout? (3) What practices do rural special education teachers find successful in preventing burnout?” (p. 32)	Study of students with extensive support needs during literacy learning. Student participants first complete a literacy assessment. Then researchers complete qualitative classroom observations to contextualize the learning process and identify supports that contributed to students’ assessment scores. Sample research question(s): What differences in learning processes and available supports exist between students with high and low scores on a literacy assessment?
Exploratory sequential	Qualitative data are collected and analyzed first, followed by quantitative data collection and analysis that further examines the same or a related phenomenon. Qualitative data may be used to develop and test a quantitative feature, such as an intervention or tool.	Unwin et al. (2021) explored practitioners’ perspectives of and experiences using sensory rooms to support children with ASD, a practice with limited evidence despite its wide use. First, practitioners completed interviews about their use of sensory rooms, perceived outcomes, and potential drawbacks. On the basis of a thematic analysis, researchers identified five themes. These themes were then used to develop a survey that was administrated to a wider group of participants. The survey further examined practitioner beliefs about the use of sensory rooms and included questions that were statistically analyzed to probe findings from interviews, such as the impact of prior experience using sensory rooms on their beliefs. Research purpose/question(s): “(1) What are the beliefs and experiences of educational practitioners about the possible outcomes and overall efficacy of using MSEs with autistic children? (2) What factors do practitioners believe may affect possible outcomes in the MSEs for autistic children?” (p. 2)	Study examining how practitioners can engage in early-intervention home visits that are responsive to the unique strengths, priorities, and concerns of families of color. First, researchers observe home-visiting sessions and interview families. Then, they develop a PD program for home visitors. They administer a pre-/postintervention survey to interventionists and the families they work with to understand the feasibility, social validity, and initial effectiveness of the PD at improving cultural responsiveness and family–professional partnerships. Sample research question(s): (1) How do families of color support the fine and gross motor skill development of their young children with disabilities? (2) To what extent do early interventionists and families find the PD feasible, useful, and effective in supporting the fine and gross motor skill development of young children with disabilities?

Note. Draws from Creswell and Plano Clark (2018). ASD = autism spectrum disorder; AT = assistive technology; IEP = individualized education program; MSE = multisensory environment; PD = professional development.

Quality Indicators for MMR

In this section, we present a four-part framework of quality indicators for MMR for consideration by special education researchers and reviewers. The framework draws on foundational work around quality within the broader MMR community in the behavioral, social, and health sciences (e.g., Fàbregues & Molina-Azorín, 2017; Heyvaert et al., 2013; O’Cathain, 2010) and the American Psychological Association’s (2020) Journal Article Reporting Standards (JARS). In our view, quality in MMR can be conceptualized in terms of (a) integration, (b) synergy, (c) strong component designs, and (d) legitimation, as described in the following sections.

Meaningful and Purposeful Integration

The integration of quantitative and qualitative research approaches in service of the research purpose distinguishes MMR from multiple-methods research and is how researchers can capitalize on the unique strengths of MMR (Fetters & Freshwater, 2015). Integration may be employed at any point throughout the research process, including through the overall research design (e.g., the core mixed-methods design), sampling, data collection, data analysis, and data interpretation (Fetters & Molina-Azorín, 2017). Researchers should specify how they integrated quantitative and qualitative research approaches and how integration served the research purpose. This may result in separate research questions for the component designs and a mixed-methods-specific question or fully integrated MMR question (Plano Clark & Ivankova, 2016). Example MMR research questions are presented in Table 1. Here, we discuss several common integration processes that support rigorous MMR, with a summary provided in Supplementary Table 2.

Integration during design planning and sampling

Fetters and Molina-Azorín (2017) identified multiple dimensions of integration that may occur during the design-planning and sampling stages. For instance, researchers may integrate philosophical assumptions (e.g., epistemologies) to inform decisions about the types of data that will be collected as well as sampling procedures (e.g., variants of random or purposive sampling, combination sampling procedures). Researchers should also develop mixed-methods-specific aims and questions, leading to integrated research questions and hypotheses that dictate the planned relationship between quantitative and qualitative approaches.

Moreover, given the contributions of a strong theoretical framework for qualitative research (Kozleski, 2017), it is equally important that MMR has philosophical clarity and a strong theoretical foundation (Collins et al., 2012) from the outset. A coherent theoretical or conceptual framework can inform qualitative data collection and analysis as well as contribute to systematic integration of qualitative and quantitative data or findings. For example, to ensure the multiple types of data build on each other within a study, a shared theoretical or conceptual framing can inform measure selection (Cole & Cawthon, 2015), protocol or code development (Love & Corr, 2022), and decisions about what quantitative variables to analyze and how to do so (Cross et al., 2020). A joint data display (see samples in Supplemental Tables 3 and 4) may also be organized around the study's theoretical or conceptual framework to facilitate integrated analysis (Guetterman et al., 2015; Love & Corr, 2022). Finally, theory can inform decision-making when quantitative and qualitative findings diverge (Fetters & Molina-Azorín, 2017), enabling researchers to make meaning of said divergence based on previously identified conceptual relationships that can be applied or empirically tested.

Integration during data collection and analysis

Integration during data collection and analysis generally involves collecting multiple types of data to build or expand on each other, construct a case, match or compare (e.g., along themes or constructs), or otherwise inform the analysis of each other (Fetters & Molina-Azorín, 2017). For instance, research with a development purpose should feature researchers analyzing one data set to inform the collection or analysis of another data set (Bazeley, 2012). Researchers may, for example, thematically analyze focus group transcripts to develop survey items. It should be clear how integration during data collection and analysis supports joint meaning making given the research purpose.

Data transformation is a common integration step (Nzabonimpa, 2018). Such transformation happens during analysis where qualitative data (e.g., text) are “quantitized” (Sandelowski et al., 2009) or when quantitative data (i.e., numeric) are “qualitized,” such as when they are represented as narrative (Onwuegbuzie & Leech, 2019). Quantitizing qualitative data may include counting codes or themes and using those counts for further quantitative analysis or using the presence or absence of a code as a variable in statistical analyses. Qualitative data are typically quantitized to limit bias, support generalizability claims, or identify statistical relationships, reflecting general goals and assumptions of quantitative research (Bazeley, 2012).

Qualitizing quantitative data might include developing codes based on quantitative data and then analyzing these codes or otherwise converting quantitative data into data such as narratives that can be analyzed qualitatively (Nzabonimpa, 2018). Notably, qualitizing quantitative data does not mean just turning numeric data into words; it reflects analyzing quantitative data according to the general goals and assumptions of qualitative research, such as using multiple sources of knowledge (e.g., theory, participant perspectives) to determine and conduct analyses, rather than attempting to control for bias (Love & Corr, 2022; Nzabonimpa, 2018). Qualitizing quantitative data can create contextually meaningful groups of participants or cases, support the identification of patterns, add descriptive meaning, enhance reliability, and support analysis that is responsive to a particular population (Onwuegbuzie & Leech, 2019).

Finally, joint data displays are almost a ubiquitous form of integration within MMR. Joint data displays visually combine multiple types of data to facilitate joint analysis (Fetters et al., 2013). The displays often take the form of a table, path diagram, or network diagram (Guetterman et al., 2015). Although joint data displays may be organized in many ways, the most common displays are organized according to statistical analyses, themes, theoretical or conceptual components, cases, research questions, or quantitative and qualitative findings (Guetterman et al., 2015). Additionally, joint display tables may match quantitized qualitative data (e.g., code counts) with statistical analyses or match qualitized quantitative data (e.g., assessment scores from groups that were determined based on qualitative findings) with qualitative findings (Guetterman et al., 2015). Although commonly used during analysis, joint data displays may also be used during interpretation and reporting (Fetters et al., 2013). Joint data displays support systematic integration by ensuring joint meaning making across multiple types of data and findings. Sample joint data display templates are presented in Supplemental Tables 3 and 4.

Integration during data interpretation and inferences

Integration may take place after all data have been collected as researchers make joint meaning or draw meta-inferences across research approaches (Fetters & Molina-Azorín, 2017; Newman & Houchins, 2018). Integration that is restricted to the final interpretation or reporting stage of the research project may be common in convergent MMR, including case studies, or research that has separate, but related, research questions for quantitative and qualitative data collection and analysis. Regardless of whether integration occurs during earlier research stages, researchers should demonstrate integration during final interpretation and conclusions because integration should be necessary to address the research purpose of an MMR study.

Using Component Designs in Synergistic Ways

The fundamental design principles of singular methodologies apply to MMR just as they do with other forms of inquiry. Consider the relative merits of different group designs, like randomized controlled trials (RCTs) and quasiexperiments. Each approach has its advantages and drawbacks. For example, RCTs tend to yield the capacity for causal inference that is better supported than that of quasiexperiments; but on the other hand, it can be more difficult to recruit study participants into an RCT relative to a quasiexperimental design. The point is there are design trade-offs. Researchers should establish that the best possible design was used given the research purpose and context at hand (Newman & Houchins, 2018).

This raises a key justification for considering MMR, or what Johnson and Turner (2003) refer to as the fundamental principle of mixed research. This principle dictates that researchers should use different strategies and collect multiple forms of data to yield “complementary strengths and nonoverlapping weakness” (Johnson & Onwuegbuzie, 2004, p. 18). Relatedly, MMR should be conducted with the goal of establishing synergy between its component parts (Fetters & Freshwater, 2015). Our endorsement of the word synergy is intentional yet tempered. Synergy means two or more things working together such that the total effect is greater than the sum of individual parts. It is what Fetters and Freshwater (2015) referred to as the 1 + 1 = 3 challenge in MMR. In MMR, synergy is about pursuing a more comprehensive understanding of phenomena that results when quantitative and qualitative methods are integrated and equally valued (Hall & Howard, 2008) whether they lead to a convergence or triangulation of findings or not. The harmonious connotation of “synergy” should not lead to researchers devaluing MMR that seeks or unexpectedly results in initiation or contradictory findings. Synergy is about researchers identifying and pursuing the “value added” of the MMR approach for their study and its ensuing conclusions (Fetters & Molina-Azorín, 2017). If researchers can establish an MMR design purpose that meets this fundamental principle, then a cardinal quality indicator is met. The reverse of this is also true.

The merits of MMR depend on clearly establishing why different design choices are used, in a way that addresses complementary strengths and nonoverlapping weakness.

To this end, methodological transparency is particularly salient in MMR. It must also be noted that the principle is to some degree emblematic of MMR given this form of research is defined by using both quantitative and qualitative forms of inquiry, but no research approach would be considered of high quality if its purpose is unclear. Scholars disseminating MMR should acknowledge explicitly within a study's method section (a) the core design employed (e.g., concurrent, explanatory sequential); (b) the qualitative and quantitative component designs and a rationale for their use, including how, when used together, they promote synergy; and (c) the methods of integration and how they promote the study purpose. These decisions and design elements should not be left for readers and reviewers to infer.

Strong Component Designs

Nastasi et al. (2021) articulate the idea that different systems of research, or inquiry frameworks, use their own philosophical foundations and approaches toward promoting and critiquing research quality. In MMR, it is important that researchers consider the quality indicators of the component designs and frameworks that are being integrated. For example, those familiar with survey research would likely consider indicators of quality when considering matters around how a sample was gathered, such as whether a sufficiently large sample was randomly drawn from a list that is well representative of a formally defined population. If the answer is yes, then several indicators of survey sampling quality are addressed (e.g., statistical power, sampling methodology, population specification), leading one to begin to conclude that whatever is observed from the survey effort is likely to be trustworthy. Survey report readers will, however, also consider survey instrument quality, the degree to which items adequately cover the topic at hand, and if the items are written in a way that avoid influencing respondents to provide socially desirable responses (Dillman et al., 2009).

These considerations represent fundamental questions about survey quality, and methodologists have described sources of survey error that provide guideposts to think critically about the quality of information (Dillman et al., 2009). There are quality indicators of survey research representing a framework for understanding this form of research. It is important to understand that this framework relies on at least one other framework; specifically, one of the sources of survey error deals with measurement, which is predicated on the field of psychometrics. Therefore, systems of quality may build on or borrow from each other.

For another example, single-case designs (SCDs) represent an extensive set of well-established quality indicators (see Horner et al., 2005), but these indicators rely at least to some degree on the Campbellian causal validity framework used to understand experimental and quasiexperimental designs (cf. Hitchcock et al., 2014; Kratochwill et al., 2013) when the SCD is used to understand a treatment effect.

Through qualitative research, scholars will often wish to explore the perspectives of different stakeholder groups to describe phenomena, build theory, understand cultural nuances, dive into the particulars of a special case, and so on (see Brantlinger et al., 2005). In the last Exceptional Children special issue on quality indicators of research, Brantlinger et al. (2005) describe a set of credibility measures for qualitative research (see Brantlinger et al., 2005, Figure 2) that provides part of the framework for identifying high-quality qualitative inquiry. From a systemic design point of view, researchers should endeavor to use multiple credibility checks (e.g., evaluate triangulation across multiple sources of information, maintenance of audit trails [research records], and peer debriefing; see Brantlinger et al. [2005] and Trainor et al. [this issue] for details).

Minimizing threats to quality is a consideration for any given framework and not just qualitative inquiry. For example, an RCT benefits from baseline measurement of participants because this allows for improved sample description, better understanding of the comparison- and control-group context, empirical checking on the outcome of randomization, and likely, improved statistical power if pretest scores can be used in impact modeling (Shadish et al., 2002). These are all indicators of RCT design quality and help address threats to causal inference while promoting analytic clarity. However, a lack of a pretest does not equate with a lack of quality. There are numerous excellent trials that lack pretest information because of considerations such as cost or because some baseline pretest might not have been feasible. The point here is that few studies can claim use of all design techniques that might be quality indicators.

Thus, there are three key points that must be appreciated when thinking about component designs in MMR: (a) all forms of inquiry have quality indicators, (b) quality indicators might be drawn from different forms of research, and (c) not all quality indicators need be used. This is because MMR quality is predicated not only on strong application of a mixed-methods design and how different forms of inquiry are integrated but also on quality criteria of component study designs (Collins, 2015). For example, consider a MMR study that combines SCD with qualitative inquiry. Both thoughtful integration of the SCD and qualitative components and the quality of the components themselves must be considered (O’Cathain, 2010; Teddlie & Tashakkori, 2009). However, not all indicators of quality are feasible or necessary, so researchers should be careful to choose different techniques to minimize threats to quality in an efficient manner based on the research's purpose.

Legitimation

For quantitative and qualitative research methods, one indicator of quality is the degree to which results and findings from a study are judged to be sound or reasonable. In quantitative research this is often referred to as “validity,” whereas in qualitative research it is often termed “trustworthiness,” “credibility,” or “dependability.” Scholars posit there should be a corollary for the inferences derived from MMR and suggest “legitimation” as a possibility (Onwuegbuzie & Johnson, 2006). “The legitimation step involves assessing the trustworthiness of both the qualitative and quantitative data and subsequent interpretations” (Johnson & Onwuegbuzie, 2004, p. 22). Hence, it can be construed as a quality indicator of MMR for purposes of this special issue. There are several different legitimation types, as presented in Table 2.

Table 2.

Legitimation Types as Quality Indicators.

Type	Description
Commensurability approximation	Degree to which meta-inferences are based on quantitative and qualitative worldviews and informed by researchers’ purposefully seeking synergy derived from applying two or more methods. Example: Researchers might consider how experimental evidence derived from an SCD and qualitative evidence about social validity combine to allow for overall inferences about intervention utility.
Conversion	Degree to which qualitative data can be quantitized and quantitative data can be qualitized to yield quality meta-inferences. Example: Providing case notes during a student's triennial evaluation otherwise based on test scores and rating scales. Does the narrative adequately capture test score data to support subsequent meta-inferences?
Inside-outside	Degree to which stakeholders’ views are accounted for, relative to the views of a researcher-observer. Example: Attempting to understand the views of students, parents, and/or teachers.
Integration	Extent to which researchers appropriately integrate quantitative and qualitative data. Example: Failure to account for all interview data and findings during an RCT would imply the presence of integration problems. This does not necessarily mean interview data are ignored (e.g., strong positive effects are found but research participants have complaints about a treatment), but an inability to integrate undermines inference quality.
Multiple validities	Degree to which different assumptions underlying component research processes are considered. Example: Ensuring statistical assumptions are addressed during impact analyses and ensuring qualitative quality indicators (e.g., member checks, triangulation, audit trails) are used to support interview findings.
Paradigmatic/ philosophical	Degree to which researchers’ worldviews or beliefs are addressed during the inquiry process. Example: Researchers who espouse a cognitive-behavioral framework might see value in showing students positive SCD results to promote treatment buy-in.
Sample integration	Considering how well sampling designs allow for making meta-inferences. Example: Mixed multiple-baseline design that includes interviewing treatment recipients and stakeholders who deliver a treatment will have strong quantitative and qualitative overlap. In contrast, an RCT with interviews of only sample members who do not respond to treatment will have weaker integration.
Sequential	Considering if there might be an order effect that yields a meta-inference. Example: Always showing positive SCD outcomes (a quantitative phase) and then interviewing if a treatment is acceptable to teachers might bias their opinions; researchers might wish to learn about teachers’ views in absence of SCD graphs.
Sociopolitical/political	Evidence of tension that can arise when members of a team value quantitative evidence over qualitative evidence or vice versa or when consumers of research privilege different forms of evidence. Example: Policy makers might push adoption of new teaching techniques based on quantitative efficacy evidence even if teachers report reservations about practice change.
Weakness minimization	Degree to which a given quantitative research approach compensates for the weaknesses of a qualitative approach and vice versa. Example: A survey study asking general education teachers about co-teaching preferences can be used to make population inferences based on a randomly drawn sample, but a survey might not effectively capture all preferences or the reasons for them; interviews can, in contrast, allow researchers to explore the matter in depth (keeping in mind it is difficult to conduct interviews with large sample sizes).

Note. Draws from Onwuegbuzie and Johnson (2006) and Johnson and Christensen (2020). RCT = randomized controlled trial; SCD = single-case design.

One consequence of the limited use of MMR in special education research is that there are few fleshed out examples of scholarship that explicitly discusses legitimation in the ways identified in Table 2. Five interrelated factors likely exacerbate the lack of examples: (a) legitimation as a concept is relatively new, (b) the American Psychological Association's publication manual's guidance on how to describe legitimation ideas represents only a first effort with sparse details (see American Psychological Association, 2020, p. 108), (c) the topic is described primarily in MMR outlets, (d) page limitations likely limit its full discussion in empirical works, and (e) legitimation is expansive. Relatedly, guidelines for determining how many and which legitimation criteria are “right” for a particular study are relatively nonexistent given that various intellectual communities (e.g., social constructivists, postpositivists, critical theorists) have different practices and values (Collins et al., 2012).

For illustrative purposes, we present two MMR studies from the broader education field that describe legitimation using phrasing presented in Table 2. It should be noted that in our judgment, many studies (both within and outside special education) in principle demonstrate these ideas without the explicit phrasing, but we highlight two that include explicit discussions of legitimation. First, Benge and colleagues (Benge, 2012; Benge, Onwuegbuzie, et al., 2018; Benge, Robbins, et al., 2018) present a body of work focusing on fifth-grade students’ vocabulary acquisition. Their broad motivation was to support teachers in direct vocabulary instruction using strategies like presenting mnemonics and cartoons. Their line of research is based on a dissertation that used a sequential MMR study that led to a model for presenting threats to legitimation.

The quantitative design in Phase 1 used a quasiexperimental approach wherein 220 students were exposed to four different instructional conditions: (a) dictionary definition only, (b) revised definition only, (c) dictionary definition plus cartoon, and (d) revised definition plus cartoon. The Vocabulary Knowledge Check Sheet (Beck et al., 2013) and a sentence generation task were the outcome measures. Qualitative work primarily entailed semistructured student interviews. Overall, the study found that students’ aggregated performance was lowest in the dictionary-definition-only condition; qualitative work showed that students preferred use of a cartoon visual to support conceptual understanding (Benge, Robbins, et al., 2018), yielding a meta-inference that we summarize as “a picture is worth a thousand words.”

In Phase 2, Benge, Onwuegbuzie, et al. (2018) interviewed the teachers of these students (n = 8) about their perceptions around the use and effectiveness of cartoon mnemonics relative to other instructional strategies, such as relying on traditional word definitions to teach vocabulary. Qualitative analyses revealed that primary teacher themes were (a) prior knowledge (the degree to which teachers thought students’ prior vocabulary knowledge played a role in student ability to capture meaning from a cartoon), (b) cognitive support (the type and degree of instructional support teachers offered), and (c) student engagement (e.g., student interest and boredom, ability to relate to a cartoon). These themes were numerically coded, allowing the authors to conduct a multivariate, exploratory technique called correspondence analysis. This step allowed the researchers to visually cluster teachers by frequency of their theme endorsement and revealed that three teachers primarily clustered around (i.e., focused on) students’ prior knowledge in their interview responses and two teachers focused more on cognitive support. One teacher referenced both themes across her interview responses, whereas three teachers tended to focus only on student engagement.

An additional correspondence analysis was conducted on 10 subthemes. This second correspondence analysis yielded a three-dimensional graph bracketed by the four lesson types, teacher, and subtheme. The graph yielded an opportunity to profile how teachers perceived the importance of student learning constructs during vocabulary acquisition. For example, Benge, Onwuegbuzie, et al. (2018) reported, “Kyleigh and Kasey were unique in that they were the only teachers who most strongly align to the piqued interest/enjoyment and connections/relate subthemes, supporting their perception that student engagement was a necessary component to their students’ ability to learn” (p. 575).

In a broader sense, this MMR approach can help researchers understand learning factors teachers recall when trying a new instructional strategy, and importantly, the approach demonstrates quantitative-qualitive synergy. Specifically, interviews were open-ended and semistructured and thereby had an exploratory element. This provides potential access to insights that might not be gleaned from closed-ended instruments that might or might not present the right items, or enough of them, to explore teachers’ cognitions and attitudes that arise when using new instructional strategies. Despite these strengths, the intersection between three themes, 10 subthemes, eight teachers, and four instructional conditions can make profiling a challenge. Correspondence analyses of quantified themes can, however, pull these factors into coherent and accessible findings. Hence, this quantitative-qualitive synergy provides an example of weakness minimization legitimation. Turning back to the larger line of inquiry, Benge (2012) presents a legitimation table (see Supplemental Table 5). In all, Benge (2012) reported this table provided a good understanding of the strengths and limitations of the MMR studies.

In a second example, Cooper and Hall (2016) describe legitimation in an exploratory concurrent triangulation design to understand the motivations behind male student athletes’ decisions to attend a historically Black college or university, their overall college experience, and factors that influenced their academic achievement. The design entailed parallel quantitative and qualitative strands carried through analyses, and integration was pursued via later triangulation of findings; it was weighted toward qualitative inquiry, with data collection including document reviews, focus groups, and interviews among 57 students who completed the Student Athlete College Experiences Questionnaire (SACEQ). The authors conceptualized integration at the study's conceptual phase, the experimental stage (during the course of the study), and the inferential stage to generate meta-inferences and engage in legitimation.

Legitimation steps included weakness minimization, conversion, commensurability, and multiple validities. In terms of weakness minimization (Onwuegbuzie & Johnson, 2006), the authors referenced the lack of flexibility and capacity for in-depth analyses that occurs during survey work and in particular because of how they used the SACEQ. The use of interviews, focus groups, and document reviews addressed this weakness by affording an opportunity to explore detailed information about the institution's history, culture, and values and how these intersected with athlete experiences. However, the SACEQ assisted the researchers by documenting the frequency and extent to which respondents were satisfied with the college experience and their motivations to attend. The authors converted qualitative themes to assess code frequency, which helped them to understand volume among theme type, a common approach for identifying the meaning and importance of some themes. The authors reported that theme quantification promoted more transparent analyses and code refinement. Commensurability was pursued through meta-inferences. The authors reported, for example, that document analyses established that university policy required athletes to attend 10 hours of study hall per week, the SACEQ found respondents thought they had a proper balance of academic and athletic work in their schedule, and interviews and focus groups provided details on why participants valued the study hall experience. In terms of multiple validities, the authors relied on content validation supporting their use of the SACEQ and psychometric reliability to support their subsequent use of data. Qualitative inquiry was supported by member checking, reflexivity, and an audit trail. In all, these four legitimation steps allowed the authors to pose meta-inferences that helped them to answer the research questions at hand and describe study limitations.

Considerations for the Future

The complexities characterizing special education research and practice, including the fact the field is interdisciplinary and inclusive of various epistemological stances, pedagogical perspectives, and views on disability, necessitate inquiry that draws on the strengths of quantitative and qualitative inquiry (Leko, 2014), thus making MMR a promising research option. We feel this is especially true for advancing special education knowledge and practice related to diversity, equity, and inclusion. To begin, MMR's roots in pluralism encourage diversity of ideas.

The adoption of MMR necessitates dismantling taken-for-granted assumptions about what “counts” as evidence and the best ways to secure it.

As Johnson (2017) points out, “MMR requires that researchers thoughtfully address competing values, methods, methodologies, ontologies, and epistemologies” (p. 167). In MMR, difference is not only expected but also valued as a way to forge new insights and reach more nuanced understandings of the world.

Additionally, MMR can answer questions about what practices work, for whom, and under what conditions. For example, despite advances in identifying and implementing evidence-based practices, outcomes for students of color with and without disabilities are less positive than those of their White peers (National Center for Education Statistics [NCES], 2019; Sullivan et al., 2013). As Klingner and Boardman (2011) noted, this can be partly explained by students of color being historically underrepresented in intervention research yet expected to respond to interventions similarly to White, monolingual students. With the application of MMR, however, questions about why outcomes differ across school contexts, disability, race, language, culture, gender, socioeconomic status, and so on can be better understood and explained. This is especially needed as the field grapples with the long-term impacts of the COVID-19 pandemic, when already-present inequities were exacerbated by the switch to virtual instruction (Jones et al., 2021).

The qualitative designs in MMR can illuminate the personal voices and counternarratives of participants who are often depicted by statistics exclusively. Yet, the inclusion of both qualitative and quantitative data in MMR holds promise for critical and emancipatory research wherein there is power in reaching the widest audiences possible, including those that might be more compelled to action based on words or numbers or both (Creswell et al., 2006). We also believe MMR holds potential for higher education faculty, of whom 75% in full-time positions are White (NCES, 2020), to further reflect on their own positionality and intersectionality in terms of race, language, culture, gender, and so on and how this influences their research. A quality indicator of qualitative research is reflexivity, or the consideration of one's identities and positions as a way to lead to interpretative transparency (Trainor & Graue, 2014). The American Psychological Association’s (2020) JARS, therefore, endorse reflexivity or positionality statements be included in the Method section of MMR articles. As more special education researchers utilize MMR, there will be more opportunities, and expectations, to engage in reflexive practices.

When considering the future of MMR in special education, we also consider its place relative to open science. We offer some initial thoughts with the caveat that there is still much to be determined on this front.

To begin, we consider preregistration and registered reports. Both hinge on researchers specifying their methods and major analytical decision points prior to conducting the planned study (Cook et al., 2018). Mixed-methods researchers would typically need to preregister the quantitative component, the qualitative component, and the integration of the two. Researchers should describe the qualitative and quantitative components, including the design, data sources, measures, procedures, and analyses. Then, researchers need to explicitly highlight how integration will occur. McLeod et al. (2022) provide an example. In their study, the authors propose to investigate how educators sustain an evidence-based intervention using an explanatory sequential MMR design with a complementarity purpose. Following the identification of significant predictors of sustainment outcomes via quantitative measures, the authors propose conducting qualitative, semistructured interviews to better understand and elaborate on the quantitative findings. The authors also propose to use the quantitative data to query participants about potential mechanisms that might influence their sustainment and adaptation of the intervention. In this proposed study, the quantitative data are integrated into the qualitative component of the study. Data analysis plans are presented for both the quantitative and qualitative components of the study, but the authors also discuss how data analyses will be integrated. For example, they plan to use the qualitative data to develop a descriptive analysis of multilevel sustainment mechanisms identified through the quantitative data. Additionally, they plan to use quantitative findings to categorize teachers into groups (e.g., low and high burnout) that can then be analyzed qualitatively to identify patterns within and across groups. As the study is implemented and methods potentially change, the authors would need to provide a rationale for those changes just as they would whether a study was mixed or not.

For MMR studies like the one just described, the registered report would need to be evaluated in terms of the qualitative and quantitative component designs and the integration of the two, resulting in the need for a greater number of reviewers with expertise across multiple methodologies, including MMR. Locating reviewers with such breadth and depth expertise, particularly in MMR, may be challenging in special education (at least until MMR becomes more common), but the extra effort may be well justified. An important benefit of MMR in the registered report process lies in its ability to explain null findings. With registered reports, studies are published whether they result in novel or positive results or not (Cook et al., 2018). A natural follow-up question would be why some inquiry efforts did not lead to positive results; herein lies an important power of MMR.

As might be expected, precise reporting of MMR, including the quantitative and qualitative component designs and their integration, is difficult to achieve with restrictive journal page limits. The practice of open materials, or the sharing of study materials online alongside a journal article (Cook et al., 2021), may alleviate pressure felt by researchers. With open materials, mixed-methods researchers can include quantitative and qualitative measures, instruments, and protocols (of which there may be several in a MMR study) in their entirety without consuming precious journal pages. Intervention materials can also be included as open materials, thereby eliminating the need for lengthy descriptions or figures.

To date, use of MMR has not been prevalent within special education scholarship, but we hope, and have reason to believe, this will change. High-quality MMR characterized by integration, synergy, strong component designs, and legitimation will produce scholarship that has been lacking in the field, especially as researchers and practitioners work to promote equity and inclusivity for students with disabilities and their families. Our purpose in presenting these quality indicators is that special education scholars will have guidance as they avail themselves of a methodology that has immense potential for addressing our field's most complex and pressing problems.

Supplemental Material

sj-docx-1-ecx-10.1177_00144029221141031 - Supplemental material for Quality Indicators for Mixed-Methods Research in Special Education

Supplemental material, sj-docx-1-ecx-10.1177_00144029221141031 for Quality Indicators for Mixed-Methods Research in Special Education by Melinda M. Leko, John H. Hitchcock, Hailey R. Love, David E. Houchins, and Maureen A. Conroy in Exceptional Children

Footnotes

ORCID iD

Melinda M. Leko

Supplemental Material

The supplemental material is available in the online version of the article.

Manuscript received March 2022; accepted October 2022.

References

American Psychological Association. (2020). Mixed methods article reporting standards.

Anguera

M. T.

Blanco-Villasenor

Losada

J. L.

Sánchez-Algarra

Onwuegbuzie

A. J.

(2018). Revisiting the difference between mixed methods and multimethods: Is it all in the name? Quality & Quantity, 52(6), 2757–2770. https://doi.org/10.1007/s11135-018-0700-2

Bastable

Meng

Falcon

S. F.

McIntosh

(2019). Using an embedded mixed methods design to assess and improve intervention acceptability of an equity-focused intervention: A methodological demonstration. Behavioral Disorders. Advance online publication. https://doi.org/10.1177/0198742919880486

Bazeley

(2012). Integrative analysis strategies for mixed data sources. American Behavioral Scientist, 56(6), 814–828. https://doi.org/10.1177/0002764211426330

Beck

I. L.

McKeown

M. G.

Kucan

(2013). Bringing words to life: Robust vocabulary instruction. Guilford Press.

Benge

C. L.

(2012). Effect of cartoon mnemonics and revised definitions on the acquisition of tier-two vocabulary words among selected fifth-grade students [Unpublished doctoral dissertation]. Sam Houston State University.

Benge

C. L.

Onwuegbuzie

A. J.

Robbins

M. E.

(2018). Effect of cartoon mnemonics and revised definitions on the acquisition of tier-2 vocabulary words among selected fifth-grade students: Phase II. Crossover mixed analysis. International Journal of Multiple Research Approaches, 10(1), 565–588. https://doi.org/10.29034/ijmra.v10n1a39

Benge

C. L.

Robbins

M. E.

Onwuegbuzie

A. J.

(2018). Effect of cartoon mnemonics and revised definitions on the acquisition of tier-2 vocabulary words among selected fifth-grade students: Phase I. Noncrossover mixed analysis. International Journal of Multiple Research Approaches, 10(1), 538–564. https://doi.org/10.29034/ijmra.v10n1a38

Bertilsson

Gyllensten

A. L.

Opheim

Gard

Hammarlund

C. S.

(2018). Understanding one’s body and movements from the perspective of young adults with autism: A mixed-methods study. Research in Developmental Disabilities, 78, 44–54. https://doi.org/10.1016/j.ridd.2018.05.002

10.

Brantlinger

Jiminez

Klingner

Pugach

Richardson

(2005). Qualitative studies in special education. Exceptional Children, 71(2), 195–207. https://doi.org/10.1177/001440290507100205

11.

Brownell

M. T.

Sindelar

P. T.

Kiely

M. T.

Danielson

L. C.

(2010). Special education teacher quality and preparation: Exposing foundations, constructing a new model. Exceptional Children, 76(3), 357–377. https://doi.org/10.1177/001440291007600307

12.

Cole

E. V.

Cawthon

S. W.

(2015). Self-disclosure decisions of university students with learning disabilities. Journal of Postsecondary Education and Disability, 28(2), 163–179.

13.

Collins

K. M. T.

(2015). Validity in multimethod and mixed research. In Nagy Hesse-Biber

Burke Johnson

(Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (pp. 240–256). Oxford University Press.

14.

Collins

K. M. T.

Onwuegbuzie

A. J.

Johnson

R. B.

(2012). Securing a place at the table: A review and extension of legitimation criteria for the conduct of mixed research. American Behavioral Scientist, 56(6), 849–865. https://doi.org/10.1177/0002764211433799

15.

Conroy

M. A.

Hetrick

Houchins

Hitchcock

(2022). Mixed and multiple methods in emotional and behavioral disorders research: A literature review [Manuscript submitted for review].

16.

Cook

B. G.

Cook

(2016). Research designs and special education research: Different designs address different questions. Learning Disabilities Research & Practice, 31(4), 190–198. https://doi.org/10.1111/ldrp.12110

17.

Cook

B. G.

Fleming

J. I.

Hart

S. A.

Lane

K. L.

Therrien

W. J.

van Dijk

Wilson

S. E.

(2021). A how-to guide for open-science practices in special education research. Remedial and Special Education. Advance online publication https://doi.org/10.1177/07419325211019100

18.

Cook

B. G.

Lloyd

J. W.

Mellor

Nosek

B. A.

Therrien

W. J.

(2018). Promoting open science to increase the trustworthiness of evidence in special education. Exceptional Children, 85(1), 104–118. https://doi.org/10.1177/0014402918793138

19.

Cooper

J. N.

Hall

(2016). Understanding black male student athletes’ experiences at a historically Black college/university: A mixed methods approach. Journal of Mixed Methods Research, 10(1), 46–63. https://doi.org/10.1177/1558689814558451

20.

Corr

Love

Snodgrass

Kern

Chudzik

(in press). Methodological training in special education doctoral programs: A mixed methods exploration. Teacher Education and Special Education.

21.

Corr

Snodgrass

M. R.

Love

Scott

I. M.

Kim

Andrews

(2021). Exploring the landscape of published mixed methods research in special education: A systematic review. Remedial and Special Education, 42(5), 317–328. https://doi.org/10.1177/0741932520924030

22.

Creswell

J. W.

Plano Clark

V. L.

(2018). Designing and conducting mixed methods research (3rd ed.). Sage.

23.

Creswell

J. W.

Shope

Plano Clark

V. L.

Green

D. O.

(2006). How interpretive qualitative research extends mixed methods research. Research in the Schools, 13(1), 1–11.

24.

Cross

F. L.

Agi

Montoro

J. P.

Medina

M. A.

Miller-Tejada

Pinetta

B. J.

Rivas-Drake

(2020). Illuminating ethnic-racial socialization among undocumented Latinx parents and its implications for adolescent psychosocial functioning. Developmental Psychology, 56(8), 1458–1474. https://doi.org/10.1037/dev0000826

25.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2009). Internet, mail and mixed-mode surveys: The tailored design method (3rd ed.). Wiley.

26.

Fàbregues

Molina-Azorín

J. F.

(2017). Addressing quality in mixed methods research: A review and recommendations for a future agenda. Quality & Quantity, 51(6), 2847–2863. https://doi.org/10.1007/s11135-016-0449-4

27.

Fetters

M. D.

Curry

L. A.

Creswell

J. W.

(2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6), 2134–2156. https://doi.org/10.1111/1475-6773.12117

28.

Fetters

M. D.

Freshwater

(2015). The 1 + 1 = 3 integration challenge. Journal of Mixed Methods Research, 9, 115–117. https://doi.org/10.1177/155868981558122

29.

Fetters

M. D.

Molina-Azorín

J. F.

(2017). The Journal of Mixed Methods Research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. https://doi.org/10.1177/1558689817714066

30.

Garwood

J. D.

Werts

M. G.

Varghese

Gosey

(2018). Mixed-methods analysis of rural special educators’ role stressors, behavior management, and burnout. Rural Special Education Quarterly, 37(1), 30–43. https://doi.org/10.1177/8756870517745270

31.

Greene

J. C.

(2007). Mixed methods in social inquiry. Jossey-Bass.

32.

Guetterman

T. C.

Fetters

M. D.

Creswell

J. W.

(2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Annals of Family Medicine, 13(6), 554–561. https://doi.org/10.1370/afm.1865

33.

Hall

Howard

(2008). A synergistic approach: Conducting mixed methods research with typological and systemic design considerations. Journal of Mixed Methods Research, 2(3), 248–269. https://doi.org/10.1177/1558689808314622

34.

Hesse-Biber

Johnson

R. B.

(2015). The Oxford handbook of multimethod and mixed methods research inquiry. Oxford University Press.

35.

Heyvaert

Hannes

Maes

Onghena

(2013). Critical appraisal of mixed methods studies. Journal of Mixed Methods Research, 7(4), 302–327. https://doi.org/10.1177/1558689813479449

36.

Hitchcock

J. H.

Horner

R. H.

Kratochwill

T. R.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2014). The What Works Clearinghouse single-case design pilot standards: Who will guard the guards? Remedial and Special Education, 35(3), 154. https://doi.org/10.1177/0741932513518979

37.

Hitchcock

J. H.

Onwuegbuzie

A. J.

(in press). The Routledge handbook for advancements in mixed methods integration. Routledge.

38.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10.1177/001440290507100203

39.

Howie

E. K.

Campbell

A. C.

Abbott

R. A.

Straker

L. M.

(2017). Understanding why an active video game intervention did not improve motor skill and physical activity in children with developmental coordination disorder: A quantity or quality issue? Research in Developmental Disabilities, 60, 1–12. https://doi.org/10.1016/j.ridd.2016.10.013

40.

Hunt

J. H.

Martin

Khounmeuang

Silva

Patterson

Welch-Ptak

(2020). Design, development, and initial testing of asset-based intervention grounded in trajectories of student fraction learning. Learning Disability Quarterly. Advance online publication. https://doi.org/10.1177/0731948720963589

41.

Johnson

R. B.

(2017). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2), 156–173.

42.

Johnson

R. B.

Christensen

L. B.

(2020). Educational research: Quantitative, qualitative, and mixed approaches (7th ed.). Sage.

43.

Johnson

R. B.

Onwuegbuzie

A. J.

(2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. https://doi.org/10.3102/0013189X033007014

44.

Johnson

R. B.

Onwuegbuzie

A. J.

Turner

L. A.

(2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. https://doi.org/10.1177/ 1558689806298224

45.

Johnson

R. B.

Turner

L. A.

(2003). Data collection strategies in mixed methods research. In Tashakkori, A., & Teddlie, C. (Eds.), Handbook of mixed methods in social and behavioral research (pp. 297–319). Sage.

46.

Jones

T. M.

Williford

Spencer

M. S.

Riggs

N. R.

Toll

George

Bruick

(2021). School mental health providers’ perspectives on the impact of COVID-19 on racial inequities and school disengagement. Children & Schools, 43(2), 97–106. https://doi.org/10.1093/cs/cdab009

47.

Klingner

J. K.

Boardman

A. G.

(2011). Addressing the “research gap” in special education through mixed methods. Learning Disability Quarterly, 34(3), 208–218. https://doi.org/10.1177/0731948711417559

48.

Kozleski

E. B.

(2017). The uses of qualitative research: Powerful methods to inform evidence-based practice in education. Research and Practice for Persons with Severe Disabilities, 42(1), 19–32. https://doi.org/10.1177/1540796916683710

49.

Kratochwill

T. R.

Hitchcock

J. H.

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2013). Single-case intervention research design standards. Remedial and Special Education, 34(1), 26–38. https://doi.org/10.1177/0741932512452794

50.

Leko

M. M.

(2014). The value of qualitative methods in social validity research. Remedial and Special Education, 35(5), 275–286. https://doi.org/10.1177/0741932514524002

51.

Love

H. R.

Corr

(2022). Integrating without quantitizing: Two examples of deductive analysis strategies within qualitatively driven mixed methods research. Journal of Mixed Methods Research, 16(1), 64–87. https://doi.org/10.1177/1558689821989833

52.

Marsh

Houchins

D. E.

Christenson

(2022). Using an integrated approach when conducting juvenile justice special education mixed research. In Hitchcock

Onwuegbuzie

(Eds.), Handbook for advancing integration in mixed methods (pp. XX–XX). Routledge.

53.

McLeod

B. D.

Sutherland

K. S.

Conroy

M. A.

Lyon

A. R.

Chapman

J. E.

Granger

K. L.

Saldana

(2022). Study protocol: Multi-level determinants of implementation and sustainment in the education sector. Journal of Emotional and Behavioral Disorders. Advance online publication. https://doi.org/10.1177/1063426622109016

54.

Nastasi

B. K.

Hitchcock

J. H.

Gutierrez

Oshrin

(2021). Promoting quality of mixed methods consultation research: A synergistic participatory framework. Journal of Educational and Psychological Consultation. Advance online publication. https://doi.org/10.1080/10474412.2021.1887743

55.

National Center for Education Statistics. (2019). The nation’s report card.

56.

National Center for Education Statistics. (2020). Characteristics of postsecondary faculty. In The condition of education 2020 (NCES 2020-144).

57.

Newman

Houchins

D. E.

(2018). Conceptualizing mixed methods questions in special education research. Research in the Schools, 24(2), 23–33.

58.

Nzabonimpa

J. P.

(2018). Quantitizing and qualitizing (im-)possibilities in mixed methods research. Methodological Innovations, 11(2), 1–16. https://doi.org/10.1177/2059799118789021

59.

O’Cathain

(2010). Assessing the quality of mixed methods research: Toward a comprehensive framework. In Tashakkori

Teddie

(Eds.), Handbook of mixed methods in social and behavioral research (2nd ed., pp. 531–555). Sage.

60.

Odom

S. L.

Brantlinger

Gersten

Horner

R. H.

Thompson

Harris

K. R.

(2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71(2), 137–148. https://doi.org/10.1177/001440290507100201

61.

Onwuegbuzie

A. J.

(2012). Introduction: Putting the mixed back into quantitative and qualitative research in educational research and beyond: Moving towards the radical middle. International Journal of Multiple Research Approaches, 6, 192–219. https://doi.org/10.5172/mra.2012.6.3.192

62.

Onwuegbuzie

A. J.

Corrigan

J. A.

(2018). What is happening now? An overview of mixed methods applications in special education. Research in the Schools, 25(2), 1–22.

63.

Onwuegbuzie

A. J.

Johnson

R. B.

(2006). The validity issue in mixed research. Research in the Schools, 13(1), 48–63.

64.

Onwuegbuzie

A. J.

Leech

N. L.

(2019). On qualitizing. International Journal of Multiple Research Approaches, 11(2), 98–131. https://doi.org/10.29034/ijmra.v11n2editorial2

65.

Petrenko

C. L. M.

Pandolfino

M. E.

Roddenbery

(2016). The association between parental attributions of misbehavior and parenting practices in caregivers raising children with prenatal alcohol exposure: A mixed-methods study. Research in Developmental Disabilities, 59, 255–267. https://doi.org/10.1016/j.ridd.2016.09.005

66.

Plano Clark

V. L. P.

Ivankova

N. V.

(2016). Mixed methods research: A guide to the field (Vol. 3). Sage.

67.

Regan

K. S.

Berkeley

S. L.

Hughes

Brady

K. K.

(2015). Understanding practitioner perceptions of responsiveness to intervention. Learning Disability Quarterly, 38(4), 234–247. https://doi.org/10.1177/0731948715580437

68.

Sandelowski

Voils

C. I.

Knafl

(2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208–222. https://doi.org/10.1177/1558689809334210

69.

Scanlon

Saenz

Kelly

M. P.

(2018). The effectiveness of alternative IEP dispute resolution practices. Learning Disability Quarterly, 41(2), 68–78. https://doi.org/10.1177/0731948717698827

70.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

71.

Sullivan

A. L.

Klingbeil

D. A.

Van Norman

E. R.

(2013). Beyond behavior: Multilevel analysis of the influence of sociodemographics and school characteristics on students’ risk of suspension. School Psychology Review, 42(1), 99–114. https://doi.org/10.1080/02796015.2013.12087493

72.

Tashakkori

Teddlie

(2010). Handbook of mixed methods in social and behavioral research. Sage.

73.

Teddlie

Tashakkori

(2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage.

74.

Trainor

A. A.

Graue

(2014). Evaluating rigor in qualitative methodology and research dissemination. Remedial and Special Education, 35(5), 267–274. https://doi.org/10.1177/0741932514528100

75.

Unwin

K. L.

Powell

Jones

C. R.

(2021). A sequential mixed-methods approach to exploring the experiences of practitioners who have worked in multi-sensory environments with autistic children. Research in Developmental Disabilities, 118. https://doi.org/10.1016/j.ridd.2021.104061

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB