Integrating Transformative Considerations and Quantitative Results Through a Participant Selection Joint Display in Explanatory Sequential Mixed Methods Studies

Abstract

Integrating philosophical or paradigmatic dimensions in mixed methods research studies facilitates the development of stronger meta-inferences. The transformative paradigm and the explanatory sequential mixed methods design share a focus on developing sampling criteria, but with different priorities. This article contributes to the field of mixed methods research by presenting a method of integrating transformative sampling considerations in explanatory sequential designs through a participant selection joint display. The approach presented addresses concerns regarding transparency of research decisions in mixed methods studies, while providing a method of centering the transformative paradigm in mixed methods integration procedures.

Keywords

transformative paradigm mixed methods joint display integration

Integration is the cornerstone of mixed methods research (MMR), and remains under constant methodological study on why, how, and when researchers mix quantitative and qualitative approaches. In seeking to develop stronger meta-inferences, methodologists have discussed the need to integrate across multiple dimensions of a study including philosophy, theory, data collection, and sampling (Fetters & Molina-Azorin, 2017; Onwuegbuzie & Collins, 2007); when integrating across multiple dimensions, more advanced and dynamic MMR designs are available with the potential to yield more insightful meta-inferences (Tashakkori & Teddlie, 2003, p. 689). Guidance is needed to further promote mixing across more practical decisions (such as sampling) in study design, particularly as the field progresses and dynamic MMR designs integrate philosophy into the conception, design, and reporting of quantitative, qualitative, and mixed phases.

The transformative paradigm of research, as promoted by Mertens (2007, 2009, 2012), provides a framework for promoting social justice and attending to the complexity in communities who are often oppressed and socially disenfranchised.¹ Ethically, the transformative paradigm compels researchers to (1) work with affected communities, (2) be cognizant of the role of oppression and intersectionality, (3) be culturally and linguistically competent when working with communities, and (4) understand that research ethics applies not just to individual participants but also to affected communities (Mertens, 2007, 2009). The transformative paradigm’s axiological assumptions guide the epistemological and ontological assumptions, which then inform methodology. Ontologically, the transformative paradigm holds that reality is socially constructed and that “what is real” is rooted in power relations (Mertens, 2009). Therefore, researchers gain the responsibility of interrogating whose version of reality is promoted and how power and oppression influence the promotion of that reality (Mertens, 2012). The transformative paradigm has been applied in studies across disciplines including education, healthcare, policy, and sustainability (Camacho, 2020; Canales, 2013; Mertens, 2018; Sweetman et al., 2010).

Introductory MMR texts discussing the transformative paradigm, however, frequently focus solely on the social justice-orientation of the paradigm, often ignoring how the paradigm seeks to promote justice (i.e., by applying highly philosophical dimensions) (Creswell & Plano Clark, 2017, pp. 36–38; Tashakkori et al., 2021, pp. 62–64). Some of these justice-oriented transformative considerations include actively pushing against the status quo, engaging in continual reflexivity, and applying rigorous methods to interrogate the role of oppression and power in the health, social, and quality of life outcomes among diverse, culturally complex communities. Sampling is crucial to the implementation of transformative methodology, which seeks to represent views customarily not represented due to structural oppression. Despite the role of sampling, however, there is limited discussion on how to practically apply transformative sampling tenets in MMR studies while maintaining rigor in core MMR designs. Therefore, the methodological aim of this article is to present a method to integrate transformative considerations and quantitative findings to develop qualitative sampling schemes. We demonstrate this method through an explanatory sequential mixed methods study from the health sciences, investigating emergency department utilization among deaf and hard-of-hearing patients.

Sampling in the Transformative Paradigm

With the aim of exposing the role of power and oppression through research, and aligned with the ontology of the paradigm, transformative sampling strategies are focused on diversity within samples and engaging populations that are typically excluded in research and, therefore, have diminished promotion of their experience (Mertens, 2009, p. 200). Central to the transformative paradigm’s methodology is caution of the “myth of homogeneity”: using generic labels and failing to recognize the complexity among groups of people (Mertens, 2009, p. 200). In avoiding this sampling threat, researchers are encouraged to consider all relevant aspects of diversity, particularly within underserved groups.—Specifically, researchers are recommended to consider “what are the dimensions of diversity that are important in [their] study? (Mertens, 2009, p. 201)—Through this process, researchers define and label these relevant dimensions, and identify strategies for recruiting participants who represent the intended diversity (Mertens, 2009, p. 213).

Despite these recommendations for sampling in transformative MMR, there is a lack of transparency on what dimensions researchers choose to prioritize in their studies and how these decisions impact study design (Camacho, 2020). Moreover, there is a lack of methodological discussion on how researchers make these decisions; for example, reviews and guidance focused on transformative MMR do not consider the sampling decisions occurring within studies (Canales, 2013; Sweetman et al., 2010).

Sampling in Explanatory Sequential Mixed Methods Studies

The explanatory sequential design is a popular MMR design consisting of a distinct quantitative phase followed by a qualitative phase (Creswell & Plano Clark, 2017). The intention of the design is to use qualitative procedures to help further explain quantitative findings. Major methodological considerations when implementing the core explanatory sequential design are focused on the priority given to quantitative and qualitative data, and how methods and results are integrated (Ivankova et al., 2006). To this end, there are two crucial integration points in the explanatory sequential design. The first occurs in the intermediate phase between quantitative results and the beginning of qualitative data collection; the second, at the end of the study when results are compared across phases (Creswell & Plano Clark, 2017; Ivankova et al., 2006).

Of interest to the present article is the integration that occurs during the intermediate phase of the design to align samples between the quantitative and qualitative phases. A central, methods-level integration technique during this intermediate phase is connecting: using the results of one type of analysis to inform the sampling in another type of analysis (Creswell & Plano Clark, 2017; Fetters, 2020; Fetters et al., 2013). For example, in the explanatory sequential design, a researcher may use results from a quantitative survey to develop purposeful sampling goals for qualitative interviews. Connecting samples across quantitative and qualitative phases facilitates the development of stronger meta-inferences (Ivankova et al., 2006), and reduces the sampling integration legitimation threat (Onwuegbuzie & Johnson, 2006, pp. 56–57).

Mixed methods researchers may use several techniques to connect samples. Joint displays are one technique which have gained traction in MMR as a tool to plan, report, and facilitate integration (Creswell & Plano Clark, 2017; Guetterman, Fetters, et al., 2015; Onwuegbuzie & Burke Johnson, 2021). Joint displays in the explanatory sequential design may be specifically designed to assist with connecting samples; for example, Guetterman et al. (2015, p. 160,164–166) describe a “participant selection display” used to demonstrate how quantitative results inform a purposeful sampling for the qualitative phase. However, these types of joint displays do not account for the unique considerations of transformative paradigms.

Methodological Goal

Sampling is a core dimension of interest in both the explanatory sequential design and the transformative paradigm of research. However, as described, the integration of transformative sampling considerations may not always be reported in MMR studies. In addition, connecting quantitative results to qualitative sampling in explanatory sequential designs encourages researchers to focus solely on the quantitative constructs in making sampling decisions, carrying the limitations of the quantitative dataset and methods to the qualitative phase. For example, large datasets (e.g., national surveys, clinical records) largely lack social-related variables, which are pertinent to the transformative paradigm. In this way, methodological guidance for the transformative paradigm and explanatory sequential design seems misaligned, and a researcher may ignore justice-oriented considerations when developing their qualitative sampling priorities.

In seeking to resolve this perceived misalignment, promote transparency in reporting MMR studies, and integrate the transformative paradigm in explanatory sequential designs, below we describe the development of a participant selection joint display including transformative considerations. The methodological innovation presented in this paper is specifically for integrating quantitative results and transformative paradigm considerations in the development of qualitative sampling schemes. Therefore, we present limited information about the quantitative methods and qualitative findings.

Example Study: Emergency Department Utilization among Deaf and Hard-of-Hearing Patients

Overview

The parent study was the first author’s dissertation which applied an explanatory sequential mixed methods design to develop and revise a conceptual model of emergency department utilization among deaf and hard-of-hearing (DHH) adults in the U.S. DHH people are an underserved and historically ignored population in health research. The DHH population is heterogenous with respect to social and medical characteristics, including development of a DHH identity, chosen language modality, age of onset of hearing loss, and etiology, which are associated with socio-behavioral antecedents influencing emergency department care processes and outcomes (James, Varnes, et al., 2021). For example, DHH people who use American Sign Language (ASL) and DHH people who use spoken English typically have different sociomedical experiences including age of onset, early childhood language access, and distal social outcomes including educational attainment, employment, and access to health promoting resources. For example, DHH people who use ASL to communicate are at high risk of not receiving effective communication access in healthcare environments, which may diminish the quality of care (James, Argenyi, et al., 2022; James, Coady, et al., 2022; McKee et al., 2011). For this reason, the goal of the study was to assess differences in emergency department care among DHH ASL-users and DHH English-speakers, in comparison to non-DHH English-speakers.

Quantitative data were from electronic health records of patients who used the emergency department at a large academic medical center in the southeastern United States. The methods and results of the quantitative phase have been published elsewhere (James, McKee, et al., 2022; James, Miller, et al., 2022). Qualitative data were from in-depth semi-structured interviews with DHH people who use ASL and English to communicate, who self-reported being patients at the sampled emergency departments during the 3 years prior to the study.

The transformative paradigm was used throughout the study, which was designed using a community-engaged research (CEnR) approach. During the preliminary phase of conceptual model development, evidence synthesis highly weighted research studies conducted by DHH authors (James, Varnes, et al., 2021). During the quantitative phase, researchers used the QuantCrit framework (Garcia et al., 2018; Gillborn et al., 2018); namely, (1) centering the role of audism (a form of oppression against DHH people), (2) recognizing the non-neutrality of clinical data, (3) using DHH status as an indicator/proxy for experiencing audism, and (4) maintaining the orientation that statistics are not value-free. During qualitative data analysis, researchers applied a transformative paradigm and Critical Disability/DeafCrit lens to interrogate the role of audism, and oppression at the intersection of audism and other systems of power/oppression. The study also convened a community advisory group of DHH people who provided feedback when framing quantitative and qualitative results, assisted with determining qualitative data collection priorities (both sampling and interview prompts), recruited participants for the qualitative phase, and disseminated results to local community members.

Sampling Integration

Researcher’s Transformative Considerations

As a point of reflexivity, the first author is not DHH but is fluent in ASL and the adult child of DHH English-speaking parents. At the time of the study, he had 6 years of experience in the local DHH community. He applied transformative paradigm principles when determining sampling priorities; specifically, questioning whose experience of seeking care in the emergency department needed to be understood and represented to highlight social inequity and lead to social change. Based on his previous studies and his commitment to social justice and health equity, perspectives of particular importance to him were from (1) people who use ASL who were not raised bilingually with English as a second language or who are immigrants to the United States; (2) Black, Indigenous and people of color who face barriers to health equity due to oppressive social structures and also receiving poorer care in health systems; and, (3) other minoritized or historically oppressed populations including LGBTQ + individuals, and individuals who engage in socially taboo health behavior (e.g., illicit drug use) or have contentious chronic health conditions (e.g., fibromyalgia) which impact how individuals use healthcare.

Community Advisory Group’s Transformative Considerations

After identifying and defining his transformative considerations, the first author sought feedback from the study’s community advisory group members to identify areas of diversity they believed would provide a deeper perspective. The feedback request was included with a training video to increase community member knowledge of qualitative research procedures and the importance of gathering different perspectives for our research inquiry and introducing members to the social-ecological model of health. The first author then described, with visual illustrations, an example of the parable of the blind men and an elephant. This parable from the Indian subcontinent describes the limited perspective of individual experiences and the role of collective wisdom. In this case, the parable was used to describe the importance of seeking perspectives from a diversity of DHH people. This training session ended with the following prompt: “Whose perspective helps us understand the full picture? Think about the DHH people that you know who go to the ER. What are their lives like?”

Community advisory group members encouraged diversity with respect to including people (1) from historically oppressed populations (including Black DHH women), (2) living in rural environments, (3) who are migrant workers, (4) who are un/underemployed, (5) who are diverse with respect to medical aspects of deafness (e.g., DeafBlind and late-deafened individuals), (6) who are socially isolated, (7) with mental health and substance use conditions, and (8) living in housing environments of poor quality.

Quantitative Results

Statistical results indicated that DHH ASL-users and DHH English-speakers had higher rates of ED utilization than non-DHH English-speakers (James, McKee, et al., 2022; James, Miller, et al., 2022). Two analyses provided insights for qualitative sampling priorities: sub-group analyses, and descriptive results on primary diagnosis for using the emergency department. Sub-group analyses among DHH English-speaking patients indicated that Black patients (compared to white patients), Medicaid insured and uninsured patients (compared to privately insured patients), and current smokers who use combustible cigarettes (compared to never smokers) had higher adjusted odds of using the emergency department in the past 36 months. Sub-group analyses among DHH ASL-users did not indicate any significant differences among quantitative dimensions.

Chief complaints/reasons for using the emergency department, operationalized as primary diagnosis code, were mostly similar among each patient segment. However, two results were highlighted during this phase (James, McKee, et al., 2022). First, DHH ASL-users made up a minority of mental health/substance use related encounters. This finding was counter to theory and previous evidence in this population indicating significant burden of mental health conditions (Glickman & Hall, 2019; James, McKee, et al., 2022). Secondly, DHH English-speaking women, aged 18–44 years old, made up the majority (69%; 31 of 45) of the top encounters for maternal/neonatal related conditions including other complications of pregnancy (25 of 32).

Developing a Participant Selection Joint Display

Researcher- and community advisory group-prioritized sampling dimensions and qualitative results were integrated using a participant selection joint display (see Table 1). The table included a column for conceptual model construct (from the preliminary conceptual model; James, Varnes, et al., 2021), researcher- and community advisory group-determined considerations, quantitative findings, and the resulting qualitative sampling priorities. For example, one of the resulting qualitative sampling priorities was to ensure adequate sampling of Black participants, and participants from rural areas. This priority was developed through alignment with the perspective of the researcher and community advisory group, in addition to evidence from the quantitative sub-group analysis indicating higher risk of emergency department utilization among Black patients (relative to white patients).

Table 1.

Participant Selection Joint display Integrating Transformative Sampling Considerations and Quantitative Findings.

Construct	Researcher-Defined Transformative Sampling Considerations	Community Advisory Group Transformative Sampling Considerations	Quantitative Findings	Qualitative Sampling Priorities
DHH patient: Predisposing—demographic and cultural considerations	•LGBT+. •Black, Indigenous, POC.	•Historically marginalized groups (e.g., Black people). •Rural. •Migrant workers due to culture, language, employment status, and discrimination. •People who are un or underemployed	•In sub-group analyses of DHH English-speakers, Black patients had higher adjusted odds (aOR = 3.636) of using the ED in the past 36 months than white patients	•Ensure adequate sampling of Black participants. •Sample some participants from rural areas/counties
DHH patient: Predisposing—DHH-specific demographics and cultural characteristics	N/A	•Late-deafened and DeafBlind individuals	N/A	•Sample participants from a variety of DHH identities
DHH patient: Enabling—financial considerations	•Food insecure	•Food insecure. •Lack of insurance	•In sub-group analyses of DHH English-speakers, Medicaid/uninsured patients had higher adjusted odds (aOR = 1.752) of using the ED in the past 36 months than patients who were privately insured	•Sample participants who are Medicaid insured or uninsured, which may also be a marker of low-income status and food insecurity
Context: Enabling—community environment	N/A	•Poor housing •Community mental health/substance use issues	N/A	•Sample participants who believe substance use is an issue in their community
Central outcome: Health status	•Chronic illness (especially those who have ‘controversial’ diagnoses, e.g., fibromyalgia). •Illicit drug use	•Chronic illness. •Mental health diagnosis	•A similar proportion of patients, across patient segments, engaged in current combustible cigarette use. •In sub-group analyses of DHH English-speakers, current smokers had higher adjusted odds (aOR = 1.719) of using the ED in the past 36 months than patients who were never smokers. •DHH ASL-users made up a minority of mental health/substance use related encounters. •DHH English-speakers had lower adjusted odds (aOR = 0.697) of having comorbid conditions on the Charlson Comorbidity Index than non-DHH English-speakers	•Sample participants who experience mental health conditions or concerns
Central outcome: Patient evaluated need – reasons for use	N/A	N/A	•DHH English-speaking women, aged 18–44 years old, made up the majority of encounters for maternal/neonatal related conditions	•Sample DHH English-speaking women who have a recent history of pregnancy

Note. This joint display was condensed for illustrative purposes.

One area of concern during the development of this joint display was the weighting of priorities across groups and acknowledgment of the difficulty in developing sampling schemes with a relatively large number of purposeful sampling priorities. Some priority areas were developed based on the sole recommendation of the community advisory group. For example, community advisory group members recommended sampling late-deafened individuals (i.e., DHH people who have adult ages of hearing loss onset) and DeafBlind individuals (i.e., people have both hearing and vision loss, and a cultural sub-group within DHH communities). Given the small prevalence of DeafBlind individuals, it would be difficult to use electronic health record data in the quantitative aim to investigate disparities in ED utilization outcomes. Therefore, the research team chose to prioritize this area based on the expertise of the community advisory group feedback. Other areas, however, were not addressed. For example, the research team, community, and quantitative results pointed to the health status outcome of mental health conditions. Due to perceived challenges in sampling participants based on this construct plus other criteria during a pandemic, in addition to the high prevalence of mental health conditions among DHH individuals we chose to not prioritize mental health conditions.

The final priority areas had strong alignment with researcher- and community-indicated priorities: (1) ensuring adequate sampling of Black participants, (2) sampling from rural areas, (3) including the diversity of DHH identities, (4) including people from low-income backgrounds, and (5) sampling people who believe substance use is an issue in their communities.

Implementing the Sampling Plan

Recruitment took place through referral from community advisory group members, disseminating advertisements through community-based partner organizations (e.g., the North Central Florida Signing Alliance and the Hearing Loss Association of America—Gainesville Chapter), and sharing advertisements through social media. In addition, to mitigate the effect of the SARS-CoV-2/COVID-19 pandemic on recruitment and to better achieve purposeful sampling (in our case, criterion sampling) with respect to the sampling priorities identified through the joint display, researchers partnered with HealthStreet—a community-engaged research referral service through the University of Florida’s Clinical and Translational Sciences Institute. HealthStreet is a research and social services referral program that recruits community members from across Florida at grocery stores, community events, barbershops, libraries, and other community locations (Cottler et al., 2013). HealthStreet participants complete an intake form and are added to a database for future research contact. Based on HealthStreet’s methodology, DHH ASL-users were not included, but DHH English-speakers were. This was a strength, as the research team had relatively weaker community-based connections with DHH English-speakers.

The research team provided HealthStreet sampling criteria (see Table 2) which were used to identify a prospective participant pool for this study. Prospective participants were contacted by HealthStreet to screen for eligibility in the study before being linked to the research team for qualitative interviews. Due to a decision by HealthStreet staff members, participant selection characteristics were changed without involvement with the research team; this change was identified after noticing that the age of recruited participants was older than anticipated and that the sample was predominantly white. Therefore, the sample of DHH English-speakers was slightly older than expected, and the purposeful recruitment based on race was not achieved. Recruitment, overall, was more difficult than initially anticipated due to the COVID-19 pandemic, with several participants screening eligible (either from HealthStreet or from direct contact) not completing the study.

Table 2.

Purposeful Sampling Priorities Applied in HealthStreet Sample Selection of DHH English-Speakers.

Characteristic	Number of Participants
Age
18–44 years old	5
45–64 years old	3
65 years or older	2
Race
Black	5
White and/or Hispanic	4
Other race	1
Food insecure	5
Medicaid insured or uninsured	4
Pregnant (with due date after October 2017)	3
Lives in a rural area	4
Neighborhood concerns of substance use	5

Note. Categories are not mutually exclusive.

Discussion

This article has sought to demonstrate the application of transformative paradigm sampling considerations in an explanatory sequential design. In doing so, we have addressed the call to integrate across multiple dimensions, to advance MMR designs, and have developed a method of methodologically aligning the transformative paradigm in the explanatory sequential design.

Contribution to the Field of Mixed Methods Research Methodology

The method described in this paper contributes to MMR methodology in four primary ways: (1) centering the transformative paradigm throughout a MMR study; (2) including transformative considerations in joint displays; (3) providing a structure to improve reporting in mixed methods community-based participatory research (MMCBPR) designs; and (4) increasing transparency in MMR reporting.

Centering the Transformative Paradigm Throughout a MMR Study

The transformative paradigm encourages researchers to prioritize constructs often not available in quantitative data, while integrating through connecting in explanatory sequential designs hinges on the use of quantitative results in developing sampling goals. This paper has demonstrated that the goal and practical integration of transformative paradigm tenets in the explanatory sequential design need not be incompatible. Instead, we argue that the integration of transformative paradigm concepts in the explanatory sequential design, through existing tools (i.e., the participant selection joint display), creates a more rigorous design that integrates across multiple dimensions.

This integration across multiple dimensions is imperative when considering the development and application of the transformative paradigm. The transformative paradigm can be readily applied in quantitative and qualitative aims, but it may not always be centered during mixed methods procedures including methods-level integration (e.g., connecting). The use of the participant selection joint display with transformative considerations addresses the need for paradigmatic alignment across quantitative, qualitative, and mixed methods strands, and keeps the transformative paradigm at the center of methodological decisions and research procedures. A description of how this method lends to the legitimation (or validity) of MMR studies is described in Table 3.

Table 3.

Relevance of the Proposed Participant Selection Joint Display with Transformative Considerations to the MMR Legitimation Framework.

Legitimation Dimension	Definition	Relevance to the Proposed Method
Integration	“The degree to which the researcher has successfully integrated quantitative and qualitative data, analysis, and conclusions” (Perez et al., 2022, p. 8)	The participant selection joint display is a tool to facilitate integration between quantitative results and qualitative sampling scheme development. The presented method innovates on existing joint displays to demonstrate the integration of paradigmatic considerations
Multiple stakeholder	“The degree to which the researcher has addressed the interests of different stakeholders (e.g., quantitative, qualitative, and MM researchers, policymakers, and groups with little power) in the report and generation of high-quality meta-inferences” (Perez et al., 2022, p. 8)	The transformative paradigm calls for the inclusion of community members throughout the research process. The proposed joint display method provides insight on how stakeholders, including the researcher and multiply marginalized community members, can provide guidance in the development of qualitative sampling criteria
Paradigmatic mixing	“The extent to which the researcher’s epistemological, ontological, axiological, methodological, and rhetorical beliefs that underlie the quantitative and qualitative approaches are successfully (a) combined or (b) blended into a usable package” (Onwuegbuzie & Johnson, 2006, p. 57,59)	As applied in the quantitative phase of the example study, it is also possible to include transformative paradigm ideals in the quantitative phase. The joint display method presented encourages the integration of transformative paradigm constructs in the connecting phase of explanatory sequential studies, strengthening the application of the paradigm
Weakness minimization	“The extent to which the weaknesses from one approach is compensated by the strengths from the other approach” (Onwuegbuzie & Johnson, 2006, pp. 57–58)	Quantitative data, particularly existing quantitative data, lack equity-relevant variables. The proposed method encourages considering what equity-relevant characteristics should be sampled in the qualitative phase of explanatory sequential designs

Including Transformative Considerations in Joint Displays

Participant selection joint displays are one of the five major types of joint displays and, although underutilized, are useful when facilitating connecting across samples (Guetterman et al., 2021). A critique of joint displays in the explanatory sequential design has been that they appear rudimentary relative to other joint displays (Guetterman, Creswell, & Kuchartz, 2015). For example, some participant selection joint displays simply present the characteristics of respondents or sites sampled (Guetterman, Creswell, & Kuchartz, 2015; Ivankova et al., 2006).

The method outlined in this paper innovates on these previous displays in two primary ways. First, we extend existing participant selection joint displays to clearly identify the linkage between quantitative findings and qualitative sampling priorities; this can be applied in any explanatory sequential MMR study. Secondly, we demonstrate the inclusion of transformative sampling considerations in the development of qualitative sampling priorities, centering the transformative paradigm. In this way, the participant selection joint display with transformative considerations intentionally integrates paradigm and practical study design decisions, facilitating stronger integration in MMR studies applying the transformative paradigm.

Providing a Structure to Improve the Reporting of Integration in Community-Based Participatory MMR Designs

Although not applied in the present study, mixed method community-based participatory research (CBPR) designs, known as MMCBPR, are an advanced application of MMR within participatory and action research studies and often align with the transformative paradigm (DeJonckheere et al., 2018). In CBPR, community members are positioned as co-researchers who make research decisions in partnership with academic researchers. Therefore, MMCBPR designs are directly relevant to the transformative paradigm, where inclusion of community members throughout the research process and focusing on community priorities can help identify opportunities to improve social justice. In a review of 129 studies, DeJonckheere et al. (2018) found that 72% of MMRCBPR studies used connecting as an integration strategy. However, they also identified a need for more detailed and comprehensive descriptions of how data are integrated in MMCBPR designs, including how specific methodological decisions are influenced by CBPR principles. The method outlined in this paper can be used to describe how integration through connecting is conducted throughout the study, reporting how community priorities are centered.

Using the Joint Display to Promote Transparency

There is increasing recognition of the importance of transparency to improve reproducibility and replicability of studies throughout all fields of science (Levitt et al., 2018; United Nations Educational, Scientific and Cultural Organization, 2020). Transparent reporting is crucial in assessing rigor in MMR studies, while also promoting replicability and open science. The use of a participant selection joint display with transformative considerations demystifies research decisions typically unreported in transformative MMR studies (Canales, 2013; Sweetman et al., 2010). For example, in MMCBPR designs, there is a lack of reporting/writing about integration during results. It is possible that this lack of reporting integration in MMR studies applying the transformative paradigm are also underreporting their integration strategies. However, this should be subject to formal study through a systematic methodological review.

Methodological Limitations

There are several methodological limitations to this approach. First, there is no empirical evidence on the appropriateness of “weighting” a transformative consideration versus a quantitative finding. For example, in the absence of quantitative data specific to ED utilization across DHH patient sub-groups (e.g., people who are DeafBlind)—we deferred to community advisory group members’ expertise and maintained this prioritization in the qualitative phase. This is consistent with the transformative paradigm’s tenet on respect for affected communities, leading researchers to prioritize the affected community’s perspective. Relatedly, there is also a question of the exhaustiveness of researcher-defined transformative considerations. For example, quantitative results suggested a clear disparity among DHH English-speaking women who had maternal/neonatal related conditions. This group was not explicitly identified for prioritization by researchers nor community advisory group members. In line the explanatory sequential design, this characteristic was prioritized for qualitative sampling to help explain quantitative results. A post-hoc reflection of transformative considerations, however, supports the prioritization of this group. If the researchers were to weigh incomplete transformative reflection higher than the need to explain quantitative results, they may have unintentionally excluded a population that provided strong qualitative insights for their research question.

In addition, the integration of transformative considerations through a participant selection joint display seeks to ensure transformative sampling goals are achieved in the qualitative phase. Transformative sampling, however, should also be considered when conducting quantitative studies. This is not addressed in the present example; however, transformative considerations should be identified and defined during these phases. This may include using a QuantCrit lens (Garcia et al., 2018; Gillborn et al., 2018), being intentional with sampling in quantitative studies, and recognizing areas where quantitative data are lacking. Researchers using the explanatory sequential design, regardless of paradigmatic orientation, should remain cognizant of the limitations of the quantitative study and how those limitations may impact the development of qualitative sampling schemes. For example, in the present study, many equity-relevant variables were not collected in the electronic health record. Without considering the limitations of the data that were available, the research team may have inadvertently ignored patient characteristics meaningful for explaining results. Ignoring this consideration would have weakened the “weakness minimization” legitimation criteria—where researchers seek to minimize the weaknesses of one approach with another (Onwuegbuzie & Johnson, 2006). Lastly, additional studies are needed to identify methodological decisions on choosing which results to further explain through the qualitative phase of explanatory sequential designs.

Recommendations for Best Practices

We have developed several recommendations to aid researchers in applying this method (see Table 4).

1. Identify and define the research team’s transformative sampling considerations. First, we recommend researchers answer the question recommended by Mertens (2009, p. 201): “What are the dimensions of diversity that are important in [your] study?” In this process, researchers should clearly define all transformative considerations of importance for use in the study. As identified in the limitations, however, attention should be provided at the level of exhaustiveness of this list based on theory, resources, and the researchers’ intents. These considerations should then be transparently reported.

2. Identify and define the community advisory group’s transformative sampling considerations. Community advisory group members who are new to research, particularly qualitative research, should be trained on the importance of seeking different perspectives. As described in this study, researchers used the parable of the blind men and an elephant. When seeking community advisory group feedback, there are several important considerations. First, will their considerations be sought in a group environment or individually? This decision should be considered similar to the methodologic decision of conducting qualitative focus groups or individual interviews (e.g., if group interaction is important). In a group setting, community advisory group members may feel more comfortable—surrounded by other community members—to speak up or challenge a researcher’s beliefs. However, group settings may also serve a role of oppressing viewpoints by members facing intersectional oppression. For example, a group member may be more open about concerns regarding racism, ableism, and other forms of oppression in a one-on-one setting. (This area should be the subject of research on the influence of group dynamics on the development of transformative sampling considerations.) Regardless of the method used, the considerations should be reported without reducing the analytic dimensions provided by community members.

3. Identify quantitative results requiring explanation. As described in the Limitations, we have concerns regarding the weighting of quantitative results and transformative sampling priorities in the explanatory sequential design. However, it remains important to implement the MMR design as intended; in the explanatory sequential design, explaining quantitative results through qualitative inquiry. Additional research is needed on the linking of quantitative results to qualitative sampling criteria, and how researchers choose which results are pertinent. These decisions should be transparently reported by researchers applying this method.

4. Integrate transformative sampling considerations with quantitative results to identify qualitative sampling priorities. Researchers should integrate all of the aforementioned data in a participant selection joint display with transformative considerations. In developing these joint displays researchers may choose to link transformative considerations, quantitative results, and subsequent qualitative sampling priorities across constructs from a theoretical model. These joint displays should be included in publications to increase transparency. Researchers facing word count limitations should include this as a supplemental document.

5. Implement qualitative sampling plan. After identifying qualitative sampling priorities, researchers should identify unique and strategic partnerships to increase the potential to achieve their transformative sampling scheme(s). When reporting the final results, it is imperative to describe which considerations were met, the potential reasons why some were not met (if any), and how exclusion of certain participants may change results.

Table 4.

Process of Integrating Transformative Considerations When Sampling in Explanatory Sequential Mixed Methods Designs.

Step	Recommendations
1. Identify and define the research team’s transformative sampling considerations	•Answer the question: “What are the dimensions of diversity that are important in [their] study?” (Mertens, 2009, p. 201) •Clearly define all transformative considerations for use in the study •Report the research team’s considerations
2. Identify and define the community advisory group’s transformative sampling considerations	•Train community advisory group members on the importance of seeking different perspectives. •Determine if community advisory group discussion of sampling considerations will be group-based or individual •Report the community advisory group’s considerations, without reducing the analytic dimensions provided by community members
3. Identify quantitative results requiring explanation	•Additional research is needed on the linking of quantitative results to qualitative sampling criteria—choosing which results are pertinent, transparently reporting
4. Integrate transformative sampling considerations with quantitative results to identify qualitative sampling priorities	•Develop a participant selection joint display, including the transformative considerations •Report the participant selection joint display in publications to increase transparency
5. Implement qualitative sampling plan	•Identify unique and strategic partnerships to increase achievement of sampling across transformative considerations •Report when transformative sampling considerations were not met and contributing reasons why

Conclusion

The transformative paradigm’s sampling tenets can be applied in explanatory sequential mixed methods studies by integrating through a participant selection joint display. Due to the transparency and methodological rigor of this method, we put forth this technique with the hope of it being accepted as best practice in transformative explanatory sequential mixed methods studies.

Footnotes

Editorial Team Note

This article was submitted to the journal before Timothy C. Guetterman started his role as coeditor, and he was entirely removed from the peer review and acceptance process.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported by grant number 1R36HS027537 awarded to TGJ from the Agency for Healthcare Research and Quality, and the National Center for Advancing Translational Sciences of the National Institutes of Health under University of Florida Clinical and Translational Science Awards UL1TR000064 and UL1TR001427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or of the National Institutes of Health.

ORCID iDs

Tyler G. James

Melissa DeJonckheere

Timothy C. Guetterman

Note

References

Camacho

(2020). From theory to practice: Operationalizing transformative mixed methods with and for the studied population. Journal of Mixed Methods Research, 14(3), 305-335. https://doi.org/10.1177/1558689819872614

Canales

(2013). Transformative, mixed methods checklist for psychological research with Mexican Americans. Journal of Mixed Methods Research, 7(1), 6-21. https://doi.org/10.1177/1558689812446022

Cottler

L. B.

Striley

C. W.

O’leary

Ruktanonchai

Wilhelm

(2013). Engaging the community in research with the HealthStreet model: National and international perspectives. In Translational Medicine-what, why and how: An international perspective (3, pp. 98-109). Karger Publishers.

Creswell

J. W.

Plano Clark

V. L.

(2017). Designing and conducting mixed methods research. Sage Publications.

DeJonckheere

Lindquist-Grantz

Toraman

Haddad

Vaughn

L. M.

(2018). Intersection of mixed methods and community-based participatory research: A methodological review. Journal of Mixed Methods Research, 13(4), 481-502. https://doi.org/10.1177/1558689818778469

Fetters

M. D.

(2020). The mixed methods research workbook: Activities for designing, implementing, and publishing projects (1 edition). SAGE Publications, Inc.

Fetters

M. D.

Curry

L. A.

Creswell

J. W.

(2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6pt2), 2134-2156. https://doi.org/10.1111/1475-6773.12117

Fetters

M. D.

Molina-Azorin

J. F.

(2017). The Journal of Mixed Methods Research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291-307. https://doi.org/10.1177/1558689817714066

Garcia

N. M.

López

Vélez

V. N.

(2018). QuantCrit: Rectifying quantitative methods through critical race theory. Race Ethnicity and Education, 21(2), 149-157. https://doi.org/10.1080/13613324.2017.1377675

10.

Gillborn

Warmington

Demack

(2018). QuantCrit: Education, policy, ‘Big Data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158-179. https://doi.org/10.1080/13613324.2017.1377417

11.

Glickman

N. S.

Hall

W. C.

(Eds.) (2019). Language deprivation and deaf mental health. Routledge.

12.

Guetterman

T. C.

Creswell

J. W.

Kuckartz

(2015). Using joint displays and MAXQDA software to represent the results of mixed methods research. In McCrudden

M. T.

Schraw

Buckendahl

(Eds.) Use of visual displays in research and testing: Coding, interpreting, and reporting data (pp. 145-177). Information Age Publishing, Inc.

13.

Guetterman

T. C.

Fàbregues

Sakakibara

(2021). Visuals in joint displays to represent integration in mixed methods research: A methodological review. Methods in Psychology, 5, 100080. https://doi.org/10.1016/j.metip.2021.100080

14.

Guetterman

T. C.

Fetters

M. D.

Creswell

J. W.

(2015). Integrating quantiative and qualitative results in health science mixed methods research through joint displays. The Annals of Family Medicine, 13(6), 554–561. https://doi.org/10.1370/afm.1865

15.

Ivankova

N. V.

Creswell

J. W.

Stick

S. L.

(2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20. https://doi.org/10.1177/1525822X05282260

16.

James

T. G.

Argenyi

M. S.

Guardino

D. L.

McKee

M. M.

Wilson

J. A.

Sullivan

M. K.

Griest Schwartzman

Anderson

M. L.

(2022). Communication access in mental health and substance use treatment facilities for deaf American Sign Language users. Health Affairs, 41(10), 1413-1422. https://doi.org/10.1377/hlthaff.2022.00408

17.

James

T. G.

Coady

K. A.

Stacciarini

J.-M. R.

McKee

M. M.

Phillips

D. G.

Maruca

Cheong

(2022). They’re not willing to accommodate Deaf patients”: Communication experiences of Deaf American Sign Language users in the emergency department. Qualitative Health Research, 32(1), 48-63. 10.1177/10497323211046238

18.

James

T. G.

McKee

M. M.

Miller

M. D.

Sullivan

M. K.

Coady

K. A.

Varnes

J. R.

Pearson

T. A.

Yurasek

A. M.

Cheong

(2022). Emergency department utilization among deaf and hard-of-hearing patients: A retrospective chart review. Disability and Health Journal, 15(3), 101327. 10.1016/j.dhjo.2022.101327

19.

James

T. G.

McKee

M. M.

Sullivan

M. K.

Ashton

Hardy

S. J.

Santiago

Phillips

D. G.

Cheong

(2021). Community-engaged needs assessment of deaf American Sign Language users in Florida, 2018. Public Health Reports, 137(4), 730-738. https://doi.org/10.1177/00333549211026782

20.

James

T. G.

Miller

M. D.

McKee

M. M.

Rotoli

Sullivan

M. K.

Pearson

T. A.

Mahmoudi

Varnes

J. R.

Cheong

(2022). Emergency department condition acuity, length of stay, and revisits among deaf and hard-of-hearing patients. Academic Emergency Medicine, 29(11), 1290-1300. https://doi.org/10.1111/acem.14573

21.

James

T. G.

Varnes

J. R.

Sullivan

M. K.

Cheong

Pearson

T. A.

Yurasek

A. M.

Miller

M. D.

McKee

M. M.

(2021). Conceptual model of emergency department utilization among deaf and hard-of-hearing patients: A critical review. International Journal of Environmental Research and Public Health, 18(24), 12901. https://doi.org/10.3390/ijerph182412901

22.

Levitt

H. M.

Bamberg

Creswell

J. W.

Frost

D. M.

Josselson

Suárez-Orozco

(2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 26. https://doi.org/10.1037/amp0000151

23.

McKee

M. M.

Barnett

Block

R. C.

Pearson

T. A.

(2011). Impact of communication on preventive services among deaf American Sign Language users. American Journal of Preventive Medicine, 41(1), 75-79. https://doi.org/10.1016/j.amepre.2011.03.004

24.

Mertens

D. M.

(2007). Transformative paradigm: Mixed methods and social justice. Journal of Mixed Methods Research, 1(3), 212-225. https://doi.org/10.1177/1558689807302811

25.

Mertens

D. M.

(2009). Transformative research and evaluation. The Guilford Press.

26.

Mertens

D. M.

(2012). Transformative mixed methods: Addressing inequities. American Behavioral Scientist, 56(6), 802-813. https://doi.org/10.1177/0002764211433797

27.

Mertens

D. M.

(2018). Transformative mixed methods and policy evaluation. Diritto & Questioni Pubbliche, 18, 247-264.

28.

Onwuegbuzie

A. J.

Burke Johnson

(Eds.) (2021). The Routledge reviewer’s guide to mixed methods analysis (1st ed.). Routledge.

29.

Onwuegbuzie

A. J.

Collins

K. M.

(2007). A typology of mixed methods sampling designs in social science research. Qualitative Report, 12(2), 281-316. https://doi.org/10.46743/2160-3715/2007.1638

30.

Onwuegbuzie

A. J.

Johnson

R. B.

(2006). The validity issue in mixed research. Research in the Schools, 13(1), 48-63. http://www.msera.org/docs/rits-v13n1-complete.pdf#page=55

31.

Perez

Howell Smith

M. C.

Babchuk

W. A.

Lynch-O’Brien

L. I.

(2022). Advancing quality standards in mixed methods research: Extending the legitimation typology. Journal of Mixed Methods Research, 17(1), 29-50. https://doi.org/10.1177/15586898221093872

32.

Sweetman

Badiee

Creswell

J. W.

(2010). Use of the transformative framework in mixed methods studies. Qualitative Inquiry, 16(6), 441-454. https://doi.org/10.1177/1077800410364610

33.

Tashakkori

Johnson

R. B.

Teddlie

(2021). Foundations of mixed methods research (2nd ed.). SAGE Publications, Inc.

34.

Tashakkori

Teddlie

(2003). Handbook of mixed methods in social and behavioral research. SAGE.

35.

United Nations Educational, Scientific and Cultural Organization (2020). UNESCO, WHO and the UN high commissioner for human rights call for “open science”. United Nations Educational, Scientific and Cultural Organization. https://en.unesco.org/news/unesco-who-and-high-commissioner-human-rights-call-open-science