Mixed Methods Research for Social Betterment and a More Just World

Abstract

Mixed methods researchers who make a commitment to use their work for social betterment take a stance that has implications for their designs. This article contributes to mixed methods by illustrating how designs informed by the transformative and Indigenous paradigms provide examples of both the value of pursuing this goal, as well as the challenges that mixed methods researchers face. Social betterment studies engage with marginalized and vulnerable populations, such as Indigenous nations, people with disabilities, women and girls, and sexual minorities. Insights from the transformative and Indigenous paradigms are provided and illustrated by mixed methods studies. Challenges are discussed along with resources that can help address these challenges.

Keywords

transformative Indigenous people with disabilities LGBTQ community engagement

The world faces many grand challenges in the form of intransigent problems for which no single solution is agreed upon (Mertens, 2015). These challenges include, but are not limited to, the status of refugees and immigrants; violence and conflict; the climate crisis and its social implications; gaps in wealth between rich and poor; disparities in economic, education, and health systems; and violations of the rights of racial and ethnic minorities, women and girls, sexual minorities, and people with disabilities (Mertens, 2018a; 2023a). The role of mixed methods researchers in addressing these grand challenges was recognized by the Mixed Methods International Research Association Task Force on the Future of Mixed Methods (Mertens, et al., 2016). Molina-Azorin and Fetters (2019; 2021; Fetters & Molina-Azorin, 2021), along with other mixed methods scholars (Guetterman et al., 2023; Mertens, 2024), reinforced the importance for mixed methods researchers to critically examine their methodologies to address how to increase the impact of their work as a strategy towards social betterment and increased justice.

Every researcher, at some point in the process of planning a study, articulates a statement of purpose. Researchers who establish a purpose that seeks to challenge current and historical forces, both systemic and structural, that support an oppressive status quo, take a stance that their research purpose is to support transformative change to increase justice and further social betterment. The global community recognizes that particular constituencies have historically been marginalized; the adoption of Sustainable Development Goals (SDGs) has an explicitly stated purpose to “leave no one behind” (United Nations, 2015). The populations identified as being left behind include Indigenous, youth, women and girls, sexual minorities, and people living in poverty, conflict, and adversity. By committing to the SDGs, the global community demonstrates the necessity of modifying our research so that it works in the interests of furthering social betterment in the form of increased social, economic, and environmental justice.

Two paradigms that support the use of mixed methods center approaches that explicitly challenge researchers to address issues of justice: the transformative paradigm and the Indigenous paradigm (Chilisa, 2020; Chilisa & Mertens, 2021; Cram & Mertens, 2015; Mertens, 2023, 2023b; Mertens & Cram, 2015). Other paradigmatic frameworks can also be used in studies that seek to further social betterment, especially when they integrate aspects of the transformative and Indigenous paradigms. In this article, the assumptions of the transformative and Indigenous paradigms are explored, especially as they apply to supporting actions that increase justice. Then, examples of mixed methods studies are used to illustrate how this community can be part of the solution to the identified grand challenges.

The Transformative Paradigm

The use of a paradigmatic framework allows researchers an opportunity to critically reflect on the assumptions that guide their methodological decisions. The transformative paradigm begins with an axiological assumption that clarifies the values and ethics that underlie a study designed to further social betterment (Mertens, 2018b; 2021; 2023a; 2024; Mertens, et al., in press). The transformative axiological assumption highlights the need to conduct culturally responsive studies that explicitly address social, economic, and environmental justice by unmasking inequities and adapting methods to reflect community priorities. The transformative ontological assumption about the nature of reality challenges researchers to identify those versions of reality that sustain an oppressive status quo and to reveal versions of reality that provide a pathway to increased justice. The transformative epistemological assumption calls researchers to challenge power structures that exclude those who are traditionally vulnerable and marginalized and to structure their studies so that their voices are included in ways that are legitimate and safe and to value community-based knowledge.

These three assumptions then lead to a methodological assumption that a transformative lens should inform the design and conduct of the studies and that mixed methods are generally useful to enable researchers to be responsive to the needs of different constituencies (Mertens, 2021). The methodological implications include building culturally responsive relationships, supporting transformative change to increase justice, consciously addressing inequities, challenging discrimination, and addressing power inequities in ways that increase the probability of sustaining change at the end of the study. Design wise, this means researchers need to provide time to build culturally responsive relationships and conduct a contextual analysis of the historical and current factors that inhibit and support transformative change. Designs need to be flexible to be responsive to community input and to changing conditions that are revealed by early data collection. Thus, no one design defines transformative methodologies. Researchers can use action research designs that focus on a specific community-identified change, or other mixed methods designs that address furthering justice and social betterment as a part of relationship building and development of strategies to support sustainable change that goes beyond the research study itself.

The Indigenous Paradigm

While the Indigenous paradigm shares some common ground with the transformative paradigm (Chilisa & Mertens, 2021; Cram & Mertens, 2015; Mertens, 2023b), it also reflects the unique history and cultures of Indigenous nations (Waapalaneexkweew (Bowman) & Dodge-Francis, 2020; Chilisa, 2020). Indigenous nations experienced colonization throughout the globe that resulted in and continues to sustain loss of land, discrimination, and denial to their cultural heritage. Thus, the Indigenous perspective on social betterment and justice includes decolonization, restoration of land rights, and the right to live as Indigenous people with their own culture and language (Chouinard & Cram, 2018). The Indigenous axiological assumption is relational and reflects the values of respect, reverence, responsibility, reciprocity, reflexivity, responsiveness, and decolonization (Chilisa, 2020; Chouinard & Cram, 2018). This concept of relationality goes beyond relationships between persons and is inclusive of human relationships with nature. The Indigenous ontological assumption adds two important aspects to the nature of reality: spirituality and the interconnectedness of all living and nonliving things (Cram et al., 2018). The Indigenous epistemological assumption aligns with their axiological and ontological assumptions by emphasizing the importance of forming respectful relationships and valuing the Indigenous knowledge and ways of knowing that comes from their communities.

The Indigenous methodological assumption derives from the other assumptions in that methodologies should be decolonizing, use traditional Indigenous strategies for community engagement, and be designed to nurture relationships that value the community’s strengths and culture ((Chilisa, 2020). Indigenous researchers make visible and challenge aspects of their culture that sustain oppression, such as beliefs about the subjugation of women (Chilisa & Tsheko, 2014). Indigenous knowledge should inform all aspects of the research studies, including engaging with people using appropriate cultural protocols and structuring the study so Indigenous knowledge is brought to bear on the study design, purpose, questions, methodologies, and data collection, analysis, interpretation, and use.

Permeable Borders Across Paradigms

Indigenous and transformative researchers advocate for appropriate inclusion of methodologies that are associated with other paradigms, such as the post-positivist, constructivist, and pragmatic paradigms. This might be characterized as the permeability of borders across paradigms, as is reflected in this excerpt:

For example, the African ethical principle of motho ke motho ka batho holds that [researchers]…have an ethical responsibility to design their work to support positive transformation in the human and physical world because we are all related. This aligns with the pursuit of social, economic, and environmental justice as an ethical remit in [research]…, an assumption found in the transformative paradigm (Mertens & Wilson, 2019). In the [Made in Africa] MAE ethical view, there are no boundaries between knowledge systems; thus, it can be integrative, bringing together Western and Indigenous perspectives. It promotes global partnerships of knowledge systems and of evaluation actors and stakeholders. It seeks to stamp out decontextualized [research]…and the silencing of non-Western ontological, epistemological, and axiological assumptions in…. [research] (Chilisa & Mertens, 2021, p. 246).

In research for social betterment, the assumptions of the transformative and Indigenous paradigms are used to frame mixed methods studies. In some studies, the integration across paradigms is explicitly discussed. Crossing borders between paradigms brings challenges, but it provides an opportunity to increase the impact of mixed methods research to support social betterment (Mertens, 2023a; Chilisa & Mertens, 2021; Cram & Mertens, 2015; Mertens, 2023b).

Mixed Methods and Social Betterment Examples

In the next section, examples of mixed methods research are used to illustrate the application of the assumptions of the transformative and Indigenous paradigms in studies that have an explicit social justice and betterment goal. These studies include strategies for building culturally responsive relationships and community engagement, recognizing the historical and current contextual factors that support an oppressive status quo, addressing power inequities, and increasing the potential for transformative action.

Transformative and Indigenous Frameworks Inform Design

Lucero et al. (2018) used a combination of the Indigenous and transformative paradigms to design their mixed methods study to address health disparities in the Native American community. Both Fetters (2020) and Mertens (2023a) identified this study as exemplifying adaptations to mixed methods approaches that have a goal of social betterment. Lucero and colleagues formed relationships through connections with Native American organizations that consciously addressed issues of sharing power, building trust, cultural identity, spiritual protocols, and participatory decision making. They conducted a contextual analysis that revealed the low socioeconomic status in the area, institutional racism, and aspects of the Native American history that created a lack of trust with the health care community. Lucero et al. described how their community-based participatory research (CBPR) reflected both the Indigenous and transformative paradigms:

While CBPR as a whole has adopted transformative ideas of endorsing community partners as knowledge co-creators and of applying research findings toward social justice, indigenous methodologies add more focused attention to culturally driven epistemologies and community control over research such as ensuring tribal (or community) data ownership after research is completed (p. 60).

One key element included the use of a cyclical design such that early data collection and discussion with community members influenced subsequent stages of the study. Based on their consultation with communities and a preliminary case study, the researchers changed their original design that was going to begin with a quantitative survey. The qualitative data from the early stages of the study revealed the need to conduct additional case studies to inform the development of a quantitative survey that would be more culturally relevant and responsive. The researchers’ willingness to modify the design served to build trust between them and community members and set the stage for greater ownership of the data that could be used to reduce health disparities in this marginalized community.

Building Coalitions and Challenging an Oppressive Status Quo

A contextual analysis of United States legislation and policies revealed that the historical framing of strategies to address drug problems as a War on Drugs had resulted in an over-representation of people of color being incarcerated (Mertens, 2018b). Legislation passed in 2003 reframed drug-related offenses as a public health issue and provided funding for mental health courts and jail diversion programs (America’s Law Enforcement and Mental Health Act, Public Law 106-515). The contextual analysis was part of a transformative mixed methods study to determine the impact of a jail diversion program in a mental health court in Salem, Oregon. The court, under Judge Mary James, undertook a process that began with building relationships with the key players who were needed to navigate the justice system and to provide support for participants in the diversion program.

Qualitative (legislation and court policies) and quantitative (statistics of incarceration by race) data were collected as part of the contextual analysis. Then qualitative data were collected through interviews with the judge, as well as by observations in the courtroom and planning meetings of the support teams for each program participant. These data revealed the need to tailor support teams to meet individual participants’ needs. The results also indicated that a support team was necessary, but an effective program also had to be built on respectful relationships between the judge, other court personnel, and the participants. As Judge James explained:

It is not that the type of treatment intervention is not as important, but it is more impactful that people feel the judge and others pay attention to them and help them adopt behaviors that make their lives better. The amount of contact with the judge is important; the tone of the court is important. There is a perception of fairness (Judge James, cited in Mertens, 2018b, p. 8).

Data collection challenges were acknowledged in the study as people with mental illness and drug addiction may not be easily accessible or willing to provide data for a study. The data also revealed that modifications of services were needed for particular groups, for example, being gay and not willing to admit that in open court or being a veteran with Post-Traumatic Stress Disorder. Quantitative data were collected on recidivism and costs of the program. These data can be used to strengthen the argument that mental health courts and diversion programs should continue to receive support and be expanded to a broader constituency.

Community Engagement With Youth

NeMoyer et al. (2021) described a multilevel participatory social justice mixed methods design in their study of the effectiveness of PhotoStories, a project designed to address youth behavioral health problems in four diverse Boston neighborhoods. The data collection included a contextual analysis with quantitative data derived from extant data bases (e.g., the Youth Risk Behavior Survey and the U.S. Census American Community Survey). These data revealed racial and ethnic diversity between neighborhoods, with three of the neighborhoods having a high Latino representation. Qualitative data were also collected from key informants who were providers of youth services in the neighborhoods. A survey for youth was developed and field tested with volunteer youth to measure such variables as health risk and protective factors.

Youth served as coresearchers and were trained in the photovoice data collection method. Qualitative data were collected from youth using the photovoice method. The youth took pictures of things they liked and things they wanted to change in their neighborhoods. The photos then served as a basis for collecting data from interviews in which the youth told researchers about the meaning of the photos. The researchers included a dissemination phase that was designed to facilitate use of the findings for social betterment through community forums to present the youth’s perspectives on assets and challenges and what could be done to better address their needs. By adopting a social justice lens to inform their design, the researchers and the coresearchers contributed to the potential for mixed methods to engage with communities in ways that motivate them to work towards an improvement of services for marginalized populations.

Reaching Marginalized Populations: Women With Disabilities and Transwomen

Stocker et al. (2020) conducted a mixed methods evaluation of an economic development program designed to address the needs of women who work from home in Pakistan. In this country, there is a stigma attached to having a child with a disability and to being a transperson. Hence, the researchers used a transformative lens to adapt their methods to be inclusive of women with disabilities and transwomen. Their initial intent was to use a quantitative survey and to invite the women who work from home to come into centers to be interviewed. However, their cultural analysis revealed that women with disabilities and transwomen seldom, if ever, left their homes because of the social stigma. The researchers changed their methods and used a case study approach that was informed by the women with disabilities and transwomen. This change in their design provided support for a change in the intervention (economic training) to be more locally based and supportive for women with disabilities. Even with these efforts, they were only able to collect data from one transwoman.

Building Coalitions: Gay and Bisexual Men and Transwomen

Miller et al. (2021) provide an example of how to use community activist strategies to be inclusive of hard-to-reach populations. They conducted a mixed methods study of an initiative to improve health care for gay, bisexual men, and transwomen in countries in which manifestation of these sexual identities is against the law, punishable by prison, or death. The unique contribution of this study was that it demonstrates how coalitions can be built through connecting advocacy organizations who have a goal of social betterment (Wolfe & Price, 2023; Community Tool Box, 2024; Treacy & Harder, 2018). The research team worked with an activist agency and identity organizations to identify representatives from this population to participate in the planning of the study. The data collection instruments were constructed to address issues of concern to the community; data collectors from the community were trained to go to clinics. In debriefing sessions, the data collectors reported a high level of stress during their clinic visits due to overt discriminatory behavior by the clinic staff. In response to this, the data collectors were given more training in how to respond in adverse contexts; additional data collectors were trained; and the project provided psychological support for the data collectors. These strategies led to transformative effects for the community, as well as in health clinics.

Discussion

The call for mixed methods researchers to contribute to social betterment provides an opportunity to advance the field, as well as challenges that need attention in future studies. Indigenous scholars (Special issue: Decolonizing evaluation: Towards a fifth paradigm, 2023) challenge researchers in Indigenous communities to integrate their world views inclusive of cultural protocols, dialogue, compassion, psychological safety, space for their voices, valuing Indigenous knowledge, and culturally responsive relationships. Lucero et al. (2018) provide guidance on how to build culturally responsive relationships with tribal members, integrate cultural and spiritual practices, and the importance of being flexible with the mixed methods design in response to community input. Their study also demonstrates the value of integrating paradigmatic perspectives.

Nonindigenous mixed methods researchers have an opportunity to learn from Indigenous scholars how to recognize the importance of spiritual connectedness to reinforce attention to social, economic, and environmental justice. “As the world faces increasing numbers of crises in the form of climate damage, violations of human rights, and inequitable societal structures, Indigenous assumptions that support strong relationships amongst humans and nature are relevant” (Mertens, 2023b, p. 11). Their work has relevance inside Indigenous communities, as well as for researchers who do not work with Indigenous communities. What risks do we take when we ignore the spiritual reality of communities? The Western separation of spirituality and science may be denying the role that spirituality plays in the health of communities, both for people and for the environment. How would we change our designs so that we do consider social, economic, and environmental justice? What do we gain as we explore the meaning of permeability across paradigmatic borders?

Consideration of the use of mixed methods for social betterment has also led to advances in understanding how to include marginalized populations. Results from research in a mental health court (Mertens, 2018b), an economic development project for women with disabilities and transwomen (Stocker et al., 2020), and health clinics for gay and bisexual men and transwomen (Miller et al., 2021) exemplify both the value and challenges of modifying mixed methods approaches to address the goal of social betterment. The pursuit of social betterment for these marginalized populations and for those who have been historically left behind challenges mixed methods researchers to improve their approaches to gender as a nonbinary concept, as well as how to examine the root causes of gender-based inequities (e.g., culturally based gender roles, norms, and power relations).

Advancements in recognizing adaptions of research methods to support inclusion of persons with disabilities (International Fund for Agricultural Development (IFAD), 2022; Kenny et al., 2023; Mertens, 2024; United Nations Evaluation Group (UNEG), 2022) can be used by mixed methods researchers to integrate these supportive strategies into their work to enhance the potential for furthering social betterment for these populations. The international development community recommends consultation with persons with disabilities and advocacy organizations such as the Global Action on Disability Network or the International Disability Alliance. Provision of appropriate accommodations is a step in the right direction, but Kenny et al. (2023) suggest that such consultations need to result in partnerships that influence the design of the study, interventions, and data collection methods using Universal Design principles (Center for Applied Special Technology, 2018). Universal Design is a framework that provides guidance in the development of data collection methods that reduce barriers and provide appropriate accommodations for persons with disabilities. The Universal Design principles include provision of multiple means of engagement and options for sustaining effort and persistence and expression and communication.

Contribution to Mixed Methods Research

Mixed methods designs for social betterment and increased justice look different from mixed methods studies that do not pursue these goals. While the examples presented here illuminate helpful strategies to enhance the potential transformative impact of mixed methods studies, the mixed methods community has an opportunity to respond to calls from Molina-Azorin and Fetters (2019; 2021; Fetters & Molina-Azorin, 2021), along with other mixed methods scholars (Guetterman et al., 2023; Mertens, 2024), to explicitly adapt their designs to enhance justice and further social betterment. This means making a commitment to include the time and resources needed to build culturally responsive relationships, develop coalitions, conduct contextual analyses, and integrate social activist strategies. In this way, we increase the chance that we can address the root causes such as cultural norms and structural barriers that have inhibited the impact of research to this purpose and increase justice for marginalized and vulnerable populations.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Donna M. Mertens

References

Center for Applied Special Technology (CAST) . (2018). Universal design for learning guidelines version 2.2. Retrieved from. https://udlguidelines.cast.org

Chilisa

(2020). Indigenous research methodologies (2nd ed.). Sage.

Chilisa

Mertens

D. M.

(2021). Indigenous Made in Africa evaluation frameworks: Addressing epistemic violence and contributing to social transformation. American Journal of Evaluation, 42(2), 241–253. https://doi.org/10.1177/1098214020948601

Chilisa

Tsheko

G. N.

(2014). Mixed methods in Indigenous research: Building relationships for sustainable intervention outcomes. Journal of Mixed Methods Research, 8(3), 222–233.

Chouinard

J. A.

Cram

(2018). Culturally responsive approaches to evaluation. Sage.

Community Tool Box . (2024). Table of contents. https://ctb.ku.edu/en/table-of-contents/assessment/promotion-strategies/start-a-coaltion/main

Cram

Mertens

(2015). Transformative and Indigenous frameworks for mixed and multi method research. In Hesse-Biber

S. N.

Johnson

R. B.

(Eds.), The Oxford handbook of mixed and multiple methods research (pp. 99–109). Oxford University Press.

Cram

Tibbitts

K. A.

LaFrance

(2018). A stepping stone in Indigenous evaluation. In Cram

Tibbetts

LaFrance

(Eds.), Indigenous evaluation (New Directions for Evaluation) (159, pp. 7–16). Wiley.

Fetters

M. D.

(2020). The mixed methods research workbook. Sage.

10.

Fetters

M. D.

Molina-Azorin

J. F.

(2021). Special issue on COVID-19 and novel mixed methods methodological approaches during catastrophic social changes. Journal of Mixed Methods Research, 15(3), 295–303. https://doi.org/10.1177/15586898211029100

11.

Guetterman

T. C.

Motohara

Molina-Azorin

J. F.

(2023). In this issue: Tribute to Michael Fetters, queer approaches to mixed methods, mixed methods community-engaged evaluation, instrument fidelity and natural language processing, teaching mixed methods, and geographic information system. Journal of Mixed Methods Research, 17(4), 335–339. https://doi.org/10.1177/15586898231202075

12.

IFAD . (2022). IFAD’s disability inclusion strategy 2022-2027. https://www.ifad.org/en/-/ifad-disability-inclusion-strategy-2022-2027#:∼:text=The%20objective%20of%20the%20strategy,all%20pillars%20of%20IFAD's%20work

13.

Kenny

Doyle

Horgan

(2023). Transformative inclusion: Differentiating qualitative research methods to support participation for individuals with complex communication or cognitive profiles. International Journal of Qualitative Methods, 22, 1–17. https://doi.org/10.1177/16094069221146992

14.

Lucero

Wallerstein

Duran

Alegria

Greene-Moton

Israel

Kastelic

Magarati

Oetzel

Pearson

Schulz

Villegas

White Hat

E. R.

(2018). Development of a mixed methods investigation of process and outcomes of community-based participatory research. Journal of Mixed Methods Research, 12(1), 55–74. https://doi.org/10.1177/1558689816633309

15.

Mertens

D. M.

(2015). Mixed methods and wicked problems. Journal of Mixed Methods Research, 9(1), 3–6. https://doi.org/10.1177/1558689814562944

16.

Mertens

D. M.

(2018a). Mixed methods design in evaluation. Sage.

17.

Mertens

D. M.

(2018b). Transformative mixed methods and policy evaluation. Diritto & Questioni Pubbliche, 18(1), 247–264.

18.

Mertens

D. M.

(2021). Transformative research methods to increase social impact for vulnerable groups and cultural minorities. International Journal of Qualitative Research, 20, 1–9. https://doi.org/10.1177/16094069211051563

19.

Mertens

D. M.

(2023a). Mixed methods research. Bloomsbury.

20.

Mertens

D. M.

(2023b). The pursuit of social, economic, and environmental justice through evaluation: Learning from Indigenous scholars and the fifth branch of the evaluation theory tree. Journal of MultiDisciplinary Evaluation, 19(44), 11–23. https://doi.org/10.56645/jmde.v19i44.749

21.

Mertens

D. M.

(2024). Research and evaluation in education and psychology (6th ed.). Sage.

22.

Mertens

D. M.

Bazeley

Bowleg

Fielding

Maxwell

Molina-Azorin

J. F.

Niglas

(2016). Expanding thinking through a kaleidoscopic look into the future: Implications of the mixed methods international research association’s Task force report on the future of mixed methods. Journal of Mixed Methods Research, 10(3), 221–227. https://doi.org/10.1177/1558689816649719

23.

Mertens

D. M.

Cram

(2015). Transformative and Indigenous frameworks for mixed and multi-method research. In Hesse-Biber

Johnson

(Eds.), Oxford handbook of mixed and multiple methods (pp. 91–109). Oxford University Press.

24.

Mertens

D. M.

Hall

Wilson

A. T.

(in press). Program evaluation theory and practice: A comprehensive guide (3rd ed.). Guilford Press.

25.

Mertens

D. M.

Wilson

A. T.

(2019). Program evaluation theory and practice (2nd ed.). Guilford Press.

26.

Miller

R. L.

Rutledge

Ayala

(2021). Breaking down barriers to HIV care for gay and bisexual men and transgender women: The Advocacy and Other Community Tactics (ACT) Project. AIDS and Behavior, 25(2021), 2551–2567. https://doi.org/10.1007/s10461-021-03216-w

27.

Molina-Azorin

J. F.

Fetters

M. D.

(2019). Building a better world through mixed methods research. Journal of Mixed Methods Research, 13(3), 275–281. https://doi.org/10.1177/1558689819855864

28.

Molina-Azorin

J. F.

Fetters

M. D.

(2021). In this issue: Intersection of realist inquiry and structural equation modeling, multilevel participatory social justice mixed methods design, qualitative vignette experiments, mixed methods research systematic methodological reviews, and mixed methods in randomized controlled trials. Journal of Mixed Methods Research, 15(4), 467–469. https://doi.org/10.1177/15586898211049622

29.

NeMoyer

Alvarez

Mukthineni

Tendulkar

Alegria

(2021). Addressing youth-focused research questions in a community context: Collecting and integrating mixed methods data at multiple ecological levels with the PhotoStories project. Journal of Mixed Methods Research, 15(4), 507–525. https://doi.org/10.1177/1558689820972916

30.

Special issue: Decolonizing evaluation: Towards a fifth paradigm. (2023). Journal of Multidisciplinary Evaluation, 19(44). https://doi.org/10.56645/jmde.v19i44

31.

Stocker

F. F.

Nizami

I.A. K.

Sarfraz

(2020). Economic empowerment of women home based workers and excluded groups in Pakistan. UN Women.

32.

Treacy

Harder

(2018). Coalition building and community mobilization. https://digitalcommons.unomaha.edu/cgi/viewcontent.cgi?article=1006&context=step_reports

33.

UNEG . (2022). Guidance on integrating disability inclusion in evaluations. https://www.un.org/sites/un2.un.org/files/2022/06/uneg_guidance_on_integrating_disability_inclusion_in_evaluation_0.pdf

34.

United Nations . (2015, September 25). Resolution adopted by the general assembly: Transforming our world: The 2030 agenda for sustainable development.

35.

Waapalaneexkweew, (Bowman, N., Mohican/Lunaape), & Dodge-Francis, C. . (2018). Culturally responsive indigenous evaluation and tribal governments: Understanding the relationship. New Directions for Evaluation, 159, 17–31. https://doi.org/10.1002/ev.20329

36.

Wolfe

S. M.

Price

A. W.

(2023). Guidebook to community consulting: Cambridge University Press.