A Novel Mixed Methods Approach for Integrating Not-for-Profit Service Data via Qualitative Geographic Information System to Explore Authentic Experiences of Ill-Health: A Case Study of Rural Mental Health

Introduction

International challenges in delivering rural mental health services to meet growing need (Bain & Munoz, 2020) demand novel methods of data collection and analysis. Methods to date are inadequate for capturing rural residents’ lived experiences as these are nuanced by place. People experiencing mental health conditions may be reticent to participate in research due to stigma and lack of confidentiality associated with rural living and mental ill-health (Byrne et al., 2019). Further, the cost of collecting new primary data in multiple rural contexts tends to be prohibitive (Pierce & Scherra, 2012), resulting in small samples over large areas, misleadingly depicting diverse rural experiences as unified (Urban Institute, 2020). As a result, researchers have suggested using mixed methods to integrate previously untapped, complementary secondary datasets—including electronic health data—to address evidence gaps (Ford et al., 2019; Watkins, 2022).

Since the early 2000s, Mixed Methods Research (MMR) has become increasingly deployed in the health sciences (Ivankova & Kawamura, 2010; Plano Clark, 2010) to develop more comprehensive understandings of social phenomena by listening to the “voices” of people affected by ill-health (Guetterman et al., 2015). The rationale behind using mixed methods is that combining qualitative and quantitative approaches leads to an integration of findings and richer detail than either method can generate alone, while simultaneously expanding the scope and breadth of the study (Creswell & Plano Clark, 2017). In other areas of social policy, “data collaboratives” that aim to identify, obtain and integrate complementary mixed datasets (Verhulst & Young, 2017) have emerged as a mixed methods approach to generating new cross-sector partnerships that leverage reused operational data from Not-for-profit organizations (henceforth, NFPs) for “social good” (Susha et al., 2017). There is a gap, however, in mixed methods approaches exemplifying how researchers can partner with NFPs to create data collaboratives in contexts that are data scarce. We therefore aimed to develop a novel mixed methods approach to help researchers understand how to ethically source and integrate NFP datasets to create a data collaborative in contexts of data scarcity. In this paper, we demonstrate how NFPs’ datasets can be ethically sourced, reused, integrated, analyzed, and “”ground-truthed” in a Geographic Information System (GIS) to provide new mixed methods-based sources of insights for rural mental health systems improvement, a particularly topical and distressing global-equity challenge (Bain & Munoz, 2020).

Brief History of Qualitative Geographic Information System Methods

The mixed methods approach we present here builds on a provenance of critical GIS thinking and related qualitative GIS-based methods developed over the last 30 years. Critical GIS was originally developed by human geographers in the 1990s who sought new imaginative possibilities for integrating “critical” human geography within existing GIS architectures (O’Sullivan, 2006). At its core, critical GIS “emerged at the interface between geographic information science and geographical social theory” (Sheppard, 2005, p. 5) in response to the epistemological privilege enjoyed by the physical sciences (Schuurman, 2000). Key contributions to a “new era” of critical GIS (Schuurman & Kwan, 2004) include Pickles (1995) descriptions of maps as representations and assertions of power and Schuurman (2000) critique of positivist approaches to GIS permeating geographical research.

During the 2000s, Knigge and Cope (2006) moved beyond critiques of mainstream positivist approaches to using GIS, instead advocating for qualitative GIS-based methods that “converge” quantitative and qualitative data in GIS representations. This allowed for iteratively building an understanding of how social space is constructed and perceived (Fielding & Cisneros-Puebla, 2009). Other significant developments in the field of qualitative GIS include feminist approaches to geospatial mapping and GIS (Elwood, 2008; Harris, 2015), most notably by Mei-Po Kwan who describes using GIS in tandem with feminist epistemologies and politics (Kwan, 2002). More recent conceptualizations position qualitative GIS as a mixed methods approach that recognizes multiple ontologies, epistemologies, datasets, methods, and visualization techniques that may work in combination or compared with each other, to enhance understandings of social phenomena (Cope & Elwood, 2009; Elwood, 2010; Schuurman & Leszczynski, 2006).

Since the mid-2000s, further evolution of qualitative GIS methods occurred (Hawthorne et al., 2015; Jung & Elwood, 2010; Kamstra et al., 2019; Kwan & Ding, 2008; Kwan & Knigge, 2006), enabling spatial representation of individuals’ experiences and how contextual factors influence their everyday practices (Cope & Elwood, 2009). Of importance are feminist geography approaches like Kwan and Ding’s (2008) geo-narrative approach that uses mixed methods and qualitative GIS (in their original study, to understand the lives of Muslim women in Columbus, Ohio). Similarly, Pavlovskaya (2002) analysis of Moscow’s post-Soviet economy mixed quantitative census data with qualitative interview data to profile Moscow’s informal economy. These mixed datasets were used to inform qualitative maps of Moscow’s informal economy. Following MMR practices, these mixed methods-based qualitative GIS efforts maintain that while all data are partial—they may be useful for understanding some types of phenomenon, but less useful for informing others—they may be “woven” together using different approaches that enable deeper understanding of social issues (Creswell, 1994; Schuurman, 2006).

Benefits of Using Not-for-Profit Data in Mixed Methods Research

Compared with traditional MMR, which is often premised on researchers situating themselves in positions of power by defining the research problem and collecting data to address predetermined research questions (D'Ignazio & Klein, 2020), in our mixed methods approach, we reuse existing geospatial data collected by NFPs through their day-to-day service provision, forgoing the need to solicit “extra data” responses from participants to hear their “voices.” Not-for-profit organizations play a significant global role in actively promoting community wellbeing (Hou & Wang, 2017) and are increasingly harnessed, via government contracts, to provide a range of health and social support services. In recording and storing data generated through their day-to-day addressing of citizens’ needs, NFPs generate datasets that capture citizen’s unsolicited authentic lived experiences (i.e., voices). These “hidden” data give potential to move NFPs’ data collection from a position of providing day-to-day accountability for individual organizations, to offering new mixed datasets that could be reused and integrated in data collaboratives. Under the appropriate ethical conditions, new data repositories generated have potential for informing mixed methods strategies for improving community wellbeing (de Las Casas et al., 2013).

To date, “big” NFP-generated datasets are mainly untapped (Cresswell-Smith et al., 2021). This could be attributed to internal financial and technical constraints to reuse and (qualitatively) analyze their service data for research (Voida et al., 2011), but is mainly due to this being an emergent and innovative practice that is not yet fully mainstream. Early adopter NFPs tend to collaborate with external partners with expertise in ethics (i.e., Institutional Review Boards) and data sharing policies to enable data reuse for research (Tripp et al., 2020). Here, the analysis of NFP service-generated data (e.g., from consumer online forums) demonstrates one way to conduct ethical and affordable secondary research to improve data availability for understanding, and thus addressing, intractable societal problems (Citrin et al., 2017; Gooding, 2017). Simultaneously, this novel mixed methods approach shows NFPs that there are ways to create new value from data they already hold (Masefield et al., 2020). Sourcing and reusing NFP data that was not originally collected for research does, however, involve harnessing emerging research practices about consent, privacy, and responsible reuse of sensitive content (Williams, 2020; Zook et al., 2017). The methods used to navigate these ethical issues are described in this paper (see Figure 1), offering novel and responsible pathways to cost-effective data analysis in MMR internationally.OPEN IN VIEWER

There are multiple benefits from developing new data-driven opportunities for collaboration between NFPs and researchers. These include making the labor of health consumers more visible in research outcomes (D’Ignazio & Klein, 2016) and unlocking new data for social good by bringing social science closer to communities (Fielding & Cisneros-Puebla, 2009). Within this backdrop, and aligned with the aims of partner NFPs, the method presented here captures the voices (i.e., lived experiences) of individuals with mental health challenges in different rural places, while minimizing the financial and emotional costs of new data capture. This mixed methods approach was developed in direct response to gaps in available tools to explore diverse lived mental health experiences in rural Australia (the location of this case study) and to advance understanding of how to reuse existing data, including that produced by NFPs, to better inform service planning (Productivity Commission, 2020).

We use qualitative GIS-based methods and purposefully sampled (Teddlie & Yu, 2007), complementary NFP service data to weave together mixed, inter-related lived experience data in a GIS to inform public service planning. In doing so, we contribute a replicable mixed methods approach that enables geospatial analyses using “big” qualitative datasets in an ethical way, advancing MMR practice by further bridging the gap between spatial and qualitative data (Cope & Elwood, 2009; DeLyser et al., 2009).

Phases of a Novel Mixed Methodology

As a novel approach to mixed methods, we use an explanatory sequential mixed methods design to describe the phases of quantitative geospatial mapping tools that are used to identify geographic areas of interest for further qualitative inquiry. Typically, an initial quantitative phase is conducted before a qualitative investigation in an explanatory sequential mixed methods design (Ivankova et al., 2006). Here, before describing the quantitative phase, we describe the methods used to ethically source NFP service data.

Phase 1 begins with sourcing NFP service-generated data, which involves the navigation of organizational data sharing policies. We established that, to source NFP data for research in Australia, we must ensure that research aims align with aims of each NFP and that service consumers must have consented to their data being reused for research. Having navigated these issues with each NFP’s data manager, we then developed a method to ethically source NFP data, while still addressing our research aims. This involved ensuring that all data would not be shared with third parties, was stored securely on university servers, remained unchanged, was completely de-identified, and any data visualizations would maintain end-user anonymity (Figure 1).

Once agreed, the research practices in Figure 1 were formalized in a research proposal and/or data sharing agreement with corresponding NFPs and Swinburne University, depending on the requirements by NFPs to share their data. These signed documents were then used to apply for and receive university ethics committee approval (R/2019/033).

Phase 2 involves reusing service-generated data for research. When negotiating data reuse with NFPs, ethical compliance must align with organizational data governance and data protection regulations (Figure 1). Data protection laws such as the General Data Protection Regulation specify the conditions for data reuse, defining primary and secondary purpose of data reuse in the European Union (General Data Protection Regulation, 2018). Data collected through service transactions, for instance, can be reused for research, but the purposes of the research must align closely with the primary purpose for the data’s collection and the consent given by subjects of the data, or the licensing arrangements established for further use of the data. This, however, is not always clearly stipulated. Analysis of data reuse opportunities for “open” data sources in the United States, for example, shows significant reuse and interoperability barriers due to insufficient licensing practices (Carbon et al., 2019). This means that, alongside ethical oversight of the research, best practice in reuse of sensitive data (here, about mental health services) involves developing a specific data sharing license or agreement, which can be based on the practices outlined in Figure 1.

Phase 3 involves integrating mixed NFP datasets, which is a key component of MMR (O'Cathain et al., 2007). Here, we integrate NFP data using a “joint display” in a GIS, which “integrates the data by bringing the data together through a visual means to draw out new insights beyond the information gained from the separate quantitative and qualitative results” (Fetters et al., 2013, p. 2143). Integrating NFP data in a GIS using a joint display is useful for demonstrating how mixed NFP datasets can be “woven” together to help generate new insights that is “more than the sum of its parts” (Fetters & Freshwater, 2015) and provides “situational and actionable knowledge” (Saunders & Mulgan, 2017) about place-based issues. To integrate NFP data in a GIS, NFP data were collected with the same XY coordinate geospatial identifier (e.g., post code) so each NFP dataset could be seamlessly imported into a GIS.

Phase 4 involves analyzing NFP data. This includes using quantitative geospatial mapping tools to identify geographic target areas of NFP data that informs further qualitative analysis. Once target geographical areas were identified, thematic coding techniques were used to analyze the qualitative data that is situated within the geographically targeted areas.

Phase 5 requires ground-truthing the results by sharing both mapped geographic areas of interest and thematic coding results with partner NFPs. The aim of this final phase is to identify flawed assumptions of mapped geographic target areas while simultaneously, providing the necessary opportunity for NFPs to assess whether the granularity of joint displays maintains end-user anonymity.

Figure 2 serves to illustrate the explanatory sequential mixed methods design that involves sourcing (Phase 1), reusing (Phase 2), integrating (Phase 3), analyzing (Phase 4), and ground-truthing NFP data (Phase 5) in an ethical way.OPEN IN VIEWER

In the following sections, we describe how we developed and applied this method in a case study about mental health in rural Australia.

Case Study: Mental Health in Rural Australia

There is a persistent gap in Australian rural mental health outcomes including higher rates of suicide, self-harm and emergency mental health hospitalizations, compared with metropolitan areas. Rates increase with geographical remoteness (Australian Bureau of Statistics, 2015); as an example, the rate of suicide among rural men aged 15–29 years is almost twice as high as in major cities (Hickie et al., 2019). Although there are scattered place-based innovations, there is general acknowledgement of a major challenge around delivering a mental health service system that is flexible to local needs and contexts, and that is affordable (Parliament of Australia, 2018).

Lack of choice, distance to specialist services and shortages of skilled local mental health practitioners are some problems persistently noted as associated with poor rural mental health outcomes (Parliament of Australia, 2018; Victoria State Government, 2020). One specific challenge is the multitude of organizations and different types of services that comprise the service system and lack of co-ordination between these services means NFP often compete for government contracts to provide services (Henderson et al., 2018). Coupled with previously mentioned challenges in establishing rural data that represents diverse rural places due to the phenomenon of “ecological fallacy” or “averaging” of data across large administrative regions (Boslaugh, 2007), fragmentation of providers means data for planning is piecemeal, and uncoordinated (Fernandez et al., 2017).

Although there are small-scale experiments with quantitative modelling of rural service availability, to date these mono-methods are not scalable and are based on intensive, expensive collection of new data (Van Spijker et al., 2019). Consequently, the Australian government laments a gap in using data, including existing public data, to better inform service planning—suggesting NFPs might work to reuse data to inform improved service provision (Productivity Commission, 2020).

Methods

Applying a Novel Approach to Mixed Methods to a Case Study

In mapping four mixed national-scale NFP datasets, we draw on an understanding of qualitative GIS as the integrating or weaving of mixed datasets together to inform one another (Cope & Elwood, 2009). Here, we partnered with NFPs that collect and structure their data each according to their distinct operational processes. The result is four unique mixed geospatial datasets, each contributing partially to understanding rural mental health experiences. Table 1 summarizes the organizations and the nature of data that were used.

OPEN IN VIEWER

Table 1.

Participating Not-for-Profits and Related Datasets.

NFP & its Goals	Data Type & Source	Contribution
Infoxchange Provides an online platform termed “AskIzzy” for citizens to search for services www.infoxchange.org/au	QUANTITATIVE: Online search requests for local mental health services	Indicator of self-assessed service need
Royal Flying Doctor Service Provides emergency and primary healthcare services for rural residents www.flyingdoctor.org.au	QUANTITATIVE: Locations of aeromedical retrievals for mental health crisis incidents	Crisis a incidents may be considered an indicator of (local) services not meeting needs
Beyond Blue Provides mental health and wellbeing support to nearly 149,000 online forum users www.beyondblue.org.au	QUALITATIVE: Posts in an online peer support forum related to less-acute, high prevalence mental health issues	Descriptions of mental health-related lived experiences
SANE Australia Provides support for people affected by complex mental health issues through support for> 10,000 online forum users www.sane.org	QUALITATIVE: Posts in an online peer support forum related to complex, low prevalence mental health issues	Descriptions of mental health-related lived experiences

Results

Drawing on the case study above, this section first examines the feasibility of mapping geospatial NFP data to explore densities of need (using Infoxchange search data) and crisis incidents (using Royal Flying Doctor Service aeromedical emergency retrieval data) in relation to population density to identify geographic areas of interest. Second, it explores the potential of qualitative online forum datasets for providing new insights about rural peoples’ lived experiences of mental health in target geographic areas of interest, which in this case study is the highest density of crisis incidents (see Figures 5 and 7).

Quantitative Geospatial Analyses

Mapped Infoxchange data (i.e., indicating need, informed by numbers of searches on AskIzzy) in Figure 4 shows that 33,093 searches were made for mental health services across 340 post codes. Figure 4 demonstrates that—as would be expected—the higher densities of searches aligns broadly with more populous rural areas. This national-scale analysis suggests the feasibility of reusing Infoxchange data to “drill-down” to explore what services are needed, when, and in relation to place, helping to predict need and plan for services. Although using density of need data could be used to identify geographic areas of interest for further qualitative inquiry, we instead chose to focus our qualitative analysis within geographic densities of crisis data that does not align with larger population densities (See Figures 5 and 7). Notwithstanding, the layer of need data could be used in future analyses to inform rural mental health service provision, with timely, granular data, which is an acknowledged gap in data for rural Australia (Productivity Commission, 2020).OPEN IN VIEWER

Figure 4.

Map showing the density of service searches (i.e., need) across rural Australia overlaid on rural population density, with darker blue areas representing higher densities or geographical “hot spots” of need.

OPEN IN VIEWER

Figure 5.

Map showing the density of aeromedical retrievals for mental health crisis incidents across rural Australia overlaid on rural population density, with darker orange areas representing higher densities or geographical “hot spots” of crisis incidents.

During the same period, the Royal Flying Doctor Service undertook 664 rural mental health emergency aeromedical retrievals (responding to crisis incidents). As shown in Figure 5, the highest density of crisis incidents was approximately 100 km south-east of the rural area with the highest population density in the state of South Australia. This means that—contrasting with Figure 4 about need—the highest density of crisis incidents does not align with areas of highest population. This visual analysis shows that mapped NFP data layers can raise anomalous or counterintuitive findings that suggest locations for more granular explorations to better understand how service challenges may be experienced and lead to crisis evacuations.

Lastly, we identified a rural sample of 288 Beyond Blue posts, made by 49 forum users across 26 separate post codes and 253 SANE Australia posts, made by 51 forum users across 39 separate post codes (Figure 6).OPEN IN VIEWER

Figure 6.

Map showing the location (i.e., postcode) of rural Beyond Blue (blue triangles) and SANE Australia (red circles) posts.

Similar to other NFP datasets, the national distribution of rural posts indicate the potential of these data to provide new sources of information about lived experiences in relation to rural places.

Qualitative Analysis of Topics of Rural Mental Health Lived Experiences

To illustrate the potential of using qualitative GIS-based methods to inform place-based planning, Figure 7 shows how we integrated quantitative need and crisis density data with qualitative forum data in a joint display, where the highest density of crisis incidents across rural Australia was used as the target geographic area of interest to identify a sample of Beyond Blue and SANE Australia forum posts for qualitative analysis (total n= 73).OPEN IN VIEWER

Figure 7.

Geolocation of need data (blue squares), combined SANE Australia and Beyond Blue qualitative forum post data (purple circles), and density crisis incident data (orange) in relation to population density.

Within this joint display, we identified 805 searches (i.e., indicating service need) that occurred across 40 post codes. When compared to the location of forum posts, each of the nine post codes where someone used an online forum also shows evidence of people searching in response to need for mental health services. The integration of quantitative need data from searches and lived experience data from forums has potential to provide a richer placed-based understanding of challenges in relation to the crisis incidents occurring in the region.

To explore this, we qualitatively analyzed the forum posts. Within the target area of interest identified by high density of crisis incidents (orange heat map), we identified 73 forum posts made by 11 different forum users from nine different post code areas. Table 2 summarizes results from the qualitative thematic coding of combined posts from the SANE Australia and Beyond Blue forums, showing numbers of posts about themes that occurred more than once.

OPEN IN VIEWER

Table 2.

Thematic Analysis of Online forum Posts.

Theme	Example Quote from Post	Number of Posts on this Topic
Themes about services
Problems with service providers	Your reply could not have come at a more opportune time. I have just had a very dismissive response via text from my mental health social worker and it stuck me a hard blow	10
Limited access & choice	I have received great professional help, but it is limited to what they can do	4
Emergent themes about rural life and mental health
Relationship breakdowns	I need encouragement to move on from a very unhealthy relationship and need to hear positive things	4
New to isolated rural place	Struggling to make friends in a new town, recently my dad passed away he held my whole family together, my sister moved to be with her boyfriend, my mum kicked me out	2

Thematic coding of qualitative posts shown in the same joint display (Figure 7) showed two overarching sets of challenges expressed. One theme was about challenges related to services. Findings here resonate with service challenges regularly highlighted in government reviews (Parliament of Australia, 2018; Victoria State Government, 2020). Alongside, two themes about challenges of social life in rural communities were raised.

From this, it can be seen that examining qualitative information about just one “hot-spot” of crisis incidents suggests the potential of using NFP qualitative data, such as from online peer support forums, to better understand the context in which crisis incidents are occurring. With more qualitative data—for example, from health service notes connected to Royal Flying Doctor Service retrievals, there is potential to extend contextual understanding leading to the potential for better understanding of what leads to crisis. Traditional government and professional reviews tend to highlight service deficiencies, but as shown in our examination of a small set of forum data, there are also “emergent” issues about personal life in rural places that will not be raised in consultations about mental health because they do not ask about these more general life issues. The case study suggests the potential to use NFP qualitative data to explain and explore quantitative findings, here need and crisis incidents.

Discussion

Internationally, it is difficult to access data that authentically reflect the needs and experiences (i.e., voices) of people living in different rural places (Urban Institute, 2020). This is due to problems with the ecological fallacy where data is collected for sparse populations over large areas, and averaged (Boslaugh, 2007). Specific to rural mental health, there is also limited routine public data collection and service data generated is tightly held—and not shared—by health services. The mixed methods approach described here shows the potential of ethically sourcing, reusing, integrating, analyzing and ground-truthing mixed NFP organizations data, with each data set partially contributing novel data layers depicting partial aspects of mental health experiences in rural places. There is still much to be explored in terms of granularity versus identifiability (i.e., in establishing the scale at which data could be applied in place-based planning) and the ways in which reused data could be more routinely available to guide community and service planning. This is an agenda that must be pursued by involving the rural citizens involved in laboring to produce the data (i.e., here, as searchers for services, service clients and peers supporting each other online), as well as staff of NFPs, suggesting important next steps of this method.

In working with four NFPs to analyze data collected through aspects of their operational service delivery over a one-year period, we found it feasible to ethically and geospatially integrate quantitative data about need and crisis, with qualitative content from forum posts about lived experiences in geographic areas of interest. The most challenging dimension of this study was initially navigating ethical data sharing policies of each NFP, guiding how the data could be “reused” for research, while ensuring all aspects of this process were conducted responsibly and adhered to organizations’ agreements with end-users about consent. The experience of building relationships and navigating ethical issues aligns with that of Susha et al. (2019) who suggested, based on an extensive review, that this aspect of mixed methods data-driven social partnerships can be so complex as to deter organizations from collaborating with reused and integrated data. Notwithstanding these challenges, we have produced a replicable method for sourcing mixed datasets that already exist but are typically “untapped.”

In doing so, this novel approach to MMR helps to address a gap around social support services generated by NFPs and their outcomes, in Australia and elsewhere around the world, in how NFP data can be reused for social good (D'Ignazio & Klein, 2020). Although our case study was exploratory and limited to a single geographic area of crisis data, it raised emergent findings that highlight the potential for generating new insights when NFPs reuse and collaborate using their service data. Based on our findings, we propose that the ethical reuse and integration of NFP datasets—drawing on the idea of data collaboratives to inform social policy—helps to expose new facets of situational experiences, here in relation to mental health in rural areas.

Qualitative analysis of online forum datasets in geographic areas of interest perhaps shows particular promise as a source of insights about lived experiences for areas perceived to be data scarce. In our case study, this is exemplified where multiple posts occurred in a geographical area where there was a high level of crisis incidents (in South Australia), that did not correlate with population density. Our qualitative analysis of posts in this area showed confirmation of issues highlighted in previous intensive government rural mental health reviews and academic studies in highlighting challenges of rural service accessibility (Productivity Commission, 2020). It also raised emergent issues we have not seen reported previously, including experiences of isolation when new to a rural place and relationship breakdown both of which affect mental health experiences, which is why people are discussing them on the forum. These latter issues could be particularly “hidden” in consumer consultations as they do not seem immediately related to mental health but appear from our analysis, to be associated with raising or exacerbating mental health challenges. Further, these are areas that are open, potentially, to targeting new types of service interventions, in particular places. These results illustrate that analyzing qualitative forum data in geographic areas of interest could extend thinking about rural health services beyond traditional service models and methods that tend to emphasize clinical services.

For NFPs, collaborating using their service data can be seen as creating new value by expanding their capacity to do research. For example, via participating in this study, SANE Australia has gained the script developed by the authors to extract posts from their forum, expanding their capacity to reuse their service data for novel ways of conducting internal place-based research and evaluation. The Royal Flying Doctor Service also has gained new geospatial insights into where crisis incidents are occurring in the highest density, prompting them to internally investigate this issue and suggesting the sharing of additional service data to better understand what factors could be influencing the high-density of crisis incidents explored here.

Data collaboratives also offer new opportunities for NFPs to work together, albeit under the right ethical conditions with regard to data ownership and sharing, to improve place-based health and service provision. For some time, online health organizations have been recognized for their functionality in enabling forum users to share and receive support (Van Dijck & Poell, 2016), as well as generating added-value by facilitating the collection of health data for research and education (Moessner et al., 2018). Integrating mixed datasets from several NFPs and relating it to place and context-specific population density data, via a GIS, offers a novel extension for using mixed health consumer and operational service data. This could be especially useful internationally for commissioners of services and public health managers by providing new ideas for service planning, and insights about population health and its drivers. This is especially true for rural areas, with regard to the acknowledged lack of datasets with adequate rural representation (Productivity Commission, 2020).

In recent years, researchers have also pointed to the potential of online health forums as a unique source of observational data for understanding health issues. Mamykina et al. (2015), for example, argue that online health forums can be viewed as generating “collective sense making” around specific health experiences using an approach that focuses on integrating post data to create shared knowledge structures uniquely capable of depicting authentic consumer experiences. At the same time, using “big” qualitative datasets to explore “grand” community challenges such as mental health, has been critiqued for the potential to commodify consumer opinion (Lupton, 2014). However, we argue that within ethical collaborative settings, there is also potential value in improving the ways the health system can access health consumer experience data.

We view collaborating with NFPs—as we did in this case study—as an important first step towards co-designing MMR approaches that emphasize end-users’ voices and lived experiences. As we highlighted above, including health consumers as partners in driving these data use processes is an essential next step in this dynamic field. Although all data used in the case study was subject to prior user consent via agreements when registering to use their service, we are concerned to more deeply engage end-users in issues around analyzing, interpreting, and applying data generated through NFP services, as well as using data for consumer empowerment and advocacy. The next step in our research is to engage online forum users in discussions about data practices and appropriate use of NFP data analytics in service design.

Contribution to the Field of Mixed Methods

This mixed methods qualitative GIS-based approach contributes to MMR in multiple ways. First, we demonstrate how the application of a novel mixed methods qualitative GIS-based analysis can be used to fill data gaps and inform mental health service-planning challenges in rural contexts that are perceived to be data scarce. In doing so, we feature an ethical mixed methods approach that integrates mixed NFP datasets, providing a pathway for future research that aims to reuse NFP health data and build on innovative MMR approaches.

Reusing geospatial NFP health datasets in a mixed methods approach draws on the richness of qualitative analyses alongside the greater scale of quantitative research in a non-compartmentalized way, contributing to the expansion of mixed methods design (Maxwell, 2016). Choosing the location of a single case study that drilled down into a South Australian hotspot area for crises was an outcome of an exploratory and iterative qualitative visual analysis that was designed to reveal potential spatial relationships between quantitative and qualitative NFP data (Knigge & Cope, 2006). Iterative qualitative visual analysis of mixed datasets is a known hallmark of qualitative GIS research (Mennis et al., 2013), which we contribute to by adding a ground-truthing phase to the identification of geographic areas of interest with participating NFPs, contributing new methods of validating geographic areas of interest in a joint display. In doing so, this mixed methods approach exploits the growing utility of qualitative GIS by accessing and integrating reused, secondary NFP data to expand the geographical examination of place (Watkins, 2022)—in this case through the lens of rural mental health. Developing this novel mixed methods approach also responds to calls for evidence-based policy making by creating data collaboratives that make the voices of health consumers more visible in research outcomes, thereby contributing novel methods to the re-distribution of who collects and has access to data (Susha et al., 2017). Although the case study presented here was exploratory, our results show the promise of adding additional datasets to explore place-based relationships between lived experiences of mental health service challenges and other contextual experiences.

Second, our mixed methods approach has potential use for NFPs, health services and policy researchers to quickly, affordably and responsibly identify potential place-based elements that are contributing to service and other challenges for people experiencing mental ill-health in varying international settings, so long as they possess the technical expertise or can partner with organizations that do to create a data collaborative. This mixed methods approach is designed in such a way that much of the data collection can be done from a distance, thus minimizing the resources needed to replicate the methods used in this case study.

Third, this case study demonstrates the potential of generating new evidence from collaborating with mixed NFP data when conventional public data collection or health service data are “too thin” for understanding place-based contexts and challenges as in the problem of the ecological fallacy in relation to rural places. In qualitatively selecting and ground-truthing a site for this case study, themes were raised that broadly verify rural mental health service provision challenges raised by large and expensive government reviews, but also provided emergent insights, such as problems of isolation when new to the area and of relationship breakdowns. Although an analysis of a small amount of qualitative data, our findings do show the potential for explanatory evidence from reusing qualitative NFP datasets. The addition of more qualitative datasets might give more insights into what is occurring in this hotspot. Similarly, the addition of more quantitative data might give more nuance as to conditions and other services provided (e.g., what mental health services people are searching for). It can be seen that ongoing additions of qualitative and quantitative data could serve to make the picture richer for service providers and planners. Further, the methods used here could be applied to other public health issues, outside of the field of rural mental health, to include other health, service, or support issues where data used to inform policy is limited and the voices of lived experience would be helpful. Developing this mixed methods approach with the involvement of NFP staff was important too—for example, as they were able to assist with ground-truthing findings.

Finally, we have demonstrated that mixed NFP datasets can be ethically reused to add new layers of complementary data, helping to generate rich and integrated evidence that is “more than the sum of its parts” and providing “situational and actionable knowledge” about place issues (Fetters & Freshwater, 2015; Saunders & Mulgan, 2017). Although this was successfully done in Australia under strict national data sharing policies, this method can be adapted to other, for example, rural contexts around the world to inform place-based service planning, as long as NFPs offer services to those rural areas, they have geospatial identifiers connected to their service data and users of their services consent to reusing of their “data” for research. As an example, NFP data from an online community called “Elefriends” that facilitates peer support for people experiencing mental health problems in the United Kingdom (Tucker & Goodings, 2017), could be integrated with other service-related “need” and/or “crisis retrieval” data to identify geographic areas of interest for further qualitative inquiry. Although this would require navigating data sharing policies shown in Figure 1—in the United Kingdom—the benefit of adopting this mixed methods approach has been shown here to generate novel insights and new collaborative opportunities between researchers and NFPs.

Although GIS science has only begun to interface with “big data,” reused NFP data and data collaboratives, it has remained largely focused on more traditional quantitative methods (Poorthuis & Zook, 2015). Critically, Sui and Goodchild (2011) argue that within this context, GIS scientists must move beyond simple analysis of XY patterns in geospatial data and begin to interrogate the ways that mixed electronic health data can inform our understandings of place. We accomplished this by exploring facets of individuals lived experiences with mental health challenges in rural places via integrated need, crisis incident data and experiential data from posts in online peer support forums. In doing so, we generated a novel joint display to which other data sets can be added for richer evidence.

Limitations

There are caveats that apply to each NFP dataset in our study; for example, only a proportion of those experiencing mental ill-health in rural areas are likely to use online forums, due to issues including affordability of internet connection, meaning the experiential data drawn on here are not comprehensive. Similarly, not everyone seeking mental health services will search online. Simultaneously, though, we are not suggesting a comprehensive picture here, rather that each dataset offers potentially useful and previously “hidden” partial information that can be woven together to complement or extend other data such as from government and health administrative services.

Using secondary data that already exist is a way of reducing research costs for governments, NFPs, and researchers and limits the demands for data collection made on vulnerable individuals and communities with mental ill-health. It also would move focus more toward identifying, obtaining, and analyzing datasets in suitable relationships with end-users and NFPs, than designing questions from perspectives of power and trying to persuade reluctant citizens to respond. Inevitably though, as highlighted by D'Ignazio and Klein (2020), even reused datasets are not “raw” and the original purpose for their collection and ways of managing and organizing their data should be borne in mind in data application and interpretation. This could, however, be said of standard research data collection which is guided by the “agenda” of the researcher or project, including consultation exercises to inform government reviews or research projects pursuing a central focus or hypothesis.

With data collaboratives also—theoretically—there is potential to continually extend the richness of the limited data pool shown here, by adding additional NFP and other datasets, with capacity to integrate them for further place-based analysis dependent on having a shared spatial identifier (i.e., post code). For example, reused data and qualitative descriptions of how services are difficult to access can be integrated with travel time data to hospitals to help inform service delivery (Barbieri & Jorm, 2019). This means that there is additional potential for dynamic, real-time data analysis as the datasets used here were not obtained through cross-sectional surveys but instead, were reused from timestamped NFP data, enabling longitudinal study. Although the resulting joint display and data analyses are necessarily only partial in showing the complexity of rural mental health experiences, nonetheless this novel mixed methods approach shows what is possible with access to multiple mixed NFP datasets.

Conclusion

With the changing context of MMR and “big” qualitative data come novel opportunities for collective sense-making in research, practice, and policy. Here, this was illustrated by developing a novel method involving mixed, reused NFP datasets, each contributing a partial understanding, collectively helping to extend the availability of data capable of informing place-based service delivery challenges. By drawing on qualitative GIS-based analyses of reused NFP datasets in a data collaborative, we were able to geospatially map discussions of lived experiences of mental health, need, and crises in relation to rural population density, illuminating mental health themes relating to rurality. This shows the potential to ethically reuse alternative sources of data to understand complex socio-spatial scenarios. Next iterations of research into the potential of reused and integrated NFP datasets for service planning might focus on accruing more data sets. This could facilitate improved efforts to “drill down” into geographic areas of interest to explore causations and explanations of why spatially distinct phenomena arise, by drawing attention from users of individual services and giving different mixed lenses through which to understand social phenomena.

Although we have indicated various future directions for this research, a priority is to involve those people whose experiences are reflected in the data, with a central aim of this research being to understand how they could gain benefit through applications of their data, but also understanding how people with mental health conditions can use data for empowerment and advocacy. This involves generating data capability beyond NFPs, a sector itself only coming to terms with the infrastructure and understanding the need for data analytics that extend to citizens. This will lead to a next generation of more equal and capable relationships between citizen consumers of services, service providers, service planners, and other health system actors.