Abstract
This article describes data collection within an Australian mixed methods study that investigated family carers’ stress levels and experiences of dementia services and the cognitive and functional abilities of individuals experiencing dementia. Qualitative and quantitative data were collected via a “progressive engagement” process involving “multidisciplinary functioning,” “unambiguous protocols,” and “participant-centered procedures.” The approach achieved high-quality data and 100% participant retention. Study details are located within a broader discussion of mixed methods research in the social sciences and a pragmatic paradigm fitting to dementia research. The article contributes to the field of mixed methods research by highlighting philosophical issues relevant to dementia researchers and by providing sufficient depth of detail for interested readers to adopt or adapt the authors’ mixed methods procedures.
Rising worldwide diagnosis rates for dementia coincide with a growing body of dementia research. A major review of literature undertaken in Canada found that despite the burden of dementia experienced globally, understanding of its causes, risk factors, and prevention strategies remains limited (Patterson et al., 2008). In Australia, dementia holds national health priority status (Australian Institute of Health and Welfare, 2007), with several national bodies sharing responsibility for those with the disease and their carers.
The complexity of dementia necessitates researchers to generate knowledge within all research traditions if a comprehensive and holistic perspective of the dementia field is to be developed. Until improved diagnostic and treatment options for dementia become available, well-being issues for those experiencing the disease and their family carers are of prime research concern. Our research as introduced here sought quantitative and qualitative insights within a single, exploratory study. It used a mixed methods research design that has been described as “a synthesis that includes ideas from quantitative and qualitative research” (Johnson, Onwuegbuzie, & Turner, 2007, p. 112). Specifically, the study aimed to investigate the following:
Family carers’ information needs and experiences of receiving dementia services within and outside the home
Family carers’ stress and well-being levels
The cognitive and functional ability of the individuals with dementia supported in the community by the family carers.
Although details of how the study design and findings met these aims are reported elsewhere (Stirling et al., 2010), this article focuses on the contribution a mixed methods approach made to acknowledging the complexity of dementia research. We judged our objective measures of dementia care recipients and caregivers could be fully meaningful only if considered in the qualitative context of the family carers’ lives. This exemplifies why a mixed methods approach is fitting to dementia research: complex research problems warrant a complex research approach. Dementia is a profoundly complicated condition and mixed methods allow for a diverse range of means by which to address the problem. Ultimately, by way of this and related studies, we wish to inform and influence policy direction in dementia care in Australia.
The features of our approach to data collection were not novel in and of themselves, but when taken together (packaged) within the context of mixed methods and dementia research, they provided a coherent and meaningful framework within which to operate. This supports the point that mixed methods research designs are similar in many respects to other traditional designs (Hanson, Creswell, Clark, Petska, & Creswell, 2005). To provide a sufficiently comprehensive description of our data collection methods for study and replication purposes, we include here both those features recognizable as generic to all good research and those relating specifically to our mixed methods dementia study.
“Progressive engagement”—the nub of our approach—refers to an increasingly meaningful and effective involvement by the research team with the study design, the research participants, and the data sets. It relates especially to building strong relations between our research team and our participants over time, with the intention of obtaining data of increasing quality.
Relevant Literature
Given our interest in the situation of family carers supporting people with dementia living in the community, we were especially interested in mixed methods studies in our topic area that addressed methodological issues of data collection. Such studies proved scant, which could relate to an uncritical acceptance of mixed methods practices (failure to give due consideration to underlying assumptions; Sale, Lohfeld & Brazil, 2002), a point to which we return. The following articles served to highlight a methodological gap in the dementia literature, as well as strike some cautionary notes about methodological difficulties from which we could learn.
An evaluation of a dementia care service in the United Kingdom specializing in early care used mixed methods as this would obtain more complex and definitive results (Pritchard & Dewing, 2001). Participants included people with dementia (regarded as a novel sample constituent) and methods included interviews, self-report questionnaires, “dementia care mapping,” focus groups, document analysis, field notes, and other contextual information. Outcomes revealed a high-quality service with parallel care planning with carers and clients and carer coaching being of particular note. These authors did not include theoretical discussion of mixed methods research per se.
In contrast, D. G. Morgan and Stewart (2002) provided an in-depth discussion of the theoretical and philosophical issues surrounding their mixed methods dementia research in Canada. Although their topic (evaluation of residential care for dementia clients) was dissimilar to ours, some comments on data collection are noteworthy. These authors took the stance that paradigms cannot be ignored because they are linked to strategies of inquiry. Concerns about conflicting sampling approaches to different components of the study were avoided as participants in a quasi-experimental component of the study (residents) were a different group from those in a grounded theory component (caregivers). Bias in order of data collection was also minimized by having data collected in these two sample groups by different people. Data collection occurred over 21 months, resulting in extraneous events that the authors reported as threatening internal validity of their study.
Numerous articles report discretely quantitative or discretely qualitative dementia research. For example, Bouwens et al. (2008) studied the relationship between motor and process skills and dementia severity measures, finding such measures were only moderately associated with observation of performance on daily activities. In another study, a multimethod evaluation was undertaken of people with dementia and their family carers’ satisfaction with a memory service; however, only the qualitative component appears to have been reported (Willis, Chan, Murray, Matthews, & Banerjee, 2007). From qualitative interviews, the researchers derived seven quality indicators by which to judge services for people with dementia. This study was in our topic area but did not meet the mixed methods expectation of integration of findings (Johnson & Onwuegbuzie, 2004).
Of particular interest was a qualitative study in which grief was found to be the major barrier faced by carers of individuals with dementia, but in which there was no assessment of dementia severity (Frank, 2008). As the author reports, this unfortunately meant the relationship between grief and the stage of disease could not be analyzed—a limitation likely remedied by mixed methods research (and specifically addressed in our study).
Social Science Context of Mixed Methods Research
Literature beyond the field of dementia about mixing research methods embraces various views on the origin of the mixed methods movement, merits of a “paradigm” perspective, legitimacy of the quantitative–qualitative divide and triangulation, and a definition for mixed methods research, all further addressed here. Definition views are highlighted in a study that sought definition of mixed methods research from 36 leaders in the field who furnished 19 different definitions—15 of which identified qualitative and quantitative research as to what is mixed, with explicit and implicit understanding that mixing occurs at all research stages (Johnson et al., 2007).
A groundswell of interest in mixed methods began about 20 years ago, but the practical demands of fields such as education, nursing, and evaluation required that various methods be employed well before this, according to Greene (2008). In social inquiry, Greene believed that a mixed methods approach has the potential to be a distinctive methodology because it embraces multiple traditions and is developing distinctive methodological components:
A mixed methods way of thinking is an orientation towards social inquiry that actively invites us to participate in dialogue about multiple ways of seeing and hearing, multiple ways of making sense of the social world, and multiple standpoints on what is important and to be valued and cherished. (p. 20)
Other authors cite the mixed methods approach as originally being an outgrowth of the triangulation of methods movement used in geography that sought to confirm results by the convergence of findings from different data sets (Dunning, Williams, Abonyi, & Crooks, 2008). With a particular interest in quality of life studies, these authors described a mixed methods approach as going beyond triangulation in that it also seeks increased understanding (comprehension) of results—it is not simply a mix and match of two or more research techniques. Mixed methods research thus has two broad goals: confirmation and comprehension. The complexity of the approach can result in divergent findings, which ironically led Dunning et al. (2008) to new insights. Two key guidelines proposed by these authors include being transparent about the methodology because mixed methods research as yet lacks a formal design and preparing in advance for dissonant results that can extend timelines and tax resources. We have heeded the transparency guideline here. By limiting our report to data collection, we can provide sufficient methodological detail for interested others to either follow or tailor our procedures to their own research. Interestingly, we also arrived at some dissonant (counter intuitive) findings, which ultimately provided an unexpected and novel outcome concerning the relationship between carers’ needs, severity of dementia in care recipients, and services used (Stirling et al., 2010).
Uncritical Acceptance of Mixed Methods
It has been suggested that a new generation of researchers (especially in health care) is uncritically adopting mixed methods research without attending to underlying assumptions of the qualitative–quantitative debate (Sale, Lohfeld, & Brazil, 2002). These authors pose the question as to how it is possible to combine the two approaches for triangulation purposes if they study different phenomena. They make a case that, indeed, the approaches cannot logically and justifiably be combined, despite some persuasive points to the contrary (such as ignoring the issue and doing what works, or acknowledging inherent quantitative features of qualitative research). Pursuing their argument, Sale et al. (2002) go on to show that despite their incommensurability, mixed methods can be applied simultaneously or sequentially in a single study if done for complementarity purposes. That is, because each method studies different phenomena, it is essential to distinguish these by labeling the respective phenomenon under study by each method, for example, a “measure of burnout” and “lived experience of burnout.” The authors argue that this produces an additive outcome rather than merely seeking to balance out the strengths and weaknesses of each approach, or to capture different aspects of the same phenomenon (Sale et al., 2002).
Reconciling the quantitative–qualitative divide has remained a dominant theme in mixed methods literature of the past decade, for example, Greene (2008), Johnson and Onwuegbuzie (2004), D. G. Morgan and Stewart (2002), and Sale et al. (2002). Standing apart from other authors’ views, Axinn and Pearce (2006) have found the divide unhelpful and have abandoned the quantitative/qualitative labels in favor of using the terms structured and unstructured throughout their text.
A Pragmatic Approach
D. L. Morgan (2007) has advocated a “pragmatic approach” as a basis for combining qualitative and quantitative methods and directing attention away from metaphysical concerns (issues related to the nature of reality and truth). In the pragmatic approach D. L. Morgan (2007) holds that, principally, the connection between theory and data is one of abduction, where reasoning moves back and forth between deduction (quantitative approach) and induction (qualitative approach). Other conceptual developments in the pragmatic approach are intersubjectivity (a duality of objectivity and subjectivity) and inferences from data being characterized by transferability—as opposed to generality (quantitative approach) and context (qualitative approach). While keenly recognizing that social science researchers are concerned with ethical issues, D. L. Morgan (2007) firmly rejected any crude notion of pragmatism that simply claims the ends justify the means—rather, “the pragmatic approach reminds us that our values and our politics are always a part of who we are and how we act” (pp. 69-70).
Educationalists Johnson and Onwuegbuzie (2004) have argued for pragmatism as “an attractive philosophical partner” to mixed methods research. Drawing on the classical ideas of Peirce and James, and Dewey, they described a pragmatic approach as offering “a practical and outcome-oriented method of inquiry that is based on action and leads, iteratively, to further action and the elimination of doubt” (p. 17). Although finding these ideas complementary to our own research, we were mindful of counter views. For example, Johnson and Onwuegbuzie emphasized that pragmatism should not quell philosophical debate and, indeed, that its weaknesses should be made clear. These authors cited practicality as possibly detracting from basic research and, from a transformative–emancipatory perspective, failing to satisfactorily identify for whom a pragmatic solution is useful (i.e., failing to identify whose interests are being served).
These multiple viewpoints in the mixed methods literature led us to regard mixed methods within a pragmatic paradigm as a fitting context for dementia research. We were also comfortable in locating our research in a school of thought that holds that mixed methods is a distinctive methodology that embraces multiple ways of making sense of the social world (Greene, 2008).
Method
Quantitative and qualitative data in mixed methods research are not limited by type (e.g., interview data, survey data) but rather by procedural issues to do with sampling and concurrent or sequential collection (Hanson et al., 2005). In this study, we used purposive random sampling (Teddlie & Yu, 2008) to recruit 20 family carers of people with dementia and the family member they cared for. This single sample generated data for both the quantitative and qualitative strands of the study and had features in common with concurrent mixed methods sampling (Teddlie & Yu, 2008). Quantitative data were collected from the family carers and family members at the outset of the study (September 2007), with subsequent collections comprising both qualitative and quantitative data from the family carers. Data were not collected in an order of priority that held qualitative or quantitative as having greater importance. However, where we believed qualitative data collection could influence an objective measure, we collected the quantitative data first to avoid this (as in the carer stress tests). In other instances, we judged a qualitative lead-in could best facilitate the collection of a quantitative item (as in our telephone calls); conversely, getting past the collection of some quantitative items we judged could provide a more appropriate lead-in to qualitative data collection (as in our interviews). In other instances, the decision as to the order in which data were collected was determined by what was most convenient or beneficial to participants (as in the cognitive testing of the family member with dementia). Our study design thus included both concurrent and sequential collection of the two kinds of data, a relevant point for data analysis and integration of findings (Hanson et al., 2005). Table 1 shows the various tools used and the types and timing of data sets collected.
Data Collection Tools, Data Type, and Timing of Collection
Data Collection: Features of Our Progressive Engagement Approach
Below, the three key features of the progressive engagement approach—multidisciplinary functioning, unambiguous protocols, and participant-centered procedures—are first shown schematically in Figure 1 and then described in terms of how they progressively facilitated collection of the data sets previously identified.
Schematic representation of progressive engagement approach
Multidisciplinary Functioning
Team membership comprised two chief investigators, one coinvestigator, a project manager, a senior research officer, and a research assistant. Additional expert advice was sought on an ad hoc basis in relation to data collection tools.
Collective expertise of the team covered a number of fields including arts and humanities, counseling, education, nursing, psychology and neuropsychology, science and neuroscience, and social work. These fields span both qualitative and quantitative research traditions—we found this essential for informed decision making and functioning in all stages of our mixed methods research, including data collection.
Three-weekly meetings were held throughout the data collection period to monitor progress and address problems before they escalated and for team members in the field collecting data to consult, share emerging insights, and occasionally debrief in a confidential, supportive atmosphere. As we were collectively and progressively drawn into the participants’ worlds via incoming data, we were able to refine guidelines and procedures to best meet study aims (e.g., develop additional interview questions and take the decision to repeat one measure in the final week of the study) plus plan for upcoming stages of data analysis and integration of findings.
Bimonthly consultations were held with an Advisory Committee representing a range of dementia service providers, the funding body, and a family carer who was also a participant in the study. The inclusive form “we” was used when referring to the study with participants, clearly indicating they were a highly valued and respected part of a large multitalented team.
Ad hoc communication between the three team members responsible for collecting data was intense (telephone, email, corridor) in relation to entering participants’ homes as a team in Week 1 of the study (travel involved a 100 km city radius) and between the two members involved over the remaining weeks. For these two members, confidential folders in a locked drawer (accessible in a common office) also served as a communication means, for example, for passing the diary and hard copy data collected monthly at interview for use in weekly telephone contact with participants. Email contact was also frequently used for two-way communication about participants’ progress in the study.
Our multidisciplinary functioning was congruent with the outcome of a review of five mixed methods investigations in the social sciences that found all research was conducted by externally funded teams with quantitative and qualitative expertise and with extant knowledge of mixed methods models (Creswell, Fetters, & Ivankova, 2004). All studies were found to be labor intensive with multiple stages of data collection. In a study of 20 mixed methods researchers in the United Kingdom, Bryman (2007) also found it was necessary to develop teams that bring together specialists from both quantitative and qualitative traditions. This researcher further reported that attempting to weld the two kinds of work could, however, lead to an unfortunate compartmentalization of roles and responsibilities that ultimately hinder the integration of findings. Our multidisciplinary functioning proved to be a productive melting pot of ideas and influences, bearing out a vigorous call by Axinn and Pearce (2006) for investigator involvement in mixed methods data collection to allow for maximum introspection.
Unambiguous Protocols
Application for ethics approval required explicit justification for quantitative and qualitative data collection, including measures to ensure that family carers, known to carry a heavy burden of care (Michon, Weber, Rudhard-Thomazic, & Giannakopoulos, 2005), were not further stressed by a mixed methods study. Procedures for gaining consent from family carers on behalf of family members with dementia were established in consultation with the university ethics committee. The process involved (over several weeks if necessary) explanation and discussion between the project manager who was responsible for recruitment and carers who liaised with family members. We note here that care recipients were moderately to severely impaired with dementia and that our study design involved them as participants in terms of determination of dementia severity but not as interviewees regarding service provision. The study was concerned specifically with family carers’ experiences of service provision.
Recruitment occurred within the Southern Region of Tasmania with assistance from, and intensive consultation with, regional dementia service providers (meeting inclusion criteria and avoiding potential bias were key considerations). Two categories of participants were required within a single sample group (20 individuals with dementia and 20 family carers of these individuals). Exceptionally clear and concise expression was required in letters of invitation, the information sheet, and the consent form to ensure necessary information was conveyed without dissuading potential participants. Excessive mixed methods detail could have made the study appear overly daunting.
Diary development and trial
The weekly diary for completion by family carers was developed in draft form by the team, validated by three experts in the field (involving some modification in the process), trialed with two family carers who met the inclusion criteria, and subsequently refined for final use (Mastagia, Toye, & Kristjanson, 2003).
Written guidelines.
Participant progress sheet—to allow detailed tracking of each participant in relation to each means of data collection and other key processes for participants (matrix format) over the 12-week data collection period
Interview guide—a guide for each monthly interview, stating purpose, and intended progress of interviews, including recording procedure and topics to be covered
Record of weekly phone calls showing participant, date, time, duration of call, plus quantitative and qualitative data
Pro forma for interview notes—used for quantitative and qualitative data collection in conjunction with monthly interviews
Meticulous data management
With consideration to the university’s policy on data storage, all mixed methods data sets (totaling 440 individual pieces by close of collection) were numerically coded and codes kept separately from participants’ identities; original hard copy data were stored securely as they came to hand in a separate building to locations where copy data were held; original electronic data (digital interview recordings) were uploaded to a dedicated secure folder with specified access on the faculty website; copy data were held in locked places in researchers’ offices and accessed only by team members (one general staff member was responsible for transferring electronic interview data to the transcriber within a confidentiality agreement).
Participant-Centered Procedures
Participant-centered procedures were paramount and pivotal to our approach to mixed methods data collection. By this, we mean that the well-being and interests of our participants were of central concern and took precedence over study aims and researchers’ interests. We knew from our own and others’ research that we could potentially lose participants from the study due to them feeling overburdened. Specific strategies adopted in relation to each method of data collection follow.
Stress and Cognition Tests, Carer Personal Profile
Week 1 was key to participants’ ongoing involvement in the study—we wanted them to feel motivated to continue despite some intensive data gathering in the estimated 2-hour session. Written consent was obtained at this time (verbal having been previously secured with utmost care as described), the personal profile completed, and the diary-keeping process explained—the diary was left in the home on this day for immediate use. We had also taken the decision to administer the stress tests (to the family carer) and the cognitive function test (to the person with dementia) on this day, primarily because we did not want other aspects of the study (e.g., qualitative interviews) to influence results. Strategies adopted to foster participant well-being and minimize stress are listed below.
Limiting number of researchers entering the home
Dementia service providers had advised that more than three people entering the home at any one time could intimidate participants and we followed this advice to good effect. Following sociable introductions, and securing written consent, one team member moved to a different room with the person with dementia to conduct the cognitive ability test, and two other team members stayed with the carer to complete the personal profile, administer the stress tests, and explain the diary. At the time of introductions, we allowed the carer to set the tone and pace; for example, some carers wished to move straight into business, whereas a cup of tea was an important ritual for others.
Sequencing of data collection
Activities with carers were ordered in the following sequence, judged to cause minimum stress: personal profile, tests, and diary explanation. The tests were also sequenced so that those requiring least personal disclosure were completed first; for example, the Bayer scale addressed the carers’ perceptions of the abilities of the person with dementia, whereas the General Health Questionnaire required carers to focus on their own mental and emotional states. We made it clear that a break could be taken at any time, and we checked how the carers were feeling if we sensed it appropriate—emotions were never far from the surface however the tests were sequenced.
Flexibility in conducting tests
All tests were of paper-and-pencil variety, and we found some carers wished the researcher to record the answers to questions (having either read the questions themselves or having the questions read to them), others wished to record answers themselves but liked the questions read aloud to them, while yet others liked to accomplish the task alone. Adapting to the carer’s preferred style helped to accomplish the data collection smoothly—our aim was to demonstrate our genuine regard and respect for the carer and sensitive material being addressed. Considerable flexibility was also required in conducting the cognitive function test in accord with the person’s personality and cognitive ability. In all instances, across all tests, we conveyed verbally and nonverbally an appreciative, affirming attitude, and we believe that on this first important visit we left the occupants of all homes feeling optimistic about the study.
Telephone Calls
Each participant received a weekly telephone call at which time both quantitative and qualitative data were documented regarding services used in the past week; assistance was offered in relation to the diary, and carers were reminded about the next monthly interview. Our participant-centered approach meant that calls were made at carers’ preferred time of day, were repeated until contact established (leaving a message if possible), and were intentionally helpful and supportive in content. Tailoring calls to participants in this way meant they varied markedly in length from a few moments to approximately 20 minutes. Our estimate of 5 minutes per call depended on the particular events in the home each week and individual experiences with the diary-keeping process. Securing the data we needed required recording a value for subsequent counting (“happy/unhappy” satisfaction rating) and empathic listening to participants who were enmeshed in complex family circumstances.
Research indicates that there is no evidence of datum loss or distortion when collected by telephone; indeed, telephone use might allow participants to disclose sensitive information more freely (Novick, 2008). In dementia research, caregivers have found telephone support more rewarding when they had met face-to-face with the provider (Spilsbury, 2006)—as ensured by our protocols—and that their caregiver burden was decreased (Stewart, Makwarimba, Barnfather, Letourneau, & Neufeld, 2008).
Diary
The diary provided core quantitative data for the study—we required comprehensive, accurate information about which particular dementia services were accessed and if they were accessed as planned. We also sought qualitative data in an unstructured section of the diary where participants recorded their “most outstanding event of the week in relation to services.” The diary’s most significant feature was a personalized format. At the first meeting with carers, details of services received inside and outside the home were entered into the dairy for the current week by the researcher, providing a baseline for all subsequent entries by the participant. Also significant was omission from the dairy of the “happy/unhappy” weekly satisfaction rating; we collected these data via telephone, believing participants could be justifiably reluctant to record this judgment in a place where it might be sighted by personnel providing the service. Differently colored pages were used to help identify sections about services received in the home, services accessed outside the home, and the free response section. At the time of each monthly interview, the diary was reviewed and a new diary for the forthcoming month was similarly personalized and left with the carer (and the previous month’s diary collected). Our weekly telephone support and monthly face-to-face assistance was aimed at maintaining a flow of complete, accurate data over 12 weeks—we needed to know exactly what changes occurred during this period. The carers’ diary-keeping skills markedly improved as the weeks passed and we attribute this largely to our participant-centered approach, which included an all-hours telephone message service that was used on several occasions, especially when carers needed to change interview times and dates.
In research using diaries with structured and unstructured components, problems with diary design, respondent fatigue, and inadequate support have been found to contribute to poor participant retention rates and the quality of data (Keleher & Verrinder, 2003). Our strategies for successful data collection via diaries were influenced by these researchers.
Interviews
Four interviews were conducted at monthly intervals over the data collection period, with several participant-centered considerations. The first team interview (as described) involved consent, the carer profile, stress and cognition tests, and diary explanation. Subsequent interviews involved diary assistance, collection of data for quantitative analysis (using pro forma), and a 20 to 30 minute taped conversation. Interviews were held in participants’ homes at carers’ preferred times, taking geographic location into consideration but not as a priority. Care of the person with dementia was a priority and planned in advance for the total interview duration (60-90 minutes) and the cost of any required respite care met within the study budget. The comfort and well-being of carers remained paramount during the four interviews and was achieved primarily by what we called “seamless transition” and “progressive disclosure.”
Seamless transition involved moving from one monthly interview to the next, and between parts of each interview, in a progressively engaging way that ultimately comprised just one research experience for participants but multiple data sets for the researchers. It required keeping a clear mental picture of each participant’s situation and progress, in tandem with the datum requirements of the study—both aided by our mixed methods written guidelines. Meticulous observation of interview dates and times, maintaining an empathic, conversational manner, and having necessary materials immediately to hand (marked-up diary for following month, interview guide, pro forma for quantitative data, voice recorder) without having to rummage and fumble were important in maintaining a flow of events, as was using an unobtrusive digital recorder (with a back-up recorder available if needed). Striving to achieve seamless transition was important in building strong relations with participants—we needed to demonstrate our authentic interest in following their progress and changing circumstances over time.
Progressive disclosure involved guiding participants via semistructured questions through a series of increasingly sensitive topics about their use of dementia services and their life as carers. This occurred over 12 weeks with each conversation tending to open where the previous one had closed (seamless transition), taking account of special and changing circumstances. For example, one thread of questioning over time began with which services were used and how they were accessed, progressing to the carer’s relationship with the organization providing the service and how this could be enhanced, to the impact of services on the carer’s quality of life and the most difficult aspects of the caring role. As the final interview neared its end, we endeavored to bring a natural, seamless, and positive closure to the study experience for participants. Disengagement was assisted by presenting a (unexpected) gift voucher in appreciation of participants’ contribution. Administering the Carers’ Checklist again at the final interview not only furnished desired data but enabled us to demonstrate to participants our genuine interest in them over the 12-week period.
Literature is replete with advice on conducting qualitative interviews. We note that the “laddered” interview technique used by Price (2001), albeit in educational research, was in some ways congruent with our own. Price proceeded through different levels of inquiry over several interviews, using a notebook (in addition to a recorder) to capture possible leads for follow-up later in the interview or in subsequent interviews. Salient points for our study from research using in-depth interviewing in the home related especially to trust factors: confidentiality, flexibility in techniques, establishing rapport, and authenticity and experience of the researcher (Borbasi, Chapman, Gassner, Dunn, & Read, 2002). Other authors have discussed relevant factors such as age, class, and gender in interviews (Manderson, Bennett, & Andajani-Sutjahjo, 2006) and interviewing on emotional topics and the impact of this on the researcher (Lowes & Gill, 2006).
Reflections on Our Progressive Engagement Approach
Overall, our approach proved very effective for mixed methods data collection in dementia research. Its component elements (multidisciplinary team functioning, unambiguous protocols, participant-centered procedures) provided us with a sufficiently complex structure within which to address the inherent complexity of our research problem and a means for articulating our work. Particularly pleasing was the 100% participant retention rate and participants’ reported enjoyment of the research process.
At the final interview, a single question (without subsequent probing) was put to carers: “What has the experience of participating in this research project been like for you?” Responses were overwhelmingly positive (23 comments contained the descriptors “interesting,” “enjoyable,” “informative,” “stimulating,” “motivating”) including a perceived value of helping others in the future (13 comments), pleasurable interaction with the researchers (10 comments), and benefits for the person with dementia (2 comments). Combined with these positive comments were six comments about initial difficulties with using the dairy, two about finding the experience daunting, and one expressing the view that talking about being unhappy “rubs it in.”
We also asked participants (excepting one who was herself in a stage of terminal illness) if they would be willing to assist with future dementia research and received an unqualified “yes” in response. Participants’ positive cooperation is noteworthy, especially given their (average) high-stress scores on the General Health Questionnaire (GHQ-30) and numerous personally reported physical and mental health issues.
Participants’ enjoyment of the study raises the concept of research as therapy (Borbasi et al., 2002; Lowes & Gill, 2006). Despite our guidelines and procedures being research oriented only (e.g., we did not provide advice or information about dementia services), several participants referred to our data gathering as being helpful and informative for them; one even described it as counseling. We note other researchers have capitalized on this phenomenon; using unstructured health diaries over 16 weeks, researchers encouraged a two-way flow of information with their participants, finding that this led to richer data (Keleher & Verrinder, 2003). In our case, we considered divulging information could alter participants’ experience of service use and thus invalidate data.
Difficulties Encountered
Our approach was threatened by a particular difficulty related to irregular participant starting dates. Although the period of data collection for each participant was 12 weeks, data were actually collected over 16 weeks as participants were recruited and became available for a first interview. Staggered commencement was appropriate but not staggered irregular commencement. Several weeks into the study the complexity of our mixed methods became apparent when multiple first and second interviews, plus telephone calls, began to mount in potentially unmanageable numbers in particular weeks. This dilemma was only fully appreciated when a composite matrix was drawn up, visually showing every participant and every data collection event over the actual calendar weeks. Consequently, the decision was taken to halt recruitment at 20 participants, instead of the anticipated 25, so as to maintain team workloads at reasonable levels. An additional protocol “Researcher progress sheet” would have enabled us to foresee this situation and will be added to future guidelines.
Other, less significant difficulties related to it being impossible at times to reconcile team members’ personal diaries (especially when participants had to change interview dates), resulting in a very few participants not receiving a three-member team visit at first interview. Similarly, in a very few instances, it was impossible to maintain a weekly phone call because the days of the week simply ran out before contact could be established. In both these situations, data were collected outside the planned timeframe. The complexity of data collection was also compounded by three participants not residing with their family member experiencing dementia (geographic considerations), and unpredictable behavior by family members with dementia when these family members were at home during interviews (consistent with our participant-centered procedures, the decision on how the person with dementia was best cared for during interviews was taken by the family carers). These situations required understanding and flexibility on our part.
The Potential of Mixed Methods Within a Pragmatic Paradigm for Data Collection in Dementia Research
We foresee that mixed methods will remain integral to dementia research, with researchers involving themselves increasingly in philosophical debates (Why are we thinking this mixed methods way? Where are we positioned amid present multiple viewpoints?) and rigor issues (What characterizes “good” mixed methods research?). We concur with Dunning et al. (2008) that mixed methods can go beyond triangulation and achieve both confirmation (e.g., in our study regarding carer burden) and understanding (e.g., regarding the relationship between services provision and dementia severity; Stirling et al., 2010). Discussion about a distinctive mixed methods methodology has rekindled debates about the quantitative–qualitative divide. Our research shared the complementarity purpose advocated by Sale et al. (2002), whereby mixed methods can be applied simultaneously or sequentially, provided care is taken in delineating the purpose for which each method is employed (e.g., in our research measurement of dementia severity vs. understanding of carer experiences). We thus achieved additive outcomes (regarding carer stress, dementia severity, and service utilization) as described earlier by Sale et al. (2002) and reported by us elsewhere (Stirling et al., 2010).
If advocates of a paradigm shift toward pragmatism are sufficiently influential, we envisage similarly comprehensive justifications for mixed methods research within a pragmatic paradigm. The views of Johnson and Onwuegbuzie (2004) were congruent with our own in that mixed methods partnered with pragmatism ultimately proved a “practical and outcome-oriented method of inquiry.”
Vexing issues persist however; for example, some will wish to keep philosophical debates alive, whereas others will prefer to remove themselves from such difficult matters. Interestingly, among the 20 mixed methods researchers interviewed by Bryman (2007), most described themselves as pragmatists, with only one wishing to take an alternative ontological stance. Optimism for mixed methods research is countered by Bryman’s (2007) research in which he identified mixed methods barriers in relation to different audiences, methodological preference, project structure, role of timelines (stages not synchronized), skill specialisms (potentially counterproductive), nature of data (some might be more striking), bridging the ontological divide, and publication issues whereby some journals emphasize either quantitative or qualitative research. Bryman expressed the view that considerable uncertainty still exists about what it means to integrate findings. Indeed, most mixed methods authors include cautionary notes about their confidence in mixed methods research. Some weigh up their hesitations: D. G. Morgan and Stewart (2002) reported that the benefits outweighed the many challenges experienced in their longitudinal dementia evaluation study, attributing the qualitative method to the ultimate success of interpreting the mixed methods results (likening it to the glue that held the results together). Such was the dynamic climate in which our research was located.
Although the literature especially highlights difficulties of integrating qualitative and quantitative findings in mixed methods research, we wish to highlight that, with dementia research, procedural pitfalls also await at the data collection stage. The most serious, we suggest, are postponing discussion of the implications of mixed methods until analyzing data, failing to develop advance protocols, and inadequate regard for extant knowledge about the lives led by those experiencing dementia and their family carers.
We take from our progressive engagement approach the suggestion that mixed methods data collection in dementia research should:
Be micromanaged by a multidisciplinary research team appropriately experienced in quantitative and qualitative methods
Be guided by project-specific mixed methods protocols that take account of the complexity of qualitative and quantitative data collection timeframes and time outlays for both researchers and participants
Be procedurally participant-centric, that is, all data collection procedures should include active measures to enhance the well-being of family carers and family members with dementia
Data collection in our mixed methods dementia research ultimately proved successful and rewarding (despite being complex, time consuming, and potentially prone to fragmentation). Our progressive engagement approach served us well in meeting all challenges and paved the way to mixed methods analysis and reporting of findings. We remain optimistic that dementia researchers will continue to collect and combine data in increasingly diverse and creative ways in the effort to counter the impacts of this baffling, global disease.
Footnotes
Acknowledgements
We thank all participants for their cooperation, time, and generous sharing of experiences.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:
We acknowledge the Australian Government Department of Health and Ageing and the Tasmanian Department of Health and Human Services for funding the project through the Home and Community Care Program.