The evaluation of complex interventions in palliative care: An exploration of the potential of case study research strategies

Abstract

Background: Complex, incrementally changing, context dependent and variable palliative care services are difficult to evaluate. Case study research strategies may have potential to contribute to evaluating such complex interventions, and to develop this field of evaluation research.

Aim: This paper explores definitions of case study (as a unit of study, a process, and a product) and examines the features of case study research strategies which are thought to confer benefits for the evaluation of complex interventions in palliative care settings.

Results: Ten features of case study that are thought to be beneficial in evaluating complex interventions in palliative care are discussed, drawing from exemplars of research in this field. Important features are related to a longitudinal approach, triangulation, purposive instance selection, comprehensive approach, multiple data sources, flexibility, concurrent data collection and analysis, search for proving-disproving evidence, pattern matching techniques and an engaging narrative. The limitations of case study approaches are discussed including the potential for subjectivity and their complex, time consuming and potentially expensive nature.

Conclusions: Case study research strategies have great potential in evaluating complex interventions in palliative care settings. Three key features need to be exploited to develop this field: case selection, longitudinal designs, and the use of rival hypotheses. In particular, case study should be used in situations where there is interplay and interdependency between the intervention and its context, such that it is difficult to define or find relevant comparisons.

Keywords

Organizational case studies evaluation studies as topic program evaluation research design

Introduction

Complex interventions are ‘conventionally defined as interventions with several interacting components’.¹ They are acknowledged to present particular evaluation challenges, for example, relating to difficulties standardising interventions, sensitivities to features of local contexts, issues using experimental methods to investigate service or policy changes, and the length and complexity of the causal chains linking intervention with outcome.¹ Many evaluation issues in palliative care concern complex, inter-related, already provided services, and so an exploration of evaluation methods in this field is warranted.

Evaluations in complex palliative care contexts are difficult to address with standard experimental methods, for a number of reasons. First, services are variable. Services incrementally change, have different components, and can be inconsistently used or provided. Multiple services from different providers (and with different funding streams) can be concurrently or sequentially provided to patients. This variability means that treatment content and ‘doses’ received can be different or difficult to determine.² Second, the deteriorating nature of participants in palliative care research means that there are frequently problems with retention, attrition or incomplete follow up.³ Third, randomisation or finding an appropriate comparator group is challenging, as there may be few providers, and there are ethical issues about withholding already-provided services. Fourth, purely quantitative experimental approaches give little information about causation, providing scant explanatory information about why certain effects were or were not found. And fifth, experimental approaches tend to examine isolated aspects of a complex system, but such systems are not simply the sum of its components, but also the intricate relationships between components.⁴

One approach that can address these complexities and requirements may be case study, as such research strategies generally address multiple variables in numerous real-life contexts, often where there is no clear, single set of outcomes.^5–9 Case study strategies therefore warrant further investigation for use in complex evaluations in palliative care settings.

What is case study research?

A common definition of case study research is elusive. Definitions of case study vary in their focus on defining the unit of study (the case), the process (the case study), and the product of the investigation (the study report).¹⁰ A hypothetical example can be used to illustrate these definitions and their application in practice. Let’s take the example of a study evaluating the introduction of a new type of palliative care worker to support patients at home, implemented across a number of primary care organisations. The intervention is complex, because it is provided in different organisations, in different ways that reflect the differences in existing palliative care provision, and has developed and changed over time.

Case based definition of case study

Case study is most commonly defined by the choice of case rather than the choice of methods or approach.¹¹ The case should describe a bounded phenomenon (i.e. person, programme, service, group, policy or instance of something), where what will or will not be studied can be defined with reference to time, place, event or activity.^5,12

In our hypothetical example, the case could be defined in a number of ways. The case could be the new palliative care worker, with each worker being the phenomenon under investigation, defined perhaps with reference to the patients they visit and the professionals they interact with. Equally, the case could be the primary care organisation and the palliative care provided within its boundaries, or perhaps the patient in receipt of palliative care. Each of these choices may be appropriate depending on the research question being asked, the focus of the evaluation, and whether cases with and without the new worker are required. Such ‘bounding’ of the case can provide an appropriate ethical framework for the study. An example is a study of dying within an intensive care unit where the case approach limited the observational ‘range’ to events concerning selected patients and was argued to be more suited to the sensitive and emotionally charged setting.¹³

Once the case is defined, the selecting of the instances of the case to be studied is crucial.¹⁴ Cases are usually selected purposively, although convenience or probability selection can also be used.

Process based definition of case study

Case study can be defined in terms of the process of enquiry:

‘A case study is an empirical enquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.’¹⁵

The process is clearly related to the bounded nature of the case and its relationship to its context:

‘A case study is a method for learning about a complex instance, based on a comprehensive understanding of that instance obtained by extensive description and analysis of that instance taken as a whole and in its context.’¹⁴

In our example, this would mean using research methods that examined the palliative care worker in context, and involve multiple methods and multiple stakeholders. Thus there may be observation of the palliative care worker, interviews with the worker, health and social care colleagues, patients, carers, commissioners, documentary analysis, etc. Case studies can also incorporate the collection of quantitative data for example surveys, or the mapping of outcome data such as place of care and death. Case study is not assigned to a fixed methodological position⁵ and bringing qualitative and quantitative evidence and methods together can be regarded as a strength of the method.⁸

Product based definition of case study

Merriam¹² defines case study in terms of the end point: ‘A case study is an intensive, holistic description and analysis of a single instance, phenomenon or social unit’. For some, the study narrative produced is critical to the success of case study as an evaluation method:

‘The challenge for evaluators is how to use those aspects of an anecdote that are effective … the immediacy, the convincingness, the attention-getting quality – and at the same time fulfil other informational requirements … such as generalisability and reliability. Case study methods, whilst not without their limitations in this regard, can help us answer this challenge’¹⁴ (p.8).

In our example, the presentation of the narrative generated from data analysis would be critical. It would need to use contextual knowledge to resonate with readers’ experiences and understandings.

The benefits of case study evaluations

Ten features of case study evaluations are thought to be beneficial (see Box 1).¹⁴ These are discussed and illustrated by examples of case studies conducted in palliative care settings, the details of which are displayed in Table 1.

Table 1.

Examples of case studies conducted in palliative care settings

Author, date, country	Research question	Case	Participants	Methods	Findings
Bergen 1992, UK	To examine and establish how district nurses and community care nurses felt standards of care for the dying were achieved, and to establish the relationship between nursing assessment and the patient’s perception of need and care	Multiple cases: the triple perspective of patient, and their district nurse and continuing care nurse	District nurses, continuing care nurses, patients	Interviews with all participants and a district nurse / continuing care questionnaire on standard setting	Most structure, process and outcome standards were met, or compensatory mechanisms were available to deal with issues. Outcomes criteria were difficult to measure and were nurse oriented
McKenna et al. 2004, Northern Ireland	To evaluate the role and work of an oncology nurse practitioner	Not explicitly stated: assumed to be the oncology nurse practitioner	Patients, managers, health care professionals and nurse practitioners	Descriptive information collected about the case holder. Interviews	Findings focused around the importance of choice, the ONP consultation, contacts outside formal appointments, the criteria for the post, the benefits of local review clinics and the quality of the postholder
Fitzsimons et al. 2007, Northern Ireland	To explore the palliative care needs of patients with a non-cancer diagnosis.	Not explicitly stated: assumed to be the patient	Patients, carers and their clinical team	Qualitative interviews, focus groups, SF36 and HADs to evaluate patients functional status and emotional state	Central theme: detioriorating health status leading to decreased independence, social isolation and family burden. Resources and support perceived as lacking.
Moriarty et al. 2007, UK	To describe and evaluate the implementation of two Macmillan nurse facilitator posts	Single case: the pilot project of two Macmillan nurse facilitators working together within a defined geographical area.	Stakeholders including managers, Macmillan nurse facilitators, GPs, Hospice staff, district nurses	Data collected in two phases over two years, which included interviews with stakeholders, focus groups, questionnaires, observation of meetings and the facilitators work, site visits and documentary analysis	Focus on the establishment of the roles and the key activities of the postholders including facilitating educational programmes, establishing a resource nurse network, and supporting the use of clinical guidelines. Role credibility was important and project ownership
Pietroburgo and Bush 2008, USA	To examine the collaboration efforts of two non-profit hospice organisations	Not explicitly stated: assumed to be the two hospice organisations	Hospice staff and board	Survey to hospice staff, interviews. Documentary analysis. Longitudinal approach to data collection	Four ongoing sources of tension that affected collaboration progress: differences in professional values and practice, struggles for community support, differences in organisational culture and competition for market share. Able to track why the third collaborative attempt was most successful.
Mahmood-Yousuf et al. 2008 Munday et al. 2007,UK	To explore the effectiveness and sustainability of the implementation of the Gold Standards Framework (GSF) within general practices	Not explicitly stated: assumed to be general practices / primary health care teams who had implemented the Gold Standards Framework within three selected primary care trust areas	District nurses, general practitioners, practice managers, specialist palliative care staff and other key informants	Interviews, observation of practice and locality meetings, inspection of practice systems and documents	High, medium and low GSF performers recognized against identified criteria. Considerable variation between practices. Recognised the importance of strong process and effective professional relationships. Professionals valued different styles of communication
Van Raak et al 2008, Netherlands	To identify the characteristics of the palliative care network that enabled or hindered cooperation amongst participants	Single case: the Palliative Care Network Embedded units of analysis: organisations that participated in the network; longitudinal design	Project leader (network coordinator), managers, support staff, doctors, nurses	Interviews and documentary analysis (project plans, letters, reports, minutes, memos)	Divergence between routines of network members hampers cooperation. Divergence may be overcome by negotiation and transformational leadership
Della Penna et al. 2009, USA	To understand the factors supporting or impeding the rapid and consistent spread of a complex programme within Kaiser Permanante (IPC – introducing interdisciplinary in-patient palliative care)	Not explicitly stated: assumed to be the sites implementing IPC programmes	National and regional leaders from within the organisation, hospitals and medical groups	In-depth semi-structured interviews	Identified factors promoting spread (evidence, impact on quality and satisfaction, costs, networking opportunities, observing original model) and factors inhibiting spread (resources, funding responsibilities, leadership)

HAD, Hospital Anxiety and Depression Scale; ONP, oncology nurse practitioner

Box 1.

Anticipated benefits of case study methods for evaluation purposes (United States General Accounting Office 1990)

Design	Benefits expected
Longitudinal	Assurance that a short-term situation that may be unrepresentative of what is happening isn’t inflated in importance
Triangulation	Assurance that reasons given for events properly reflect influences from many different sources
Purposive instance	Ability to match questions asked and later generalisation of findings at level appropriate to the questions
Data collection	Benefits expected
Comprehensive	Assurance that important conditions, consequences and reasons for these have not been overlooked
Flexible	Broader perspectives; increased assurance that what is important on the scene rather than centrally will be examined
Multiple data sources	Assurance that a full picture will be obtained and that bias associated with self-protection or self-interests will be minimised
Analysis	Benefits expected
Concurrent with data collection	Assurance of ability to collect data to test alternative interpretations and make rapid adjustments in design
Search for disproving–proving evidence	Assurance that alternative interpretations have been thoroughly searched for and checked – identification of instances that don’t fit general pattern; understanding of outliers
Chain-of-evidence and pattern matching techniques	Permits fairly direct assessment of how convincingly the evidence of conclusions are related
Reporting	Benefits expected
Actual instances persuasiveness	Assurance of authenticity – generalise from objective experiences

Design features

Case study design features which are thought to benefit complex evaluations include facilitating a longitudinal approach, triangulation, and selecting purposive instances.

One longitudinal case study was the two-year evaluation of a Macmillan nurse facilitator role.¹⁶ This allowed investigation of both the processes involved in the establishment of the posts and the agreed indicators of success. Who the facilitators were (i.e. background and role) was as important, if not more so, than what they did (i.e. training). Another longitudinal case study examined hospice collaboration over a 12-year period.¹⁷ This facilitated understanding that the long-term processes and relationships over time shaped the end product of the collaboration. These longitudinal case study designs minimised the possibilities that an unrepresentative short-term situation wasn’t inflated in importance.¹⁴ The time-ordered analyses allowed investigation of events where the consequences were larger or smaller than expected.

Case studies using triangulation include an evaluation of the Gold Standards Framework,¹⁸ of referrals within community palliative care services,¹⁹ and studies of bereavement services.²⁰ Triangulation generally requires different methods of data collection (such as interviews, observations, routine data and documentary evidence), brought together as data collection proceeds.⁶ In these studies triangulation included observation and audit information used together to examine and corroborate Gold Standards Framework adoption,¹⁸ and comparison of the similarities and differences between self-reported and observed roles of staff.¹⁹ The advantages are that these data give multiple reference points for verifying patterns, ruling out alternative explanations and clarifying meaning.^14,21

The choice of case can be for different reasons: because they have a particular interest (i.e. an oncology nursing role,²² a particular instance of end-of-life care,²³ cooperation in a palliative care network,²⁴ and new hospice services²⁵); they are representative (i.e. maximising variation between in-patient palliative care sites,²⁶ variation between sites implementing the Gold Standards Framework,^18,27 or between patients with chronic illness²⁸); or are typical (i.e. evaluating the care of the dying in one locality²⁹). These careful selections of cases should facilitate an appropriate matching of the questions and later generalisation of findings, ensuring the best possible explanations of phenomena.^14,21

Data collection features

Case study data collection features which are thought to benefit complex evaluations include their comprehensive nature, use of multiple data sources, and flexibility.

Many of the exemplar studies highlight comprehensive data collection approaches as particular strengths of their studies. Bergen²⁹ highlights the triple perspective (patient, district nurse, continuing care nurse) of respondents commenting on the same situation; McKenna et al.²² collect data from patients, managers, the oncology nurse and colleagues; Mahmood-Yousuf et al.²⁷ highlight these comparisons as a strength of the study, examining dynamic relationships within practices, and Payne et al.²⁰ found the approach encouraged the collection of concurrent data, so that participants are referring to similar time frame. The benefits of these comprehensive approaches should be that important conditions and consequences are not overlooked, and any bias associated with self-protection or self-interests reduced.¹⁴

Flexible, open interview techniques are frequently used, facilitating a focus on participants topics and issues arising during the study^20,22,28,29. However, studies also report what appear to be less flexible data collection approaches, based predominantly on pre-existing literature²⁶. Few studies directly report changes in data collection plans as a result of initial findings, although there are reports of extending data collection to include unanticipated health care professionals.²⁰ These flexibilities mean that what is important ‘on the scene’ is examined rather than centrally determined prior to study commencement.¹⁴

Analysis features

Case study analysis features which are thought to benefit complex evaluations include conducting analysis concurrently with data collection, searching for disproving-proving evidence, and chain of evidence and pattern matching techniques.

Methods of analysis in the example palliative care studies draw from different traditions including content analysis,^16,29 matrix analysis,¹⁸ interpretative phenomenological analyses,²² and Colazzi’s phenomenological analysis.²⁸ Some report the use of techniques such as pattern matching and constructing chains of evidence,^18,19 or explanation building strategies,²⁴ but reporting constraints appear to preclude sufficient reporting of analysis to determine the use of these features. Analysis concurrent with data collection should give assurance that data can be collected as needed to test alternative interpretations and to make rapid adjustments in design. This is linked with the search for proving–disproving evidence, which means that alternative explanations have been thoroughly searched for, often through identification of instances that don’t fit patterns and understanding outliers. Constructing chains of evidence and pattern matching in analysis should then permit assessment of how convincingly the evidences of conclusions are related.¹⁴

Publication features

The case study as a published narrative serves two functions, to engage the reader and to enable a judgement of the quality of the case study methods and procedures. In the exemplar studies, some use direct quotes from respondents extensively,¹⁹ whilst others create a narrative about the case(s).^16,24 Case studies are also presented differently to illuminate different findings, such as two reports of the same study which respectively mostly use direct quotations²⁷ or narrative descriptions of high, medium and minimal performing cases.¹⁸. Four criteria are proposed to assess case study reports: resonance (between the report and its underpinning paradigm); rhetoric (the form, structure and presentation of the case); empowerment (to evoke and facilitate action); and applicability (facilitation of inferences that have applicability in the readers context).³⁰ The exemplar studies demonstrate how challenging it is to achieve these criteria within the confines of a journal report.

The limitations of case study evaluations

Case study strategies are often assumed to be limited because other forms of knowledge are believed to be superior, such as the context-independent knowledge generated by a trial, or because of concerns that generalisation is not possible from a limited number of cases, or about the risk of bias.³¹ Authors such as Flyvberg³¹ clearly articulate counter-arguments to these assertions, but this does not mean that case studies are without limitations.

One limitation is the possibility of partiality or the inability to generalise from the findings when there is subjectivity, inaccuracy or bias within the case study.¹⁴ Case study methods inherently require subjective and judgemental elements, but when proper procedural safeguards (such as a clear audit trail) are used these elements should not diminish the value of the method.¹⁴ Case studies can also be expensive, complex and time-consuming because they frequently require in-depth data collection in multiple settings for extended periods. The use of multiple data collection methods can also make analysis and the interpretation of findings difficult.

Research should be problem-driven not methods-driven, and case study may not be appropriate to answer the evaluation question posed. It is imperative to consider what type of evaluation question is asked, and then examine both the strengths and limitations of different types of case study for answering it, as well as alternative research strategies.

Discussion

Case studies have been used in palliative care settings to examine a range of research questions, including evaluation focused questions. Key features of case study research strategies, especially the use of multiple perspectives, facilitate the achievement of evaluation goals. However, there is still much potential to use case study research strategies in evaluation research in palliative care, as some features of case study research that are thought beneficial in evaluation contexts have not been fully exploited. Researchers seeking to use case study strategies need to consider three main issues in order to develop and improve the evaluation of complex evaluations: case selection, longitudinal design, and the use of rival hypotheses.

Appropriate case selection is described as critical to the success of case study evaluation methods, but few case studies either define the case explicitly or give a clear rationale for case selection. As an example, the rationale for case selection is not explicitly discussed in the study of Gold Standards Framework adoption. The authors discuss a rationale for the choice of Primary Care Trusts involved in the study, but the Primary Care Trust is not the ‘case’, rather the GP practice is described as the case unit, and it is not clear how this was selected.¹⁸ In a similar fashion, the case study of inpatient palliative care changes does not define the case nor the ‘key dimensions of interest’ against which participants were sampled.²⁶ It is essential that researchers clearly identify the case to allow a match between the case and the research questions asked, and to facilitate the generalisable explanation of phenomena in a way that is meaningful to those responsible for managing change.

Longitudinal case-study designs also offer much potential to evaluators. Exemplar longitudinal case studies do not explicitly discuss or draw upon the features of a longitudinal study thought to be important in evaluation research, nor do they indicate the importance of the longitudinal nature of the studies in determining the findings. Whilst longitudinal studies are notoriously difficult to analyse,³² they have much potential to evaluators. Not only can a longitudinal case study design be used to examine change after an intervention, but also the collection of multiple data sources from different perspectives at different time points allows understandings of how, why and when change occurred. This is powerful data for evaluators and practitioners alike in determining the use of complex interventions.

The explicit specification and testing of hypotheses and rival hypotheses is also a key case-study feature that has potential to be further developed in the use of case-study evaluations of complex interventions. Yin⁸ in fact argues that the case study feature with the greatest potential impact on evaluation is this substitution of explicit rival hypotheses for the use of control or comparison groups. This recognises the difficulties in conducting evaluations of complex interventions, or where a programme or organisations are the focus of evaluation. In these situations, evaluators are unlikely to have the control (or finances) available to manipulate a control or comparison group. In addition, suitable control or comparison groups may be difficult to identify. Yin argues that rigorous evaluation designs can still be developed.⁸ Rather than dealing with rival explanations by inference (where a control group rules out rival explanations without necessarily identifying them), evaluators can specify the key rival explanations at the commencement of an evaluation. This allows them to be directly investigated, for example, by tracking alternative sequences of events within the intervention to be evaluated⁸.

Case study, therefore, has potential to contribute to the field of evaluating complex interventions in palliative care and elsewhere. Case study is not suitable for all evaluations, and has a range of limitations. However, when researchers are considering options for designing rigorous evaluation research, case study strategies should be one of the options under serious consideration.

Footnotes

Funding

Catherine Walshe was funded by an NIHR Primary Care Researcher development award to conduct case study research upon which some of this article is based. Catherine Walshe is currently funded by a RCUK post-doctoral fellowship.

References

Medical Research Council. Developing and evaluating complex interventions: new guidance 2008. London: MRC, 2008.

Lipsey

Cordray

. Evaluation methods for social intervention. Annu Rev Psychol 2000; 51:345–375.

Jordhøy

Kaasa

Fayers

Underland

Ahlner-Elmqvist

. Challenges in palliative care research; recruitment, attrition and compliance: experience from a randomised controlled trial. Palliat Med 1999; 13:299.

Anderson

Crabtree

Steele

McDaniel

. Case study research: the view from complexity science. Qual Health Res 2005; 15: 669–685.

Luck

Jackson

Usher

. Case study: a bridge across the paradigms. Nurs Inq 2006; 13: 103–109.

Yin

. Enhancing the quality of case studies in health services research. Health Serv Res 1999; 34: 1209–1224.

Walshe

Caress

Chew-Graham

Todd

. Case studies: a research strategy appropriate for palliative care? Palliat Med 2004; 18: 677–684.

Yin

. Discovering the future of the case study method in evaluation research. Eval Pract 1994; 15: 283–290.

Yin

. Applications of case study research. Thousand Oaks, CA: Sage, 2003.

10.

Anthony

Jack

. Qualitative case study methodology in nursing research: an integrative review. J Adv Nurs 2009; 65: 1171–1181.

11.

Stake

. Case studies. In: Denzin

Lincoln

(eds). Handbook of qualitative research. Thousand Oaks, CA: Sage Publications, 2000, pp. 435–454.

12.

Merriam

. Qualitative research and case study applications in education. San Francisco: Josey-Bass publishers, 1998.

13.

Seymour

. Negotiating natural death in intensive care. Social Sci Med 2000; 51: 1241-1252.

14.

United States General Accounting Office. Case Study Evaluations. Washington, DC: United States General Accounting Office, 1990.

15.

Yin

. Case study research. Design and method. 3rd ed. Thousand Oaks, CA: Sage, 2003.

16.

Moriarty

O’Hara

Byron

. Macmillan nurse facilitators for palliative care: evaluation of a pilot project. Int J Palliat Nursing 2007; 13: 334–343.

17.

Pietroburgo

Bush

. Coming to terms: a case study of hospice collaboration challenges. Am J Hospice Palliat Med 2008; 24: 487–492.

18.

Munday

Mahmood

Dale

King

. Facilitating good process in primary palliative care: does the Gold Standards Framework enable quality performance? Fam Pract 2007; 24: 486–494.

19.

Walshe

Chew-Graham

Todd

Caress

. What influences referrals within community palliative care services? A qualitative case study. Social Sci Med 2008; 67: 137–146.

20.

Payne

Field

Rolls

Hawker

Kerr

. Case study research methods in end-of-life care: reflections on three studies. J Adv Nurs 2007; 58: 236–245.

21.

Stake

. Case studies. In: Denzin

Lincoln

(eds). Handbook of qualitative research. Thousand Oaks, CA: Sage, 1994.

22.

McKenna

McCann

McCaughan

Keeney

. The role of an outreach oncology nurse practitioner: a case study evaluation. Eur J Oncol Nursing 2004; 8: 66–77.

23.

Cooke

Hurley

. A case study exploring the ethical and policy dimensions of allocating acute care resources to a dying patient. J Clin Nurs 2007; 17: 1371–1378.

24.

van Raak

Paulus

Cuijpers

te Velde

. Problems of integrated palliative care: A Dutch case study of routines and cooperation in the region of Arnhem. Health Place 2008; 14: 768–778.

25.

Ingleton

Field

Clark

. Multidisciplinary case study as an approach to the evaluation of palliative care services: two examples. Int J Palliat Nursing 1997; 3: 335–339.

26.

Della Penna

Martel

Neuwirth

Rice

Filipski

Green

. Rapid spread of complex change: a case study in inpatient palliative care. BMC Health Serv Res 2009; 9: doi: 10.1186/1472-6963-9-24510.1186/1472-6963-9-245.

27.

Mahmood-Yousuf

Munday

King

Dale

. Interprofessional relationships and communication in primary palliative care: impact of the Gold Standards Framework. Br J Gen Pract 2008; 58: 256–263.

28.

Fitzsimmons

Mullan

Wilson

Conway

Corcoran

Dempster

. The challenge of patients’ unmet palliative care needs in the final stages of chronic illness. Palliat Med 2007; 21: 313–322.

29.

Bergen

. Evaluating nursing care of the terminally ill in the community: a case study approach. Int J Nurs Stud 1992; 29: 81–94.

30.

Lincoln

Guba

. Judging the quality of case study reports. In: Huberman

Miles

(eds). The qualitative researchers companion. Thousand Oaks, CA: Sage, 2002.

31.

Flyvbjerg

. Five misunderstandings about case-study research. In: Seale

Gobo

Gubrium

Silverman

(eds). Qualitative research practice. London: Sage, 2004, pp. 420–434.

32.

Faithfull

. Analysis of data over time: a difficult statistical issue. J Adv Nurs 1997; 25: 853–858.