Clinically representative research in the psychological therapies: an emerging paradigm

Abstract

This paper considers the tension between the ‘biology’ and the ‘therapeutic relationship’ camps within mental health nursing, which is focussed on differing conceptualisations of what it is that constitutes evidence and appropriate lines of inquiry. I argue that the struggle that mental health nurses have experienced in delivering evidence-based practice also resides in the thorny issue of methodological inquiry and how it is written into nursing practice. I would suggest that what is alienating to practitioners is the way in which evidence and evidence-based enquiry is generated. This paper offers a line of methodological inquiry that is based in clinical representativeness, a way of conducting and evaluating research to produce an evidence base that is informed from the ‘bottom up’ by ‘practice’. This paper not only contributes to the knowledge base of clinically representative research (CRR) but examines the ways in which this knowledge base has been produced and is currently perpetuated. The following themes are addressed:

Processes and outcomes in CRR: I identify key treatment outcomes and processes operating in practice settings and consider with what confidence we can make conclusions on observed outcomes in these settings. I also examine key moderating and mediating processes influencing such outcomes.

Practice of CRR: I reflect on the paradigm of CRR giving particular attention to how the paradigm has developed and is maintained. I look at how the relationship of CRR to effifficacy research has influenced paradigm development.

Future directions of CRR: I summarise the conclusions of CRR to date and identify points of consensus, disagreement and clarity suggesting in what areas the field has moved forward and whether there are instances where points of conflict have contributed to growth.

Keywords

clinically representative research evidence-based practice paradigm development practice-based evidence psychological therapies

In writing a paper on clinically representative research (CRR) in the psychological therapies I would like to begin by flagging a contemporary area of academic debate in mental health nursing, that which concerns the standing of the ‘therapeutic relationship’. This debate is essentially paradigmatic concerning the split in the profession between the therapeutic relationship and biology camps. The therapeutic relationship remains one of the dominant cornerstones of mental health nursing practice. The long-standing, if currently less acknowledged influence of Peplau (1952) has especially influenced its importance within British mental health nursing (Forchuk and Reynolds, 2001). In nursing practice, a large amount of nursing time is spent on administration, meaning that time spent talking with patients is often minimal, and when it does happen is rarely purposely therapeutic or informed by any theoretical framework or evidence base (Cameron et al., 2005). Forchuk and Reynolds (2001) argue that Mental Health Nurses (MHNs) struggle to implement evidence-based practice (EBP) and that this struggle is grounded in the divided identity of ‘psychiatric/mental health nursing’ itself: A profession divided between the ‘therapeutic relationship’ camp, focussing primarily on therapeutic factors, and the ‘biology’ camp, focusing on the primacy of biological factors and mental health problems.

One example of this ‘division’ was played out within British mental health nursing during the 1990s in the series of papers by two influential nurses Barker and Gournay. Barker criticised biomedical reductionism – the emphasis on EBP, randomised controlled trials (RCTs), and biological explanations of mental illness – supporting the work of Peplau (Barker, 1998; Barker et al., 1997, 1998). In contrast, Gournay criticised Peplau’s model among other un-validated nursing models, espousing instead a primarily biological underpinning to mental illness, arguing for ‘hard’ evidence from RCTs as the way forward for helping people with mental health problems (Gournay, 1995, 1997, 1999). Cameron et al. (2005: 70) describe how the ‘barrage of claims and counterclaims’ left ordinary nurses feeling confused, disillusioned and disempowered. Perraud et al. (2006), argue that in the course of this debate the strength of the biological paradigm loosened the traditional mooring of the therapeutic relationship and eroded the core identity of mental health nursing.

The reason I highlight this tension between the ‘biology’ and the ‘therapeutic relationship’ camps within mental health nursing here is that it seems to centre around differing conceptualisations of what it is that constitutes evidence and appropriate lines of methodological inquiry. That is proponents of the ‘biology’ camp who tend to favour hard evidence derived from RCTs would contest that models such as Peplau’s remain unvalidated due to the nature of the ‘non hard’ evidence that supports them. This relates to the debate of what it is that defines evidence. I would argue that the struggle that mental health nurses have experienced in delivering EBP resides in this thorny issue of what constitutes valid methodological inquiry and how it is written into nursing practice. By way of explanation I would argue that what may be alienating to practitioners, as encapsulated in the debate described above, in the is the way in which evidence and evidence-based enquiry is generated.

In this paper I offer a line of methodological inquiry that is based in clinical representativeness. By this I mean a way of conducting and evaluating research to produce an evidence base that is informed from the ‘bottom up’ by ‘practice’, an evidence base that will meet both the EBP agenda and support the development of psychological therapies in practice settings. It is my argument that refining paradigms of methodological enquiry will go some way to bridging schisms between ‘biology’ and ‘therapeutic relationship’ camps in mental health nursing.

I was fortunate enough to work with a team of researchers at the Psychological Therapies Research Centre who pioneered clinically representative ways of researching psychological therapies. This paper is partly based on a body of publications I submitted for my PhD by publication and which has been derived from working in this team dedicated to the furtherance of CRR, rendering research accessible to practitioners, and ultimately enabling research designs to be co-produced by practitioners. The publications reported here draw on key observed processes and outcomes in psychotherapy and contribute to an understanding of how the practice of CRR is currently developing. The publications that I present in this paper have, in my view, been cohesive in contributing to the development of the paradigm of CRR, and I present them here to invite a response and further develop the paradigm. As I engaged in team working my psychological therapies publications were multiple authored. Throughout this paper I use the singular 1st person pronoun (I) to refer to the current manuscript and the plural 1^st person (We) when referring to my joint-authored papers.

To arrive at a working definition of CRR I draw on key seminal texts including Barkham and Margison (2007), Nathan et al. (2000), Roth and Parry (1997) and Shadish et al. (1997, 2000) Broadly speaking, CRR is research that evaluates the effects of therapies as practised in real world, routine settings. CRR is, as it stands, a highly diffuse concept both in terms of nomenclature and definition. It encompasses highly diverse research designs and agendas, all of which purport to reflect real world settings. In recognition of such diversity, Shadish et al. (1997) conceived of CRR as a ‘spectrum’ of grades (i.e. 1–3), ranging from less to more clinically representative, with increasing levels of stringency in criteria for clinical representativeness (Table 1).

Table 1.

Box of definitions.

Term	Definition
Evidence-based practice	Evidence-based practice promotes the collection, interpretation, and integration of valid, important and applicable patient-reported, clinician-observed, and research-derived evidence. The best available evidence, moderated by patient circumstances and preferences, is applied to improve the quality of clinical judgments and facilitate cost-effective care.
Efficacy research	Efficacy research focuses on the measurable effects of specific interventions. In the clinical trial, one or more experimental treatments are compared with one or more control treat-ments. To maximise the likelihood of detecting treatment effects, factors that might obscure them are eliminated in efficacy studies as much as possible.
Clinically representative research	Clinically representative research is research that evaluates the effects of therapies as practised in real world, routine settings. Clinically representative research encompasses highly diverse research designs and agendas, all of which purport to reflect real world settings and in this supporting statement covers effectiveness research and practice base evidence.
Effectiveness research	Effectiveness research aims to establish whether efficacious treatments are feasible and have measurable, beneficial effects across broad populations in real world settings. As defined in this supporting statement, effectiveness research in contrast to practice-based research imposes constraints found in research trial methodology such as randomisation, treatment protocols, and manuals.
Practice-based evidence	Practice-based evidence promotes the collection of evidence derived from routine settings and measures what normally happens in practice settings. Practice-based evidence is driven by the aim of showing that procedures work and are effective in improving the quality of care in real-life practice settings.
Common factors/Non-specific factors	Common factors or non-specific factors refers to those factors of psychotherapy that are present in most, if not all, approaches to therapy. These techniques cut across all theoretical lines and are present in all psychotherapeutic activities.
Paradigm	Ways of looking at the world that define both the problems to be addressed and the range of admissible evidence that may be used to effect solutions.
Discourse	Set of rules or assumptions for organising and interpreting the subject matter of an academic discipline or field of study.
Epistemology	Branch of philosophy concerned with the nature and origin of knowledge. As the study of knowledge, epistemology is concerned with the following questions: What are the necessary and sufficient conditions of knowledge? What are its sources? What is its structure, and what are its limits?
Benchmarking	Benchmarking techniques utilise published outcome data as a comparator and can be employed to evaluate new service delivery models. Benchmarking is viewed as a key strategy for understanding and contextualising the effectiveness of routine data
Systematic review	Systematic reviews are designed to provide a comprehensive accounting of all evidence relating to a particular question or problem area. Studies are located through methods such as electronic database searching and the review follows a protocol to determine which studies have the necessary level of quality and are thus eligible for inclusion in the review of the available literature
Reliable and clinically significant improvement	Reliable and clinically significant improvement yields two criteria. First there must be reliable change – that is, change that has occurred over the result of therapy over and above the fluctuations of a measuring instrument. Second there must be a clinically significant result which indicates that the client’s score has moved sufficiently such that it falls within the range of scores expected from the general population rather than from the clinical population.

More recently, a distinction has been made between effectiveness research (which imposes constraints found in research trial methodology such as randomisation, treatment protocols, and manuals) and practice-based research (which measures what normally happens in practice settings). These two approaches are differentiated in Cahill et al. (2010) and Barkham and Margison (2007). I propose that CRR can be re-configured as an umbrella term covering these two inter-related, but conceptually distinct, approaches. This conceptualisation of CRR complements and maps on to Shadish’s spectrum of grades of clinical representativeness (interested readers may refer to Shadish et al. (1997) for more background on the criteria employed in that study to differentiate between grades). This conceptual distinction is useful and important: Effectiveness research and practice-based research have important differences in terms of aims, study design, and agenda. However, both offer an evidence base that is distinct from efficacy research, which derives from formal randomised trials and has as its aim the analysis of the specific effects of a given intervention. Accordingly, the work reported in this paper provides examples of research from both the effectiveness and practice-based components of the spectrum. In order to contextualise discussion of these issues more clearly, I have provided a box of working definitions of terms that I use throughout this paper.

The primary purpose of this paper is to outline the ways in which the paradigm of CRR is developing. In so doing, I am proposing an epistemology for the way in which psychological therapies – their evaluation and practice – are generated.

Key issues

In considering key issues in CRR, I acknowledge the importance of not only contributing to the knowledge base of CRR (through providing examples of primary and secondary psychological therapies research in this paper) but to examining the ways in which this knowledge base has been produced and is currently perpetuated (by providing a worked example of how the conceptual map has helped to chart the development of CRR). In essence, an examination of paradigm development. In this paper, I focus on a subset (listed below) of the key issues reported in Cahill (2010), which present as especially pertinent to this themed issue and the interests of practitioner-researchers:

Outcomes and processes in CRR: What are the key treatment outcomes and processes in practice settings? What can be said about the observed outcomes in practice settings? What are key moderating and mediating processes influencing such outcomes? With what confidence am I able to make a judgement on outcomes in clinically representative settings?

Practice of CRR: How has the paradigm of clinically representative research developed? How does this body of publications sit within the paradigm? Are there instances where points have conflict have contributed to growth? What are the indications for the future development of the paradigm?

Future directions of CRR: What are the conclusions to date? Where are the points of consensus and disagreement? Where are the points of clarity? In what areas has the field moved forward?

Outcomes and processes in CRR

In this section I consider some key outcomes and processes in psychological therapy across a range of settings differing in levels of clinical representativeness.

Assessment outcomes

These refer to the range of outcomes (as defined by the Clinical Outcomes in Routine Evaluation-Outcome System) following clients’ assessment prior to therapy. These types of outcomes are of significance in that they represent the client’s experience of seeking therapy and mark the beginning of the therapeutic journey. In an analysis of 34 primary care services, I and co-authors found that a mean of 81% of clients were accepted into therapy, with significant variation among services ranging from 34% at the lower limit to 100% at the upper (Cahill et al., 2006). Such variation flags ethical considerations in terms of a real possibility that large numbers of clients in some services have a negative experience of seeking therapy. However, I would emphasise that assessment outcomes are especially context dependent and context bound, being determined by a range of contextual factors influencing proportion of clients taken up for therapy such as availability of alternative services, waiting times, population demographics, commissioner expectations, and referrer pressures to take ‘hard to help’ clients.

The matter of context raises pressing methodological questions for practitioners and researchers alike concerning how these benchmarks are produced and interpreted, and how this evidence contributes to the knowledge base of CRR. Responsible reading of benchmarks, accompanied with a degree of reflection, has potential to lead to an approach that is responsive to context. For example, regarding the proportion of clients accepted to therapy, Cahill et al. (2006) note that a high level of clients accepted into therapy might indicate a high level of appropriate referral or it could mean that the service is being uncritical in determining which clients are suitable for therapy. It is only knowledge of practitioner and service parameters that can result in appropriate measurement against benchmarks.

Therapeutic outcomes

These refer to end state outcomes following a period of therapy. Consideration of therapeutic outcomes in CRR conducted with colleagues at Leeds Depression Research Clinic (LDRC) indicated that therapy completers returned superior therapeutic outcomes – a better end-state – than non-completers (Cahill et al., 2003). Seventy-one percent of the completers but only 13% of non-completers achieved reliable and clinically significant improvement (RCSI) on their Beck Depression Inventory (BDI) scores: a finding consistent with clinical lore and common sense clinical expectations. Reliable and clinically significant improvement refers to change that is reliable (change that occurs over and above the imprecise fluctuations of a measuring instrument) and change that is clinically significant (that is change that returns the client to the normal population. Interested readers are referred to Jacobson and Truax (1991) for a full explication of this concept and criteria. However, this seemingly clear-cut finding raises some interesting methodological and clinical questions for practitioner-researchers.

First, the magnitude of the differences between these two groups of clients was only made possible by the methods of RCSI. The groups were clearly differentiated on the parameter of ‘reliable and clinically significant improvement’ (71% vs. 13%), which suggests that a client has made enough reliable change across therapy to rejoin the normal population. In contrast, observed group differences between completers and non-completers after matched numbers of sessions provided by standard inferential statistics (repeated measures ANOVA) elided the clinical relevance of the different outcomes between groups.

Second, while providing greater differentiation between groups, RCSI methods provided greater differentiation within groups. It was found that, while standard inferential statistics simply demonstrated that non-completers as a group made statistically significant change over therapy, RCSI methods indicated that although only 13% of these clients achieved the more stringent criteria of reliable and clinically significant improvement, the majority (70%) of these clients had achieved reliable improvement (that is change that was reliable but not of a sufficient magnitude to return them to the normal population). This important therapeutic outcome, while not meeting the desired outcome of RCSI is nevertheless indicative of a degree of improvement that would be of clinical importance to the practitioner and of personal importance to the client.

What Cahill et al. (2003) highlight is that, in order to produce outcome data that are methodologically and clinically useful, it is essential to employ measurement techniques and outcome indices that are fit for purpose. I describe this issue in terms of responsiveness in outcome measurement, an issue that will recur throughout this paper.

Benchmarking outcome data from the Cahill et al. (2003) study against data from efficacy research, provided support for another trend consistent with clinical lore and EBP wisdom: effects observed in efficacy trials tend to be attenuated when transferred to real world settings. Accordingly, Cahill et al. (2003) observed that the pre-post effect sizes (ESs) for cognitive behavioural therapy (CBT) at 1.71 were lower than those observed in efficacy randomised controlled trials (RCTs) by about 0.7. However this difference when translated into differences in percentiles, was modest amounting to approximately 3%. Reduced therapy effectiveness has been confirmed by other benchmarking studies. That is, Paley et al. (2008) found that the ESs from psychodynamic interpersonal therapy (PIT) as practised in routine setting were less favourable than when used in efficacy trials. Moreover, Cahill et al. (2010) provided evidence of reduced effectiveness of therapy in routine settings in a systematic review of practice-based research on psychological therapies. When benchmarked against efficacy studies, practice-based studies yielded effect sizes that fell short of the selected benchmark.

While these publications converge in demonstration of superior outcomes observed in efficacy research (research trials), they also raise questions pertinent to the establishment of CRR as a unique paradigm and the politics of reading outcomes from such research. So, while I acknowledge that publications such as Cahill et al. (2003), and Paley et al. (2008) provide evidence of therapies’ reduced effectiveness in clinically representative settings, I would also, following Shadish et al. (2000), underline important differences between the research agendas of efficacy research and CRR and the implications this has for ‘reading’ the outcomes. That is efficacy research is concerned with establishing the effectiveness of a treatment in controlled conditions while CRR is concerned measuring what normally happens in practice and whether procedures are transferable from controlled settings. The practice of benchmarking CRR against efficacy trials, while on one level appearing to confirm the hierarchy of evidence, also draws attention to the issue of responsiveness in the reading of results. That is, we should be aware of important differences in design issues between clinical trials and natural settings such as inclusion and exclusion criteria, client heterogeneity, therapist differences. These differences do of course impact on outcomes. Such information is essential for framing the reading of therapeutic outcomes.

Moving away from comparisons with efficacy research to comparisons within CRR reveals other issues relevant to methodology and theories about therapy effectiveness. For example, Paley et al. (2008), in benchmarking against other CRR studies, found that PIT was comparable with results of other therapies (including CBT) in routine settings. I suggest that this finding provides a degree of support for broad equivalence of outcomes in routine settings as observed by Stiles et al. (2006, 2008). What does equivalence of outcomes indicate? Put another way, do treatment specific effects become more attenuated in routine settings? One argument could be that in routine settings where therapists do not practice under constraints observed in RCTs, equivalent outcomes indicate therapists’ responsiveness to clients in the context of the unfolding therapeutic relationship rather than adherence to any specific model. Such therapist responsiveness is, I suggest, indicative of trans-theoretical processes that take place in psychotherapy.

The issue of therapist responsiveness in therapy is interestingly mirrored by the issue of researcher responsiveness in methodology. Cahill et al.’s (2010) review of practice-based research underlines the importance of ensuring that measurement techniques are responsive to the research question and to the agenda of building up an evidence base that is relevant to the epistemology of CRR. The review demonstrated a high incidence of studies that did not report sufficient data for the calculation of ESs and rates of recovery (17/31: 55%), meaning that they could not be benchmarked. One consequence will be the generation of an evidence base that does not accurately reflect the magnitude of therapeutic outcomes in clinically representative settings.

In addition, I and my co-authors encountered some challenges linked with quality appraisal methods. In using the Downs and Black (1998) quality appraisal instrument, we found that the criteria were ill-matched to the design and agenda of practice-based research. Although the Downs and Black checklist had been designed to evaluate both randomised and non-randomised trials, it did not prove to be responsive to both the design features and theoretical underpinnings of practice-based research. Some features seen only in terms of deficit (e.g. lack of control group, heterogeneous nature of patient population, non-manualised treatments) enable practice-based research to accurately reflect routine practice. To address this disparity, we revised and adapted the checklist to render it applicable for practice-based research (Cahill et al., 2010).

Therapeutic processes in CRR

This section considers key processes in psychotherapy, derived predominantly from an evaluation of cognitive therapy that was conducted in the LDRC.

In Hardy et al. (2001), I and co-authors found that the impact of client interpersonal styles (under-involved) on outcome as measured by the BDI was mediated by the therapeutic relationship between client and therapist. That is, when therapeutic alliance was entered into the regression equation, under-involved interpersonal style was no longer a significant predictor. As with the outcome research described above, we suggested this provides increasing support for the influence of common factors in therapy practised in routine settings. We argued that what took place in therapy was ‘appropriate responsiveness’ whereby therapists optimise the reactions of individual clients to treatment as it unfolds, leading to an attenuation of relationships between individual characteristics (such as interpersonal styles) and outcome. As in consideration of outcome research in routine settings, this prompts the question of whether the phenomenon of therapist responsiveness becomes more pronounced in routine settings. In a related LDRC study using an increased sample size (Saatsi et al., 2007), we found further support for the mediation model. In this study clients’ secure interpersonal style was predictive of treatment outcome but this impact was again mediated by the therapeutic alliance.

Similarly in Hardy et al. (2005) we observed that sudden gains (reduction in clients’ symptom intensity during psychotherapy, which occurs suddenly between the beginning of one session and the beginning of the next, rather than gradually over the course of treatment) took place in a range of treatment conditions (i.e. therapies) in addition to cognitive behavioural therapy. This finding led us to believe that, rather than sudden gains indicating a cognitive shift resulting from model specific factors, sudden gains could be suggestive of more trans-theoretical processes in psychotherapy. That sudden gains were less stable (that is, the sudden gains were not sustained throughout therapy) in routine treatment conditions, could also be indicative of trans-theoretical model of psychotherapy, such as ‘appropriate responsiveness’ noted in the earlier Hardy et al. (2001) study. According to this logic, responsiveness is allowed greater expression in routine settings where treatment adherence is not monitored, manuals and protocols are not used, and therapists are free to use a variety of techniques.

Building from research carried out at LDRC, I led on a health technology assessment (HTA) review of measures of therapist–patient interaction (Cahill et al., 2008; Hardy et al., 2007). The report contained a detailed conceptual model with three developmental processes identified as necessary for the provision of an effective therapeutic relationship: establishing a relationship, developing a relationship, and maintaining a relationship (Figure 1). Key therapeutic processes establish what might be called ‘mini outcomes’ or ‘objectives’ for each phase. It is assumed that, although these processes develop across therapy, there will be a cycling through these processes within a therapeutic meeting or over a number of weeks or months. For example, a therapeutic relationship may be well developed when there is a break in treatment, resulting in patient-reported dissatisfaction. The therapist will work to repair this rupture in the relationship and may also return to the use of engagement skills. Therapist, patient, and contextual factors determine the nature of the roles and frameworks within which the therapeutic interactions take place. These, in turn, impact on the processes and outcomes of each developmental process of the relationship.

Figure 1.

Conceptual model of the therapeutic relationship.

The reason I present the map here is, not only to indicate the evidence base for the key trans-theoretical processes operating in psychological therapy, but to prefigure the basis for paradigm development discussed later in this paper. As I argue later, the conceptual map of the psychotherapy relationship is highly adaptable to the conceptual map of discourse development, and ultimately paradigm development in so far as shared frameworks of understanding may be said to have the properties of a relationship upon which contextual factors and dynamic processes exert their influence.

Practice of CRR

A key question relates to the processes by which the paradigm of CRR is being established. Barkham and Margison (2007) attribute the emergence of practice-based evidence (PBE) to the unease felt when one paradigm such as EBP assumes dominance. Accordingly, they present the theory of chiasmus to describe the construction of PBE via a reversal in the order of words in the parallel phrase ‘evidence-based practice’. Hence, a complementary paradigm of PBE emerges that transcends the either-or dichotomy. According to such a paradigm, efficacy research and CRR are not pitched against the other but combine to generate an evidence base that draws on the differing characteristics of the two approaches.

While the concept of chiasmus is an advance on dichotomy, it may still be restricting in terms of paradigm development in that the phrase ‘practice-based evidence’ is defined as the inverse of EBP. Barkham and Margison (2007: 442) insert the phrase ‘practice-based evidence’ into Sackett’s (1996) definition of evidence-based medicine to generate an alternative paradigm so that ‘practice-based evidence is the conscientious, explicit, and judicious use of current evidence drawn from practice settings in making decisions about the care of individual patients. Practice-based evidence means integrating both individual clinical expertise and service-level parameters with the best available evidence drawn from rigorous research activity carried out in routine clinical settings’. By such an insertion, this complementary model of paradigm development is still defined by the parameters of evidence-based medicine which, to a certain degree, limits the possibilities of PBE.

Conceptual map of discourse development

As a contribution to the development of the paradigm of CRR, I draw on the conceptual map of discourse development presented in Figure 2, derived from the conceptual map of the therapeutic relationship (Figure 1) (Cahill et al., 2008; Hardy et al., 2007) and first presented at the 5th International Mixed Methods Conference 2009 (Freshwater and Cahill, 2009).

Figure 2.

Conceptual map of discourse development.

Freshwater and Rolfe (2004: 58) point to Thomas Kuhn’s definition of a paradigm as ‘ways of looking at the world that define both the problems that can legitimately be addressed and the range of admissible evidence that may bear on their solutions’. It is therefore reasonable to posit that discourses directly underpin paradigm development. This conceptual framework can be used as a heuristic aid to shape awareness and reflection about how a paradigm is formed.

Freshwater (2007: 135) defines a discourse as ‘set of rules or assumptions for organizing and interpreting the subject matter of an academic discipline or field of study’. However, the sets of rules and assumptions draw their energy from the reader/audience and as such are constantly open to dynamic processes that I would argue are inherently relational in as much as the reader/audience interpret and respond to the discourse. It therefore seems appropriate for discourse development to be understood as the formation of a relationship and underpinned by the processes of engagement, maintenance and development which all involve responsiveness and relationality between parties. Understanding paradigm development as a relational activity allows unpacking of the processes through which the discourses surrounding evidence are established in the area of psychological therapies. This conceptual framework also allows for the recipient of any given research paradigm to assume a more active stance in so far as they can position themselves within the discourse and impact on its course and development.

Development of this conceptual map concerns issues of knowledge generation and production: that is, epistemology. I believe it is important to cultivate an awareness of how discourses surrounding evidence are produced and perpetuated as it enables critical reflection on the validity of evidence and how such evidence is used: a key activity of research and academic scholarship. Moreover, the ability to stand outside a research paradigm and observe its development enables one to impact its direction. Using the conceptual map as a guide in the section that follows, I suggest how the paradigm of CRR has emerged within the context of an evidence-based culture and hypothesise how it might develop.

Paradigm shifts

Considering the development of CRR with recourse to the conceptual map (Figure 2), I may attribute the shift to contextual factors such as consumers (practitioners) seeking clinically meaningful research that applies to routine practice which is relevant at the level of the individual client. This has been outlined earlier. In terms of establishing a practice-based discourse, this has, in part, been activated by seminal publications (e.g. Shadish et al., 1997, 2000), by high quality CRR, and by a dedicated readership.

In the maintenance phase, a way to actively perpetuate the discourse of CRR is to ensure that research outputs are high impact and likely to attract funding. To ensure impact, it is essential to include check on quality control. Cahill et al.’s (2010) review of practice-based research identifies potential methodological improvements (e.g. comprehensive reporting and appropriate statistical techniques). Similarly, in the adoption of systematic review technologies to CRR, Cahill et al. (2010) suggest that appraisal criteria need development so as to enhance the quality and impact of systematic reviews of CRR.

The development phase is most critical for the future of CRR in terms of how the discourse can be progressed and refined. Areas of progression such as appraisal criteria for practice-based studies (Cahill et al., 2010) and systematic reviews (Cahill et al., 2008) have been highlighted. Attention must also be given to potential fractures and anomalies within the discourse. For instance, Cahill et al. (2010), indicated areas of inconsistency between research studies in procedures used (e.g. in reporting data and estimating effects of clinical significance). Moreover, Cahill et al. (2008), highlighted anomalies characterising our early attempts to rate the alliance measures, such as using a rating scheme designed for the appraisal of outcome measures (Fitzpatrick et al., 1998). In this case I contend that these initial points of dissonance have potential to lead to growth and the strengthening of the paradigm. And, consequently, there is now a rating tool for the appraisal of studies associated with client-practitioner interaction measures (Cahill et al., 2008) and a quality appraisal tool adapted from the Downs and Black instrument (Downs and Black, 1998) for the appraisal of practice-based studies in systematic reviews.

Development of a discourse can sometimes lead to deconstruction of a discourse. By this I mean the process whereby the ‘unpacking’ of terminology during the development and refinement of a discourse can lead to significant reformulation of the discourse. In this case I would argue that the discourse of CRR is still in the development phase. With further scholarship and debate in this area (which is relatively young) the discourse may undergo significant restructuring and said to be deconstructed.

Future directions of CRR

In this section I offer some tentative concluding remarks based on my consideration of CRR in the psychological therapies.

Refinement of CRR methodology

As identified above, specific aspects of CRR methodology would benefit from further development, benchmarking being a fitting exemplar: it has been applied innovatively in CRR to enable detection of equivalence of outcomes between differing therapeutic approaches (as in Paley et al., 2008) as well as important differences in outcomes between routine settings such as primary and secondary care (Cahill et al., 2010).

Continued development of quality appraisal criteria

The application of systematic review technologies to the evaluation of CRR and the development of process measures holds promise but is challenging. The main issues are the development of appropriate inclusion and sensitive quality appraisal criteria that do not unfairly advantage practice-based research but which are responsive to its theoretical basis. Specific quality appraisal criteria have been presented in the form of a rating tool in Cahill et al. (2010).

Scope for refining the paradigm of CRR

In this paper I have presented an explanation for how the paradigm of CRR has emerged and indications for its development. In particular, I have highlighted how points of dissonance may limit the uptake of CRR. In proposing this new paradigm of enquiry I would emphasise that it is a paradigm that is still emerging. In line with my proposed conceptual map of discourse development, the paradigm of CRR is open to a number of dynamic processes and contextual factors. The key feature of the paradigm is responsiveness to context both in the generation and interpretation of evidence and a creative approach to points of dissonance as a way of refining the paradigm.

Key points

Researchers and practitioners have differing conceptualisations of what constitutes evidence and appropriate lines of methodological enquiry.

This paper offers a line of methodological enquiry that is based in clinical representativeness.

Clinically representative research encompasses effectiveness research (which imposes some of the constraints founds in formal trial methodology) and practice-based research (which evaluates what normally happens in practice settings).

This paper provides examples of psychological therapies research from both effectiveness research and practice-based research and thus outlines the way in which the paradigm of clinically representative research is developing.

Footnotes

Conflict of interest statement

None declared.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Jane Cahill is a Senior Research Fellow at the School of Healthcare, University of Leeds, who has published widely in the field of psychological therapy effectiveness research. Her current programme of research includes the therapeutic alliance, practice-based evidence approaches, workforce mental health issues, mental health and employment support and mixed methods research approaches. Jane has recently led on an externally funded knowledge transfer partnership (KTP) project which concerned the development and validation of a model of employment support for mental health service users. This project was in collaboration with Leeds Partnership NHS Foundations Trust and had strong service user involvement and representation.

References

Barker

(1998) The future of the theory of interpersonal relations? A personal reflection on Peplau’s legacy. Journal of Psychiatric and Mental Health Nursing, 5(3): 213–220.

Barker

Keady

Croom

(1998) The concept of serious mental illness: Modern myths and grim realities. Journal of Psychiatric and Mental Health Nursing, 5(4): 247–254.

Barker

Reynolds

Stevenson

(1997) The human science basis of psychiatric nursing: Theory and practice. Journal of Advanced Nursing, 25(4): 660–667.

Barkham

Margison

(2007) Practice-based evidence as a complement to evidence-based practice: From dichotomy to chiasmus. In: Freeman

Power

(eds) Handbook of Evidence-Based Psychotherapies: A Guide for Research and Practice, Chichester, UK: Wiley, 443–476.

Cahill J (2010) Clinically representative research: An emerging paradigm. PhD Thesis, University of Leeds, UK.

Cahill

Barkham

Hardy

(2003) Outcomes of patients completing and not completing cognitive therapy for depression. British Journal of Clinical Psychology, 42(2): 133–143.

Cahill

Barkham

Hardy

(2008) A review and critical appraisal of measures of therapist patient interactions in mental health settings. Health Technology Assessment, 12(4): ix–47.

Cahill

Barkham

Stiles

(2010) Systematic review of practice-based research on psychological therapies in routine clinic settings. British Journal of Clinical Psychology, 49(4): 421–453.

Cahill

Potter

Mullin

(2006) First contact session outcomes in primary care psychological therapy and counselling services. Counselling and Psychotherapy Research, 6(1): 41–49.

10.

Cameron

Kapur

Campbell

(2005) Releasing the therapeutic potential of the psychiatric nurse: A human relations perspective of the nurse–patient relationship. Journal of Psychiatric and Mental Health Nursing, 12(1): 64–74.

11.

Downs

Black

(1998) The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of Epidemiological Community Health, 52(6): 377–384.

12.

Fitzpatrick

Davey

Buxton

(1998) Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment, 2(14): 1–74.

13.

Forchuk

Reynolds

(2001) Clients’ reflections on relationships with nurses: Comparisons from Canada and Scotland. Journal of Psychiatric and Mental Health Nursing, 8(1): 45–51.

14.

Freshwater

(2007) Discourse, responsible research and positioning the subject. Journal of Psychiatric and Mental Health Nursing, 14(2): 111–112.

15.

Freshwater D and Cahill J (2009) Practice of mixed methodologies in mental health nursing research. In: 5th International Mixed Methods Conference, Harrogate, UK, 8–10 July 2009.

16.

Freshwater

Rolfe

(2004) Deconstructing Evidence-Based Practice, London, UK: Taylor & Francis.

17.

Gournay

(1995) What to do with nursing models. Journal of Psychiatric and Mental Health Nursing, 2(5): 325–327.

18.

Gournay

(1997) Responses to: ‘What to do with nursing models’: A reply from Gournay. Journal of Psychiatric and Mental Health Nursing, 4(3): 227–231.

19.

Gournay

(1999) In research, money talks. Nursing Times, 95: 18–18.

20.

Hardy

Cahill

Barkham

(2007) Active Ingredients of the Therapeutic Relationship that Promote Client Change: A Research Perspective, New York, NY: Routledge.

21.

Hardy

Cahill

Shapiro

(2001) Client interpersonal and cognitive styles as predictors of response to time-limited cognitive therapy for depression. Journal of Consulting and Clinical Psychology, 69(5): 841–845.

22.

Hardy

Cahill

Stiles

(2005) Sudden gains in cognitive therapy for depression: A replication and extension. Journal of Consulting and Clinical Psychology, 73(1): 59–67.

23.

Jacobson

Truax

(1991) Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1): 12–19.

24.

Nathan

Stuart

Dolan

(2000) Research on psychotherapy efficacy and effectiveness: Between Scylla and Charybdis? Psychological Bulletin, 126(6): 964–981.

25.

Paley

Cahill

Barkham

(2008) The effectiveness of psychodynamic-interpersonal therapy (PIT) in routine clinical practice: A benchmarking comparison. Psychology and Psychotherapy: Theory, Research and Practice, 81(2): 157–175.

26.

Peplau

(1988) Interpersonal Relations in Nursing, London, UK: MacMillan. (Originally published in 1952.).

27.

Perraud

Delaney

Carlson-Sabelli

(2006) Advanced practice psychiatric mental health nursing, finding our core: The therapeutic relationship in 21st century. Perspectives in Psychiatric Care, 42(4): 215–226.

28.

Roth

Parry

(1997) The implications of psychotherapy research for clinical practice and service development: Lessons and limitations. Journal of Mental Health, 6(4): 367–380.

29.

Saatsi

Hardy

Cahill

(2007) Predictors of outcome and completion status in cognitive therapy for depression. Psychotherapy Research, 17(2): 185–195.

30.

Sackett

Rosenberg

Gray

(1996) Evidence based medicine: What it is and what it isn't. BMJ, 312(7023): 71–72.

31.

Shadish

Matt

Navarro

(1997) Evidence that therapy works in clinically representative conditions. Journal of Consulting and Clinical Psychology, 65(3): 355–365.

32.

Shadish

Navarro

Matt

(2000) The effects of psychological therapies under clinically representative conditions: A meta-analysis. Psychological Bulletin, 126(4): 512–529.

33.

Stiles

Barkham

Mellor-Clark

(2008) Effectiveness of cognitive-behavioural, person-centred, and psychodynamic therapies in UK primary-care routine practice: Replication in a larger sample. Psychological Medicine, 38(5): 677–688.

34.

Stiles

Barkham

Twigg

(2006) Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies as practised in UK National Health Service settings. Psychological Medicine, 36(4): 555–566.