PROTOCOL: Personal Budgeting Interventions to Improve Health and Social Care Outcomes for People with a Disability: A Systematic Review

The Problem

More than a billion people – or about 15% of the world's population – are estimated to live with some form of disability, and these rates are increasing over time (World Health Organisation [WHO], 2013). The International Classification of Functioning, Disability and Health (ICF), defines disability as an umbrella term for impairments, activity limitations and participation restrictions. According to the WHO, disability is the interaction between individuals with a health condition (e.g., cerebral palsy, Down syndrome, and depression) and personal and environmental factors (e.g., negative attitudes, inaccessible transportation and public buildings, and limited social supports) (WHO, 2013). The WHO (2013) recognises that disability is extremely diverse, but that generally, rates of disability are increasing due to population ageing and a greater prevalence of more chronic health conditions, whilst people with disabilities also have less access to health care services and, therefore, more unmet needs than ever before. There is further evidence to suggest that people with disabilities have lower life expectancies.

The many different needs of people with a disability, learning difficulty or mental health problem tend to be met through a range of activities, which may be described, collectively, as ‘social care’. These might include help with personal hygiene, dressing and feeding, or general life skills such as shopping, keeping active, and socialising (Malley et al., 2012). In recent years, the disability and mental health sectors have witnessed a significant shift towards community-based health and social care services that attempt to place the service user at the centre of decision-making and service delivery. A growing body of policy now describes how people with disabilities should be autonomous and self-determined members of society. The concept of self-determination has its roots in self-determination theory, which is based on human motivation, development and wellness. According to Deci and Ryan (2008) the theory focuses on the type and quality of motivation as a predictor of performance and well-being outcomes, as well as social conditions that are improved by such motivations. Autonomous motivation, in particular (compared to controlled motivation) — whereby intrinsic and extrinsic motivation allows individuals to identify with an activity's value and integrate it into their sense of self — can lead to better psychological health, performance and a shift toward healthier behaviours. Compared to amotivation, ‘controlled motivation’ can lead to improvements. However, these improvements are limited, since individuals feel pressure to think, feel and behave in certain ways (in order to avoid shame or to gain approval from the external regulation), when functioning under a system of reward or punishment. Self-determination theory also examines the impact of self-determination on life goals and aspirations and can be applied to a wide range of domains, including relationships, work, education and health care (Deci & Ryan, 2008). The findings of a recent meta-analysis of 184 studies – based on self-determination theory in health care and health promotion contexts – showed positive relationships between the satisfaction of psychological needs, autonomous motivation and positive health outcomes (Ng. et al (2012). A number of more specific studies which have examined self-determination in a sample of people with a disability, found similarly positive outcomes (Perreault & Vallerand, 2007; Saebu, Sørensen, & Halvari, 2013).

One way to achieve self-determination is by means of a personal budget (“Convention on the Rights of Persons with Disabilities,” 2006). Personal budgets are rooted in the Independent Living Movement and the associated Independent Living Fund, whereby people with a disability self-directed their support by hiring a ‘personal assistant’ (PA) to gain more control over their lives and services. While the concept of independent living varies internationally, all approaches emphasise choice and control whilst acknowledging that personal budgets are just one way to achieve their goals (Jon Glasby & Littlechild, 2009). A personal budget, also known as ‘individualised funding‘, is an umbrella term for various funding mechanisms that aim to provide personalised and individualised support services for people with a disability. Whilst the terminology may vary, the principles remain consistent and are based on self-determination, choice and, very often, person centred planning. Thus, personal budgets aim to place the service user at the centre of the decision making process, thereby recognising their strengths, preferences and aspirations and empowering them to shape public services, social care and support by allowing the service user to identify their needs, and to make choices about how and when they are supported (Carr, 2010). As a result, many international governments are recommending personal budgets as a means to empower individual service users or their advocates, whilst ensuring transparency in the allocation and use of resources.

For example, in Ireland, there are several key policy goals (e.g. enshrined in the Value for Money and Policy Review of Disability Services (Department of Health, 2012)) which promote the use of ‘individual needs assessments‘. These assessments can lead to a personal budget which can then be used to purchase services from within existing (limited) resources (Keogh, 2011). In the UK, personal budgets are common and are facilitated by standardised resource allocation systems that include a robust needs assessment. Furthermore, a social care outcomes framework is in place to monitor how well social care services are delivering the most meaningful outcomes for people with disabilities whilst also addressing any shortcomings therein (Department of Health, 2013). The monitoring process is supported by tools such as the Adult Social Care Outcomes Toolkit (ASCOT) which was used, for example, in an evaluation of personal budgets commissioned by the UK Department of Health (Forder et al., 2012). This tool comprises eight conceptually distinct attributes or domains including: personal cleanliness and comfort; food and drink; control over daily life; personal safety; accommodation cleanliness and comfort; social participation and involvement; occupation; and dignity (Malley et al., 2012).

There are several types of personal budget which can be used to address these kinds of health and social care needs; the two most common involve either a direct payment model or a brokerage service.

A direct payment involves the funds being given directly to the person with a disability, who then self-manages this money to meet their individual needs, capabilities, life circumstances and aspirations (Aiseanna Tacaiochta, 2014b). This may include the employment of a personal assistant to help with everyday tasks and/or the purchase of services from private, voluntary or community service provider organisations (Carter Anand et al., 2012). Direct payments often involve considerable administrative duties for the person with a disability and are more likely, therefore, to be utilised by people with a physical or sensory disability and less so by those with an intellectual or developmental disability. However, in some cases, a person with a mild intellectual disability may have the skills to manage the direct payment, with or without the support of family members or other natural supports (or informal care). More severe intellectual disabilities would most likely require some kind of family/natural support. This review endeavours to determine whether the benefits of direct payments are affected by the type and degree of disability, or indeed the involvement of third parties whether paid or unpaid.

A brokerage model or ‘managed’ personal budget, on the other hand, whilst it provides a similar amount of freedom for the person with a disability around choice and control of services utilised, involves a broker assuming responsibility for administrative tasks and providing support, guidance and information to enable the person to successfully plan, arrange and manage their support services or care plans (Carr, 2010). The tasks of a broker include working with the person with a disability to develop an individual action plan, as well as researching options within the community to fulfil the goals in the action plan. The broker can also assist in negotiating costs with service providers and are available for support of the individual when necessary (PossibilitiesPlus, 2014). Brokerage models tend to have a far reaching impact across service provision and local authority purchasing by encouraging more flexible and innovative solutions for user-orientated services, whilst also influencing the development of payment schemes (Zarb, 1995).

Whilst the involvement of brokers is ongoing, their presence in the life of the individual tends to be more intensive in the initial transition (i.e. from traditional services) and set-up stages. During this period, the broker will help to develop the ‘circle of support‘, either from scratch when none currently exists, or by expanding an existing support structure to include extended family members, such as aunts, uncles, cousins, friends and members of the wider community. During this initial period, the broker may also assist in the recruitment of staff for day-to-day support. For this reason, this review will seek to determine whether or not these intervention effects differ based on the level and quality of support available, both paid and unpaid. Some research suggests that the circle of support is integral to the successful implementation of such an intervention (Curryer, Stancliffe, & Dew, 2015; Fleming, McGilloway, & Barry, 2015b). Furthermore, the quality of paid support may also affect outcomes since the provision of broker/facilitator training has been found to be a successful element of individualised models of support (Fleming et al., 2015b; Lord & DeVidi, 2015).

A third type of model, the Cash and Counselling model, is found predominantly in the US and allows the user the flexibility to choose between a self-managed and a professionally managed/assisted account. This represents a combination of the direct payment and brokerage models described above (National Resource Center for Participant-Directed Services, 2014). In many jurisdictions, the brokerage/support function which facilitates planning and implementation, is separated from the ‘fiscal management’ supports which handle the accounting and human resource issues, but not the personal planning/support/monitoring element. While these can be conflated in some cases, it is generally considered important to maintain the independence of the brokerage/planning function from the fiscal dimension to avoid conflict of interest. The separation of the two allows individuals or advocates who do not wish to have any planning support to secure the ‘payroll’ services required without any obligation to avail of planning and monitoring supports.

While ‘individualised funding’ is emerging as an umbrella term for the various funding mechanisms, terminology remains unclear. A decade ago, ‘cash-for-care’ or ‘cash and care’ were predominant umbrella terms when reviewing evidence over several decades from the US, UK and EU (Caroline Glendinning & Kemp, 2006; Ungerson & Yeandle, 2008). These early studies highlighted the risks associated with the marketisation and indirect privatisation of care services whereby ‘consumers of care’ increasingly act as employers without necessarily having the human resource skills or knowledge of available care choices (Woods, 2008). In contrast, evidence suggests that people availing of personal budgets are capable of acquiring the necessary skills, or indeed able to outsource certain tasks in order to successfully bypass the service providers and contract their support services directly(Fleming et al., 2015b). Thus, there exists a tension between individuals with a disability, who can secure potential cost savings while having more autonomy, and traditional service providers who need to maintain contractual agreements with staff members within their organisations.

Further tensions may also exist for frontline staff between their ethical obligations to promote empowerment and self-determination whilst honouring their legal obligations to limit access to personal budgets (Ellis, 2007). Another challenge for staff relates to risk management. A balancing act is required to facilitate positive risk-taking whilst ensuring that the personal budgeting-specific risks, such as financial abuse, neglect or physical/emotional abuse, are avoided. This requires careful consideration and planning, but risk management can vary considerably. For example, during the piloting of personal budgets in the UK, local authorities conducted risk assessments but in some cases relied on annual reviews, thereby placing the onus of responsibility on individuals or families in the interim (C. Glendinning et al., 2008). Carr and Robbins (2009) also highlight the region-specific contextual factors, such as culture and policy, which can influence implementation of personal budgets. For example, in Canada, the US and the Netherlands, it is compulsory to use an independent support broker, whilst in the UK and US, ‘personal assistants’ are the preferred option for those receiving personal budgets. The eligibility criteria may also differ at initial implementation depending on the region. For example, in Canada, the focus was on younger people with learning disabilities whereas the Swedes focused on adults with physical disabilities; furthermore, very few regions accommodated people with mental health problems. Objectives also differed; for example, Australia initially focussed on tackling fragmented service provision, particularly in rural areas, while the US concentrated on solving staff shortages in long-term care facilities (Carr & Robbins, 2009).

All of the above interventions, regardless of delivery mode, involve a transitionary period which can be difficult for individuals and families to manage, particularly when national systems of allocating resources are not in place and families have to negotiate the release of funds from a regional disability manager, as is the case, for example, in Ireland (Fleming et al., 2015b). This period of transition can also be a time of great uncertainty for individuals and their families (where applicable) who have left a form of service provision to which they have been accustomed, often for many years. As a result, the length of time that the intervention has been in place may considerably affect its real or perceived effects. Furthermore, socio-demographic factors may have a similar impact; for example, an older person may have been using traditional forms of services for much longer than a young adult transitioning from mainstream school or another form of secondary education. Thus, past experiences, such as institutionalisation, may dramatically affect an older person's ability to adapt to this new model of service provision. Equally, more people living in rural areas have been found to avail of day services when compared to urban dwellers, potentially due to a lack of alternatives within the community (Fleming, McGilloway, & Barry, 2015a). This dependence on traditional day services may impact an individual's ability to adjust to the new model, or could limit the potential for community integration due to a lack of community services for the general population. Therefore, this review will take such confounding factors into consideration, both in the inclusion/exclusion criteria and in the subgroup analysis.

The Intervention

For the purposes of this review, the intervention will include any form of personal budget, regardless of the name given to the model of delivery. As indicated above, these models may be described in many different ways. For example, Webber et al. (2014) identified: ‘Individual Budgets’; ‘Recovery Budgets’; ‘Personal Budgets’; ‘Direct Payments’; ‘Direct Health Budgets’; and ‘Cash and Counselling’. Others include ‘third party managed’ personal budgets, direct payments managed by an appointed person and individual service funds. However, a personal budget, to be included in this review, must have the following fundamental characteristics: (1) It must be provided by the state as financial support for people with a physical, sensory, intellectual, developmental disability or mental health problem; (2) the recipient must be able to freely choose how this money is spent in order to meet their individual needs; (3) the individual can avail of “brokerage” services or any equivalent service which supports them in terms of planning and managing how the money is used over the lifetime of the funding period; (4) the recipient can also independently manage the personal budget, in whatever way is feasible, such as setting up a “Company Limited by Guarantee” as is the case in Ireland (Aiseanna Tacaiochta, 2014a); and (5) the personal budget may be provided as a ‘once-off pilot intervention for a defined period of time (minimum 6 months), or it can be a permanent move from more traditional forms of funding arrangements that exist nationally or regionally.

Personal budgeting interventions are implemented with a view to delivering a range of positive health and social care outcomes over time. It is expected that a persons' quality of life will improve (e.g. socially, personally, environmentally and in terms of their physical / psychological health) as a result of their increased autonomy, choice and control over daily life decisions and greater social integration and interaction. Client satisfaction is also expected to improve due to greater self-determination. By increasing independent life skills (i.e. taking on more responsibilities such as shopping and household chores) physical functioning also has the potential to improve. Many of these quality of life measures if improved, would arguably generate greater cost benefits, although evidence is limited. The limited pool of evidence would suggest that personal budgets can be cost effective, ranging from 7% to 16% in the US (Conroy, Fullerton, Brown, & Garrow, 2002) to 30% to 40% in the UK (Zarb & Nadash, 1994). In contrast, another UK study suggested that personal budgets may not result in cost savings but do represent value for money (John Glasby & Littlechild, 2002). Stainton, Boyce, and Phillips (2009) support these more conservative findings showing relative cost neutrality for personal budgets compared to independent service providers, however personal budgets were more cost effective than traditional in-house service provision. Furthermore they reported greater user satisfaction for those availing of personal budgets, highlighting the link between client satisfaction, quality of life and cost benefits.

Rationale for the Intervention

The international move towards personal budgets has led, in turn, to a growing interest in identifying methods, more generally, that might offer the most potential in terms of informing effective and efficient resource allocation, particularly in the context of recent economic reforms. However, these strategic and policy decisions would appear to be evolving on the basis of locally sourced or anecdotal evidence, since there appears to be a lack of high quality experimental studies in the area (Webber et al., 2014). Nonetheless, current international evidence suggests many benefits of personal budgets, such as increased choice and control, and a positive impact on quality of life (QoL), cost effectiveness and reduced service use (Field, 2015; Webber et al., 2014). A theory of change seeks to explain the pathways/mechanisms that lead to change (in this case positive change) and to determine the links between activities, outputs and outcomes (Taplin, Clark, Collins, & Colby, 2013). In the case of personalised budgets, people with disabilities are meant to have more autonomy over their lives (e.g. by having a say in every decision that affects them) which, in turn, acts as a mechanism to enhance self-determination, something that most people without a disability take for granted. A mantra that resonates globally within the disability sector is “Nothing about us, without us” (Charlton, 1998). This aptly illustrates the fundamental need to place the person with a disability at the centre of decision making. Thus, personal budgets and attendant services are designed as a vehicle/mechanism for potentially improved health and social care outcomes. Such individualised funding arrangements are also important in shifting the power dynamic from service providers and placing it in the hands of individuals with a disability (or their families).

Glendinning et al. (2008) reported mixed findings in their RCT on the impact of a personal budget on health, social care and personal outcomes within their subgroup analyses. Outcomes varied according to age or mental health status, whilst the type of disability did not appear to play an important role (C. Glendinning et al., 2008). Furthermore, health outcomes may vary across various jurisdictions where different rules exist on what can or cannot be funded from a personal budget – particularly health services which may have different eligibility rules by region. Importantly, international evidence on personal budgeting models suggests that there is no ‘one size fits all’ approach for everyone; hence, there is considerable variation with regard to: levels of choice and control given to service users; the professionals involved; the type of funder; and the limitations in both the services available for purchase and administrative structures/ processes (Carter Anand et al., 2012).

It is notable that the type of study design also varies considerably in the evaluation of personal budgets. Studies include, but are not limited to: RCTs (C. Glendinning et al., 2008; Shen et al., 2008); quasi-experimental trials with controls (Forder et al., 2012; Foster, Brown, Phillips, & Schore, 2003; Teague & Boaz, 2003); and without controls (Spaulding Givens, 2011); cross-sectional surveys (Hatton & Waters, 2011; Lawson, Pearmain, & Waters, 2010); and qualitative studies (Coyle, 2009; Homer & Gilder, 2008; Maglajlic, Brandon, & Given, 2000).

Prior Reviews

We are aware of only two reviews, to date, which have specifically examined personal budgets for people with a disability or mental health problem. Both of these included quantitative and qualitative data. The first, by Carter Anand et al. (2012), was a rapid evidence assessment rather than a rigorous systematic review. As a result, the search strategy had some major limitations, such as the exclusion of non-English studies and a geographical restriction to 7 countries including: the United States; Australia; Germany; Great Britain; Ireland; Netherlands and New Zealand. The authors acknowledged that the search strategy had resulted in a limited evidence base, which precluded the possibility of drawing strong conclusions about the implementation and impact of personal budgets. However, they also indicated that the qualitative evidence derived from service users tended to reflect positive views about the initiatives. The review did not report on the characteristics of included studies, or on study results in any detail. Furthermore, there was no detail about whether or not a meta-analysis was conducted, or the methods by which the qualitative data were synthesised. In addition, no subgroup analyses were conducted despite an apparent broad definition of disability (e.g. various types and level of physical and intellectual disabilities, inclusion of older people and those with mental health problems). Finally, while quality was assessed, no information was provided on any assessment of bias.

The second more recent review by Webber et al. (2014) closely followed the EPPI-Centre methodology for conducting a systematic review, appraising methodology and assessing the research quality and reliability (Gough, Oliver, & Thomas, 2012). Once again however, non-English studies were excluded, but more importantly, the focus of this systematic review was on mental health only; other physical or learning disabilities were included only if they coexisted with mental health problems. Fifteen studies were included in the review and the main findings showed that personal budgets can have positive outcomes for people with mental health problems in terms of choice and control, impact on QoL, service use and cost-effectiveness (Coyle, 2009; Davidson et al., 2012; C. Glendinning et al., 2008; Spandler & Vick, 2004). However, methodological shortcomings, such as variation in study design, sample size, and outcomes assessed, were reported to limit the extent to which the study findings could be accurately interpreted or generalised. This was compounded by considerable variation in the support models included, but without any attempt to undertake a sub-group analysis (e.g., ‘Personal Budget’ versus ‘Direct Payment’ versus ‘Recovery Budget’ versus ‘Cash and Counselling‘). Consequently, the authors concluded that more large, high quality, experimental studies were required before any definitive conclusions could be reached (Webber et al., 2014).

Contribution of this Review

We are not aware of any systematic review that focuses on the effectiveness of personal budgets in relation to people with a disability of any form, including mental health problems. Given the new policy imperative around personal budgets and the growing pool of studies in this area, there is now a need for a systematic review of these models (when compared to a control) across a spectrum of disabilities, in order to assess their effectiveness in relation to health and social care outcomes. A supplementary synthesis of the non-controlled evaluations and qualitative studies will also be included in order to capture these valuable findings in an area that is relatively new. Due to the complex nature of implementing innovative initiatives that challenge the status quo, many qualitative studies have been undertaken to capture important perspectives, successes and challenges and these cannot, therefore, be overlooked in this review.

This review will: (1) assess the effectiveness of personal budgeting interventions; (2) utilise subgroup analyses to explore how effects may differ by various client and intervention parameters; and (3) appraise and synthesise the experiences of key stakeholders. The ultimate aim of this review is to provide useful, robust and timely data to inform service providers/organisations working in the field of disability and to provide a rigorous evidence base on which decisions by policy makers (and drivers) can be made around different resource allocation/personal budgeting models to support greater choice and control by individuals in their daily lives.

Characteristics of the Studies Relevant to the Objectives of the Review

Eligible study designs for questions relating to the effectiveness of the personal budgeting intervention will include randomised, quasi-randomised and cluster-randomised controlled trials. Due to the complex nature of the intervention and attendant ethical constraints, randomisation may not be possible since the aim of personal budgets is to increase choice and control, and randomisation limits this option. Therefore, non-randomised studies (e.g., controlled before and after studies, cross-sectional surveys, longitudinal studies or cohort studies) will be considered in this part of the review. Randomised and non-randomised studies will be analysed separately. We will not include single-case designs, pre-post studies without a control group, non-matched control groups, or groups matched post-hoc after results were known.

For the qualitative synthesis, eligible studies will include: ethnographic research; phenomenology; grounded theory; participatory action research; case studies; or mixed methods studies if qualitative methods have been used to gather data. Methods used to collect the qualitative data in primary studies will include: interviews; focus groups; observation; open-ended survey questions; and documentary analysis.

Criteria for Inclusion and Exclusion of Studies in the Review

For the quantitative element of this review, where a control group exists, support services may take two forms: (1) traditional ‘services as usual’ (e.g., predetermined group activities, provided in a congregated setting and financed through block funding to service providers whereby previous annual spend for a service provider is used to estimate the required funding for the upcoming year (National Disability Authority, 2011); or (2) a different type of personalised support which does not include a personal budget where, for example, a service user might access services through a congregated setting where finances are centralised, but where an individualised plan is used to determine service user needs and preferred activities. However, the individualisation of planned responses may be limited, for example, by majority preferences within the group, staffing limitations or pre-existing service options.

We will exclude personal budgeting interventions that are provided to families, guardians/ other carers, or where the person with a disability does not have an active role in the decision making and planning process and cannot exercise control over the use of funds. However, studies may be included where an advocate is managing the funds after an individual assessment of need takes place and provided that the funds are being used to meet the needs identified during the assessment.

A personal budget which is provided by the persons' family or another private means will not be included, as this review is interested in the use of public funds for people with a disability. Furthermore, private sources of funding introduce confounding factors which would lead to uncontrollable bias.

Search Strategy for Finding Eligible Studies

The Campbell Collaboration policy brief for searching studies and information retrieval, informed the search strategy as presented below (Hammerstrøm, Wade, Hanz, & Klint Jørgensen, 2009). In addition, an information retrieval specialist within Maynooth University was consulted during the preparation of search strings. Padraic Fleming, the lead author, will conduct the searches once the protocol has been peer-reviewed and approved by Campbell Collaboration. It is expected that searches will be conducted in the first quarter of 2016; exact dates will be reported. Studies in any language and from any country will be reviewed for inclusion, provided the abstract is in English.

As recommended by Higgins and Green (2011), the search strategy will be reported in an appendix of the systematic review. This will be reported separately for each database utilised and will be copied and pasted exactly as it was performed. This will ensure that all searches are reproducible. Examples of copied and pasted search strings are provided in Appendix A.

Data Extraction and Study Coding Procedures

Titles will be reviewed initially in Endnote by the lead author to remove any studies which are clearly irrelevant (e.g. non-human or pharmaceutical studies). Excluded studies will be reported (see sample flowchart in Appendix B). Following this, the screening of studies in relation to inclusion/exclusion will be undertaken in two stages. The first stage will involve citation and abstract; the second will involve full text documents. Two independent researchers will be involved at each stage. Both (PF & MH) are co-authors of this protocol and have a deep understanding of the research questions and outcomes of interest. However, prior to data extraction and coding, the two independent reviewers will meet to discuss and pilot the extraction and coding procedures on a sample of abstracts. A third party (SMcG) will resolve any disagreements between the two independent researchers, where a resolution cannot be agreed through discussion and consensus. Inter-rater reliability will be calculated on a sub-sample of papers using kappa statistic, as recommended and will be based on making simple inclusion/exclusion decisions. Values of kappa between 0.40 and 0.59 reflect a fair level of agreement between reviewers, whilst values from 0.60 to 0.74 reflect good agreement; 0.75 indicates excellent agreement (Higgins & Green, 2011; Chapter 7.2.6). The inter-rater reliability score will be recorded and if there is a considerable level of disagreement, the reviewers will revisit the eligibility criteria and coding schemes together to ensure consistency in interpretation.

To pre-empt such disagreements, both reviewers will discuss the inclusion/exclusion criteria and the various tools being used to assess study quality and risk of bias. Any potential differences in interpretation will be discussed and resolved insofar as possible. A number of known studies will be used to pilot the data extraction and coding procedures in order to support this process.

Risk of Bias

Risk of bias will be evaluated using a range of tools (depending on study design) by two independent reviewers (PF and MH). The main areas of bias include: selection bias; performance bias; detection bias; attrition bias; and reporting bias (Higgins & Green, 2011; Chapter 8). ‘The Cochrane Collaboration's tool for assessing risk of bias’ (Table 2 in Appendix C) will be used to appraise randomised, quasi-randomised and cluster-randomised controlled trials. All non-randomised study designs will be appraised for quality and risk of bias using the appropriate tool from the Critical Appraisal Skills Programme (CASP). CASP consists of a set of eight critical appraisal tools designed to read and check health research for trustworthiness, results and relevance, specifically for systematic reviews, randomised controlled trials, cohort studies, case control studies, economic evaluations, diagnostic studies, qualitative studies and clinical prediction rule. (Critical Appraisal Skills Programme (CASP), 2014)

The overall quality of the evidence will be reported using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) for quantitative data and Confidence in the Evidence from Reviews of Qualitative research (CerQual) (Lewin et al., 2015). GRADE takes into consideration: within-study risk of bias (methodological quality); directness or relevance of evidence; heterogeneity; precision of effect estimates; and risk of publication bias (Higgins & Green, 2011; Chapter 12.2.1). A ‘high’ GRADE score indicates further research is unlikely to change reviewer's confidence in the estimate of effect, while a ‘low’ score indicates further research is very likely to impact reviewer's confidence and the estimate of effect. CerQual scores are based on methodological limitations, relevance, adequacy of data and coherence and are also scored from high to very low (Lewin et al., 2015).

The risk of bias score and GRADE / CerQual score will be reported in Section I of Table 1 along with an explanation to support the score based on evidence from primary studies (Appendix C).

Synthesis Procedures and Statistical Analysis

The main study findings, limitations, risk of bias and evidence quality score will be reported in Table 1 (Appendix C). A meta-analysis of data will be conducted using RevMan for all quantitative studies included in the review, Stata will be used for data that require analysis prior to the meta-analysis (Review Manager (RevMan), 2012; StataCorp, 2013). When inputting data, numerical accuracy, including magnitude and direction of effect will be checked. Initially, summary statistics will be presented for all studies and the appropriate effect estimates reported. Randomised and non-randomised studies will be analysed separately. It is expected, however, that there will be a range of intervention models and outcome measurements utilised, as outlined previously. Study results will therefore be organised in a hierarchy as recommended by the Cochrane Collaboration (Higgins & Green, 2011; Chapter 4.8). As highlighted by Webber et al (2014) there is no ‘one size fits all’ approach to individualised funding, hence the various mechanisms used to deliver services. Therefore, provided that sufficient studies have been identified, studies will first be pooled and analysed within subgroups such as: brokerage; direct-payment; cash for counselling. However, if a sufficient number of studies has not been identified to allow reliable estimation of random effects mean and variance, then all studies will be analysed together regardless of the mechanism of delivery, calculating the mean effect for each outcome such as: Quality of Life, Client Satisfaction, Adverse Psychological outcome etc. (Higgins & Green, 2011; Figure 4.8a) Intervention type will then be used as a moderator in the subgroup analysis.

Treatment of Qualitative Research

Drawing conclusions

Chandler et al (2012) recommends that when formulating conclusions, only studies that are included in the meta-analysis should be referenced. Therefore, no single study will be heavily relied upon, but rather, every effort will be made to report the synthesised findings in a balanced way. It is anticipated that the findings of this review will provide a clear indication of the effectiveness of personal budgets in improving health and social care outcomes and in so doing, inform future policy and practice decisions. Furthermore, the subgroup analysis should shed some light on what type of model works best for different types of disability or other socio-demographic groupings. Where appropriate, gaps in knowledge will be identified and areas for future research highlighted.