Abstract
1 Background
Globally, 43 million people are forced migrants because of violence and persecution, and 10.5 million of them are classified as refugees (UNHCR, 2011). Resettlement involves moving refugees to a third country; often, refugees have particular needs, such as physical safety, that cannot be met in the country where they initially sought protection (UNHCR, 2012). Despite the large number of refugees resettled and the continuation of resettlement programs, testimonial and correlational evidence suggests outcomes are variable. This systematic review will identify and evaluate available evidence as to whether interventions designed to improve refugees' economic self-sufficiency and well-being are meeting those goals.
1.1 DESCRIPTION OF THE CONDITION
In each of the three largest resettlement countries by current volume – the United States (US), Canada, and Australia – successful economic adjustment has been a central goal of the refugee resettlement policy (Waxman, 2001). Furthermore, refugees often define economic outcomes such as employment as important to their own lives (Valtonen, 1998).
Despite this emphasis, resettlement programs may have mixed outcomes. Australia admitted over a half million humanitarian entrants from 1939 to 2001, but survey evidence suggests recent refugees from Bosnia, Afghanistan, and Iraq have high unemployment rates relative to the general population (Waxman, 2001). The same study also found prior financial status, employment, and qualifications show no statistical correlation with employment outcomes (Waxman, 2001). Similarly, the United States has resettled over three million refugees since 1975 (US Department of State, 2o11; US Senate, Committee on Foreign Relations, 2010). Yet, their resettlement system is often described as ‘failing to meet the basic needs of the refugee populations they are currently asked to assist’ (US Senate, Committee on Foreign Relations, 2010). Economic hardship is further negatively correlated with refugee well-being. For Sudanese in Canada, economic hardship has been associated with being two to four times more likely to experience mental distress (Simich, Hamilton, & Baya, 2006), and for Cambodians in the United States, unemployment has been correlated with PTSD and major depression (Marshall et al, 2005).
The condition of resettled refugees is dynamic, like the refugee programs and the populations they serve. Contemporary programs are a result of more recent contexts, so this review will only look at studies undertaken or published since 1980, which can be characterised as something of a ‘watershed’ year.
The legislation and structure of refugee resettlement in the United States, Australia, Canada, and Europe changed dramatically from the late 1970s. The United States, by far the largest resettlement country, passed the 1980 Refugee Act, which serves as the basis for the mandate and structure of its current program. Australia's contemporary approach to refugees emerged in December 1979, when the Community Refugee Resettlement scheme came into force. This scheme included housing, social, and employment support (Refugee Council of Australia, 2012). Canada's current refugee program is based on the 1976 Immigration Act and further shaped by the 1982 entrenchment of the Canadian Charter of Rights and Freedoms in their Constitution, the 1997 Refugee Resettlement Model, and the 2002 Immigration and Refugee and Protection Act's incorporation of the 1951 Convention and 1967 Protocol into Canadian law (Casasola, 2001; Citizenship and Immigration Canada, 2011; Gauthier, 2010). European resettlement likewise changed dramatically after the 1970s, partly as a result of legal changes such as Germany's 1992 revision of their constitutional definition of ‘refugee’ (Hailbronner, 1994).
1.2 DESCRIPTION OF THE INTERVENTION
This review will examine any intervention designed broadly to increase the economic self-sufficiency and well-being of resettled refugees compared to a control or comparison group receiving ‘services as usual’ or an alternative intervention. The broad scope of interventions in this review reflects the paucity of previous reviews on interventions' effectiveness and reflects the diversity of approaches to promoting economic self-sufficiency and well-being. These interventions typically last from three months to two years and may include services such as employment casework to discuss goals and expectations; mediation between employers and employees; translation and paperwork assistance; employment mentorship; and interview training. These interventions may be delivered by the resettled state, for-profit organisations, or non-profit organisations including secular and faith-based organisations.
The contexts of the interventions also vary. The number of resettlement countries has changed over the years, but in 2008, refugees left for 28 different resettlement countries. The United Nations High Commissioner for Refugees (commonly called UNHCR: A Refugee Agency) reports that the following states currently have resettlement programs: Argentina, Australia, Brazil, Bulgaria, Canada, Chile, the Czech Republic, Denmark, Finland, France, Hungary, Iceland, Ireland, Japan, the Netherlands, New Zealand, Norway, Paraguay, Portugal, Romania, Spain, Sweden, the United Kingdom, Uruguay, and the United States (UNHCR, 2012). In any one country, there may be multiple providers and multiple interventions offered to improve the economic self-sufficiency and well-being of refugees.
1.3 HOW THE INTERVENTION MIGHT WORK
The hypothesised pathways for improving conditions for resettled refugees are not always specified in the literature. Broadly speaking, an intervention might target individual refugees, their surroundings, or the link between the two. For example, interventions may aim to work through targeting individuals through casework and employment skill development or through targeting employment demands and matching refugees with available employment. Interventions may also work by attempting to change community attitudes towards refugees or through complex interventions targeting multiple goals at both the level of the individual and the community.
Social cognitive career theory (SCCT) is one framework to explain how refugee economic interventions may work. SCCT focuses on the interactions between environmental and personal factors as the important linkages for careers (Yakushko, Backhaus, Watson, Ngaruiya, & Gonzalez, 2008). This idea implies that career counselling can improve outcomes for refugees by focusing on items such as ‘…knowledge about career options, process of obtaining and maintaining a job, cultural norms with regard to work, developing work-related cultural competencies, balancing work and education, and balancing work and family life’ (Yakushko et al., 2008). Rather than narrowing on a paradigm of causal change, this review aims to understand the impacts of interventions
1.4 WHY IT IS IMPORTANT TO DO THIS REVIEW
Despite the importance of and attention to refugee economic self-sufficiency, a knowledge gap exists about outcomes from interventions designed to improve refugee economic self-sufficiency and well-being. For example, interventions offered through the US government include the Office of Refugee Resettlement's four approaches to improving employment outcomes, but the government itself reports that ‘little is known about which approaches are most effective in improving the economic status of refugees’ (United States Government Accountability Office, 2011, p. 20). To our knowledge, this is the first systematic review to compile evidence about outcomes from interventions designed to improve refugee economic self-sufficiency and well-being.
2 Objectives
This review systematically collects and synthesizes evidence from prospective, controlled evaluations of interventions designed to improve economic outcomes for refugees. The review aims to answer the following questions:
Do interventions designed to improve the economic self-sufficiency and well-being for refugees affect participants' labour force participation, employment, use of cash assistance, income, job retention, and quality of life? Do effects differ depending on program content, program provider, populations served, or the setting?
3 Methodology
3.1 CRITERIA FOR INCLUSION AND EXCLUSION OF STUDIES
3.1.1 Types of studies
Studies must use a prospective, controlled methodology: a randomised controlled trial (RCT) design; a quasi-randomised controlled trial design (QRCT, i.e. participants are allocated by means which may not be expected to influence outcomes, such as alternating allocation, birth date, the date of the week or month, case number or alphabetical order); or a nonrandomised controlled design (i.e. quasi-experimental design). Nonrandomised controlled studies must provide information on baseline comparability of the cohorts and use statistical tools to adjust for baseline differences. For all studies, participants must have been prospectively assigned to study groups or a control group (i.e. alternative intervention or ‘services as usual‘), and studies must measure control group outcomes concurrently with intervention group outcomes.
Additionally, studies must have been conducted or published since 1980 (see Section 1.1).
3.1.2 Types of participants
The review includes refugees who meet the domestic legislative definition of a refugee for the country of the intervention and:
are formally assisted to resettle by the government (i.e. are resettled refugees but not asylum-seekers); have been served by a refugee resettlement entity; and, fall between the ages of 18 and 64 at the time of intervention.
If, for any reason, the sample in the study does not fall completely within those parameters, we will contact the author in order to obtain disaggregated data for the population that meets the criteria of a, b, and c. If we are unable to obtain disaggregated data, we plan to use sensitivity analyses based on studies with mixed populations.
3.1.3 Types of interventions
Eligible interventions will include any designed to broadly increase the economic self-sufficiency and well-being of resettled refugees compared to a control or comparison group receiving ‘services as usual’ or an alternative intervention. Interventions typically last from three months to two years and may include services such as employment casework to discuss goals and expectations, mediation between employers/employees, translation and paperwork assistance, employment mentorship, and interview training.
3.1.4 Types of outcome measures
Studies must report at least one primary or secondary outcome.
3.1.4.1 Primary outcomes
The primary outcome is economic activity as measured by employment rate or labour force participation rate.
The unemployment or employment rate measures the number of people without or with jobs compared to those searching for jobs in a population of interest (e.g. adult refugees) whereas the labour force participation rate measures the proportion of the population of interest (i.e. including those not actively looking for a job) participating in the labour force.
3.1.4.2 Secondary outcomes
Secondary outcomes include, for resettled refugees,
Level of cash assistance, as measured by the percentage or portion of the population accessing specialised refugee cash assistance or public cash assistance, Income, as measured by overall annual income, salary rate, or average hourly wage. Job retention. Quality of life, as measured by ‘generic indicators’ including scales such as the Euroqol, or the short-form SF-36 or SF-12 that measure individual functioning, family functioning, social support, or general health related quality of life (Jenkison & McGee, 1998). Measures must both make sense across different intervention evaluations and must not be tied to specific clinical mental health diagnoses which are not the focus of this review. Disease-specific measures, patient-generated measures such as the Patient-Generated Index of Quality of Life, and DSM mental health diagnoses are outside of the purview of this review.
Data on secondary outcomes will likely be self-reported by the study participants and/or collected via records from governmental agencies or non-governmental organizations.
3.2 SEARCH METHODS FOR IDENTIFICATION OF STUDIES
We will use bibliographic databases, targeted websites, and personal communication to find relevant studies. No language restrictions will be applied to potential studies identified through searches.
3.2.1 Search terms
The following will be the key search terms to cover the population, intervention and methods:
Population: resettle* OR re-settle* OR refuge* OR force* ADJ *migrant* OR asylum* OR humanitar* ADJ entrant* OR humanitar* ADJ settle*
Intervention: AND economic OR job* OR employ* OR mone* OR work* OR well-being OR wellbeing OR well ADJ being OR quality NEAR life
Methods: AND outcome* OR evaluat* OR effect* OR efficacy OR compar* OR experiment* OR trial OR control* OR random* OR study OR studies OR assessment OR impact* OR research*
These terms are in line with other meta-analyses on refugees and suggested terminology for limiting methodology to quantitative methodology (Bollini, Pampallona, Wanner, & Kupelnick, 2009; Bronstein & Montgomery, 2011; Clark & Mytton, 2007; Fazel, Wheeler, & Danesh, 2005; Gagnon & Tuck, 2004; Lipsey & Wilson, 2001; Littell, Corcoran, & Pillai, 2008; Morton, 2011; Robjant, Hassan, & Katona, 2009).
An example of the search strategy for Applied Social Sciences Index and Abstracts (ASSIA) is listed below. The strategy may be modified for the different databases. We will report full details of the modifications in the completed review.
all(resettle* OR re-settle* OR refuge* OR force* ADJ *migrant* OR asylum* OR humanit* ADJ entrant* OR humanit* ADJ settle*) AND all(economic OR job* OR employ* OR mone* OR work* OR wellbeing OR well-being OR well ADJ being OR quality NEAR/4 life) AND all(outcome* OR evaluat* OR effect* OR efficacy OR compar* OR experiment* OR trial OR control* OR random* OR study OR studies OR assessment OR impact* OR research*)
3.2.2 Databases
We will search the following databases and citation indices (Greenhalgh, 2010; Hammerstrøm, Wade, & Jørgensen, 2010):
Applied Social Sciences Index and Abstracts (ASSIA) Business Source Elite Cochrane Central Register of Controlled Trials (CENTRAL) CINAHL Dissertation Abstracts International Education Resources Information Center (ERIC) EconLit IDEAS International Bibliography of the Social Sciences (IBSS) ISI Index to the Social Sciences and Humanities OpenGrey PAIS International PolicyFile PsycINFO Social Care Online Social Science Citation Index SocIndex SocioFile Sociological Abstracts
3.2.3 Searching other resources
In order to capture unpublished reports, the above list includes some databases known for their grey literature: ISI Index to the Social Sciences, OpenGrey, PAIS International, PolicyFile, and Dissertation Abstracts International.
Additionally, we will manually search the Journal of Refugee Studies since its inception (1988).
We will also search the following websites of relevant research, policy, and governmental organisations:
Center for Migration and Refugee Studies, American University in Cairo: www.aucegypt.edu/gapp/cmrs
Centre for Refugee Research, University of New South Wales: www.crr.unsw.edu.au/
Centre for Refugee Studies, York University: http://crs.yorku.ca/
Centre for Research on Migration, Refugees and Belonging, University of East London: www.uel.ac.uk/cmrb
Department for Work and Pensions Social Research Branch: www.dwp.gov.uk/asd/asd5/index.html
Forced Migration Online Digital Library: www.forcedmigration.org/digital-library
Manpower Demonstration Research Corporation: www.mdrc.org
Mathematica Policy Research: www.mathematica-mpr.com
National Centre for Social Research: www.natcen.ac.uk/natcen/pages/op_employment.html
National Institute for Social Work: www.nisw.org.uk/about.html
Norwegian Refugee Council, Evaluations: www.nrc.no/?aid=9160729
Office of Refugee Resettlement, U.S. Department of Health and Human Services, Administration for Children and Families: www.acf.hhs.gov/programs/orr/
Refugee Services, New Zealand: www.refugeeservices.org.nz
Refugee Studies Centre, Oxford University: www.rsc.ox.ac.uk
Urban Institute: www.urban.org
UNHCR: The UN Refugee Agency, Policy Development and Evaluation: http://www.unhcr.org/pages/4a1d28526.html
Furthermore, we will use personal contacts and contact all authors of included studies to try to locate studies that are ongoing or unpublished. The bibliographies of included studies will also be inspected for potentially relevant studies.
3.3 DATA COLLECT ION AND ANALYSIS
3.3.1 Selection of studies
The studies will be screened and selected through a three-part process. Through each stage, the number of papers excluded and retrieved will be noted in a PRISMA chart.
First, all titles and available abstracts obtained from the search will be reviewed separately by both review authors. If there is any doubt about the relevance of the title or differences in opinion between the two review authors, the full paper will be obtained and reviewed. Second, all papers retrieved will be screened by two review authors against the inclusion criteria using the screening guide (in the Appendices, Section 8.1).
Thirdly, if there is any dispute between the two review authors, an arbiter from the Centre for Evidence-Based Intervention at Oxford University will be consulted to help decide if that paper meets the inclusion criteria. Relevant primary investigators will be contacted if necessary to establish eligibility. The review authors will complete the data collection sheet for included studies (in the Appendices, Section 8.2). Evidence of effectiveness will be examined only for studies meeting the screening criteria.
3.3.2 Data extraction and management
All data will be independently extracted and coded by the review authors in the data extraction sheet. The initial data extraction sheet includes information on the context, study design, study sample, and outcomes. A separate risk of bias sheet will be used to assess and record the quality of included studies. The data extraction sheet will be piloted with several studies and revised as necessary through discussion between reviewer authors. Relevant primary investigators will be contacted as necessary for missing or unclear information. Disagreement on extraction and coding will be resolved through consulting an independent reviewer at the Centre for Evidence-Based Intervention at the University of Oxford or the Campbell Methods Group. Data will be stored online, and appropriate analyses will be performed using RevMan5, SPSS, or STATA.
3.3.3 Assessment of risk of bias in included studies
As previously stated, the internal validity of the studies will be ensured by only including studies with a design of a prospective randomised controlled trial, quasi-randomised controlled trial, or nonrandomised controlled trial which adjusts for baseline differences.
For included studies, additional risk of bias will be assessed using both i) categories outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011: Section 8.6) and ii) predetermined criteria previously published in two systematic reviews and modified to adapt to the parameters of this study as shown in appendix 8.3 (Morton, 2011; Zief, Lauver, & Maynard, 2006). Review authors will independently complete both of the risk of bias sheets. Relevant primary investigators will be contacted if necessary in regards to missing information. If there is any dispute between the two review authors, an arbiter from the Centre for Evidence-Based Intervention at Oxford University or the Campbell Methods Group will be consulted.
i) Using the data extraction form, review authors will independently assess each study for risk of bias on the following criteria as outlined by the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011: Section 8.6):
Sequence generation (was the allocation sequence adequately generated?) Allocation concealment (was allocation adequately concealed?) Blinding of participants, personnel and outcome assessors (was knowledge of the allocated intervention adequately prevented during the study?) Incomplete outcome data (were incomplete outcome data adequately addressed?) Selective outcome reporting (were reports of the study free of suggestion of selective outcome reporting?) Other sources of bias (was the study apparently free of other problems that could put it at a high risk of bias?)
One example of ‘other sources of bias’ may be including participants who have received employment services from more than one agency. This could lead to bias, as participants in this subgroup may be more likely to demonstrate an effect after having received a higher dosage of employment services.
For each domain, review authors will independently assign each included study to one of the following categories:
High risk of bias Low risk of bias Unclear or unknown bias
ii) Additionally, we will carry out a narrative assessment of bias based on the second risk of bias table in Appendix 8.3. This table is adapted to this review to accommodate both randomised controlled trials and high-quality quasi-experimental designs. In particular, these criteria address how a study controls for baseline differences, reassignment, and attrition. Studies will be considered to be of higher quality if they meet What Works Clearinghouse standards for overall and differential attrition (Mathematica Policy Research, Inc., 2011: 13–14), use statistical tools to adjust for all baseline differences that are statistically significant before the intervention, and use original assignment of intervention and other groups as the basis for analyses. These standards have been used in recent systematic reviews (Del Grosso, Kleinman, Esposito, Martin, & Paulsell, 2011; Mathematica Policy Research & Child Trends, 2012). This assessment of the risk of bias is not intended to be a ranked or quantitative exercise. Rather, the exercise facilitates discussion about the strengths and weaknesses of the research designs and their implementation for included studies.
3.3.4 Measures of treatment effect
Binary data
For dichotomous outcomes, such as employed or unemployed, we will report relative risks (i.e. risk ratios) (Higgins & Green, 2011: section 9.2.2).
Continuous data
In line with the systematic review for work programmes for welfare recipients (Smedslund et al., 2004), for outcomes that are continuous variables, such as salary, and reported on the same scale of measurement, we plan to use the mean difference (i.e. weighted mean difference). For outcomes reported on different scales, we plan to use Hedges' g to report standardized mean differences (SMDs) (Higgins & Green, 2011: section 9.2.3). We will report the 95% confidence intervals for mean differences and standardized mean differences.
Synthesising binary and continuous data
If measures arise that may report results across studies using both binary and continuous data (e.g. salary), both review authors will assess and discuss whether it is logical and appropriate in the context of the study and wider field to convert the continuous data into dichotomous data. The cut-off point for the dichotomous data must be meaningful and reasonable. We will consult experts from the Campbell Methods Group and the Centre for Evidence-Based Intervention as necessary, and report synthesised data as appropriate.
3.3.5 Unit of analysis issues
Cluster-randomised trials
We anticipate that allocation to the intervention group will be on the individual level. In the event of clustering, for example on the community level, we anticipate that investigators will have controlled for a clustering effect in their results. We will contact authors for further information if this is unclear. If the clustering effect was not controlled for, we will request individual participant data to calculate an estimate of the intra-cluster correlation coefficient (ICC), and, if that is not available, we will obtain external estimates of the ICC from similar studies. We will then enter these data into RevMan to analyse effect sizes and confidence intervals using the generic inverse variance method (Higgins & Green, 2011: 16.3.3).
Repeated observations on participants
One potential complication of a systematic review of studies in this area is that most studies will report multiple outcomes, and some may report the same outcome (e.g. salary) at multiple time points. The statistical methods outlined require that findings (e.g. standardized mean differences) come from unique samples.
In order to address this problem, all findings meeting the criteria of this review will be coded, but, for meta-analysis, we plan to use the data from the longest follow-up that is based on the full sample (i.e. not affected by attrition) (Higgins & Green, 2011: Section 9.3.4). We will use the attrition guideline standards set by What Works Clearinghouse, accounting for different levels of overall and differential attrition as well as the primary investigator's judgment about whether the source of attrition is at random or endogenous (Mathematica Policy Research, Inc., 2011: 13–14; Mathematica Policy Research & Child Trends, 2012).
3.3.6 Dealing with missing data
Where a study has missing data, we will contact the primary author to request additional information. For trials reporting outcomes only for participants completing the trial, the primary author will be asked to provide additional information to permit intention-to-treat analyses. Studies in which participants are analysed as members of the groups to which they were originally assigned (intention-to-treat analysis), studies that include only those participants who were willing or able to provide data (available-case analysis), and studies that analyse participants who adhered to the study's design (per-protocol analysis) will be analysed separately.
Where obtaining missing data is not possible or investigators are unresponsive, we will make assumptions regarding whether the data are ‘missing at random’ or ‘not missing at random’ and will follow the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011: Section 16.1.2). Data that are not missing at random are likely to be missing for reasons related to the outcomes of the missing data. For example, if a participant agrees to take part in an employment services intervention, but is unhappy with not finding a job or the job found, the participant may be unwilling to complete any follow-up interviews or questionnaires on his or her experience. In such a situation, where dichotomous data are missing, we will impute data with the assumption that the participants experienced the less favourable outcome. We will also explore the possibility that those missing experienced the positive outcome (found work) and impute data based on this assumption. We will conduct sensitivity analyses to examine the impact on the results of changes in the assumptions made about missing data.
Where studies have missing summary data, such as missing standard deviations, we will derive these where possible, using calculations provided in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2011: Section 16.1.3).
We will specify the methods used to address any missing data in the results tables. If imputation was not possible, we will outline the reasons for this in the text.
3.3.7 Assessment of heterogeneity
Heterogeneity by study demographics, setting, program characteristics, and study quality will be noted in a narrative format and in a table.
If meta-analysis is used, statistical heterogeneity will be reported both using a Q statistic and its p value, the I2 statistic, and by visual inspection of forest plots. Experts on this issue will be consulted as necessary.
3.3.8 Assessment of reporting biases
Reporting bias may be present both as a result of publication bias and because of selective reporting. Extensive searches will be conducted to attempt to identify unpublished or ‘grey’ literature, and we will include both published and unpublished studies. We will use funnel plots for information about possible publication bias if we find sufficient studies (e.g. at least ten studies, Higgins & Green, 2011: 10.4.1). However asymmetric funnel plots are not necessarily caused by publication bias (and publication bias does not necessarily cause asymmetry in a funnel plot). If asymmetry is present, we will consider possible reasons for this.
We will also attempt to identify any pre-published study protocols to check that all pre-specified items appear in the final reports. Additionally, review authors will attempt to contact researchers with regard to missing data. All missing data and concerns about reporting biases will be reported.
3.3.9 Subgroup analysis
For our data extraction, we will record subgroup results as reported by:
gender of the participants (Beiser & Hou, 2003; Siraj, 2007); language proficiency level for the national language of the resettlement country at baseline (Mamgain & Collins 2003; Potocky-Tripodi, 2004); and, ethnic group of the participants (e.g. this may be split by countries of origin; Siraj, 2007).
If sufficient data exists and comparison conditions are similar, subgroup analyses in RevMan will examine potential differential effects of interventions, dividing studies using meta-regression where appropriate (Higgins & Green, 2011: Section 9.6) according to:
program content grouped if there are clear delineations in approach, intensity, and content (e.g. mediation between employers/employees, employment mentorship, and language training) program provider as defined as government provider, coethnic community provider, other non-profit provider, or a private provider; population served by: gender of the participants, language proficiency level for the national language of the resettlement country, and, ethnic group of the participants the setting by country (e.g. USA, Canada, and Australia)
In the event of sufficient studies and data for subgroup analyses, these analyses will be accompanied by a discussion of their potential pitfalls. No conclusions from subgroup analyses will be drawn and interpretation of relationships will be cautious, as they are based on subdivision of studies and indirect comparisons.
3.3.10 Sensitivity analyses
We will perform sensitivity analysis to assess whether the findings of this review are robust with respect to the decisions made in the process of obtaining them. For example, sensitivity analyses may be performed excluding studies according to study quality issues (including those with low sample size and high risk of bias) and by separating studies by design (i.e. randomised controlled trials from quasi-experimental designs). For methodological quality, we will consider sensitivity analysis for each major component of the risk of bias checklists. Sensitivity analysis will further be used to examine the robustness of conclusions in relation to the quality of data (outcome measures based on different time intervals).
3.3.11 Data synthesis
Meta-analysis will be used if appropriate. If sufficient studies are identified, we plan to analyse the effects of differing types of interventions, time points, and/or comparison conditions separately. Meta-analyses will be conducted for each outcome construct separately, combining dichotomous and continuous outcomes as appropriate as discussed in ‘Section 3.3.4 Measures of Treatment Effect’. Some outcome constructs may have insufficient studies for meta-analysis to be appropriate. The standard for study design is fairly strict to be included in this review, so we will not separate studies by study design, but study design will be explored through sensitivity analyses.
3.3.12 Statistical procedures and conventions
A random effects model will be assumed given the expected heterogeneity across studies. If there are sufficient studies, analyses will be performed using RevMan5, SPSS, or STATA.
3.3.13 Narrative synthesis
We aim to use a narrative synthesis of the results including a table and accompanying description to present information such as strength of the study design, risk of biases, population differences, context of the intervention, and context of the results. Depending on the quantity of papers that meets the inclusion criteria, we may follow the three step suggestion of Petticrew and Roberts (2005): ‘(i) organizing the description of the studies into logical categories; (ii) analyzing the findings within each of the categories; and (iii) synthesizing the findings across all included studies.‘ If few or no studies meet the inclusion criteria, we plan to discuss excluded studies.
3.3.14 Treatment of qualitative research
Qualitative data from included studies will be useful in contextualizing the results and determining the risk of bias, and will be reported in the review.
4 Timeframe and updating
The authors will aim to complete the searches for published and unpublished studies within two months of approval of the protocol. The authors aim to complete the review within a year and to update the review within three years.
5 Sources of support
5.1 INTERNAL FUNDING
The lead review author is funded by the Rhodes Trust and the Truman Foundation as part of her DPhil in the Centre for Evidence-Based Intervention at the University of Oxford. The second review author is employed by the University of Oxford.
5.2 EXTERNAL FUNDING
There is currently no external funding.
6 Declarations of interest
The authors are not aware of any conflicts of interest.
Footnotes
8 Appendices
1
Categories specified based on experience and literature. May be modified based on obvious splits for included studies. For use only if sufficient studies for meta-analysis.