Abstract
Linked article:
BACKGROUND
The Problem, Condition or Issue
Diarrhoeal diseases are the second highest cause of death in low-income countries and the fifth highest cause of death in the world (WHO, 2011). In an update of the Global Burden of Disease study, it was shown that unsafe water, sanitation and handwashing caused nearly 5 per cent of DALYs (Disability-Adjusted Life Years) for males and females in poor communities (GBD Risk Factor Collaborators, 2015).
Water, Sanitation and Hygiene (WASH) interventions consist of (1) water supply (water quantity) and water treatment (water quality), including operation and maintenance of the water source (“Water”), (2) latrine construction, latrine use, latrine hygiene, faeces disposal practices, discouraging the practice of open defecation, disposal of solid waste and wastewater, and vector control (“Sanitation”), and (3) promotional activities around personal hygiene (e.g. handwashing, facial washing, showering/bathing practices, menstrual hygiene) and domestic hygiene (“Hygiene”) (DFID, 2013). The actual construction of WASH interventions, such as construction of a water source or latrine, is called the “hardware” element of the intervention. On other hand, implementation of participatory approaches to promote safe hygiene practices, establish community-based management systems for the WASH facilities, create up-front demand and encourage community participation and ownership is called the “software” element of the intervention (Peal, 2010). The latter is particularly important to ensure long term sustainability of behaviours and technical durability of facilities since it was shown that the impact of WASH interventions on the burden of disease falls over time (Cairncross, 2010; Waddington, 2009).
One of the targets of the Millennium Development Goals was to halve the amount of people without sustainable access to safe water and sanitation by 2015. In 2012 it was published that the target for water supply has been met, however, 780 million people still do not have access to safe water, with rural populations having five times less access than urban populations. The target for sanitation has not been met at all, and it is estimated that 2.5 billion people have no access to improved sanitation, with 30 per cent of access in Sub-Saharan Africa and 41 per cent of access in South Asia. Moreover, 1.1 billion people still practice open defecation (WHO/UNICEF, 2010; DFID, 2013).
The Intervention
In order to improve effectiveness of WASH interventions, more and more attention is currently going to the design of programmes and the selection of approaches to promote WASH behaviour change. Several approaches have been developed over the last two decades, and are currently being applied in practice to promote uptake of WASH interventions and to achieve WASH behaviour change, including for example marketing approaches (e.g. Public Private Partnership for Handwashing with Soap) and participatory, community-based approaches (e.g. Community Led Total Sanitation) (Peal, 2010). It is not always clear which of these approaches is the most effective in relation to sanitation and hygiene behaviour change, and other outcomes leading to behaviour change (e.g. learning outcomes) or longer term outcomes that follow from behaviour change (e.g. mortality, morbidity). In the WASH sector, evaluations of programmes tend to focus on intended outcomes and impacts (whether the intervention worked and what effect it had on outcomes) but not on appraising the process of implementation and establishing how the use of a specific approach leads to changes in outcomes. However, decision makers need to know the critical factors in process implementation that ensure that impacts are achieved and sustained, and how scaling up is best achieved.
For the purpose of this review, and because of limitations in time and resources, we will focus on approaches to promote handwashing and sanitation interventions, with behaviour change as the main outcome. Since adherence to water, sanitation and hygiene programmes is known to be highly associated with factors such as gender, socioeconomic status, education and occupation, equity factors will be taken into account in this systematic review (DFID, 2013). Since the effect of WASH interventions on health outcomes (such as diarrhoea, cholera, trachoma, helminth infections) has been shown in many existing individual studies and systematic reviews (Cairncross, 2010; Dangour, 2013; Fewtrell, 2005; Peletz, 2013; Stocks, 2014; Strunz, 2014; Taylor, 2015; Waddington, 2009), and practicing/showing the right behaviour is a pre-requisite for health impacts, health outcomes will also be looked at in those studies that measured behaviour change. Although it would be relevant to include studies that measured cost-effectiveness, this is outside the review scope.
The objective of this systematic review is to identify promotional elements and those factors in the implementation process that lead to behaviour change. This will guide governments and international bodies to select effective behaviour change methodologies to formulate effective handwashing and sanitation policies and programmes.
In the context of this review, we will use the following definitions:
How the Intervention Might Work
We have built a Theory of Change (ToC) framework illustrating the hypothesized causal links, explaining how (elements of) handwashing and sanitation promotional approaches are expected to lead to the intended short-term, intermediate and longer-term outcomes, and how different factors could influence the implementation of the promotional approaches (see Figure 1). The following sources of information were used to inform the ToC: In the scoping phase of this project (overview of existing systematic reviews), we identified a systematic review of WASH behavioural models (Dreibelbis, 2013). The review did not fulfil our selection criteria, but was used as a basis for the development of the ToC. The RANAS model for behaviour change, cited in this review, is one of the few models that is applicable across multiple WASH practices and interventions. RANAS stands for “Risks, Attitudes, Norms, Abilities, and Self-regulation”, which are called “behavioural factors” that determine behaviour. Norm factors represent the perceived social pressure towards a behaviour. Self-regulation factors represent a person's attempt to plan and self-monitor a behaviour. The model is based on psychosocial theories including the Health Belief Model (Rosenstock, 1974), the Protection Motivation Theory (Floyd, 2000), the Health Action Process Approach (Schwarzer, 2008), the Theory of Planned Behavior (Fishbein, 2010). The entire framework, containing behavioural factors and behavioural outcomes, was integrated in the ToC as short-term and intermediate outcomes, respectively. The contextual factors that are part of this model are included in a box with factors that can influence all steps of the ToC. In addition to the RANAS model, the IBM-WASH framework (standing for “The Integrated Behavioural Model for Water, Sanitation, and Hygiene”) is another model providing guidance in the design and evaluation of behaviour change interventions (Dreibelbis, 2013). A couple of additional contextual factors (division of labour, available space) were added to the ToC. A more recent model for behaviour change, which was applied in the development of handwashing programmes, is the Evo-Eco approach or BCD Behaviour Determination model (Aunger, 2014; Aunger, 2015). Since this model was not included in the review by Dreibelbis (2013), we initially did not use it as a source of information for our ToC. The 6 systematic reviews that were included in the scoping phase (overview of existing systematic reviews, see below) contained supportive information for certain behavioural outcomes (such as “use”), however they could currently not be used to further refine the ToC or confirm any of the links in the model, since only a limited number of outcomes were included and not enough information is available on the specific content of the intervention. The PROGRESS framework, which is developed to provide an equity lens in the conduct, reporting and use of research (O'Neill, 2014). The factors described by the PROGRESS acronym, including for example gender and disability, illuminate inequities in health and could be of importance to make subgroup analyses in the phase of data synthesis in this systematic review. These factors were added to the box with “contextual factors”, if not covered yet. The Checklist for implementation (“Ch-IMP”), which is composed of a list of process and implementation related factors, relevant in understanding aspects of intervention implementation (Cargo, 2015). This checklist served as a source of factors that plays a role before short-term outcomes can occur, and relevant factors were added to the ToC. In addition, the SURE framework, containing a checklist for identifying factors affecting the implementation of a policy option, was used to inform these factors and the contextual factors, if not covered yet (The SURE Collaboration, 2011). The draft ToC was discussed in detail and approved by our different team members and Advisory Group members, including methodological as well as content experts.
Theory of change framework concerning the effect of promotional approaches intended to improve handwashing and sanitation behavioural factors (short-term outcomes), handwashing and sanitation behaviour change (intermediate outcomes) and reduce morbidity and mortality (longer-term outcomes). Green boxes contain short-term, intermediate or longer-term outcomes. Primary outcomes are indicated in boxes with a black border. Blue boxes contain factors that can influence the implementation of the promotional approaches.
The ToC contains six different (elements of) promotional approaches aimed at inducing handwashing and sanitation behaviour change. Furthermore it contains (1) short-term outcomes, consisting of five “behavioural factors” (knowledge, skills and attitude, norms, self-regulation), (2) intermediate outcomes, consisting of the different elements that compose “behaviour change”: intention, use and habit, and (3) longer term outcomes, including health outcomes such as mortality and morbidity due to agents with faecal-oral transmission. Health outcomes will be included since these are the final intended outcomes for which behaviour change is a pre-requisite. However, data on health outcomes will only be included from studies that also report behavioural outcomes, which ensures that these outcomes are linked (and taking into account confounding factors such as other causes of morbidity or mortality). The “behaviour change” outcomes are the primary outcomes in this review, while the other outcomes are included as secondary outcomes. These outcomes will be measured in quantitative research.
In addition to the “core structure” of the ToC, three types of factors that are able to influence the implementation of the promotional approaches were added to the model: (1) programme environment factors and recipient-related moderators, (2) process evaluation factors (such as recruitment, attrition, reach, dose, fidelity, adaptation, engagement, satisfaction and acceptability), and (3) recipient-related contextual factors (including socio-cultural, physical and personal contextual factors of the recipients). These factors will be mainly identified from qualitative research. Examples of such factors are equity factors such as gender, which, at the analysis level, will be used in subgroup analyses (if possible).
The current ToC is not a static model and will be further refined during the course of reviewing the literature. We will also explore the Evo-Eco approach or BCD Behaviour Determination model as a source of information for the ToC.
Why it is Important to do the Review
In a first scoping phase an extensive overview of
To answer these research questions we only included systematic reviews that investigated the effectiveness (research question 1) or implementation aspects (research question 2) of WASH promotion programmes on behaviour change outcomes. Systematic reviews where no approach was used to promote the WASH intervention and/or did not report behavioural change outcomes (e.g. only health-related outcomes) were excluded.
Different databases (The Cochrane Library, Medline (Pubmed), Embase (Ovid), Web of Science (Science citation index-expanded, Social Sciences Citation index), ERIC (EbscoHost), Cinahl (EbscoHost) and the Campbell Library) were searched from the date of inception until October 15 2015. In addition, different websites (IRC International Water and Sanitation Center, Social Science research network (SSRN), WHO, World Bank, USAID/EHPROJECT, UNICEF and International Center for Diarrhoeal Disease Research) were searched for grey literature. From 3775 database references, and 199 references identified as grey literature, 6 systematic reviews were included for data extraction and quality appraisal, including 5 reviews related to research question 1 and 1 review related to the second research question. We used the ROBIS tool to assess the risk of bias of the included systematic reviews (Whiting, 2016).
The following WASH interventions were included in the reviews: water quality (Fiebelkorn, 2012), hygiene hand sanitizers (Mah, 2008; Ejemot-Nwadiaro, 2015) and multiple WASH interventions (water, sanitation, hygiene) (Evans, 2014; Hulland, 2015; Joshi, 2013). No systematic review focused on water supply or sanitation only.
The (multiple) WASH interventions were promoted using different approaches as follows: via social marketing principles (Mah, 2008; Evans, 2014), via community-led total sanitation (Hulland, 2015), via educational and/or communication channels (Ejemot-Nwadiaro, 2015; Hulland, 2015; Joshi, 2013) or via multiple promotional approaches (community mobilization, health education, motivational interviewing, role modeling, social marketing: Fiebelkorn, 2012). No systematic review on the use of financial incentives or other approaches to promote WASH interventions was found.
In the systematic reviews information on the promotional approach of the interventions is scarce, which prevents us from making any further conclusions. In none of the systematic reviews, meta-analyses were performed, because of heterogeneity in population, type of intervention and outcome measurement.
Only 1 systematic review reported data on implementation factors that can influence WASH behaviour (sustained adoption) (Hulland, 2015). Systematic reviews concerning other implementation factors were not identified. Evidence from the systematic review by Hulland (2015) suggests that the most influential programme factors associated with sustained adoption include frequent, personal contact with a health promoter over a period of time. While the Hulland review investigated factors that affect sustained adoption of WASH technologies (e.g. promotion via frequent, personal contact), this project will bring the focus on factors that influence the approach to promote WASH technologies (e.g. culture as a barrier to use a financial incentive).
The quality (risk of bias) of the included systematic reviews was considered as low (Ejemot-Nwadiaro, 2015; Evans, 2014; Fiebelkorn, 2012; Hulland, 2015) to high (Joshi, 2013, Mah, 2008).
Detailed information about this first scoping phase (methodology, results, conclusion) will be published this year (2016) in a separate peer-reviewed publication.
In summary, based on our scoping review, we concluded that in the context of our 2 research questions, there is still an evidence gap with for example no systematic collection of evidence regarding several promotional approaches (e.g. financial/non-financial incentives) or specific WASH interventions (e.g. sanitation) in relation to behaviour change as an outcome. In addition, there is a lack of qualitative information (in systematic reviews) about (implementation) factors that can influence approaches to promote WASH interventions. Finally, it is also worthwhile and unique to collect and analyse the evidence about the effectiveness and implementation factors of all WASH promotion programmes by a uniform, rigorous and systematic approach in order to make comprehensive evidence-based recommendations Therefore, we conclude that a systematic collection, extraction and analysis of qualitative/quantitative data on the effectiveness of promotional approaches aiming to promote handwashing and sanitation behaviour change outcomes is relevant and timely.
The included studies of this systematic review will be mapped and analysed along the ToC, to elucidate which promotional approaches will lead to behaviour change (and improved health outcomes) and to help understand what factors affect the success or failure of a particular promotional approach. This theory-based approach will help policy-makers to understand the reasons for differing levels of programme participation and the processes determining behaviour change.
Stakeholder engagement will happen throughout the project and a stakeholder specific dissemination strategy will be used to communicate the findings of the project tailored to the requirements of these stakeholders. Specific policy messages or policy implications will be formulated.
OBJECTIVES
This review will be a “Mixed methods research synthesis”, consisting of a strand of quantitative and a strand of qualitative evidence. In this way we aim not only to answer the question “what works”, but we will also inform policy makers on “why, for whom and under which circumstances” a programme will work.
The overall goal for this systematic review is to show which promotional approaches are effective to change handwashing and sanitation behaviour, and which implementing factors affect the success or failure of such an intervention.
This goal will be achieved by answering two different review questions, in a quantitative and qualitative arm of the review:
METHODOLOGY
The methodology applied to develop this systematic review is in accordance with the Campbell Collaboration Guidelines on Systematic Review Methods.
I. Mixed Methods Research Synthesis design (MMRS)
For this review a segregated concurrent type of MMRS design will be used (Heyvaert, 2016). In this type of design the quantitative and qualitative studies will be analysed separately (Figure 2).
Schematic overview of the segregated concurrent type of Mixed Methods Research Synthesis design that will be used in this review.
We will use a comprehensive search to identify relevant literature. Quantitative and qualitative study designs will be split out in the screening phase. Primary mixed method studies will be considered for inclusion when quantitative and qualitative results/findings can be separated out. Design specific critical appraisal instruments will be used to assess the quality of each study type. Quantitative evidence will be analysed using statistical pooling techniques. The qualitative evidence will be synthesized using a “Best fit framework synthesis” approach (Booth, 2015; Carroll, 2013).
The analysis of both strands of evidence will feed into an overall discussion and conclusion section.
Criteria for including studies in the review
In order to answer Impact evaluations using an experimental design (randomised controlled trials (RCTs) with assignment at individual or household/community (cluster) level, quasi-randomised controlled trials) Impact evaluations using a quasi-experimental design (non-randomized controlled studies, taking into account confounding variables at the design or analysis stage) Observational analytic studies such as cohort studies and case-control studies.
Quasi-experimental and observational analytic studies are included since these are prevalent in the WASH literature, because randomized assignment is not always feasible or ethical.
Uncontrolled studies will be excluded, as well as case series, research methodology reports/manuscripts, editorials and economic analyses.
In order to answer
The programme should contain a direct promotional approach, such as: Training/health education Psychosocial theories (this is the case in programmes such as Focus, Opportunity, Ability, Motivation (FOAM), IBM-WASH, Access Build Create Deliver Evaluate (ABCDE), Evo-Eco or BCD Behaviour Determination model, and RANAS) Community-based participatory approaches (this is the case in programmes such as Community Led Total Sanitation (CLTS), Participatory Rural Appraisal (PRA), Participatory Hygiene and Sanitation Transformation (PHAST), Self-esteem, Associative Strengths, Resourcefulness, Action-Planning, and Responsibility (SARAR), community reunion, community hygiene club/mother club, community health clubs (CHC), child-to-child approach (CtC), Urban Led Total Sanitation (ULTS), Community Approaches to Total Sanitation (CATS), Methodology for Participatory Assessments (MPA), Community Action Planning (CAP), Child Hygiene and Sanitation Training/Transformation (CHAST), and the model home approach) Marketing approaches, which combine enterprise approaches with demand stimulation, including: (1) marketing of a single intervention (e.g. Saniya, Public Private Partnership for Handwashing with Soap (PPPHWS)), (2) marketing of sanitation goods and services (e.g. Support to Small Scale Independent Providers (SSIP), SaniMart, SanMark, Total Sanitation and Sanitation Marketing (TSSM)) Incentives: (1) financial (national government subsidies programmes, community-based cross subsidies, vouchers, cash transfers, loans/micro-credits) or (2) non-financial (e.g. food) Advocacy (activities targeting policy/decision makers, for example community meetings or shifting perception of general public like events with celebrities) Other promotional elements, such as the use of pride or disgust, the use of several behaviour change techniques (e.g. a specific technique to specifically influence “norms”), etc. Any combination of the approaches mentioned above (Multichannel approach)
The programmes that are mentioned as examples in the above list (e.g. CLTS) can contain different promotional elements, depending in the context for which the programme was developed. As a consequence, the content of such a programme is not necessarily the same.
Any of the approaches above can be delivered using one or more different communication strategies: Interpersonal communication: peer to peer, home visits, focus group; either of these approaches could work with change/transformation agents such as hygiene promoters, WASH Committees, champions/natural leaders who are not part of community leadership system, community leaders (chefs, elected village/ appointed village leaders, councillors, etc.), religious leaders, teachers, village health workers, local government staff (dealing with WASH, social services, health, etc.), volunteers (e.g. Red Cross volunteers), lecture, workshops, games, material provision with demonstration, quiz Mass media communication: poster, TV, radio spot, radio program, billboards, newspapers, outdoor/transit advertising, megaphones, hygiene day, stickers, paintings Traditional communication: songs, folk drama and theatre, concerts, rallies, parades, cinema show
Programmes using no promotional approaches will be excluded.
Primary outcome: behaviour change in the following domains: (a) use of handwashing and sanitation interventions (handwashing: with or without soap (or alternatives such as ash) and/or hand disinfection with alcohol based gels at key times - before eating, before food preparation, after visiting the toilet and after children's faeces disposal or cleaning the baby's bottom; sanitation: latrine/toilet construction, latrine/toilet use, latrine/toilet maintenance, latrine/toilet hygiene, number of people practicing open defecation, safe disposal of child faeces;): uptake of the interventions, adherence to the interventions, longer-term use of the interventions, (b)
Secondary outcomes: knowledge, skills, attitude, norms, self-regulation concerning the practice of handwashing and sanitation interventions; morbidity and mortality due to agents associated with faecal-oral transmission. Studies reporting data on morbidity and mortality will only be included if also data on primary outcomes (behaviour change) are available.
We will include outcomes that are measured via direct observation, demonstration (where a participant is asked to show how a behaviour is practiced), as well as self-reported, parent-reported or teacher-reported outcomes (which could be prone to recall bias or inaccuracy). Health-related outcomes could also be measured by laboratory measurements, e.g. amount of antibodies or via PCR methods, or as Weight-for-age z-score, which is a proxy marker for diarrhoea (DFID, 2013).
In order to answer
If no raw data is provided, studies will be excluded.
Any post hoc decisions about or changes to the selection criteria will be justified documented in the review, possibly accompanied by sensitivity analyses.
II. Search strategy
Searching for studies will be done according to the principles stated by Hammerstrom (2010).
The following databases will be searched from 1980 to present: MEDLINE (PubMed interface) Embase (OVID interface) Cochrane Central Register of Controlled Trials Applied Social Sciences Index and Abstracts (ASSIA) (Proquest) Sociological Abstracts (Proquest) International Bibliography of the Social Sciences (IBSS) (Proquest) Social Sciences Citation Index (SSCI) (Web of Science) PsycINFO (EBSCOHost) WHO Global Index Medicus (http://www.globalhealthlibrary.net/php/index.php) The Education Resources Information Center (ERIC) (EBSCOHost) Global Health (CABI) 3ie Impact Evaluations database
A sensitive search strategy is developed by an information specialist (Vittoria Lutje) and tested in an iterative way for each database separately, based on existing search strategies from existing WASH systematic reviews, our ToC and our selection criteria. A combination of index terms (where relevant) and free text words (in title/abstract) is used, with attention to possible synonyms and words used in key papers. De-duplication of the references will be done by the information specialist using Reference Manager 12. All searches and search dates will be documented.
In order to find unpublished material and programme documents relevant for research questions 1 and 2, we will contact the following research groups and organizations and/or check the following websites: IRC International Water and Sanitation Centre Water, Engineering and Development Centre, UK Water and Sanitation for the Urban Poor (WSUP) iDE Global WASH Initiative WaterSHED ( CLTS Foundation SHARE (Sanitation and Hygiene Applied Research for Equity) consortium ( WHO: Department of Child and Adolescent Health and Development (WHO) Water, Sanitation and Health Program (WHO) ( World Bank: World Bank World Bank Water and Sanitation Program United Nations Children's Fund (UNICEF) Oxfam International WaterAid International Centre for Diarrhoeal Disease Research, Bangladesh (ICDDR,B) Development Media International (DMI) ( R4D (Research for Development) UK DFID Government of India website ( International Water Centre – Australia Social Science Research Network Electronic Library Susana project database (
This list of sources is based on the advice and networks of our team members and Advisory Group members.
The citation and reference lists of included references will be searched (e.g. by searching in Social Sciences Citation Index, Scopus and Google Scholar). In addition, the “Related Articles” feature of the databases, if present, will be used. Finally, citation and reference lists of known WASH systematic reviews will be searched.
Content experts (including the Advisory Group) will be consulted for missing studies.
Via EPPI reviewer software, study selection will be performed independently and in parallel by two evidence reviewers (Thashlin Govender and Hans Van Remoortel). Titles and abstracts of the references identified by the search will be scanned in a first phase, making use of the text mining possibilities of EPPI reviewer. Full texts of relevant articles will be retrieved, and references that meet the selection criteria will be included for further analysis. Any discrepancies between the two reviewers will be resolved by consensus, and in case of disagreement a third reviewer will be involved (Emmy De Buck). A PRISMA study selection flowchart will be provided (Moher, 2009) and a list of excluded studies with the reason for exclusion will be provided.
Any relevant retraction statements and errata will be examined.
III. Description of methods used in primary research
Below the methodology of some quantitative studies, included in the systematic reviews that we identified in the scoping phase of this review (overview of existing systematic reviews), is described briefly: Pickering, 2013: Cluster-RCT with six schools, comparing the effect of hand washing with soap, hand sanitizer versus no intervention on student hand hygiene behaviour. The intervention groups received a hygiene promotion kit, including posters, stickers and a classroom activity. A participatory discussion on germ theory and hygiene was held with the teachers. Allocation sequence was unclear and allocation concealment was not described. Diarrhoeal rates (via interviewing the students about diarrhoeal symptoms) and hand washing rates (via direct observation of field staff) were measured, and length of follow-up was eight weeks. A cluster-adjustment method was used. Langford, 2011: Cluster-RCT comparing a community-based hand washing programme versus no intervention in children in Nepali slums. Group meetings were organized with the intervention group, and posters and drama were used in addition. The allocation sequence was generated by flipping a coin, and allocation concealment was unclear. Morbidity (via a symptom checklist collected from the children's mothers by trained fieldworkers) and hand washing practices (via structured observations and questionnaires of fieldwork assistants) were measured and length of follow-up was 6 months. It is unclear if a cluster-adjustment method was used.
IV. Criteria for determination of independent findings
Multiple reports of the same study will be treated together.
If a study is included with more than two intervention arms, only intervention and control groups that meet the eligibility criteria will be included. Where studies report multiple relevant intervention arms (e.g. factorial designs), we will report findings from both arms and undertake relevant sub-group analyses.
Where studies report multiple effect sizes for different outcome constructs, we will analyse them separately. Where studies report multiple effect sizes for similar outcome constructs (e.g. self-reporting versus enumerator observation versus pathogen tests for adherence) we will analyse these outcomes separately. In cases where we need to identify a single effect size from multiple reported effect sizes in any given study, we will use synthetic effect sizes (averaging effect sizes before meta-analysis), or use the outcome indicator, which is similar to other studies, or the effect size that is deemed of lowest risk of bias.
V. Details of study coding categories
Data extraction (including quality assessment) will be done by two reviewers independently (Thashlin Govender and Hans Van Remoortel).
Risk of bias in the individual studies (experimental studies) will be analysed at the study level by using the Cochrane Risk of Bias tool (Higgins, 2011). For quasi-experimental studies, the risk of bias tool provided by 3ie and the Cochrane tool for non-randomised studies (ACROBAT-NRSI), will be used (available from: http://www.3ieimpact.org/media/filer_public/2012/12/26/jorge_hombrados_and_hugh_waddington_conference-session12-b_3ie_dhaka_colloquium.pdf, and: http://bmg.cochrane.org/cochrane-risk-bias-assessment-tool-non-randomized-studies-nterventions-acrobat-nrsi). The different choices made during the risk of bias assessment will be justified by providing information directly from the study. A specific question will be added to the risk of bias assessment concerning the rigour of the outcome measurement, especially for handwashing, since it is known that over-reporting often takes place when using questionnaires (Manun'Ebo, 1997; Contzen, 2015).
The GRADE approach will be used to assess the overall quality of evidence included in this review. This approach is based on the limitations in study design, imprecision, inconsistency, indirectness, and publication bias (Atkins, 2004). The online tool of the GRADE Working Group (“GDT” or “Guideline Development Tool”) will be used for the GRADE assessment process.
A level of evidence for the “body of evidence” will be assigned, ranging from high, moderate, low to very low, as part of the GRADE process (Atkins, 2004).
Data concerning the publication date, study design, study population, details of the intervention, outcome measures, and study quality will be independently extracted by the two reviewers.
For the intervention information on the targeted activity (handwashing, sanitation) will be extracted, as well as information on the promotional approach. The promotional approach should be clearly described, including: (1) who is providing the approach, (2) who is receiving the approach, (3) the exact content of the promotional approach (presence of promotional elements such as: health education, psychosocial theories, community-based participatory approach, marketing, incentives, advocacy, and other elements such as pride/disgust/behaviour change techniques), and (4) process evaluation factors (recruitment, attrition, reach, dose, fidelity, adaptation, engagement, satisfaction, acceptability). All these different elements will be extracted separately. If this information is not reported in the paper, study authors will be contacted and related programme reports will be checked. If a substantial amount (which is decided post-hoc, based on the number of included studies) of the above mentioned information is missing, no further data extraction will be carried out for these studies, since it will not be possible to make any meaningful conclusions on the effectiveness of different promotional approaches.
Outcomes measured on different time points following the intervention will be extracted separately.
For each dichotomous outcome, we will extract the number of participants experiencing the event and the number of participants in each treatment group, or we will extract the information necessary to estimate odds and risk ratios, including group means and sample sizes. For each continuous outcome that can be assumed normally distributed, we will extract means, standard deviations (or information to estimate standard deviations), and number of participants in each group. For skewed continuous data, medians, ranges, and p-values for non-parametric tests will be extracted.
Any discrepancies between the two data extractors will be resolved through discussion or consulting other review co-authors. In the event that data are lacking, authors of the study will be contacted to obtain more detailed information. If studies are using different scales the direction of interpretation will be explained and it will be clearly indicated in case directions were reversed. Data will be entered into a meta-analysis software and checked for accuracy.
Two tables will be developed: (1) characteristics of the included studies, containing a summary of the characteristics of the participants, interventions, outcomes and other relevant information; (2) summary of findings, containing a summary of the results of all the included studies.
A quality appraisal will be done at the study level by using the CASP Qualitative Checklist to reveal limitations in study design (Critical Appraisal Skills Program 2014).
The CerQual approach will be used to assess the overall confidence in the qualitative evidence synthesis. This assessment is based on the methodological limitations of the qualitative studies contributing to a review finding, the relevance to the review question of the studies contributing to a review finding, the coherence of the review finding, and the adequacy of data supporting a review finding (Lewin, 2015). The assessment will be performed by the two reviewers independently.
A level of confidence in the review findings will be assigned, ranging from high, moderate, low to very low confidence, as part of the CerQual process (Lewin, 2015).
Also for question 2, data concerning publication date, study design, study population, details of the intervention, evaluation measures, and study quality will be independently extracted by the two reviewers. Similar information on the intervention will be extracted as described for question 1. In addition, other factors influencing implementation (such as programme environment factors, recipient-related factors, and socio-cultural, physical and personal contextual factors, as included in the ToC) will be extracted.
Quantitative as well as qualitative data will be extracted using a codebook developed for this purpose (see Appendices 1 and 2). The codebook is based on the deconstitution of the ToC, so that elements of the ToC become fields in the codebook. All items of the codebook will be incorporated in EPPI reviewer software, so that data extraction can be performed easily in parallel by the two reviewers.
In the codebook each variable will be theoretically and operationally defined, which will facilitate intercoder and intracoder agreement during the data extraction process. We will conduct a pilot trial of the codebook to check its adequacy, and make changes if necessary, in an iterative process. It will be clearly stated which codes were developed pre-hoc and post-hoc.
VI. Statistical procedures and conventions
Evidence relating to question 1 (effectiveness of promotional approaches) will be synthesized in a quantitative way (meta-analysis), where possible. Where meta-analysis is not possible or is deemed inappropriate, we will report results from individual studies separately. The measures I2 and τ2 will be used as a measure of presence of heterogeneity, which will be further explored (e.g. by using moderator analysis). We will consider statistical heterogeneity to be substantial when the value of I2 is greater than 50 per cent. We will use random effects meta-analysis to produce an overall summary, if an average treatment effect across trials is considered meaningful. Fixed effect meta-analysis will not be applied because its homogeneity assumption will likely not be satisfied in this systematic review. Included experimental studies will be categorized and analysed according to the different interventions/comparisons. Experimental and non-experimental studies will be analysed separately. Pre-specified primary/secondary outcomes data will be extracted separately. Binary outcomes will be analysed using risk ratios (+ 95% confidence intervals (CI)). For continuous data, mean differences (+ 95% CI) will be calculated. Standardized mean differences (+ 95% CI) will be calculated when different units were used. In the random effects meta-analysis, Mantel-Haenszel (M-H) methods will be used for binary outcomes and the Inverse-Variance (I-V) method will be used for continuous outcomes. Where possible, we will use the robust variance estimation approach to combining effect size estimates (e.g. Hedges, 2010). Unit of analysis issues will be carefully considered in order to adjust for the clustering effect (cluster randomized trials) and/or for multiple testing (multi-arm trials). Cluster randomized trials will be scrutinized and, where analyses would not have been adjusted for clustering, we will adjust the estimates using values of intra-cluster correlations (ICC's) from the literature. The effect of missing data on the overall results will be assessed through sensitivity analysis with respect to data imputation. If only a treatment-on-the-treated (TOT) analysis is available, an intention-to-treat (ITT) analysis will be estimated based on the available data on adherence. We will test for significant differences between ITT and TOT findings in a moderator analysis.
If substantial statistical heterogeneity is detected, investigation of potential sources of heterogeneity will be investigated through moderator or subgroup analysis. If post hoc subgroup analyses will be conducted it will be clearly stated that these analyses are post hoc and exploratory in nature. Depending on the data that will be extractable, moderator and subgroup analyses may be conducted with respect to (1) different types of promotional approaches (training/education versus community-based participatory approaches versus marketing approaches) since we want to give policy advice concerning effective promotional elements, (2) the targets of the study (individual, household, community) (3) the setting where the approach has been applied (rural, urban, informal-urban; see Peal, 2010) (Fiebelkorn (2012) reported differential behaviour change near the city and among the rural population; see also DFID, 2013), (4) the scale at which the approach has been applied (small scale (one village, several villages) vs. larger scale (sub-district, district, province or region, national); see Hulland, 2015), and (5) other equity factors such as socioeconomic status, occupation and education (O'Neill, 2014) (adherence to water, sanitation and hygiene programmes is known to be highly associated with these confounding factors; see DFID, 2013). Sensitivity analyses may be performed with respect to the quality of studies to test the robustness of the meta-analysis by assessing whether results are not influenced by the inclusion or exclusion of low quality studies. The meta-analysis results that will be observed may suggest additional factors that can be used for subgroup and sensitivity analyses. Any interpretation of differences by moderator or subgroup will draw on statistical significance tests (e.g. ANOVA or meta-regression). The presence of any publication bias will be assessed through the visual inspection of funnel plots and also by formal statistical testing (e.g. using Eggertest).
Statistical support will be provided by the statistician, who is part of the review team (Alfred Musekiwa) and meta-analyses will be conducted using Stata version 14 software. Meta Analysis results will be displayed by use of forest plots.
Evidence conclusions will be formulated in a narrative way, but mentioning the effect sizes (and CI) where possible, and taking into account the level of evidence. If possible, differences in results will be explained by giving a description of likely explanatory factors. A statistically non-significant p-value (e.g. larger than 0.05) will be interpreted as a finding of uncertainty (“no evidence of effect”) unless confidence intervals are sufficiently narrow (no imprecision according to the GRADE approach) to rule out an important magnitude of effect (“evidence of no effect”). Accuracy of numeric data in the review will be checked against the data as available from the original study.
VII. Treatment of qualitative research
For the qualitative evidence synthesis we will use the “Best fit framework synthesis” approach (Booth, 2015; Carroll, 2013).
A first step of this approach is to identify an existing model for a particular health behaviour, in this case “WASH behaviour”. In the scoping phase of this project (review of reviews), existing models for WASH behaviour change were identified, including the RANAS model and IBM-WASH model (Mosler, 2012; Dreibelbis, 2013). These models, which were included in our ToC, will be used as “a priori framework”. In addition to the information from the WASH behaviour change models, elements from the “Checklist for implementation” (Cargo, 2015), the SURE framework for the implementation of a policy option (The SURE Collaboration 2011), and the PROGRESS framework to consider equity issues (O'Neill, 2014) were used to inform the a priori framework.
In the second step of this approach, we will code data from individual qualitative studies against the a priori themes of our ToC model representing factors that are able to influence the implementation of the promotional approaches ToC model (i.e. programme environment factors and recipient-related moderators, process evaluation factors and recipient-related contextual factors). Inductive, thematic analysis techniques will be used with data that cannot be accommodated within these themes.
The conclusions of both strands of evidence will be integrated at the end point of the review process in the conclusion and discussion section. In addition, the conclusions will be coupled back to the ToC. Conclusions will be based only on findings from the synthesis (quantitative or narrative) of studies included in the review.
In the discussion section of the review, policy implications of the findings will be discussed, taking into account local considerations. In addition to the policy messages, implications for research will be formulated.
TABLE EXAMPLE: CHARACTERISTICS OF INCLUDED STUDIES
TABLE EXAMPLE: SUMMARY OF FINDINGS
Footnotes
SOURCES OF SUPPORT
This review is supported and funded by WSSCC in partnership with 3ie, and co-funded by Belgian Red Cross (± 10%). We would like to acknowledge the following members of the Advisory Group for their valuable input and feedback on the protocol, via electronic means and/or in the face-to-face meeting, held in Cape Town from 10-12 February 2016: Alfonso Alvestegui, UNICEF Mozambique Colex Chapendeka, Malawi Red Cross Society Chaitali Chattopadhyay, WSSCC Libertad Gonzalez, The Netherlands Red Cross Society Harun Joho, British Red Cross Brian Kae Enriquez, Philippine Red Cross
DECLARATIONS OF INTEREST
The authors are not aware of any conflicts of interest arising from financial or researcher interests.
REVIEW AUTHORS
| Name: Emmy De Buck |
| Title: Senior researcher, Manager |
| Affiliation: Centre for Evidence-Based Practice, Belgian Red Cross |
| Address: Motstraat 40 |
| City, State, Province or County: Mechelen |
| Postal Code: 2800 |
| Country: Belgium |
| Phone: +3215443514 |
| Email: |
| Name: Taryn Young |
| Title: Director, Professor |
| Affiliation: Centre for Evidence-Based Healthcare, Stellenbosch University |
| Address: PO Box 241 |
| City, State, Province or County: Cape Town |
| Postal Code: 800 |
| Country: South Africa |
| Phone: +27219389001 |
| Email: |
ROLES AND RESPONSIBILITIES
Content: Hans Mosler is the coordinator of several Watsan research projects in low- and middle-income countries resulting in more than 40 peer-reviewed publications. He is specifically experienced in WASH behaviour change. Axel Vande Veegaete has performed numerous assessment and evaluation missions for development and rehabilitation programmes for the International Federation of the Red Cross and Red Crescent Societies and Belgian Red Cross in the field of water, sanitation and hygiene in Burundi, India, Malawi, Mozambique, Namibia, Nepal and Rwanda (1998-2003). Systematic review methods: Emmy De Buck, Taryn Young, Hans Van Remoortel, Axel Vande Veegaete, Philippe Vandekerckhove, Margaret Cargo and Karin Hannes co-authored several systematic reviews, including systematic reviews in the domain of humanitarian aid. Taryn Young and Karin Hannes are both involved in the Cochrane Collaboration and co-authored several Cochrane systematic reviews. Karin Hannes is also involved in the Campbell Process & Implementation Methods Group and is particularly experienced in qualitative evidence syntheses. Margaret Cargo is experienced in theory-driven approaches to systematic effectiveness reviews, process evaluation of complex health interventions and mixed method evidence-synthesis of complex health interventions. Thashlin Govender is first author of several publications in the epidemiological field on the health and sanitation status of specific low-cost housing communities as contrasted with those occupying backyard dwellings in the City of Cape Town, South Africa. Statistical analysis: Alfred Musekiwa provides biostatistical support to the University of Pretoria (South Africa), and statistical consulting within and statistical consulting within Centers for Disease Control and Prevention (CDC), National Institute of Communicable Diseases (NICD) and National Department of Health (NDoH) including giving statistical support to PhD and MPH degrees in health sciences. He has computer proficiency regarding the following statistical software packages: Stata, SAS, Epi-Info, SPSS. Information retrieval: Vittoria Lutje has 13 years of experience as a specialist in the retrieval and assessment of scientific and clinical information. She designs and tests highly specialized search strategies for Cochrane systematic reviews and other systematic review groups.
PRELIMINARY TIMEFRAME
Registration, respectively, the review area is opened up for other authors. Submission of draft review: To be determined
Submission of final review: 28 February 2017
PLANS FOR UPDATING THE REVIEW
This review will be updated every 5 years, as a collaboration between the Centre of Evidence-Based Practice of Belgian Red Cross and the Centre for Evidence-Based Healthcare of Stellenbosch University.
AUTHORS' RESPONSIBILITIES
By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.
A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.
You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.
PUBLICATION IN THE CAMPBELL LIBRARY
The support of the Campbell Collaboration and the relevant Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review and subsequent updates in the Campbell Library. Concurrent publication in other journals is encouraged. However, a Campbell systematic review should be published either before, or at the same time as, its publication in other journals. Authors should not publish Campbell reviews in journals before they are ready for publication in the Campbell Library. Authors should remember to include a statement mentioning the published Campbell review in any non-Campbell publications of the review.
I understand the commitment required to undertake a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:
Form completed by: Emmy De Buck
Date: 18 February 2016