Abstract

1. Background
1.1 Description of the Problem
Millennium Development Goal 4, as defined by the United Nations in 2000, aims to reduce the 1990 under-five mortality rate (U5MR) by two thirds in the year 2015. A reduction U5MR will reflect the impact of child survival interventions and global policies aimed at international development (Mosley 2003). Nearly 98% of the 7.6 million child deaths globally each year occur in Low-and-Middle-Income countries (LMICs) and 64% of them are from preventable infectious causes namely sepsis, pneumonia, diarrhea and Malaria (Liu 2012). About 40% of the child deaths occur among neonates (0-28 days), from preventable causes such as preterm birth and intrapartum complications and sepsis (Liu 2012 and Lawn, 2012). Though U5MR has decreased from 9.6 million in 2000 (Black 2003) to 7.6 million in 2010, reduction of U5MR could still improve in many developing countries where U5MR remains high (Liu 2012). A slower decline of U5MR has been attributed to stagnant neonatal mortality rate (NMR) (Save the Children 2013) and prevailing high childhood morbidity rates (Murray 2007). Annual decline of NMR during 2000-2010 was slower than annual mortality decline among children 1-59 months when compared to 1990-2000 (Lawn 2012). Hence, child survival strategies should focus on infectious and neonatal causes of mortality (Liu 2012) and direct the resources towards vulnerable socio-economic groups for improved neonatal survival (Lawn 2012).
Available evidence suggests that interventions such as exclusive breastfeeding, clean delivery and skilled attendance at delivery, tetanus toxoid immunization to pregnant mothers, newborn resuscitation, and appropriate management of infections can prevent most neonatal deaths (Darmstadt 2005). In addition, prompt and appropriate management of acute diarrheal diseases (ADD), acute respiratory infections (ARI), and malaria and childhood under-nutrition can also prevent child mortality (Jones 2003). Research has shown that comprehensive training of health care providers (HCPs) on management of childhood morbidities at health facilities and in outreach services, can prevent a significant proportion of childhood mortality (Gove 1997). Studies have reported that available child health services are under-utilized, care-seeking behavior during acute illnesses is neither prompt (timely) nor appropriate (Sreeramareddy, 2006a; Sreeramareddy, 2012) which could be a reason for a slower progress towards achieving MDG-4 in many LMICs. Stagnation in neonatal mortality rate has been attributed to unsafe delivery and newborn care practices particularly in underserved rural communities and urban slums (Osrin, 2002; Sreeramareddy, 2006b; and Seward, 2012). Counseling the parents/care takers about symptoms and danger signs of child illness may improve the care-seeking behavior during the episodes of childhood illness. Therefore, behavior change communication aimed at improving utilization of child health services, seeking prompt and appropriate treatment during illness, improving perinatal and child care practices assume importance in LMICs where U5MR remains high.
1.2 Description of the interventions
Previously child survival interventions were usually implemented as separate disease specific programs for common causes of child mortality. These interventions and others such as exclusive breastfeeding, vaccination and oral rehydration therapy (ORT), are effective, children attending primary care settings are often known to have multiple and overlapping morbidities which need an integrated approach to treat them adequately (Bryce 2005b). The World Health Organization (WHO) and United Nations Children Fund (UNICEF) jointly with other technical partners developed the Integrated Management of Childhood Illness (IMCI) strategy to reduce U5MR in 1997 (Tulloch 1999). IMCI strategy aims to address the limitations encountered in disease specific child health programs by integrating treatment of common childhood morbidities that cause child mortality. Since its conception and after piloting the strategy in a few countries, more than 70 countries have fully implemented IMCI strategy (Lambrechts 1999). Three components of the IMCI strategy are:1) improving case management skills of healthcare providers (HCPs) particularly in the outpatient facilities (health worker interventions), 2) strengthening health systems (health system interventions) and 3) improving family and community health practices (WHO 2003; WHO 2005) (community interventions).
Health worker interventions are implemented by training healthcare providers about use of evidence-based, locally adapted guidelines for managing the leading causes of childhood illnesses. Training is provided in-service for duration of standard 11 days or shortened 5-10 days and a follow-up visit by the trainer after one month to reinforce the newly acquired skills for using IMCI guidelines and job-aids such as wall charts, diagnostic algorithms, etc. Health system interventions are mainly related to the key policies and management of health systems such as Ensuring availability of essential drugs and supplies (job aids), Organization of health facilities for rapid evaluation and management of sick children, and Establishing appropriate referral system, health information management system, monitoring and supervision of healthcare providers
Implementation of household and community interventions of IMCI is done in one or more of the following ways: Counseling parents/caregivers at health facilities about care seeking behavior, danger signs, and home management of childhood illness, compliance to treatment advice, child feeding and utilization of preventive services Community-based peer educators providing information during in-home counseling sessions to parents/caregivers and family members of under-five children about nutrition and health practices and proper referral practices Dissemination of IMCI practices through mass media (mainly television and radio), distribution of posters and brochures to educate the general public about child health issues Community mobilization involves active and meaningful participation of the communities to establish a partnership between health workers and households with support from their communities. This is achieved by involving women's groups, community leaders, religious institutions etc through meetings arranged by healthcare providers to sensitize and motivate them about IMCI interventions.
1.3 How the interventions might work
Training component of Integrated Management of Childhood Illness (IMCI) is expected to improve case management skills of healthcare providers, thus reducing mortality from acute respiratory infections, acute diarrheal diseases and malaria. The health system interventions would improve the infrastructure of health facilities i.e. health staff, job aids, and treatment facilities (drugs) for treatment of childhood illnesses. While the health worker training and health system interventions together would improve the quality of care provided at health facilities; the family and community interventions may improve family's care seeking behavior and community childcare practices. Improved quality of care as a result of training and health system interventions together with community interventions may lead to increase in utilization of child health services. All the three IMCI interventions are expected to act synergistically towards reducing child mortality (Ahmed et al. 2010; Lulseged 2002). A causal chain (figure 1) shows various family and community interventions of IMCI strategy and ensuing programmatic pathways of intermediate and final health outcomes. Family and community interventions are expected to improve care seeking behaviors of family members/caregivers for common childhood illnesses (choosing appropriate care and seeking prompt care (within 24 hours), community practices regarding perinatal and newborn care, caregiver's compliance to HCP's treatment and advice. Family interventions are also expected increase uptake of preventive services, improve child feeding practices also. In addition to these expected change, social mobilization through involvement of women's groups, community leaders will raise the general awareness and demand for improving household and community practices with regards to child health. All these improvements will increase the utilization of preventive and curative services and overall improvements in health and nutritional status of under-five children, thus decreasing child mortality (Bryce, Victora, Habicht, Black, & Scherpbier 2005b).
1.4 Why it is important to do this review
Evidence about impact of Integrated Management of Childhood Illness (IMCI) is available from community trials, WHO multi-country evaluation report and a few systematic reviews. Primary studies which have assessed the impact of IMCI strategy on health care providers' performance have demonstrated that IMCI strategy can improve the quality of care at the health facilities (Amaral et al. 2004; Amaral et al. 2005; Amaral et al. 2008; Arifeen et al. 2005; Bryce et al. 2005a; Huicho et al. 2005). Primary studies which have assessed the impact of health care provider training alone as an intervention to strengthen IMCI program have shown a variable effect on outcomes of childhood illness (Ahmed, Mitchell, & Hedt 2010; Goga & Muhe 2011; Lulseged 2002). However, evidence from primary studies about the impact of one or more of IMCI interventions on child mortality is inconclusive (Amaral, Leite, Cunha, &Victora 2005; Arifeen et al. 2009; Bhandari et al. 2012; Huicho, Davila, Gonzales, Drasbek, Bryce, &Victora 2005). A report about Multi-Country Evaluation on Effectiveness of Integrated Management of Childhood Illness (IMCI), Cost and Impact (MCE-IMCI) suggested that IMCI delivery systems should be expanded to include other potential channels such as private health sector and adopt stronger community-based approaches rather than being just health facility-based (Bryce, Victora, Habicht, Black, & Scherpbier 2005b). A mixed methods multi-country survey has highlighted that financial shortages, logistic difficulties in follow-up after training in IMCI case management training and human resource shortages (in terms of number and/or skills) as challenges to IMCI scale-up (Goga et al. 2009; Goga and Muhe 2011).
To date, we found three systematic reviews on this topic by searching in Index Medicus, Scopus and Cochrane library. A narrative review that focused on parents/caregiver's role in IMCI strategy has underscored that ICMI trained workers lacked communication skills about counseling the parents/caregivers about danger signs and homecare of sick child (Paranhos et al. 2011). A systematic review comparing the effect of standard 11-days training versus shortened 5-10 days training on quality of care has concluded that standard (11-days) IMCI training was marginally better and underscored the need for implementing other IMCI interventions irrespective of the training duration. This review also found that for a third of sick children, the HCPs did not adhere to IMCI guidelines (Rowe et al. 2012). A Cochrane systematic review protocol aims to synthesize the evidence on effect of integration of individual IMCI components as a healthcare package on both HCPs and its beneficiaries (Gera et. al. 2012). However, a Cochrane review of both experimental and quasi-experimental studies has shown that there is very little evidence that integrating primary health care services at the point-of-delivery may improve the utilization and outputs of healthcare delivery (Dudley & Garner, 2011). However, among the published reviews, one review did not perform meta-analysis but only summarised the results on impact of IMCI interventions on mortality rates and utilization rates of child health care services (Rowe et al. 2012) while another review qualitatively synthesized the caregiver's (mothers and/or family members) child health care practices (Paranhos et al. 2011). A published Cochrane systematic review protocol aims to synthesize impact of various combinations of three IMCI interventions on mortality, quality of care and key IMCI indicators. Though family and community interventions of our review overlap with interventions to be included in a review by Gera et al. we aim to focus only on various components of family and community interventions (see section 3.1.2) and assess entire intermediate outcomes related to proximate determinants to childhood mortality. Moreover, none of the above reviews have focussed on the impact of IMCI interventions on community child health care practices as underscored by another review (Paranhos et al. 2011) or focussed the review on the impact of standalone community interventions on child mortality rates plus childcare practices of the family and community. Moreover, published reviews have not compared the impact of supply side interventions (health worker and health system interventions) with demand side interventions (community interventions). In addition, from the existing reviews evidence is lacking about the impact of family and community interventions on compliance to homecare of a sick child, care seeking behavior and preventive practices (Thompson and Harutyunyan 2009). Hence, synthesizing the evidence about impact of family and community interventions of IMCI strategy on its beneficiaries is timely and policy relevant. We aim to report the effect of household and community interventions inclusive of social mobilization on under-five mortality and other intermediate outcomes such as care seeking and community child care practices. We also aim to compare the outcomes demand side interventions (community) with outcome of supply side interventions (health worker plus health system).
2. Objective of the review
Main objective
To assess the effect of family and community interventions under Integrated Management of Childhood Illness (IMCI) strategy on childhood mortality rates, utilization of child health services and community child health practices in LMICs. To compare the outcomes of health worker interventions combined with health system and/or community interventions with outcomes of family and community interventions only To compare outcome of health worker intervention together with health system and/or family and community interventions with outcomes of health worker interventions only.
3. Methods
3.1 Criteria For Considering Studies For The Review (PICOs)
3.1.1 Participants
Newborns (aged up to 28 days after birth) and children aged less than five years Parents/family members/caregivers of newborns and under-five children
If the studies/trials were randomized at the level of villages/wards (i.e. cluster randomized trial) then all participants (as listed above) living in the clusters studied will be eligible. All the populations to be included will be residing in the low-and-middle-income countries (LMICs) as defined by World Bank (The World Bank Group, 2011).
3.1.2 Interventions
We will include the family and community interventions, which is the third component of IMCI strategy listed in the IMCI document published by UNICEF (UNICEF 1999). These interventions are: Individual counseling provided by the HCPs to the caregivers either at health facilities or home Peer-counseling provided to the caregivers, and their family members at home Behavior change communication about child health practices disseminated through posters, brochures and mass media Social mobilization through involvement of women's groups, community leaders etc
We will include all potential studies which have examined the effect of health worker training together with household and community interventions or family and community intervention only or health worker training only. We will also include the studies which have examined the effect of all the three IMCI interventions.
Comparison Groups
Comparison groups in controlled study designs will be those individuals and clusters or communities where routine mother and child health services were ongoing without implementation of IMCI household and community interventions.
3.1.3 Study Types
Studies which have adopted the following designs and are addressing the household and community interventions as described in UNICEF's IMCI document will be included:
Experimental study designs:
Individual Randomized Controlled Trials (iRCTs) Cluster-Randomized Controlled Trials (cRCTs) Quasi-Randomized Trials (qRCTs) Non-Randomized Trials (NRTs)
Quasi-experimental study designs:
Controlled before-and-after (CBA) studies Regression-discontinuity designs (RDD) Interrupted time series (ITS) studies
We will include either individual or cluster RCTs where in unit of randomization is at individual or cluster (village or district) level and in each study comparison group will differ only in their exposure to the intervention i.e. baseline characteristics and potential confounding factors are adjusted for.
qRCT or NRT the trials should have a concurrent comparison group, (for example no IMCI intervention) and groups adjusted for baseline characteristic and investigator allocates into groups by non-random methods (in qRCT allocation into groups is done by pseudo-random sequence).
In CBA studies allocation into the different comparison groups are not made by the investigators and outcomes measurement is done in both intervention and control groups before the intervention is introduced and once again after the intervention has been introduced.
RDD studies are pretest-posttest design which the investigators demonstrate interventions effects by assigning an intervention using a cut-off i.e. above or below a threshold
In ITS studies, investigators obtain data on outcomes from the same population several times before and after intervention.
We will include cRCT, qRCT, NRT and CBA studies, if these studies have studied at least two intervention sites and two control sites.
We will include ITS studies, if the time point of intervention was clearly defined and data on outcomes is available from at least three time points before and after the intervention.
Considering that risk of bias and measures of effect to be different according to types of studies we will synthesize the results separately for RCTs and non-randomized trials (qRCT, NRT, CBA, RDD and ITS studies)
3.1.4 Outcomes
3.1.3.1 Primary outcomes:
Neonatal mortality rates (first 28 days of life) Post-neonatal mortality rates (after 28 days up to, one year of age) Infant mortality rates (first year of life) Under-five mortality rates (up to five years of age)
All mortality rates will be calculated as number of deaths per 1000 live births.
3.1.3.2 Secondary outcomes
(Any of the outcome measures listed below and/or any other outcomes of interest that fit into the categories listed below if identified during the search for studies and/or data extraction will be included and be given an operational definition)
Newborn care practices such as (% of newborns) % of newborns for whom breast-feeding initiation was done within one hour after birth % of newborns who were not given any pre-lacteal feeds % of newborns who were exclusively breast-feeding at four weeks after birth % of newborns who were given skin-to-skin care on the first day of life % of newborns who were appropriately clothed on the first day of life % of newborns for whom nothing was applied on the umbilical cord
% of children aged less than six months who were exclusively breastfed % of children aged 6–9 months who received breast milk and complementary feeding % of children aged 0–23 months who were stunted (defined as ≤ 2 weight-for-height Z score) % of children aged 24–59 months who were wasted (defined as ≤2 weight-for-height Z score)
Proportion of caretakers who sought appropriate care during the illness within last two weeks Proportion of care takers who sought prompt (within 24 hours) care during illness in last two weeks Proportion of care takers who continued feeding the child during illness Proportion of care takers who adhered to the health care providers' advice on treatment
Improvement in utilization of public health facilities or private health facilities Increase in immunization coverage according to expanded program of immunization (EPI) Increase in the proportion of skilled attendance at birth Increase in the proportion of childbirths at health facilities Outcomes measuring the efficacy of community mobilization program (Indicators that measure if IMCI strategy has been successful in educating community members/caregivers about childcare and healthcare seeking behavior) Proportion of care takers i.e. family members/parents who were counseled in the previous six months by a community health worker about child feeding, care-seeking etc Proportion of care takers who had attended a session about community mobilization during last six months. Proportion of mother/care takers who have knowledge about Oral Rehydration Solution and/or home available fluids for management of diarrhea at home Proportion of mothers who have knowledge about at least two danger signs of a sick child
Decrease in the incidence of acute respiratory infections and diarrhea (as a result of improved nutritional status)
3.2 Search methods for identification of studies
3.2.1 Electronic searches
We will perform a comprehensive electronic search for primary studies in both ‘mainstream’ databases and ‘specialist databases'. To minimize the publication bias, we will search all available databases and the titles of the journals covering the fields of child health, public health, and global/international health. To avoid language bias and publication bias, we will perform the searches without any language and publication status restrictions.
The following ‘mainstream’ electronic databases will be searched for primary studies.
PubMed
EMBASE (Athens)
Popline
CINAHL (Cumulative Index to Nursing and Allied Health Literature)
Ovid
Index Copernicus
CAB-Direct (Global Health)
LILACS (Latin American and Caribbean health sciences)
Web of Science
SCOPUS
Science Citation Index Expanded
Social Sciences Citation Index
Sosiological Abstracts
World Health Organization Library Information System (WHOLIS)
African Healthline (bibliographic databases on African health issues)
African Index Medicus
IndMed
Western Pacific Region Index Medicus
Index Medicus for South-East Asia Region
Australasian Medical Index
International Bibliography in Social Sciences (Athens)
The Campbell Library
The Cochrane Central Register of Controlled Trials
The database of Abstracts of reviews of Effectiveness
FRANCIS - bibliographic database in social sciences
BDSP (Banque de Donnees en Sante Publique) – French database on public health literature
MEDCARIB - Caribbean health sciences literature
JSTOR, Wiley Inter-science, Science Direct
HINARI (Health InterNetwork Access to Research Initiative)
We will develop the search strategy in MEDLINE as detailed in the annexure-5. The MEDLINE search strategy will be translated into other databases using appropriate vocabulary. Some suitable primary studies may have been published as monographs and reports or as research articles in journals that are not indexed in electronic databases listed above. So we will search the following ‘specialist’ electronic databases and non-governmental organizations' websites as further sources for primary studies.
WHO/CAH reports - World Health Organization's, Department of Child And Adolescent health (www.who.int/child_adolescent_health/en/)
HCPP (Health Care Provider Performance) database
UNICEF (United Nations Children's Fund)
JOLIS Library Catalog - World Bank & IMF Libraries of the World Bank
British Library for Development Studies (BLDS) - a database about economic and social issues in developing countries at Institute for development studies
ID21 – a database reporting the UK-based international development research
Database on Prince Leopold Institute of Tropical Medicine, Antwerp, Belgium
WHO Regional Index Medicus for Eastern Mediterranean Region (EMRO)
PAHO - PAHO HQ Library Catalog
WHO Global Health Library
DFID (Department for International Development)
ELDIS (Electronic Development and Environment Information System)
OpenGrey
Partnership for Maternal, Newborn and Child Health
Global Strategy for Women's and Child Health
Healthy Newborn Network
Concern International
Hellen Keller International
World Vision International
Concern Worldwide
Save the Children
We will also use ‘Google’ and ‘Google Scholar’ search engines with one more of the following search terms; ‘IMCI’, ‘care-givers' counseling’, ‘community mobilization’, ‘social mobilization’, ‘Millennium Development Goal 4’, ‘MDG-4’, ‘under-5 mortality’, ‘child mortality’, ‘infant mortality’, and ‘neonatal mortality’. In Google search we will use cited references feature to identify further studies.
3.2.2 Searching other resources
We will hand search the journal titles and conference proceedings that have not been hand searched on behalf of the Campbell collaboration. We will refer to Cochrane Collaboration's master list of journals and conference proceedings. The reference lists of retrieved studies will be searched to identify additional studies. Authors of the primary studies and investigators of IMCI evaluations will be contacted to identify any further published or unpublished studies. Experts in the field of health systems research or authors of any other relevant reviews will be contacted to know if they are aware of any relevant studies. For the ongoing trials, we will look into the registered trials at the following trial registries: a) International Clinical Trials Registry Platform (ICTRP), World Health Organization (WHO) http://www.who.int/ictrp/en/ and b) ClinicalTrials.gov, US National Institutes of Health (NIH) http://clinicaltrials.gov/
Lancet
Bulletin of World Health Organization
British Medical Journal
BMC Public Health, BMC Pediatrics, BMC International Health and Human Rights, BMC Health services Research
PLoS Medicine
PLoS One
Global Health Action
International Conference of Pediatrics
World Congress of Public Health
World Congress of Epidemiology
3.3 Data collection and analysis
3.3.1 Selection of studies
The search results from various sources i.e. electronic databases, hand searching and other sources will be merged into a reference manager file. Duplicate references will be deleted and a list of titles, abstracts and full text articles will be generated. Eligibility of the retrieved studies will be assessed by a two-stage process by two review authors (CTS and TNS). At first, two review authors, CTS and TNS will independently screen the lists of titles, abstracts and full text articles by applying pre-defined screening criteria (appendices 1 and 2). At this stage, if any titles/abstracts are deemed not eligible will be excluded. If uncertain in the first stage, we will reassess by reading the full text at the second stage. If any disagreements have to arise during study selection, they will be resolved either by discussion or consulting a third review author (HNHK). If insufficient or ambiguous information is present in the studies, we will contact primary authors for further information or seeking clarification. All these processes will be guided and entered into flow diagram provided in the Review Manager 5.1.
3.3.2 Data extraction and management
At least two independent reviewers (among CTS, TNS, and HNHK) will extract the data using pre-designed data extraction form. Data extraction form will be piloted on a few studies and revised if necessary (see Appendix 1). Extracted data will be stored electronically in RevMan or Stata software. We will ensure inter coder-reliability by following methods: pilot testing the coding form, 2) creating a detailed codebook and 3) training about use of codebook and coding forms. The following information about details of the included studies will be recorded; 1) characteristics of the study, 2) participants, 3) comparison groups, 4) outcomes, 5) statistical analysis (measure of effect size reported) and 5) conclusions (for details see appendix 3). Data to be extracted for the possible meta-analysis includes the number of withdrawals, exclusions and loss to follow-up, and number included in the final analysis, baseline characteristics, types of statistical analysis used, specific measures of effect size reported i.e. both unadjusted and adjusted for overall sample and sub groups. We will also extract data about frequency of dichotomous outcomes, mean and standard deviation for the continuous outcomes. Data about relative risk as a measure of effect size for RCTs NRTs, and CBA studies.
If the same data has been published as more than one publication, the study with the most complete results will be included into meta-analysis. Disagreements will be resolved by consulting a third reviewer or an independent reviewer having expertise in content and methods. Any disagreements that cannot be resolved would be addressed by contacting the study authors. Finally, if all these attempts were unsuccessful, the disagreement would be reported in the review.
3.3.3 Assessment of risk of bias in included studies
Two review authors (CTS and TNS) will independently assess the risk of bias for each included study. Disagreements will be resolved by a third reviewer who has methodological and statistical expertise (HNHK). Assessment of quality of the studies will be done by keeping Preferred Reporting Items for Systematic Reviews (Moher 2009) guidelines as reference and using the risk of bias model in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2008) for methodological quality of RCTs. For non-randomized studies, we will pay particularly attention to selection bias, such as baseline differences between the groups, and the potential for selective outcome reporting (Higgins 2008).
The risk of bias assessment will be based on five dimensions (described below). The questions about assessment of risk of bias will be piloted and modified (appendix 3) and risk of bias will be rated as low risk, high risk, and uncertain risk of bias. Assessment of risk of bias will inform the sensitivity analysis to be conducted (see 5.6.2 Sensitivity analysis). We will report the assessment of risk of bias for each included study.
Risk of Bias Dimensions:
Selection bias
Selection bias is systematic baseline differences between the groups (i.e. observable factors that are not adequately accounted for) which may compromise comparability between the groups.
Performance bias
Performance bias refers to systematic bias and confounding related to intervention fidelity and/or exposure to factors other than the interventions and comparisons of interest that may confound the outcome measured. Blinding of participants and intervention delivery is not applicable for IMCI strategy due to the nature of this intervention
Detection bias
Detection bias deals with systematic differences between groups in relation to how outcomes are determined, including blinding of outcome assessors. Participants who do not undergo IMCI family and community interventions before the end of the study should be censored from the outcome data and if not adequately accounted for may have the potential for introducing bias. Therefore, censoring of the participants is a potential source of detection and attrition bias (see below).
Attrition bias
Attrition bias deals about completeness of the sample and follow up data. This bias refers to systematic differences between the participants who did not (drops outs, migrated etc) complete the follow-up and those who completed the follow-up.
Reporting bias
Reporting bias refers to both publication bias (see 5.5.3 assessment of publication bias) and selective reporting of outcomes data and results.
Other sources of bias
We will examine for other potential sources of bias after extraction of data about study designs and the statistical analyses in the included studies. The focus will be to assess if the study authors have reported other potential sources of bias and if they have dealt with these biases adequately.
For individual randomised controlled trials, non-randomised controlled trials and CBA studies the following sources of bias will be included: sequence generation (selection bias) allocation sequence concealment (selection bias) blinding of participants and personnel (performance bias) blinding of outcome assessment (detection bias) incomplete outcome data (attrition bias) selective outcome reporting (reporting bias) comparability of baseline outcomes and the characteristics and protection from contamination
For cluster- randomised trials, specific attention will be paid to following types of biases: recruitment bias baseline imbalance loss of clusters incorrect analysis and comparability with individually randomised trials (Higgins 2011)
3.3.4 Measures of treatment effect
Dichotomous outcomes will be analyzed using relative risk (RR) ratio and its 95% confidence intervals (CIs). For continuous data, we will report mean and standard deviation for the outcome measures. If continuous data for the outcomes were measured on a same scale between trials, we will use mean difference (MD) with 95%CIs otherwise we will calculate standardized mean difference (SMD) and its 95% CIs. If the means, standard deviations, and/or effect sizes are not available, we will use chi-squared values and correlation coefficients. For rare outcome binary or continuous data (ex: prelacteal feeds, not seeking any healthcare), we will use Peto OR and Mann Whitney test. Hedges' ‘g’ will be used to correct for small sample size. For non-randomized studies, adjusted effect sizes will be calculated to account for the influence of confounding factors by propensity score matching. Exact criteria for selection of effect sizes will be a developed ad hoc during the coding phase
3.3.5 Unit of analysis issues
The included studies are cRCTs or CBA studies, we will use the reported cluster adjusted risk ratios and their 95% CIs. If unadjusted for clustering, we will use intracluster correlation coefficient (ICC), if available or else we will impute ICC from any other included study. If multiple interventions groups are included in the trial, community interventions paired with training and health system intervention will be compared with the control group. If the included studies report variable and multiple time points of outcomes assessments, we will only focus on comparison of baseline and final endpoint even though it is known that effects of interventions may diminish over time.
3.3.6 Dealing with missing data
We will attempt to contact the authors to obtain the missing data and any other information irrespective of whether, it can assumed to be ‘missing at random’. If it will not be possible to obtain the missing data, we will impute the missing values, and all the assumptions made for any imputation will be recorded. We will either use intention-to-treat analyses, if reported or contact the authors to carry out re-analyses. In the absence of intention-to-treat analyses and fair indications that data are not missing at random, we will consider missing data to constitute a risk of bias and report in risk of bias assessment.
Imputation made will be for missing aggregate data such as missing SD of for change-from-baseline, from SD for the same outcome from another study, or impute SD using recommended methods (Abrams 2005). If necessary, we will impute the missing data using appropriate statistical methods (White, 2009). In addition to this, we will report attrition rates of >50% in any groups and explore its effect on the outcomes by doing a sensitivity analyses.
3.3.7 Assessment of heterogeneity
We will measure heterogeneity using I2 statistic. I2 test measures the statistical heterogeneity across the studies. I2 statistic describes the percentage of total variation across studies due to heterogeneity rather than chance (Higgins, 2002). I2 can be readily calculated from basic results obtained from a typical meta-analysis. I2 is given by 100%×(Q - df)/Q, where Q is Cochran's heterogeneity statistic and df is the degrees of freedom (Higgins, 2002). Negative values of I 2 are considered as zero so that I 2 lies between 0% and 100%. (Higgins, 2002). We will also calculate Cochran's Q-test and τ2 statistic.
3.3.8 Assessment of publication biases
Publication bias will be evaluated using a quasi-statistical method of drawing a Funnel Plot. Funnel plots will be drawn to investigate relationships between effect size and study precision using the ‘trim and fill method’. Meaningful Funnel plots will require an adequate number of studies with a diverse range of sample sizes (Hayashino 2005). Hence, a funnel plot analysis will be drawn if there are at least ten studies with appropriate data. We will also conduct formal statistical tests for funnel plot asymmetry, using Begg's and Egger's methods (Egger 1997).
3.4 Data synthesis
Analysis will be conducted in RevMan 5 and/or Stata as appropriate. Meta-analysis will be done separately for RCTs and n-RCTs in accordance to current recommendations (Higgins, 2011). Due to diversity in interventions, participants and outcomes we anticipate the use of random effects models. If we find at least two studies that evaluated similar interventions and reported similar outcomes, we will calculate pooled risk ratios, mean differences or standardised mean differences using a random-effects model. Pooled estimates will be calculated by generic inverse variance method and will be generated as Forrest plots. If there are not enough (at least two) studies evaluating similar interventions and reporting a similar outcomes, we will report the median and range of effects, if relevant, or measures of effect from individual studies. We will do separate meta-analysis for the following IMCI interventions or their combinations 1) community interventions only, 2) health worker interventions only, 3) health worker interventions combined with health system and/or community interventions. If meta-analyses is possible, we will compare the outcomes across combinations of IMCI interventions. If meta-analyses was not possible we will prepare a ‘Summary of findings’ table, including an assessment of the quality of evidence for each of the main outcomes or types of outcomes listed above. Quality of evidence will be assessed using the GRADE approach for each outcome and will be classified in four levels as ‘high’, ‘moderate’, ‘low’ and ‘very low’ (Guyatt 2008).
3.4.1 Subgroup analysis and investigation of heterogeneity
We will perform the following sub-group analysis, if data becomes available. The difference in outcome measures that may be explained by certain factors will be considered for sub-groups analysis. We will test the following sub-groups. If IMCI-family interventions were given as stand alone or along with training and/or health system component. We expect that the effects may be greater with addition of these IMCI components.
Country settings such as Low Income Country (LICs) or Middle Income Countries (MICs) as defined by World Bank. The effects may be higher in LICs compared to MICs. Countries classified according to current U5MR as a bench mark. For example, we will classify countries with U5MR > 100 per 1000 live births and U5MR <100 per 1000 live births. The effects are expected to be greater in countries having high U5MR.
The uncertainty around heterogeneity will be explored with sub-group sensitivity analysis. Further, heterogeneity will be explored in meta-regression considering the potential covariates for comparison and as a possible explanation for heterogeneity. The coefficient for each covariate will be checked for statistical significance at conventional p-value of <0.05. This method has been considered as valid method for combining estimates from different types of study designs and to pool common estimate of interest (Harris, 2008).
3.4.2 Sensitivity analysis
Sensitivity analysis will be used to examine the rigor of conclusions in relation to the quality of data and approaches to analysis. Sensitivity analysis will be used to investigate the possibility of study design influencing outcomes. We will perform sensitivity analysis encompassing all the probable studies included into meta-analysis. We will test our findings by modifying any assumptions we made about missing data within a plausible range of values and by removing those studies with high risk of bias, if there are studies with different levels of risk of bias. We will report sensitivity analyses as a summary table. We will attempt to generate individual forest plots for some of the sensitivity analysis we will undertake.
3.4.5 Narrative analysis
To capture the major studies and give a sense of research in IMCI family and community interventions as previously mentioned, we will include relevant studies. To make our analysis more transparent, we will report these studies in a separate narrative analysis that will focus on intervention characteristics and contextual factors. The narrative analysis will enhance our understanding of IMCI family and community interventions included in the review and inform the discussion section.
Acknowledgements
The authors would like to acknowledge the help provided by the ICMR systematic review support Group.
Advisory group:
Dr. Prathap Tharyan, (He is an Editor with the Cochrane Schizophrenia Group, Coordinator of the South Asian Cochrane Network) Expertise in systematic review methods
Anthony Costello (Professor of International Child Health and Director of the UCL Institute for Global Health) His main scientific expertise is i the evaluation of community interventions to reduce maternal and newborn mortality
Contributions of authors
Declarations of interest
None
Sources of support
