PROTOCOL: School feeding for improving the physical and psychosocial health of disadvantaged elementary school children

The rationale for school feeding programs

Across the world, over 300 million children are chronically hungry; in developing countries, 1 in every 3 children under five fail to reach their full growth potential, largely because of chronic malnutrition ( UN 2000 ). Malnutrition is inextricably linked to poverty; millions of families cannot meet basic needs for energy and protein; even more cannot provide their children with adequate micronutrients or with properly balanced diets ( Martelin 1994 ; Darnton Hill 1998; WHO 1996 ). There is increasing evidence that malnutrition in childhood has devastating consequences, including poor physical and cognitive development, lowered resistance to illness, and mortality ( WHO 1996 ; Nelson 2000 ). The Food and Agriculture Organization reports that 6 million children die each year because of hunger. Even mild malnutrition can lead to weight loss, stunted growth, and increased susceptibility to disease ( Worobey 1999 ). Malnutrition and micronutrient deficiencies can lead to poor cognitive and behavioral functioning as well as to lower academic achievement ( Worobey 1999 ; Meyers 1989 ; Pollitt 1995 ).

School feeding programs have the potential to address several of these problems ( Macdonald 1979 ). Their major objectives are to improve nutrient intake, decrease hunger and malnutrition, increase school attendance and enrolment, and to improve cognition and academic performance ( Levinger 1984 ; Gleason 1995). Those who implement school feeding programs feel that, over the long term, they may encourage students to stay in school longer, and that literacy rates, particularly among women will rise (GHC 2002; Arya 1991). Many school feeding programs are aimed specifically at poor and low-income children.

Yet, there is some controversy over both the short and long-term effectiveness of school feeding programs. According to one supporter “Research has shown that when food is provided at school, hunger is immediately alleviated, and school attendance is doubled” p 1, (GHC 2002). Others say that school feeding programs are not a cost-effective solution, and that they address a symptom, rather than the root causes of hunger ( MacIntyre 1992 ). Furthermore, there is a potential danger that the delivery of some school feeding programs may lead to stigmatization and dependency ( MacIntyre 1992 ; Levinson 1995). Regardless of their viewpoint, most experts agree that more thorough evaluation of school feeding programs is needed in order to resolve this controversy, and to determine whether school feeding programs are indeed a cost-effective and equity promoting use of resources.

Previous reviews

We have thus far been unable to identify an existing systematic review on the effectiveness of school feeding programs. However, we have identified some non-systematic reviews and summaries ( Pollitt 1995 ; Levinger 1984 ; Politt 1998 ; Felson 1995). These reviewers have noted that most studies on school feeding programs are observational, and that it is difficult to draw firm conclusions from them. There is some evidence that school feeding programs have the potential to enhance school attendance and nutrient intake, but these are based either on single studies or on narrative reviews. Evidence of effects on school performance is similarly equivocal. One reviewer ( Levinger 1984 ) has suggested that school feeding may be more effective for poor children, but observes that effectiveness for children in different socio-economic positions has not been systematically summarized or evaluated.

None of these reviews have provided a comprehensive, systematic picture of the effectiveness of school feeding programs or of their potential for reducing socioeconomic inequalities in health:

The reviewers did not perform systematic searches. Therefore, the evidence presented in these reviews may be only a partial representation of the relevant evidence.

Many countries and organizations have invested large amounts of money in school feeding programs. It is therefore important to learn whether or not this is an effective and cost-effective intervention for improving the health, nutritional status, school enrolment and school performance of disadvantaged children. It is also important to learn whether these feeding programs have the potential to decrease socio-economic inequalities in health.

This review seeks to answer these questions. It focuses on the effectiveness of school feeding for disadvantaged elementary school children. It is the first in a series of eight reviews on school feeding/supplementation.

The reviews will be broken down by type of feeding and age of the children.

Feeding

Pre-school and formal day-care

Supplementation

Pre-school and formal day-care

The overarching reviews will summarize the most important findings from the smaller reviews in a policy-relevant manner.

Types of studies

Data from Randomized Controlled Trials (RCTs), non-randomized Controlled Clinical Trials (CCTs), Interrupted Time Series (ITS), and Controlled Before and After (CBA) will be examined. Results from each type of study will be tabulated and analyzed separately. All other types of studies will be excluded. We will also exclude studies done in laboratories rather than in the school setting, as laboratory studies do not have direct programmatic relevance. We will accept either no treatment controls (lunch, breakfast at home or no feeding) or placebo controls (e.g. very low energy foods or drinks).

Social interventions such as school feeding are highly complex and political. Because these programs give food to children who need it, there is resistance to having some students or schools in a no-meals control group for the sake of research; there is also sometimes resistance to random allocation (Leiberman 1976). Thus, researchers studying school meals have to work within existing structures; randomized controlled trials may be impractical or impossible to carry out under these conditions. Often, however, researchers are able to use interrupted time series designs or controlled before and after studies.

Moreover, while the randomized controlled trial (RCT) has an important place in determining whether a particular complex intervention produces a particular predefined outcome (the ‘can it work?’ question), traditional RCTs rarely answer ‘what’, ‘why’ and ‘how’ questions such as ‘Why was (or wasn't) this delivery approach used in practice? or ‘what are the barriers to this initiative working outside the research setting?

Types of participants

Children and adolescents aged 5 to 13 who attend elementary school. We will cover both developing and developed countries, although they will be dealt with separately.

Participants must be either:

1)

All from economically disadvantaged groups only or

Types of interventions

Programmes can comprise:

Meals (breakfast or lunch).

These interventions must be administered in the elementary school setting.

We will exclude nutrition education in schools or at home, obesity prevention programs, breastfeeding programs, food stamps, modifications to school meals to change nutrient, fat content, or appeal to participants, community kitchens, and food banks.

Types of outcome measures

Changes in the intervention group and changes relative to the control/comparison group will be examined.

Physical Health outcomes: nutritional status (weight and height gain (adjusted for age and sex when given), peak bone mass, micronutrient status).

Cognitive outcomes: intelligence test scores, psychomotor and mental development, attention, memory, reasoning, vocabulary, on-task behaviour, and school achievement.

Behavioural outcomes: school enrolment, school attendance, and behaviour problems. All outcomes should be relevant for the age group.

Reduction of dental caries will be excluded, as will increased nutritional knowledge. Intermediate physical health outcomes such as reduction of hunger and nutrient intake will also be excluded.

Adverse outcomes: stigmatization, dependency, disruptive behaviour at school, and obesity or excessive weight loss.

Cost outcomes: where possible, we will consider cost-effectiveness

Reductions in socio-economic inequalities in health: Interventions will be classified as effective for reducing inequalities in health, potentially effective for reducing inequalities in health, ineffective for reducing socio-economic inequalities in health, or uncertain.

Effective: We will consider an intervention effective for reducing socio-economic inequalities in health if improvements in health are greater for children in lower socioeconomic groups than in higher groups.

Electronic searches

We have worked with an information specialist (JM) to develop a search strategy. This search strategy will continue to be refined to identify articles that we know to be relevant. The search will be performed on the following electronic databases:

MEDLINE and PreMedline, EMBASE, Cinahl, PsycINFO, ERIC, Sociofiles, HMIS (Health Management Information Consortium), Healthstar, LILACS, System for Grey literature in Europe, Cochrane Controlled Trials Register, C2-SPECTR (Social, Psychological, Educational and Criminological Trials Register), Health Development Agency database of interventions to reduce health inequity, Social Science Index, and Dissertation Abstracts International.

Search strategy (which will be modified as required across databases):

1

milk.sh,tw.

An Internet search will be carried out using Google. In addition to this, key people from organizations focusing on nutrition, hunger, and international development will be contacted by email. These e-mails will introduce our review, and ask for help in identifying studies on school feeding programs which we may have missed. The organizations we intend to approach are listed below:

World Food Program: http://www.wfp.org

We will hand-search the American Journal of Clinical Nutrition, Journal of Nutrition, European Journal of Clinical Nutrition, Nutrition Reviews, Public Health Nutrition, and Social Sciences and Medicine for the past five years. In addition, references of retrieved articles and relevant reviews will be scanned for eligible studies.

Personal contacts

We plan to contact Sally Grantham Mc-Gregor, Ernesto Pollitt, and other authors of the primary studies. Leading resesearchers on interventions to reduce health inequities including Johann MacKenbach, Anne-Marie Gepkens, and Margaret Whitehead will also be contacted.

1. Selection of studies

The abstracts and titles of articles retrieved by the electronic and hand searches will be scanned independently by two reviewers (BK and VR) for eligibility, according to the inclusion criteria above. Full copies of all those deemed eligible by one of the reviewers will be retrieved for closer examination. All studies which initially appear to meet inclusion criteria from this first screening but on closer inspection do not meet the inclusion criteria will be detailed in the table of excluded studies.

2. Data extraction/management

Data will be independently extracted by three reviewers (BK, VR, and DF) who will thoroughly review each other's work. Our data abstraction forms are based on the data collection forms from the Effective Practice and Organization of Care (EPOC) review group, albeit heavily modified for the purposes of this review. We will extract data on study design, description of the intervention (including process), details about participants (including number in each group), length of intervention and follow-up, definition of disadvantaged, health, cognitive and behavioural outcomes, cost-effectiveness, critical appraisal (see below), and statistical analysis. Where possible, we will record effects by socio-economic position, and by other socio-demographic variables, including place of residence, gender, race/ethnicity, and age. Consensus will be reached by discussion and consultation with a third reviewer, if necessary.

After the data abstraction is complete, tables of included and excluded studies will be drawn up. Separate sets of tables will be completed for developing and developed countries.

Within each of these sets of tables, interventions will be further grouped according to type of study, and intensity and type of intervention.

We will carefully describe the interventions given to both the experimental and control groups, having noted that many studies give no intervention to children in control groups and that these children may or may not have had meals at home. In other studies, we have noted that children in the control group are given a low calorie meal or unfortified snack or drink, but the same amount of attention and supervision as children in the intervention group; this controls for the effects of positive attention.

The nutritionists (BM and JK) will also read each primary study in order to determine: 1) whether or not the intervention was nutritionally adequate, 2) whether nutritional status was measured appropriately 3) whether the outcomes are clinically meaningful (the last is for the data analysis/results/discussion sections). They will draw up tables on the quality of the intervention, and will add their comments to the completed abstraction forms.

Process of implementation

The following process elements will be abstracted (list modified from the work of a systematic review by Arblaster et al [Arblaster 1996]):

Intensity of approach (portion size, enery/protein content, percentage of requirements)

Methodological quality of included studies

Two reviewers (BK and BS) will independently rate the quality of each study using the criteria outlined below.

In assessing methodological quality of the RCTs, we will consider allocation concealment, baseline measurement, reliable primary outcome measures, blinded assessment of primary outcomes, protection against contamination, co-intervention, and loss to follow-up. Double-blinding is unlikely to be applicable in this context, and will not be assessed. No overall score will be given.

In assessing methodological quality of the CBAs, we will consider baseline measurement, blinded assessment of primary outcomes, reliability of outcome measures, co-intervention, and protection against contamination.

In assessing methodological quality of the ITS designs, we will consider protection against secular changes, protection against detection bias, reliability of the outcome measures, co-intervention, and completeness of data set.

Each aspect of quality is described in more detail below. These descriptions are modified from those in the EPOC checklist.

Concealment of allocation (protection against selection bias)

Adequate: -if the unit of allocation was by class or school and the randomization process was described explicitly, e.g. the use of random number tables or coin flips, or if the unit of allocation was by student and there was some form of centralised randomisation scheme, an on-site computer system or sealed opaque envelopes were used.

Unclear: -the unit of allocation is not described explicitly or the unit of allocation was by student and the authors report using a ‘list’ or ‘table’, ‘envelopes’ or ‘sealed envelopes’ for allocation.

Inadequate: the authors report using alternation such as reference to case record numbers, dates of birth, day of the week or any other such approach (as in CCTs), or allocation was by student and the authors report using any method that was entirely transparent (e.g.an open list of random numbers or assignments), or allocation was altered.

Baseline measurement

Adequate: student outcomes were measured prior to the intervention, and no substantial differences between study groups were found;

Unclear: baseline measures were not reported, or it was difficult to tell whether baseline measures were substantially different across study groups

Inadequate: differences at baseline in main outcome measures likely to undermine the post intervention differences (e.g. are differences between the groups before the intervention were similar to those found post intervention).

Reliable primary outcome measure(s)

Adequate: For psychological tests, reliability will be considered adequate if internal consistency or parallel forms reliability is greater than or equal to .8 or stability across time is adequate for the time period used. If reliability is not reported, reliability will also be considered adequate if well-validated, standardized psychological tests are used (e.g. the WISC-R). For weight and height, reliability will be considered adequate if height and weight measures are well-described and methods are in line with established protocols. Important details would be training of the anthopometric team, number of replicates of measurement, and type and calibration of equipment.

Unclear: if reliability is not reported for outcome measures and tests are not well-known. For weight and height, unclear will be used if measures and methods of measurement are not clearly described.

Inadequate: if well-known and validated standardized tests are not used, or if lesser known tests are used and internal consistency or alternate forms reliability is lower than .8. The reliability of weight and height will be considered inadequate if the team is untrained, if the measurement is not replicated, and if equipment is not well-calibrated.

If some outcome variables were measured reliably and others were not, each will be scored and reported separately.

Blinded assessment of primary outcome(s) (protection against detection bias)

Adequate: If the authors state explicitly that the primary outcome variables were assessed blindly OR the outcome variables are objective (eg. standardized group test scored by machine).

Unclear: if not specified in the paper;

Inadequate: if the outcomes were not assessed blindly.

Protection against contamination

Adequate: If controls and experimental groups were in different classes or schools, and it is unlikely that controls received meals, snacks, or milk.

Unclear: If controls and experimental groups were in the same classes and it is not clear whether controls received meals, snacks, or milk.

Inadequate: If controls received the intervention.

Co-intervention

Adequate: if interventions other than school feeding were avoided or used similarly across comparison groups

Unclear: use of interventions other than school feeding were not reported and cannot be verified by contacting the investigators

Inadequate: dissimilar use of interventions other than school feeding across comparision groups.

Loss to follow-up

Adequate: Loss to follow-up less than 20% and equally distributed across comparison groups

Unclear: losses to follow-up not reported

Inadequate: losses to follow-up greater than 20%

Protection against secular changes

The intervention is independent of other changes

Adequate: intervention occurred independently of other changes over time Unclear: Not specified

Inadequate: if reported that the intervention was not independent of other changes

Interrupted time series only: Sufficient data points to allow reliable statistical inference

Adequate: At least 3 points have occurred before and after intervention and authors have done repeated measures analysis or ANOVA or multiple t-tests. And at least 30 observations per data point.

Unclear: Not specified in paper

Inadequate: if any of the conditions above are not met.

In addition to this, TG and BK will use a narrative review technique to further explore the quality of included studies. As part of this narrative summary, we will set out the strengths, weaknesses, and contributions of each study in a tabular form.

Data synthesis

Continuous data. RCTS, CCTs, and CBAs. Where baseline data are available from RCTs/CCTs and CBAs, pre-intervention and post-intervention means will be reported for both study and control groups and the absolute change from baseline will be calculated (change in study group values minus change in control group values), along with standard deviations and 95% confidence intervals). If standard deviations (S.d.s) for change are not given, we will calculate them using the formula: SQRT(ABS((sdbaseline^∧2+ sdendof study^∧2) − 2∗Rho∗(sdbaseline∗sdendofstudy))) where Rho represents the correlation between baseline and end of study (we will estimate it at 0.4). When baseline data are not available, results will be expressed as the relative percentage change (difference between post-intervention values in the study and control groups expressed as a percentage of post-intervention values in the control group).

Interrupted time series. We will calculate relative and absolute mean difference in before and after values. When possible, we will use time series regression to calculate mean change in level and mean change in slope.

Discrete outcomes

For discrete outcomes (e.g. malnourished versus well-nourished), we will present the relative risk of the outcome compared to the control group. We will also calculate the risk difference, which is the absolute difference in the proportions in each treatment group. Finally, we will calculate the number needed to treat to achieve one person with the desired outcome.

When possible, comparisons will be reported by socioeconomic group as well as by other relevant socio-demographic variables including baseline nutritional status, gender, race/ethnicity, and place of residence. Where results by socio-economic variables are not available in the primary articles and reports, we will request these data from the authors and recalculate effect sizes and p-values.

Data synthesis

Data will be synthesized qualitatively for all studies. Process data will be summarized and used to interpret results. We plan to do a realist review approach to unpack the contribution of each process element (listed above) to the measured outcome. Our approach will be based on the “Would it work here?” framework developed by Gomm ( Gomm 2000 ); TG will work to implement this methodology in our review.

We will then conduct quantitative meta-analysis, if possible, conducting separate analyses for each outcome across: 1) developing vs. developed country; 2) different study designs (ie ITS, RCT and CBA). The results will be interpreted using clinical significance as well as statistical significance. Nutritionists involved with this review will be asked to judge the clinical significance of the outcomes related to nutritional status, and a neuropsychologist will be asked to judge the clinical significance of the psychological and behavioural outcomes.

Assessment of heterogeneity

We will use the following methods to assess heterogeneity:

1)

Common sense (e.g. are the interventions, participants or outcomes so different that they cannot be combined?). This will be based on a synthesis of the process elements.

Exploring heterogeneity

If heterogeneity exists, we will examine potential sources using the following steps:

Subgroup analysis

Subgroup analysis:

We will conduct subgroup analysis across two factors: socio-economic position, and baseline nutritional status (proxy for SES). We hypothesize that school meals may be more effective for children who are:

Most disadvantaged, poorest, lowest socioeconomic status

Meta-regression

If useful, and with consultation from our biostatistician, we will conduct meta-regression to look at the relation of size of effect to characteristics of the trials. The characteristics we will use in the meta-regression are:

-

gender

Random effects models. Where heterogeneity cannot be explained by subgroup analysis or meta-regression, we will use random effects models to present pooled results. This model assumes that the true effect estimates vary across studies due to both within study differences and between study differences.

We will also explore heterogeneity qualitatively using the “Would it work here” framework developed by Gomm.

Sensitivity Analysis

Sensitivity analysis will be used to evaluate whether the pooled effect sizes are robust across components of methodological quality. For methodologic quality, we will conduct sensitivity analysis for each major component of the quality checklists (eg blinded assessment, randomization, reliable primary outcome). We will also conduct sensitivity analyses to determine whether differences exist in results if we exclude those studies with estimated standard deviations and to assess the implications of using the imputed correlation coefficient in estimating standard deviations for change.

Publication bias

The impact of publication bias will be explored using funnel plots to assess the relationship between effect size and study precision, though formal statistical methods for assessing this may not be appropriate given heterogeneity in the included study designs.

Funnel plots will be drawn to investigate any relationship between effect size and study precision (closely related to sample size). Such a relationship could be due to publication or related biases or due to systematic differences between small and large studies. If a relationship is identified clinical diversity of the studies will be further examined as a possible explanation. (See also Egger 1997 ).

Description of studies

Methodological quality of included studies

Results

Discussion

Reviewers’ conclusions

Implications for practice

Implications for research