Abstract
Background:
Evidence indicates that tailored educational resources such as e-Bug in schools can improve students’ knowledge of infection prevention. This study aimed to (1) understand the feasibility of using soap use data as a proxy for measuring student handwashing behaviours and (2) refine a study method that could be implemented by local authorities and schools to assess impact as a result of the implementation of e-Bug.
Design:
A stepped-wedge design was employed, and 10 primary schools were randomly assigned to groups where teachers were trained to use e-Bug, either in July 2019 (early intervention group) or in January 2020 (late intervention group). Pre/post student knowledge-retention questionnaire data and soap use measurements were collected from all schools before and after the early group teachers used e-Bug. The sample included 16 soap use measures, 561 students in the early group and 601 students in the late group. Statistical analysis involved single/multivariable mixed-effects linear and logistic regression.
Setting:
A local authority in England.
Results:
In 15–18 questions, there was a significant relationship between training and students answering questions about microbes, hand/respiratory hygiene and antibiotic use/resistance correctly (degree of change varied). There was not enough power to comment on the relationship between the intervention and soap use.
Conclusion:
This study informs the feasibility of cluster, randomised, controlled trials in educational settings to measure the impact of educational resources. It can be used to inform future studies and highlights the need to ensure adequate sample size to estimate soap use and improve the measurement accuracy of data collection tools.
Introduction
The impact of infections in school children is multifaceted, affecting health and schooling. In semi-closed settings such as schools, some infections can spread faster, resulting in school absenteeism and causing children to miss out on educational opportunities, which may mean that they suffer academically (Cauchemez et al., 2009; Department of Education, 2015; Jackson et al., 2014; Public Health England [PHE], 2014). Meanwhile, infections among teachers potentially mean teacher absences, delays in children’s learning, and schools possibly losing public funding due to absenteeism (Lau et al., 2012; Miller et al., 2008).
The inappropriate use of antibiotics contributes to the increasing global health threat of antimicrobial resistance (AMR) (Llor and Bjerrum, 2014). Partly due to a high burden of respiratory and gastrointestinal illness in this age group, children receive a high proportion of the antibiotics prescribed in the community (Enserink et al., 2013, 2014). A study published in 2018 in the UK (Williams et al., 2018) showed that prescriptions for children made up 74.4% of antibiotic prescriptions for respiratory infections. Previous UK public surveys (McNulty et al., 2007, 2019) have found persistent misunderstandings regarding AMR, and schools are a key setting to improve knowledge of how to prevent and manage common infections and expectations concerning antibiotic use. This aligns with the UK’s 5-year national action plan (UK Government, 2019) to work with ‘educators and local authorities to ensure that all school leavers understand how to wash hands, prevent infections and use antimicrobials appropriately’. Previous studies (Hayes et al., 2020, 2021b) have demonstrated low knowledge of antibiotics and AMR in students because of low coverage in the National Curriculum. However, in most cases following an educational intervention, antibiotic knowledge was retained for at least 6 weeks (Young et al., 2017). Similar findings were also observed following children acting out a musical play related to antibiotic use (Hall et al., 2019).
Infection prevention and control (IPC) serves as an important measure in preventing communicable illnesses and is key in reducing demand for antimicrobial use and AMR (Costelloe et al., 2010). Good hand hygiene practices are one of the most effective ways to prevent and control infections, with some researchers estimating that a million deaths could be prevented globally every year if regular hand washing habits were adhered to (Curtis and Cairncross, 2003; Mathur, 2011).
The average days lost to illness during the academic year 2018/2019 in the UK was 8.4 days per student (Department of Education, 2020). IPC interventions have been shown to reduce infection among children and are associated with lower school absenteeism rates (Kimel, 1996; Lau et al., 2012; Talaat et al., 2011; Willmott et al., 2016). One large-scale intervention promoting handwashing in Chinese primary schools (Bowen et al., 2007) demonstrated a significant reduction of up to 54% in student absence. Numerous other studies have similar findings, including an English study with children aged 4–5 years in Leicestershire primary schools and studies conducted in Spain, Colombia, Finland, France and Sweden (Munn et al., 2020; Randle et al., 2013; Younie et al., 2020), strengthening the link between hygiene education and use of hand washing school-based interventions, with reductions in absenteeism. Similarly, the use of hand sanitiser in schools also has the potential to reduce absenteeism caused by respiratory illness (Azor-Martínez et al., 2014; Dyer et al., 2000; Meadows and Le Saux, 2004; Munn et al., 2020; White et al., 2001).
e-Bug is an educational resource developed and operated by the UK Health Security Agency (UKHSA, 2021) with the aim of increasing children’s knowledge and awareness of microbes, preventing the spread of infection and AMR. e-Bug resources are endorsed by the National Institute for Health and Care Excellence (NICE) and when used in school settings have been shown to markedly improve students’ knowledge of hygiene and AMR topics (Eley et al., 2019b; Lecky et al., 2010). e-Bug train-the-trainer sessions are run by local authorities and provide educators with training to use e-Bug materials (Eley et al., 2019a).
e-Bug’s influence on student/teacher knowledge has been evaluated previously (Eley et al., 2020; Farrell et al., 2010; Fernandes et al., 2019; Lecky et al., 2010). An initial evaluation of e-Bug resources (Lecky et al., 2010) was conducted as a pan-European study involving students aged 9–11 and 12–15 years across England, France and Czech Republic. The study found a significant improvement in IPC and AMR knowledge in students following use of the e-Bug teaching pack in schools. In a recent study (Hayes et al., 2021a), IPC and AMR knowledge were evaluated in educators following face-to-face e-Bug train-the-trainer intervention via pre and post questionnaires. The study found that the e-Bug train-the-trainer intervention significantly improved the confidence, knowledge and skills of educators to teach IPC and AMR topics to children, particularly among teachers without a scientific background. This study supports previous work (Eley et al., 2020; Hayes et al., 2021a) with primary educators and e-Bug training, and also evaluates the wider social and health outcomes following training.
Although the impact of e-Bug on knowledge has been evaluated, less has been done to understand how this change in knowledge equates to improved IPC practices, such as handwashing, and outcomes, such as reducing absenteeism (Eley et al., 2020; Farrell et al., 2010; Fernandes et al., 2019; Lecky et al., 2010). This feasibility study therefore had a twofold aim: (1) to understand the feasibility of using soap use data as a proxy for tracking handwashing behaviour in students and (2) to refine a study method that can be practically implemented by local authorities and schools to assess changes in students’ knowledge and behaviour as a result of the implementation of e-Bug. Findings from the study will be used to inform a larger study focusing on the impact of e-Bug on students’ knowledge and behaviour around AMR and IPC.
Methods
The study was implemented by the South Gloucestershire Local Authority (SGLA) with support from the e-Bug team in the UK Health Security Agency (UKHSA) (formerly PHE) and funding for supplementary supplies (£3,000). Schools were invited to take part in the project through the local Health in Schools programme, to run as a silver award project or as a stand-alone project (South Gloucestershire Council, 2017).
Research governance approval was obtained for the project through the South Gloucestershire Council’s Research Governance procedure, and the project was explained in detail through letters sent to the schools (South Gloucestershire Council, 2022). Signed consent was obtained from each participating school leader for use of anonymised school-level data. No student-specific data were collected or stored, and the questions asked of pupils were like routine curriculum assessment questions, used to assess a pupil’s increased knowledge in a given subject.
Interested schools responded to a letter sent out by the local authority to 90 South Gloucestershire schools (Figure 1). The study employed a stepped-wedge design, with 5 out of 10 schools recruited randomly assigned, using a random number generator, to an ‘early intervention’ group and the remaining 5 to a control or ‘late intervention’ group (see Figure 1 for full details regarding recruitment process).

Recruitment flow diagram.
Three-hour e-Bug train-the-trainer workshops were delivered to early and late intervention group teachers in June 2019 and January 2020, respectively, and included six to eight educators who were taught to use the e-Bug resources and cascade the information to other educators in their schools. Schools were able to decide how many classes and teachers they wanted to include. Once teacher training was completed, each class would have been provided with four 30-minute sessions on each of the core e-Bug topics over the following term: microbes, hand hygiene, respiratory hygiene and antibiotics.
Early intervention schools conducted e-Bug themed lessons during Term 1 (September/October 2019) and late intervention schools during/after January 2020. Pre/post knowledge-retention tests were used to assess changes in students’ responses over time. These had been piloted in a previous pilot study and employed simple true/false questions specific to e-Bug learning objectives (South Gloucestershire Council, 2019). Tests were either filled out by the pupils individually or in younger age groups, or recorded by grouping responses using a blind show of hands (de Wit et al., 2012). Schools were able to decide which primary school classes were targeted and included in the data collection, so the number of pupils and classes assessed varied per school (see Table 1).
School characteristics and sample size.
All data taken from the uk.gov primary school comparison calculator: https://www.compare-school-performance.service.gov.uk/compare-schools?for=primary.
Early – September/ October 2020; Late – January Feburary 2021.
This primary school was formerly combined with a secondary school and has not been in place long enough to have data for 2018/2019. It was excluded from the soap use analysis because of missing data.
Soap use data for all schools were measured over 6-week term prior to any training in June/July 2019 (first data collection point) and then again following training of the intervention schools in November/December 2019 (second data collection point). Schools received information and questionnaires that guided data collection but could choose the dispensers they assessed. Soap used was determined by the school using fractional estimates of the amount of each container used (‘3/4 500 ml container used or 2/3 of a 5 L container’). Knowledge data were collected from all schools before the intervention in September 2019 (first data collection point) and then followed up after the training in the early intervention schools in the November/December term (second data collection point). The same groups of children were followed in each school over the course of the study. A third round of data collection (February/March 2020) was not completed because of the outbreak of coronavirus disease (COVID-19).
Data were entered and cleaned in Microsoft Excel files which were then imported into STATA version 15.1, in which all data manipulation and analysis were performed.
Linear regression was used to assess the soap use outcome and logistic regression to compare knowledge outcomes. The p-value was obtained by means of the likelihood ratio test or Wald test. All models for both outcomes contained the three fixed effects of interest, namely, group (early or late intervention), receipt and use of e-Bug training, and first (summer/early autumn term) versus second data collection point (late autumn term), as main effects. A stepwise modelling procedure was adopted, which differed depending on the outcome. Significance level was taken to be 5%, and evidence of substantial confounding was deemed if a change in 20% or more occurred in the coefficient or odds ratio, respectively, following removal of a variable from the model.
Individual-level data were used in the analysis of the knowledge data by expanding the amalgamated data that were collected by some schools. For each knowledge question, the response of ‘true/false/do not know’ was converted to ‘correct/incorrect’. Non-response was treated as missing and omitted from the analysis as ‘do not know’ was considered an incorrect response. School characteristics from 2018/2019 were extracted from the UK government find and compare school’s website and made available for analysis (see Table 1) (Gov.UK, 2019). Mixed-effects regression was used with class or key stage nested within school as random effects. South Gloucestershire local authority collected the data, but data entry and analysis were conducted by UKHSA and 50% of the data for each school were double-coded to check for accuracy.
When the outcome was knowledge, first an analysis was undertaken without any of the school-level fixed effects (‘single variable’ analysis) and retaining both random effects, regardless of the magnitude. Then a backward stepwise approach including all the school-level fixed effects was adopted, beginning with those with missing data and removing them from the model if neither statistically significant nor substantially confounding for the fixed effects of interest after determining their appropriate functional form. The process concluded when no variable with missing data could be removed, and the appropriate function of each continuous variable had been ascertained.
For the soap use outcome, each of the continuous variables was examined one at a time for the most parsimonious function fitting adequately. School type and number of students were added and removed one at a time if neither significant nor substantially confounding. Following removal of a variable, another was added, and the process ended when no further variables could be removed.
Results
In total, there were 16 early/late soap use toilet measures and 561/601 early/late student responses from all the 10 participating schools. Data from one school were excluded from the model because they had no school characteristic information to use as they were newly formed. Discrepancies in the sample size between the two data collection periods were related to teacher sickness absence during the first-round data collection for one class and variations in student absenteeism.
There was no significant difference in soap use noted in the analysis between June/July and November/December, between the school groups or between groups receiving training or not (Table 2 and Figure 2). However, the sample size significantly limited the power to detect the change between the intervention and control groups. Observations in the group that received training had higher log soap use (Coeff. 0.20, confidence interval [CI], −0.57, 0.97, p = .6) after adjusting for school type (maintained/ academy), school group (intervention/ control), absence rate and time of measurement, but not significantly so.
Log soap use.
CI: confidence interval.
Model based on 30 observations. Adjusted for school type and absence rate, and school random effect.

Soap use over time by intervention group (n = 16) – unadjusted for school type and absence rate, and school random effect.
Questions related to microbes, hand hygiene and respiratory hygiene generally showed improvement in most single variable and multivariable results when comparing students who had received e-Bug training with those who had not yet received training (Table 3). Questions about basic microbiology and respiratory hygiene showed a larger improvement when comparing the groups than hand hygiene. The correct response rate for some of the hand hygiene questions was already high in both groups at baseline.
Odd of correct response to true/false questions about microbes, hand hygiene and respiratory hygiene if the child had received e-Bug education from e-Bug-trained teachers – adjusted for group and time.
Adjusted for school and class effects.
Adjusted for variables that could have included: year group, class, school type, free school meals, absence rate and English not first language.
Most questions related to knowledge of antibiotic use showed an increase in correct responses within the group of children who received early training (Table 4). Children who received training in early intervention schools were more likely to respond that antibiotics do not kill viruses, that most coughs and colds get better without them and that people should not be keeping leftover antibiotics at home than children who received training later. However, the questions related to antibiotics killing bacteria, using other people’s antibiotics and the development of resistance did not show significant improvement in knowledge when compared with the late intervention group. Again, there were high levels of baseline knowledge for some of these questions in both groups, which makes detecting significant changes difficult.
Odd of correct response to true/false questions about antibiotic use if the child had received e-Bug training – adjusted for group and time.
Adjusted for school and class effects.
Adjusted for variables that could have included: year group, class, school type, free school meals, absence rate, English not first language and total £ spent per pupil.
Discussion
This study addressed an unmet research need: namely, evaluating the impact of educational interventions on observed (as opposed to predicted) health-related behaviour. It also contributes to the growing evidence base of the impact resources like e-Bug can have on knowledge among IPC and AMR topics (Eley et al., 2019a, 2020; Hayes et al., 2021a).
Measuring soap use to assess handwashing behaviour has previously been evaluated, especially in healthcare settings (Dyson and Madeo, 2017; Rutter et al., 2019). While in-person observational methods have been previously considered the ‘gold standard’ as they allow evaluation of multiple aspects of handwashing, there are issues with bias (Hawthorne effect), cost, practicality and privacy when done in schools (Bruchez et al., 2020; Purssell et al., 2020; Rutter et al., 2019). A recent literature review (Rutter et al., 2019) sought to inform the assessment of hand hygiene in schools and assessed the value of counting consumption of hand washing materials (like soap) as a proxy to behaviour change. Strengths included a mitigation of the Hawthorn effect and ease of comparing soap use across schools, and challenges highlighted that calculating consumption is difficult in practice and timing/other key aspects of handwashing are not assessed. Compared to other mechanisms, the review found measuring handwashing consumables an effective way of assessing behaviour change for intervention studies in schools and suggests that digital hand hygiene surveillance devices might reduce the considerable access and workload issues of manually counting consumption (Dyson and Madeo, 2017; Rutter et al., 2019). However, the cost and feasibility of schools and local authorities to purchase and instal these devices to evaluate soap use outside of an externally funded study are prohibitive (Rutter et al., 2019).
The sample size of 10 schools was determined by what was possible for the local authority with the resources available. A small sample size, measurement error/bias and the disparity in sample size between the early and late intervention schools (10 measures vs 6, respectively) contributed to the large amount of error noted in the analysis of the soap use data. Schools measuring from larger bulk containers instead of from the dispensers had fewer measures to report per school, decreasing the sample size. Measuring using fractional estimates of containers also introduced measurement bias as these could be more subjective, especially if measurement guides (like a ruler with a marker pen) were not used or not feasible (e.g., when school used pre-filled dispenser sachets). Our recommendations are that future studies:
improve soap use measurement accuracy, precision and comparability between intervention groups. This would include measuring soap use per dispenser instead of bulk containers and calculating use by weight or by investing in digital hand hygiene surveillance devices;
consider seasonal respiratory viruses and the increase in rinse-free handwashing. Rinse-free handwash may be useful in preventing the spread of some microbes that commonly cause infection, are acceptable to teachers and students, and may contribute to a reduction in school absenteeism (Munn et al., 2020). Updated e-Bug resources recommend the use of rinse-free handwash when soap and water are not available; however, handwashing with soap and water should be used at key times like after using the toilet or when hands are visibly soiled (Centers for Disease Control and Prevention, 2021; UKHSA, 2021). Measuring the use of hand sprays/gel and assessing the location of hand sanitising stations and soap dispensers could be used to inform hand hygiene behaviour in future studies;
future studies will need to be larger, using sample size calculations to determine the appropriate study size to detect practically important differences as statistically significant, with sufficient power.
Before and after true and false questions were administered by educators in this study. The same educators who taught the class administered the questions. True and false questions can be useful as they are relatively easy to construct and interpret, can be answered quickly, and scored easily and objectively (Blerkom, 2009). However, questions with only two responses are easier to guess and you are limited to simple questions with ‘black and white’ answers (Blerkom, 2009). A benefit of having questions with only two responses was evident in this study when a blind show of hands was needed to assess responses in the youngest age groups. However, the simple design could be problematic in a questionnaire administered three times in a stepped-wedge design, especially for older students who may be more likely to remember the questions/answers. Reading and vocabulary capabilities of children between the ages of 4 and 11 years also vary considerably and change rapidly over time, making it difficult to develop a questionnaire that is appropriate for the youngest and oldest children. This may explain why there were higher levels of non-response for the younger age groups compared to the older age groups and why some questions showed a high level of baseline knowledge or change in both groups (e.g., baseline knowledge about not using someone else’s antibiotics was high in both groups). Also, results on some of the questions could have also been influenced by learning from other sources in the autumn months, such as from national cold/flu campaigns.
Recommendations for questionnaire design in future studies include:
tailoring the test questions for the key stage 1 (KS1) and key stage 2 (KS2) groups, especially as the updated e-Bug learning resource packs now target KS1 and KS2 groups with different lesson plans (UKHSA, 2021). Question themes would need to remain similar if data are to be aggregated. A blind show of hands will remain necessary for the youngest age groups if a paper questionnaire is used, so true/false questions will be the most appropriate (4–7 years/KS1). However, simple multiple-choice questions might be more appropriate and challenging for the older age groups (7–11 years/KS2);
developing a simple digital data collection tool or surveys would be useful if students have reasonable access to computers/tablets. This would reduce workload, making the data entry and analysis more feasible for local authority staff. It would also minimise errors in data collection and bias associated with reading levels if there was an audio question option and reduce the risk that the educators could influence the children’s responses during testing;
using student codes would mean that pupil’s responses could be electronically matched, which would improve the study’s accuracy and potentially allow for additional analysis on IPC knowledge in schools, including areas specific to gender and equality; however, this would add additional complexity for the teachers administering the test questions.
Initial findings indicate improved before and after student knowledge response in IPC and AMR topics, aligning with similar e-Bug research. Randomisation allowed reduced risk of selection bias between the early and late intervention schools. Using school-level data to account for varying socio-economic determinants using logistic regression analysis was a useful way of controlling for confounding and explanatory factors that would have influenced the results. This feasibility study was led by a local authority and implemented within local schools at low cost. However, a research team within UKHSA entered and analysed the data. This was to test the data analysis method and support when capacity issues were raised during the initial outbreak of the COVID-19 pandemic. Because one school was missing data from the UK government compare website (Gov.UK, 2019), it was excluded from the multivariable analysis, which would have further decreased the sample size for the early intervention schools. A third round of data collection in all schools after full e-Bug roll-out (February 2020) was not possible because of the COVID-19 outbreak, so change in response to questionnaires or soap use was not assessed for all schools after training. Additional plans for qualitative data collection with educators to understand their experiences, the feasibility of the study methodology and to collect information on student absenteeism were also not possible because of the timing of the COVID-19 outbreak.
Beyond e-Bug research, other papers evaluating interventions to improve handwashing and IPC have demonstrated changes in knowledge and behaviour (Chittleborough et al., 2013; Tousman et al., 2007; Younie et al., 2020). A mixed-methods study (Chittleborough et al., 2012, 2013) conducted by a prior version of UKHSA in England focusing on handwashing in primary schools indicated that an educational intervention was well received but difficult to roll out sustainably. It highlighted facilitators to sustaining good hand hygiene behaviour, including adult modelling, student time and priorities, attractive facilities and knowledge/awareness. In the USA, absenteeism rates decreased during implementation of an intervention, and the majority of parents noticed an increase in frequency of their child washing their hands (Tousman et al., 2007). Educational handwashing interventions were also successful in delivering knowledge and behaviour change during the COVID-19 pandemic in young children (Younie et al., 2020). Researchers in Leicester (Younie et al., 2020) developed educational resources around microbes and handwashing to help young children understand the concept and practice behind the importance of good hand hygiene. A range of resources were developed to support this, including a children’s storybook, a song, online games, a website and a visual UV gel handwashing activity. During evaluation, the interventions demonstrated immediate knowledge and behavioural improvements, which lasted a month after implementation. Researchers observed around a 23% increase in children washing in-between their fingers and a 25% increase in children washing around their wrists and under their nails (Younie et al., 2020).
Conclusion
The findings from this study can inform studies that seek to measure the impact of educational resources (e.g., e-Bug) on student knowledge and behaviour in the school setting. Larger studies using similar methodology could help address this research need in the future; however, several lessons have been learned that will help improve future work in this area. Data collection materials should be tailored to key stage group and include digital data collection tools to support early readers. In addition, schools need specific guidance to ensure data are collected in a standardised way across all schools and supported to improve the accuracy of soap use measurements, ideally through digital hand hygiene surveillance devices and/or standardised soap dispensers. Multivariable analysis should include additional information that would allow for further understanding of confounding and/or explanatory variables.
Footnotes
Acknowledgements
We acknowledge the support of South Gloucestershire Local Authority together with teachers, students and staff of the participating schools. We also thank Julie Brooke and Eirwen Sides for assistance.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: At the time of this intervention, all authors except N.Q.V. worked on the e-Bug programme or for South Gloucestershire Council and were involved in development or delivery of the train-the-trainer workshops.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the development and delivery of the intervention were supported and funded by UKHSA and South Gloucestershire Council.
