Abstract
Linked article:
BACKGROUND
The need for evidence-based health care competencies
Evidence-based medicine (EBM), introduced in 1991, has its roots in the field of clinical epidemiology and was listed as “one of the 15 greatest medical milestones since 1840” in the British Medical Journal (Montori & Guyatt, 2008). The most commonly used definition of evidence-based medicine (EBM) describes it as “the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients” (Sackett, Rosenberg, Muir Gray, Haynes, & Richardson, 1996). It thus requires practitioners to bring together external evidence about the effects of new tests, treatments and interventions, clinical judgement and expertise, and the patient's clinical state, values, preferences, needs and predicament.
EBM is now more commonly referred to as evidence-based practice (EBP) or evidence-based health care (EBHC), as EBM is not limited to medical doctors, but should be adopted by all healthcare practitioners. Practicing EBHC typically involves five steps: (i) formulating an answerable question from a healthcare problem; (ii) finding the best available evidence applicable to the question; (iii) critically appraising the evidence for validity, clinical relevance, and applicability; (iv) applying the results of the evidence in the healthcare setting; and (v) evaluating the performance (Dawes et al., 2005). An important aim of EBHC is that beneficial, effective health care practices are adopted, and that harmful and ineffective ones are abandoned. Consequently, this requires healthcare professionals to recognise their deficiencies in knowledge and to adopt a philosophy of life-long learning, which is the backbone of practicing EBHC (Greenhalgh & Macfarlane, 1997).
The importance of the knowledge, skills, and attitude learnt through the principles of EBHC is also highlighted in the Lancet report on the health professional for the 21st century (Frenk et al., 2010), which proposes that healthcare professional training become transformative. One of the fundamental shifts of transformative learning aligns almost perfectly with the steps of EBHC: the shift from memorization of facts to “critical reasoning that can guide the capacity to search, analyse, assess and synthesise information for decision-making” (Frenk et al., 2010).
In addition, Glasziou and colleagues (2011) have urged educational institutions to teach medical students skills that enable them to become life-long learners so that they are able to combine external evidence from research with their own expertise and their patients' values and preferences. They emphasize that teaching EBHC skills should form an integral part of the medical curriculum and be re-iterated and practiced throughout undergraduate and postgraduate training (Glasziou, Burls, & Gilbert, 2008; Glasziou, Sawicki, Prasad, & Montori, 2011).
In a recent evaluation of an online module of EBM, we have proposed a set of EBHC competencies that all healthcare professionals should ideally possess once they graduate (Rohwer, Young, & van Schalkwyk, 2013). These competencies comprise five key competencies that mirror the five steps of EBHC, as well as enabling competencies that include basic knowledge of epidemiology and biostatistics. Figure 1 is a graphic representation of the EBHC competencies.
EBHC key and enabling competencies
A number of systematic reviews have explored the effects of teaching EBHC to healthcare professionals, both at the undergraduate and postgraduate stages. From these reviews, it is evident that teaching EBHC to students and health care professionals leads to increased EBHC knowledge and skills (Coomarasamy & Khan, 2004; Flores-Mateo & Argimon, 2007; Ilic, 2009; Norman & Shannon, 1998). The question is therefore no longer whether we should be teaching EBHC, but rather how we should be teaching it. Khan and Coomarasamy (2006) have proposed a hierarchy of teaching EBHC, where integrated and interactive teaching is seen as the most effective way of teaching EBHC (Khan & Coomarasamy, 2006). This, however, only refers to traditional lecture-based or face-to-face teaching of EBHC and does not include the increasingly popular method of electronic learning.
Electronic Learning (e-learning)
Electronic learning or e-learning (as the term is widely used) strategies have been widely adopted by educators around the world, and it is not surprising that it has also become a buzzword amongst medical educators. E-learning—also called web-based learning, online learning, distributed learning, computer-assisted instruction, and internet-based learning— can be defined as the delivery of training material via information and communication technology (ICT), including the Internet, CD-ROMs, DVDs, smartphones, and other media, both inside and outside of the classroom (Frehywot et al., 2013; Ruggeri, Farrington, & Brayne, 2013; Ruiz, Mintzer, & Leipzig, 2006).
Pure e-learning refers to the use of e-learning materials only, without any face-to-face classroom methods. Although it seems to be synonymous to completely online learning, the difference between pure e-learning and fully-online learning refers to the delivery platform. While online learning relies on a web-based delivery platform and requires Internet access, pure e-learning can take place without internet access, e.g., using CD-ROMs to deliver the educational content. Blended learning combines e-learning components with other traditional face-to-face, lecture-based learning in and outside of the classroom (Frehywot et al., 2013).
Potential advantages of e-learning include improved accessibility of educational materials at a time convenient to the learner; individualised or personalised learning, where the learner decides on the amount, pace and place, which allows personal tailoring of the learning experience and meeting of individual learner objectives; lower training costs; collaborative learning environments; the ability to track learner activity; and provision of consistent and standardized training to a larger number of learners all over the world (Clark, 2002; Ruggeri et al., 2013; Ruiz et al., 2006; Welsh, Wanberg, Brown, & Simmering, 2003).
Potential disadvantages of e-learning include costs related to the technologies as well as the staff needed, although most costs seem to occur at the development stage of e-learning courses, and social isolation or the lack of interaction among the participants in the e-learning activity or course. The lack of face-to-face networking and peer interaction can be seen as a huge drawback, possibly making e-learning less attractive and less useful. Poor instructional design, technical problems, and de-individualised instruction can also be regarded as disadvantages of e-learning (Cook, 2007; Welsh et al., 2003).
E-learning is a complex intervention with multiple components and dimensions that may interact differently in different contexts (Wong, Greenhalgh, & Pawson, 2010). In the literature, different components, levels, dimensions, and categories of e-learning have been described.
Ruiz et al. (2006) describe the components of e-learning as being (i) the development of content, (ii) management of the content, and (iii) delivery of the content. Regarding development of content, digital learning objects, i.e., “any grouping of digital materials structured in a meaningful way and tied to an educational objective,” are the fundamental components of lessons, modules, and curricula, created with the help of instructional design and pedagogical principles. Examples of digital learning objects include tutorials, case-based learning scenarios, hypermedia, simulations, and game-based learning modules (Ruiz et al., 2006).
Management of content encompasses the administrative functions (storing, indexing, cataloguing) needed to make content available to learners through portals, repositories, digital libraries, learning management systems (e.g., Blackboard WebCT), search engines, and e-Portfolios (Ruiz et al., 2006).
Content can be delivered in a synchronous or asynchronous way. Synchronous delivery of content refers to real time, instructor-led learning, where all learners receive information simultaneously, and communicate and interact directly with other learners by logging onto platforms like virtual classrooms (e.g. teleconferencing, Internet chat forums, instant messaging). Asynchronous delivery of content occurs when the transmission and receipt of information occurs at different time points. It can include pre-recorded presentations or podcasts, PowerPoint slides, or even more sophisticated applications like simulations. While asynchronous communication is done by means of email, online bulletin boards, listservs, newsgroups, wikis, or weblogs, it enables learners to participate in the learning activity at any time of day, from any desired location (Ruiz et al., 2006; Welsh et al., 2003).
Ruggeri et al. (2013) describe the different dimensions and attributes of e-learning programmes under four headings: synchronicity (asynchronous vs. synchronous), location (same place vs. distributed), independence (individual vs. collaborative), and mode (electronic-only vs. blended) (Ruggeri et al., 2013).
Cook (2005) describes four levels of instructional design in computer-based learning: medium, which refers to the mode of delivery of instruction (e.g., textbook, face to face, computer-based, television); configuration, referring to the “big picture” differences within a given media format (e.g., computer: web-based discussion board, web-based, or CD-ROM-based tutorial, vs. face to face: small-group discussion, lecture); instructional method, referring to teaching techniques that support learning processes (e.g., learning activities, self-assessment questions, clinical cases, simulations, group discussions); and presentation, referring to elements of the given medium that enhance the intervention, but do not qualify as instructional methods (e.g., hyperlinks, multimedia, font simulation fidelity) (Cook, 2005).
Our understanding of the complexity of e-learning of EBHC and the different components, contextual factors, and interactions that are needed to achieve the desired outcomes are graphically depicted in Figure 2 (Rohwer & Rehfuess, 2013).
Systems-based logic model on e-learning of EBHC
In an effort to tease out the characteristics and dimensions of e-learning interventions in order to be combined meaningfully, we have only listed the broad categories that we thought of as being most important.
How e-learning May Work
A number of studies, including a few systematic reviews, have been done in various fields of medical and health science education, examining whether different types of e-learning have an effect on knowledge and skills.
A systematic review on Internet-based learning amongst healthcare professionals (Cook, Levinson, Garside, Dupras, Erwin, & Montori, 2008) found large, statistically significant effects for knowledge, skill, and behaviour outcomes when comparing Internet-based educational interventions to no educational interventions. When comparing Internet-based educational interventions with non-Internet-based interventions, there was a very small statistically significant effect for knowledge outcomes but no statistically significant effect for skill and behaviour outcomes. Internet-based learning in the healthcare professions can thus be beneficial and neither superior nor inferior to traditional educational models.
Other studies have examined how e-learning works, focusing more on the essential components for successful learning, as well as certain contextual factors that influence learning.
A realist review by Wong et al. (2010), looking at “what works, for whom and in what circumstances” when considering Internet-based medical education, concluded that online courses need to engage the learners to use the technology, which is more likely to happen when the technology is perceived as being useful and easy to use; and that interactivity should be a key characteristic of an online course, since learners want to be able to enter a dialogue with tutors and peers (Wong et al., 2010).
A systematic review by Cook and colleagues (2010) looking at ways to improve Internet-based learning had similar conclusions. They found an association between interactivity, practice exercises, repetition, and feedback, as well as improved learning outcomes, although they also found significant heterogeneity which needs to be taken into consideration when interpreting the results (Cook et al., 2010)
E-learning technologies afford a new learner paradigm based on adult learning theory, explaining that adults learn by relating new information to past experiences; tailoring learning to their unique needs (open learning); and applying learning in practice, resulting in more effective and efficient learning experiences. The attributes of accessibility and convenience (distributed learning) personalise the learning, because students decide when and where they are receptive to learning. Learning becomes an individual experience where adults learn because they want to learn–not because they are told to learn. This shift from “expert-led teaching to user-lead learning“ results from intrinsic motivation and offers a much stronger learning stimulus. Some evidence suggests that learning by means of e-learning is more efficient and that learners gain knowledge faster, which in turn translates back to improved motivation and enhanced learning (Clark, 2002; Dabbagh, 2005; Ruggeri et al., 2013; Ruiz et al., 2006).
When considering these attributes and pedagogical underpinnings of e-learning, one can argue that these fit perfectly within the EBHC paradigm. Enhanced learning occurs through internal motivation rather than external drivers, and requires acknowledgement of shortcomings (knowledge gaps) and adoption of a reflective approach towards one's own practice. This alignment between attributes of e-learning and EBHC foci suggests that making use of e-learning in this context could be of value.
The pathway from EBHC e-learning activities to the desired outcomes, i.e. increased knowledge, skill, attitude and behaviour, as well as health outcomes is depicted in Figure 3 (Rohwer & Rehfuess, 2013).
Process-orientated logic model showing the pathway from EBHC learning to desired outcomes
It is important to note that this pathway does not take place in a vacuum, but that ultimately, improved health care delivery and improved health outcomes do not only rely on evidence-based practice, but are influenced by a variety of other factors within the healthcare context, such as socio-cultural, socio-economic, epidemiological, legal, ethical, and political factors on a national and international level (See Figure 2).
Why it is Important to do this Review
A recent overview of systematic reviews (Young, Rohwer, Volmink, & Clarke, 2013) that included 16 systematic reviews examining the effects of educational activities on EBHC found that when comparing single interventions (a workshop, journal club, lecture, or e-learning) with multifaceted interventions (a combination of different strategies, such as lectures, tutorials, e-learning, and journal clubs), multifaceted clinically integrated educational activities were more likely to increase EBHC knowledge, skills, attitude, and behaviour. Although some of the studies included in the systematic reviews related to e-learning activities, it is still unclear whether e-learning of EBHC leads to increased knowledge, skills, attitude, and behaviour. In addition, it is still unknown whether some e-learning strategies are more useful and effective than others.
This review considers e-learning of EBHC to be a complex intervention and focuses on the effectiveness of the interventions, as well as the implementation and contextual factors in delivering the intervention. In teasing out these issues, we hope to be able to formulate evidence-based recommendations for EBHC teachers and program developers.
OBJECTIVES
Primary objective: To assess the effectiveness of e-learning of EBHC on increasing EBHC competencies in healthcare professionals.
Secondary objectives: To assess the effectiveness of specific dimensions of e-learning in increasing EBHC competencies. To assess how educational context influences the effectiveness of EBHC e-learning. To assess how implementation approaches influence the effectiveness of EBHC e-learning.
METHODOLOGY
Study inclusion criteria
Types of studies.
We will consider the following study designs for inclusion in the review, as defined by the Effective Practice and Organisation of Care (EPOC) group of The Cochrane Collaboration (http://epoc.cochrane.org/sites/epoc.cochrane.org/files/uploads/EPOC%20Study%20Designs%20About.pdf): Randomized controlled trials (RCTs) An experimental study in which people are allocated to different interventions using methods that are random. Cluster randomized controlled trials An experimental study in which groups of people (clusters) are allocated to different interventions using random methods. EPOC recommends that only studies with at least two intervention sites and two control sites are included, in order to minimise confounding due to study site. Non-randomised controlled trials (NRCTs) An experimental study in which people are allocated to different interventions using methods that are non-random Controlled before-after studies (CBAs) A study in which observations are made before and after the implementation of an intervention, both in a group that receives the intervention and in a control group that does not. In this case, decisions about allocation to different groups are not made by the investigators. EPOC recommend that only studies with at least two intervention and two control sites are included, in order to minimise confounding due to study site. Interrupted time series (ITS) A study that uses observations at multiple time points before and after an intervention. Studies should have a clearly defined point in time when the intervention occurred with at least three data points before and three data points after the intervention.
In addition, we will include studies supporting the included intervention studies that yield information on the process of implementation. These studies may be quantitative (e.g., process evaluations, quantitative interview studies) or qualitative in nature (e.g., focus groups, qualitative interview studies). The information gathered may be reported within the above included intervention studies, or cited within these and published separately. We will not include these supporting studies in the data synthesis, but rather report their results separately, in a tabular or narrative form.
Types of participants
We will include all healthcare professionals, including doctors, dentists, nurses, occupational therapists, physiotherapists, dieticians, audiologists, mental health professionals, psychologists, counsellors, and social workers; at undergraduate, postgraduate, or continuing medical education (CME) level; working in primary, secondary, or tertiary environments.
Types of interventions
We will include any educational intervention (a co-ordinated educational activity) that includes any or all of the 5 steps of EBHC (asking questions, searching the literature, critically appraising the literature, applying the results, evaluating the process) and is delivered via an electronic platform (e-learning only), or makes use of e-learning in a supplementary way (blended learning). E-learning (web-based learning, online learning, distributed learning, computer-assisted instruction) will be regarded as: delivery of training material via any electronic media (Internet, CD-ROMs, DVDs, smartphones, and other media), independent of the delivery platform.
Types of comparison
We will include the following comparisons: No educational intervention. Any educational intervention that includes any or all of the 5 steps of EBHC (asking questions, searching the literature, critically appraising the literature, applying the results, evaluating the process) that is delivered via an electronic platform, but with different components than the intervention (e.g. interactivity vs. no interactivity; synchronicity vs. asynchronicity). Any educational intervention that includes any or all of the 5 steps of EBHC (asking questions, searching the literature, critically appraising the literature, applying the results, evaluating the process) and is delivered via face-to-face learning, with no e-learning components. These control interventions will be categorized into the same broad categories as the e-learning interventions.
Types of outcome measures
Primary outcomes in this review are: EBHC knowledge, measured by evaluating knowledge scores with a validated pre-and post-training questionnaire, such as the Fresno test (Ramos, Schafer, & Tracz, 2003) or Berlin test (Fritsche, Greenhalgh, Falck-Ytter, Neumayer, & Kunz, 2002). EBHC skills, measured by evaluating skill scores with a validated pre-and post-training questionnaire, such as the Fresno test (Ramos et al., 2003) or Berlin test (Fritsche et al., 2002). EBHC attitude, measured with Likert-scale questions pre- and post-training (e.g., Baum, 2003; or by self-report). EBHC behaviour, measured by self-reported behaviour changes (e.g., amount of questions formulated, amount of searches done) or more objective measurements (e.g., proportion of clinical cases where evidence was consulted).
Secondary outcomes in this review are: Process outcomes Satisfaction of students with method of learning, measured with Likert-scale questions post-training, or open-ended questions yielding qualitative data. Satisfaction of educators with method of learning, measured with Likert-scale questions post-training, or open-ended questions yielding qualitative data. Enablers of the method of learning EBHC, measured with Likert-scale questions post-training, or open-ended questions yielding qualitative data. Barriers to the method of learning EBHC, measured with Likert-scale questions post-training, or open-ended questions yielding qualitative data. Cost (e.g., a sum of all the monetary cost involved in the training). Attrition of learners, measured quantitatively by looking at the proportion of learners that complete the training. Behaviour outcomes Learner adherence, measured either quantitatively by looking at the proportion of learners that adhere to the prescribed learning, or qualitatively by asking learners whether about their learning experience. Evidence-based practice, measured qualitatively through self-report of practitioners. Non-health outcomes Evidence-based guideline implementation, measured quantitatively (e.g., by looking at practice audits). Health care delivery, measured qualitatively through self-report of practitioners. Health outcomes Individual health outcomes; these refer to clinical patient-orientated outcomes, measured quantitatively. Population health outcomes; these refer to health outcomes of the community or the entire population measured quantitatively (e.g., disease incidence rates).
Information retrieval
Electronic searches
We will develop a comprehensive search strategy consisting of relevant terms (Appendix 1) and search electronic databases (MEDLINE, EMBASE, ERIC, CINAHL, CENTRAL, SCOPUS, Best Evidence Medical Education, Web of Knowledge, Google Scholar, PsycInfo) and dissertation databases (ProQuest) for relevant studies. We will not apply any language restrictions.
Searching other resources
We will complement our search with a thorough examination of reference lists of identified studies and will contact experts in the field to identify any on-going or unpublished studies. We will also search trial registries (ICTRP) for on-going studies.
Coding procedures and categories
Two authors (either AR and TY; or AR and ER) will independently screen the search outputs and abstracts for relevant studies. Full texts of studies with seemingly relevant abstracts will be retrieved and assessed for eligibility using the pre-specified inclusion criteria. Studies will be classified as either included, excluded, awaiting assessment, or on-going.
Two authors (either AR and TY; or AR and ER) will independently extract data from relevant studies by electronically completing pre-piloted data extraction forms made available via a link to Google forms (Appendix 2). In case of differences in extracted data, we will discuss these to reach consensus, and if unresolved, further discussion will involve the third author. In case of missing data, we will contact the original study author for clarification.
We will place all interventions (e-learning) and control interventions (e-learning and face-to-face interventions) into one of the following broad categories: Single intervention: standalone EBHC learning Single intervention: integrated EBHC learning Multifaceted intervention: standalone EBHC learning Multifaceted intervention: integrated learning
In addition, e-learning interventions and control interventions will be further broken down into the following subcategories: Pure e-learning: individual learning Pure e-learning: collaborative learning Blended learning: individual learning Blended learning: collaborative learning
An e-learning intervention will therefore be placed in 1 of 16 categories and a non-e-learning intervention in 1 of 4 categories.
Data on the following will be extracted from included studies (see Appendix 2 for detailed data extraction form): Study design Type of study Duration of study Country where study was conducted Participants Number of participants Type of health care professionals Level of education Interventions and control Theory underlying intervention Intervention design Educational content (learning objectives; EBHC key and enabling competencies) Duration, intensity, dose, timing of intervention Intervention delivery Synchronicity (synchronous vs. asynchronous learning) Characteristics of facilitators and tutors (e.g. commnication skills, teaching skills, engagement with learners) Institutional characteristics (e.g. cost, capacity, culture) Outcomes: Primary and secondary outcomes Measurement details (e.g. definition of outcome, tools used to measure outcome) Timepoint at which outcomes were measured Results: Measures of effect with 95% confidence intervals Qualitative data Educational context: Location Learner context (background knowledge of EBHC, computer literacy, learning style) Institutional context (structure of course within larger curriculum, role models) Socio-economic context (Access to internet, access to information, affordability of educaton, availability of personal computer, availability of electricity)
Assessment of methodological quality
For each included study, two authors (either AR and TY; or AR and ER) will independently judge the risk of bias for different domains (selection bias, performance bias, detection bias, attrition bias, reporting bias, other bias) according to the Cochrane Effective Practice of Care's (EPOC) suggested risk of bias criteria (http://epoc.cochrane.org/sites/epoc.cochrane.org/files/uploads/Suggested%20risk%20of%20bias%20criteria%20for%20EPOC%20reviews.pdf) We will resolve discrepancies through discussion and consultation with the third author.
Risk of bias in cluster randomized controlled trials
In addition, we will look at the following risk of bias issues in case of cluster randomised controlled trials: Recruitment bias: We will describe whether participants were recruited before or after randomization of clusters. We will regard studies as having low risk of recruitment bias if participants were recruited before randomization of clusters; high risk of bias if they were recruited after randomization; and unclear risk of bias if information about the timing of recruitment is unclear. Baseline imbalance: We will describe any baseline imbalances between individuals and clusters. Loss of clusters: We will describe number of clusters lost as well as reasons for attrition. Incorrect analysis: We will describe whether analysis was adjusted for clustering.
Risk of bias in RCTs, NRCTs and CBAs
Risk of bias for interrupted time series (ITS) studies
Compatibility with RCTs randomised by individuals: We will describe whether the intervention effects may be systematically different from individually randomised controlled trials (i.e., whether it was likely that the effect size was over- or underestimated).
Measures of intervention effect
Dichotomous data
We will present dichotomous summary risk ratios with 95% confidence intervals. This includes data for the outcomes: attrition, learner adherence, evidence-based guideline implementation, individual health outcomes, and population health outcomes.
Continuous data
We will present continuous data as the mean difference with 95% confidence intervals if outcomes are measured in the same way, and standardized mean difference with 95% confidence intervals if outcomes are measured in different ways (i.e. with different tools between studies). Continuous outcomes will include EBHC knowledge and skill scores, EBHC attitude scores, and EBHC behaviour, as well as costs and satisfaction with module scores.
Other data
Outcomes not measured numerically will be reported in a qualitative manner. These outcomes include self-EBHC behaviour, barriers and enablers to the method of learning EBHC, reports of evidence-based practice, and health care delivery.
Unit of analysis issues
In cluster-randomized trials, the unit of allocation is a group, rather than an individual. Since individuals within clusters tend to behave in a similar way, the data cannot be seen as being independent and thus have to be adjusted. If clustering is ignored, estimates will be too precise and will hence receive more weight in the meta-analysis.
If the cluster randomized controlled trials that we will include have sufficiently accounted for the cluster design, we will include the effect estimates in the meta-analysis. If clustering has been ignored, we will adjust the data by inflating the standard errors by multiplying them by the square root of the design effect (Higgins & Green, 2008). We will then include the data in the meta-analysis.
Dealing with missing data
We will report levels of attrition for all included studies. We will do sensitivity analysis to explore the impact of studies with high levels of missing data.
We will carry out analyses on an intention-to-treat basis, so that all participants will be analysed in the group that they were allocated to, regardless of which intervention they received.
Assessment of heterogeneity
We will assess heterogeneity by describing variability among studies regarding participants, interventions, and outcomes (educational heterogeneity), as well as variability in study design and risk of bias (methodological heterogeneity).
Educational heterogeneity will be explored by clearly documenting the characteristics of participants; all components of the intervention relating to intervention design and delivery; and outcomes and measurement of outcomes in table format. In addition, the educational context (setting, learner context, institutional context, socio-economic context) in which the intervention was delivered will be explored and reported (see Figure 2).
Methodological heterogeneity will be explored by clearly documenting different study designs as well as risk of bias for each study.
We will assess statistical heterogeneity by looking at the I2 statistic as well as the Chi2 test statistic. We will consider an I2 value of more than 30%, and a p-value of less than 0.10 for the Chi2 test for heterogeneity to be an indicator of important heterogeneity. In case of considerable heterogeneity, we will perform random-effects meta-analysis instead of fixed-effect meta-analysis or consider only reporting outcomes in tabular or narrative format.
Assessment of reporting biases
We will assess reporting biases by using funnel plots if there are 10 or more studies in a meta-analysis. We will visually inspect the funnel plot for asymmetry. In the presence of asymmetry, we will perform statistical tests for funnel plot asymmetry. We will use the test proposed by Egger and colleages (1997) for continuous outcomes and the one proposed by Harbord and colleagues (2006) for dichotomous outcomes.
Data synthesis
We will consider studies within the same intervention category as being sufficiently homogenous to be pooled and will synthesize results by performing random-effects meta-analysis of quantitative outcomes, using Review Manager (RevMan, 2011). We will convert means and standard deviations to standardized mean differences (Hedges g effect size). We will report the effect estimate with 95% confidence intervals.
We will also consider other factors affecting heterogeneity, such as participants, outcome measurements, educational context, and study design. If we consider heterogeneity between studies to be high, we will not pool data, but rather report on results of individual studies in a tabular and narrative manner, clearly describing participants, interventions, comparisons, outcomes, results, and measurement tools.
Subgroup analysis
Should we identify considerable heterogeneity, we will perform subgroup analysis on the following primary outcomes: EBHC knowledge EBHC skills EBHC attitude EBHC behaviour
We will perform subgroup analysis on the following subgroups: Type of healthcare professional (e.g. medical vs. allied healthcare professionals) Level of education of healthcare professionals (undergraduate vs. postgraduate vs. continuing medical education) Measurement tool used for outcomes (e.g. Fresno test vs. Berlin test) Synchronicity (synchronous vs. asynchronous delivery) Duration of the intervention
Sensitivity analysis
We will perform sensitivity analysis on primary outcomes to see what effects the quality of studies and the study design have on the effect estimate. In addition, we will explore the impact of studies with high levels of missing data.
Supporting studies
If we find studies supporting the included intervention studies that yield information on the process of implementation, we will not include them in the data synthesis, but rather report their results separately, in a tabular or narrative form.
REVIEW AUTHORS
ROLES AND RESPONSIBLIITIES
Content: Anke Rohwer, Taryn Young Systematic review methods: Anke Rohwer, Taryn Young, Eva Rehfuess Statistical analysis: Anke Rohwer (with consultation of biostatistician) Information retrieval: Anke Rohwer (with consultation of librarian)
SOURCES OF SUPPORT
Institute for Medical Informatics, Biometry and Epidemiology, University of Munich, Germany Centre for Evidence-based Health Care, Stellenbosch University, South Africa
This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 30614)
INTEGRATE-HTA http://www.integrate-hta.eu
Effective Health Care Research Consortium: AR and TY are supported by the Effective Health Care Research Consortium, which is funded by UKaid from the UK Government Department for International Development.
This research has also been supported by the US President's Emergency Plan for AIDS relief (PEPFAR) through HRSA under the terms of T84HA21652 and via the Stellenbosch University Rural Medical Education Partnership Initiative (SURMEPI).
REQUEST SUPPORT
Systematic Searches
Field expertise – Education reviews
DECLARATIONS OF INTEREST
None known.
PRELIMINARY TIMEFRAME
Approximate date for submission of review: July 2014
AUTHOR DECLARATION
By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.
A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.
You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.
The support of the Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review, and subsequent updates in the Campbell Library. The Campbell Collaboration places no restrictions on publication of the findings of a Campbell systematic review in a more abbreviated form as a journal article either before or after the publication of the monograph version in Campbell Systematic Reviews. Some journals, however, have restrictions that preclude publication of findings that have been, or will be, reported elsewhere and authors considering publication in such a journal should be aware of possible conflict with publication of the monograph version in Campbell Systematic Reviews. Publication in a journal after publication or in press status in Campbell Systematic Reviews should acknowledge the Campbell version and include a citation to it. Note that systematic reviews published in Campbell Systematic Reviews and co-registered with the Cochrane Collaboration may have additional requirements or restrictions for co-publication. Review authors accept responsibility for meeting any co-publication requirements.