Abstract
The national shortage of pediatric neurologists is worsening, yet referral rates by pediatricians are high. Suboptimal training of pediatric residents in care of patients with neurologic disease may be a contributing factor. We formed a partnership between the Boston Children’s Primary Care at Longwood clinic and Child Neurology Residency Training Program. The educational intervention included lectures, observed neurologic examinations, in-person and virtual triage, and an electronic medical record–based consult system. Residents in other primary care clinics served as the comparison group. Intervention-group residents reported significantly improved confidence in diagnosis of chronic/recurrent headache, attention deficit hyperactivity disorder (ADHD), and developmental delay; initial management of ADHD and developmental delay; and secondary management of ADHD, developmental delay, and concussion/traumatic brain injury. Comparison-group residents reported significantly improved confidence only in diagnosis of developmental delay. Our multipronged intervention is a promising approach to improving pediatric resident training in pediatric neurology and may be generalizable to subspecialty collaborations for other residency programs.
Introduction
Neurologic concerns are common in pediatrics. At least 10% of pediatric office visits involve a neurologic chief complaint, and in a large pediatric primary care network, 11% of subspecialty visits were to neurologists.1,2 A survey of pediatricians indicated that, on average, 61% “always” or “almost always” referred patients for new-onset seizures, 21% tics, and 7% migraines. 3 Physicians with lower self-assessment scores of neurologic knowledge and skill in performing a neurologic exam refer significantly more than other physicians. 4 Child neurologists are also more likely than other pediatric subspecialists to report lower-complexity referrals, 5 further suggesting discomfort among primary care providers with managing common neurologic issues. Lack of adequate resident training could be a contributing factor, as demonstrated by a survey of recent resident graduates that found that 74% reported a need for more residency training on developmental-behavioral and general neurological conditions. 6 A survey of pediatric residents assessing knowledge of seizures and epilepsy demonstrated significant gaps; less than half of senior residents correctly answered questions on febrile seizures and mechanism of a common antiepileptic drug. 7
Due to the national shortage of pediatric neurologists, 63% of pediatricians report at least a 5-week delay when referring to neurology. 5 This shortage is only expected to worsen, with estimates of a 20% shortfall of child neurologists by 2025. 8 Recruiting and training new pediatric neurologists has proven difficult, but enhancing neurology education of pediatric residents is another potential solution. 5 Furthermore, limited knowledge of neurologic conditions hampers coordination of care between primary care and subspecialty providers. Adequate knowledge can empower pediatricians to interface with subspecialists and families to identify care needs and assign roles, key elements of patient- and family-centered care. 9
Our hospital has separate residency programs and inpatient services for pediatric neurology and pediatrics. As a result, pediatric residents care for relatively few neurology patients despite the prevalence of neurologic disease. We sought to enhance our residents’ education by forming a partnership between the Child Neurology Residency Training Program and the Boston Children’s Hospital Primary Care at Longwood (BCH-PCL) clinic. Our BCH-PCL-Neurology partnership developed and implemented a multipronged approach to pediatric resident neurology education with 2 goals: improving proficiency with the neurologic examination and increasing confidence in diagnosis and management of common outpatient neurologic issues.
Methods
Setting and Participants
From August 2013 to July 2015, pediatric residents at the BCH-PCL clinic (n = 65 in 2013-2014, n = 62 in 2014-2015) participated in the educational intervention. BCH-PCL clinics sessions start with a 45-minute educational talk. Two years of neurology residents (n = 7 in 2013-2014, n = 16 in 2014-2015) participated in the intervention, with logistical planning provided by study authors. Pediatric residents at other primary care clinic sites were the comparison group (n = 76 in 2013-2014, n = 84 in 2014-2015). Other than separate clinic locations, these 2 groups experience no significant difference in clinical exposure or rotations. Our study was granted exempt status by the Boston Children’s Hospital Institutional Review Board (IRB-P00009363).
Intervention
Four features of the intervention were implemented in a staggered fashion (Figure 1).
Timeline of intervention components and data collection.
Interactive Lecture Series
Preclinic educational sessions were provided by neurology trainees and faculty. Year 1 topics included headache and concussion/traumatic brain injury (TBI); in Year 2, sessions on the neurologic examination and sleep were added. Each lecture was delivered daily during a 1-week period.
Observed Neurologic Examinations
During Year 1, neurology residents observed pediatric residents completing a neurologic examination on a primary care patient. Both pediatric and neurology residents reported difficulty finding appropriate time for observations in the primary care clinic and a preference for more abnormal examination findings. In response to this feedback, during Year 2, pediatric residents attended neurology clinic before their own clinic session, where they performed an examination on a neurology patient while being observed by a neurology resident. Written and verbal feedback were provided using a structured assessment tool. Residents also received access to instructional videos demonstrating the neurology examination before their session.
Neurology In-Person and Virtual Consultation
During the first 6 months of the intervention, a senior neurology resident was present in the BCH-PCL workroom 1 to 2 afternoons per week to answer neurology questions and address examination findings, enhancing pediatric residents’ learning in real time. Due to intermittent utilization, we transitioned to a virtual pager, with neurology residents available by page during selected clinic sessions to answer questions and triage concerns.
Electronic Medical Record–Based Consult System
We designed and implemented a standardized consult request form within our electronic medical record (EMR) that enabled the intervention group to receive asynchronous advice and request referrals. Neurology residents responded electronically to pediatric residents to provide education, triage referral urgency, and recommend additional workup when indicated.
Data Collection
We surveyed residents in the intervention and comparison groups by email in August 2013 to obtain baseline data and again in June 2015 to assess the efficacy of the intervention. Outcome measures included resident confidence in the diagnosis and management of 4 common neurologic conditions (chronic/recurrent headache, attention deficit hyperactivity disorder [ADHD], concussion/TBI, and developmental delay) and in performing components of the neurologic examination (mental status, cranial nerves, fundoscopy, motor, reflexes, sensory, and coordination). Survey respondents scored the confidence measures on 10-point Likert-type scales (“not at all confident” to “extremely confident”); for analysis, we collapsed measure scores into 5 categories (1/2, 3/4, 5/6, 7/8, 9/10). The postsurvey for intervention residents also included questions to assess resident use and perceived educational value of the intervention components. Educational value measures were scored on a 4-point Likert-type scale (from “Not helpful” to “Very helpful”). Neurology residents also completed an email survey at the end of the intervention assessing time burden and perceived educational value of observed examinations and EMR-based consult system.
Data Analysis
We determined the frequency of attendance at each lecture and scores of perceived educational value of intervention components. We evaluated whether scores for each confidence measure changed significantly pre- and postintervention by constructing ordinal logistic regression (ologit) regression models for each outcome (eg, resident confidence in diagnosis of ADHD) with survey period (pre- vs postintervention, specified as a dichotomous variable) as the predictor of interest, adjusting for year of residency and intervention versus comparison group. Data analysis was performed using Stata/SE 11.2 (StataCorp, College Station, TX).
Results
The response rate was 44% (62 of 141 [65 intervention, 76 comparison] residents) for the preintervention survey and 27% (40 of 146 [62 intervention, 84 comparison] residents) for the postintervention survey. We found no difference in postgraduate year distribution between the intervention and comparison groups in either survey. Nine of 16 neurology residents (56%) responded to the survey.
Reported lecture attendance ranged from 50% to 68% per lecture. The percentage of respondents who gave a score of 3 or 4 when rating the educational value was 60% (9/15) for the neurological examination and 62% (8/13), 64% (7/11), and 86% (12/14), respectively, for the lectures on headache, sleep, and concussion/TBI. Having neurology lectures in clinic was reported to be “very helpful” or “somewhat helpful” by 87% (19/22) of respondents.
During the first cycle of observed examinations, 84% (n = 55) of intervention-group residents had ≥1 neurologic examination observed by a neurology resident, of whom 69% felt the experience was “somewhat” or “very” helpful. All neurology residents (n = 6) observed pediatric resident examinations, with 67% finding the exercise educationally valuable; however, 67% noted that finding time to observe this examination in a busy pediatric resident clinic was difficult.
In the second year, 95% (n = 59) of intervention-group residents had a neurologic examination observed, with 52% (n = 31) completing the examination feedback survey. Overall, 55% reported that the session improved their neurologic examination skills by at least a “moderate amount”, and 73% reported it was “moderately” or “extremely” helpful to receive feedback on their exam skills. Sixty percent watched the instructional examination videos, with 78% finding the videos “moderately” or “extremely” helpful.
During the 17 months following introduction of the EMR-based consult system, residents requested 36 separate consults. All residents reported that this tool was “somewhat” or “very helpful” for patient management. Neurology residents estimated that these consults collectively took <60 minutes per week.
Intervention-group residents were significantly more confident post- versus preintervention in the diagnosis of chronic/recurrent headache (adjusted odds ratio [AOR] 4.2 [95% CI 1.3, 13.6], P = .02), ADHD (AOR 3.6 [95% CI 1.1, 11.4], P = .03), and developmental delay (AOR 5.0 [95% CI 1.6, 16.1], P = .007; Table 1). They also reported greater confidence post- versus preintervention in the initial management of ADHD (AOR 3.1 [95% CI 1.0, 9.3], P = .047) and developmental delay (AOR 8.0 [95% CI 2.4, 26.6], P = .001) and in the secondary management of ADHD (AOR 4.1 [95% CI 1.3, 12.6], P = .01), developmental delay (AOR 5.6 [95% CI 1.8, 17.5], P = .003), and concussion/TBI (AOR 3.7 [95% CI 1.3, 11.1], P = .02). Comparison-group residents only reported significantly improved confidence in diagnosis of developmental delay (AOR 3.6 [95% CI 1.1, 12.2], P = .04).
Mean Preintervention Resident Confidence Scores and Adjusted Odds of the Association of Scores and Survey Time.
P < .05; bP < .01.
Intervention-group residents did not report significant changes in confidence in any of the neurologic examination components. Comparison-group residents reported significantly improved confidence only in the motor examination (AOR 3.5 [95% CI 1.1, 11.6], P = .04).
Discussion
Our novel, multipronged intervention was well received and increased confidence in the diagnosis and management of several common neurologic conditions. This confidence may translate to greater proficiency and autonomy in diagnosis and management of these conditions in the future. Our comparison group, who had identical clinical exposure outside of primary care clinic, reported improved confidence in only one area of common neurological diagnoses.
No significant change in confidence with the neurologic examination was reported. Our intervention to improve examination skills was composed of 1 lecture, at most 2 observed examinations, and access to instructional videos (viewed by only 60% of residents). These limited and brief experiences may not have provided sufficient longitudinal instruction to effect significant change. In addition, baseline self-assessed examination skills were high, with median values of 7 or 8 on a 10-point Likert-type scale for almost all components. Feedback from pediatric residents revealed a desire to spend more time in neurology clinic to practice examination skills under supervision, while neurology residents wanted longer sessions with pediatric residents. Future changes in neurologic exam training will include more frequent and/or prolonged observations, a focused session on fundoscopy, and serial use of standardized examination forms to document true improvement.
Although lectures were generally well received and subspecialty lectures were felt to be valuable, attendance varied (50% to 68%). This reflects a common problem with resident conferences, namely that despite valuable content, resident schedules (eg, night rotations, postcall) and competing priorities often lead to low attendance. Under the flipped classroom model, materials for residents to review independently with subsequent opportunities to ask questions and correct common mistakes may lead to more exposure and perceived value. 10 Those residents who viewed the instructional examination videos did find them useful. Clinical behaviors can be enhanced by a combination of in-person and brief electronic learning, as is afforded by our EMR-based consult system. 11 Further revision and distribution of these electronic resources will allow us to better evaluate their utility.
Importantly, the neurology residents responsible for the educational interventions generally reported a reasonable time demand, particularly for the EMR-based consult system. Additionally, these interventions allowed increased opportunities for teaching, an important aspect of training.
Our study has limitations. Statistical power was limited by the number of residents who completed the surveys. In addition, because our setting is a tertiary care center with a neurology residency program that fully staffs its own primary inpatient service, our intervention might not be applicable to all pediatric residencies. Last, our primary outcomes are self-assessments, which may not reflect true change in clinical behavior or skill. Ongoing analysis of our residents will seek to demonstrate whether our intervention results in fewer or more appropriate referrals to pediatric neurology.
Our intervention not only improved resident confidence in knowledge of common neurologic conditions, but also provided useful information regarding the value and burden of various educational strategies. We chose to focus on education in neurologic conditions because of the gap in our pediatric residency training and the national shortage of child neurologists. However, given growing hyperspecialization within tertiary care centers, other collaborations among training programs could also be valuable. The limited time commitment, improvement in resident confidence, and expansion of subspecialty learning and teaching opportunities make this an attractive model with potentially wide applications.
Author Contributions
EZ conceptualized and designed the study, conducted the analysis, interpreted data, and drafted and revised the manuscript. MBL, AAP, RG, and DKU conceptualized and designed the study and reviewed and revised the manuscript. CR conceptualized and designed the study, conducted the analysis, interpreted data, and reviewed and revised the manuscript. SLT conceptualized and designed the study, conducted and supervised the analysis, interpreted data, and reviewed and revised the manuscript. All authors approved the final manuscript as submitted and agree to be acountable for all aspects of the work.
Footnotes
Acknowledgements
The authors would like to acknowledge the hard work of Larissa Wenren, Christine Shrock, and Coral Stredny in the data collection for this study.
Authors’ Note
Early data from this work have previously been presented in poster format at the following conferences: (a) Pediatric Academic Societies Annual Meeting, April 26, 2015, San Diego, California; (b) Harvard Medical School Resident/Fellow as Teacher Curriculum Showcase, September 17, 2014, Boston, Massachusetts; and (c) Harvard Medical School Medical Education Day, October 28, 2014, Boston, Massachusetts.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the HMS Center for Primary Care’s Academic Innovations Collaborative (AIC) and Boston Children’s Hospital.