Enhancing Pediatric Trainees Assessment of Common Respiratory Complaints in Children

Abstract

Technology enhanced learning (TEL) modules on common pediatric respiratory conditions have often not been aligned to appropriate pedagogy. This study appraised the impact of a new TEL module on the skills and knowledge of pediatric trainees. The TEL module on common pediatric respiratory conditions was piloted using a combination of medical educational theories and feedback from participants was collected. The feedback from 32 participants was largely positive as demonstrated by high Likert scores on the evaluation forms. The post-module quiz scores significantly improved when compared with the pre-module quiz scores (67.0 ± 21.9% versus 42.5 ± 23.8%, P < .0001). Participants reported that the TEL module was time efficient and tailored to different levels of learners providing a self-directed and flexible learning approach based on their clinical practice. Trainees perceived that this new TEL module improved their skills and knowledge, enhancing their management of pediatric respiratory conditions going forward.

Keywords

technology enhanced learning postgraduate training pediatrics flexible learning

Brief Points

What is already known on this topic?

Online lectures have become popular since the COVID-19 pandemic.

In pediatric respiratory medicine, there is a dearth of well developed educational resources.

Many educational resources developed are not based on medical education theory.

What this paper adds?

This study showed that pediatric trainees who took the educational resource created were more confident in managing common respiratory conditions in childhood.

The TEL module developed was flexible, self-directed and based on medical education theory.

The process of the module developed here can be applied to any area of postgraduate pediatric medicine.

Introduction

Technology enhanced learning (TEL) in medical education has been revealed to be as successful as regular teaching approaches in a number of meta-analysis studies, provided these are underpinned by appropriate pedagogy.¹ Since the COVID-19 pandemic, there has been an explosion in E-learning resources for postgraduate trainees.² With this increase, there are many examples of TEL not being adequately underpinned by educational theory resulting in poorly designed modules ^3,4. Pediatric trainees often have demanding on-call schedules that can impose time restrictions and limits access to learning resources hence making it challenging to participate in active learning. The advantages of TEL are that it promotes self-directed learning (SDL) and active learning while facilitating student’s access and flexibility to a diverse amount of information in many forms in their own time and space.⁵

In the field of pediatric respiratory medicine, one of the most challenging areas is the clinical assessment of young children with common respiratory complaints.^6,7 This is mainly due to confusing terminology and a wide variation in clinical phenotypes.^8
-11 Respiratory conditions account for most presentations to pediatric hospitals, yet the assessment of the clinical findings in these children is mixed.¹²

The aims of this study were to assess the impact of a new pediatric respiratory TEL module on the knowledge of pediatric trainees and to also explore their perceptions on the newly developed TEL module.

Methods

Participants

A TEL module on common pediatric respiratory conditions was piloted in July 2020. All pediatric trainees registered on the national training schemes were invited via email to take part. Informed consent was taken from all participants. Participation was voluntary, and all data collected were anonymized.

Module Design

The TEL module created took the form of a recorded short online interactive lecture. There are numerous studies that demonstrate video-based lectures as an effective tool to align classroom learning and workplace practice.¹³ For the purposes of this study, we piloted a TEL module on common pediatric respiratory topics; namely wheezing illnesses, community-acquired pneumonia, and chest x-ray interpretation. The content comprised simulated case scenarios, patient management pathways, and premodule and postmodule multiple-choice questions (MCQs)/quiz.

Application of Medical Education Theory

The TEL module design was underpinned by Mayer’s multimedia principles and also was supported by educational theory such as cognitive load theory (CLT).^14,15 Implementing Mayer’s multimedia design principles helps create effective and engaging contents whilst developing the online lectures.¹⁵ A discipline-specific online lecture was chosen, which adhered to CLT, providing learners with an efficient learning tool that reduces extraneous cognitive load.

Evaluation of Trainee Feedback

A mixed-method approach, collecting data from qualitative and quantitative methods, was applied in this study. Participants filled out an online evaluation form via Survey Monkey after completing the module. The evaluation form consisted of a number of 5-point Likert scale questions and open-ended questions which contributed to the collected quantitative data and qualitative data, respectively.

A random selection of 4 participants then took part in an online focus group discussion (FGD) to explore in depth about the trainee’s perceptions on the module. This small FGD group, comprised 2 senior house officers and 2 senior registrars, allowed us to obtain more detailed feedback on the module and informed consent was obtained from participants. The FGD was video recorded after obtaining permission from all participants. The recordings were deleted after being transcribed verbatim, and anonymous transcripts were analyzed for codes, categories, and themes. The recording was then deleted.

All participants took a multiple-choice quiz pre-TEL and post-TEL module and the scores were collated and compared. Both quizzes consisted of the same 3 case histories with MCQs on investigation and management. In total, there were 6 MCQs, 2 on each case history and both the prequiz and postquiz were compulsory for all participants to complete the module.

Statistical Analysis

Statistical analysis was performed using paired Student t-test comparing the numerical data collected from the prequiz and postquiz. A P value of <.05 was considered statistically significant.

Thematic analysis of the data collected from participant evaluation forms and the FGD was performed using the QUAGOL (Qualitative Analysis Guide of Leuven) method.¹⁶

Results

Overall Results

Of 83 registered pediatric trainees eligible to participate in the research study, 71.1% (59/83) participants completed the TEL module and quiz. Feedback from 54.2% (32/59) participants was collected through the evaluation form while 4 participants took part in the FGD. The male to female ratio of participants was 1:5, respectively. All participants who took part in the study were nonconsultant general pediatric trainees, comprising 50.9% (30/59) senior house officers, 30.5% (18/59) specialist registrars, and 18.6% (11/59) registrars. Slightly more participants had previous TEL experiences at postgraduate level 56.3% (18/59) compared with undergraduate level (43.7% (14/59). All participants took the module only once and the mean time to complete was 37.6 ± 21.9 minutes.

Quantitative Data

The mean Likert score across the 7 questions on the responders’ overall impression of the module was 3.98 ± 0.11 (range 1-5) (Figure 1). Of the 32 participants who completed the evaluation form, 84.3% (27/32) felt that the module improved their knowledge and met the learning outcomes as described in the TEL module.

Figure 1.

The horizontal bar graph shows percentages of participants’ responses based on the Likert scale as indicated in the legend, against the items in the questionnaire.

The participants strongly agreed that the TEL module enhanced their confidence in managing respiratory conditions in children, giving an average Likert score of 4.1 ± 0.17 (Figure 2). 90.6% (29/32) of participants felt that the module content was applicable to their clinical practice.

Figure 2.

The horizontal bar graph shows percentages of participants’ responses based on the Likert scale as indicated in the legend, against the items in the questionnaire.

The mean ± SD score in the post MCQ was 67.0 ± 21.9%, showing a significant rise in the score (P < .0001) when compared with 42.5 ± 23.8% in the pre-MCQ.

Qualitative Data

Preconceptions of learners

Participants felt that previous TEL modules consisted of inadequate learning materials, making engagement with the materials challenging. However, after attending the current TEL module, they felt otherwise and were pleased to have engaged in the module.

I think I went into it [TEL Module]probably with the pre-existing negative perception almost that it was going to be something that would take up time and that I would just click through the slides, probably have half an eye on it and just get to the end of it. I was very pleasantly surprised that that wasn’t the case. (FG2)

Time limitations

Participants due to their demanding work schedule considered learning flexibility an important positive factor. They liked the idea that they could take the online lectures at a time that best suited them within their own environment.

All the difficulties of getting to course days have been eliminated by being able to complete these modules online and in your own time. (P3)

Different levels of learners

The participants perceived that this educational resource should be available for all level of learners from undergraduate medical students to consultants. Trainees of varying levels felt that the learning received added value to their existing knowledge and skill.

The learning concepts of the online lecture would be applicable to physicians at any level from medical students to consultants. (P19)

Discussion

This study sought to develop an online training module on the assessment of common pediatric respiratory conditions for trainees. Overall, the TEL module developed had an encouraging impact on pediatric trainees, reflected in the positive Likert scores and the reported change in knowledge and confidence with managing respiratory conditions relevant to their clinical practice.

Teaching pediatric trainees in a busy clinical environment is challenging. Trainees report that they do not have the time to study due to the significant service commitments and senior clinicians report that they do not have the time to develop training modules for trainees.¹⁷ However, clinically relevant training modules can be easily developed by clinicians and quickly digested by trainees. A recent review of online lectures at undergraduate level examined 45 studies and reported that the attainment of knowledge was equivalent or better with online lectures compared with traditional lectures.¹⁸ Studies have acknowledged the importance of developing TEL modules with a distinct theoretical framework, clear outcomes, and standardized assessment tools, rather than merely embedding latest digitized teaching tools for online learning.¹⁹ This study created a TEL module which was underpinned by multimedia principles and CLT and led to perceived improved clinical skills and knowledge.

The use of preteaching and postteaching quizzes has been shown to improve student performance in other studies.²⁰ The postlecture quiz scores were significantly higher when compared with the prelecture quiz scores implying active learning and the successful transfer of clinical skills and knowledge to trainees. The high Likert scores reported for the content of the module and confidence in managing respiratory conditions in children infers that the improved quiz scores likely relate to high participant engagement and active learning.

As no one evaluation tool can provide all the trainee feedback needed to answer the research questions, a focus group was added as an additional data source. Focus groups are a good way of bringing together participants for exploratory research and can strengthen any preliminary data collected.²¹ The data from the focus group corroborated and expanded on the data from the evaluation form. It unlocked a number of themes and codes from which we drew more robust qualitative results.

Through this TEL module, trainees experienced a positive change in their attitudes to e-learning. Recent studies examining the impact of online learning during the COVID-19 pandemic have reported that a lack of prior exposure to online learning can negatively affect student engagement.²² Similar to our study, the authors reported that many learners feared reduced student engagement prior to the introduction of TEL.

A recent study assessed the perceptions of medical students on the rollout of online teaching during the COVID-19 pandemic and found that flexible learning opportunities led to better participant engagement.²³ Participants reported that they enjoyed the flexibility of being able to access the online lecture in their own time.

We included participants of different levels of experience to understand how we should cater for the individual needs of learners. By simplifying the study topics into a smaller number of basic concepts, medical educators can improve knowledge recall and retention.²⁴ By simplifying the information and delivering acceptable and appropriate amount of knowledge to trainees, the cognitive load was reduced thus increasing the perceived effectiveness of the module for everyone.

A major strength of this study was that in addition to using an evaluation form, findings were triangulated from multiple sources of data from participants through a quiz and an FGD. In addition, the use of a mixed model approach collecting both qualitative and quantitative data made the findings from this study more robust.

Since only 59 participants took part in this single-center study, it may be difficult to generalize the study findings. Also, we cannot comment whether or not the perceived improvements in knowledge and skills will transfer to clinical practice. To counteract the small sample size, an FGD and a quiz strengthened the study. Another drawback in this study was the lack of social interaction and peer feedback as standalone online lecture was designed without any face-to-face component. However, a lack of social interaction was not voiced as a negative perception in this study. Finally, the effects of the COVID-19 pandemic on e-learning was not examined which might have provided more insight. This study was designed prior to the COVID-19 pandemic but an opportunity was missed to examine how doctors might have adapted to the abrupt sudden switch over to online learning as other authors have discussed.²³ Online learning is popular since the COVID-19 pandemic will hopefully lead to improvements in online learning going forward.

Conclusions

Pediatric trainees perceived that this new TEL module improved their skills and knowledge, enhancing their management of pediatric respiratory conditions going forward. We created a learner-centered, flexible, and SDL module grounded by robust educational learning theory that can be applied to different postgraduate pediatric education settings.

Author Contributions

DWC: designed and carried out the study in addition to writing up the manuscript. PMN: designed the TEL module and edited the manuscript. QA: oversaw the study design, supervised the execution the study and the application of medical education theory to the study.

Footnotes

Acknowledgements

We would like to acknowledge Edel Hynes, Mairead Graham and Rebecca Murphy from the education department at Royal College of Physicians in Ireland (RCPI) who brought their multimedia and IT expertise to the study.

ORCID iD

Des W Cox

Ethics Considerations

The study was approved by the ethics committees of the RCPI and the University of Dundee.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Childs

Blenkinsopp

Hall

Walton

Effective e-learning for health professionals and students—barriers and their solutions. A systematic review of the literature—findings from the HeXL project. Health Info Libr J. 2005;22(suppl 2):20-32.

Goh

Sandars

A vision of the use of technology in medical education after the COVID-19 pandemic. MedEdPublish (2016). 2020;9:49.

Hao

Peng

Ding

, et al. Application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic: a systematic review. Nurse Educ Today. 2022;108:105183.

Seymour-Walsh

Weber

Bell

Pedagogical foundations to online lectures in health professions education. Rural Remote Health. 2020;20(2):6038.

Mooney

Bligh

JG.

Information technology in medical education: current and future applications. Postgrad Med J. 1997;73(865):701-704.

Hensley

Review: clinical assessment is inaccurate for diagnosing community-acquired pneumonia. Evidence Based Medicine. 1998;3(3):89.

Gajdos

Beydon

Bommenel

, et al. Inter-observer agreement between physicians, nurses, and respiratory therapists for respiratory clinical evaluation in bronchiolitis. Pediatr Pulmonol. 2009;44(8):754-762.

Wootton

Feldman

. The diagnosis of pneumonia requires a chest radiograph (x-ray)-yes, no or sometimes? Pneumonia (Nathan Qld). 2014;5(suppl 1):1.

Al aseri

. Accuracy of chest radiograph interpretation by emergency physicians. Emerg Radiol. 2009;16(2):111-114.

10.

The L. A plea to abandon asthma as a disease concept. The Lancet. 2006;368(9537):705.

11.

Van Sickle

Magzamen

Maenner

Crane

Corden

TE.

Variability in the labeling of asthma among pediatricians. PLoS ONE. 2013;8(4):e62398.

12.

Florin

Ambroggio

Brokamp

, et al. Reliability of examination findings in suspected community-acquired pneumonia. Pediatrics. 2017;140(3):e20170310.

13.

Lau

KHV

Farooque

Leydon

Schwartz

Sadler

Moeller

. Using learning analytics to evaluate a video-based lecture series. Med Teach. 2018;40(1):91-98.

14.

Young

Van Merrienboer

Durning

Ten Cate

Cognitive load theory: implications for medical education: AMEE guide no. Med Teach. 2014;36(5):371-384.

15.

Mayer

RE.

Multimedia Learning. Cambridge: Cambridge University Press; 2001.

16.

Dierckx

Casterlé

Gastmans

Bryon

Denier

QUAGOL: a guide for qualitative data analysis. Int J Nurs Stud. 2012;49(3):360-371.

17.

Pettersson

Olofsson

Implementing distance teaching at a large scale in medical education: a struggle between dominant and non-dominant teaching activities. The Official Journal of the IFIP Technical Committee on Education. Educ Inf Technol. 2015;20(2):359-380.

18.

Tang

Coret

Qureshi

Barron

Ayala

Law

. Online lectures in undergraduate medical education: scoping review. JMIR Medical Education. 2018;4(1):e11.

19.

Bajpai

Semwal

Bajpai

Car

AHY

. Health professions’ digital education: review of learning theories in randomized controlled trials by the digital health education collaboration. Journal of Medical Internet Research. 2019;21(3):e12912.

20.

Raupach

Grefe

Brown

Meyer

Schuelper

Anders

Moving knowledge acquisition from the lecture hall to the student home: a prospective intervention study. Journal of Medical Internet Research. 2015;17(9):e223.

21.

Stalmeijer

Mcnaughton

Van Mook

WN.

Using focus groups in medical education research: AMEE guide no. Med Teach. 2014;36(11):923-939.

22.

Longhurst

Stone

Dulohery

Scully

Campbell

Smith

CF.

Strength, Weakness, Opportunity, Threat (SWOT) analysis of the adaptations to anatomical education in the United Kingdom and republic of Ireland in response to the Covid-19 pandemic. Anat Sci Educ. 2020;13(3):301-311.

23.

Dost

Hossain

Shehab

Abdelwahed

Al-Nusair

Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students. BMJ Open. 2020;10(11):e042378.

24.

Manning

Spicer

Golub

Akbashev

Klein

The micro revolution: effect of Bite-Sized Teaching (BST) on learner engagement and learning in postgraduate medical education. BMC Medical Education. 2021;21(1):69.