An Expert Evaluation of Oncology Website Resources for Use in Pediatric Oncology Clinical Nursing Education in Low-Resource Settings

Abstract

Background: Online healthcare information is often used by pediatric oncology nurse educators in low- and middle-income countries (LMICs) for teaching clinical nurses as part of their initial orientation or continuing education. Access to peer-reviewed nursing journals via paid subscriptions or sub-specialty nursing textbooks in these settings is rare. This project identified and evaluated websites appropriate for pediatric oncology nurse educators in LMICs for teaching staff nurses, and for clinical staff nurses engaging in self-directed learning. Method: A strategic Google search for childhood cancer websites and an appropriate scoring tool was conducted. The Currency, Relevance, Authority, Accuracy, and Purpose Test, along with a previously published scoring rubric that was further adapted by the authors for pediatric oncology were used. Pediatric content, language options, and reading levels were appraised. Results: Of 86 identified websites, 51 met the inclusion criteria for evaluation. Websites were classified as highly recommended (n = 36), recommended (n = 12), or not recommended (n = 3) based on scores (range 14–30; maximum possible score = 30). Half offered content in multiple languages. Most websites were 9–10th-grade reading level. Discussion: Childhood cancer information appropriate for clinical nurse orientation and self-directed learning by LMIC nurses is available on free websites. Some information (diagnosis, chemotherapy, psychosocial support) is repeated across websites, while some is lacking (pediatric cancer genetics and health equity disparities). Reading levels are higher than recommended for health literacy. The reviewed websites were rarely peer-reviewed, inconsistently updated, and generally self-regulated. However, 48 websites on childhood cancer were deemed appropriate pediatric oncology clinical nursing education resources.

Keywords

childhood cancer pediatric oncology nursing internet education

Introduction

The 2018 WHO Global Initiative for Childhood Cancer (GICC) strives to increase countries’ capacity to provide quality services for children with cancer to address the inequalities in access to specialized care and essential medications between low- and middle-income countries (LMIC) and high-income countries (HIC). It is universally acknowledged that effective childhood cancer care requires a specialized pediatric oncology workforce to implement strategies including community awareness efforts, adapted protocol-driven care, and improved interdisciplinary care in local settings (World Health Organization, 2022). Nurses are in a privileged position to implement initial cancer care, manage effects related to cancer care, and participate in end-of-life care and survivorship follow-up. Despite this situation, the global shortage of specialized pediatric oncology nursing workforce is most apparent in LMICs (Challinor et al., 2020; Simonyan, 2017).

Although LMIC nurses have access to baccalaureate or diploma-level education, their specialty-specific education is limited, often due to a lack of experienced nursing faculty (So et al., 2016). Nurses often receive instruction in only introductory pediatric and adult oncology concepts in undergraduate nursing programs (Banayat et al., 2021). Specialty pediatric oncology nursing education is generally provided in the workplace and usually includes (a) didactic content along with precepted experiences by senior nurses, (b) competency evaluation before independent practice, and (c) continuing education throughout a nurses’ career. Implementation of specialty education, however, is often limited and unstructured and is largely dependent on the availability of expertise from local and international partners. Education for specialized oncology nursing roles is also frequently lacking (Day et al., 2015). Access to resources is a key barrier to implementing specialty pediatric oncology nursing education in LMICs. Nurses in LMICs who care for patients with cancer often have limited access to specialty texts and peer-reviewed publications, particularly in their native languages, to augment their clinical education (Eniu et al., 2019; Galassi et al., 2017). Additionally, many nurses who acquire specialty education are not recognized for their knowledge and skills by their workplaces or by other governing authorities (Challinor, 2022; Diaw et al., 2020; Yang & Zhang, 2013).

Identifying appropriate and accurate materials to support nursing education can be challenging. Paywalls prevent access to specialized educational materials or journal subscriptions (Wilson, 2019), and the cost of subscriptions is often a barrier. Therefore, nurse educators in these settings often rely on freely available, Internet-based information, including content pertaining to childhood cancer, to educate nursing staff. Free, Internet-based information is also used by individual staff nurses themselves who seek additional information on childhood cancer care (Berland et al., 2020). The content may not be current or relevant, and the date of publication or update may not be obvious. The authority of content creators is not always verified, and the content may not be peer-reviewed. Use of inaccurate content pulled from low-quality websites may lead to errors or inappropriate nursing care, especially when providing support and information to patients and families. Many nurses in LMICs who understand English are nonnative English speakers; therefore, the presence of website content in multiple languages is important. Not all websites with important pediatric oncology content, however, have translations available.

No earlier review of pediatric oncology nursing websites appropriate for use by clinical nurse educators or self-directed clinical staff nurses was located through literature search. Previous work has focused on evaluation of websites for other pediatric subspecialties (Fast et al., 2012), evaluation of pediatric oncology website content and usability for patients and families (Knijnenburg et al., 2009; Ruble et al., 2019; Stinson et al., 2011), or use by health care professionals in rural settings (not specific to nursing; Martiniuk et al., 2022).

The objective of this review was to identify and evaluate websites for use by pediatric oncology nurse educators to educate staff nurses, and for staff nurses who engage in self-directed learning in clinical settings (e.g., hospitals and clinics) in LMICs. The websites were reviewed for their currency, relevance, accuracy, authority, and purpose. Language availability and reading level of sites were also evaluated. Attention was given to determining website literacy level, non-English language options, and relevance to pediatric oncology nursing in LMICs.

Method

Search Strategy

No previous website evaluation of pediatric oncology nursing resources was located through an extensive search of PubMed, CINAHL, and Google Scholar databases (2011–2021). The authors developed the search strategy. All three are masters- or PhD-prepared expert pediatric oncology nurses with extensive experience in pediatric oncology nursing education. One nurse lives and works in an LMIC setting; the other two nurses are from HICs with considerable experience working with nurses in LMICs. A medical librarian was also engaged to develop and refine the search strategy, which was conducted between February and March 2021.

The author team conducted an exhaustive Google search to identify websites using the following search terms: pediatric oncology website, childhood cancer website, pediatric oncology nursing website, pediatric oncology nursing practice website, psychosocial childhood cancer websites, palliative care children with cancer, pediatric psycho oncology society, AYA cancer support websites, palliative care childhood cancer, cancer research UK, childhood cancer genetics information, and cancer genetics education. Inclusion criteria for websites were as follows: all websites in English with pediatric oncology information relevant to nursing. Exclusion criteria were (a) journal articles, (b) websites not available in English, (c) patient or survivor blogs, (d) consolidation websites (e.g., list of links to other resources), (e) patient/family financial support or advocacy websites, (f) websites for hospitals (e.g., advertising services), (g) “news” websites (e.g., links to news media headlines), (h) website with a paywall, and (i) nonfunctional link. Additionally, websites that addressed only precision medicine or hematopoietic stem cell transplants and other related cellular therapies, which are not generally used in LMIC nursing practice, were excluded. Websites could be primary sources of information, for example, a website designed to provide information written by the organization. Online educational repositories such as websites with a collection of webinars, seminars, or other educational materials that are housed for the purpose of educating health care providers were also included. After applying the inclusion and exclusion criteria, the remaining websites were divided into the following sections by specific topic: pediatric hematology/oncology, psychosocial, pediatric palliative care, adolescent and young adult cancer, patient and caregiver support, and pediatric oncology nursing practice.

Instrument

Multiple website scoring tools were considered for this review: the Criteria for Evaluating Web Resources from Kent State University (2018) Libraries, the Kirkpatrick Evaluation Framework by Campbell et al. (2019), the National League of Nursing (2020) Hallmarks of Excellence, and the Currency, Relevance, Authority, Accuracy, and Purpose (CRAAP) Test of Meriam Library (2020). The CRAAP Test was selected because the scoring categories aligned most closely with the purpose of this review. The other tools considered were deemed less appropriate since they did not evaluate for relevance, focused on pedagogy and assessing training outcomes, or focused on the implementation of academic programs.

The CRAAP Test is widely used by librarians to teach resource evaluation to higher education students. Developed in 2004, the tool assists librarians in evaluating the credibility and reliability of academic sources and has since been adapted to evaluate online content. The tool has five domains: currency, relevance, authority, accuracy, and purpose (see Table 1). The original CRAAP Test has a set of criteria for each domain but does not include a scoring rubric. A group of pediatric neurosurgical healthcare providers created and applied a scoring rubric using the CRAAP domains to evaluate pediatric neurosurgical websites for use by patients and family members (Garcia et al., 2018). These authors assessed the internal consistency reliability of the newly rubric and deemed it acceptable with a Cronbach's alpha = .83.

Table 1

CRAAP Tool Domain Details

Domain	Evaluation
Currency	Timeliness of the information
Relevance	Importance of the information for your needs
Authority	Source of the information
Accuracy	Reliability, truthfulness, and correctness of the content
Purpose	Reason the information exists

Note. CRAAP = Currency, Relevance, Authority, Accuracy, and Purpose Test.

For this study, the CRAAP Test and scoring rubric previously published (Garcia et al., 2018) were further refined (Tables 2 and 3) to evaluate pediatric oncology websites for use by nurses and applied to the websites meeting inclusion criteria. An overall score between 0 and 30 points was possible, with higher scores indicating greater credibility and reliability. The website evaluation was completed during March–April 2021.

Table 2

Adapted CRAAP Test With Scoring Rubric; Maximum Total 30 Points Across Five Categories

Currency (4 points maximum)	0	1	2	3
Date created, updated, or reviewed	No year	>3 years	1–3 years	<1 year
Embedded links functional^a	No	Yes
Relevance (4 points maximum)	0	1
Information appropriately related to topic or question	No	Yes
Intended audience	Not relevant	Healthcare provider^b
Information appropriate level for nurses	No	Yes
Comfortable citing source in an academic publication	No	Yes
Authority (7 points maximum)	0	1	2
Identity of author/source	None identified	Group connected to topic	Authoritative resource
Authors’ credentials	None identified	Group connected to topic	Authoritative resource
Author (or organization) qualified to write on topic	No	Yes
Contact information of author provided	No	Yes
URL identifies relevance of source—Does the URL reveal anything about the author or source? (i.e., edu; .gov; .org; .net)	No	Yes
Accuracy (7 points maximum)	0	1	2
Source of information	Unclear	Medical professional or personal experience	Organizational authority
Information supported by evidence	No	Yes
Information has been reviewed or refereed (peer or committee review)	No	Yes
Information verifiable via another source (including personal knowledge)	No	Yes
Information presented with unbiased, unemotional tone	No	Yes
Grammar errors and/or typos	Yes	No
Purpose (8 points maximum)	0	1	2	3
Purpose of information	Sell something	Persuade or entertain	Inform	Teach
Intentions or purpose is clear	No	Yes
Nature of information	Propaganda	Opinion	Facts
Point of view appears objective and impartial	No	Yes
Political, ideological, cultural, or religious biases	Yes	No

Note. CRAAP = Currency, Relevance, Authority, Accuracy, and Purpose Test; URL = uniform resource locator.

All embedded links that were tested, no less than three.

Can also be patient/family, but content must be usable for healthcare provider.

Table 3

Modifications Made to Reference CRAAP Test With Scoring Rubric^a

CRAAP category	Original measurement and scoring	Modification	Rationale
Currency	Date created or information updated	Added a score of 0 if no year visible on website for created or updated	Critical information for health website
	Date created or information updated	Modified to: information updated within 3 years	Reflects the rapid changes in pediatric oncology science
	Embedded links functional	Not revised
Relevance	Information appropriately related to topic or question	Not revised
	Intended audience	Removed patient/family	To meet selection criteria of study
	Information appropriate level for needs	Modified to: information appropriate level for nurses (medical terminology used as appropriate)	To meet selection criteria of study and clarify scoring
	Comfortable citing source	Modified to: comfortable citing source in academic publication	Make level of citation explicit
Authority	Identity of author/source: 1 = patient	Removed option of patient	Target audience for study was nurses
	Author's credentials: 1 = patient	Removed option of patient	Target audience for study was nurses
	Author (or organization) qualified to write on topic	Not revised
	Contact information of author provided	Not revised
	URL identifies something relevant of source—Does the URL reveal anything about the author or source? examples:.com .edu .gov .org .net
Accuracy	Derivation (source) of information: 1 = personal experience 2 = professional experience	Combined personal and professional experience	Source of information for reviewed websites are difficult to ascertain if from medical professional practice or personal experience since all websites from patient or survivor blogs are excluded from the review
	Derivation (source) of information: 3 = evidence-based review	Changed to: organizational authority	Many websites in pediatric oncology are from well-known societies and associations, or hospitals, or other organizational authorities
	Information supported by evidence	Not revised
	Information has been reviewed or refereed (peer or committee review)	Not revised
	Information verifiable via another source	Added: including personal knowledge of the expert nurse scoring the website	Reviewers must have some level of nursing education expertise
	Information presented with unbiased, unemotional tone	Not revised
	Grammar errors and typos	Not revised
	Purpose of information	Not revised
Purpose	Intentions or purpose is clear	Not revised
	Nature of information (fact, opinion, or propaganda)	Not revised
	Point of view appears objective and impartial	Not revised
	Political, ideological, cultural, religious biases	Not revised

Note. CRAAP = Currency, Relevance, Authority, Accuracy, and Purpose Test; URL = uniform resource locator.

Garcia e al. (2018).

Data Management and Analysis

All websites meeting inclusion criteria were scored by the three researchers independently using the modified CRAAP rubric. Scores for subitems for each domain were added to calculate an overall CRAAP score for each website. The three researchers agreed on a three-tier scoring ladder to categorize the websites according to the level of author recommendation: highly recommended for teaching clinical nurses at hospitals or clinics (CRAAP score ≥25), recommended (CRAAP score 20–24) for informational purposes or as a reference, and not recommended (CRAAP score ≤19), meaning the site had inaccurate information or had broken links and would not be appropriate for use as a nursing educational or informational reference. If only two of the three researchers scored a website in the same (recommendation) ladder category, the website was then revisited and reconsidered by all three researchers in discussion of individual written scoring comments until consensus was achieved for the appropriate recommendation category. The final CRAAP score for each website was obtained by averaging the scores from each of the three researchers.

Websites were also evaluated for explicit presence of pediatric content (i.e., use of terms “pediatric,” “child,” or “adolescent”), reading level, and language(s) available on the website. The reading level was determined using the WebFX (n.d.) online tool used to calculate the Flesch Kincaid Grade Level, based on the average US school grade level necessary to comprehend the website content.

Results

Websites Included for Review

Eighty-six unique websites were initially identified during the strategic search process; 25 were excluded after applying exclusion criteria. An additional 10 websites were further excluded because they had broken links or required a paid subscription or were duplicates of websites retained for review. A total of 51 websites were included in this evaluation.

The majority (n = 39; 76%) of websites had explicit presence of pediatric oncology content, while the rest had content relevant to oncology nursing in general. Less than half (n = 25; 49%) of the websites offered information in language/s other than English (see Tables 4 –6). The reading level of evaluated websites ranged from grade level 2 to 16, with mode at grade level 9 and median of grade level 9.7.

Table 4

Highly Recommended Websites for Use in Clinical Nurse Education in Low- and Middle-Income Countries (CRAAP Score ≥25)

Website	URL	Language/s	Reading level (Grade US)	Pediatric content
Pediatric hematology/oncology
American Cancer Society: Information for Health Care Professionals	https://www.cancer.org/health-care-professionals.html	English, Chinese, French, Haitian, Creole, Hindi, Korean, Polish, Portuguese, Russian, Spanish, Tagalog, Ukrainian, Vietnamese	9	No
British Columbia (BC) Cancer	http://www.bccancer.bc.ca/health-professionals	English	11	No
Cancer Research Institute	https://www.cancerresearch.org/	English and Spanish	10	Yes
Cancer Research United Kingdom (UK)	https://www.cancerresearchuk.org/	English	7	No
Center for Disease Control & Prevention (CDC): Cancer Resources for Health Care Providers	https://www.cdc.gov/cancer/health-care-providers/	English and Spanish	9	No
Children's Hospital of Philadelphia (CHOP) Open Pediatric Education Network	https://open.chop.edu	English	13	Yes
Children's Oncology Group	https://childrensoncologygroup.org/	English	13	Yes
Cure4Kids	www.cure4kids.org	English, Spanish, Chinese, Portuguese, Russian, Arabic	11
Merck Manual (Professional Version)	https://www.merckmanuals.com/professional/pediatr ics/pediatric-cancers	English, Spanish, German, French, Italian, Japanese, Portuguese, Russian, Chinese, Vietnamese	13	Yes
National Cancer Institute (NCI): PDQ® Comprehensive Database	https://www.cancer.gov/publications/pdq	English, Spanish, Chinese, Japanese, Arabic	11	Yes
OncoLink for Healthcare Professionals	https://www.oncolink.org/healthcare-professionals	English	12	Yes
World Health Organization (WHO)	https://www.who.int/news-room/factsheets/detail/cancer-in-children	English, Chinese, Arabic, French, Russian, Spanish	4	Yes
Psychosocial
Mattie Miracle Foundation: Psychosocial Standards	https://www.mattiemiracle.com	English	4	Yes
Pediatric palliative care
American Society of Clinical Oncology (ASCO)	https://www.cancer.net/navigating-cancercare/advanced-cancer/caring-terminally-ill-child	English (some downloads available in Spanish)	8	No
African Palliative Care Association	https://www.africanpalliativecare.org/resources-centre/	English, French, Portuguese	13	No
Courageous Parents Network	https://courageousparentsnetwork.org/	English and Spanish	11	Yes
Dougy Center	https://www.dougy.org/	English and Google Translate widget	8	Yes
International Children's Palliative Care Network	http://www.icpcn.org/	English and Google Translate widget	8	Yes
National Alliance for Grieving Children	https://childrengrieve.org/resources	English	8	Yes
National Institutes of Health (NIH): Clinical Center: Pain and Palliative Care	https://www.cc.nih.gov/palliativecare/resources.html	English and Spanish	10	Yes
National Hospice and Palliative Care Organization	https://www.nhpco.org/palliative-careoverview/pediatric-palliative-and-hospice-care/	English and Spanish	11	Yes
Paediatric Palliative Care Australia	https://palliativecare.org.au/children	English, Arabic, Vietnamese, Chinese	11	Yes
Adolescent and young adult cancer
LiveStrong	https://www.livestrong.org/we-can-help/young-adults	English	7	Yes
Patient and caregiver support
American Cancer Society: Cancer in Children	http://www.cancer.org/cancer/cancerinchildren/index	English, Spanish, Arabic, Russian, Haitian Creole, Hindi, Korean, Polish, Portuguese, French, Tagalog, Vietnamese	7	Yes
American Society of Clinical Oncology (ASCO) Cancer.net Patient/Family site	https://www.cancer.net/navigating-cancercare/advanced-cancer/caring-terminally-ill-child	English	11	Yes
Cancer Support Community	https://www.cancersupportcommunity.org/	English	8	No
ChemoExperts	https://www.chemoexperts.com/	English	11	No
Children's Oncology Group: Patients and Families	https://childrensoncologygroup.org/index.php/patients-and-families	English	9	Yes
Children with Cancer United Kingdom (UK)	https://www.childrenwithcancer.org.uk/	English	9
Coalition Against Childhood Cancer	https://cac2.org/interest-groups/awareness/childhood-cancer-fact-library/	English	9	Yes
CureSearch	https://curesearch.org/Cancer-Resources	English	9	Yes
HealthyChildren.org	https://www.healthychildren.org/English/healthissues/conditions/cancer/Pages/Childhood-Cancer.aspx	English, Spanish and audio	10	Yes
Nemours KidsHealth	http://kidshealth.org/parent/_cancer_center/cancer_center.html	English, Spanish and spoken and increased text size possible	8	Yes
Together by St. Jude Children's Research Hospital	https://together.stjude.org/en-us/care-support/palliative-care.html	English, Arabic, Burmese, Chinese, French, Hindi, Russian, Spanish	9	Yes
Pediatric oncology nursing practice
Association for Pediatric Hematology and Oncology Nurses	https://aphon.org/	English	12	Yes
Oncology Nursing Society	https://www.ons.org	English	11	No

Note. CRAAP = Currency, Relevance, Authority, Accuracy, and Purpose Test; URL = uniform resource locator.

Table 5

Recommended Websites for Use in Clinical Nurse Education in Low- and Middle-Income Countries (CRAAP Score 20–24)

Website	URL	Language/s	Reading level grade (US)	Pediatric content
Pediatric hematology/oncology
Leukemia Lymphoma Society	https://www.lls.org/continuing-education	English and Spanish	9	Yes
National Comprehensive Cancer Network (NCCN)	https://www.nccn.org	English, Pediatric Acute Lymphoblastic Leukemia (only) in Indonesian Italian, Japanese, Portuguese, Russian, Spanish	13	Yes
National Human Genome Research Institute	https://www.genome.gov/dna-day/15-ways/cancer-genomics	English and Spanish	10	No
Pediatric Oncology Group of Ontario (POGO)	https://www.pogo.ca/	English	11	Yes
Psychosocial
American Childhood Cancer Organization	https://www.acco.org/	English	9	Yes
Pediatric palliative care
Canadian Hospice Palliative Care Association	https://www.chpca.ca/	English and French	9	Yes
Children's Cancer and Leukemia Group	https://www.cclg.org.uk/CSOIR/Introduction-to-palliative-care-in-children-and-young-people-with-cancer	English	9	Yes
Adolescent and young adult cancer
Planet Cancer	https://stupidcancer.org/eventstrashed/planet-cancer/	English	12	No
UK Teenage Cancer Trust	https://www.teenagecancertrust.org/	English	8	Yes
Pediatric oncology nursing practice
Royal College of Nursing	https://www.rcn.org.uk/	English	8	Yes
Atlantic Provinces Pediatric Hematology Oncology Network (APPHON)	https://www.apphon-rohppa.com/en/teaching-education	English and French	15	Yes
International Society of Nurses in Cancer Care	https://www.isncc.org/page/InsightResources	For pediatric-related resources: English, Polish, Czech, Chinese, Spanish, Portuguese	Varied	Yes

Note. CRAAP = Currency, Relevance, Authority, Accuracy, and Purpose Test; URL = uniform resource locator.

Table 6

Websites Not Recommended for Use in Clinical Nurse Education in Low- and Middle-Income Countries (CRAAP Score ≤19)

Website	URL	Language/s	Reading level grade (US)	Pediatric content
Pediatric hematology/oncology
Paediatric Oncology International Network for Training and Education	https://cancerpointe.com/	English	9	Yes
Psychosocial
Hospital Educator and Academic Liaison Association	https://www.healassociation.org/about/	English	10	No
Adolescent and young adult cancer
Australia Canteen	https://www.canteen.org.au/	English	8	Yes

Note. CRAAP = Currency, Relevance, Authority, Accuracy, and Purpose Test; URL = uniform resource locator.

CRAAP Scores

CRAAP scores ranged from 14 to 30. Well over half (n = 36; 71%) of the websites scored ≥25 and were considered highly recommended (Table 4). Less than one quarter (n = 12; 24%) were recommended (score 20–24) (Table 5), and only three (6%) scored ≤19 and were considered not recommended for use in creating pediatric oncology nursing education materials (Table 6).

The websites within the following assigned topics had the highest average CRAAP scores: Patient and Caregiver Support, Pediatric Palliative Care, and Pediatric Hematology/Oncology (i.e., general disease and symptom management information). The lowest scoring topics were: Adolescent and Young Adult, and Psychosocial.

Because the search strategy excluded nonrelevant and nonpurposeful websites, the relevance and purpose domains, though important, tended to score highly across websites. The domains with the most variance in CRAAP scores were currency, authority, and accuracy. Confidence in website quality was also diminished if the date of review or update was greater than 3 years prior. If a website was under review, it was scored as “review >3 years old.” Twenty-four percent of websites (n = 12) had not been updated within the past 3 years and/or had broken links. Limitations in accuracy included lack of clarity on the source of information and whether it had been reviewed or refereed (n = 13 websites; 25%). Six websites (12%) did not identify the author with credentials, qualifications, or contact information. Highly scored websites consistently identified authors with credentials and affiliation or contact information (authority), along with a publication or review date (currency). Contact information for authors was deemed as noncritical. Websites received higher scores if organizational or author/s contact information was identified, or the author was clearly identified with credentials and could thus be easily located by an Internet search and contacted such as for permission to use proprietary website content to avoid copyright infringement. Additionally, websites missing the author's name, credentials, and contact information were difficult to consider authoritative as were those without references, a specified review process, or reviewer qualifications. To increase confidence and allow for recommendation of websites, this information should be added to individual pages or an “About” section on websites. Almost all the sites reviewed were impartial and presented information free of bias, emotion, or grammatical errors.

Scoring comments by the researchers are included in Supplemental Table 1 to provide future users with additional detailed information, such as if a website was rated highly but included a limited breadth of content. Both positive (e.g., “website includes excellent videos for teaching”) and constructive comments (e.g., “website owner is unclear”) were recorded.

Discussion

In settings with limited resources, subspecialty and specialty nursing educators and clinical nurses must rely on professional teaching materials that are easy to access, free, accurate, and up-to-date. Many settings are unable to acquire oncology nursing textbooks due to a lack of local availability, high cost, and limited use due to rapid changes in cancer care. Donated texts that are current are often stored in a place that is inaccessible to on-duty nurses due to their high value and risk of theft. Thus, the Internet serves as a readily available and free source of nursing information, however, websites are self-regulated and rarely peer-reviewed. Guidance on website suitability for use by nurse educators in LMICs to teach other staff nurses and for self-directed learning of staff nurses has been unavailable to date. Our findings benefit nurses seeking appropriate, accurate, and timely information to create teaching materials for clinical nurses. The existence of unreferenced, outdated, author-unidentified, or not fit-to-purpose websites on childhood cancer reveals the need for expert evaluation and a robust, freely accessible, curated online database of resources for nurses working with children with cancer, especially in LMICs. Nearly all of the included websites reviewed were judged to be highly recommended or recommended, although there was significant redundancy in topics such as disease descriptions and treatment side effects.

Important Attributes

The breadth and depth of available online resources revealed through the systematic search were beyond the initial expectations of the authors, who have experience searching for pediatric oncology clinical nursing and for use in family education. Key elements were often missing from many of the websites including the date of publication and/or review (currency); a list of references (supported by evidence); whether information had been reviewed or refereed; and whether the author was identified with credentials (this could include an author, review board, scientific committee, medical director/advisor, etc.). Attention to these attributes could enhance the overall value of these websites to support their overall utility for pediatric oncology nursing education.

Several criteria that could be easily addressed were consistently identified as barriers to websites earning full points for credibility. These included the exclusion of the date when the website was created and updated. Because pediatric oncology is a rapidly changing field, regular review and updating of content is essential to maintain currency and accuracy due to advancements in diagnostics, disease and treatment management protocols, technology, and medications. Clearly identifying the author team has the potential to enhance credibility as well. Ensuring all hyperlinks are active, that websites easily allow users to find the information they are seeking and adhere to standard guidelines for web design have all been identified as important to website usability (Knijnenburg et al., 2009).

Utility for Nurses Caring for Children With Cancer in LMICs

Despite great strides being made in various sectors involved in childhood cancer care to reach the goal of the WHO GICC, nursing has continued to struggle to keep up the momentum because of challenges such as the global shortage of human resources for health (World Health Organization, 2016). In LMICs such as in the Philippines, where the childhood cancer burden is high, nurses taking care of children with cancer are usually general nurses, with little to no specific training on childhood cancer care. Though some children with cancer are admitted to university, teaching, or tertiary hospitals and cancer centers in metropolitan areas, many receive care in hospitals in rural areas where nurses have limited to no access to childhood cancer publications, periodicals, subscriptions, or specialized training (Ting et al., 2020). Interactive digital platforms used for specialized training for pediatric oncology physicians are not as easily available to nurses, who must rely on more static Internet resources (Martiniuk et al., 2022). Therefore, health information is usually accessed through crude Google searches from mobile devices. The results from this review are valuable to help nurses in LMICs such as the Philippines to be able to identify which of these websites are highly recommended, and the most credible, reliable, accurate, author-verified, and fit-for-purpose.

A recent study in multiple LMICs noted that many key topics were not included in materials used for nursing onboarding and continuing education programs (Morrissey et al., 2019). Categorizing websites under common childhood cancer topics is helpful to ease the search for specific childhood cancer concerns. Identifying which websites are available in multiple languages will also help nurses whose primary language is not English. Reading level is helpful especially to those adapting the content for educating their patients and their families, as well as those who do not speak or read English as a primary language. Previous studies that have evaluated pediatric oncology websites have also found reading levels of 10th to 11th grade (US) overall (Ruble et al., 2019; Stinson et al., 2011), which is higher than the standard 6–8th grade (US) recommendation (Stossel et al., 2012). These results complement our findings that most websites were at a 9–10th-grade level (US).

Because websites are frequently changed or updated and new resources are continuously being created, any database reviewing these resources must itself be up to date as well. Nurses in LMICs should be cautious about using health information found on websites whose sources cannot be verified as authoritative. The curated list of recommended websites we have created allows nurses to access accurate and vetted information. Recommendations are unbiased, as scores are assigned according to the modified CRAAP rubric.

By creating a resource of recommended pediatric oncology websites and sharing the modified CRAAP scoring tool, along with information on reading level and available languages, we have provided a curated database for nurses and other professionals to create educational materials for nurses working in pediatric oncology. The next steps for this work include creating a database with wiki-functionality for additional pediatric oncology websites to be identified and proposed by users, who can then provide feedback on scoring.

Evaluation Limitations

When reviewing websites, it was often difficult to define the purpose (e.g., inform vs. teach) and reviewer scoring may have been inconsistent depending on the purpose of the specific page of the website that was evaluated. Some websites had quite a narrow scope, such that although they scored high, they had very limited information on childhood cancer overall which may be of limited utility for creating educational materials at a broader scope. As authors may have had organizational involvement in various websites, subjective ratings and biases may have been introduced. We sought to minimize the possibility of bias, however, by ensuring all websites were scored by three separate nurses and that consensus was achieved. Additionally, due to the rapidly changing Internet landscape, key childhood cancer websites may have been missed in the search for this study, particularly since many websites are in non-English languages and difficult to access in English searches. Finally, although the review of online resources behind a paywall, such as research databases requiring subscriptions, may be useful for nurses in academic settings, these were excluded from this review as they are not widely accessible to nurses in LMICs.

Evaluation Strengths

This evaluation offers a curated list of a large collection of Internet resources in English (with approximately half of the resources also available in multiple other languages) for nurses to create teaching materials for other nurses, or for families of children and adolescents with cancer. The expertise of the three authors allowed for deliberate scoring and determination of the highest-quality content. The results of this study will allow pediatric oncology clinical nurses in various settings with limited access to subscription-based resources to focus on the Internet resources most likely to offer trusted and accurate content, while avoiding those that are less appropriate or of dubious origin. The methodology used in this study could be helpful for nurses who speak languages other than English to conduct a similar evaluation on childhood cancer websites in their native languages.

Recommendations

Following this review, an online database of recommended websites that are freely available will be created and maintained on the International Society of Paediatric Oncology (SIOP) website for use by nurses seeking information for education regarding the care of children with cancer, especially in limited-resource settings. This resource aligns with the SIOP Baseline Standard for Nurses in LMIC Standard 2:

All new nursing employees receive a formalized paediatric oncology induction (orientation) programme to include two weeks of theory and clinical skills training followed by 3–4 weeks spent with an experienced nurse preceptor. The induction programme should include specific learning objectives. Evidence of successful completion should be obtained before new nurses provide direct patient care. Content should include: review of paediatric cancers, administration of chemotherapy and management of side effects, management of venous access (central and peripheral), control and prevention of infections, administration of blood products, management of neutropenic sepsis, early detection and treatment of oncology emergencies, assessment and management of pain, nutritional support, education for patients and families, palliative care including death and dying and spiritual and psychological issues.

This resource also aligns with the SIOP Baseline Standard for Nurses in LMIC Standard 3:

Nurses receive continuing education and training to maintain and increase their paediatric oncology clinical skills and knowledge. A minimum of 10 h continuing education and training per year is recommended. (Day et al., 2015, p. 113)

To improve access, utility and credibility, pediatric oncology websites must demonstrate currency, relevance, authority, accuracy, and purpose. Nurses should be consulted on health information website development from inception and through the development process if the intention is to disseminate accurate information on childhood cancer. Collaboration with nurses will ensure websites are fit-for-purpose and provide a credible resource that can also be used for nursing training and patient/family education. Pediatric oncology patients, survivors, and families should also be involved in development and review of childhood cancer websites. Acknowledging the role of nurses, patients, and families in website development including website design and user experience will improve trust in the website content and accessibility.

Childhood cancer websites should consider that clinical nurses will use the content to create teaching materials for nurses and families, particularly in rural or limited-resource settings, and ensure that authority, currency, and applicability are addressed. Nurses have a long reach in both nursing and patient and family education, as well as community awareness raising and advocacy.

To assist pediatric oncology clinical nurses worldwide in accessing reliable information online, a centralized online list, curated by expert nurses within the international community, must be maintained and reviewed at least annually to remain current. Continual evaluation of websites with a standardized tool such as the CRAAP Test will help avoid bias. This evaluation should be conducted by at least three nurse experts (including at least one nurse currently from an LMIC) to maintain a minimal level of reliability and objectivity. Validation of the modified CRAAP scoring rubric in the future would aid in ensuring that websites are accurately and reliably evaluated.

The National Academy of Medicine in the US convened an independent advisory group to consider social media sources of health information, including websites (Kington et al., 2021). They noted that the public rarely depends on a single Internet source of health information, and routinely visits multiple sites. Government websites are generally vetted before public access which provides some assumed credibility. However, the authors recommend that a thorough review for quality assurance of any health information website should be conducted prior to using the content for public consumption on social media. This can be extrapolated for our study purpose, indicating that a quality review must be done before websites are used for teaching.

Finally, attention must be given to missing components of childhood cancer websites including cancer genetics and counseling, and social determinants of health and their impact on cancer treatment and survival. In addition, website content should be written at a 6–7th US grade level to account for health literacy levels of the general public as well as nonnative- or second-language English-speaking nurses across LMICs.

Conclusion

We have described the landscape of available pediatric oncology-related websites for use in developing nursing educational materials, especially in LMICs. Following elimination of websites that were not fit-to-purpose, the majority of the remaining websites were found to be highly recommended. It is reassuring that robust information about childhood cancer is freely available worldwide in multiple languages. Although most of the websites in this study were at a reading level higher than recommended for general health literacy, this was not necessarily too high for clinical nurses used to medical terminology in English. We recognize that many websites are not stable (URL may no longer be valid, information is too old [e.g., >5 years], or information may require registration or strong bandwidth unavailable in many LMICs). There is much redundancy across websites, which is to be expected given that many sites offer diagnosis information, descriptions of pediatric chemotherapy, and similar psychosocial support content. However, we noted a dearth of information regarding cancer genetics related to pediatric cancers, genetic counseling, and international and national overviews of cancer disparities (e.g., ethnic, socioeconomic, and geographic differences in incidence, survival, and psychosocial needs). Nurses were rarely mentioned or identified as authors or reviewers; in fact, these persons were often not noted at all. Nurses should participate more in website development and have visible recognition of their contributions.

There is much information on pediatric oncology available on the Internet for clinical nurses preparing educational materials for other nurses. The CRAAP Test provides an easy and brief method for evaluating a website prior to using information. The highly recommended websites are significant resources for clinical nurses seeking robust information to use for teaching other nurses and those engaged in self-directed learning.

Supplemental Material

sj-docx-1-jpo-10.1177_27527530231190370 - Supplemental material for An Expert Evaluation of Oncology Website Resources for Use in Pediatric Oncology Clinical Nursing Education in Low-Resource Settings

Supplemental material, sj-docx-1-jpo-10.1177_27527530231190370 for An Expert Evaluation of Oncology Website Resources for Use in Pediatric Oncology Clinical Nursing Education in Low-Resource Settings by Aprille C. Banayat, Julia Challinor and Elizabeth Sniderman in Journal of Pediatric Hematology/Oncology Nursing

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Julia Challinor

Supplemental Material

Supplemental material for this article is available online.

References

Banayat

A. C.

Dychangco

M. E. A.

Manahan

L. T.

Howard

S. C.

Sullivan

C. E.

(2021). Philippines Cancer control plan: Nursing priorities from nursing service delivery working group of the Philippine National Childhood Cancer Control Workshop. Cancer Care Research Online, 1(2), Article e0008. https://doi.org/10.1097/CR9.0000000000000008

Berland

Capone

Etcher

Ewing

Keating

Chickering

(2020). Open education resources to support the WHO nurse educator core competencies. International Nursing Review, 67(2), 282–287. https://doi.org/10.1111/inr.12583

Campbell

Taylor

Douglas

(2019). Effectiveness of online cancer education for nurses and allied health professionals: A systematic review using Kirkpatrick Evaluation Framework. Journal of Cancer Education, 34(2), 339–356. https://doi.org/10.1007/s13187-017-1308-2

Challinor

J. M.

(2022). Addressing the global pediatric oncology nursing workforce gap: An overdue imperative. Asia-Pacific Journal of Oncology Nursing, 9(2), 77–78. https://doi.org/10.1016/j.apjon.2022.01.006

Challinor

J. M.

Alqudimat

M. R.

Teixeira

T. O. A.

Oldenmenger

W. H.

(2020). Oncology nursing workforce: Challenges, solutions, and future strategies. The Lancet Oncology, 21(12), e564–e574. https://doi.org/10.1016/s1470-2045(20)30605-7

Day

Challinor

Hollis

Abramovitz

Hanaratri

Punjwani

(2015). Paediatric oncology nursing care in low- and middle-income countries: A need for baseline standards. Cancer Control, 111–116. http://www.cancercontrol.info/cc2015/paediatric-oncology-nursing-care-in-low-and-middle-income-countries-a-need-for-baseline-standards/

Diaw

Sibeoni

Manolios

Gouacide

J. M.

Brami

Verneuil

Revah-Levy

(2020). The lived experience of work-related issues among oncology nurses: A metasynthesis. Cancer Nursing, 43(3), 200–221. https://doi.org/10.1097/NCC.0000000000000774

Eniu

Peccatori

F. A.

Pavilidis

Siano

Fessi

(2019). Challenging cancer inequities within Europe through education and training: ESO’s vision. Cancer Control, 75–80. http://www.cancercontrol.info/wp-content/uploads/2019/07/Dr-Alexandru-Eniu.pdf

Fast

A. M.

Deibert

C. M.

Hruby

G. W.

Glassberg

K. I.

(2012). Evaluating the quality of Internet Health Resources in Pediatric Urology. Journal of Pediatric Urology, 9(2), 151–156. https://doi.org/10.1016/j.jpurol.2012.01.004

10.

Galassi

Pradhan

Palat

LeBaron

V. T.

(2017). Global oncology volunteering: Making an effective contribution. Oncology Nursing Forum, 44(5), 530–533. https://doi.org/10.1188/17.ONF.530-533

11.

Garcia

Daugherty

Ben Khallouq

Maugans

(2018). Critical assessment of pediatric neurosurgery patient/parent educational information obtained via the internet. Journal of Neurosurgery: Pediatrics, 21(5), 535–541. https://doi.org/ 0.3171/2017.10.peds17177

12.

Kent State University. (2018). Interpersonal and communication skills for healthcare professionals: Criteria for evaluating web resources . https://libguides.library.kent.edu/c.php?g=447762&p=3055365

13.

Kington

R. S.

Arnesen

Chou

W-Y. S.

Curry

S. J.

Lazer

Villarruel

A. M.

(2021). Identifying credible sources of health information in social media-principles and attributes. NAM Perspectives. Discussion Paper, National Academy of Medicine, Washington, DC. https://doi.org/10.31478/202107a

14.

Knijnenburg, S. L., Kremer, L. C., Versluys, A. B., Beek, J. R., & Jaspers, M. W. (2009). Development and evaluation of a patient information website for childhood cancer survivors. Studies in health technology and informatics, 150, 342–346.

15.

Martiniuk

Challinor

Arora

R. S.

Handayani

S. A.

Lam

(2022). Digital solutions to facilitate education, training and professional support for paediatric oncology and other health professionals in rural and other low-resource settings. Health Education, 122(2), 232–259. https://doi.org/10.1108/HE-02-2021-0035

16.

Meriam Library, California State University, Chico. (2020, September 2). Literature reviews. https://libguides.csuchico.edu/c.php?g=414315&p=2822716

17.

Morrissey

Lurvey

Sullivan

Challinor

Forbes

Abramovitz

Afungchwi

Hollis

Day

(2019). Disparities in the delivery of pediatric oncology nursing care by country income classification: International survey results. Pediatric Blood & Cancer, 66(6), Article e27663. https://doi.org/10.1002/pbc.27663

18.

National League for Nursing. (2020). Hallmarks of excellence . https://www.nln.org/education/teaching-resources/professional-development-programsteaching-resources/hallmarks-of-excellence-%C2%A9-dffbb05c-7836-6c70-9642-ff00005f0421

19.

Ruble

K. J.

Paré-Blagoev

E. J.

Cooper

S. L.

Jacobson

L. A.

(2019). Assessment of online resources for returning to school during and after treatment of childhood cancer. Journal of Cancer Education, 35(5), 876–884. https://doi.org/10.1007/s13187-019-01537-y

20.

Simonyan

(2017). Lived experiences of pediatric oncology nurses in Armenia: A qualitative study [Masters thesis]. American University of Armenia. https://sph.aua.am/files/2017/07/Tamara-Simonyan_2017.pdf

21.

W. K. W.

Cummings

G. G.

de Calvo

L. E.

Day

S. W.

Houlahan

Nevidjon

B. M.

Schneider

Teahon

Were

P. A.

(2016). Enhancement of oncology nursing education in low-and middle-income countries: Challenges and strategies. Journal of Cancer Policy, 8, 10–16. https://doi.org/10.1016/j.jcpo.2016.03.002

22.

Stinson

J. N.

White

Breakey

Chong

A. L.

Mak

Low

K. K.

Low

A. K.

(2011). Perspectives on quality and content of information on the internet for adolescents with cancer. Pediatric Blood & Cancer, 57(1), 97–104. https://doi.org/10.1002/pbc.23068

23.

Stossel

L. M.

Segar

Gliatto

Fallar

Karani

(2012). Readability of patient education materials available at the point of care. Journal of General Internal Medicine, 27(9), 1165–1170. https://doi.org/10.1007/s11606-012-2046-0

24.

Ting

Benedict Sacdalan

Sarita Abarquez

Uson

(2020). Treatment of cancer patients during the COVID-19 pandemic in the Philippines. Ecancermedicalscience, 14(14), 1040. https://doi.org/10.3332/ecancer.2020.1040

25.

WebFX. (n.d.). Readability test. https://www.webfx.com/tools/read-able/

26.

Wilson

FR.

(2019). An ethnographic study that explores the policy and cultural influences on the continuing professional development of nurses and their utilization of computer technology in a community hospital in Uganda [Doctoral dissertation]. University of Chester. https://chesterrep.openrepository.com/handle/10034/622130

27.

World Health Organization. (2016). Health workforce requirements for universal health coverage and the sustainable development goals. (Human Resources for Health Observer, 17). https://apps.who.int/iris/handle/10665/250330

28.

World Health Organization. (2022). Global initiative for childhood cancer . https://www.who.int/publications/m/item/global-initiative-for-childhood-cancer

29.

Yang

Z. H.

Zhang

L. L.

(2013). 我国肿瘤专科护士培训现状与展望 [The current situation and prospect of oncology nurse training in China]. Chinese Nursing Management, 13(12), 55–57. https://doi.org/10.3969/j.issn.1672-1756.2013.012.018

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB