Abstract
SchoolhealthLink, a Web-based information service for Missouri school nurses, is a promising resource to reach school nurses isolated from traditional professional networks. It also may serve as an important adjunct to stay abreast of the latest health information. Using a strategy to identify the health information needs of school nurses and to test the site early in its development, this pilot study found school nurses reported SchoolhealthLink was an easy-to-use method to find high-quality, up-to-date information relevant to school nursing practice, was better than existing methods to access information, and could change the way they practiced. In order to continue to build SchoolhealthLink, a partnership with Barnes College of Nursing and the Missouri Association of School Nurses has been established to work together to continue to add health information resources useful to school nursing practice.
INTRODUCTION
The World Wide Web (WWW), with its vast array of health information resources, holds great promise for reaching school nurses often isolated from traditional professional networks and is likely to serve as a resource for school nurses. School nurses are facing increasing demands to deliver more complex services to children in their schools. Web-based knowledge may fill their information needs, although the quality and relevance of resources available vary widely. To serve the needs of school nurses, the authors built SchoolhealthLink, a multipurpose WWW site that links school nurses to existing Web-based disease-specific information and serves as a filter to child health knowledge. The authors capitalized on a vast computer infrastructure in Missouri where over 97% of public schools report connections to the Internet and 92% of classrooms are wired for the Internet (Missouri Department of Elementary and Secondary Education, 2002). Missouri schools are also rich in professional nursing resources with over 90% of schools having school nursing services.
The basic design strategy for SchoolhealthLink was Brennan’s (1995) ComputerLink model, which involves intended users in a needs assessment to address the unique needs of school nurses, followed by usability testing. Results of the needs assessment to determine school nurses’ perceptions of beneficial health information resources and the priority for making these resources available on SchoolhealthLink have been reported (Bachman, Brennan, Patrick, & Cole, 2000). In addition to a needs assessment, Brennan and Gustafson et al. (1993) stress the importance of an evaluation strategy that includes pilot testing of the computer system early in its development to determine usability and to assure quality and relevance of the resources. SchoolhealthLink is intended to assist school nurses to quickly locate relevant high-quality health information resources useful for their practice.
LITERATURE REVIEW
There are a number of health information support systems demonstrating applications of computer networks that tap into the vast array of health information available on the WWW. These systems are designed for patients with a health crisis or medical concern and for primary care providers. Patient-centered health information systems include models that use a variety of strategies to filter health information from the WWW to provide focused and tailored health information for concerns such as HIV/AIDS, asthma, smoking cessation, living with alcohol abuse, or stress (Brennan, Caldwell, Moore, Sreenath, & Jones, 1998; Gustafson et al., 1999). Physician-centered health information support systems use a variety of strategies to provide health information filtered from the WWW for physicians and patients/families on topics such as child health, safety, and the management of insulin-dependent diabetes (Porcelli & Lobach, 1999; Riva, Bellazzi, & Stefanelli, 1997).
The unprecedented growth of health information resources demands that valid instruments be developed to evaluate Web-based health information systems.
There are no reports of health information systems that (a) take into account the needs of school nurses derived from a needs assessment, (b) provide health information resources specifically focused for school nursing practice, or (c) have developed an evaluation strategy to determine if the health information system meets the needs of school nurses. Although the informatics community has developed a wide variety of health information systems, long-term evaluation of these systems is lacking (Gell, 1999). The unprecedented growth of health information resources demands that valid instruments be developed to evaluate Web-based health information systems (Jadad & Gagliardi, 1998; Jones, 1999; Kim, Eng, Deering, & Maxfield, 1999; Robinson, Patrick, Eng, & Gustafson, 1998). Although a wide variety of criteria are currently being used (Kim et al.), no online instrument has been developed to determine the usefulness of the health information resources available on the WWW for school nurses.
The goals of system design were to develop (a) materials that match the identified needs of Missouri school nurses to health information resources; (b) a high-quality, easy to use health information system; and (c) online evaluation instruments. The purpose of this pilot test of SchoolhealthLink was to (a) determine the usefulness, quality, and relevance to school nurses of information resources related to diabetes and lice; (b) determine the usefulness, quality, and relevance of SchoolhealthLink; and (c) determine whether to retain special features that encouraged user participation.
THEORETICAL FRAMEWORK
Rogers’ (1995) theory of the diffusion of innovations guided the development of SchoolhealthLink. This theory proposes that to be successful, technology innovations must be (a) perceived as better than existing methods; (b) compatible with existing values, past experiences, and needs; (c) easy to use and understand; (d) easily accessible for trial and error; and (e) have results that are readily observed. The challenge for the authors was to design SchoolhealthLink so that (a) school nurses will find it equal to or better than methods they currently use to find health information, (b) health information provided is relevant and will meet their needs, (c) school nurses will be able to find health information quickly and easily, (d) it will be available for testing by school nurses, and (e) the results of using SchoolhealthLink to access health information resources will be apparent to school nurses. Thus Rogers’ theory provides the framework that may result in adoption of the use of SchoolhealthLink as a new health information resource for school nurses.
METHODS
System Design
SchoolhealthLink includes six sections: (a) “Home” welcomes school nurses, explains the purpose of SchoolhealthLink, and provides a disclaimer; (b) “About Us” identifies the SchoolhealthLink team and provides links to team members; (c) “A to Z Index” permits school nurses to initiate searches for health information resources using keywords listed in the A to Z index page; (d) “Evaluate Site” gives school nurses an opportunity to evaluate SchoolhealthLink; (e) “Help Build Site” asks school nurses to recommend quality WWW sites with health information useful to school nursing; and (f) “Contributors” thanks those who contributed health information resources to SchoolhealthLink (http://schoolhealthlink.umsl.edu).
Material Development
Based on a prior survey of school nurses (Bachman et. al, 2000), information resource development focused on two priority health topics, diabetes and lice, for the pilot test of SchoolhealthLink. A filtered database was constructed. This was accomplished by filtering a vast array of information resources available to manageable sets of quality resources relevant to school nursing practice. Instead of a large number of links with a myriad of information, the filtering process attempts to provide links to the best sites with information specific to school nursing practice.
To accomplish this, the authors capitalized on the emergence of Metadata standards to build the index of Web resources (Dempsy & Heery, 1998). The Dublin Core (Dempsey & Heery; Dubline Core Metadata Initiative, 2003) is a metadata element set of 15 terms used to describe resources. To help assure the quality and relevance of the resources, seven Dublin core elements were found to be useful to the SchoolhealthLink database and provided the structure for indexing the SchoolhealthLink WWW resources. These elements were
The subject and keyword tag: A term selected from the Medical Subject Headers, a controlled vocabulary developed by the National Library of Medicine (2003) for indexing terms to facilitate the organization of resources into a category, for example lice and diabetes.
Title or name: The title given the resource by the creator, modified by the investigator, so the title would reflect types of health information resources school nurses could expect to find on the Web site.
Uniform Resource Locator (URL): The Web address to establish a link from the Access database to the search.
Author or creator: A person or organization creating the intellectual content used as a screening criteria to assure that the source was reliable. Patterned after Healthfinder (U.S. Department of Health and Human Resources, 2003), sites were chosen primarily from local and state government agencies; national voluntary, nonprofit, and professional organizations serving the public interest; universities, other educational institutions, and libraries; and organizations partnering with government agencies to provide information to the public.
Date: The date the Web site was modified to assure currency of the information.
Language: The content of the resource was evaluated to determine if the language used was suitable for the nurse, parent, child, or a combination.
Description: An expanded description of the content of the resource of a particular link. For example, the frequently asked questions about the lice resources was expanded to more fully describe the resources, such as photos, diagrams, prevention strategies, treatment, or safety tips.
Some resources were chosen because the health information was suitable for school nurses to share with children and parents. Therefore, two reading-level scores were added to the evaluation. The Flesch Reading Ease Score, based on the average number of syllables per word and words per sentence, is a 100-point scale where the higher the score, the easier to understand (aim for score of 60–70). The Flesch-Kincaid Grade Level Score rates text on grade-school level (for example, 8.0 means an 8th-grader can understand the document). After the health information resources were selected, a sample of the text for each site was copied into Microsoft Word to determine the Grade Level Score and Reading Ease Score.
Health Information System
NetObjects Fusion 4.0 software (Website Pros, Inc., Jacksonville, FL) was selected to build the SchoolhealthLink because of ease of use, freeing the developers to focus on the site from a content perspective, rather than a coding perspective. The prototype SchoolhealthLink was housed on the Health Management and Informatics server at the University of Missouri—Columbia. The Internet Information Server 4 (IIS4) was used to determine use patterns of school nurses (user Internet Protocol address, pages they accessed, and date and time of link).
Instrument Development
Two questionnaires were developed and put online to evaluate the site and gain information on user participation. The first questionnaire, “Evaluate Site,” was designed to evaluate the pilot SchoolhealthLink site for usefulness, quality, and relevance of the sites. To gather data on the content validity of the instruments, the questions were based on the works of several researchers (Kim et al., 1999; Jones, 1999; Nielsen, 1993; Rogers, 1995). Based on this review, seven criteria were selected as important to evaluate SchoolhealthLink. They were content, currency of the information, authority of the source, ease of use, links, intended audience, and user support. The instrument consisted of three parts. Part I evaluated diabetes health information resources, Part II evaluated lice health information resources, and Part III evaluated the SchoolhealthLink site.
To encourage user participation, SchoolhealthLink provided “Help Build Site” and “Contributors” pages. Used only during the testing phase, the second questionnaire was placed on the “Contributors” page to determine (a) whether a list of contributors who used “Help Build Site” is important; (b) if so, what information should be published about the contributors; and (c) if the list of contributors might evolve to something more, such as a school nurse support group.
The questionnaires were reviewed by six nursing doctoral students, a nursing faculty member at one university, and by multidisciplinary informatics fellows and faculty at another university to assure clarity, completeness, and appropriateness. Based on the comments from the reviewers and the literature, revisions were made to the questionnaires.
Evaluation of SchoolhealthLink
A formative strategy was used during the test phase to evaluate SchoolhealthLink prior to launching. A formative strategy is ongoing evaluation of a system as it is being developed to optimize the quality of the system and operates to suggest changes to the system to make it more successful (Mohr, 1995). The evaluation plan sought to evaluate the SchoolhealthLink system and the health information resources for school nurses.
The officers of the Missouri Association of School Nurses (MASN) were asked to list 5–10 school nurses in their area who might be willing to test the site. An e-mail message was sent to 25 school nurses, inviting them to view SchoolhealthLink and complete the “Evaluate Site” questionnaire. After viewing the 13 Web information resources related to diabetes and the 4 Web information resources related to lice, the school nurses were asked to respond to 10 “yes” or “no” questions about the usefulness, quality, and relevance of the resources. Open-ended questions encouraged school nurses to make suggestions about the diabetes and lice resources. The school nurses were then asked to check a “yes” box after the title of the link if any of the diabetes or lice links should be eliminated. An open-ended question allowed the school nurses to write reasons the link should be eliminated. They were then asked to respond to eight “yes” or “no” questions about the SchoolhealthLink site, followed by three open-ended questions asking about errors found on the site, other encouraging comments about the site, and the number of years they have been a school nurse.
In addition, an e-mail message was sent to five officers of the MASN inviting them to view the “Help Build Site” and “Contributors” pages on SchoolhealthLink. After viewing these two pages, they were asked to complete the questionnaire found on the “Contributors” page and check “yes” or “no” to the following questions: (a) Is asking visitors to SchoolhealthLink to suggest Web sites that may be useful to school nurses a good idea? (b) Do you think a list of contributors is a good idea? and (c) If you agree that a list of contributors is a good idea, what information should be published about the contributors (Name, e-mail address, state, school district)? Finally, there were two open-ended questions to determine what other types of information about contributors could be included and to allow for comments and suggestions.
RESULTS
Of the 25 school nurses who received e-mail messages, 10 (40%) viewed the SchoolhealthLink site and completed the “Evaluate Site” questionnaire. Years as a school nurse ranged from 3 to 26 years (M = 9.1 years). An additional three school nurses did not complete the online evaluation but sent e-mail messages with positive comments about SchoolhealthLink. Thus 13 (52%) responded to the e-mail request to evaluate SchoolhealthLink. The reading ease score for diabetes and lice health information resources ranged from 22 to 92.1 (M = 50.4, SD = 17.6; aim for the score approximately 60–70). Reading level for diabetes and lice health information resources ranged from grade 2.9 to 12 (M = 10, SD = 2.4).
The responses to the 10 questions evaluating the resources on diabetes and lice are found in Table 1. On the open-ended questions related to diabetes, two school nurses wrote: “Glad you are available!” “The site was excellent. Very comprehensive with additional resources.” On the questions relating to resources about lice, nine of the school nurses indicated that none of the links should be eliminated. One school nurse recommended that one lice link should be eliminated because the site “Does not support a nit-free policy, which most schools have adopted and runs the risk of contradictory information.” Three respondents found spelling errors on SchoolhealthLink (Table 2).
Of the 23 different days visitors came to the SchoolhealthLink site, 62 visited on Monday through Friday, and 11 visited on the weekend.
Of the five e-mails sent to officers of the MASN, four (80%) completed the “Contributors” online evaluation instrument. When queried about asking visitors to contribute to the SchoolhealthLink site, all responded “yes.” When asked if a list of contributors was a good idea, three responded “yes,” whereas one responded “no” and stated, “I prefer the list of contributors be kept but not on the Web page.” All four respondents responded “yes” that the list of contributors could include name, e-mail address, state, and school district. Suggestions for other information about contributors included age, type of children the nurse serves, and agency or business that employs the nurse.
When asked if the list could evolve into something more, such as a school nurse support group, responses were “You need to be careful about who would monitor the participants. We have a school nurse listserv in our state and have had some interesting things discussed (or opinions expressed). We have had to delete some folks from the listserv,” “Missouri has a school nurse e-mail communication site. SchoolhealthLink should remain a professionally regularly reviewed professional(s) resource site. Opening it to others could diminish its quality,” and “Absolutely, I see this as a great way to keep from reinventing the wheel and an opportunity to bounce ideas off of a broad range of colleagues.”
During the testing period for the diabetes and lice sites, there were 93 individual visitors to the SchoolhealthLink. Although the site was not publicized, there were other visitors to the site. Some visitors viewed only a few of the links, whereas others visited the entire site. The time of day the site was visited ranged from 8:00 a.m. to 10:54 p.m. Of the 248 times recorded for the various pages, 165 were on the school day during the hours of 8:00 a.m. to 4:00 p.m., and 83 occurred after school hours. Of the 23 different days visitors came to the SchoolhealthLink site, 62 visited on Monday through Friday, and 11 visited on the weekend.
DISCUSSION
The research findings of Rogers (1995), Nielsen (1993), Kim et al. (1999), and Jones (1999) provided useful criteria for development of the online survey “Evaluate Site.” Survey instruments were an integral part to determine whether school nurses can use SchoolhealthLink to link to existing WWW-based, disease-specific information designed to enhance their school nursing practice.
Grade Level and Reading Ease scores provided a measure of whether the health information resources selected for SchoolhealthLink were understandable to the nurses and suitable to share with children and parents. Although the mean Reading Ease score for the diabetes was 50.4 and for lice 10—information resources below the aim of 60–70—only a randomly selected section of the text was used to test the Reading Ease level. Within this randomly selected text, medical terminology was included, which may have decreased the ease of reading score.
An interesting finding is that one school nurse reported there was no new information about lice, suggesting an experienced school nurse with expertise about lice. Of all the links, there was only one recommendation—one link to lice should be eliminated. The reason given was the site did not support the nit-free policy common in Missouri schools. A nit-free policy is a controversial issue among school nurses. In their position statement, “Nit Free Policies in the Management of Pediculosis,” The National Association of School Nurses (2003) states “nit-free policies disrupt the educational process and should not be viewed as an essential strategy in the management of head lice.” This position may be in conflict with existing school district policy. However, this supports Brennan’s (1998) premise that an expected benefit from the appropriate use of computer technology is that school nurses can keep up-to-date on the latest health information. Knowledge of the position by NASN and Harvard School of Public Health (Pollack, 2003) included on SchoolhealthLink could result in a change in district policy. A benefit to the school child would be fewer missed days of school.
Results suggest that the “Help Build Site” Web page was a good idea. Although the officers (n = 4) responded that a list of contributors could include a variety of information, for ease of building the site the authors determined that only the name and school district would be included on the “Contributors” page. Results were inconclusive when MASN officers were queried about whether the contributor’s list could evolve to a school nurse support group. Concerns expressed about whether the contributors list could evolve to a school nurse support group were that it would (a) be duplicative of the existing Missouri school nurse listserv, (b) open up SchoolhealthLink and diminish the quality, (c) be difficult to monitor disruptive contributors to a listserv, and that (d) care should be taken. Yet one of the officers stated that it would be an opportunity to “bounce ideas off a broad range of colleagues.” In view of these concerns, further study is needed to determine if a contributor support group would be beneficial to SchoolhealthLink.
Finally, results suggest school nurses found that the SchoolhealthLink site was easy to use, an efficient way to get health information, and supported their practice. An important finding was that school nurses indicated that SchoolhealthLink could change the way they practice. Another important finding was that school nurses found SchoolhealthLink to be better than existing methods of getting health information, such as textbooks, journal articles, school nurse protocols, and materials provided by companies that provide school health supplies. It may be that resources may be easier than reading journals, going to the library, or attending conferences. These findings support Rogers’ (1995) theory of the diffusion of innovations that proposes that for a system such as SchoolhealthLink to be successful, it must be perceived as equal or better than existing methods, be efficient and easy-to-use, and be relevant to practice.
LIMITATIONS OF THE STUDY
The accuracy of the readability and ease of reading scores was limited because only a random sample of the content of health information resources was scanned to determine scores. The sample was not randomly selected. In addition, the response rate to the online “Evaluate Site” was lower than desired. This may be due to the fact that the request was sent at the end of the school year when school nurses are busy with end-of-year tasks. The responses to the “Evaluate Site” online survey may have been more positive than in a general Missouri school nurse population because they were selected by MASN officers because they were perceived by the officers as willing to test SchoolhealthLink.
. . . results suggest school nurses found that the SchoolhealthLink site was easy to use, an efficient way to get health information, and supported their practice.
IMPLICATIONS FOR SCHOOL NURSING PRACTICE
Although the findings of the present study are based on a small sample of school nurses (n = 10, n = 4), there are several important implications that may be drawn from the results. Using the strategy described by Brennan (1995) and Gustafson et al. (1999) to test SchoolhealthLink, there is evidence that a Web-based health information resource is an easy-to-use method for school nurses to find high-quality, up-to-date health information relevant to school nursing practice. The school nurses in this pilot study indicated that SchoolhealthLink was better than methods currently used to get health information and could change the way they practiced, supporting the continued development of SchoolhealthLink.
Health information found on SchoolhealthLink can be shared with children and families. Further development of SchoolhealthLink may include links to health information resources specifically for children or parents. There is a need for continued online evaluation of SchoolhealthLink. This is important in order to tap into the expertise of the school nurses and provide sites to meet the ever-changing health information needs of school nurses.
Now that the SchoolhealthLink (2003) infrastructure is in place, a partnership with the Barnes College of Nursing and the MASN has been established to work together to continue to filter and add health information to the site. The recently launched MASN Web site (http://www.missourischoolnurse.org) provides school nurses with a link to contribute Web-based health information resources, or they may send resources to the SchoolhealthLink “Help Build Site” page. If the school nurse grants permission, her or his name and school district will be added to the “Contributors” page to acknowledge the contribution to the site. Unique to SchoolhealthLink is the ongoing evaluation strategy using the “Evaluate Site” online questionnaire to assure the health information needs of Missouri school nurses for high-quality, relevant health information resources are met. Using this partnership, the SchoolhealthLink project will be an ongoing endeavor to provide health information resources useful to enhance school nursing practice.
Footnotes
ACKNOWLEDGMENT
The authors wish to acknowledge the support of grant F38 LM00069 from the National Library of Medicine and from the Small Grants Fund, University of Missouri—St. Louis.