Clinical Evaluation of NOC Outcomes

PURPOSE

To evaluate NOC outcomes in field sites across the care continuum; to identify core outcomes and linkages among diagnoses, interventions, and outcomes.

METHODS

Researchers tested 190 patient outcomes, developed and tested for content validity and coded in a taxonomic structure in an earlier phase of the research, for reliability, validity, sensitivity, and usefulness in 10 field sites: 2 academic, tertiary hospitals; 3 community hospitals; 1 long-term care nursing home; 1 communitybased parish nursing practice; 1 ambulatory, academic nursing center staffed by nurse practitioners; and 2 community health nursing agencies providing home care (visiting nurses) in 4 midwestern states.

Reliability: Two types of interrater agreement were assessed. Near agreement was defined as the numerical ratings that do not differ by more than 1 value on the 5-point Likert-type scale and include both the label and its indicators. Absolute agreement is the percent of absolute or total agreement between 2 nurses' ratings for the outcome, irrespective of the indicator ratings.

Validity: For most of the outcomes, standardized tools that measure the same or similar concept were located to estimate construct validity. No standardized tools were located for some NOC outcomes, especially those that describe discrete physiologic outcomes. To estimate the validity of these outcomes, specified information was extracted from patients' health records. Pearson correlation was used to determine similarity between the NOC measure and the standardized tool.

Sensitivity: Sensitivity is the extent that a measure captures change in a phenomenon. This sensitivity to change is often called responsiveness to change in health outcomes research. The smaller the change the measure is able to gauge, the more responsive it is. The sensitivity of NOC outcomes is examined by comparing two measures of the outcome on the same patient at two different times.

Usefulness: Because the outcome labels will be used by nurses to monitor and evaluate the effects of interventions for patients, they must be clear with unambiguous definitions and measurement scales that nurses can rate (score) with ease and confidence. Field-site nurses were asked to note their comments about difficulties using the outcomes and measures, and to provide suggestions for revisions on the data collection forms. The investigators also met with the nursing staff to obtain feedback.

FINDINGS

Data from approximately 2,300 patients have been collected. Based on the results to date, there is cautious optimism that most of the NOC outcomes and measures will demonstrate an acceptable degree of interrater reliability. The percent agreement on many of the outcomes is ≥80%, and the majority of weighted Kappa coefficients are ≥.60. Validity testing of the outcomes, while encouraging, shows a greater variance in correlation scores. This variance is directly related to the compatibility of the two measures.

CONCLUSIONS

The research team is continuing to develop, validate, and classify outcomes and outcome measures for individuals, families, and communities. Several issues are being addressed by the research–for example, the issue of whether the outcome indicators should be rated, and if so, how the final rating for the outcome should be determined. One site is rating the outcome label only and not the indicators. If these labelonly ratings are shown to be as reliable and valid as when indicators are rated, it may suggest that nurses need only rate the outcome label. This is an important issue, given that most nurses are highly concerned about the burden of documentation.