The Interrater Reliability of Nursing Outcome Classification (NOC): “Caregiver Performance: Direct Care”

Abstract

PURPOSE

The article investigates the interrater reliability of Nursing Outcome Classification (NOC) “Caregiver Performance: Direct Care” in Indonesian language.

METHODS

Two raters assessed 31 caregivers of clients with mental disorders using NOC “Caregiver Performance: Direct Care” in January 2016. This NOC was translated into Indonesian and each indicator was operated in accordance with caregivers’ need in taking care of clients. Kappa and percent agreement were used for the analysis.

FINDINGS

The overall kappa value of NOC “Caregiver Performance: Direct Care” was 0.752, while the percent agreement was 80.

CONCLUSIONS

The interrater reliability of NOC “Caregiver Performance: Direct Care” in Indonesian was at the level of substantial reliability.

IMPLICATION FOR NURSING PRACTICE

NOC “Caregiver Performance: Direct Care” can be used in the clinical setting.

Keywords

Caregiver interrater reliability kappa value NOC percent agreement

Get full access to this article

View all access options for this article.

References

Altman

D. G.

(1991). Practical statistics for medical research. London; New York: Chapman and Hall.

American Educational Research Association . (1999). Standards for educational and psychological testing. Retrieved January 2, 2016, from http://www.apa.org/science/programs/testing/standards.aspx

Arikunto

(2013). Prosedur penelitian: Suatu pendekatan praktik. Jakarta, Indonesia: Rineka Cipta.

Azwar

(2013). Metode penelitian. Yogyakarta, Indonesia: Pustaka Pelajar.

Bretz

Milkovich

G. T.

Read

(1992). The current state of performance appraisal research and practice: Concerns, directions, and implications. Journal of Management, 18, 321–352.

Cook

D. A.

Beckman

T. J.

(2009). Does scale length matter? A comparison of nine- versus five-point rating scales for Mini-CEX. Advances in Health Sciences Education, 14, 655–684.

Danielson

(2011). Evaluation that help teachers learn. Educational Leadership, 68(4), 35–40.

Dunbar

S. B.

Koretz

D. M.

Hoover

H. D.

(1991). Quality control in the development and use of performance assessments. Applied Measurement in Education, 4(4), 293–303.

Fowler

I. L.

Carr

V. J.

Carter

N. T.

Lewin

T. J.

(1998). Patterns of current and lifetime substance use in schizophrenia. Schizophrenia Bulletin, 24(3), 443–455.

10.

Given

B. A.

Given

C. V.

(1991). Family caregiving for the elderly. Annual Review of Nursing Research, 9, 77–101.

11.

Graham

Milanows

Miller

(2012). Measuring and promoting inter-rater agreement of teacher and principal performance ratings. Retrieved January 2, 2016, from http://files.eric.ed.gov/fulltext/ED532068.pdf

12.

Gwet

K. L.

(2014). Handbook of interrater reliability. Gaithersburg, MD: Advanced Analytics, LLC.

13.

Hawari

(2012). Schizophrenia. Holistic approach bio-psycho-socio-spiritual (3rd ed.) [in Indonesian]. Jakarta, Indonesia: Fakultas Kedokteran Universitas Indonesia.

14.

Heneman

H. G.

Milaowski

A. T.

(2003). Continuing assessment of teacher reactions to a standards-based teacher evaluation system. Journal of Personnel Evaluation in Education, 17(2), 173–195.

15.

Hoyt

W. T.

Kerns

(1999). Magnitude and moderators of bias in observer ratings: A meta-analysis. Psychological Methods, 4, 403–424.

16.

Johnson

R. L.

Penny

J. A.

Gordon

(2008). Assessing perfomance. New York: The Guilford Press.

17.

Linn

R. L.

Baker

E. L.

(1996). Can performance-based student assessments be psychometrically sound? In Baron

J. B.

Wolf

D. P.

(Eds.), Performance-based student assessment: Challenges and possibilities. Ninety-fifth yearbook of the National Society for the Study of Education. Chicago: University of Chicago Press.

18.

Marchira

C. R.

(2012). The influence of quick interactive psychoeducative intervention about schizophrenia toward caregiver knowledge, regular control, medicine adherence and relapse on psychotic patient in the first psychotic episode (Doctoral dissertation). Universitas Gadjah Mada, Yogyakarta, Indonesia.

19.

McHugh

M. L.

(2012). Interrater reliability: The kappa statistic. Journal of Croatian Society of Medical Biochemistry and Laboratory Medicine, 22(3), 276–282.

20.

Ministry of Health of Republic Indonesia . (2006). Mental health nursing standard. Jakarta, Indonesia: Ministry of Health.

21.

Mohr

W. K.

(2006). Psychiatric mental health nursing. Philadelphia: Lippincott Williams & Wilkins.

22.

Moorhead

Johnson

Maas

M. L.

Swanson

(2013). Nursing Outcomes Classification (NOC). St. Louis, MO: Elsevier Mosby.

23.

Myford

C. M.

Wolfe

E. W.

(2009). Monitoring rater performance over time: A framework for detecting differential accuracy and differential scale use. Journal of Educational Measurement, 46(4), 371–389.

24.

National Institute of Health Research and Development . (2013). RISKESDAS 2013. Retrieved January 2, 2016, from http://dinkes.bantenprov.go.id/read/hasil-riskesdas-2013.html

25.

Orem

D. E.

(1991). Nursing: Concepts of practice. St. Louis, MO: Mosby Year Book.

26.

Pharm

N. G.

Bell

J. S.

Chen

T. F.

(2013). Interrater agreement and interrater reliability: Key concepts, approach, and applications. Research in Social and Administrative Pharmacy, 9(3), 330–338.

27.

Putri

Rositawati

(2015). Descriptive study of self compassion on mother undergoing hemodialysis in Al-Ihsan local hospital Bandung. Paper presented at the Prosiding Psikologi.

28.

Ramirez Garcia

J. K.

Chang

C. L.

Young

J. S.

Steven

R. L.

Janis

H. J.

(2006). Family support predicts psychiatric medication usage among Mexican American individuals with schizophrenia. Social Psychiatry and Psychiatry Epidemiology, 41(8), 624–631.

29.

Rushforth

H. E.

(2007). Objective Structural Clinical Examination (OSCE): Review of literature and implication for nursing education. Nurse Education Today, 27(5), 481–490.

30.

Sari

A. N.

(2014). Effectivity of chronic disease management program in treating diabetes mellitus type 2 of family physician in Turi District, Sleman region. Yogyakarta, Indonesia: Universitas Gadjah Mada.

31.

Sudiyanto

(2009). Hambatan Terapi Pasien Schizophrenia dan Penyelesaiannya Ditinjau dari Sudut Pandang Pasien dan Keluarga. Jiwa: Majalah Psikiatri, Indonesian Psychiatric Quarterly, 1, 73–84.

32.

Utami . (2014). Laporan bulanan puskesmas kalasan. Yogyakarta: Kalasan Public Health Center.

33.

Tsuai

A. S.

Barry

(1986). Interpersonal affect and rating errors. Academy of Management Journal, 29(3), 586–599.