Inter-Rater and Intra-Rater Reliability of the Kansas University Sitting Balance Scale

PURPOSE: Objective, measurable language is an expectation in occupational therapy (OT) documentation (Sames, 2014), but despite this expectation, therapists have been using non-measurable terms such as good and poor to describe sitting balance (Kluding, Swafford, Cagle, & Gajewski, 2006). Sitting balance is a client factor and performance skill that has an effect on both patient function and length of hospital stay (Oh et al., 2013). Acknowledging the importance of balance, and their own non-objective documentation, Kluding et al., created The Kansas University Sitting and Standing Balance Scales (KUSSBS) in 2006. The sitting and standing balance scales of the KUSSBS use the same zero to five numbering classification and terminology. However, intra-rater and inter-rater reliability were only established for the standing scale.

The aim of this study was to establish inter-rater and intra-rater reliability for the Kansas University Sitting Balance Scale (KUSitBS) by answering the question: Does the KUSitBS have acceptable intra-rater and inter-rater reliability for use by occupational therapists with an adult population?

DESIGN: This descriptive study is a randomized, cross-sectional design utilizing videotaped sessions of a simulated subject. To inform the design of the study, a literature review was conducted on the use of videotaped sessions, number of raters, time interval between ratings, and the use of simulated patients. Simulated patients were not found in any balance instrument reliability studies, but they are extensively used in physician studies and correlated significantly to ‘real’ patients in a study of manual muscle testing.

METHOD: Four OTs working in an acute care hospital were recruited to rate a videotaped performance of a simulated patient using the KUSitBS. The patient simulation video was performed by three master of OT graduate students and directed by the principal investigator (PI); the performance was scripted and rehearsed for accuracy of the scenario. The simulation was videotaped by a professional videographer and each scenario level of the KUSitBS edited into a random order. The recruited OTs were provided a brief introduction to the KUSitBS and a written copy of the scale. They viewed each scenario on the video only once, rating each using the KUSitBS. Exactly two weeks later, the participants were e-mailed a blank rating form and YouTube link to the simulation video. Having committed to viewing each scenario only once, they viewed and rated the levels at home and e-mailed the completed rating form to the PI.

RESULTS: Inter-rater reliability was assessed using two-way random, absolute-agreement, single-measures inter-class coefficients (ICC). The resulting ICC was in the good range, ICC = .877, p < .01, indicating that the OT raters had a high degree of agreement and suggesting that sitting balance on the KUSitBS was rated similarly across coders.

Intra-rater reliability was assessed using a two-way mixed, absolute-agreement, single-measures inter-class coefficients (ICC). The resulting averaged ICC was in the excellent range, ICC = .914, indicating that each OT rated sitting balance on the KUSitBS similarly at initial assessment and then again two weeks later.

CONCLUSION: In order to have confidence in using any assessment tool, intra- and inter-rater reliability must be established. This study established reliability of the KUSitBS and advances its use with adult populations.

The KUSitBS is an objective and measurable method of describing balance that requires no special equipment or training, and can be used during any functional activity of importance to the client. The use of the KUSitBS could enhance professional documentation by occupational therapists working in adult settings.

References

Kluding, P., Swafford, B.B., Cagle, P., & Gajewski, B.J. (2006). Reliability, responsiveness, and validity of the kansas university standing balance scale. Journal of Geriatric Physical Therapy, 29(3), 93-9.

Oh, H-M., Im, S., Ko, Y.A., Ko, S.B., & Park, G-Y. (2012). The sitting-unsupported balance score as an early predictor of functional prognosis in stroke patients: a pilot study. Annals of Rehabilitation Medicine, 37(2), 241-246.

Sames, K.M. (2014). Documentation in practice. In B. Boyt Schell, G. Gillen, & M. Scaffa (Eds.), Willard & Spackman’s occupational therapy (12th ed., pp. 466-475). Baltimore, MD: Lippincott Williams & Wilkins.