Abstract
Introduction
Lower limb musculoskeletal disorders place a heavy burden on healthcare systems. Appropriate management of these conditions is critical, however access to appropriate physiotherapy services is difficult for those in geographically remote areas or those with mobility or transport difficulties. The aim of this study was to evaluate the accuracy and reliability of an online musculoskeletal physiotherapy assessment of the knee complex using telerehabilitation compared to traditional face-to-face assessment.
Methods
In a repeated-measures design, 18 subjects who sought treatment for knee pain underwent a traditional face-to-face assessment and a remote telerehabilitation assessment. Telerehabilitation assessments were conducted with participants performing facilitated self-palpation, self-applied modified orthopaedic tests, active movements and functional tasks.
Results
Primary pathoanatomical diagnoses were in exact agreement in 67% of cases and were similar in 89% of cases. The system of pathology was found to be in agreement in 17 out of 18 cases (94%). Comparisons of objective findings from the two physical assessments demonstrated substantial agreement (kappa = 0.635) for categorical data and binary data (chi-squared = 400.36; p < 0.001). A high level of intra-rater (89%) and moderate level of inter-rater (67%) reliability was evident for telerehabilitation assessments.
Discussion
Telerehabilitation assessment of the knee complex appears to be feasible and reliable. This study has implications for clinical practice and the development of physiotherapy services to address the burden of lower limb musculoskeletal pain and disability.
Introduction
In musculoskeletal physiotherapy practice, pain and injury in the knee joint complex is one of the most common presenting conditions. The typical types of injury vary over the lifespan. Children and adolescents predominantly present with soft tissue injuries and developmental disorders. 1 With increasing age and engagement with sport and vocation, traumatic knee injuries become more prevalent. In athletic populations, the knee is the most commonly injured joint and patella-femoral pain has been reported as the most common presentation in private sports physiotherapy practices.2,3 In older populations the incidence of knee pain can be as high as one in three 4 with an increasing incidence of degenerative knee disorders such as osteoarthritis. Knee-related disorders contribute significantly to the burden of chronic musculoskeletal pain and disability, at considerable cost to society.5,6
Physiotherapists provide conservative management for the assessment and treatment of musculoskeletal knee pain. Interventions can be for primary treatment or for rehabilitation after surgical procedures. There are however, a number of patient contexts that can hinder equitable access to physiotherapy and thus lead to chronicity and poor outcomes. These include those living in rural or remote areas where services are scarce and those who have restricted mobility or travel difficulties because of physical, neurological or other impairment.7,8
Telerehabilitation, the provision of rehabilitation services using telecommunications technology as the delivery medium, 7 may offer a solution to some of these issues of access to services.9,10 There is increasing evidence for the efficacy of telerehabilitation in the provision of remote services:11,12 systematic reviews have reported positive outcomes in the areas of the rehabilitation of motor function, 13 rehabilitation for those with multiple sclerosis, 14 rehabilitation interventions for those with cardiac and pulmonary disease15,16 and telerehabilitation for stroke care 17 among others.
The ability to complete an accurate, objective, detailed and reliable patient interview and physical assessment is critical if telerehabilitation systems are to be widely implemented in physiotherapy clinical practice. 18 Many early studies in telerehabilitation were focused on the validation of individual elements of an examination, validating components such as range of motion,19–21 gait assessment 22 and the patient interview. 23 However, more recent research has expanded on earlier efforts to look at the diagnostic accuracy of remote assessment when individual assessment items are combined. Diagnostic accuracy studies have been reported for musculoskeletal disorders in the area of the hip and ankle,24,25 the shoulder and elbow26,27 and lower back.28,29 To the best of our knowledge, no studies have investigated the validity of comprehensive telerehabilitation assessments of the knee complex. The aims of this study are to establish the criterion validity, inter- and intra-rater reliability of a telerehabilitation-based musculoskeletal physiotherapy assessment of the knee complex when compared to traditional face-to-face assessment. We hypothesise that a musculoskeletal assessment of the knee joint complex can be successfully performed via telerehabilitation.
Methods
This study incorporates a repeated-measures design to establish the criterion validity and both inter- and intra-rater reliability of remote physical assessments of the knee via telerehabilitation.
Participants
Over a two-month period, sequential patients with symptoms of knee pain who presented to a Physiotherapy Musculoskeletal and Sports Injuries Clinic in Brisbane, Australia were invited to participate in the study. Participants aged 18 years or older with an adequate level of cognition and communication to follow commands and the ability to independently mobilise were considered for the study. Participants were excluded if they had concomitant medical conditions (e.g. severe respiratory disease) that would prevent them from safely completing a physical examination. Participants provided written consent before participating and the study was approved by the relevant Medical Research Ethics Committee.
Procedure
Participants underwent a patient interview, face-to-face physical assessment and an online physical assessment carried out from a different location via a telerehabilitation system. All assessments were completed within one session of approximately 90 min duration with a brief rest period of 10 min between face-to-face and online assessments. Different examiners conducted each assessment and were blinded to their colleagues' assessment results. The orders of examinations were randomly assigned on recruitment using a balanced block design of size six. Patient interviews were led by the randomised examiner (face-to-face or online) with the other examiner listening as a passive observer. To enable the reliability aspects of the study, the telerehabilitation patient assessment was captured using a recording function on the telerehabilitation system and these recordings were used to perform subsequent assessments (see Figure 1). Recorded telerehabilitation assessments were reviewed again by the original clinician at a time approximately one month after the initial assessment to enable the evaluation of intra-rater reliability (TR2). One month was considered adequate to limit test-retest bias from the perspective of the rating physiotherapist. Video recordings were also viewed by an independent therapist who used these to perform an additional assessment which was used to establish inter-rater reliability (TR3).
Flow of participants through the study. TR1: initial telerehabilitation assessment; TR2: telerehabilitation assessment completed by original therapist one month after initial assessment on videorecorded assessment produced in TR1; TR3: telerehabilitation assessment completed by an independent therapist on videorecorded assessment produced in TR1.
Face-to-face assessments
Face-to-face assessments were conducted in the traditional manner 30 by a physical therapist in the room with the participant. The assessment was not limited in any way and included all components of a physical assessment such as observation, postural examination, palpation, joint range of motion testing, orthopaedic special tests, manual muscle tests, neurological and neurodynamic testing, gait analysis and functional task analysis. Assessment items were chosen at the discretion of the examiner according to the presenting symptoms.
Telerehabilitation assessments
Online assessments were led in real-time by a remote physiotherapist via the eHAB telerehabilitation system (Version 2; NeoRehab, Brisbane, Australia). This system is a portable device which enables real-time videoconferencing to the home of the patient and includes features such as store and forward videorecording, advanced multimedia capabilities to enable the display of pre-recorded instruction videos, exercises etc. and advanced measurement capabilities to enable the measure of patient parameters such as joint range of motion and linear distances during the videoconference. The system connects to the Internet via either a Wi-Fi broadband connection of via the 3G mobile telephone network. More details on the system are provided elsewhere.8,26,27
The online examiner completed the physical examination through the modification of conventional assessment techniques and by the participant applying modified diagnostic tests to themselves. Palpation of the area of injury was completed by the participant under the guidance of the therapist via the real-time videoconference. The therapist demonstrated what was to be done via the videoconference and used pre-recorded video and still images that were shown by the system to the patient to instruct them in the technique. The patient was asked to comment on various parameters during the palpation such as tissue texture, areas of pain, amount of pain etc. Postural examination, gait analysis and active movement were completed in front of the system and the therapist produced video recordings and used the system measurement tools to quantify to resulting movements. Self-resisted manual muscle tests were performed by the participant and modified special orthopaedic tests were self-applied to look for instability and/or pain. For example, participants were asked to report pain intensity and location while applying a passive varus force through their slightly flexed knee while sitting to test the integrity of the lateral collateral ligament. Guidance on how to apply such a force was given verbally by the online examiner and also demonstrated via pre-recorded video-clips which showed a person completing the self-test. Guarding, pain response and the amount of resulting movement was observed and measured by the remote therapist via the videoconference. Neurological and neurodynamic movements were also reproduced by the patient after receiving instruction on how to complete the assessment tasks via the videoconference.
Prior to the commencement of the study, two trial subjects were assessed by each therapist to hone the procedure, use of technology, self-examination techniques and scoring system.
Outcome measures
Example of rating scale for conversion of physical examination findings for analysis.
VAS: visual analogue scale; ROM: range of movement; MCL: medial collateral ligament.
Subjects completed a satisfaction questionnaire at the end of the examinations by marking a 100 mm line to indicate their opinion on a series of questions. The rating scale was bounded by the descriptors ‘complete dissatisfaction’ and ‘extremely satisfied’. The questionnaire assessed six items;
How confident were they with the Internet method of a musculoskeletal assessment. Would they recommend this to a friend who is unable to travel. Do they think this method of assessment is as good as traditional face-to-face assessment. Could they see the physical therapist clearly at all times. Could they hear the physical therapist at all times. What is their overall satisfaction.
Data analysis
All data were analysed using MedCalc version 10.4.8.0 (MedCalc Software, Ghent, Belgium).
To determine validity, data were compared between the face-to-face assessment and the original online assessment. Primary pathoanatomical diagnoses were coded as the same, similar or different, and analysed using descriptive statistics. System diagnoses were analysed using percentage agreement and chi-squared (χ2).
Individual clinical findings codified using the rating scale produced either categorical or binary data. Binary data were compared using χ2 statistics and percentage agreement. Quadratrically weighted kappa statistics (k) and percentage of exact agreement were used for categorical data. The value of k is commonly used in validity papers as it effectively discounts the proportion of agreement that is expected by chance. The strength of agreement between 0 and 1 can be judged as 0.00–0.20 slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.8 substantial agreement and 0.81–1.00 almost perfect agreement. 31 Values of k over 0.40 were considered to be clinically acceptable. Statistical significance was set at p < 0.05 for all tests.
Inter- and intra-rater reliability was analysed using the same statistical methods used to determine validity. Participant satisfaction ratings were analysed descriptively.
Results
Eighteen subjects (eight male and 10 females) with a mean age of 23 ± 7 years participated in the study. All participants underwent both examinations.
Validity
Primary diagnoses provided by both the face-to-face and the telerehabilitation examiner for each participant.
F: female; M: male; TR1: initial telerehabilitation assessment; GR: grade, ACL: anterior cruciate ligament, Med: medial.
The proportions of observed exact and similar agreement for the categorical clinical measures were 83% and 93% respectively. The weighted k coefficient for the categorical data was 0.64 signifying substantial agreement. 31 Significant agreement was present for the binary data (χ2 = 400.4; p < 0.001) which was found to be in exact agreement in 98.8% of cases.
Intra-rater reliability
Repeated assessments of the participants made by a single examiner on the recorded telerehabilitation videoconference were in exact agreement in 16 out of 18 cases (89%). This increased to 18 out of 18 cases (100%) for similar agreement. The system of pathology was found to be in agreement in 17 out of 18 cases (94%) between the two assessment methods. This level of agreement was found to be significant (χ2 = 12.5; p < 0.001).
The proportions of observed exact and similar agreement for the categorical clinical measures were 97% and 100% respectively. A weighted k coefficient for the ordinal data was 0.98 signifying almost perfect agreement. Significant agreement was present for the binary data (χ2 = 1121.4; p < 0.001) which was found to be in exact agreement in 100% of cases.
Inter-rater reliability
Assessments of the participants made by two independent clinicians on the recorded telerehabilitation videoconference were in agreement in 12 out of 18 cases (67%). This increased to 17 out of 18 cases (94%) for similar agreement. The system of pathology was found to be in agreement in 12 out of 18 cases (67%) between the two assessment methods. This level of agreement was found to be significant (χ2 = 9.39; p = 0.002).
The proportions of observed exact and similar agreement for the categorical clinical measures were 94% and 99% respectively. A weighted k coefficient for the ordinal data was 0.94 signifying almost perfect agreement. Significant agreement was present for the binary data (χ2 = 982.4; p < 0.001) which was found to be in exact agreement in 99% of cases.
Survey
Patients reported satisfaction on survey questions was high and is presented in Figure 2.
Mean participant responses to survey on 100 mm visual analogue scale (VAS). Error bars display +1 standard deviation (SD) of mean. Question 1: confidence with physical examination; 2: recommended to a friend who is unable to travel; 3: as good as face-to-face; 4: visual quality; 5: auditory quality; 6: overall satisfaction.
Discussion
The results of this study supports our hypothesis that a musculoskeletal assessment of the knee joint complex can be successfully performed via telerehabilitation. In this study, the validity of the telerehabilitation assessments was demonstrated through a high level of agreement between diagnoses performed via telerehabilitation when compared to traditional face-to-face assessments. There are two occurrences (11%) where the diagnosis across the modalities was different, which require consideration. The first case (Case 1) involved an injury from blunt trauma to the knee. The face-to-face examiner recorded subluxation as part of the diagnosis together with soft tissue injury while the telerehabilitation diagnosis was focussed on the soft tissue injuries that resulted from the trauma. In both of these cases it could be argued that the immediate short-term management of pain reduction, management of swelling and restoration of functional movement would have been similar. The second case (Case 17) was diagnosed as a superior Tibio-fibular joint injury by the face-to-face examiner while the telerehabilitation therapist diagnosed the disorder as iliotibial friction syndrome. These conditions usually present quite differently in terms of the mechanism of injury and behaviour of symptoms although the location of symptoms can be quite similar. It is likely that the manual examination which was available in the face-to-face examination, where the superior tibio-fibular joint could be passively mobilised to reproduce symptoms, was a factor in this differing diagnosis. In this case, initial management of the patient may have indeed been different and treatment progress may have been sub-optimal. In light of this finding, adaptations to the clinical protocol of the telerehabilitation assessment of the knee should be made to better evaluate this joint. These adaptations could include a more thorough patient palpation of the superior tibio-fibular joint, the addition of anterior, posterior or compressive forces to the joint to provoke pain and more thorough questioning about the mechanism of injury.
The assessment of four cases in this study resulted in similar diagnoses between face-to-face and telerehabilitation assessments. In all cases, the main focus of the diagnosis was consistent between the assessments, however additional elements of the diagnosis were different. For example, in Case 4 both diagnoses identified a grade 1 lateral collateral ligament strain as the primary pathology with one diagnosis suggesting anterior cruciate ligament involvement while the other suggested meniscal involvement. In each case, the initial management of the patient is unlikely to have been different given the diagnoses.
The level of agreement between the methods of assessment in this study are not dissimilar to the level of agreement that has been reported in the literature for two physiotherapists who both examine the same patient in the traditional face-to-face manner. For example, a study by Cooperman et al. examined the validity of two physiotherapists and two orthopaedic surgeons who examined the knees of 32 patients, 13 of whom had an anterior cruciate ligament (ACL) rupture. 32 This study found that the percentage of agreement on the results of the Lachman's test was 76% for all examiners. In a similar study by McClure et al., percentage agreement of 56–90% was found between three physiotherapists performing the tibiofemoral joint abduction tests of the medial collateral ligament of the knee. 33
The results of the inter-rater and intra-rater reliability analysis for the telerehabilitation examinations was also high with 67% and 89% agreement between methods respectively. The fact that the inter-rater reliability agreement was similar to the validity results gives weight to the notion that it may have been patient performance differences, clinical reasoning and the skills of the therapists, rather than the technology per se, that was the source of the error.
The success of the remote assessments, despite the absence of physical contact between the examiner and participant during the online examination is noteworthy. It is well known and widely accepted that direct palpation and refined orthopaedic examinations are significant diagnostic tools in clinical practice.34 In the telerehabilitation environment participants were instructed to self-palpate anatomical structures for pain and texture. The online therapist encouraged the subject to compare the sensory feel to the unaffected limb to assist this process. The main limitation to self-palpation was the inability of the subject to palpate posterior knee structures accurately. Subjects were also instructed by pre-recorded videos in how to apply modified orthopaedic tests and report their findings. While these tests were obviously difficult to teach and were not always performed with a high level of accuracy, they did appear to give the remote clinician enough information from which to draw conclusions. This is evident in the high level of agreement between face-to-face and telerehabilitation clinical tests (categorical and binary data) found in this study.
As noted, functional assessment was a key component of the telerehabilitation assessment, especially where it was not practical for a subject to perform a specific orthopaedic test on himself or herself. For example, participants were unable to directly test the patency of the anterior cruciate ligament (e.g. by anterior draw or Lachman’s test) however functional testing (e.g. hopping or changing direction) and biomechanical frame-by-frame analysis on the telerehabilitation system enabled the successful diagnosis of this pathology. Indeed, there are advocates for the use of these clinical tests in face-to-face practice. 35
Overall, the participant satisfaction ratings of telerehabilitation assessments were with a high overall reporting of satisfaction (Figure 2). High ratings were observed for auditory and visual components of the videoconference, suggesting that communication limitations reported in previous research were not experienced in this study. In accordance with other studies,26,27 a high number of participants expressed a preference for traditional, face-to-face assessment if given the choice, however they were very supportive of telerehabilitation processes if remote consultations were the only option.
There were a number of difficulties encountered in this study. Although not specifically recorded, online examinations took longer than the traditional face-to-face assessments. This was due to the need to explain positioning, the self-application of modified tests and qualification of findings. Anecdotally, the time taken to complete the assessment was only marginally more and was still considered acceptable by the participants and the physiotherapists. Future studies should quantify the difference in time required to perform online assessments as this may impact upon the viability of a telerehabilitation service. Due to inherent differences in practitioner test choice and inherent differences between face-to-face and telerehabilitation consultations, it was not possible to perform a statistical analysis comparing findings on individual specific orthopaedic tests. This would have been interesting to investigate. On the technical side, communication failures on two occasions interrupted the telerehabilitation sessions. Although this was inconvenient and may have reduced the participant’s confidence in using the system, neither interruption appeared to have any detrimental effect on the assessment outcomes.
A number of study limitations were also identified. Firstly, the repeated-measures study design enabled participants to learn from the initial examination, and this may have subsequently influenced the physical findings of the second examination. The study design also frequently provoked symptoms during the second examination in irritable and latent participants, a finding that has been observed by other authors who used repeated-measures study design.26–28 Consequently, this may have resulted in different clinical findings, influencing the agreement seen between the validity findings. Secondly, the mean age of 23 ± 7 years of participants was low and consequently there was a high prevalence of acute and sub-acute conditions. These conditions may be easier or harder to diagnose when compared to developmental or chronic conditions and this limits the generalisability of the results to a defined population. Future research should examine more diverse populations. Finally the inter-rater reliability results may be inflated because the order and selection of tests by the initial telerehabilitation clinician influenced what was observed by the second clinician.
This study demonstrates that telerehabilitation-mediated musculoskeletal assessment and diagnosis of disorders of the knee is valid and possesses good inter- and intra-rater reliability. This study has implications for clinical practice and the development of physiotherapy services to address the burden of chronic musculoskeletal pain and disability.
Footnotes
Acknowledgments
The authors would like to thank the management and staff at The University of Queensland Physiotherapy Musculoskeletal & Sports Injuries Clinic for accommodating this research.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: one author (TGR) has a material interest in the telerehabilitation system used in this manuscript. To remove bias, this author was not involved in the collection or analysis of the data for this paper.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
