Abstract
Background
The present study aimed to assess the accuracy and characterize the learning curve of surgeon-lead shoulder ultrasound scans (USS) performed in outpatients for suspected rotator cuff tears, with intra-operative findings considered as the gold standard.
Methods
From 2009 to 2011, all patients having arthroscopic shoulder surgery by the senior author were identified. Clinic letters were reviewed to identify those who had undergone USS in clinic. This was then compared with the operating findings.
Results
A total of 66 patients had an USS and proceeded to arthroscopic shoulder surgery during that time. Overall sensitivity and specificity was 0.86 and 0.70, respectively. Comparing values from 2009 to 2011: specificity improved from 0.50 to 0.8; sensitivity remained much the same with 2009 values of 0.88 to 2011 values of 0.86.
Conclusions
The results reflect good sensitivity and specificity, which was comparable with that reported in the literature. There was an improvement in specificity over time displaying a learning curve and emphasizing the question of how much experience in shoulder USS is required before it can be relied upon as the patient's primary imaging preoperatively?
INTRODUCTION
Rotator cuff tears are a common presenting pathology characterized by pain and loss of function. Prompt diagnosis and management are an integral part of the patient pathway. The use of ultrasound scans (USS) in a shoulder outpatient clinic to diagnose rotator cuff tears is a convenient way of assessing the patient. It is cheap, readily available and a safe form of imaging. The disadvantages of USS are that it is heavily dependent upon the skill and experience of the user [1].
Although USS is more commonly performed by radiologists, its use in the clinic by orthopaedic surgeons conveys the advantage that the scan is performed in the context of the history and clinical examination as obtained by the surgeon and thus may help the surgeon to better correlate scan findings. Accurate radiological findings are important not only to aid clinical management of the patient, but also to help plan operative intervention in terms of equipment and theatre time required.
Results of clinic based USS previously published in the literature have been shown to be comparable to that of musculoskeletal radiologists. Jeyam et al. compared the shoulder USS of musculoskeletal radiologists to USS carried out in clinic with favourable results [2]. In the diagnosis of full-thickness rotator cuff tears, the radiologist has a sensitivity of 94% compared to 92% for the surgeon. However, the radiologist displayed superior sensitivity at diagnosing partial-thickness tears, with 100% success compared to 85.7% for the surgeon.
These results are echoed in other studies, such as that of Ziegler in a series of 282 patients [3], who quotes a sensitivity of 95.9% and 94.1% and a specificity of 94.3% and 96.1% for full-thickness tears and partial-thickness tears, respectively. However, there are other published studies reporting comparatively inferior accuracy for diagnosing cuff tears. A retrospective review of 85 patients undergoing USS by a radiologist using shoulder arthroscopy as the gold standard revealed sensitivities and specificities of 66% and 54%, respectively [4].
There is also good evidence that USS for rotator cuff tears is as accurate as magnetic resonance imaging. A meta-analysis of 65 articles performed by de Jesus et al. compared the use of magnetic resonance imaging (MRI), MRI arthrography and USS in the diagnosis of rotator cuff injuries [5]. It was found that MRI arthrography was statistically superior to the other two imaging modalities in the diagnosis of full-thickness and partial-thickness tears. There was no statistical difference between plain MRI or USS in the diagnosis of both full- and partial-thickness tears. Interestingly, USS was superior to MRI in diagnosing partial-thickness tears, although this was not supported by a favourable p-value.
The present study aimed to assess the accuracy of an orthopaedic surgeon performing shoulder USS in a district general hospital in an outpatient clinic, as well as to characterize the learning curve.
MATERIALS AND METHODS
The surgeon performing the scans had previously attended two courses on shoulder ultrasonography and carried out his practice in a similar manner to that of the recently published guidelines from the British Society of Shoulder and Elbow Surgeons (BESS) [6]. All scans were carried out using a General Electric LOGIQ e ultrasound scanner (GE Healthcare, Milwaukee, WI, USA). A 12-MHz linear probe was used. All patients who underwent arthroscopic shoulder surgery from 2009 to 2011 under the care of the senior author were identified. Clinic letters were cross-referenced to select only those patients who had undergone surgeon-lead USS before their surgery, and scan findings in terms of cuff pathology were recorded. Data were categorized as negative or positive for a rotator cuff tear (i.e. partial and full thickness).
A comparison was made with the intra-operative findings using the operative notes. A 2 × 2 table was created from this and sensitivity, specificity and predictive values were calculated. Further analysis by year of surgery was conducted to characterize the learning curve of the scanning surgeon.
RESULTS
Over the 3-year period, the consultant had carried out 213 shoulder arthroscopies of which 66 had had USS performed by the surgeon in outpatients before their surgery. The overall raw results are shown in Table 1 in a 2 × 2 format. The sensitivity, specificity and predictive values of both yearly data and overall are shown in Table 2.
A table showing overall results for ultrasound
A table showing yearly results of ultrasound
CI, confidence interval.
A yearly improvement can be seen in both positive predictive value (2009: 0.70 to 2010: 0.79 to 2011: 0.95) and specificity (2009: 0.50 to 2010: 0.75 to 2011: 0.80). Sensitivity remains much the same throughout the 3 years, and the negative predictive value showed a decline in 2011 (0.57) compared to the previous years (2009: 0.75; 2010: 0.82). The overall sensitivity and positive predictive value was 0.86 and 0.84, respectively. As can be seen from Fig. 1, the accuracy of the surgeon improves throughout the 3 years of data sampling, with the overall accuracy being 80%. The graph displays the learning curve of the surgeon, although, as a result of the data being collected over a 3-year period, it is difficult to appreciate what the maximal accuracy of scanning from the surgeon would be because the curve is yet to plateau. Because of the low numbers of patients scanned in year 1, this can be interpreted as an ultrasound orientation period as described in the BESS training and competency pathway. Therefore, the most important comparison is between years 2 and 3 where there are greater numbers of patients scanned. The broad confidence intervals shown in Table 2 can be attributed to the relatively small numbers of patients in each yearly breakdown.
A graph to show accuracy of ultrasound over time in years.
DISCUSSION
The results overall reflect the good accuracy of USS as performed by the operating surgeon. The sensitivity, specificity and predictive values are comparable to most of the previously reported literature on the topic. A low negative predictive value from the series indicates that there were several false positives in the data. Several of these were in fact partial-thickness cuff tears on arthroscopic findings. The previous literature suggests that partial-thickness tears are more difficult to detect compared to full-thickness tears [7 –9] and substantiates this outcome. Some bias does exist in this series because not all patients undergoing USS in clinic would have come to surgery, and these therefore may represent an element of selection bias in possibly reducing a theoretical number of true negatives.
The added advantage of the use of USS in clinic is that it can give instant feedback for the clinician's findings in the immediate context of the history and examination. USS can help clarify the diagnosis of rotator cuff tears compared to other conditions that can mimic it on examination, such as calcific tendonitis and adhesive capsulitis [10] The accuracy of shoulder USS reported in the literature can vary greatly. Naqvi et al. reported a sensitivity of 0.88 and specificity of 0.89 in the diagnosis of rotator cuff tear by radiologists with a special interest in musculoskeletal pathology [7]. A more recent series in the setting of a UK district general hospital found much lower than expected values for radiologist lead scans, with a sensitivity of 0.48 and a specificity of 0.79 [4]. However, Jeyam et al. reported a sensitivity of 0.92 and a specificity of 1.0 in the diagnosis of full-thickness tear by surgeons, as well as a comparable accuracy of radiologists and surgeons [2].
We can compare our surgeon's experience with that of Murphy et al. [11] (as referenced by the BESS guidelines). In this series, an ‘accelerated learning method’ is described where patients are scanned on the same day as their operation, giving instant feedback for the surgeon. In their study, they had an initial orientation period of 50 scans followed by a further 50 scans. In their second set of 50 scans from four different operators, they had an overall accuracy of 94% and sensitivity of 90% for full-thickness tears. This can be compared to our years 2 and 3 (n = 52 scans) with an accuracy of 83% and a sensitivity of 86%. Some of the discrepancy can be explained by Murphy et al. [11] only including full-thickness tears compared to our data not discriminating between full- and partial-thickness tears. However, it does raise the point that a more focused learning method may be advantageous. Along with the consideration of reviewing overall numbers of scans, there also appears to be a relevance of frequency of scanning (i.e. scans per annum). Our data reveal that, with a comparatively low frequency of scanning compared to Murphy et al. [11], not only can a reasonable accuracy of scanning be achieved, but also an improvement in results can also be attained. Our results do show an improvement over time particularly with sensitivity and positive predictive value indicating a learning curve. The question that this raises is when is a surgeon competent enough at the USS imaging modality to rely upon clinic USS as the primary imaging investigation preoperatively? The low numbers of scans in our series from year 1 make comparison with the latter 2 years difficult but, by viewing this year as a period of orientation by the surgeon, a more confident comparison can be made between years 2 and 3. The shape of the learning curve will vary between operators, although the number and frequency of the USS is an important factor.
What the findings of the present study paper does support is the importance of appraising one's own results to ensure they are reliable, or perhaps further experience or training is required.
The above question has been addressed by the BESS guidelines [6] on shoulder USS for rotator cuff injuries, which recommend a clinical review of the results at 50 and 100 scans. If the results at that point are comparable to the published results, then the guidelines support the clinical use of shoulder USS in clinic. Accordingly, the operator can expect to assess their own learning curve at fixed points, and compare their results with the published evidence. However, as can be seen from our data, the 66 USS were collected over 3 years, which would indicate that, in a district general hospital setting, there would be a sizeable gap between beginning to collect data and having sufficient experience to use USS in clinic as the lead imaging modality.
Conclusions
When used in an outpatient clinic setting, shoulder ultrasound is an accurate diagnostic tool that gives instant feedback to the operating surgeon. The reliability of it as an imaging modality is both operator-dependent and related to the amount of scanning experience that the operator has.