Abstract
Background:
Osteochondritis dissecans (OCD) is a vexing condition for patients, parents, and physicians because of the frequent slow healing and nonhealing that leads to prolonged treatment. Several features on plain radiographs have been identified as predictors of healing, but the reliability of their measurement has not been established.
Purpose:
To determine the inter- and intrarater reliability of several radiographic features used in the diagnosis, treatment, and prognosis of OCD femoral condyle lesions.
Study Design:
Cohort study (Diagnosis); Level of evidence, 3.
Methods:
Pretreatment anteroposterior, lateral, and notch radiographs of 45 knees containing OCD lesions of the medial or lateral femoral condyle were reviewed in blinded fashion by 7 orthopaedic physician raters from different institutions over a secure web portal at 2 time points over a month apart. Classification variables included lesion location, growth plate maturity, parent bone radiodensity, progeny bone fragmentation, progeny bone displacement, progeny bone contour, lesion boundary, and radiodensity of the lesion center and rim. Condylar width and lesion size were measured on all views. Interrater reliability was assessed using free-marginal kappa and intraclass correlations. Intrarater reliability was assessed using the Cohen kappa, linear-weighted kappa, and intraclass correlations based on measurement type.
Results:
Raters had excellent reliability for differentiating medial and lateral lesions and growth plate maturity and for measuring condylar width and lesion size. In the subset of knees with visible bone in the lesion, the fragmentation, displacement, boundary, central radiodensity, and contour (concave/nonconcave) of the lesion bone were classified with moderate to substantial reliability. The radiodensity of the lesion rim and surrounding epiphyseal bone were classified with poor to fair reliability.
Conclusion:
Many diagnostic features of femoral condyle OCD lesions can be reliably classified on plain radiographs, supporting their future testing in multifactorial classification systems and multicenter research to develop prognostic algorithms. Other radiographic features should be excluded, however, because of poor reliability.
Osteochondritis dissecans (OCD) has recently been redefined as a focal idiopathic alteration of subchondral bone with risk for instability and disruption of adjacent articular cartilage that may result in premature osteoarthritis. 13 This condition is most commonly located in the femoral condyles of the knee. 18 OCD presents a unique challenge for treating physicians due to a lack of evidence surrounding its origin, the nonoperative and operative treatment indications, and the most appropriate treatments for each stage of the condition. 13 The American Academy of Orthopaedic Surgeons (AAOS) clinical practice guideline on OCD found only weak or inconclusive support for all OCD-related diagnostic and treatment protocols.5,6
Based on retrospective studies, approximately 30% to 50% of skeletally immature patients with stable OCD lesions of the knee will fail to heal with nonoperative or operative treatment.3,4,11,16,21,26,30,31 The lack of high-level evidence makes it difficult to determine which treatment will be most beneficial to a patient. Simple predictive nomograms that incorporate radiography, magnetic resonance imaging (MRI), and clinical findings have been introduced to help guide treatment decision making but have yet to be validated.22,31 To our knowledge, there have been no studies assessing the reliability of OCD staging or classification systems using radiographs, which is a prerequisite for research on predictors of OCD healing.
Radiographs and MRI provide complementary information during the initial assessment of OCD lesions. Radiographs are useful for making the initial diagnosis and for assessing skeletal maturity, while MRI is particularly important for lesion staging and determination of stability due to the visibility of articular surface defects and subchondral delamination. Since radiographs are currently the standard of care for assessing OCD healing over the course of OCD treatment, it is essential that features visible on radiographs can be classified reliably. Several features on plain radiographs, including open growth plates, lesion location, size, and sclerosis,18,22,27,31 have been identified as predictors of a healing response to treatment; however, the reliability of these radiographic features has not been established. The purpose of this multicenter study was to determine the interrater and intrarater reliability of several radiographic features of OCD femoral condyle lesions.
Materials and Methods
Study Design
The study was performed by members of the Research in Osteochondritis of the Knee (ROCK) group. This multicenter research group, devoted to the investigation of cause, diagnostic criteria, management, and optimization of physical activity for patients with OCD, collaboratively developed a list of radiographic features that were speculated to be predictors of healing of OCD knee lesions based on clinical experience and literature-based reports. 25 The purpose of this study was to determine the reliability of each radiographic feature, which is a prerequisite for future study of the effect each feature may have at predicting healing success.
Before classification, physician raters were instructed on the individual radiographic features through a PowerPoint tutorial. The radiographs used in the tutorial were not included in the study, and the raters who participated in the testing phase of the study did not participate in case selection or preparation. The features and instructions provided for classification are listed in the Appendix (available online at http://ajsm.sagepub.com/supplemental). Parent bone refers to the epiphyseal bone surrounding the OCD lesion, and progeny bone refers to any bone within the OCD lesion itself, if present (Figure 1). 24 The location of the lesion was classified for both the anteroposterior and lateral planes. In the anteroposterior images, the lesion could be located on the medial or lateral femoral condyle. In the lateral images, the lesion could be located on either the anterior or posterior portion of the condyle. The lesion could also be located in the trochlear groove or on the patella, but these lesions were not included in our sample.

The parent bone (black arrow) is considered the epiphyseal bone surrounding the osteochondritis dissecans lesion, and the progeny bone (white arrow) refers to the bone within the osteochondritis dissecans lesion.
All institutions obtained institutional review board approval with a waiver of informed consent. Pretreatment anteroposterior, lateral, and notch radiographs of 45 symptomatic knees (24 left, 21 right) containing OCD lesions of the medial or lateral femoral condyle from 42 patients (24 male, 18 female; average age: 13.8 ± 4.5 years; range, 7.1-32.0 years) from 2 institutions were uploaded to a secure research web portal and presented to raters in random order (Figure 2). Both skeletally immature and mature patients were included to assess the reliability of growth plate maturity classification. Seven orthopaedic/sports medicine surgeon raters (2 fellowship trained in pediatric orthopaedic surgery and sports medicine, 2 in sports medicine, 2 in medical pediatrics with a certification of added qualification [CAQ] in sports medicine, and 1 in general orthopaedic surgery with a CAQ in sports medicine) performed ratings at 2 time points a minimum of 1 month apart. The web portal allowed raters to zoom, pan, and measure the radiographs from any computer with Internet access but did not allow changes in contrast and brightness, which had been optimized before upload. Raters were not allowed to return to completed ratings. The web portal used branching logic to skip questions not relevant to the raters based on previous answers and then automatically populated the raters’ answers into a secure, online database. Progress was monitored in real time, and comparative progress updates were emailed to raters during the course of the study to incentivize timely completion.

Screenshot of the research web portal showing (left to right) anteroposterior, lateral, and tunnel radiograph views of the knee for each patient.
Statistical Analyses
The reliability of categorical variables (lesion location, progeny bone fragmentation [Figure 3A] and boundary [Figure 3, B and C]) was determined using free-marginal kappa (κf) 32 for interrater and Cohen kappa (κc) for intrarater reliability. The reliability of ordinal variables (growth plate maturity, parent bone radiodensity, progeny bone displacement [Figure 4], lesion contour [Figure 5], and progeny bone center radiodensity [Figure 6] and rim radiodensity) was determined using intraclass correlations (ICC) derived from 2-way mixed-effects analysis of variance (ANOVA; single measures) 17 for interrater and linear-weighted kappa (κl) for intrarater reliability. The reliability of continuous variables (knee width and lesion width and depth) was determined using ICCs derived from 2-way mixed-effects ANOVA (average measures) for inter- and intrarater reliability. Standard error is reported for Cohen and linear-weighted kappas, and 95% CIs are reported for ICCs. The criteria of Landis and Koch 23 were used for the magnitude of the reliability coefficient: 0 to 0.2 = poor, 0.21 to 0.4 = fair, 0.41 to 0.6 = moderate, 0.61 to 0.8 = substantial, 0.81 to 1.0 = excellent agreement.

(A) The progeny bone was classified as fragmented (arrow) if there were ≥2 pieces of progeny bone. (B) The boundary between the parent bone and progeny bone was classified as distinct (arrow) if there was a higher or lower density line separating them and (C) indistinct (arrow) if there was not.

(A) The progeny bone was classified as nondisplaced if the lesion was completely intact, (B) partially displaced if part of the lesion remained intact, or (C) totally displaced if the lesion had completely detached from the parent bone.

The predominant shape/contour of the articular side of the osteochondritis dissecans lesion was classified as (A) convex, (B) linear, or (C) concave.

The radiodensity of the center of the progeny bone was compared with the unaffected parent bone on all views and classified as being (A) more, (B) less, or (C) the same density.
Results
Radiographic Features With Excellent Reliability
Inter- and intrarater reliability were excellent (ICC/κ >.83) for differentiation of medial and lateral lesions and growth plate maturity as well as for measurement of knee width and lesion width and depth on anteroposterior, lateral, and notch views (Table 1). Based on majority ratings, there were 27 medial femoral condyle lesions and 18 lateral femoral condyle lesions, and the growth plate was classified as open in 25 knees, closing in 8 knees, and closed in 12 knees.
Interrater and Intrarater Reliability of Radiographic Features of Osteochondritis Dissecans Knee Lesions a
κc, Cohen kappa; κf, free-marginal kappa; κl, linear-weighted kappa.
Analysis included a subset of 16 patients with visible progeny bone as agreed by all 7 raters.
Radiographic Features With Moderate to Substantial Reliability
Anterior and posterior lesions were differentiated with moderate interrater reliability (κ = .49) and substantial intrarater reliability (κ = .72) for the 18 lesions that were classified by all raters as visible on lateral view. According to majority ratings, there were 13 posterior lesions and 5 anterior lesions. The visibility of the progeny bone was classified with moderate interrater (κ = .45) and substantial intrarater (κ = .67) reliability. Reliability analyses on radiographic features of the progeny bone were limited to the 16 cases where the progeny bone was rated as visible within the lesion on any view by all 7 raters. The fragmentation and displacement of the progeny bone were classified with moderate interrater (κ = .54, ICC = .52, respectively) and substantial intrarater reliability (κ = .64 and .80, respectively). The boundary of the progeny bone was classified with substantial interrater (κ = .62) and moderate intrarater reliability (κ = .55). The radiodensity of the progeny bone center was classified with moderate interrater (ICC = .52) and intrarater (κ = .57) reliability.
Radiographic Features With Poor to Fair Reliability
Lesion contour, when classified as convex, linear, or concave, had poor interrater (ICC = .33) and moderate intrarater reliability (κ = .53). When classified as nonconcave or concave, lesion contour improved to substantial interrater reliability (κ = .65) without compromising intrarater reliability (κ = .47). Parent bone radiodensity was classified with fair interrater (ICC = .39) and moderate intrarater (κ = .47) reliability. Combining categories for parent bone radiodensity was not beneficial as the parent bone was rarely classified as less dense by any rater. The radiodensity of the progeny bone rim was classified with poor interrater (κ = .11) and fair intrarater (κ = .32) reliability. Combining categories for progeny bone rim radiodensity was not beneficial as the progeny bone rim was rarely classified as more dense.
Classification Time Burden
The average time for each rater to rate and measure each OCD case was just under 4 minutes at 225 seconds (range, 150-318 seconds).
Discussion
This study is the first to evaluate the interrater and intrarater reliability of several radiographic features used in the diagnosis and classification of OCD femoral condyle lesions. Excellent reliability was demonstrated for 4 features (medial/lateral locations, growth plate maturity, condyle width, lesion size). Moderate to substantial reliability was demonstrated for 6 features (anterior/posterior locations, fragmentation, boundary, displacement, central radiodensity, and contour). Poor to fair reliability was demonstrated for 2 features (the radiodensity of both the lesion bone rim and the surrounding epiphyseal bone). Identifying reliable radiographic features will be critical to allow multicenter study of this condition. The relative rarity of this condition will require multicenter collaborations to develop algorithms to predict healing, which are critical to develop optimal treatment programs.
Previous research has shown inconsistent results on healing rates of lateral versus medial femoral condyle lesions in children.3,18,22,31,33 Hefti et al 18 found lesions of the medial femoral condyle to have a better prognosis than lesions of the lateral femoral condyle. Other prognostic studies3,22,31 have not found a difference. Physicians in our study were able to differentiate medial and lateral OCD lesions on radiographs with excellent reliability. Discrimination of anterior and posterior OCD lesions on the lateral view was less reliable, likely because many lesions were considered not visible on lateral view or were located on the border between the anterior and posterior condyle. MRI may be a better option for determining the OCD location in the lateral/sagittal plane. Growth plate immaturity has previously been correlated with healing9,18,26 and was classified based on knee radiographs with excellent reliability in the current study. The reliability was better than that reported for the Risser grading system 28 or the manual version of the Tanner-Whitehouse method. 29 Other studies have found that bone age assessments using knee radiographs are less influenced by height and body mass index than hand radiographs. 1
Larger lesion size has been identified by 2 previous studies as a negative predictor of healing at 6 or 12 months after nonoperative treatment in children.22,31 Our study supports the applicability of measurement of lesion size on plain radiographs (both direct measurement and normalized to the patient’s knee size), as all measurements had excellent reliability between raters and across ratings.
The presence of a sclerotic rim has also been associated in previous work with a worse prognosis in pediatric patients. 27 The radiodensity of the rim of the parent bone adjacent to the OCD lesion was considered a measure of sclerosis in our study and demonstrated only fair to moderate reliability. Reliability may be improved by clarifying the question from “what is the radiodensity of the rim of the parent bone” to the more straightforward question of “what is the sclerotic rim density.” Future research is warranted to better clarify the precise significance of the sclerotic rim and the reliability of its assessment on radiographs.
Features of the intact OCD lesion (progeny bone), such as fragmentation, displacement, and boundary, have not been assessed as predictors of healing in previous studies but did demonstrate moderate to substantial reliability in our study. The contour of the lesion was classified with greater reliability using the binary categories of concave versus nonconcave than the tertiary categories of concave, linear, and convex. The comparative radiodensity of the rim of the OCD lesion’s progeny bone demonstrated insufficient reliability for use in a classification system and should be dropped from predictive model testing.
OCD lesions of the knee are currently classified using several 4- and 5-stage systems, including the Berndt and Harty 2 classification system for plain radiographs; the De Smet et al, 8 Hefti et al, 18 Dipaola et al, 10 and Chen et al 7 classification systems for MRI; and the Guhl 15 and Ewing and Voto 14 classification systems for arthroscopy. None of these classification systems have undergone rigorous reliability testing. While 4- and 5-stage systems may provide good recall in a clinical setting, a staging system’s value is based on its ability to accurately guide diagnosis and treatment and to predict the final outcome of a specific treatment. When compared with arthroscopy, the accuracy of these radiographic and MRI staging systems range considerably (30%-100%),7,10,19,20 and their predictive ability is currently poorly understood. For example, the majority of pediatric patients with femoral condyle OCD present in stage 1 (according to De Smet et al 8 ), which is typically treated nonoperatively. Outcomes after nonoperative treatment vary widely, with rates of healing ranging from 50% to 94% in the literature.12,26,31 These simple staging systems currently do not discriminate between treatment options for children within a stage.
Instead of using a staging system, our multicenter research group recommends development of a multifactorial feature-based classification system. Features may include aspects of the patient’s medical history, symptoms, physical examination, radiographs, MRI, and arthroscopy, and the features that physicians can reliably agree on can be tested in a multivariate predictive model of healing.
There are several potential limitations to this study. First, all physician raters were members of a research study group focused on OCD and may not be representative of general, pediatric, or sports medicine physicians. Second, the radiographic technique may have varied between the 2 institutions, such as the knee flexion angle on the notch view. However, it is unlikely that these differences would affect reliability since all physicians rated the same images. Moreover, including radiographs from 2 institutions may also increase generalizability. Third, the quality of the images, including brightness and contrast, could not be altered by raters as they could in the clinic setting. However, these features were optimized before classification by the research design team. Fourth, raters also had the option to view images on personal computers that may not have as high of resolution as high-definition radiology monitors. However, this may reflect an increasingly common reality of clinical practice for many providers who use mobile digital technology. Finally, because only 16 of the 45 knees showed bone within the OCD lesion, reliability may be underpowered for some features, such as those relating to the progeny bone. A more robust determination of the reliability of the features assessed in the current study might be achieved with a larger sample. One important next step is also to determine how accurately these features can be classified using radiographs in comparison to arthroscopy and/or MRI.
This study identified several radiographic features of OCD femoral condyle lesions with high inter- and intrarater reliability. The radiographic characteristics of the OCD that have high rater reliability should be tested in a predictive model of time to healing and ultimate healing rates, given the fact that one of the most common questions that parents and patients ask a sports medicine physician during their first clinic visit is when they can return to sports. This study lays the groundwork to better answer questions such as these through the development and validation of a prognostic model to predict and improve OCD lesion healing rates. Future research is needed to test the reliable radiographic features identified in this study in multifactorial models to determine independent predictors of OCD lesion healing.
Footnotes
Contributing Authors
M. Lucas Murnaghan, MD, MEd (Division of Orthopaedics, Hospital for Sick Children, Toronto, Ontario, Canada); Carl Nissen, MD (Connecticut Children’s Medical Center, Farmington, Connecticut, USA); Jennifer Weiss, MD (Kaiser Permanente, Los Angeles, California, USA); Rick Wright, MD (Department of Orthopaedic Surgery, Washington University School of Medicine at Barnes-Jewish Hospital, St Louis, Missouri, USA); Gregory D. Myer, PhD (Division of Orthopaedic Surgery and Division of Sports Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio, USA; Micheli Center for Sports Injury Prevention, Waltham, Massachusetts, USA; Department of Pediatrics and Orthopaedic Surgery, University of Cincinnati, Cincinnati, Ohio, USA; Sports Medicine Sports Health & Performance Institute, The Ohio State University, Columbus, Ohio, USA).
Presented at the 40th annual meeting of the AOSSM, Seattle, Washington, July 2014.
One or more of the authors has declared the following potential conflict of interest or source of funding: This study was supported by an AOSSM Smith & Nephew Grant.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
