Abstract
Purpose
Radiology descriptions of intracranial metastases following radiotherapy are often imprecise. This study sought to improve such reports by creating and disseminating a structured template that encourages discrete categorization of intracranial lesions.
Methods
Following initiation of the structured template, a retrospective review assessed patients with intracranial metastases that underwent radiotherapy, comparing ‘pre-template’ with ‘post-template’ reports. A total of 139 patients were included; 94 patients (67.6%) were imaged pre-template, 45 (32.4%) post-template. Reports were assessed for discrete versus non-specific descriptions of lesions: ‘(presumed) new metastases’, ‘treated metastases’, and ‘indeterminate lesions’. Non-specific language was subdivided based on the type of lesion(s) described: e.g. ‘stable enhancing foci’ was deemed a non-specific description of ‘treated metastases’.
Results
Non-specific descriptions of lesions were used in 25/94 reports (26.6%) pre-template, and eight reports (17.8%) post-template. No significant difference was found in the frequency of inappropriate/ambiguous descriptions of intracranial lesions following template initiation (P = 0.52). However, only 27/45 (60.0%) of the reports in the post-template time period used the structured report; the other reports were written as free prose. Of the reports that did use the structured template, the authors used significantly less ambiguous language structured template (P = 0.02).
Conclusion
When utilized, a structured report template resulted in decreased non-specific descriptions and improved discrete characterization of intracranial metastases in patients treated with radiation. However, the frequency of non-specific language usage before and after template initiation was unchanged, probably due to poor compliance with template utilization.
Introduction
In radiology in general, and in oncological radiology in particular, there is a need for objective characterization of pertinent findings.1–5 Traditional free text or ‘prose’ format reports often suffer from inconsistencies in both language and style, which can ultimately impact the clarity of communication between radiologists and referring physicians. 6 Recently, there has been a push for the increased use of structured radiology templates.7,8 Such templates both encourage the incorporation of pertinent imaging findings, and encourage the use of a more standardized lexicon to describe disease-specific findings.5,6
Characterization of intracranial metastases following radiation therapy presents a uniquely challenging task for radiologists. 9 Patients with multiple lesions may have a combination of successfully treated metastases, progressive disease, and indeterminate findings within the same examination.10,11 The interpretation of such studies can require intensive research into patients' histories and comparison with multiple prior examinations. 11 Radiology reports are frequently non-specific, lacking sufficient clarity to be of use to the referring physicians. Non-specific description of intracranial metastases can be a particular source of consternation to referring physicians, whose treatment planning depends on precise definitions of new, treated, and indeterminate lesions.
Therefore, this study aimed first to implement and then disseminate a structured report template for use in patients with intracranial metastases who have undergone radiation therapy. The report template categorized intracranial lesions into one of three distinct entities: (a) ‘new metastasis’, defined as being a new enhancing lesion remote from a treatment site; (b) ‘treated metastasis’, defined as stable and/or decreased size of a previously treated lesion; and (c) ‘indeterminate, disease progression versus treatment related changes’, defined as increased size of a previously treated enhancing lesion. The purpose of this study was subsequently to compare the precision with which radiology reports were written before and after the template initiation.
Methods
Patient selection
Institutional review board approval was obtained for the purposes of this study. Following the initiation of the structured report, a retrospective review was performed of all patients with known intracranial metastases who underwent magnetic resonance imaging (MRI) between 1 January 2017 and 26 February 2018. Patients were included who had: (a) prior treatment of intracranial metastases with gamma knife radiotherapy; and (b) contrast-enhanced magnetic resonance with sequences suitable for the evaluation of malignancy. Exclusion criteria were: (a) radiotherapy completed for a prior central nervous system (CNS) malignancy (e.g. meningioma, astrocytoma); (b) studies completed immediately after postoperative resection of an intracranial metastasis to assess for margins, complications, etc.; (c) limited magnetic resonance, not suitable for malignancy evaluation (e.g. stealth MRI for radiation planning, quick-stroke protocol MRI); and (d) suboptimal image quality limiting readers' abilities to assess intracranial findings. A total of 150 patients with known intracranial metastases underwent MRI between 1 January 2017 and 26 February 2018, of which 11 were excluded as they were imaged in the predefined transition period of 11 November 2017 and 20 November 2017. The same pool of radiologists read reports in the pre and post-template periods.
Development and initiation of report template
A structured report template was implemented within our institution, to be used in examinations completed for patients with metastatic disease to the CNS that had undergone prior radiation therapy. The template was designed by a neuroradiologist within our department, using feedback from several other neuroradiologists and radiology residents. The report separated any intracranial metastases into one of three categories: (a) ‘(presumed) new metastases’: new enhancing intracranial lesions, remote from prior gamma knife treatment site; (b) ‘treated metastases’: stable or decreased size of previously treated lesions, considered unchanged disease burden or positive treatment response; and (c) ‘indeterminate’: increased size of a previously treated enhancing intracranial lesion, considered to be uncertain if it represented an enlarged metastasis and/or radiation necrosis. The ‘findings’ section of the report template also included areas for the radiologist to provide pertinent information: date(s) of prior stereotactic radiosurgery, date(s) of prior surgical resection, and the presence or absence of leptomeningeal involvement.
The structured template was implemented on 11 November 2017. Radiologists at our institution were informed of the template over the following 10 days (11–20 November 2017).
Data collection
Reports were dichotomized into ‘pre’ and ‘post’ initiation of the structured template: ‘pre-template’ dates were defined as 1 January 2017 to 10 November 2017, and ‘post-template’ dates were defined as 21 November 2017 to 26 February 2018. Reports between 11 November 2017 and 20 November 2017 were excluded, as these were considered to be during the transition time in which the institution's radiologists were informed of the new practice.
All reports were assessed for the use of ‘appropriate’ descriptions of intracranial findings based on the categories defined in the structured template: (a) ‘new metastases’; (b) ‘treated metastases’; and (c) ‘indeterminate’ or ‘possible radiation necrosis versus increased size of metastases’. Language that was deemed similar enough to these categories to remain of clinical value was also accepted, as detailed in Table 1. For example, ‘stable metastases’ was accepted as a description of unchanged disease burden, and ‘decreased size of metastases’ was accepted as a presumed positive response to treatment. Language deemed inappropriately ambiguous, conversely, was noted to be ‘inappropriate’: examples include ‘increased size of metastatic lesion’ (inappropriate because the language does not include the possibility of radiation necrosis) and ‘stable enhancing foci’ (inappropriate as it acknowledges neither the malignancy nor the prior treatment). When possible, inappropriate descriptions of lesions were subdivided into one of three of the above categories: e.g. ‘stable enhancing foci’ was specifically considered an ambiguous description of ‘treated metastases’, while ‘increased size of enhancing lesion’ was considered an ambiguous description of an ‘indeterminate’ lesion.
Comparison of reports before (‘pre-template’) and after (‘post-template’) the dissemination and initiation of the structured report template within the radiology department. No significant difference was noted in the overall use of ambiguous language before and after template initiation (P = 0.52). A significantly greater proportion of reports included descriptions of leptomeningeal involvement as a pertinent positive or negative and included history of prior radiotherapy within the report (P < 0.0001 and P = 0.0005, respectively).
Reports were also assessed for whether the structured template was used, if the reports included the presence or absence of leptomeningeal involvement, and if the patient's history of prior radiotherapy was included within the ‘history’ section of the report.
Statistical analysis
Comparison P values were calculated using a generalized estimating equation in the logistic form. A P value of less than 0.05 was considered statistically significant. Statistical analysis was performed using statistical analysis software (SAS), version 9.4 (SAS Institute Inc., Cary, NC, USA).
Results
Patient population
A total of 139 patients were included in the cohort; 88 women (63.3%). Ninety-four patients (67.6%) were imaged in the pre-template time period; 45 (32.4%) were imaged post-template initiation. The average patient ages pre and post-template were 59.4 ± 12.0 and 61.6 ± 10.4, respectively. Of patients' primary malignancies, the most common was lung (n = 40), followed by breast (n = 31), melanoma (n = 27), and renal cell carcinoma (n = 14).
Effect of structured report on lesion categorization
Inappropriate/ambiguous descriptions of intracranial lesions were used in 25 reports (26.6%) pre-template, and in eight reports (17.8%) post-template. Indeterminate lesions were described inappropriately in 13/32 reports pre-template (40.6%) and in six of 20 reports post-template (30.0%). Treated metastases were described inappropriately in 13/74 reports pre-template (17.6%), and in two of 32 reports post-template (6.3%). Of the 13 reports with inappropriate descriptions of treated metastases, seven had mixed appropriate and inappropriate descriptions of lesions (i.e. some lesions were correctly described, whereas others were described with ambiguous language). None of the reports inappropriately described new metastases. Leptomeningeal involvement (or lack thereof) was mentioned in seven of 94 (7.5%) of pre-template reports and in 22/45 (48.9%) of post-template reports. A history of radiotherapy was mentioned in 53/94 (56.4%) of pre-template reports, and in 32/45 (71.1%) of post-template reports.
Following the initiation of the structured report, 27/45 of the reports (60.0%) used the structured template. Inappropriate/ambiguous language was used in two of 27 reports (7.4%) that used the structured template, and in six of 18 (33.3%) of the unstructured reports. Again, none of the reports described new metastases inappropriately. Regarding descriptions of treated metastases, one of 23 reports (4.4%) that used the structured template included inappropriate language, whereas one of nine (11.1%) of the unstructured reports included inappropriate descriptions. Indeterminate lesions were inappropriately described in one of 11 reports (9.1%) that used the template, and in five of nine (55.6%) unstructured reports. Leptomeningeal involvement (or lack thereof) was described in 22/27 (81.5%) of reports that used the structured template, and in none of 18 (0.0%) unstructured reports. A history of prior radiation therapy was included in 27/27 (100.0%) of template reports, and in five of 18 (27.8%) of unstructured reports.
Comparing the reports written before and after the initiation of the template, no significant difference was noted in the overall use of inappropriate descriptions of lesions (P = 0.52), between the frequencies of ambiguous descriptions of treated metastases (P = 0.27), or between the frequencies of ambiguous descriptions of indeterminate lesions (P = 0.75) (Figure 1). Reports in the post-template period mentioned leptomeningeal involvement (or lack thereof) and history of prior radiation therapy significantly more than pre-template reports (P < 0.0001 and P = 0.0005, respectively).
Regarding the post-template period, reports that used the structured template had significantly fewer ambiguous descriptions of lesions compared to unstructured reports (P = 0.02), although no significant difference was specifically noted between descriptions of treated metastases or indeterminate lesions (P = 0.31 and P = 0.06, respectively) (Figure 2). Among the post-template period reports, leptomeningeal involvement (or lack thereof) and history of prior radiotherapy were mentioned significantly more in reports that used the structured template compared to the unstructured reports (P < 0.0001 for both).
Comparison of reports following the initiation of the template, subdivided based on radiologists’ use of the structured template or use of the unstructured/free prose template. Reports written using the structured template had significantly less use of ambiguous language (P = 0.02), although no difference was noted specifically between descriptions of ‘treated metastases’ or ‘indeterminate lesions’ (P = 0.31 and P = 0.06, respectively).
Discussion
This study set out to assess the impact of a structured radiology report template for patients with intracranial metastases status post gamma knife therapy. The results indicate that the use of a structured radiology template to describe lesions after radiotherapy led to improved categorization of lesions, and decreased use of ambiguous language. However, the results also noted that no significant difference was found in the use of ambiguous language before and after the initiation of the structured report template. This is related to poor compliance/use of the structured reports, although it is possible that variations in patient groups, reading radiologists, or bias introduced by the assessment of report language played a role.
The most crucial improvement noted using the structured template was the more concrete categorization of intracranial metastases. The use of ambiguous terminology is a common problem in radiology reports. 12 For example, Lee et al. found a significant discrepancy between radiologists’ and non-radiologists’ interpretations of vague phrases such as ‘consistent with’, ‘evidence of’, and ‘compatible with’. 13 Prior to the template initiation, the most common ambiguous description of intracranial lesions was ‘increased size of enhancing foci’, which omitted the possibility of radiation necrosis. The observed improvements following the dissemination of the template were likely to be due to a number of reasons: the force-field template format discouraged characterization of lesions outside of one of three predetermined categories, and the report design left little room for free prose text. In addition, the template probably served as a type of checklist reminder for a list of findings for the radiologist to assess, a mechanism proposed by prior authors. 1 For example, use of the template led to better reporting of prior radiation therapy by encouraging radiologists to describe precise treatment dates, a benefit of structured reporting that has previously been described. 14 However, it is also possible that improvements may have also been at least partially due to an overall increased awareness of this subject within our department related to the template's dissemination.
Besides being clear and accurate, a radiology report should also be written in a way that meets a referring provider's preferences. 12 Prior studies have found that ordering providers believe that the use of structured reports improves the transmission of clinical information, and that referring providers find structured reports to have superior clarity and clinical utility.15,16 For example, the use of structured report templates in rectal cancer MRIs led to both increased satisfaction and confidence of report accuracy of the referring surgeons.17–19
Various authors have predicted, and championed, the move towards structured templates as the mainstay for radiology reports.19,20 Already many benefits of structured reporting have been recognized: reports have improved reproducibility, superior organization, and can be more readily mined for informatics data.14,21 Report accuracy, too, may benefit: the use of a checklist-style report was found to decrease the missed non-fracture findings by residents reading cervical spine computed tomograms, and a report with dropdown options improved adherence to prostate imaging reporting and data system (PI-RADS) in patients with prostate cancer.22,23
Nevertheless, structured templates have also received several criticisms. Reports with incomplete sentences have more typographical errors due to the limitations of voice recognition software. 24 Some radiologists feel that the constraints of structured reports may inappropriately simplify conclusions in patients with complex clinical findings. 25 Others state that the use of structured reports is too time consuming. 25 In the current study, although the use of a structured template led to more precise descriptions of intracranial metastases among the patients in the post-template cohort, the overall use of ambiguous language did not significantly change between pre and post-template groups, probably related to poor (60.0%) compliance with the template. This suggests that although structured templates may be beneficial in the appropriate setting, radiologist preferences of free prose text, lack of training, or lack of incentive to change may be roadblocks to adherence to structured reports. 2 Ultimately, the use of free text and/or structured reports may depend on the clinical scenario and preferences of the referring doctor and radiologist.
There are a number of limitations to this study, including its retrospective nature, small patient cohort, and single institution setting. In addition, a relatively short time (10 days) was allowed for the dissemination of the structured report template throughout the radiology department. Hence the suboptimal compliance of the use of structured reports in the post-template period may have been partially related to the incomplete adoption of the template by the institution's radiologists after the transition period. Next, although the template was designed by a neuroradiologist, it was not validated by an independent neuroradiologist prior to its implementation. In addition, the retrospective design of the study did not allow for the radiologists to be surveyed; hence, it is uncertain if template non-compliance was due to radiologist preference or simply lack of awareness of the template's existence. Furthermore, it is unknown whether the use of structured reports led to differences in treatment strategies. Future studies could explore if the use of structured reports in the setting of intracranial metastases affects patient management or patient outcomes. Finally, no structured report can account for all possible situations encountered in diagnostic imaging. When imaging findings do not clearly fit into one of the predetermined categories, radiologists must be ready to use clinical judgment to guide the formation of a report.
Conclusion
Examples of acceptable and ambiguous language for each category.
Supplemental Material
Supplemental material for The value of structured radiology reports to categorize intracranial metastases following radiation therapy
Supplemental Material for The value of structured radiology reports to categorize intracranial metastases following radiation therapy by John Benson, Matthew Burgstahler, Lei Zhang and Matthew Rischall in The Neuroradiology Journal
Footnotes
Author contribution
This article is not under consideration for publication elsewhere. All authors have participated sufficiently to take public responsibility for the content.
Funding
None was utilized to produce the manuscript.
Conflict of interest
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
