Abstract
This study identifies the treatment outcome domains used in recently published studies on the treatment of hand fractures and joint injuries with the aim to inform development of a core outcome set. Seven databases were searched from January 2014 to March 2019 for randomized and quasi-randomized studies and large prospective observational studies. We identified 1777 verbatim outcomes in 160 eligible studies. From the verbatim outcomes we distinguished 639 unique outcomes, which we categorized into 74 outcome domains based on the World Health Organization International Classification of Functioning, Disability, and Health framework. The primary outcome was appropriately identified in only 65% (72/110) of randomized and quasi-randomized controlled trials. Of the 72 studies with a primary outcome identified, 74% (53/72) had an appropriate power calculation. The vast heterogeneity in outcome selection across studies highlights the need for a core outcome set of what outcomes to measure in future clinical research on hand fractures and joint injuries.
Introduction
The recent James Lind Alliance Priority Setting Partnership on common conditions affecting the hand and wrist incorporated the opinions of patients, carers and clinicians. It highlighted the need for research to answer uncertainties concerning both the treatment of injuries in the hand and wrist and the methods of best assessing patient outcomes from treatment (James Lind Alliance, 2017). Outcome selection is a fundamental aspect of clinical research. However, when different researchers select outcomes independently, there is the risk of inconsistency in outcomes used across studies. There is also the risk that researchers omit outcomes that may be of priority to patients themselves (Kirwan et al., 2003).
A core outcome set (COS) is an agreed minimum set of outcomes that should be measured and reported in all clinical trials or research studies in a specific area of health (Williamson et al., 2017). It should have input from key stakeholders, including healthcare professionals and patients. Use of a COS increases consistency across studies, allowing more trials to be included in future meta-analyses and helping to measure clinically relevant outcomes. Selective reporting bias is also reduced since it becomes apparent if COS outcomes are not fully reported.
The aims of this systematic review were to: (1) Identify and map the outcome domains measured in recent clinical studies of hand fractures and joint injuries. (2) Assess selective outcome reporting bias in these studies. (3) Compare outcome domains reported on the treatment of patients with distal radial fractures (DRFs) versus other hand and wrist injuries (non-DRFs). Epidemiological studies have indicated a difference in the typical age and sex distribution of the patient populations of DRF and non-DRF injuries (Karl et al., 2015; Van Onselen et al., 2003; Van Staa et al., 2001). Such differences may influence outcome selection by researchers.
Methods
The design of this systematic review was guided by the Core Outcome Set-STAndards for Development (COS-STAD) (Kirkham et al., 2017). The protocol was prospectively registered on the PROSPERO international prospective register of systematic reviews (CRD42019126299).
Scope and eligibility criteria
We defined hand fractures and joint injuries as phalangeal, metacarpal, carpal and DRFs (with or without distal ulna) and an injury to a joint between any of these bones. These injuries included dislocation, subluxation, volar plate injury, avulsion injury, ligamentous tears/sprains/ruptures and closed tendon ruptures/tears. We excluded complex hand injuries (i.e. ‘mangled hand’, amputations requiring replantation), primary nerve injuries, burns and open tendinous injuries, as such injuries likely have very different outcome domains of interest.
Study types included randomized controlled trials (RCTs) and pilot or feasibility studies, quasi-randomized controlled trials (qRCTs) and prospective observational studies with 100 or more patients. Detailed inclusion and exclusion criteria are reported in Online Table S1.
Study identification
We compiled search strategies under the guidance of an information specialist experienced in the hand surgery literature (DG). Key search strategy concepts were: (1) bones, joints, tendons and ligaments of the hand, carpus and distal radius. (2) Generic terms for fractures and joint injuries. (3) Specific hand fracture and joint injury terms. We combined (1) and (2) with AND, then added to these by combining with (3) using OR. We identified relevant free-text terms and subject headings for each database. Databases searched were Ovid MEDLINE, Ovid Embase, Cochrane Central Register of Controlled Trials (CENTRAL), PubMed, CINAHL (EBSCO), PEDro and Ovid PsycINFO.
We conducted a staged search strategy, as outlined in the COMET Initiative handbook (Williamson et al., 2017), with an initial search run on 29 March 2019. An example of the search strategy and descriptions of the staged search method and study selection process are provided in Online Appendix S1.
Risk of bias assessment
We determined the outcomes captured by studies rather than the quantitative results obtained. However, selective outcome reporting can offer insight into which outcomes authors truly prioritize. Kirkham et al. (2010) describe an outcome matrix for the assessment of outcome reporting bias based on the premise that any outcome specified for inclusion should be reported in the final publication. We used a modified version of this, as summarized in Online Table S2.
We deemed the primary outcome to be one of the following (in descending order). (1) The outcome upon which the study sample size calculation was based. (2) The primary outcome specified in the study. (3) The outcome that appeared to correspond most closely with the study aim. If there was no clear primary outcome, we considered all outcomes in the study as secondary outcomes. We performed independent two-reviewer assessment of outcome reporting status (SRD for all outcomes, second assessment divided between CM and BAM). We excluded generic ‘adverse event’ or ‘complication’ outcomes from the assessment except in cases where specific named complications were identified as being standalone study outcomes.
Data synthesis
We analysed all extracted verbatim outcomes for similarity in meaning through discussion (SRD for all and either CM or BAM). ‘Verbatim outcome’ means the literal outcome. For example, ‘finger flexion’ and ‘flexion of the finger’ would technically constitute different ‘verbatim outcomes’, but one unique outcome if measured in the same way. We split verbatim outcomes with similar terminology but different meaning into two unique outcomes where results for these outcomes could not be reasonably pooled in a meta-analysis. For example, ‘finger flexion’ constitutes two unique outcomes if reported in degrees of joint movement in some studies but as a percentage compared with the contralateral limb in others. We categorized unique outcomes into domains based on the World Health Organization International Classification of Functioning, Disability, and Health (WHO ICF) framework (World Health Organization, 2001) by using the WHO ICF linking rule guide (Cieza et al., 2005).
We analysed patient-reported outcome measure (PROM) instruments by categorizing the individual items and components of any scales into WHO ICF outcome domains (Macefield et al., 2014). Time points of outcomes are often heterogeneous. To determine ‘meaningful’ heterogeneity resulting from use of multiple and varying time points for outcome assessment, we created time point ‘ranges’ representing typical ‘follow-up windows’ and categorized our findings according to these ‘ranges’.
Results
The PRISMA study flow diagram is shown in Figure 1 (Moher et al., 2009). A table of all 160 included studies is provided in Online Appendix S2. Most studies were single-centre and based in Europe and Asia (Table 1). A total of 20,228 participants were recruited from 39 countries. Most studies were RCTs. The primary outcome was appropriately identified in 65% (72/110) of RCTs and qRCTs. Of those 72 studies with a primary outcome identified, 74% (53/72) had an appropriate power calculation.
PRISMA flow diagram. Study characteristics. n: number of studies within each category of a characteristic; N: total number of studies for which data were available for the characteristic. Percentages differ from 100% due to rounding effects.
Outcomes
There were 1777 verbatim outcomes. The number of outcomes reported per study varied from 1 to 36, with a median of ten outcomes (interquartile range 6 to 14). Verbatim outcomes were deduplicated and rationalized to 639 unique outcomes. Of these unique outcomes, 71% (456/639) were used in only a single study, 20% (128/639) were used in two to four studies and 8.9% (57/639) were used in five or more studies.
Clinicians and healthcare professionals were the outcome assessors for 66% (1181/1777) of verbatim outcomes (Figure 2). There was heterogeneity in time point ‘range’ for outcome assessment as summarized in Table 2. The modal time point ‘range’ was 6 weeks to 6 months (28% of verbatim outcomes, 1109/3936).
Pie chart demonstrating number and proportion of outcomes by assessor category. Time points of verbatim outcomes. n: number of verbatim outcomes within a given time point range; N: total number of different verbatim outcomes when accounting for time point at which outcome was assessed/measured.
Outcome domains
We mapped the 639 unique outcomes to 74 outcome domains using the WHO ICF framework (World Health Organization, 2001). The presence of each outcome domain in individual studies was noted. Further details are depicted in Online Appendix S3.
While many of the unique outcomes linked to a single WHO ICF domain, some (in particular PROMs) linked to multiple domains. Certain outcomes did not map onto the framework at all, the most common being adverse events/complications (58% of studies, 93/160), patient satisfaction (24% of studies, 38/160) and bone healing (23% of studies, 36/160).
Comparison of DRFs and non-DRF studies
Participant age and sex distribution in distal radial fracture (DRF) and non-DRF studies.
Top five PROMs and ten outcome domains most commonly used across all studies and in distal radial fracture (DRF) vs non-DRF studies.
DRF: distal radial fracture; PROM: patient-reported outcome measure; VAS: visual analogue scale; DASH: disability of the arm, shoulder and hand; PRWE: patient-rated wrist evaluation; QuickDASH: abbreviated version of DASH; EQ-5D-3L: EuroQOL-5D-3L tool.
Outcome reporting bias
Figure 3 depicts the reporting status of outcomes across the different study types, with RCTs and qRCTs subdivided based on trial registration status. This reflects the reporting bias for these outcomes. Of the RCTs and qRCTS, only 20% (22/110) were prospectively registered. Fewer than half of the outcomes in RCTs and qRCTs and two-thirds in prospective observational and randomized pilot/feasibility studies were ‘completely’ reported.
Cumulative bar chart showing percentage and number of outcomes within each reporting status category across study types.
Discussion
This review reveals several fundamental methodological issues in outcome selection for clinical research on hand fractures and joint injuries. It is important to raise awareness of these issues among hand surgeons, who will form a key stakeholder group in any future consensus work. A wide range of heterogeneous outcome domains and outcome time points are reported in the recent literature on hand fractures and joint injuries. Such variation hinders meta-analysis and predisposes to ‘research waste’ (Ioannidis et al., 2014; Yordanov et al., 2018).
The high number of unique outcomes is partially explained by the broad scope of injuries being covered. However, even at the more fundamental outcome domain level we identified 74 distinct domains. Only three domains were reported in over 75% of the studies; sensation of pain, mobility of joint functions (range of movement) and muscle power function (grip/pinch strength, performing certain actions). Even these were measured in a variety of ways and at various time points, hindering or precluding meta-analysis.
A prior study limited to a small selection of journals found that objective clinical measures (e.g. grip strength, range of motion, functional status), quality of life and morbidity were the most common outcomes assessed (Chung et al., 2006). Weinstock-Zlotnick and Mehta (2016) reported on outcomes for wrist fractures and ligament injuries from RCTs between 2005 and 2015. Though lacking details in terms of WHO ICF outcome domains, they found range of movement, grip strength and pain were the most common physical outcome measures used, while DASH and PRWE (patient-reported wrist evaluation) were the most common PROMs. Their findings are in broad agreement with ours, indicating that priorities in outcome selection for studies preceding our search window were similar.
Goldhahn et al. (2014) undertook a literature review as part of a process, which aimed to establish a core set for DRF. Though highlighting some commonly used outcomes, they did not present detail on the heterogeneity of outcomes identified. They found that radiological outcomes (e.g. healing and alignment), grip strength, range of motion and pain were most common, present in 68%, 49%, 49% and 38% of studies, respectively. The pain outcome was used much less commonly than the near-universal use we found. Their reported frequency of radiological outcomes was higher than we found, but this is because they combined outcomes that we considered distinct domains of ‘healing’ (bone healing) and ‘alignment’ (structure of upper extremity).
We compared outcome selection in DRF and non-DRF studies and found considerable overlap. Though the rank order of most common PROMs and outcome domains varied slightly, the top five PROMs and top ten outcome domains were the same (Table 4). Hence similar outcomes appear to be considered relevant to both populations. However, most PROMs reflect multiple domains, giving rise to greater apparent overlap. The most common multi-domain PROMs used were DASH (Hudak et al., 1996), PRWE (MacDermid et al., 1998) and QuickDASH (Beaton et al., 2005). DASH captures all of the ten most common outcome domains, while PRWE and QuickDASH each capture eight of the ten most common domains used (except for mobility of joint functions and muscle power functions).
The International Committee of Medical Journal Editors has deemed prospective trial registration in a public registry a condition for publication since 1 July 2005 (De Angelis et al., 2004). The updated Consolidated Standards of Reporting Trials statement (CONSORT) in 2010 contains clear recommendations for registration and outcome reporting (Schulz et al., 2010). Despite these standards being set, Lee et al. (2018) found that only 31% (28/90) of RCTs on DRFs were registered. Only 16 trials specified a primary outcome measure at registration, and seven of these ended up reporting either a different/additional primary outcome or none at all.
We found marked selective reporting bias in the recent literature of hand fractures and joint injuries, which is in agreement with previous studies of different populations. Many outcomes were not reported at all, despite being specified in the publication or trial registration. Multiple others were reported incompletely with only a brief comment or lacking sufficient detail for meta-analysis. All represent non-adherence to reporting standards.
We also found unexpected outcomes, with duration of surgery being the most common. The prospectively registered studies had a lower proportion of ‘unexpected’ outcomes as compared with retrospectively registered trials. It is possible that prospective registration correlates with a higher methodological quality in general, which is reflected in this marker of outcome reporting bias. An assessment of overall study design and risk of bias across all domains was beyond the primary scope of this study. Other reviews of hand fractures and joint injuries have highlighted issues of inadequate outcome assessment and large variation in reported outcomes (Handoll and Vaghela, 2004; Poolman et al., 2006; Verver et al., 2017). This review specifically quantifies the magnitude of the problem.
One limitation of this review was the exclusion of studies for which a publication in English could not be obtained (n = 22, Figure 1). However, for almost every country of origin where this occurred there were other studies with an English publication available maintaining some representation of these countries in the review. A theoretical limitation was the date range used, but we made this choice to focus on outcomes used in the more recent literature through a staged search approach, as recommended by the COMET Initiative (Williamson et al., 2017).
This review contributes to a longlist of outcome domains, laying the foundations for COS development. The next step is to formally and extensively explore the patients’ perspective through interviews and focus groups with those who have direct experience of these injuries. Information from both will be processed through consensus work in the form of a Delphi study and a final consensus meeting. Key stakeholders will be involved throughout to develop a COS of what key outcomes should always be reported in all future studies of the treatment of hand fractures and joint injuries, with the aim of improving the evidence base that guides clinical practice.
Supplemental Material
sj-pdf-1-jhs-10.1177_1753193420983719 - Supplemental material for Developing a core outcome set for hand fractures and joint injuries in adults: a systematic review
Supplemental material, sj-pdf-1-jhs-10.1177_1753193420983719 for Developing a core outcome set for hand fractures and joint injuries in adults: a systematic review by Sandeep Rajiv Deshmukh, Christos Mousoulis, Ben A. Marson, Douglas Grindlay, Alexia Karantana and the Core Outcome Set for Hand Fractures and Joint Injuries in Adults Group* in Journal of Hand Surgery (European Volume)
Footnotes
Acknowledgements
The Core Outcome Set for Hand Fractures and Joint Injuries in Adults Group members: Sandeep Rajiv Deshmukh1, Douglas Grindlay1, Christina Jerosch-Herold5, Alexia Karantana1, Paul Leighton3, Ben A Marson2, Alan A Montgomery4, Christos Mousoulis1, Jeremy N Rodrigues6, Ryan Trickett7. 1Centre for Evidence Based Hand Surgery, Academic Orthopaedics, Trauma and Sports Medicine, Division of Rheumatology, Orthopaedics and Dermatology, School of Medicine, University of Nottingham, UK 2Trauma Outcomes Group, University of Nottingham, Queen’s Medical Centre, Nottingham, UK 3Centre of Evidence Based Dermatology, Faculty of Medicine and Health Sciences, School of Medicine, University of Nottingham, UK 4Nottingham Clinical Trials Unit, Faculty of Medicine and Health Sciences, School of Medicine, University of Nottingham, UK 5School of Health Sciences, University of East Anglia, UK 6Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, & Stoke Mandeville Hospital, Buckinghamshire Healthcare NHS Trust, Aylesbury, UK 7Cardiff and Vale University Health Board, University Hospital of Wales, UK
Declaration of conflicting interests
The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: this research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. SRD is funded by PhD studentship at the CEBHS, University of Nottingham, which receives funds from the British Society for Surgery of the Hand. BAM is funded by a National Institute for Health Research (NIHR) Doctoral Fellowship [NIHR300240] and JNR is funded by a National Institute for Health Research (NIHR) Postdoctoral Fellowship [PDF-2017-10-075]. This article presents independent research funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
