Do Sideline Tests of Vestibular and Oculomotor Function Accurately Diagnose Sports-Related Concussion in Adults? A Systematic Review and Meta-analysis

Abstract

Background:

Sports-related concussion (SRC) assessment tools are primarily based on subjective assessments of somatic, cognitive, and psychosocial/emotional symptoms. SRC symptoms remain underreported, and objective measures of SRC impairments would be valuable to assist diagnosis. Measurable impairments to vestibular and oculomotor processing have been shown to occur after SRC and may provide valid objective assessments.

Purpose:

Determine the diagnostic accuracy of sideline tests of vestibular and oculomotor dysfunction to identify SRC in adults.

Study Design:

Systematic review; Level of evidence, 4.

Methods:

Electronic databases and gray literature were searched from inception until February 12, 2020. Physically active individuals (>16 years of age) who participated in sports were included. The reference standard for SRC was a combination of clinical signs and symptoms (eg, the Sport Concussion Assessment Tool [SCAT]), and index tests included any oculomotor assessment tool. The QUADAS tool was used to assess risk of bias, with the credibility of the evidence being rated according to GRADE.

Results:

A total of 8 studies were included in this review. All included studies used the King-Devick test, with no other measures being identified. Meta-analysis was performed on 4 studies with a summary sensitivity and specificity of 0.77 and 0.82, respectively. The overall credibility of the evidence was rated as very low.

Conclusion:

Caution must be taken when interpreting these results given the very low credibility of the evidence, and the true summary sensitivity and specificity may substantially differ from the values calculated within this systematic review. Therefore, we recommend that clinicians using the King-Devick test to diagnose SRC in adults do so in conjunction with other tools such as the SCAT.

PROSPERO Registration:

CRD42018106632.

Keywords

SRC concussion King-Devick VOMS diagnosis SCAT

Sports-related concussion (SRC) accounts for approximately 5% of all sports-related injuries,² occurring at an average rate of 0.28 (95% confidence interval [CI], 0.27-0.28) per 100 athlete exposures.¹⁴ SRC is a mild traumatic brain injury induced by biomechanical forces, characterized by a short-term neurological impairment resulting in a heterogeneous range of signs and symptoms.²⁵ Symptoms may be exhibited immediately or occur hours after injury, and the quantity, severity, and duration of concussion symptoms vary between people and consecutive concussions.²⁵

The evidence suggests that after an SRC, some individuals may experience long-term negative sequalae from repeated exposures.²³ Unfortunately, even with the potential for long-term side-effects, SRC symptoms remain underreported at the time of incidence,²⁶ with the potential for players to avoid reporting or failing to recognize symptoms as potential SRC. In light of this, further emphasis has been placed on the development of objective tests to identify SRC based on detecting impairments after SRC that do not rely on self-reported symptoms.²⁶

Standard SRC assessment tools are primarily based on subjective assessments, or self-reporting, of somatic (eg, headache), cognitive (eg, memory loss), and psychosocial or emotional (eg, irritability) symptoms.³³ However, measurable impairments to vestibular⁴⁰ and oculomotor processing¹³ have also been demonstrated after SRC. Vestibular dysfunction will present as dizziness in 67% to 77% of all cases and may result in an athlete experiencing impaired balance.⁴⁰ Oculomotor dysfunction can present as an impairment to eye tracking^4,13,35 and reduced spatial accuracy.⁴ Including these systems in assessment procedures for SRC, which do not rely on self-reported symptoms, may reduce the number of unidentified cases.¹⁹

Recent studies have reported on diagnostic tools designed to identify vestibular and oculomotor impairments after SRC,¹¹ such as the King-Devick test and the vestibular/ocular motor screen (VOMS). Although a previous systematic review pooled King-Devick results for concussed and nonconcussed individuals, these results were not consistently compared with a standardized test and they included both adolescent and adult participants.⁸ As such, limited information is available regarding the diagnostic accuracy of these tools within the intended population. Our review aimed to evaluate the diagnostic accuracy of these tools to assist clinicians in choosing the most appropriate diagnostic test to objectively diagnose athletes with SRC.

Methods

This systematic review and meta-analysis was undertaken in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.

Systematic Review Registration

The protocol for this systematic review was registered on the PROSPERO database: CRD42018106632.

Data Management

Data were managed and stored on password-protected laptop computers. The final data spreadsheet is available from the authors upon reasonable request.

Criteria for Considering Studies for This Review

Types of Studies

Studies that used an oculomotor assessment as the index test to diagnose SRC were included, provided they also used an appropriate reference standard to confirm the diagnosis of SRC (eg, Sports Concussion Assessment Tool Version 3 [SCAT3]). Studies were excluded if they did not have a case-control design (ie, included only participants with suspected SRC and did not include participants without suspected SRC).

Types of Participants

Physically active adults who participated in a sport at reasonable risk of SRC were included. Based on a previous Cochrane Database of Systematic Reviews concussion protocol, participants older than 16 years were considered adults, so we considered studies in which the sample was predominately older than 16 years (eg, mean age >16 years old).²⁸

Reference Standard

The reference standard for diagnosing SRC was based on a combination of clinical signs and symptoms, and the assessing clinician confirmed the presence of SRC.²⁵ Any studies that compared the index test to a clinical diagnosis based on signs and symptoms were included. No definitive diagnostic tools for SRC exist, because no radiological imaging or biomarkers are 100% sensitive and specific for SRC.¹¹ Therefore, many different assessment tools are available to assist clinical diagnosis of SRC, such as the Sport Concussion Assessment Tool (SCAT),^36,38 and any could be used as a reference standard to be included.¹¹

Index Tests

All vestibular and oculomotor assessment tools that provide a diagnosis of SRC were included in this review.

Search Methods for Identification of Studies

Search strategies were used from inception, updated until February 12, 2020, and are documented in Appendix A (available in the online version of this article). Only peer-reviewed human studies were included, and the searches were adapted to individual databases as necessary.

Electronic Searches

Searches used free text terms (Appendix B, available online) to identify published articles on the following electronic databases: PubMed, CINAHL (Medline R), CINAHL (EBSCO), and Web of Science. Additional searches were conducted on the Cochrane Central Register of Controlled Trials, metaRegister of controlled trials (www.controlled-trials.com/mrct), clinicaltrials.gov (www.clinicaltrials.gov), and the World Health Organization International Clinical Trials Registry Platform (apps.who.int/trialsearch/) for completed trials.

Searching Other Resources

Reference lists from reviews and key articles were checked to supplement electronic searches. In addition, ePublication lists of key journals in the field were screened to pick up studies that have yet to be indexed in the databases.

Data Collection and Analysis

Selection of Studies

Two pairs of reviewers (A.R.D. and K.M.; S.A.H. and M.C.M.) independently assessed the titles and abstracts of potential studies identified by the search strategy for their eligibility. If the eligibility of a study was unclear from the title and abstract, the full article was assessed. Studies that did not match the inclusion criteria for this review were excluded. Disagreements between reviewers regarding study inclusion were resolved by consensus. Studies were not anonymized before assessment.

Data Extraction and Management

Two of the authors (A.R.D., K.M.) independently extracted data from included studies and input the data into Microsoft Excel, with the final datasheet being transferred into Review Manager Version 5.3.³² Disagreements between the authors were resolved by a third author (M.C.M.) who assessed relevant studies as required. The following information was extracted: primary author, year of publication, study design, study population (diagnosis), sample size, baseline demographics (age, sex, height, weight, and body mass index), index test (including threshold for diagnosis), reference test, time period between index and reference tests, and information on diagnostic accuracy (true positives, false positives, true negatives, false negatives, sensitivity, and specificity).

Assessment of Risk of Bias in Included Studies

Two authors (A.R.D., K.M.) independently assessed risk of bias for included studies. Disagreements between authors were resolved by a third author (M.C.M.) who assessed relevant studies as required, and all studies were cross-checked for agreement by a fourth author (S.A.H.).

The QUADAS tool was used to assess the risk of bias in the included studies.⁴¹ The original tool involves judgments on 14 items; 3 of these items assess the quality of reporting and not methodology. Therefore, only 11 items were included for assessment as recommended in section 9.3.2 of the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.³⁴ Each of the 11 items was prespecified to be judged as either meeting criteria (Yes), not meeting the criteria (No), or being unclear as to whether the criteria was met (Unclear) (Appendix C, available online). The QUADAS items were entered into Review Manager³² and presented as the risk of bias for individual and overall studies.

Dealing With Missing Data

Where insufficient data were presented in the study report, authors were contacted to request access to the missing data. This enabled additional studies to be included in the meta-analysis. Studies for which true-positive, false-positive, true-negative, and false-negative numbers were not available after contact with the corresponding author were excluded from the meta-analyses.

Assessment of Other Biases

The possible influence of publication or small study biases on review findings was considered based on the sample size. Sample sizes of participants with suspected concussion (eg, participants who had head impact) of <50 were considered high risk, samples of 50 to 200 were considered moderate risk, and samples >200 were considered low risk.³ Funnel plots were not assessed because <10 studies were included in the meta-analysis.³⁹

Data Synthesis

All data synthesis was completed in accordance with sections 10.2.3 and 10.2.5 of the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.²² The sensitivity and specificity for both the reference standard and index tests were assessed using Review Manager³² by inputting the 2 × 2 cross classification of index test results and the reference standard. These results are presented in the form of a forest plot generated by Review Manager.³² The diagnostic accuracy for each index test was determined using the area under the receiver operating characteristic (ROC) curve. These results are presented in the form of a summary ROC (sROC) plot generated by Review Manager.³²

Meta-analysis was performed in STATA using the metandi command. Summary sensitivity and specificity were presented in addition to positive and negative likelihood ratios and positive and negative predictive values.^1,22 Area under the ROC curve and interpretation of the diagnostic accuracy were not assessed due to the small number of included studies.³⁷

Assessment of Heterogeneity

Given the narrow inclusion criteria for studies (SRC in adults resulting from involvement in a sport at risk of head impact), we anticipated that clinical heterogeneity would be limited. Results were graphed using Review Manager³² to include a ROC scatter plot to visually explore the extent of statistical heterogeneity.

Assessment of the Credibility of the Body of Evidence

Assessment of the quality of the body of evidence was undertaken using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach,¹⁰ which can be adapted for use in diagnostic accuracy studies.⁹ The GRADE approach involves making an overall judgment on the quality of the body of evidence based on the overall risk of bias, consistency of results, directness of the evidence, precision of the research, sample size, and publication bias.

Results

Study Selection

A total of 1361 studies were identified, with 809 remaining after the removal of duplicates. After screening, a total of 8 studies met the inclusion criteria and were included in the current systematic review (Figure 1). The expanded reasons for exclusion of studies after full test screening are presented in Appendix D (available online). All of the resulting 8 studies used the King-Devick test as the only assessment of oculomotor function. All included studies compared participants with no incidence of head impact versus participants with SRC.^{7,12,16,17,20,21,27,31}

Figure 1.

Flow of identification, screening, eligibility, and inclusion for the literature reporting on the diagnostic accuracy of sideline tests of vestibular and oculomotor dysfunction for sport-related concussion in adults.

Data Extraction

For all included studies, the study design and setting are presented in Table 1. All studies were prospective in design with comparisons against a healthy control group. Data collection for the studies occurred from 2010 onward, with a medical or health professional administering the majority of tests when specified. A total of 4 different assessments were used by the included studies as the reference standard. These included SCAT3,^12,17,31 SCAT2,^16,21 the Military Acute Concussion Evaluation (MACE),^7,20 and the Pitch Side Concussion Assessment Version 2 (PSCA2).²⁷

Table 1

Study Design and Setting for Included Studies^a

	Galetta⁷ (2011)	Hecimovich¹² (2018)	King¹⁶ (2012)	King¹⁷ (2015)	Leong²⁰ (2013)	Leong²¹ (2015)	Molloy²⁷ (2017)	Naidu³¹ (2018)
Study design	Prospective cohort	Prospective cohort	Prospective cohort	Prospective cohort	Prospective cohort	Prospective cohort	Prospective cohort	Prospective cohort
Method	Healthy control	Healthy control	Healthy control	Healthy control	Healthy control	Healthy control	Healthy control	Healthy control
Setting	USA, college athletes	Australia, semi- professional	New Zealand, amateur	New Zealand, amateur	USA, amateur	USA, college athletes	New Zealand, semi- professional	Canada, elite
Sport	Varsity football, sprint football, soccer, basketball	Australian football	Rugby league	Rugby union, rugby league	Boxing	Football, basketball	Rugby union	Football
Data collection period	1 season, 2010-2011	1 season, 2016	1 season (12 games), 2011	3 teams (1 per season), 2012-2014	3-round competition	1 season, 2012-2013	1 season, 2014	2015-2016
Personnel completing data collection
Reference test	NR	Medical staff	NR	Health professional	Ringside physician	Physician or athletic trainer	Medical staff	Physician or athletic trainer
Index test	NR	Medical staff	NR	Health professional	Layperson	Physician or athletic trainer	Medical staff	Physician or athletic trainer
Time from injury to assessment	Immediate (except for 3 athletes)	10 min	<30 min	<15 min	<15 min	Immediate	Immediate	<24 h
Reference test	MACE	SCAT3	SCAT2	SCAT3	MACE	SCAT2	PSCA2	SCAT3
Index test	K-D	K-D	K-D	K-D	K-D	K-D	K-D	K-D
Note		KD self- administered at baseline.		Baseline test— unspecified who delivered the assessments.Only players with change in K-D scores (>3 s) or witnessed concussions (n = 8) were assessed with SCAT3.	Bias with comparison between parent (layperson) and medical professional.

K-D, King-Devick test; MACE, Military Acute Concussion Evaluation; NR, not reported; PSCA2, Pitch Side Concussion Assessment Version 2; SCAT2, Sports Concussion Assessment Tool Version 2; SCAT3, Sports Concussion Assessment Tool Version 3.

All included studies used the King-Devick test as the index test; however, SRC diagnosis standards differed between the 8 studies. SRC was diagnosed if the score for the postinjury assessment was slower (in seconds) than the individual baseline,^12,31 if the score was slower than the individual baseline and/or if there were any errors,²⁷ or if the score was >3 seconds slower than baseline¹⁷; 4 studies did not report the diagnostic threshold.^7,16,20,21

Across the 8 included studies, the total sample size ranged from 22 participants¹² to 219 participants,⁷ with between 1 and 84 suspected concussions assessed.^21,31 The majority of included studies examined an exclusively male population.^{12,16,17,27,31} Of the 3 studies including both male and female patients, females represented less than a fifth of the total sample (6.7%,²⁰ 13.3%,²¹ 20.3%⁷). Table 2 details the overall sample size, number of injured participants, demographic characteristics, index test, and reference tests used.

Table 2

Participant Information for Included Studies^a

	Galetta⁷ (2011)	Hecimovich¹² (2018)	King¹⁶ (2012)	King¹⁷ (2015)	Leong²⁰ (2013)	Leong²¹ (2015)	Molloy²⁷ (2017)	Naidu³¹ (2018)
Sample, n
Total sample	219	22	50	104	34	127	57	150
Sample with suspected head impact^b	10	7	5	39	1	11	21	84
Control	206	15	45	65	33	116	33	63
Missing data	3	0	0	0	0	0	3	3
Age, y, mean ± SD
Total	20.3 ± 1.4	19.6 ± 2.3	19 ± 4.0	23.7 ± 5.0	NR	19.6 ± 1.2	24.1	25.4 ± 2.9
Injured	20.3 ± 1.0	NR	NR	22.9 ± 4.4	23	19.6 ± 1.2	23.8 ± 2.9	26.4
Control	20.3 ± 1.4	NR	NR	NR	25.8 ± 8.3	19.5 ± 1.2	23.4 ± 3.0	27.1
Sex, n, male/female
Total	182/37	22/0	50/0	104/0	30/4	119/8	54/0	150/0
Injured	9/1	7/0	5/0	39/0	1/0	11/0	21/0	84/0
Control	170/36	15/0	45/0	65/0	29/4	108/8	33/0	63/0
Concussion history
Total	26% of sample ≤1 (range, 1-7) concussions	NR	62% of sample ≤1 (range, 1-6) concussions	NR	NR	NR	NR	NR
Injured	NR	NR	NR	NR	NR	NR	Mean ± SD 2.7 ± 2.2 (range, 0-6) concussions	Mean 1.9 (range, 0-4) concussions
Control	NR	NR	NR	NR	NR	NR	Mean ± SD 1.5 ± 1.6 (range, 0-5) concussions	Mean 1.1 (range, 0-4) concussions
Head injury events, n	10	7	5	52 (52 concussions from 39 players)	6	11	22 (19 concussions from 18 players, with complete data)	86 (84 with complete data)
Note	K-D performed pre-post, with MACE performed in only n = 12, with n = 1 concussion. Total gender split provided was based off the total sample of n = 219 whereas injured and control gender split is based off the sample excluding the missing data so n = 216.		3 concussions identified by medic, but 2 concussions were identified using K-D. Two different mean ± SD ages provided: Methods (N = 50) 22.4 ± 4.1 y; Table 1 (N = 50) 19.3 ± 4.0 y	8 concussions identified by medic; 44 concussions were not witnessed.		Only football players (n = 102) reported concussion.	Of the 176 players, 33 controls were randomly recruited.	Of the 1223 players, 64 controls were randomly recruited, with 63 controls eligible for inclusion. Total gender split provided was based off the total sample of n = 150 whereas injured and control gender split is based off the sample excluding the missing data so n = 147.

Injured sample <50 represents high risk of bias; 50-200 represents moderate risk of bias.

For 6 studies, the data required to calculate sensitivity and specificity were provided within the text or the corresponding authors provided the data on request.^{12,17,20,21,27,31} Sensitivity and specificity were reported between 53% and 100% and between 69% and 97%, respectively (Table 3). Predictive values were included within the article for 2 studies.^27,31 Of the remaining 6 corresponding authors contacted for predictive values, 2 authors provided these upon request.^20,21 We could not include 4 studies in the meta-analysis because the published studies did not report the true positives, false positives, true negatives, and false negatives, and the data were not provided after a written request to the corresponding author.^7,12,16,17

Table 3

Comparison of King-Devick Oculomotor Test Against the Concussion Reference Test Per Included Study^a

	Galetta⁷ (2011)	Hecimovich¹² (2018)	King¹⁶ (2012)	King¹⁷ (2015)	Leong²⁰ (2013)	Leong²¹ (2015)	Molloy²⁷ (2017)	Naidu³¹ (2018)
Reference test	MACE	SCAT3	SCAT2	SCAT3	MACE	SCAT2	PSCA2	SCAT3
Index test	K-D	K-D	K-D	K-D	K-D	K-D	K-D	K-D
Sensitivity of K-D, % (95% CI)	NR	98 (87-100)	NR	100 (93-100)	100 (30-100)	100 (72-100)	53 (NR)	62 (NR)
Specificity of K-D, % (95% CI)	NR	96 (80-100)	NR	94 (84-100)	97 (84-100)	70 (55-83)	67 (NR)^d	84 (NR)
False negative	^b	^b	^b	^b	0^c	0^c	9	32
False positive					1^c	13^c	11	10
True negative					32^c	31^c	22	53
True positive					1^c	11^c	10	52

Values could not be retrieved from the author upon request.

Provided by author upon request.

The specificity we have provided differs to what is provided within the full text publication as the number within the publication is incorrect. This error was clarified by the corresponding author as a likely transcription error.

Risk of Bias

One study was at low risk of bias across all QUADAS items,¹² whereas the other 7 studies were at an unclear risk of bias.^{7,16,17,20,21,27,31} A total of 4 studies had low applicability concerns,^12,17,27,31 and 4 were unclear.^7,16,20,21 The individual and grouped risk of bias and applicability concerns are presented in Figures 2 and 3, respectively. The quality of reporting for included studies resulted in a large number of items being classified as unclear.

Figure 2.

Risk of bias across the individual studies included in this review.

Figure 3.

Risk of bias across all studies included in this review.

One study was considered to have a moderate risk of sample size bias, with a head impact sample of 50 to 200.³¹ The remaining 7 studies were considered to have a high risk of sample size bias, with a head impact sample of <50.^{7,12,16,17,20,21,27} Publication bias could not be assessed given the small number of included studies.

Data Synthesis

The individual sensitivity and specificity of the King-Devick test (including 95% CI) for each included study are presented in Figure 4. The sensitivity of the King-Devick test ranged from 0.53 to 1.00, and specificity ranged from 0.67 to 0.97.

Figure 4.

Study sensitivity and specificity of the King-Devick test.

The meta-analysis of diagnostic accuracy indicated a summary sensitivity of 0.77 (95% CI, 0.31-0.96) and a specificity of 0.82 (95% CI, 0.66-0.91). The positive predictive value of 67.9% indicated that only 68 of every 100 athletes with a positive index test result will have concussion; however, 32 will not (false positive). Conversely, the negative predictive value of 77.1% indicated that 77 of every 100 athletes with a negative index test result will not have concussion; however, 23 participants will actually have concussion (false negative).

The positive likelihood ratio was 4.24 (95% CI, 1.78-10.09), indicating that the index test was 4 times more likely to detect positive results (diagnosed concussion) in participants with concussion than in participants without concussion. The negative likelihood ratio was 0.28 (95% CI, 0.06-1.33), indicating the index test was 0.3 times less likely to clear a participant of concussion among participants with concussion than in individuals without concussion. Because <5 studies were included in the meta-analysis, area under the curve and diagnostic accuracy could not be determined.²² In addition, given the small number of studies, sensitivity analysis could not be conducted by excluding a subgroup of studies to assess their influence on the overall effect size and measures of heterogeneity.

Assessment of Heterogeneity

The sROC curve in Figure 5 displays heterogeneity within the results. Heterogeneity is concluded given the distance between each data point.

Figure 5.

Summary receiver operating characteristic (sROC) curve displaying the sensitivity and specificity of the King-Devick test.

Credibility of the Evidence

The credibility of the evidence received an overall GRADE of very low. Because only 1 study was at low risk of bias and the overall sample sizes were small to moderate, the credibility of the level of evidence was downgraded twice for study limitations. There was evidence of heterogeneity of results, so the credibility of the evidence was downgraded once for inconsistency. The confidence intervals for the results were moderate, and therefore the credibility of the level of evidence was downgraded for imprecision. The overall applicability concerns were low, and therefore the evidence was not downgraded for indirectness. Publication bias could not be assessed, and therefore the credibility of the level of evidence was downgraded once as a conservative estimate.

Discussion

Finding objective measurements to diagnose SRC that do not relate to player self-reporting of symptoms is integral to player well-being and safety, particularly given the known culture of underreporting of SRC symptoms.²⁶ In this setting, there is value in the use of vestibular/ocular screening tools to assess whether athletes with suspected SRC have objective features of concussion. When using these tests, clinicians do not need to rely on athletes to self-report symptoms, and the features can be present immediately^7,16,17 as opposed to basing a diagnosis on subjective symptoms, which may take time to develop.²⁵ Testing objective tools in adult populations is also important; although some evidence suggests that brain trauma in a younger person with a developing brain can be more detrimental (eg, longer recovery time),²⁴ it is proposed that a diagnostic tool is more relevant in an adult population. The rationale is that in a sport setting, children and adolescents are usually managed conservatively (ie, “If in doubt, sit them out”),⁶ but this can become increasingly more challenging with adult sports participants.

This systematic review was able to identify only 1 test of oculomotor function, the King-Devick test, that has been investigated in an adult population (>16 years of age) including participants with and without potential concussion.^{7,12,16,17,20,21,27,31} The inclusion of only 1 test was unexpected given that a number of different tests have been proposed in this space.^29,42 However, commonly cited studies of vestibular and ocular assessment tools, such as the VOMS, entail adolescents younger than 16 years.^29,42 A number of studies using different vestibular and oculomotor screening tools were excluded because the study design compared mean differences in the primary outcome measure between concussed and nonconcussed participants (often many days after the incident).¹⁸ Unfortunately, this design does not provide true positives, false positives, true negatives, and false negatives, which are required to assess diagnostic accuracy.

Our meta-analysis of the 4 studies that had sufficient data to be included demonstrated a summary sensitivity of 0.77 and a specificity of 0.82, resulting in a positive likelihood ratio of 4.24 and a negative likelihood ratio of 0.28. However, the quality of the evidence is very low, meaning that the true sensitivity and specificity are likely to be different than the estimate reported.^9,10 We would recommend to clinicians that the King-Devick test may be of value but should be used in conjunction with other existing screening tools, such as the SCAT, to provide a more thorough picture of an athlete who has sustained head trauma.^15,31

The number of concussed participants within studies was unexpectedly low; one study had only a single suspected concussion within the cohort, which could have been due to the small sample size, low incidence of SRC, or poor self-reporting of SRC. Likewise, few studies provided data on the participants’ history of concussion, and previously concussed participants were not included as a subgroup within the analysis of any study. We noted that the populations recruited within studies for diagnostic accuracy of the King-Devick test tend to be males younger than 25 years. This may have applicability concerns if one is planning to use the King-Devick tests in a different population (eg, female masters athletes). One study was identified during screening that used the King-Devick test to diagnose concussion in female amateur rugby union players, but the study design did not allow for investigation of diagnostic accuracy.¹⁸ We suggest that clinicians consider the potential lack of validity for different populations when determining whether to use the King-Devick test to assess their athletes.

Future studies that include a more diverse sample of participants (eg, more participants >25 years of age and/or female participants, as well as those with and without a history of SRC) are needed to determine whether the King-Devick test is applicable in different populations. Authors of future studies should also be more thorough in reporting their results; true positives, true negatives, false positives, and false negatives should be reported to assist interpretation of results and future meta-analysis.²²

Limitations

Sports-related concussion is a nonspecific, umbrella diagnosis related to a collection of different signs and symptoms. As a nonspecific diagnosis, there will likely be significant heterogeneity within and between the characteristics of included participants in studies. The characteristics of included participants (eg, a history of SRC, presence of a headache or other signs and symptoms) can all be considered their “clinical phenotype.” These clinical phenotypes can be valuable in sports medicine research because they help clinicians and researchers determine whether the sample in a research study is comparable with the cohort they see in clinical practice.³⁰ Research into differences between people with the same condition, to select targeted interventions for better outcomes, is known as precision medicine.⁵ Therefore, it is imperative that researchers include better reporting of participant baseline characteristics and diagnostic criteria (including inclusion and exclusion criteria) so that results can be appropriately pooled in meta-analyses and confounders accounted for within modeling. Adjusting for confounders was not possible in this meta-analysis due to poor reporting (eg, only 2 studies separately reported concussion history in both the control and concussed groups).

The lack of a definitive diagnostic tool for SRC²⁵ is the biggest barrier to accuracy within all SRC studies related to diagnostic accuracy. However, to limit the effect of this on our results, we included only studies that had used a clinical assessment tool incorporating a combination of clinical signs and symptoms, such as the SCAT.³⁸ This can be viewed as problematic; however, because the SCAT is recommended by the most recent consensus statement and it is not possible to truly investigate its diagnostic accuracy due to the absence of a biomarker or imaging finding to definitively diagnose SRC, we believe this is the most appropriate method. The diagnostic criteria for the King-Devick test, if reported, varied within studies, but due to the small number of included studies within the meta-analysis, a sensitivity analysis was not performed.

Finally, a number of authors of this systematic review (A.R.D., M.H., D.K., and M.C.M.) are authors of some of the included studies. However, at least 1 independent reviewer (S.A.H. or K.M.) was included as the second researcher for each step of screening and data extraction in an attempt to minimize any potential bias. All analysis was conducted by an independent author (S.A.H.) who has no previous publication within this area to ensure this did not affect the results and to minimize any potential bias.

Conclusion

This systematic review identified 8 studies that investigated vestibular/oculomotor screening tools in diagnosing SRC in an adult population. The King-Devick test was the only index test performed across all the included studies. A total of 4 studies proceeded to meta-analysis, with 4 excluded due to missing data, and the summary sensitivity and specificity were 0.77 and 0.82, respectively. The overall credibility of the evidence was rated as very low, and caution must be taken when interpreting these results given that the true summary sensitivity and specificity may differ substantially from the values calculated within this systematic review. Therefore, we recommend that clinicians using the King-Devick test to diagnose SRC in adults do so in conjunction with other tools such as the SCAT.

Supplemental Material

sj-pdf-1-ajs-10.1177_03635465211027946 – Supplemental material for Do Sideline Tests of Vestibular and Oculomotor Function Accurately Diagnose Sports-Related Concussion in Adults? A Systematic Review and Meta-analysis

Supplemental material, sj-pdf-1-ajs-10.1177_03635465211027946 for Do Sideline Tests of Vestibular and Oculomotor Function Accurately Diagnose Sports-Related Concussion in Adults? A Systematic Review and Meta-analysis by Sarah A. Harris, Alasdair R. Dempsey, Katherine Mackie, Doug King, Mark Hecimovich and Myles C. Murphy in The American Journal of Sports Medicine

Footnotes

Acknowledgements

The authors thank Dr Dana Hince (Institute for Health Research) for her consultation on completing the meta-analysis using STATA.

Submitted December 26, 2020; accepted March 5, 2021.

One or more of the authors has declared the following potential conflict of interest or source of funding: S.A.H. is supported by a WA Health Translation Network Biostatistics Research Fellowship. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.

An online CME course associated with this article is available for 1 AMA PRA Category 1 Credit™ at . In accordance with the standards of the Accreditation Council for Continuing Medical Education (ACCME), it is the policy of The American Orthopaedic Society for Sports Medicine that authors, editors, and planners disclose to the learners all financial relationships during the past 12 months with any commercial interest (A ‘commercial interest’ is any entity producing, marketing, re-selling, or distributing health care goods or services consumed by, or used on, patients). Any and all disclosures are provided in the online journal CME area which is provided to all participants before they actually take the CME activity. In accordance with AOSSM policy, authors, editors, and planners’ participation in this educational activity will be predicated upon timely submission and review of AOSSM disclosure. Noncompliance will result in an author/editor or planner to be stricken from participating in this CME activity.

ORCID iD

Doug King

References

Cochrane Methods. Software for meta-analysis of DTA studies. The Cochrane Collaboration; 2018. Accessed August 9, 2018. https://methods.cochrane.org/sdt/software-meta-analysis-dta-studies

Daneshvar

Nowinski

McKee

Cantu

RC.

The epidemiology of sport-related concussion. Clin Sports Med. 2011;30(1):1-17.

Dechartres

Trinquart

Boutron

Ravaud

Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ. 2013;346:f2304.

DiCesare

Kiefer

Nalepka

Myer

GD.

Quantification and analysis of saccadic and smooth pursuit eye movements and fixations to detect oculomotor deficits. Behav Res Meth. 2017;49(1):258-266.

Edwards

Dworkin

Turk

, et al. Patient phenotyping in clinical trials of chronic pain treatments: IMMPACT recommendations. Pain. 2016;157(9):1851-1871.

Elkington

Manzanero

Hughes

. Position statement: concussion in Sport Australia. Accessed December 4, 2020. https://www.sportaus.gov.au/

Galetta

Brandes

Maki

, et al. The King-Devick test and sports-related concussion: study of a rapid visual screening tool in a collegiate cohort. J Neurol Sci. 2011;309(1-2):34-39.

Galetta

Liu

Leong

Ventura

Galetta

Balcer

LJ.

The King-Devick test of rapid number naming for concussion detection: meta-analysis and systematic review of the literature. Concussion. 2016;1(2):Cnc8.

Gopalakrishna

Mustafa

Davenport

, et al. Applying Grading of Recommendations Assessment, Development and Evaluation (GRADE) to diagnostic tests was challenging but doable. J Clin Epidemiol. 2014;67(7):760-768.

10.

GRADE Working Group. Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Accessed October 6, 2018. https://gdt.gradepro.org/app/handbook/handbook.html

11.

Graham

Rivara

Ford

; Committee on Sports-Related Concussions in Youth; Board on Children, Youth, and Families; Institute of Medicine. Sports-Related Concussions in Youth: Improving the Science, Changing the Culture. National Academies Press; 2014.

12.

Hecimovich

King

Dempsey

Murphy

The King-Devick test is a valid and reliable tool for assessing sport-related concussion in Australian football: a prospective cohort study. J Sci Med Sport. 2018;21(10):1004-1007.

13.

Heitger

Jones

Macleod

Snell

Frampton

Anderson

TJ.

Impaired eye movements in post-concussion syndrome indicate suboptimal brain function beyond the influence of depression, malingering or intellectual ability. Brain. 2009;132(10): 2850-2870.

14.

Hootman

Dick

Agel

Epidemiology of collegiate injuries for 15 sports: summary and recommendations for injury prevention initiatives. J Athl Train. 2007;42(2):311-319.

15.

Howitt

Brommer

Fowler

Gerwing

Payne

DeGraauw

The utility of the King-Devick test as a sideline assessment tool for sport-related concussions: a narrative review. J Can Chiropr Assoc. 2016;60(4):322-329.

16.

King

Clark

Gissane

Use of a rapid visual screening tool for the assessment of concussion in amateur rugby league: a pilot study. J Neurol Sci. 2012;320(1):16-21.

17.

King

Gissane

Hume

Flaws

The King-Devick test was useful in management of concussion in amateur rugby union and rugby league in New Zealand. J Neurol Sci. 2015;351(1-2):58-64.

18.

King

Hume

Clark

Pearce

AJ.

Use of the King-Devick test for the identification of concussion in an amateur domestic women’s rugby union team over two competition seasons in New Zealand. J Neurol Sci. 2020;418:117162.

19.

Kontos

Deitrick

Collins

Mucha

Review of vestibular and oculomotor screening and concussion rehabilitation. J Athl Train. 2017;52(3):256-261.

20.

Leong

Balcer

Galetta

Evans

Gimre

Watt

The King-Devick test for sideline concussion screening in collegiate football. J Optom. 2015;8(2):131-139.

21.

Leong

Balcer

Galetta

Liu

Master

CL.

The King-Devick test as a concussion screening tool administered by sports parents. J Sports Med Phys Fitness. 2013;54(1):70-77.

22.

Macaskill

Gatsonis

Deeks

Harbord

Takwoingi

Analysing and presenting results. In: Deeks

Bossuyt

Gatsonis

, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. The Cochrane Collaboration; 2010:chap 10. http://srdta.cochrane.org/

23.

Manley

Gardner

Schneider

, et al. A systematic review of potential long-term effects of sport-related concussion. Br J Sports Med. 2017;51(12):969-977.

24.

Manzanero

Elkington

Praet

Lovell

Waddington

Hughes

DC.

Post-concussion recovery in children and adolescents: a narrative review. J Concussion. 2017;1:2059700217726874.

25.

McCrory

Meeuwisse

Dvorak

, et al. Consensus statement on concussion in sport—the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017;51(11):838-847.

26.

Meier

Brummel

Singh

Nerio

Polanski

Bellgowan

PSF

. The underreporting of self-reported symptoms following sports-related concussion. J Sci Med Sport. 2015;18(5):507-511.

27.

Molloy

Murphy

Gissane

The King-Devick (K-D) test and concussion diagnosis in semi-professional rugby union players. J Sci Med Sport. 2017;20(8):708-711.

28.

Moore

Mawdsley

Jackson

Atherton

MJ.

Psychological interventions for persisting postconcussion symptoms following traumatic brain injury. Cochrane Database Syst Rev. 2017;(8):CD012755.

29.

Mucha

Collins

Elbin

, et al. A brief vestibular/ocular motor screening (VOMS) assessment to evaluate concussions: preliminary findings. Am J Sports Med. 2014;42(10):2479-2486.

30.

Murphy

Gibson

Moseley

Rio

EK.

Are you translating research into clinical practice? What to think about when it does not seem to be working. Br J Sports Med. 2021;55(12):652-653.

31.

Naidu

Borza

Kobitowich

Mrazik

Sideline concussion assessment: the King-Devick test in Canadian professional football. J Neurotrauma. 2018;35(19):2283-2286.

32.

The Nordic Cochrane Centre TCC. Review Manager Version 5.3. The Cochrane Collaboration; 2014.

33.

Prince

Bruhns

ME.

Evaluation and treatment of mild traumatic brain injury: the role of neuropsychology. Brain Sci. 2017;7(8):105.

34.

Reitsma

Rutjes

AWS

Whiting

Vlassov

Leeflang

MMG

Deeks

JJ.

Assessing methodological quality. In: Deeks

Bossuyt

Gatsonis

, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. The Cochrane Collaboration; 2009:chap 9. http://srdta.cochrane.org/

35.

Samadani

Ritlop

Reyes

, et al. Eye tracking detects disconjugate eye movements associated with structural traumatic brain injury and concussion. J Neurotrauma. 2015;32(8):548-556.

36.

SCAT2. Br J Sports Med. 2009;43(suppl 1):i85-i88.

37.

Simundic

AM.

Measures of diagnostic accuracy: basic definitions. EJIFCC. 2009;19(4):203-211.

38.

Sport concussion assessment tool—5th edition. Br J Sports Med. 2017;51:851-858.

39.

Sterne

Sutton

Ioannidis

, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002.

40.

Valovich McLeod

Hale

TD.

Vestibular and balance issues following sport-related concussion. Brain Injury. 2015;29(2):175-184.

41.

Whiting

Rutjes

Reitsma

Bossuyt

Kleijnen

The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3:25.

42.

Yorke

Smith

Babcock

Alsalaheen

Validity and reliability of the vestibular/ocular motor screening and associations with common concussion screening tools. Sports Health. 2017;9(2): 174-180.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.33 MB