Test-Retest Reliability of the Blast Exposure Threshold Survey in United States Service Members and Veterans

Abstract

The Blast Exposure Threshold Survey (BETS) is a recently developed measure of lifetime blast exposure. Although promising, it is considered a fundamental tenet to establish that the BETS (and other measures like it) have good psychometric properties before it can be recommended for clinical use. The purpose of this study was to examine the test-retest reliability of the BETS in a military sample. Participants were 83 United States service members and veterans prospectively recruited from three military medical treatment facilities and from the community. Participants were classified into two broad groups as part of a larger study: traumatic brain injury (TBI; n = 41; mild-severe TBI) and controls (n = 42; injured and non-injured controls). Participants completed the BETS, Neurobehavioral Symptom Inventory, and a brief structured interview to gather basic demographic, military, and injury-related information (e.g., age, education, deployments, etc.). In addition, participants completed the BETS on a second occasion (T2) 3 weeks following the first administration (T1). Using Spearman rho correlation analyses, the test-retest reliability of the BETS Generalized Blast Exposure Value (GBEV) was classified as “acceptable” (r = 0.76). However, when comparing individual responses across T1 and T2, 33% of the sample reported significant inconsistencies in the endorsement of the five weapons categories. The most problematic inconsistency (∼10% of the sample) related to the failure of some participants to consistently endorse, or not endorse, exposure to a weapons category at T1 and T2 (e.g., T1 = exposure present; T2 = exposure absent). Less problematic, but also of concern, was the failure of some participants (∼23%) to consistently report the same number of years in which they were exposed to a weapons category from T1 and T2 (e.g., T1 = 10 years; T2 = 5 years). Factors associated with inconsistent reporting from T1 to T2 included higher GBEV scores, older age, higher number of years in the military, higher number of deployments, and higher blast exposure. This is one of the first studies to comprehensively examine the test-retest reliability of the BETS GBEV. Overall, the test-retest reliability of the GBEV was considered statistically acceptable and provides support for the use of the GBEV in both clinical and research settings. Concerningly, however, substantial inconsistencies were found in the basic reporting of weapons exposure in 33% of the sample that need to be addressed. Future researchers should identify ways to improve the BETS to increase response consistency over time.

Keywords

blast exposure threshold survey generalized blast exposure value lifetime blast exposure test-retest reliability

Introduction

For more than a decade, there has been growing concern regarding the potential negative effects of repeated low-level blast exposure in military personnel.1 This concern led to the two National Defense Authorization Acts passed by the United States Congress in 2018 and 2020 (NDAA; Section 734 and Section 717) that called for increased emphasis on the need to document and understand the effects of repetitive blast exposure in military personnel.2 Since then, an increasing number of studies have demonstrated the association between repeated low level blast exposure and a variety of negative clinical outcomes;^3–15 though findings are mixed depending on what clinical outcomes are examined.^16–18

A main limitation in the study of the impacts of repeated blast exposure, which likely contributes to the conflicting findings in the available literature, is the lack of validated metrics to quantify blast exposure. In order to document and understand the effects of repetitive blast exposure, it is critical that we use tools that can measure blast exposure in a reliable and valid manner. At present, there are two broad methods for measuring blast exposure: objective and subjective methods. Objective blast exposure methods refer to wearable blast sensors that gather blast data at the time of exposure.¹⁹ These methods are generally considered to be the gold standard for measuring blast. However, objective blast exposure methods are greatly limited by the fact that they can only measure blast exposures prospectively and are unable to quantify the many blast exposures that service members and veterans (SMVs) may have experienced in the past. As such, the use of subjective blast exposure measures that aim to quantify a person’s lifetime blast exposure (LBE) plays a valuable role in our attempts to understand the effects of long-term blast exposure in military personnel. Unfortunately, one of the biggest difficulties in measuring LBE is that it relies on retrospective recall of blast exposures over many years, which can be subject to reporting bias (e.g., underreporting, overreporting, inconsistency, inaccuracy). As such, the development and use of well-validated tools to measure LBE is critical for the advancement of our understanding of the effects of repeated low-level blast exposure.

Until recently, a comprehensive measure of LBE has not been available. However, four new measures have been developed: Blast Exposure Threshold Survey (BETS),²⁰ Blast Ordnance and Occupational Exposure Measure (BOOM),²¹ Blast Frequency and Symptom Severity (B-FASS),²² and Salisbury Blast Interview.²³ While all four of these measures show promise (see Turner et al.,²² for a review), there are no published studies that have examined the psychometric properties of these tools to support their use. The one exception to this was a study by Lange and colleagues²⁴ that provided data to support the convergent and discriminant validity of the BETS in 202 United States SMVs with and without a history of traumatic brain injury (TBI).

While the establishment of convergent and discriminant validity for the BETS in the study by Lange and colleagues²⁴ has provided some progress toward determining whether the BETS can be confidently used as a valid and reliable measure of LBE, other psychometric data are also needed in order to confidently support the use of the BETS in this manner. At minimum, in addition to discriminant and convergent validity, it is critical for the test-retest reliability of the BETS to be established. The purpose of this study was to examine the test-retest reliability of the BETS over a 3-week interval in United States SMVs.

Materials and Methods

Participants

Participants were 83 United States SMVs prospectively enrolled in the Defense and Veterans Brain Injury Center-TBI Center of Excellence (DVBIC-TBICoE) 15-Year Longitudinal TBI Study. Participants were recruited and classified into two broad groups: TBI (n = 41) and controls (n = 42). Participants were predominantly recruited from outpatient clinics and inpatient wards from the Walter Reed National Military Medical Center (62.7%), as well as from the Naval Medical Center San Diego (10.8%), 33 Area Branch Clinic Camp Pendleton (6.0%), and via community-based recruitment initiatives (20.5%, e.g., Yellow-ribbon events, social media, flyers). General exclusion criteria included being less than 18 years of age, lack of proficiency in conversational English, and a history of significant neurological or psychiatric condition[s] unrelated to the injury event or deployment (e.g., meningioma, bipolar disorder).

Participants were selected from a larger sample of SMVs that had been evaluated in the DVBIC-TBICoE 15-Year Longitudinal TBI study between March 2023 and April 2024 if they: (a) had completed the BETS on two occasions without missing items, (b) had completed the BETS on two occasions within 3 to 4 weeks, (c) had completed the Validity-10 scale with no missing items, and (d) had completed a structured interview with no missing demographic information. The mean time between baseline and follow-up for the sample was 3.2 weeks (SD = 0.7; median = 3.0; IQR = 3.0–3.3).

Participants were included and classified in the control group if they met one of the following criteria: (1) Injured controls (IC, n = 22): had sustained an orthopedic and/or soft-tissue injury; there was no evidence of intracranial abnormality (ICA) or an altered state of consciousness as a result of the injury (e.g., Glasgow Coma Scale [GCS] <15, alteration of consciousness [AOC], loss of consciousness [LOC], or post-traumatic amnesia [PTA]); the presenting complaint was not due to a neurological condition/disorder (e.g., cerebrovascular accident); (2) non-injured controls (NIC, n = 20): no history of an orthopedic and/or soft-tissue injury, and no history of TBI.

Participants were included in the TBI group if they had sustained a neurotrauma and could be classified based on TBI severity as follows: [a] uncomplicated mild TBI: GCS ≥13 (if available), PTA ≤24 h, LOC ≤30 min, and/or AOC present, and the absence of trauma-related ICA on computed tomography or magnetic resonance imaging; [b] complicated mild TBI: (i) GCS = 13–15, PTA ≤24 h, LOC ≤30 min, and/or AOC present, and (ii) ICA present; [c] moderate TBI: LOC >30 min to 24 h, PTA >1–7 days, and ICA present or absent; [d] severe TBI: LOC >24 h, PTA >7 days, and ICA present or absent; and [e] penetrating TBI: a breach of the cranial vault and/or dura mater by an external object (e.g., bullet, shrapnel) and/or skull fragment (i.e., depressed skull fracture). In addition, a small number of participants were classified as having a “suspected mild TBI” who had sustained a biomechanically plausible mechanism of injury; however, a TBI could not be confidently ruled-in or ruled-out based on the presence of confounding factors.²⁵

The protocol under which these data were collected was approved by the Institutional Review Board of the Walter Reed National Military Medical Center, Bethesda, MD. This study was completed in accordance with the guidelines of the Declaration of Helsinki. The study was undertaken with the understanding and written consent of each participant.

TBI diagnosis and classification

Diagnosis and classification of TBI were based on a medical record review and a comprehensive lifetime TBI history interview. The lifetime TBI history interview was completed by masters-level clinical research personnel who were specifically trained (by RTL and SML) to evaluate the presence and severity of TBI. The TBI history interview consisted of the Ohio State University TBI identification method²⁶ and an extended semi-structured clinical interview designed to (a) extract more detailed information to estimate the presence/duration of LOC, PTA, AOC, and retrograde amnesia, and (b) gather military-specific information regarding injury circumstances (e.g., type of blast, protection worn, etc.). Final determination and classification of TBI severity was undertaken by consensus, giving consideration to all information, during case conferencing with the interviewer and a PhD-level clinician/scientist trained in neuropsychology and TBI diagnostic interviewing (RTL or SML). In addition, the total number of lifetime TBIs is generated from this interview.

Measures and procedure

Participants completed the BETS,²⁰ Validity-10 scale,²⁷ and a brief structured interview at baseline. In addition, participants completed the BETS a second time 3 to 4 weeks following baseline.

The BETS is a measure designed to gather detailed information related to a person’s LBE to a variety of weapons classified in five categories: [i] Small and medium arms (e.g., hand held pistols, rifles, shotguns, machine guns); [ii] Large arms, often shoulder-fired, that can be carried on a person (e.g., rocket-propelled grenade [RPG], Carl-Gustaf, grenade launcher);, [iii] Artillery, missile weapon systems, or large arms carried by vehicle, aircraft, or boat (e.g., Howitzer, multiple rocket launcher); [iv] Smaller explosives or grenades (flashbangs, small improvised explosive device [IEDs], landmines); and [v] Larger explosives or targeted explosives in close range (e.g., breaching explosives, disposal explosives, heavy charges). For each weapons category, the respondent is first asked whether they have ever been exposed to the weapons category. If endorsed “Yes,” the respondent is then required to provide information on how many (a) years, (b) months per year, (c) days per month, and (d) rounds per day they were exposed to these weapons. The BETS is scored in two main steps. First, a blast exposure count (BEC) is calculated for each of the five weapon categories by multiplying how many years, months/year, days/month, and rounds/day they were exposed to the weapons (e.g., 3BEC = BEC score for weapons category 3). Second, a Generalized Blast Exposure Value (GBEV) is calculated using a weighted formula as outlined by the test developers.²⁰ The GBEV represents the units of blast exposure over a person’s lifetime. Higher GBEV values reflect greater LBE.

The Validity-10 scale was used as a symptom validity test (SVT) designed to detect symptom exaggeration. The Validity-10 scale consists of 10 items from the Neurobehavioral Symptom Inventory²⁸ that are considered atypical and infrequently endorsed by individuals following TBI. Clinical validation studies have supported its use for this purpose.^29–34 As recommended by Vanderploeg and colleagues,²⁷ a cutoff score of >22 was used to classify symptom exaggeration (i.e., SVT Fail). Typically, participants are excluded from a study if they fail this measure. However, in a departure from our routine research practices,^15,35,36 participants were purposely not excluded for scoring below the recommended cutoff on the Validity-10 for two reasons. First, SVTs are used to detect symptom exaggeration on self-report symptom measures. We did not include any self-report symptom measures in this study. Second, we wanted to examine SVT as a factor related to the completion of the BETS.

Participants also completed a brief structured interview designed to gather basic demographic, military, and injury-related information. For the purposes of this study, the following variables were of interest: age (in years), education (in years), sex (male/female), number of years in the military, and number of deployments.

Descriptive statistics for demographic, military, and injury-related variables and select measures in the total sample is presented in Table 1.

Table 1.

Descriptive Statistics for Demographic, Military, and Injury-Related Variables and Select Measures in the Total Sample

	M	SD		n	%
Age (in years)	44.3	10.2	Control group/TBI severity
Education (in years)	15.6	2.4	Non-injured control	20	24.1
Years in the military	17.3	9.1	Injured control	22	26.5
Time since injury (in months)b	163.6	67.0	Suspected mild TBIa	7	8.4
Validity-10 total score	10.2	8.3	Uncomplicated mild TBI	25	30.1
Number of deployments	2.5	2.0	Complicated mild TBI	3	3.6
Number of TBIsb	1.4	0.7	Moderate TBI	1	1.2
BETS-GBEV (baseline)	28,288,487	145,905,787	Severe TBI	2	2.4
BETS-GBEV (follow-up)	22,048,988	141,315,907	Penetrating TBI	3	3.6
Lifetime blast exposure (1-item)	1,496	11,507	Sex: Male (vs female)	63	75.9
			Ethnicity: White (vs non-white)	59	71.1
			SVT: pass (vs fail)	72	86.7

N = 83.

TBI could not be confidently ruled-in or ruled-out after sustaining a biomechanically plausible mechanism of injury.

TBI group only.

BETS, Blast Exposure Threshold Survey; GBEV, generalized blast exposure value; SVT, symptom validity test (Validity-10); TBI, traumatic brain injury.

Statistical analysis

Statistical analysis was undertaken using SPSS 27.0. First, to examine the test-retest reliability of the GBEV and five BEC scores within the total sample, control group, and TBI group from Time 1 (T1) to Time 2 (T2), Spearman rho correlation analysis was used (due to a non-normal distribution). Test-retest reliability was interpreted using the classification system recommended by George and Mallery³⁷ as follows: r ≥ 0.90 (Excellent); r = 0.80–0.89 (Good); r = 0.70–0.79 (Acceptable); r = 0.60–0.69 (Questionable); r = 0.50–0.59 (Poor); and r < 0.50 (Unacceptable).

Second, to examine the consistency in responses on the BETS from T1 to T2 at an individual level (in the total sample, control group, and TBI group), two select questions from the BETS were examined in each of the five weapons categories separately (i.e., “Have you ever been exposed to [weapons category]”; and “How many years were you exposed?”). For each weapons category, four mutually exclusive “Test-retest Response Categories” were created by classifying participants based on (a) the endorsement of being exposed, or not exposed, to a weapons category at T1 and T2, and (b) the reported number of years exposed to the weapons category at T1 and T2. The four “Test-retest Response Categories” were defined as follows: (a) “Exposure Consistently Absent”: weapons category consistently endorsed as absent at T1 and T2; (b) “Exposure Inconsistent”: weapons category endorsed as present at T1 but endorsed as absent at T2 [or vice versa]; (c) “Exposure Present-Years Consistent”: weapons category endorsed as present at T1 and T2 and the reported number of years exposed was consistent from T1 to T2; (d) “Exposure Present-Years Inconsistent”: weapons category endorsed as present at T1 and T2 but the reported number of years exposed was inconsistent from T1 to T2. Inconsistent reporting of the number of years exposed to a weapons category was defined as a difference of <80% in the years reported when comparing T1 and T2 (e.g., 60% difference: T1 = 10 years, T2 = 6 years [6/10 = 60%]; 40% difference: T1 = 8 years, T2 = 20 years [8/20 = 40%]. Consistent reporting of the number of years exposed to a weapons category was defined as a difference of ≥80% in the years reported when comparing T1 and T2 (e.g., 90% difference: T1 = 10 years, T2 = 9 years [9/10 = 90%]; 91% difference: T1 = 22 years, T2 = 20 years [20/22 = 91%]. The prevalence of each “Test-retest Response Category” for each of the five weapons categories was reported as a percentage. In addition, the “number of inconsistent weapons categories” was calculated by summing the number of weapons categories classified as either “Exposure Inconsistent” or “Exposure Present-Years Inconsistent” for each participant (maximum of five weapons categories).

Third, to examine factors associated with inconsistent reporting (total sample only), the number of inconsistent weapons categories was compared with select demographic, military, and injury-related variables using Pearson or Spearman rho correlations (where appropriate due to normal or non-normal distributions) for continuous variables and t-tests for categorical variables (see Table 4 for a list of variables). Any variable found to be significantly associated (p < 0.05) with the number of inconsistent weapon categories was identified as a factor associated with inconsistent reporting for subsequent analyses.

Fourth, to examine the influence of factors associated with inconsistent reporting on test-retest reliability, Spearman rho correlation coefficients were calculated for GBEV and the five BEC scores in subgroups dichotomized by each of the factors identified from the previous step. For continuous variables, two subgroups were created based on a median split in scores (e.g., age low ≤44 vs. age high ≥45). Spearman rho correlations were calculated from T1 to T2 for GBEV and the five BEC scores in the two subgroups for each factor associated with inconsistent reporting.

Results

Test-retest reliability

Spearman rho correlation coefficients for GBEV and BEC scores from T1 to T2 in the total sample, and for the control and TBI groups, are presented in Table 2. In the total sample, the test-retest reliability for GBEV and four of the five BEC scores was classified as acceptable (r = 0.71–0.79). The one exception was BEC3 with test-retest reliability classified as poor (r = 0.58). When excluding participants who failed SVTs in the total sample (i.e., Total Sample + Pass SVT), there was a slight improvement in test-retest reliability for GBEV (from r = 0.76 to r = 0.78). However, this increase was negligible and was not considered a meaningful improvement. For this reason, all further analyses did not exclude participants who failed SVTs. Note that exploratory analyses revealed that the test-retest reliability of the GBEV and BEC scores did not meaningfully change when excluding participants who failed SVTs (data not shown, available on request).

Table 2.

Sperman Rho Correlation Coefficients for GBEV and BEC Scores from Time 1 to Time 2 in the Total Sample and for the Control and TBI Group

	n	GBEV	BEC1	BEC2	BEC3	BEC4	BEC5
Total sample	83	0.76	0.74	0.76	0.58	0.79	0.71
Total sample + pass SVT	72	0.78	0.80	0.73	0.59	0.77	0.81
Control groupa	42	0.83	0.79	0.71	0.54	0.79	0.67
TBI groupb,c	34	0.71	0.66	0.84	0.62	0.78	0.76

Includes injured controls and healthy controls.

Includes uncomplicated mild, complicated mild, moderate, severe, and penetrating TBI.

Does not include suspected mild TBI.

BEC, blast exposure count (weapons category 1 thru 5); GBEV, generalized blast exposure value; SVT, symptom validity test (Validity-10); TBI, traumatic brain injury.

In the control group, the test-retest reliability for GBEV was classified as good (r = 0.83). In contrast, lower test-retest reliability was found in the TBI group but remained classified in the acceptable range (r = 0.71). The majority of BEC scores in both groups had test-retest reliability coefficients classified as acceptable to good (r = 0.71–0.84). However, BEC3 and BEC5 in the control group and BEC3 and BEC1 in the TBI group had unacceptably low test-retest reliability coefficients that ranged in classification from poor to questionable (r = 0.54–0.67).

Prevalence of Test-Retest response categories

The prevalence of the four “Test-retest Response Categories” across the five BETS weapons categories in the total sample, and in the control and TBI group, is presented in Table 3. In the total sample, approximately two-thirds of the sample provided consistent responses across the five weapons categories from T1 to T2 (i.e., Column A + B = 62.4–73.5%). However, approximately one-third of the sample provided inconsistent and problematic responses across the five weapons categories from T1 to T2 (i.e., Column C + D = 26.5–37.4%). The majority of inconsistent responses across the five weapons categories (Column D = 10.8–31.3%) were the result of participants who consistently endorsed being exposed to a weapons category at both T1 and T2 but reported an inconsistent number of years of exposure to the weapons category from T1 to T2 (e.g., T1 = 10 years vs. T2 = 3 years). The remaining inconsistent responses across the five weapons categories (Column C = 8.4–15.7%) included participants who reported being exposed to a weapons category at T1, but then endorsed that they were not exposed to the same weapons category at T2 (or vice versa). Similar results were found when comparing the prevalence of the four Test-retest Response categories across the five BETS weapons categories in the control and TBI groups separately.

Table 3.

Prevalence of the “Test-Retest Response Categories” Across the Five BETS Weapons Categories in the Total Sample and the Control and TBI Group

	Consistent responses		Inconsistent responses
	A. Exposure consistently absent	B. Exposure present—years consistent	C. Exposure present—years inconsistent	D. Exposure inconsistent	Total consistentresponses(A + B)	Total inconsistantresponses(C + D)
Total sample
Weapons category 1	1.2	67.5	31.3	0	68.7	31.3
Weapons category 2	21.7	41.0	26.5	10.8	62.7	37.3
Weapons category 3	19.3	43.1	22.9	14.5	62.4	37.4
Weapons category 4	15.7	49.4	26.5	8.4	65.1	34.9
Weapons category 5	30.1	43.4	10.8	15.7	73.5	26.5
Control groupa
Weapons category 1	2.4	61.9	35.7	0	64.3	35.7
Weapons category 2	26.2	38.1	23.8	11.9	64.3	35.7
Weapons category 3	19.0	40.5	23.8	16.7	59.5	40.5
Weapons category 4	16.7	50.0	21.4	11.9	66.7	33.3
Weapons category 5	31.0	40.5	9.5	19.0	71.5	28.5
TBI groupb
Weapons category 1	0	67.6	32.4	0	67.6	32.4
Weapons category 2	20.6	44.1	23.5	11.8	64.7	35.3
Weapons category 3	23.5	41.2	23.5	11.8	64.7	35.3
Weapons category 4	14.7	55.9	23.5	5.9	70.6	29.4
Weapons category 5	32.4	41.2	11.8	14.7	73.6	26.5

Total sample (n = 83), control group (n = 42), TBI group (n = 34). A = exposure consistently absent (weapons category endorsed as absent at T1 and T2); B = exposure present—years consistent (weapons category endorsed as present at T1 and T2 and the reported number of years exposed was consistent from T1 to T2; >80%); C = exposure present—years inconsistent (weapons category endorsed as present at T1 and T2 and the reported number of years exposed was inconsistent from T1 to T2; <80%); D = exposure inconsistent (weapons category endorsed as present at T1 but endorsed as absent at T2 [or visa versa]); weapons category 1 = small and medium arms (e.g., hand held pistols, rifles, shotguns, machine guns); weapons category 2 = large arms, often shoulder-fired, that can be carried on a person (e.g., RPG, Carl-Gustaf, grenade launcher); weapons category 3 = artillery, missile weapon systems, or large arms carried by vehicle, aircraft, or boat (e.g., Howitzer, multiple rocket launcher); weapons category 4 = smaller explosives or grenades (flashbangs, small IEDs, landmines); weapons category 5 = larger explosives or targeted explosives in close range (e.g., breaching explosives, disposal explosives, heavy charges).

Includes injured controls and healthy controls.

Uncomplicated mild, complicated mild, moderate, severe, and penetrating TBI; does not include suspected mild TBI.

BETS, Blast Exposure Threshold Survey; TBI, traumatic brain injury.

Examination of the “number of inconsistent weapons categories” reported by each individual in the total sample (data not shown) revealed that the mean number of inconsistent weapons categories reported was 1.7 (SD = 1.4; Range = 0–5). Overall, 72.3% of the total sample had inconsistent responses on 1-or-more weapons categories, 50.6% had inconsistent responses on 2-or-more weapons categories, 28.9% had inconsistent responses on 3-or-more weapons categories, and 10.8% had inconsistent responses on 4-or-more weapons categories. Only 27.7% of the sample had consistent responses in all five weapons categories.

Factors associated with inconsistent responding

The relationship between the number of inconsistent weapons categories with select demographic, military, and injury-related variables is presented in Table 4. The total number of inconsistent weapons categories was significantly related to age (p = 0.011), years in the military (p = 0.003), time since injury (p = 0.019), and GBEV at baseline (p = 0.010). As such, these variables were included as factors associated with inconsistent responding in subsequent analyses. In addition, the number of deployments approached significance (p = 0.066) and was also included in subsequent analyses for exploratory purposes. Of particular note, the total number of inconsistent weapons categories was not related to symptom exaggeration (i.e., Validity-10 total, p = 0.387; Validity-10 Fail, p = 0.221) or the presence or absence of TBI (p = 0.666).

Table 4.

Relationship between the Total Number of Inconsistent Weapons Categories with Demographic, Military, and Injury-Related Variables in the Total Sample

	p	r
Pearson correlation
Age (in years)	0.011	0.28
Education (in years)	0.727	0.04
Years in the military	0.003	0.33
Time since injury (TBI only)	0.019	0.40
Validity-10 total score	0.387	−0.10
Spearman Rho correlation
Number of deployments	0.066	0.20
Number of TBIs	0.688	−0.05
BETS-GBEV (baseline)	0.010	0.28
BETS-GBEV (follow-up)	0.065	0.20
Lifetime blast exposure (1-item)	0.234	0.14

	p	d
T-tests
TBI present (vs no TBI)	0.666	0.10
Male sex (vs female)	0.931	0.02
Validity-10 fail (vs pass)a	0.221	0.44

N = 83.

Validity-10 fail < Validity-10 pass (M = 1.1, SD = 1.2 vs M = 1.7, SD = 1.4).

BETS, Blast Exposure Threshold Survey; GBEV, generalized blast exposure value; TBI, traumatic brain injury.

Influence of factors on test-retest reliability

Spearman rho correlation coefficients for GBEV and BEC scores from T1 and T2 in dichotomized subgroups across the five factors identified above that are associated with inconsistent responding are presented in Table 5. Overall, differences in test-retest reliability coefficients for GBEV were found across dichotomized subgroups for all five factors associated with inconsistent responding, with the exception of time since injury. Higher test-retest reliability coefficients were found in subgroups that had (a) a lower number of years in the military, (b) a lower number of deployments, and (c) younger in age. For each of these subgroups, test-retest reliability coefficients were classified as “good” (r = 0.82–0.85). In comparison, unacceptably low test-retest reliability was found in subgroups that had (a) a higher number of years in the military, (b) a higher number of deployments, and (c) older in age. For each of these subgroups, test-retest reliability coefficients were classified as poor to questionable (r = 0.58–0.69). In addition, higher test-retest reliability coefficients were found in those participants who had lower GBEV scores (r = 0.65, questionable) compared with higher GBEV scores (r = 0.39, unacceptable). Nonetheless, both coefficients fell below the acceptable classification.

Table 5.

Spearman Rho Correlation Coefficients for GBEV and BEC Scores from Time 1 to Time 2 in Dichotomized Subgroups Across the Five Factors Associated with Inconsistent Responding

		GBEV	BEC1	BEC2	BEC3	BEC4	BEC5
	n	r	r	r	r	r	r
Lifetime blast exposure
Lower GBEVa,b	41	0.65	0.69	0.62	0.72	0.70	0.72
Higher GBEV	42	0.39	0.62	0.45	0.38	0.71	0.51
Years in the military
Lower (3–18)	41	0.82	0.80	0.90	0.67	0.69	0.76
Higher (19–40)	42	0.69	0.67	0.56	0.45	0.79	0.59
Age
Lower (24–44)	40	0.85	0.83	0.89	0.68	0.72	0.73
Higher (45–67)	42	0.61	0.66	0.59	0.44	0.77	0.64
Number of Deployments
Lower (0–2)	46	0.82	0.73	0.85	0.54	0.73	0.78
Higher (3–10)	37	0.58	0.63	0.46	0.51	0.78	0.51
Time since injuryc
Lower (1–12 years)	17	0.79	0.79	0.87	0.62	0.77	0.84
Higher (13–25 years)	17	0.74	0.54	0.56	0.62	0.77	0.69

N = 83.

±GBEV = 515,007.

When using a GBEV cutoff of 200,000 to create groups, a similar pattern of test-retest reliability coefficients was found: GBEV <200,000 = 0.61, GBEV ≥200,000 = 47.

TBI group only.

BEC, blast exposure count (weapons category 1 thru 5); BETS, Blast Exposure Threshold Survey; GBEV, generalized blast exposure value; TBI, traumatic brain injury.

Discussion

The purpose of this study was to examine the test-retest reliability of the BETS over a 3-week period in a sample of United States SMVs. There are three important findings from this study that warrant highlighting.

Statistically, the test-retest reliability of the BETS GBEV was “acceptable”. In the total sample, the test-retest reliability for the BETS GBEV was r = 0.76, which falls firmly within the “acceptable” classification (i.e., r = 0.70–0.79; George and Mallery, 2003). Establishing the psychometric properties of measures used to assess a condition or behavior is considered a fundamental tenet for any new measure to ensure accurate and reliable interpretation of results. There are many types of psychometric data that can be generated (e.g., internal consistency reliability, content validity, criterion validity, test-retest reliability, convergent validity, discriminant validity). Given that the BETS (and other measures like it) relies on a person’s self-report of blast exposures over many years, which can be subject to reporting bias, the establishment of test-retest reliability, convergent validity, and discriminant validity is considered to be the bare minimum threshold that must be established before the BETS can be confidently recommended for use as a valid and reliable measure of LBE in a clinical or research setting. In an earlier study by our group,²⁴ we provided data to support the convergent and discriminant validity of the BETS in a sample of 202 SMVs. In the current study, these data support the test-retest reliability of the BETS in a sample of 83 SMVs. Together, these two studies provide important psychometric data that meet the minimum statistical threshold for the BETS to be recommended for use in a clinical or research setting.

Concerning response inconsistencies were found on the BETS over time. Despite the statistically acceptable test-retest reliability found in this sample, these data may be somewhat misleading. When examining the individual responses from T1 to T2, approximately one-third of the sample provided inconsistent and problematic responses across the five weapons categories separately. When considering all weapons categories simultaneously, 72.3% of the total sample had inconsistent responses on 1-or-more weapons categories, and 50.6% had inconsistent responses on 2-or-more weapons categories. The most problematic inconsistency was the failure of some participants to consistently endorse, or not endorse, exposure to a weapons category at T1 and T2 (e.g., T1 = exposure present; T2 = exposure absent). Inconsistent responding in this manner is perplexing and difficult to explain. It is possible that the definitions of the weapons categories on the BETS are insufficient and require expansion or revision or are obsolete and need updating. However, both of these explanations seem unlikely given that the BETS was only recently developed, and operational definitions of the weapons categories were created based on end-user feedback. Nonetheless, obtaining additional feedback from SMVs regarding the operational definition of the weapons categories in a large group of SMVs is recommended to ensure accuracy and appropriateness of the weapons categories used.

Less problematic, but also concerning, was the failure of some participants to consistently report the same number of years in which they were exposed to a weapons category from T1 and T2 (e.g., T1 = 10 years; T2 = 5 years). While an arbitrary cutoff of <80% difference in years reported from T1 to T2 was used to define inconsistent reporting in this study, there were a large number of notable inconsistencies much lower than this cutoff (e.g., [a] T1 = 3 years; T2 = 19 years [16% diff]; [b] T1 = 21 years; T2 = 6 years [29% diff]; [c] T1 = 3 years; T2 = 10 years [30% diff]). Inconsistent responding on the BETS in this manner is concerning and can potentially affect GBEV scores. The inconsistencies noted here are likely the result of how the information is gathered relating to LBE. The BETS requires the respondent to report the number of times they were exposed to a specific weapons category in their lifetime. The problem with this method is that blast exposure, particularly blasts caused by larger weapons, typically occurs in periods of time by way of combat deployments or training, rather than consistent weekly or monthly exposure throughout a person’s lifetime. As such, some SMVs likely have difficulty reporting blast exposures in this manner because they are required to average the number of exposures across their entire lifetime, rather than reporting periods of blast exposure as they naturally occur. This is supported by anecdotal reports from SMVs during the course of study who noted difficulty completing the BETS because of this issue. One potential solution to this problem for the BETS would be to ask the existing questions stratified by periods of time that makes sense for the SMV (e.g., combat deployments, training periods, or other meaningful periods in their life). Ideally, these periods of time would be identified using a flexible approach that makes sense for the SMV, rather than dictated a priori by the measure. However, one significant disadvantage of this approach is that the BETS would become much longer and potentially too cumbersome to complete. Nonetheless, efforts to revise the BETS with this issue in mind would be considered prudent.

Response inconsistencies were associated with age, years in the military, number of deployments, and GBEV. Overall, inconsistent responses and lower test-retest reliability were found in participants that were (a) older in age, (b) had a higher number of years in the military, (c) had a higher number of deployments, and (d) had higher GBEV scores. There is a hypothetical relationship between these four variables that might explain why these factors have been identified as being associated with inconsistent reporting. Participants who are older in age have typically served in the military for a longer period of time. The longer period of time a person serves in the military, the more likely they will have been deployed on multiple occasions and/or completed multiple periods of training involving blast exposure. This combination of factors would then result in a high likelihood that an SMV has a high number of blast exposures. Given the difficulty of retrospectively recalling blasts over time, intuitively, it would be reasonable to conclude that it would be much more difficult for a person to consistently recall a large number of blast exposures over a long period of time compared with recalling a small number of blast exposures over a short period of time. Consequently, as the frequency and years of blast exposure reported increases, our confidence in the accuracy of what is reported would subsequently decrease.

In sum, this study provides data to support the test-retest reliability of the BETS, and when combined with earlier work by Lange and colleagues,²⁴ now meets the minimum statistical threshold for the BETS to be recommended for use in a clinical or research setting (i.e., test-retest reliability, convergent validity, and discriminant validity). That said, a number of response inconsistencies were identified on the BETS that are concerning. However, it is important to appreciate that the response inconsistencies reported on the BETS are not considered to be unique to the BETS. Rather, it is likely the result of how the information is gathered relating to LBE (i.e., exposures averaged over a person’s lifetime) that is a problem commonly faced by similar measures (e.g., BOOM and B-FASS). In an effort to improve the BETS, it is considered prudent to examine the possibility of revising the BETS to gather blast-related information in periods of time that make sense to the SMVs (e.g., deployments, training, or other meaningful periods in their life) rather than over their entire life. In addition, obtaining feedback from SMVs regarding the operational definition of the weapons categories to ensure accuracy and appropriateness is further recommended.

Transparency, Rigor, and Reproducibility Statement

This study was not formally registered. The analysis plan was not formally preregistered, but the lead author certifies that the analysis plan was prespecified. A minimum sample size of 83 participants was planned based on availability of data. Data collection was performed by investigators who were aware of relevant participant characteristics. Data analyses were performed by investigators who were aware of relevant characteristics of the participants. Statistical analysis was performed by the lead author using SPSS. No replication or external validation studies have been performed or are planned/ongoing at this time to our knowledge. Summary/aggregate data and additional information on the methods and statistical analyses will be provided on request. Analytic code used to conduct the analyses presented in this study is not available in a public repository. They may be available by emailing the corresponding author. The authors agree to provide the full content of the article on request by contacting the corresponding author.

Authors’ Contributions

R.T.L.: Conceptualization, data curation, formal analysis, methodology, supervision, and writing—original draft. L.M.F.: Conceptualization, supervision, and writing—review and editing. S.M.L.: Conceptualization, data curation, supervision, and writing—review and editing. K.C.G.: Data curation and writing—review and editing. S.B.: Data curation and writing—review and editing. J.M.B.: Supervision and writing—review and editing. L.D.H.: Supervision and Writing—review and editing. T.A.B.: Conceptualization, methodology, supervision, and writing—review and editing.

Footnotes

Acknowledgments

The authors would like to express gratitude to the SMVs for their time and commitment to participating in this research. The authors would also like to acknowledge the efforts of the larger team of research coordinators, research associates, research assistants, program managers, and senior management who contribute to the DVBIC-TBICoE 15-Year Longitudinal TBI Study.

Author Disclosure Statement

All authors have no conflicts to disclose.

Funding Information

This work was prepared under Contract HT0014-22-C-0016 with DHA Contracting Office (CO-NCR) HT0014 and, therefore, is defined as U.S. Government work under Title 17 U.S.C.§101. Per Title 17 U.S.C.§105, copyright protection is not available for any work of the U.S. Government. For more information, please contact dha.TBICOEinfo@health.mil.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the official policy or position of the Defense Health Agency, Department of Defense, or any other U.S. government agency.

References

Engel

, Hoch

, Simmons

. The Neurological Effects of Repeated Exposure to Military Occupational Blast Implications for Prevention and Health. RAND Corporation: Santa Monica, CA; 2018.

National Defense Authorization Act for Fiscal Year 2020 2019 (USA) 116th Congress (2019-2020). Available from: https://www.congress.gov/bill/116th-congress/senate-bill/1790

Gill

, Cashion

, Osier

, et al. Moderate blast exposure alters gene expression and levels of amyloid precursor protein. Neurol Genet, 2017; 3(5):e186.

Martindale

, Shura

, Rostami

, et al. Research letter: Blast exposure and brain volume. J Head Trauma Rehabil, 2021; 36(6):424–428.

Robinson

, Clark

, Milberg

, et al. Characterization of differences in functional connectivity associated with close-range blast exposure. J Neurotrauma, 2017; 34(S1):S53–S61.

Rowland

, Stapleton-Kotloski

, Martindale

, et al. Alterations in the topology of functional connectomes are associated with post-traumatic stress disorder and blast-related mild traumatic brain injury in combat veterans. J Neurotrauma, 2021; 38(22):3086–3096.

Carr

, Polejaeva

, Grome

, et al. Relation of repeated low-level blast exposure with symptomology similar to concussion. J Head Trauma Rehabil, 2015; 30(1):47–55.

Reid

, Miller

, Lange

, et al. A multisite study of the relationships between blast exposures and symptom reporting in a post-deployment active duty military population with mild traumatic brain injury. J Neurotrauma, 2014.

Martindale

, Ord

, Rule

, et al. Effects of blast exposure on psychiatric and health symptoms in combat veterans. J Psychiatr Res, 2021; 143:189–195.

10.

Bailie

, French

, Lippa

, et al. High levels of blast exposure are associated with worse neurobehavioral functioning following traumatic brain injury in US military service members and veterans. J Neurotrauma, 2022 in press.

11.

Vartanian

, Rhind

, Nakashima

, et al. Blast effects on post-concussive and mental health outcomes: Data from Canadian armed forces breachers and snipers. J Mil Veteran Fam Health, 2022; 8(s2):82–96.

12.

Lange

, French

, Bailie

, et al. Clinical utility of PTSD, resilience, sleep, and blast as risk factors to predict poor neurobehavioral functioning following traumatic brain injury: A longitudinal study in U.S. military service members. Qual Life Res, 2022; 31(8):2411–2422.

13.

Tsao

, Stentz

, Rouhanian

, et al. Effect of concussion and blast exposure on symptoms after military deployment. Neurology, 2017; 89(19):2010–2016.

14.

Bailie

, Lippa

, Hungerford

, et al. Cumulative blast exposure during a military career negatively impacts recovery from traumatic brain injury. J Neurotrauma, 2024; 41(5–6):604–612.

15.

Lange

, French

, Lippa

, et al. High lifetime blast exposure using the blast exposure threshold survey is associated with worse warfighter brain health following mild traumatic brain injury. J Neurotrauma, 2024; 41(1–2):186–198.

16.

Lippa

, Bailie

, French

, et al. Lifetime blast exposure is not related to cognitive performance or psychiatric symptoms in US military personnel. Clin Neuropsychol, 2024:1–23.

17.

Lippa

, Yeh

, Kennedy

, et al. Lifetime blast exposure is not related to white matter integrity in service members and veterans with and without uncomplicated mild traumatic brain injury. Neurotrauma Rep, 2023; 4(1):827–837.

18.

Lippa

. A review of long-term outcomes of repetitive concussive and subconcussive blast exposures in the military and limitations of the literature. Clin Neuropsychol, 2024:1–36.

19.

Roy

, Keyser

, Rowe

, et al. Methodology of the investigating training associated blast pathology (INVICTA) study. BMC Med Res Methodol, 2022; 22(1):317.

20.

Modica

, Egnoto

, Statz

, et al. Development of a blast exposure estimator from a department of defense-wide survey study on military service members. J Neurotrauma, 2021; 38(12):1654–1661.

21.

Rhodes

, Sandlain

, Ollinger

, et al. Development of the Blast Ordnance and Occupational Exposure Measure for Self-Reported Lifetime Blast Exposures Military Medicine. ePub Ahead of Print; 2022.

22.

Turner

, Sloley

, Bailie

, et al. Perspectives on development of measures to estimate career blast exposure history in service members and veterans. Front Neurol, 2022; 13:835752.

23.

Rowland

, Martindale

, Spengler

, et al. Sequelae of blast events in Iraq and Afghanistan War veterans using the Salisbury blast interview: A CENC study. Brain Inj, 2020; 34(5):642–652.

24.

Lange

, French

, Lippa

, et al. Convergent and discriminant validity of the blast exposure threshold survey in United States military service members and veterans. J Neurotrauma, 2024; 41(7–8):934–941.

25.

Silverberg

, Iverson

, Cogan

, et al.; ACRM Mild TBI Diagnostic Criteria Expert Consensus Group. The American congress of rehabilitation medicine diagnostic criteria for mild traumatic brain injury. Arch Phys Med Rehabil, 2023; 104(8):1343–1355.

26.

Corrigan

, Bogner

. Initial reliability and validity of the Ohio State University TBI identification method. J Head Trauma Rehabil, 2007; 22(6):318–329.

27.

Vanderploeg

, Cooper

, Belanger

, et al. Screening for postdeployment conditions: Development and cross-validation of an embedded validity scale in the neurobehavioral symptom inventory. J Head Trauma Rehabiliation, 2014; 29(1):1–10.

28.

Cicerone

, Kalmar

. Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury. J Head Trauma Rehabil, 1995; 10(3):1–17.

29.

Armistead-Jehle

, Cooper

, Grills

, et al. Clinical utility of the mBIAS and NSI validity-10 to detect symptom over-reporting following mild TBI: A multicenter investigation with military service members. J Clin Exp Neuropsychol, 2018; 40(3):213–223.

30.

Dretsch

, Williams

, Staver

, et al. Evaluating the clinical utility of the Validity-10 for detecting amplified symptom reporting for patients with mild traumatic brain injury and comorbid psychological health conditions. Appl Neuropsychol Adult, 2017; 24(4):376–380.

31.

Sullivan

, Lange

, Edmed

. Utility of the neurobehavioral symptom inventory validity-10 index to detect symptom exaggeration: An analogue simulation study. Appl Neuropsychol Adult, 2016; 23(5):353–362.

32.

Lippa

, Lange

, Bailie

, et al. Utility of the Validity-10 scale across the recovery trajectory following traumatic brain injury. J Rehabil Res Dev, 2016; 53(3):379–390.

33.

Lange

, Brickell

, Lippa

, et al. Clinical utility of the neurobehavioral symptom inventory validity scales to screen for symptom exaggeration following traumatic brain injury. J Clin Exp Neuropsychol, 2015; 37(8):853–862.

34.

Lange

, Brickell

, French

. Examination of the mild brain injury atypical symptom scale and the validity-10 scale to detect symptom exaggeration in US military service members. J Clin Exp Neuropsychol, 2015; 37(3):325–337.

35.

Lange

, French

, Lippa

, et al. Posttraumatic stress disorder is a stronger predictor of long-term neurobehavioral outcomes than traumatic brain injury severity. J Trauma Stress, 2020; 33(3):318–329.

36.

Lange

, Yeh

, Brickell

, et al. Postconcussion symptom reporting is not associated with diffusion tensor imaging findings in the subacute to chronic phase of recovery in military service members following mild traumatic brain injury. J Clin Exp Neuropsychol, 2019; 41(5):497–511.

37.

George

, Mallery

. SPSS for Windows Step by Step: A Simple Guide and Reference. 11.0 Update, (4th ed.). Allyn & Bacon: Boston; 2003.