Attitudinal Predictors of No-Show to Substance Use Disorder Treatment Intake in Veterans

Abstract

Background:

Substance use disorder (SUD) exacts a devastating psychosocial and medicolegal toll in Veterans. Despite the availability of quality SUD care provided by the Veterans Health Administration, many Veterans with SUD who initially seek treatment do not attend their intake appointment that helps determine their assigned level of care. Demographic and administrative factors have been used to predict outpatient no-shows, but whether patient-reported attitudes can be feasibly probed prior to the intake appointment to potentially predict intake no-show is largely unexplored.

Objectives:

We wished to determine whether patient attitudes about SUD and SUD treatment can be feasibly probed in veterans and whether such attitudes could potentially predict intake no-show.

Methods:

In a preliminary feasibility study, we administered by mail, smartphone app, or by in-person or telephone verbal interview a list of potential standard-of-care intake interview questions regarding attitudes about self and SUD treatment in n = 79 veterans scheduled for SUD treatment intake, of whom 29% did not attend the intake. To examine the power of attitudinal factors to predict intake no-show, we utilized decision tree-based machine learning (ML) analysis of veteran responses about addiction beliefs and other patient factors.

Results:

ML analyses indicated that older age, longer time before intake date as well as self-reported low levels of medical adherence were independently predictive of intake no-show. Regarding attitudinal factors, no-show was also predicted by low endorsements of craving, and high endorsements of having “an addiction,” trust in previous SUD treatment providers, and existing engagement in the recovery community.

Conclusion:

These preliminary data suggest the feasibility and potential predictive utility of querying Veterans slated for treatment intake about their attitudes regarding SUD and SUD treatment. Future replication studies with a larger sample size could yield briefer, lower-burden assessments of attitudinal no-show risk factors that could be targeted by pre-intake motivational interventions.

Keywords

addiction treatment machine learning no-show attendance risk

Introduction

Substance Use Disorders (SUDs) exact a substantial psychosocial and economic toll,¹ especially in active-duty service members²; as well as in veterans,^3,4 who show increased rates of binge drinking and substance use compared to civilian age peers (eg, 38% past-year illicit substance use⁵). For example, 59% of military personnel deployed in Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) reported binge drinking.⁶ In veterans, Fuehrlein et al⁷ reported a 42.2% prevalence of lifetime alcohol use disorder (AUD), and a past-year rate of probable AUD was 14.8%. SUDs can also increase risk for, exacerbate, or result from psychiatric disorders frequently found in military and veteran populations.⁸ For example, veterans with lifetime AUD had significantly increased odds of mood disorders (two to fourfold), drug use disorder (tenfold), lifetime suicide attempt (fourfold), and current suicidal ideation (twofold), compared to veterans without lifetime AUD.⁷ In OEF/OIF veterans, the effect of post-traumatic stress disorder (PTSD) symptoms on mental-health-related quality of life was explained only by comorbid alcohol-related psychosocial consequences.³ Finally, SUD is also linked with increased suicidal behavior,⁹ intimate partner violence, and other violence.¹⁰

For these reasons, it is critical to connect veterans with the comprehensive SUD care available to them at facilities operated by the Veterans Health Administration (VHA).¹¹ The VHA offers several programs and initiatives devoted to encouraging veterans with substance misuse to contact a local VHA facility to enroll in cognitive behavioral therapy (CBT) treatment programs for SUD.¹² VHA treatment enrollment generally involves a private intake interview, wherein a social worker or other provider queries the veteran on substance use history and related psychosocial problems as well as related medical and psychiatric history. This intake interview information is then evaluated by the SUD care team for recommended level of SUD treatment, ranging at most facilities from once-weekly outpatient group sessions up to multi-week residential treatment.

A key challenge to SUD treatment enrollment, however, is the volatility of patient motivation for behavior change. Anecdotally, SUD treatment providers frequently observe waxing and waning of an individual’s motivation for treatment and motivation to abstain generally. This volatility is also evident in ecological momentary assessment studies of persons in SUD recovery, from whom ratings of intention to stay in treatment as well as ratings of drug craving can show marked fluctuations within and across days.¹³ One costly manifestation of low or inconsistent motivation for treatment is not attending scheduled SUD treatment appointments. In an analysis of VHA administrative data on over 270 000 veterans identified as having an SUD and referred for VHA SUD treatment, only ~30% ultimately attended at least 1 session of treatment, where rates of treatment initiation were especially low in veterans who were older, married, or referred in non-SUD care contexts.¹⁴ Similar studies in civilian settings have shown that as many as 42% of patients who have undergone an SUD treatment intake appointment fail to attend any subsequent therapy sessions (discussed in Weisner et al.¹⁵). In addition to impairing SUD recovery, no-shows for SUD treatment intake incur a cost burden to the VHA as a facet of the larger problem of outpatient appointment nonattendance faced by health systems. Moreover, VHA does not double-book appointments like the private sector. Thus, no-shows have greater potential for wasting provider time and delaying care for other veterans.

Indeed, patient no-shows have prompted the widespread practice of appointment double-bookings in private sector health care. Double-booking strategies have been informed by novel machine-learning (ML) from administrative records to identify scheduled patients most likely to not show up.^16,17 These ML approaches typically rely on: (1) sociodemographic variables, (2) appointment scheduling variables (eg, day of the week), and (3) prior history of no-shows.¹⁸ Although no-show-predictive factors (and even values of predictors such as male vs female) have varied across studies and contexts, ML studies have generally found that lower socioeconomic status (SES), lack of insurance coverage, distance to clinic, weekday, visit time of day, appointment lead time, and previous no-shows were predictive of future no-shows, with previous no-shows the most predictive variable.^16,19 While these approaches have at times achieved high predictive accuracy, reliance on purely administrative and demographic data (eg, low SES) for no-show prediction and double-booking has been thought to perpetuate health inequities.²⁰

Moreover, the relative impacts of no-show predictors or motivations for attendance observed in other medical care contexts may differ in the SUD population or may differ for SUD treatment itself, such that traditional techniques to improve attendance may not be as effective for SUD care. For example, whereas use of text message reminders showed significant (29%-39%) reduction in non-attendance rates for general hospital appointments,²¹ neither pre-treatment reminder phone calls,²² nor text reminders²³ appreciably reduced SUD treatment initiation no-show. This suggests that simple forgetfulness or disorganization may be less of a factor for SUD treatment intake attendance than for other care. For example, veterans with SUD are more likely to be homeless or otherwise socioeconomically marginalized²⁴ and so may face more logistical barriers to intake attendance, such as transportation, which could be ameliorated.²⁵ Veterans with SUD are also highly likely to have significant psychiatric comorbidities^6,8 that could depress attendance, where for example no-show for colonoscopy follow-ups among veterans was predicted by presence of comorbid psychiatric disorder.²⁶

We contend that impediments to SUD care attendance may also be fundamentally different from care for other severe medical conditions like diabetes or chronic kidney disease in that insight and attitude toward care may play a more central role in following through with initiation of care. As with other mental illnesses, SUD diagnosis itself is subjective and can be prone to patient denial due to lack of insight.²⁷ Moreover, SUD has appreciable potential for spontaneous remission in that substance use is theoretically volitional, and individuals can attain abstinence without professional assistance.²⁸ Finally, because cost barriers to VHA SUD treatment are low in veterans, the role of attitudes and other psychological factors in decisions to attend an intake interview may be uniquely impactful. First, VHA SUD care is widely promoted and is typically provided to veterans at little to no cost to them. Second, the rapid expansion of telehealth for VHA addiction care at in the wake of the COVID-19 pandemic²⁹ has reduced transportation barriers to treatment engagement, by including telehealth-based intake appointments.

These factors collectively suggest that non-attendance for SUD intake by a veteran may be unusually dependent on individual attitudes about his or her condition, such as whether the individual sees himself or herself as having an addiction that requires professional assistance, or whether the veteran believes that therapy will help. Despite how attitudes about SUD therapy and its providers have been linked to SUD treatment success,³⁰ very little is known about how differences between prospective SUD care patients in their self-efficacy, personality, or specific attitudes about addiction and SUD treatment may portend failure to show up for the initial SUD intake appointment itself. For example, are psychological factors such as self-efficacy for change or the patient’s perceived importance of SUD recovery predictive of SUD treatment intake attendance like they are predictive of dropout from ongoing treatment³¹? Finally, identifying problematic attitudes and other psychological barriers to SUD treatment initiation is important because as with transportational barriers,²⁵ some of these beliefs and attitudes could be potentially modifiable during and after initial conversations between a prospective patient and a treatment program, such as with motivational interviewing³² during or following the intake appointment scheduling call.

Presented here is an analysis of data collected under a larger parent program improvement project intended to determine whether certain expressed patient attitudes about addiction and its treatment (eg, patient commitment to success; thoughts about previous SUD providers) can be feasibly asked of veterans at or after scheduling of the treatment intake appointment and whether patient responses may be predictive of VHA SUD treatment engagement and outcomes at different levels of care. In brief, we first solicited from local VA Medical Center and academic affiliate SUD providers individual questions they thought were seldom asked but could yield useful insights about patient attitudes and motivation that would affect outcomes. We then compiled this list of potential interview questions and presented them to veterans scheduled to undergo an in-person or telehealth SUD treatment intake appointment. Participant responses to the questions were related to future attendance at their intake interview as scheduled, using machine learning (ML) models. Although our ML analyses were intended to be agnostic and data-driven, based on previous administrative dataset findings coupled with anecdotal clinical impressions, we nevertheless predicted that delay to the appointment and (candidate intake interview question) responses indicative of poor medical adherence as well as low commitment or low self-efficacy toward recovery would be predictive of treatment intake no-show. We also predicted that answering the candidate interview questions would not be perceived by the veterans as unfeasibly onerous.

Methods

Overview

All participant recruitment methods and study procedures were reviewed and approved by the local Veterans Affairs Medical Center (VAMC) Institutional Review Board as protocol (IRBNet) 1572841. These data were originally collected as part of a parent study on specific patient-level predictors of treatment success at different levels of SUD care among veterans enrolled in the VAMC Substance Abuse Treatment Program (SATP). We utilized a subset of the data to examine the predictive power of patient attitudes regarding addiction (from candidate intake interview questions) together with age and time-to-appointment (as variables found in several studies to predict initial treatment appointment no-show) to predict intake appointment no-show. To minimize overfitting a small sample size, additional demographic variables were not entered.

Participants

Inclusion Criterion

Participants in the parent study were veterans who contacted the local VAMC SATP and had either recently undergone a treatment intake interview or were scheduled for that interview, but who had not yet begun therapy. Each veteran was referred to the pilot study staff by the coordinator of intakes into the SATP. Out of an initial pool of n = 899 Veterans, n = 710 were contacted (by telephone, including voice mail). Of those contacted, n = 116 agreed to participate and answered the candidate intake questions.

Exclusion Criterion (for this Analysis)

Due to the focus of this analysis on prediction of future intake no-show, Veterans who already completed their intake appointment were excluded. Each eligible veteran was then telephoned by a research team member, who described the interview questions about SUD and treatment attitudes that would be asked. The verbal willingness (consent) of the veteran to participate was collected and electronically documented. Our final sample of 79 participants included 69 males, ranged in age from 24 to 75 (mean 50.8 ± SD 13.7), and was predominantly composed of Black veterans (n = 50), with n = 26 white, and n = 3 Hispanic. For this sample herein, the intake appointment was scheduled within the next 2 weeks (median = 7 days).

Candidate Interview Questions About Self, SUD, SUD Treatment, and SUD Providers

Each veteran was administered a compiled list of 40 potential intake interview questions (see Supplemental Online document). These questions pertained to attitudes about self, about addiction and about addiction treatment and were compiled from a survey of SUD treatment providers at the VAMC and at its local academic affiliate. Providers were asked to identify patient beliefs and attitudes thought to impact SUD treatment success, including those pertinent to self-efficacy. Most items were Likert-scale format, where participants rated on a scale of 1 to 5 the degree to which they agreed with statements like: “I trust and respect my previous substance abuse treatment providers,” and “I am confident that I will achieve the goals that I set for myself.” Other more medical items included: “Do you have non-cancer related chronic pain (consistent daily pain for 3 months or longer)? (_____Yes or ____No).” Each question also gave the veteran the option: “prefer not to answer.” Participants completed the list of questions by paper mail, phone interview, in person interview, or by using the Inquisit 6 Web app (Millisecond Software LLC, Seattle, WA).

Veteran Perception of Questions

To probe feasibility and tolerability of this information gathering, following completion of the list of questions about self, SUD, and SUD treatment, participants were asked to complete a 2-item survey about their perceptions of the potential interview questions. The first, relevance-focused question asked them to rate: “I found the questions about myself and my attitudes about addiction to be”: with response options ranging from 0 =“Irrelevant/Stupid” to 5 = “OK,” to 10 = “Interesting/Important”. The second, tolerability-focused question asked: “I found answering the questions about myself and my attitudes about addiction to be”: with response options ranging from 0 = “Very unpleasant” to 5 = “OK,” to 10 = “Pleasant.”

Intake Attendance

EMR and SATP program director tracking logs were used to determine treatment intake appointment attendance. Veterans who called the SATP back immediately after missing their intake and completed it within 24 hours were counted as having attended their originally scheduled intake appointment. In all, n = 23 (29%) of the 79 veterans of this analysis sample did not attend their scheduled intake appointment at all. The subsequent analytic plan is aimed at identifying predictors (features) of these 23 no-show veterans compared to the 56 veterans who attended their scheduled intake.

Statistical Analysis

First, a decision tree-based machine learning (ML) model was applied to specific individual questions to identify potentially predictive variables. Notably, ML allows for concurrent entry into the same predictive model several potentially highly intercorrelated variables. A companion analysis focused on the global interpretability of some predictor variables (eg, directionality of that variable’s effect and the role of certain individual participant responses) using SHAP (SHapley Additive exPlanations) analysis, thereby explaining the expected behavior of the fitted ML model with respect to the distributions of the predictor variables.

Question responses together with demographics and other variables such as delay to intake and mode of question set administration were related to future intake attendance on 2 levels: First, a gradient-boosted decision tree-based machine learning (ML) model, XGBoost,³³ was applied to specific individual question set items to identify potentially predictive variables. As with other ML-based investigations, this analysis was intended to agnostically (ie, data-driven) reveal a set of attendance-predictive factors.

All statistical analyses were performed using R software (version 4.4.1, Vienna, Austria). Because some veterans declined to answer some questions, we imputed all missing values in the question responses using the predictive mean matching (PMM) approach. For this purpose, after performing appropriate statistical tests, we assumed that observations for certain variables were missing at random (MAR)³⁴ in that the probability of a value being missing in 1 question appeared unrelated to probability of missing data from that participant in another variable. The PMM approach is a semi-parametric imputation method, which has been widely recommended for imputing missing data, particularly under the MAR assumption, and it does not assume a specific distribution for the variable with missing data (such as normality), making it a flexible and robust imputation technique.³⁵ The PMM approach was implemented using the ‘mice’ package in R.³⁶

As part of the ML analysis, the XGBoost algorithm was applied to identify the most significant predictors of failure to attend from individual question responses, demographic data, and the mode of question set administration. XGBoost (Extreme Gradient Boosting) is a powerful ensemble decision tree algorithm widely used for its efficiency and performance.³⁷ It belongs to the class of gradient boosting methods, where multiple decision trees are constructed sequentially, with each new tree correcting the errors of its predecessor. This method iteratively builds trees, giving more weight to predictors that improve the model’s accuracy, ultimately producing a robust model that learns from its errors and progressively enhances predictive performance. Finally, XGBoost analysis requires no assumptions of the normality or other distributions of values of its input variables, which is critical for detection of predictive features with values that show a skewed distribution, as may be found in some question set items.

XGBoost is known to be robust to class imbalance due to its ability to incorporate customized loss functions and weighting schemes. In our analysis, we used the scale_pos_weight parameter to upweight the minority class (ie, Veterans who did not show up for medicine intake) by taking scale_pos_weight = sqrt(number of negative cases/number of positive cases), thereby ensuring that the model does not overfit to the majority class. In the context of our study, a positive case refers to a Veteran who did not show up for their scheduled intake (ie, a no-show), whereas a negative case refers to a Veteran who attended their scheduled appointment. This weighting of the minority class avoids the limitations of sampling-based methods such as Synthetic Minority Oversampling Technique³⁸ or downsampling, which can introduce bias or loss of information.³⁹ Beyond class weighting, XGBoost incorporates L1 and L2 regularization which helped reduce overfitting, and is highly efficient computationally, which allowed for extensive hyperparameter tuning and cross-validation.

In addition, XGBoost also performs embedded feature selection by evaluating how each variable contributes to reducing the model’s loss function across all decision trees. This built-in mechanism enables the model to capture complex nonlinearities and interactions among variables, which makes it more effective than traditional univariate or stepwise feature selection methods that treat predictors independently. This offers a more nuanced and data-driven approach to identifying influential predictors affecting no-show behavior among Veterans.

Our feature inclusion strategy involved initially training the model on all available predictors, followed by interpretability-based assessment (SHAP values) using the model’s output. Features with minimal contribution were not emphasized in the interpretation, but we avoided manual exclusion during model training to preserve the objectivity of the data-driven selection process.^38,39 The model was fit on 80% of the data (training set) using the “xgboost” package in R³³ with hyperparameters eta = 0.3, max_depth = 10, subsample = 0.9, colsample_bynode = 1, and gamma = 0.5. These hyperparameter choices directly controlled model complexity, thereby preventing overfitting. Within each iteration, the model chose 90% of the training set data randomly to fit the trees and calculated training set accuracy based on the remaining 10% of the data. We tested the prediction accuracy of the model based on the remaining 20% of the data (test set) to demonstrate the predictive performance of our model. Due to the small sample size, we used repeated random sub-sampling validation⁴⁰ to assess our model’s predictive performance. Specifically, we performed 100 iterations where, in each iteration, the dataset was randomly partitioned into training and test sets. The model was trained on each training set and evaluated on the corresponding test set. We then calculated the average training accuracy and average test set prediction accuracy across these 100 iterations to assess model performance.

Second, this analysis focused on the global interpretability of the predictor variables using SHAP (SHapley Additive exPlanations),⁴¹ thereby explaining the expected behavior of the fitted ML model with respect to the distributions of the predictor variables by computing Shapley values (SHAP values). Shapley values quantify the contribution of each predictor variable to a model’s output by comparing the change in prediction when a feature’s value is replaced with an average baseline value. Notably, variables identified in SHAPley analysis may not be the same as predictive/impactful variables of the core (non-directional) ML analysis. Rather, the SHAPley is a complementary approach to detect and clarify the impact of unusual or idiosyncratic variable values (cases) toward the full model, such as for question set items with highly skewed distributions with outliers. The SHAP values were calculated using the “SHAPforxgboost” package in R.

Results

Patterns of Question Responses

The distribution of values of the individual question items (n = 39 Likert scale, n = 1 binary (yes/no)) encompassed the entire range of possible values for all 39 Likert-scale items (see Supplemental Table 1 online). However, none of the individual items elicited a normal distribution of response ratings (all Shapiro-Wilk test P < .001), where some items showed a skewed distribution of responses, underscoring the importance of the machine learning analysis approaches. Notably, 23 variables, including demographic variables, were found to have complete data with no missing values. To examine the missing data patterns in our dataset, we first conducted an exploratory analysis to assess the extent and distribution of missing values across individual questionnaire items. The overall proportion of missing data in the dataset was 1.23% and the question with the highest proportion of missing responses was “I trust and respect my previous substance abuse treatment providers,” with approximately 18% missing values. Questions with the next-highest incidence of missingness pertained to craving intensity, current engagement in the recovery community, and following provider recommendations (for full details on missingness mechanism and quality of imputation, please see Supplemental Online Information). In brief, we performed Little’s MCAR test⁴² to conclude that the missingness mechanism in the data was not MCAR, and performed imputations via PMM under the assumption of MAR. We then examined item means, medians, standard deviations, and item intercorrelations before and after performing multiple imputations using PMM. We focused on the distribution of answers to the “trust” question before and after imputation, as it had the highest proportion of missing values.

One participant did not respond to the 2-item survey about the questions themselves. Among the n = 78 respondents, with respect to the response anchor points (on a 0-10 scale), the veterans found the questions more important and relevant than pleasant to answer. Mean relevance rating was 7.8 ± SD 2.7 (median = 8), whereas mean pleasantness rating was only 5.7 ± SD 3.2 (median = 5), (repeated-measures ANOVA F(1,77) = 26.114, P < .001), with several ratings of “10” on both questions (see Supplemental Online Information for histograms). Notably, nearly half (n = 34) of participants rated question relevance as 10, with only n = 7 participants rating below the midpoint of 5 (“OK”), with n = 4 participants rating “0.” More participants (n = 21) rated pleasantness as lower than the midpoint 5 = “OK,” with n = 51 of 78 respondents rating either 10, 5, or 0. Ratings of question relevance and pleasantness of answering them mildly but significantly correlated positively within-subject (Spearman rank-order r = .283, P = .012).

ML Model Performance and Individual Variable Importance to Predicting Intake No-Show

On average, our model ensemble yielded a training set accuracy of approximately 90% (SD = 5.6%) and a test set prediction accuracy of approximately 75% (SD = 8.5%). The top 10 features (items) predictive of no-show, in terms of relative (but not directionally informative) contribution to overall prediction model accuracy, are ranked in Figure 1. The importance of each feature is quantified by its average “Gain,” which reflects the fractional contribution of that feature to the model, determined by the cumulative gain from its splits across trees and averaged across all model iterations. A higher percentage indicates a more significant predictive feature. This indicated that the most impactful feature/variables predictive of intake no-show included age and delay to the appointment. Attitudinal or neurobehavioral predictive items of the question set included (self-reported) propensity to miss appointments or other treatment non-adherence, craving frequency, perceptions of the success of management his/her other mental health issues, extant engagement with the recovery community, and trust in previous providers.

Figure 1.

Top variables predictive of treatment intake no-show. Shown are bar graphs showing in descending order the top variables predictive of no-show for the SUD intake appointment, as determined by the XGBoost machine-learning model. The X axis represents “Gain,” which indicates the relative contribution of each feature to the model. A higher Gain value signifies greater importance of the predictor in the model’s performance.

SHAP Values Determining Item Directional and Outlier Effects on Prediction

The complementary Shapley plot of Figure 2 demonstrates the effect of each variable (and specific individual participant scores of that variable) to predict intake attendance in the context of all other concurrently entered variables. This plot demonstrates that the omnibus prediction of no-show was especially driven by the oldest participants (age), participants with the longest delays to intake. With regard to veteran beliefs as predictors, no-show was higher in participants reporting more self-perceived extant involvement with the recovery community and higher endorsement of having an addiction, but with lower frequency and intensity of craving. Interestingly, no-show was predicted by having high self-reported trust in and respect for previous SUD treatment providers. Conversely, intake attendance was predicted by endorsement of having well-managed mental health conditions.

Figure 2.

SHAPley plot illustrating directional effect of variables on the predictive model. Shown is a SHapley Additive exPlanations (SHAP) beeswarm plot explaining the expected behavior of the fitted ML model with respect to perturbation or omission of cases with certain values of original variables. Each dot represents the SHAP value of a case/participant, where a dot’s deviation from zero indicates that case’s SHAP value’s absolute contribution to the prediction model. The dot colors represent the feature values, where warmer colors (yellow/orange) denote low feature values and cooler colors (purple/blue) denote high feature values. The plot also explains the relationship between the original/actual predictor variable values and intake attendance outcomes, with dots to the right of zero vertical line indicating a negative effect on predicted intake attendance and dots to the left of the zero vertical line indicating a positive effect on predicted intake attendance. Participant age emerged as the strongest predictor of intake attendance where low values (younger age) were concentrated to the left of 0 (ie, promoting intake attendance), and high values for age were concentrated to the right of 0, indicating reduced intake attendance, thereby showing that older veterans had a lower overall intake attendance. Some variables’ contribution to the model were driven by outlying values, such as exceptionally long delays until the intake appointment and their negative linkage to attendance.

Discussion

In veterans slated for SUD treatment intake, we posed several potential novel intake interview questions regarding their attitudes about their addiction, self-efficacy, treatment, and providers, and analyzed their responses with ML to determine which items may be predictive of intake no-show and so could have value if added to the interview as standard of care. The core ML-derived model in tandem with the follow-on SHAP value analysis (to clarify the directional impact of predictive factors) revealed that older age and especially long delays until intake were by far the most no-show predictive variables. Among attitudinal and neurobehavioral patient factors of primary interest herein, self-report of lower incidence of craving, lower severity of craving episodes, belief in having an addiction, perception of already being engaged with the recovery community, and perception that comorbid mental health conditions were not being well-managed were each predictive of no-show in the main model. Finally, we note that mode of administration of the question set (eg, by phone or mailed paper) did not emerge as a significant predictive factor. For comparison, we applied a penalized logistic regression with Firth’s correction,⁴³ which identified only 3 significant predictors of non-attendance, namely, treatment delay, impulsivity, and recovery self-efficacy (see Supplement_results online document for details).

Longer delays until an outpatient appointment have been found in many administrative dataset studies to increase risk of no-show.^16,19 We have observed that some veterans will request a more distant appointment date. In these cases, such requests may be a potential marker of lower treatment motivation (procrastination). Unfortunately, this information was not recorded. Our finding that self-reported propensity to not attend scheduled medical appointments and to not adhere to provider instructions were also highly predictive of no-show also replicates several health system data analyses showing that perhaps the biggest predictor of outpatient no-show is previous incidence of no-show.^16,19 With regard to feasibility, that this relationship also appears to hold when relying on patient-reported compliance ratings also suggests that prospective SUD treatment patients have appreciable insight into their low propensity to attend appointments or to follow physician advice or instructions and are also willing to divulge this.

Contrary to our expectations, responses to questions that pertained to self-efficacy were not predictive of intake no-show. Notably, self-efficacy has generally been shown to have a positive relationship with treatment outcomes.³¹ Many items encompassed different facets of self-efficacy, such as: “Please indicate your belief in your ability to achieve recovery below” (scale 0-10), “I am confident that I will achieve the goals that I set for myself,” and “I am confident that I will achieve the goals that I set for myself.” Possibilities for the absence of motivational factors as predictive of no show include basic lack of insight, or even investigator-pleasing responses indicative of motivation for change and self-efficacy that are not genuine.

This specific constellation of predictive items prompts speculation of whether they collectively suggest some kind of archetypical veteran who does not attend intake. For example, there could be 2 sub-populations of veterans who are motivated to initially contact VA to reduce their problematic substance use: (1) younger veterans with more severe overall psychiatric symptomatology⁴ (including craving) with minimal experience with cycles of SUD treatment (and relapse after treatment) who would likely attend intake, and (2) more experienced (and older) veterans who have already undergone 1 or more SUD treatment regimens and remain motivated to change, but see little utility in engaging VHA providers if they have already learned and internalized key concepts from VHA cognitive behavioral therapy (because they also trust their previous providers), and so may resolve to initiate behavior change on their own and opt to not follow through with intake and another round of treatment. Anecdotally, clinical experiences with several older SATP patients suggest that this subpopulation of veterans exists. However, as noted, this account is entirely speculative and would be more definitively supported in a more expansive study of patient-reported items that probed this concept more directly. Alternatively, a more parsimonious explanation of the role of older age in no-show is that older veterans may simply eschew change in behavior from lifelong patterns.

We note that craving strength values as contributing to the SHAP value analysis, but craving strength was not identified in the primary predictive ML model. Conversely, endorsement of satisfaction with self was identified in the primary model, but not in the SHAP value analysis. This is due to the differences in how SHAP values and the XGBoost model assess feature importance. XGBoost’s native feature importance metric, such as gain, measure how much a feature contributes to reducing the inherent loss function. These metrics are often biased toward features used in higher-depth splits or those with higher cardinality. In contrast, SHAP values provide a model-agnostic approach, assigning each feature an importance score based on its marginal contribution to the predictions across all possible feature combinations.

With regard to patient reactions to the potential interview questions, responses to the 2-item survey about the questions on addiction and attitudes indicated that veterans typically found the questions to be relevant and important to their addiction and treatment (median rating of 8 on 0-10 scale). This comports with the origin of the questions as having been furnished by experienced SUD care providers. Ratings of the pleasantness of answering these questions were markedly lower, with n = 11 participants rating this as “0 – Very Unpleasant.” This could be explained by how some questions could prompt uncomfortable self-reflection, especially negatively-valenced questions such as “I feel I do not have much to be proud of.” However, in light of the perceptions of the high relevance of the questions to addiction treatment, we caution that ratings of low pleasantness may not translate into actual unwillingness to address these questions in practice, especially if veterans perceive utility in answering them. Finally, we note that in post hoc logistic regression analyses, there was no direct relationship (P > .10) between responses on either of these 2 survey questions and intake no-show.

This study should be interpreted as preliminary, due to several caveats. First, the parent project (and our present sub-analysis) sample size was constrained by funding and by COVID-19 lockdowns and not determined by an a priori power analysis, nor were there any analogous data in the treatment literature to envision a likely effect size. However retrospective power analysis indicates that the power to detect a medium effect size (f² = 0.15; Cohen⁴⁴) under an ordinary least squares framework using n = 79 samples is close to 95%. Since we have used XGBoost in our analysis, the power is not exactly quantifiable due to the lack of an underlying parametric distribution. However, the regression-based power assessment provides a useful benchmark, indicating that the study was sufficiently powered for feature selection involving moderate associations.⁴⁴ Our small sample size nevertheless lacked power to detect more subtle predictive relationships of other attitudes and also did not enable inclusion of additional demographic variables (to avoid overfitting) nor division into separate model formation and test datasets as a preferred standard.

Second, this pilot study was to probe feasibility and general proof of concept or probing veterans about certain attitudes as part of the treatment intake interview and was not designed or intended to develop a psychometric scale. Moreover, because we did not deploy a psychometrically validated questionnaire about SUD and SUD treatment attitudes, this makes our study challenging to replicate, and our results may not generalize other VAMC or other patient samples. Finally, a key limitation of any study reliant on patient self-report is the potential for investigator/provider-pleasing or other non-veridical patient responses, or the possibility that a respondent might lack insight into his or her true feelings or motivations.

This pilot study nevertheless suggests that at least some portion of SUD treatment seekers can be candid about less-than-adherent health behaviors in that we were able to detect intuitive directional relationships between intake no-show and low self-ratings of propensity to show up for scheduled health care appointments. Finally, we caution that the large majority of approached Veterans were non-responsive or otherwise unwilling to participate, which is consistent with the lower rate of patient engagement in research that has no prospects of direct medical benefit to that patient. As such, our findings may only be specific to the most motivated patients. Also, due to the large representation of Black veterans at the VAMC of study, our findings may not generalize to other VAMC or to civilian SUD populations.

Conclusion

This preliminary study suggests that asking about certain sensitive patient attitudes about addiction and its treatment can yield authentic answers and could be useful during intake scheduling to identify patients at risk of no-show to intake, where the topics asked about are perceived as meaningful by many veterans seeking SUD care. Future research with larger samples and that probe both patient attitudes about utility of treatment (especially repeat treatment) as well as about additional structural barriers²⁵ could shed additional light on the relative impact of modifiable risk factors for SUD intake non-attendance. The most predictive patient attitudes identified in a larger replication study could be packaged in a brief, clinic-deployable text- or phone-based assessment at intake scheduling, to identify veterans who may disproportionately benefit from more engagement following intake scheduling. Should these findings be replicated in a larger study and sample, it would also explain in retrospect why typical reminders to prevent no-show (such as phone calls or texts) are not as effective for SUD care.²² It may be that cursory pre-intake interim contacts that do not modify patient attitudes and motivation are generally not effective, where these contacts could feature more motivational engagement with the Veteran³² (p. 3).

Supplemental Material

sj-docx-1-sat-10.1177_29768357251397727 – Supplemental material for Attitudinal Predictors of No-Show to Substance Use Disorder Treatment Intake in Veterans

Supplemental material, sj-docx-1-sat-10.1177_29768357251397727 for Attitudinal Predictors of No-Show to Substance Use Disorder Treatment Intake in Veterans by James M. Bjork, Indranil Sahoo, Angela J. Zaur, Kathryn Polak and Jarrod Reisweber in Substance Abuse: Research and Treatment

Supplemental Material

sj-docx-2-sat-10.1177_29768357251397727 – Supplemental material for Attitudinal Predictors of No-Show to Substance Use Disorder Treatment Intake in Veterans

Supplemental material, sj-docx-2-sat-10.1177_29768357251397727 for Attitudinal Predictors of No-Show to Substance Use Disorder Treatment Intake in Veterans by James M. Bjork, Indranil Sahoo, Angela J. Zaur, Kathryn Polak and Jarrod Reisweber in Substance Abuse: Research and Treatment

Footnotes

Acknowledgements

This project was funded by an un-numbered award from the Jesse Family Foundation. The authors would like to thank Josh Leach, MSW, for participant referrals.

Correction (January 2026):

Article updated; for further details please see .

Author Note

The views expressed herein are solely those of the Authors and do not represent the views or policy of the US Department of Veterans Affairs.

ORCID iD

James M. Bjork

Ethical Considerations

All participant recruitment methods and study procedures were reviewed and approved by the local Veterans Affairs Medical Center (VAMC) Institutional Review Board.

Consent to Participate

All participants provided written or verbal (during COVID-19 distancing) informed consent.

Author Contributions

JMB and JR conceptualized this study. JR and KP developed the question set. JMB acquired funding. AJZ obtained and curated the data. IS performed the statistical analyses, and JMB initially drafted the manuscript. All authors contributed to the writing, review, and final approval of the manuscript prior to submission.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by an un-numbered award from the Jesse Family Foundation.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

Completely de-identified data may be furnished to qualified investigators, as considered by the first author.

Supplemental Material

Supplemental material for this article is available online.

References

Bouchery

Harwood

Sacks

Simon

Brewer

RD.

Economic costs of excessive alcohol consumption in the U.S., 2006. Am J Prev Med. 2011;41(5):516-524.

Johnson

Eick-Cost

Jeffries

Russell

Otto

JL.

Risk of alcohol use disorder or other drug use disorder among U.S. service members following traumatic brain injury, 2008-2011. Mil Med. 2015;180(2):208-215.

Angkaw

Haller

Pittman

, et al. Alcohol-related consequences mediating PTSD symptoms and mental health-related quality of life in OEF/OIF combat veterans. Mil Med. 2015;180(6):670-675.

Tanielian

Jaycox

LH.

Invisible Wounds of War: Summary and Recommendations for Addressing Psychological and Cognitive Injuries. RAND Corporation. 2008.

Hoggatt

Lehavot

Krenek

Schweizer

Simpson

Prevalence of substance misuse among US veterans in the general population. Am J Addict. 2017;26(4):357-365.

Crum-Cianflone

Powell

LeardMann

Russell

Boyko

EJ.

Mental health and comorbidities in U.S. military members. Mil Med. 2016;181(6):537-545.

Fuehrlein

Mota

Arias

, et al. The burden of alcohol use disorders in U.S. military veterans: results from the national health and resilience in veterans study. Addiction. 2016;111:1786-1794.

Dworkin

Bergman

Walton

Walker

Kaysen

DL.

Co-occurring post-traumatic stress disorder and alcohol use disorder in U.S. military and veteran populations. Alcohol Res. 2018;39(2):161-169.

Bohnert

Ilgen

Louzon

McCarthy

Katz

IR.

Substance use disorders and the risk of suicide mortality among men and women in the US Veterans Health Administration. Addiction. 2017;112(7):1193-1201.

10.

Elbogen

Johnson

Wagner

Sullivan

Taft

Beckham

JC.

Violent behaviour and post-traumatic stress disorder in US Iraq and Afghanistan veterans. Br J Psychiatry. 2014;204(5):368-375.

11.

Teeters

Lancaster

Brown

Back

SE.

Substance use disorders in military veterans: prevalence and treatment challenges. Subst Abuse Rehabil. 2017;8:69-77.

12.

DeMarce

Gnys

Raffa

Kumpula

Karlin

BE.

Dissemination of cognitive behavioral therapy for substance use disorders in the Department of Veterans Affairs Health Care System: description and evaluation of Veteran outcomes. Subst Abuse. 2021;42(2):168-174.

13.

Preston

Kowalczyk

Phillips

, et al. Exacerbated craving in the presence of stress and drug cues in drug-dependent patients. Neuropsychopharmacology. 2018;43(4):859-867.

14.

Harris

AHS

Bowe

Predictors of initiation and engagement in VA substance use disorder (SUD) treatment. Psychol Serv. 2008;5(3):228-238.

15.

Weisner

Mertens

Tam

Moore

Factors affecting the initiation of substance abuse treatment in managed care. Addiction. 2001;96(5):705-716.

16.

Carreras-García

Delgado-Gómez

Llorente-Fernández

Arribas-Gil

Patient no-show prediction: a systematic literature review. Entropy. 2020;22(6):675.

17.

Srinivas

Salah

Consultation length and no-show prediction for improving appointment scheduling efficiency at a cardiology clinic: a data analytics approach. Int J Med Inform. 2021;145:104290.

18.

Deina

Fogliatto

da Silveira

GJC

Anzanello

MJ.

Decision analysis framework for predicting no-shows to appointments using machine learning algorithms. BMC Health Serv Res. 2024;24(1):37.

19.

Dantas

Fleck

Cyrino Oliveira

Hamacher

No-shows in appointment scheduling – a systematic literature review. Health Policy. 2018;122(4):412-421.

20.

Shanklin

Samorani

Harris

Santoro

MA.

Ethical redress of racial inequities in AI: lessons from decoupling machine learning from optimization in medical appointment scheduling. Philos Technol. 2022;35(4):96.

21.

Hasvold

Wootton

Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review. J Telemed Telecare. 2011;17(7):358-364.

22.

Gallefoss

Gabrielsen

Haugland

Clausen

Vederhus

JK.

Effects of a brief pre-admission telephone reminder on no-show and dropout rates in substance use disorder treatment: a quasi-experimental study. Subst Abuse Treat Prev Policy. 2022;17(1):61.

23.

Blaauw

Riemersma

Hartsuiker

Hoiting

Venema

The influence of a short message service reminder on non-attendance in addiction care. Subst Use Misuse. 2019;54(14):2420-2424.

24.

Tsai

Rosenheck

RA.

Risk factors for homelessness among US veterans. Epidemiol Rev. 2015;37:177-195.

25.

Tsai

Szymkowiak

Beydoun

Dasilva

Transportation as a determinant of health: evaluating the Veterans Affairs rideshare program for homeless patients. Health Aff. 2024;43(8):1090-1099.

26.

Madi

Kilani

Rotramel

, et al. Predictors of no-shows after an endoscopic mucosal resection in veterans: a retrospective analysis. Cureus. 2024;16(12):e76462.

27.

Raftery

Kelly

Deane

, et al. Insight in substance use disorder: a systematic review of the literature. Addict Behav. 2020;111:106549.

28.

Sobell

Ellingstad

Sobell

MB.

Natural recovery from alcohol and drug problems: methodological review of the research with suggestions for future directions. Addiction. 2000;95(5):749-764.

29.

Connolly

Stolzmann

Heyworth

Weaver

Bauer

Miller

CJ.

Rapid increase in telemental health within the Department of Veterans Affairs during the COVID-19 Pandemic. Telemed E Health. 2021;27:454-458.

30.

Meier

Barrowclough

Donmall

MC.

The role of the therapeutic alliance in the treatment of substance misuse: a critical review of the literature. Addiction. 2005;100(3):304-316.

31.

Kadden

Litt

MD.

The role of self-efficacy in the treatment of substance use disorders. Addict Behav. 2011;36(12):1120-1126.

32.

Substance Abuse and Mental Health Services Administration. Chapter 3—motivational interviewing as a counseling style. In: Enhancing Motivation for Change in Substance Use Disorder Treatment: Updated 2019. Substance Abuse and Mental Health Services Administration; 2019:35-64.

33.

Chen

Guestrin

XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2016: 785-794. doi:10.1145/2939672.2939785

34.

Nisbet

Handbook of Statistical Analysis and Data Mining Applications. Elsevier Science & Technology; 2009.

35.

Bailey

Andridge

Shoben

AB.

Multiple imputation by predictive mean matching in cluster-randomized trials. BMC Med Res Methodol. 2020;20(1):72.

36.

Zhang

Multiple imputation with multivariate imputation by chained equation (MICE) package. Ann Transl Med. 2016;4(2):30.

37.

Price

Yamazaki

Fujihara

Sone

. XGBoost: interpretable machine learning approach in medicine. In: 2022 5th World Symposium on Communication Engineering (WSCE). IEEE; 2022: 109-113. doi:10.1109/wsce56210.2022.9916029

38.

Chawla

Bowyer

Hall

Kegelmeyer

WP.

SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. 2002;16:321-357.

39.

Garcia

EA.

Learning from imbalanced data. IEEE Trans Knowl Data Eng. 2009;21(9):1263-1284.

40.

Burman

A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods. Biometrika. 1989;76(3):503-514.

41.

Lundberg

Erion

Chen

, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2(1):56-67.

42.

Little

RJA

. A test of missing completely at random for multivariate data with missing values. J Am Stat Assoc. 1988;83(404):1198-1202.

43.

Heinze

Schemper

. Solution to the problem of separation in logistic regression. Stat Med. 2002;21(16):2409-2419.

44.

Cohen

, ed. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates; 1988.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB

0.03 MB