Rates and Predictors of Follow-Up Care and Treatment Initiation Among Women Veterans After a Positive Alcohol Screen

Abstract

Background:

Gender-related disparities in access to alcohol-related care exist in the Department of Veterans Affairs’ (VA) health care system. Understanding differences in the use of alcohol-related care in the context of potentially important covariates (e.g., race, ethnicity, younger age, and military sexual trauma [MST]) is critical to support the health and well-being of women Veterans. This study examined differences in the use of alcohol preventive care among women Veterans.

Methods:

From VA administrative data (2010-2016), we drew a sample of women Veterans (n = 280) who screened positive for at-risk drinking in inpatient/outpatient settings. We conducted a chart review to abstract variables from the medical record. Then, we employed logistic regression to predict receipt of any follow-up (brief intervention and/or referral to treatment) and initiation of treatment, as with covariates of race, ethnicity, age, and clinical characteristics such as MST.

Results:

Seventy-four percent (n = 207) of the sample received any follow-up. Of those referred to treatment (n = 115), 73% (n = 84) initiated treatment. Hispanic women were 71% less likely to receive follow-up care than non-Hispanic women. Women Veterans 21 to 24 years were less likely to initiate treatment than those 25 to 29 years. While women Veterans who endorsed MST were more likely to receive follow-up care than others, they were no more likely to initiate treatment.

Conclusions:

Given the rapid growth of the women Veteran population, their access to alcohol-related care is vital. Without culturally competent, clinically and developmentally appropriate alcohol prevention messaging for Hispanic and younger women Veterans with trauma, differences in access to care and disparities in outcomes will persist.

Keywords

at-risk drinking women Veterans ethnicity and emerging adulthood

Highlights

● Women veterans who screened positive for at-risk drinking had high rates of follow-up and treatment initiation.

● Hispanic women Veterans were less likely to access follow-up care.

● Women Veterans of emerging adult age were less likely to initiate treatment.

● Those with military sexual trauma were more likely to receive follow-up and no more likely to initiate treatment.

Introduction

At-risk drinking, defined as drinking above recommended limits, is problematic to health and well-being. While over one-quarter of women Veterans have high rates of past-year at-risk drinking (31%) including binge drinking (26%),¹ there are gender-related disparities^2,i in access to alcohol-related care for women relative to men in the Department of Veterans Affairs (VA) and in the general population.^3,4 In one study, Hoggatt et al.⁵ found that lowering the screening question for binge drinking on the VA clinical reminder for women Veterans from ≥6 drinks to ≥4 drinks on one occasion increased the identification of binge-drinking in women Veterans by 15%. While disparities in access to alcohol-related care have been studied in VA,⁶ equitable implementation of alcohol prevention among women Veterans requires developing a better understanding of differences in access to alcohol-related care between groups of women Veterans.⁷ Indeed, the need for alcohol-related care in this growing population of women Veterans is further complicated by additional demographic issues such as race, ethnicity, age, and experience of military sexual trauma (MST), which in turn uniquely impact and can exacerbate disparities in medical and psychiatric outcomes.^8
-13 Knowing rates and disparities in the receipt of alcohol-related care of marginalized groups of women Veterans is critical to women Veteran health.

VA Alcohol Care, Female Gender, Race, and Ethnicity: Alcohol-preventive screening, including screening, brief intervention, and referral to treatment (SBIRT), is efficacious, cost-effective, and is routinely offered within VA.^14,15 Studies examining racial as well as ethnic differences in the utilization of preventive care by women Veterans who screen positive for at-risk drinking have frequently focused on brief interventions (BI). For example, in a sample of Veterans with unhealthy alcohol use, black patients received BI at lower rates than white patients and the lowest rates of BI were among black women.³ In another study examining BI, black and Hispanic women who screened positive for unhealthy alcohol use were less likely to receive BI for alcohol-related care than white women.¹⁶

VA Alcohol Care, Female Gender, and Emerging Adult Age: In contrast, fewer studies have examined access to alcohol-related care for women Veterans with at-risk drinking by different age groups. Because women Veterans of younger age (18-44 years) and those returning from wars in Afghanistan and Iraq have some of the highest at-risk drinking rates 43% and 51%, respectively,¹ it is important to examine younger age women Veterans’ utilization of care. This is especially relevant to women Veterans of emerging adult age, defined as age 18 to mid-20s,¹⁷ because the initiation of substance use typically occurs before the age of 25 years. While the literature identifies critical problems for women Veterans who engage in at-risk drinking, including domestic violence, blackouts, physical injuries, and tolerance,¹⁸ there is a paucity of information on health-related risks for women Veterans of emerging adult age. Substance use (e.g., including alcohol) is associated with negative developmental and mental health (MH) outcomes.^19
-21 Impacts of alcohol use on the developing brain include risk of early activation of the dopamine system²² that increases the chance of later substance use,²³ and addiction. During the emerging adult developmental period, alcohol-related risks are exacerbated due to the social vulnerabilities of emerging adults characterized by shifting social networks, fewer responsibilities to family, and residential mobility.^24,25 Prevention of and treatment for at-risk drinking in emerging adults represent a critical window of opportunity that could prevent escalation of the alcohol-related disease process across the life course, especially for the subset of emerging adults who develop alcohol use disorders (AUD).^26,27

VA Alcohol Care, Female Gender, and Trauma: Beyond race, ethnicity, and age, clinical factors such as a history of MST are important to consider as they pertain to women Veteran health care access.¹³ In this study, we consider how MST may exacerbate access to care and specifically in the clinical course of care for at-risk drinking. MST is defined as experiences of sexual assault or repeated threatening and sexual harassment and has been a key area of clinical focus in VA, including screening and follow-up for this group in the context of unhealthy alcohol use. Women Veterans who use alcohol and endorse MST are at elevated risk of morbidity and mortality,^28,29 yet VA’s care for this group has been variable. Explanations for women Veterans’ inconsistent alcohol-related care are multifactorial and may include that women with AUD in general have lower rates of treatment seeking,²⁶ and other barriers to care exist. Within VA, these include an incentivized clinical reminder that is gender-neutral, delivered to men and women Veterans alike.³⁰

Study Aims: In light of the paucity of research on women Veterans’ utilization of care following a positive screen for at-risk drinking, we sought to measure the rate of follow-up among women Veterans, defined to include BI, and/or referral to treatment (RT).^31
-33 We chose to investigate both of these outcomes as women Veterans who screen positive for at-risk drinking may be provided BI, and/or receive referrals to specialty treatment through VA’s electronic consult system as part of the SBIRT model of care. We also investigated the rate of treatment initiation, defined as attendance at the first appointment after follow-up, for those women who received a referral to additional treatment services. Our research questions on utilization of alcohol-related care were guided by the Andersen model, often used to explain a patient’s use of health care.^34,35 According to this model, the predictors of using health services (including inpatient care, physician visits, dental care) can be classified into 3 types of variables: predisposing (e.g., race, age, and health beliefs), enabling (e.g., insurance, income), and need factors (e.g., MH diagnoses and traumatic experiences).

Thus, two research questions drove our study design. First, we investigated whether age, race, ethnicity, and MST status predicted any follow-up care among women Veterans with at-risk drinking. Second, we investigated whether those same factors specifically predicted treatment initiation. Based on prior literature indicating the shifting networks and mobility of emerging adults compared with those of young adults (up to 29 years of age) and adults,^24,25,36 we hypothesized that women Veterans of emerging adult age were more likely to receive a recommendation for follow-up but less likely to follow through on that recommendation. Further, we hypothesized that MST increases a woman Veteran’s likelihood of receiving follow-up from a provider but may limit the initiation of treatment among those who are referred.

Methods

We conducted a retrospective observational database study from administrative data including the electronic medical record (EMR) of women Veterans ranging in age from 21 to 65 years (N = 280) who screened positive for at-risk drinking on the Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) ≥3^37,38 in inpatient and outpatient settings in 1 VA medical center in the Northeastern United States between 2010 and 2016. The AUDIT-C screen results include responses about the patient’s drinking pattern, and a resulting numeric score scaled from 0 to 12 (see Table 1). The Institutional Review Boards at VA Boston and Brandeis University approved this study.

Table 1.

VA AUDIT-C Screening Tool.

AUDIT-C
Q1: How often did you have a drink containing alcohol in the past year?
Answer	Points
Never	0
Monthly or less	1
2-4 times a month	2
2-3 times a week	3
4 or more times a week	4
Q2: How many drinks did you have on a typical day when you were drinking in the past year?
Answer	Points
None, I do not drink	0
1 or 2	0
3 or 4	1
5 or 6	2
7-9	3
10 or more	4
Q3: How often did you have 6 or more drinks on 1 occasion in the past year?
Answer	Points
Never	0
Less than monthly	1
Monthly	2
Weekly	3
Daily or almost daily	4

The patient’s AUDIT score is obtained by summing their scores across the 3 questions. Each item is scaled 0 to 4, but the overall score is 0 to 12. From “The AUDIT” by U.S. Department of Veterans Affairs, 1990. Copyright 1990 by World Health Organization. www.hepatitis.va.gov/provider/tools/AUDIT-C.asp#S1X

Abbreviation: AUDIT, Alcohol Use Disorders Identification Test.

Sample Selection

Using administrative data from VA’s Pyramid Analytics (PA), the study team identified from the population those Veterans who served during wars in Iraq and Afghanistan,^39,ii through a flag in the PA data source that allows for data segmentation. We further specified these data to a group of women Veterans who received an inpatient or an outpatient service at one VA Healthcare System during the study period 10/1/2010-10/312016.iii The study team matched these data by patient identifier to data in the regional research data warehouse for women Veterans who were screened for at-risk drinking. This sample consisted of women Veterans who were screened through standard screening procedures using the AUDIT-C clinical reminder. If multiple scores existed for the same woman Veteran, the study team included only the most recent positive screen (n = 284). We chose a cut point of 3 or more points for the AUDIT-C, based on an extensive literature review and on the VA substance use disorder (SUD) clinical guidelines, which recommend that providers consider a screen positive for unhealthy alcohol use in women Veterans if the AUDIT-C is ≥3 points.^14,15,37,38

We transferred these administrative data of women Veterans (n = 284), their positive AUDIT-C score (ranging from 3 to 12), as well as their social security number, sex, and date of birth to a relational database. We then confirmed the AUDIT-C score to minimize false positives¹⁸ via best practices for chart review in the EMR.^40,41 Once the sample of women Veterans data set was in the relational database, and the AUDIT-C score was confirmed, all other variables for the study were pulled from the VA’s EMR from administrative and clinical note sections; study staff manually recorded these in the data abstraction tool (Supplemental Material 2). This sample identification, chart review, and variable construction have also been more fully described in prior publications.^33,39

For the present analysis, in which we examined a modified binge-drinking variable (terms are defined below in the Need Factors section), we removed 4 observations in which responses to the questions to the AUDIT-C question 3 regarding drinking 6 or more drinks on an occasion in the past year were missing resulting in a final sample of 280 (Figure 1).

Figure 1.

Consort diagram.

As part of the chart review, the study team abstracted predisposing (e.g., demographic and social factors and health beliefs), enabling, and need variables (Table 2).^33,42 Data training, weekly data quality meetings, and a quality check of 10% of the observations were conducted.

Table 2.

Descriptive Characteristics of Women Veterans Who Screened Positive for At-Risk Drinking and Characteristics of Those Who Screened Positive and Were Then Referred to Treatment.

Demographic and clinical characteristics	Screened positive for at-risk drinking (n = 280)		Referred to treatment (n = 115)
Demographic and clinical characteristics	Frequency	Percent	Frequency	Percent
Age group
18-24 (“emerging adult”)	27	9.6	16	13.9
25-29	98	35.0	41	35.7
30-38	100	35.7	35	30.4
39->65	55	19.6	23	20.0
Race
White	200	71.4	78	67.8
Black	42	15.0	18	15.7
Asian	12	4.3	7	6.1
Am Indian/Nat Hawaiian/Pac Is	7	2.5	3	2.6
Declined to answer/unknown	19	6.8	9	7.8
Ethnicity
Non-Hispanic	240	85.7	97	84.4
Hispanic	30	10.7	14	12.2
Declined to answer/unknown	10	3.6	4	3.5
Marital status
Married/in partnership	71	25.4	29	25.2
Never married/single	131	46.8	50	43.5
Divorced/separated/unknown	78	27.9	26	31.3
Military branch
Army	197	70.4	84	73
Air force	24	8.6	10	8.7
Marine corps	28	10.0	11	9.6
Navy	31	11.1	10	8.7
At-risk homeless: Yes	34	12.1	17	14.8
Health coverage: at least partial	162	57.9	75	65.2
Alcohol risk level
3-4, low risk	160	57.5	57	49.5
5-7, moderate risk	71	25.4	39	33.9
8-12, high risk	49	17.5	19	16.5
Binge-drinking behavior
Any	175	62.5	80	69.6
3-4, low risk	55	19.6	22	19.1
Psychiatric problem/diagnoses
Alcohol use disorder	83	29.6	41	35.7
Tobacco use	116	41.4	53	46.1
Substance use disorder	155	55.4	70	60.1
Depressive disorder	115	41.1	54	47.0
PTSD	125	44.6	60	52.2
MST	117	41.8	56	48.7
Number of lifetime medical problems (numeric with SD)	2.7	SD2	2.7	SD2
Houshold median income (numeric in dollars with SD)	66 980	22 249	68 301	22 606
Provider gender
Female	241	86.1	102	88.7
Male	39	13.9	13	11.3

Abbreviations: PTSD, post-traumatic stress disorder; MST, military sexual trauma; SD, standard deviation.

Level of risk includes women Veterans who engaged in binge drink behavior; ^bTotal frequency/percent of women Veterans who engaged in binge drink behavior across alcohol risk levels.

Measures: Dependent Variables

Follow-Up: Follow-up, one of two key dependent variables in this study, is defined as occurring if the patient received a provider-delivered BI and/or an RT within 30 days of the index AUDIT-C.^31,32 The BI is operationalized as an EMR indication that a provider-delivered brief counseling or advice to drink within normal limits or abstain from alcohol use. The study team searched for brief counseling using the key terms “Couns*” and “Alc*” in the EMR within the specified date range. Brief advice is noted as a check box in structured data in the EMR. These types of BI were collected on each patient and then collapsed into 1 binary dependent variable for the presence/absence of BI.

The RT represents receiving a consult to either a SUD or MH consult as recorded in the “Consult” tab. The VA electronic consult system allows primary care providers to refer directly to specialty services if clinically indicated as part of the SBIRT model of care.^33,39 We included referrals to both MH and substance use treatment services in tandem for several reasons. First, we acknowledge that VA-based outpatient general MH clinics are intended to treat Veterans with a variety of issues, including alcohol use-related problems and diagnoses.⁴⁷ Second, many smaller VA clinic settings may not have specialty alcohol services readily available. In those instances, it may be more clinically appropriate to deliver general MH services for women Veterans with AUD, rather than referring them to remote clinics where specialty services may be available, but at a prohibitive distance away. Third, many women Veterans who screen positive for AUD may have other co-occurring issues that can be more easily treated in a generalist MH setting rather than a specialty substance use treatment clinic.

Treatment Initiation: Treatment initiation, the second key dependent variable, is defined as attending a MH and/or substance use program visit within 60 days of receiving an RT.

Independent Variables

Predisposing Factors: Predisposing factors include demographic characteristics, social factors, and health beliefs that may impact a patient’s choice of access to care. The six predisposing variables included in the study were age, gender, race, ethnicity, military branch, and marital status.³³ Based on the literature on women Veterans’ drinking patterns across the lifespan,^24,25,36 we created a categorical variable based on age at the time of screening to distinguish emerging adults (18-24 years), young adults (25-29), and two groups of adults: (30-38; and mid to older 39 to >65). We considered the emerging adult category to include women aged 18 to 24 years. However, no women Veterans under the age of 21 years met study criteria, so in our analyses, the emerging adult category only included women Veterans who were 21 to 24 years of age. Similarly, no women Veterans in this sample over the age of 65 met study criteria, so our mid- to older adult age category included women Veterans who were 39 to 65 years. The race and ethnicity variables were created by abstracting data from two standardized fields in administrative data, which were derived from patient self-report. We classified race into five mutually exclusive groups that reflected patient self-report data on race for the unadjusted analysis (Supplemental Material 1), and we used VA’s binary ethnicity variable (Hispanic/Latino or not). For the regression models, we created a binary variable for white versus all other. While race and ethnicity were highly correlated (χ² (8) = 108.24, P ≤ .001), we included them both in the same statistical analyses as per the Andersen model.

Enabling Factors: Enabling factors included two financial characteristics whether at-risk for homelessness and median household income (represented by the patient’s zip code, a proxy for socioeconomic status) and one organizational variable (provider sex). Given that this study examines differences in access and issues of equity, including an organizational level variable in the model is important as it provides a variable that can be modified, if differences in access are found. The study team defined “at-risk for homelessness” as a Veteran having an appointment with the U.S. Department of Housing and Urban Development – Veterans Affairs Supportive Housing (HUD-VASH) team within 6 months of the index AUDIT-C. To detect whether a Veteran was at-risk for homelessness, the EMR “Notes” tab was searched six months before and after the index AUDIT-C for the following search terms: HCHV, HUD-VASH, HUD/VASH, and HUD VASH.³⁹ Since the patient’s zip code could vary over time, the value closest to the time of the index AUDIT-C was used.⁴³

Need Factors: In this study, need for treatment was captured by the AUDIT-C score, which was specified by different risk levels (3-4/mild; 5-7/moderate; 8-12/severe), as well as the presence of medical, MH, or substance use diagnoses, endorsing binge drinking and/or MST.

Definitions of Elements of Variables Used to Create Need Factors

While the AUDIT-C (Table 1) is scored on a scale of 0 to 12 (scores of 0 reflect no alcohol use), our sample, as described, includes AUDIT-C scores 3 to 12, based on the literature review of what justification of scores reflect a positive screen.^14,15,37,38

At-risk drinking over the past year, defined in this study as an AUDIT-C ≥3, has been found to have a sensitivity and specificity score of 0.66/0.94 for this group.³⁷

We employed a modified binge-drinking variable that is more closely aligned with the definition of binge drinking for women of more than 4 drinks in 1 sitting in about 2 hours.^44,45 We define binge drinking as occurring if a woman Veteran reported drinking 5 or more drinks on a typical day when she drank over the past year or consumed 6 or more drinks in one sitting at least once in the past year. The justification for modification of the binge-drink variable is that the current AUDIT-C binge-drink question (question 3) results in a definition of binge drinking that underestimates binge drinking by 1 drink for males and 2 drinks for females.^39,iv

MST is an indicator checked by the provider on a standard delivered screening tool asking about experiences of sexual assault or harassment experienced while serving in the military.

Analyses

Descriptive and bivariate statistics for independent and dependent variables were computed using the Stata statistical package.⁴⁶ We examined independent variables, including the association between younger age and alcohol risk level (mild, moderate, severe). We ran logistic regression and a Heckman bivariate probit model to identify predictors of follow-up (BI and/or RT) and treatment initiation (completed referral). An unadjusted logistic regression model to predict receipt of follow-up for those of emerging adult age was estimated in addition to a second logistic regression model to predict treatment initiation. Since the Heckman model yielded qualitatively similar results to the logistic regression, we report only the results of the logistic model.

For analyses predicting treatment initiation specifically, several variables were removed from this logistic regression model to conserve degrees of freedom, as the sample for this outcome was smaller due to its inclusion of only those who received an RT (N = 115). For these models, we removed variables that were not significant in the bivariate analysis (Table 2 and Supplemental Material 1) and included medical diagnoses, marital status, military branch, and homelessness status.

To evaluate the contribution of age, race and/or ethnicity, and MST for follow-up and treatment initiation, we started with unadjusted models (including one predictor at a time). Then, variables were added in blocks according to the Andersen model—first adding a predisposing variable block, then need factors. The last group of enabling variables (e.g., financing and organizational factors) made up the final model presented (Table 3).

Table 3.

Predictors of Follow-Up Care and Treatment Initiation, Among Those Women Veterans Who Screened Positive for At-Risk Drinking.

Demographic, clinical, and organizational-level characteristics	Receipt of follow-up (n = 280)		Treatment initiation (n = 111)
	OR	95% CI	OR	95% CI
Age group
18-24 (“emerging adult”)
25-29	0.38	0.99-1.48	4.69*	1.11-19.82
30-38	0.32	0.08-1.24	2.17	0.56-8.45
39->65	0.42	0.10-1.73	4.22	0.86-20.76
Race
White
Black	1.61	0.59-4.41	0.72	0.19-2.75
Asian	0.48	0.11-2.05	0.42	0.06-2.73
Am Indian/Nat Hawaiian/Pac Is^a	0.77	0.66-8.90	-	-
Declined to answer/unknown	1.32	0.28-6.20	0.55	0.04-6.54
Ethnicity
Non-Hispanic
Hispanic/Latino	0.30*	0.10-0.88	3.51	0.46-26.69
Declined to answer/unknown	0.67	0.11-5.14	1.27	0.05-32.91
Marital status^b
Married
Never married/single	0.81	0.36-1.82
Divorced/separated/unknown	1.62	0.63-4.18
Military branch^b
Army
Air force	0.59	0.19-1.80
Marine corps	0.41	0.15-1.15
Navy	0.90	0.31-2.62
At-risk homeless^b
Yes	0.89	0.27-2.95
SUD and MH problems/diagnoses
Alcohol use disorder	3.94**	1.28-12.03	1.46	0.43-4.97
Binge drink	6.25***	3.10-12.96	5.87**	1.87-18.44
Tobacco use	0.64	0.33-1.26
MST	2.11*	1.02-4.30	0.79	0.28-6.52
At least 1 medical problem^b	1.88	0.65-5.47
Provider sex female	4.49***	1.80-11.15	1.36	0.39-6.76

Abbreviations: MH, mental health; MST, military sexual trauma; SUD, substance use disorder.

Perfect predictor and dropped 3 participants; ^bVariable dropped in second logistic regression to maintain data integrity of smaller sample.

P ≤ .05. **P ≤ .01. ***P ≤ .001.

Results

Sample Characteristics

This study sample consisted of women Veterans (N = 280), who screened positive for at-risk drinking, of whom a subset was referred to MH or substance use treatment (N = 115; Table 2).

Fifteen percent of the sample were African American, 2.5% were American Indian or Native Hawaiian, 4.3% were Asian, and 71.4% were white. Regarding ethnicity, 11.0% were Hispanic. Race and ethnicity were not associated with age categories. More than 80.0% of the sample were under 40 years of age; 10.0% were emerging adults aged 21 to 24 years, and one-third were young adults (25-29 years). Descriptively, in the full sample, 57.5% screened positive for low-risk alcohol use (AUDIT-C, 3-4), 25.4% screened positive for moderate risk alcohol use (AUDIT-C, 5-7), and 17.5% screened positive for high-risk alcohol use (AUDIT-C, 8-12); yet the level of alcohol risk did not differ significantly across age groups. Also of note, 62.5% of the women Veterans endorsed binge drinking. In this sample, 41.8% had reported incidents of MST.

In this sample, we report on the rate of those women Veterans who screened positive for at-risk drinking who received any follow up care: 73.9% (n = 207). Among those who received follow-up care, 41.7% (n = 115) received a referral to either a MH or SUD program, and one patient was referred for a suicide consult. Further, 73.0% (n = 84) of those who received a referral attended an initial treatment appointment (treatment initiation; Figure 2).

Figure 2.

VA alcohol prevention screening flow chart: women Veteran follow-up and treatment initiation after the receipt of a positive screen for at-risk drinking.

Unadjusted Analyses: In the unadjusted analysis (Supplemental Material 1), AUD, binge drinking, and trauma were positively associated with receipt of follow-up. Rates of follow-up and treatment initiation for emerging-age adults were not statistically significantly different from women Veterans in other age groups. Age and race/ethnicity, similarly, were not associated with either dependent variable.

Multivariable Models

Receipt of any follow-up

Across all models, women Veterans with MST had higher odds of receipt of follow-up than those without MST (Table 3). Those of emerging age were no more likely to receive follow-up than other age categories. Further, in adjusted models, follow-up was no more likely for white patients than for other race categories. Those of Hispanic ethnicity had lower odds of receiving follow-up. In our final model (Table 3), controlling for predisposing, need, and enabling factors, Hispanic women Veterans were 71.0% less likely to receive follow-up than their non-Hispanic counterparts (P = .028; Table 3). This result was examined more closely by estimating a model that also interacted binary variables for race (black, white) and ethnicity (Hispanic, non-Hispanic). This new model showed that those who were Hispanic and white were 86% less likely than non-Hispanic white others to receive follow-up.

Several additional findings are noted. First, after controlling for all other factors, patients seen by female providers were more likely to receive any type of follow-up (Table 3). Second, women Veterans with AUD and those with binge drinking were more likely than others to receive follow-up.

Treatment initiation

Among women Veterans with a referral, those of emerging adult age were less likely to attend the treatment appointment. Race, ethnicity, and MST were not associated with treatment initiation.

Discussion

We present an observational study of the clinical course of women Veterans after they screen positive for at-risk drinking in one VA locality, using both administrative data and chart abstraction. Overall rates of follow-up and treatment initiation (73.9% and 73.0%, respectively) in this study were higher than those found in prior studies.^31,32,48 While our data do not permit us to disentangle reasons for these higher rates of follow-up and treatment initiation, we surmise that differences may reflect multiple factors at the patient and organizational levels, including (1) a higher health and MH severity rate in this inpatient/outpatient cohort; (2) facility-specific culture regarding the importance of follow-up after a positive screen on the AUDIT-C; or (3) the electronic consult system that facilitated the referral process.

Follow-Up Care (BI or RT)

Our analyses suggest that, after screening positive for at-risk drinking, Hispanic women Veterans were less likely to receive any follow-up than non-Hispanic women Veterans. We note that Hispanic women Veterans in our sample were predominantly white. In prior research, factors shown to contribute to the underutilization of Hispanic women Veterans alcohol-related health care included stigma, lack of culturally competent service, and poor identification of alcohol misuse in minority groups.⁴⁹

Among women Veterans with at-risk drinking, those with MST were more likely to receive follow-up care from providers than those without MST. Similarly, women Veterans with a documented AUD diagnosis, or who self-reported binge drinking, were more likely to receive follow-up care. Race was not associated with follow-up in our sample.

Treatment Initiation Among Women Veterans Referred to Treatment

Among women Veterans who were referred to treatment after screening positive for at-risk drinking, almost three quarters, (73.0%) went on to initiate alcohol-related treatment. Of note, women Veterans who endorsed binge drinking were more likely to initiate treatment than those who did not. These treatment initiation rates are higher than 65%, the rate cited in two prior VA published studies: one of returning male Veterans³¹ and a second study with a sample of Veterans with high-risk for severe AUD presenting to VA primary care settings between 2010 and 2014.⁴⁸ High rates of treatment initiation in this current study may be attributable to several factors. First, some Veterans may have been screened while in inpatient units coupled with the VA’s integrated health care system that includes a continuum of care for MH and SUDs. Second, VA has a sophisticated EMR, which facilitates completed referral through electronic delivery of a consult. Alternatively, they may reflect the characteristics of the complex and vulnerable patient group seen at this VA facility.

Consistent with hypotheses, emerging-age adult women Veterans who received an RT were less likely to initiate treatment than older age women Veterans.⁴² This missed opportunity for those of emerging adult age to access care at this developmental juncture may be due to the distinct developmental needs of this age group.⁵⁰ Other research suggests that digital health applications may appeal to this age group, and may therefore represent a promising method for enhancing treatment engagement.^51
-53

Findings in the study suggest that women Veterans who have experienced MST are no more likely to initiate treatment than those without a documented history of MST. This finding may reflect that women Veterans in VA with trauma could experience several complex patient-level factors that impede the referral process, including trust in the provider, in the setting, and system-level care coordination among agencies.^54,55

This study has potential limitations. First, the data were from one VA locality, resulting in a small sample size that limited the number of variables in the model and the analysis of different subgroups of patients by ethnicity, race, and age, particularly since no one was younger than 21 years. Second, chart review may not have captured all referrals, in which case, the follow-up variable would underrepresent the rates of follow-up. However, because referrals included those to MH settings, the referrals related to the positive alcohol screen may be overestimated. Third, while the findings of this study may be relevant to other localities within VA, we caution against generalizing these findings to other health systems or patient populations. Fourth, while our findings indicate significant differences in access to care between groups, we do not have enough information regarding patient needs and preferences to know whether these represent disparities.² Fifth, it is possible that women Veterans may have screened positive for other MH issues at the same visit in which the AUDIT-C was administered and that this may have impacted referrals to treatment following the visit. And lastly, we acknowledge that the data from this study date back several years (i.e., the data were collected from women Veterans treated between 2010 and 2016). However, given that the proportion of women in the Veteran population continues to rise, we believe these data remain important for guiding alcohol-related clinical care for this population.

This study has a number of strengths, including a combination of VA administrative data and rigorous chart review drawing data from VA’s EMR. We were also able to include Andersen model need variables like MST, which at the time of study was not readily accessible in the national data warehouse. Our data source yields new knowledge regarding access to care. Although a local study, these results are likely similar to a study with national findings because gaps in performance measurement (e.g., BI) used in a portion of this dataset³³ were similar to the gaps found in a larger national study.⁵⁶ Moreover, we extended the use of the Andersen model to women Veterans with at-risk drinking, which guides our research questions and provides a modifiable organizational level variable (provider sex) that informs our practice and policy implications.

Implications for Practice and/or Policy

While rates of follow-up and treatment initiation were high in this study and may indicate that VA continues to improve access to alcohol prevention care, there were differences in treatment initiation for those of emerging adult age compared to young adult women Veterans. There is literature that indicates that stigma contributes to younger Veterans not accessing treatment for MH concerns at the VA.⁵⁷ While women Veterans endorsing MST were more likely to receive follow-up care, they were no more likely to initiate treatment than those who did not endorse MST. Implementing novel clinical preventive interventions through digital health may be less stigmatizing to emerging adult women Veterans, and delivering alcohol-related care that is psychologically safe for all including MST survivors is essential and may encourage attendance at appointments for alcohol-related care at VA.

This study has identified lower rates of access for Hispanic women Veterans. These findings are important because of a missed opportunity to minimize the adverse consequences of excessive alcohol use in a vulnerable group of young Hispanic women. Women are more susceptible to the negative medical consequences associated with alcohol use but have been historically less likely to be identified and treated for unhealthy drinking.⁵⁸ These vulnerabilities are exacerbated among minoritized populations^9,11 including Hispanic women who have been identified as one of the most disadvantaged ethnic groups with regard to racial and ethnic disparities in health care access and utilization.⁵⁹ In the context of the literature and the results of this study, it is essential to further investigate access to alcohol-related care for Hispanic women Veterans with at-risk drinking in VA.

Screening for at-risk drinking in primary care may help health systems identify patients who may be in need of additional supports or follow-up services. As such, there is a significant opportunity in strategies that leverage providers in these settings, who meet regularly with patients to support health maintenance and treatment engagement, if needed. Successful implementation of evidence-based practices like SBIRT relies upon multiple factors including provider training.^60,61 Integrating routine training on alcohol prevention in women Veterans for providers who regularly perform SBIRT, through mechanisms like VA’s annual web-based and in-person training system, could be supportive to this end. This training could be a simulated interactive online training or live training with role play, which could be similar to VA current clinician training approach for cardiopulmonary resuscitation. Annual training would improve providers’ skill-based practice in delivering a motivational conversation for BI that is non-stigmatizing, trauma-sensitive, unbiased, and perhaps appeals to youth.

Conclusions

In a sample of women Veterans with at-risk drinking in one VA medical center, there were differences in access to care for women Veterans with certain demographic and clinical characteristics. Specifically, age, ethnicity, and clinical variables were associated with follow-up and treatment initiation. Ensuring appropriate follow-up to meet the needs of each of these women Veteran groups (e.g., Hispanic, those with MST, and those of emerging adult age) and delivering training to VA providers may improve the implementation of alcohol prevention care for women Veterans. Furthermore, qualitative studies may help identify what specifically is contributing to lower-than-optimal follow-up and treatment initiation rates, including understanding patient and provider preferences and experiences of what is driving the differences in receipt of evidence-based alcohol preventive care for Hispanic women Veterans. Women Veterans are the fastest growing population in the VA, and to our knowledge this is the first study that examines access to women Veteran alcohol-related preventive care in VA while accounting for important covariates in the domains of race, ethnicity, age, and trauma.

Overall, innovations are needed to best deliver evidence-based approaches to reducing alcohol-related risks in subgroups of VA’s women Veterans. In the context of a growing population and VA’s strategic goal to support women Veterans across the life course, investing in this diverse group through programmatic training and research on policy implementation may prevent further differences in access to care and ensuing health disparities.

Supplemental Material

sj-docx-1-saj-10.1177_29767342251351107 – Supplemental material for Rates and Predictors of Follow-Up Care and Treatment Initiation Among Women Veterans After a Positive Alcohol Screen

Supplemental material, sj-docx-1-saj-10.1177_29767342251351107 for Rates and Predictors of Follow-Up Care and Treatment Initiation Among Women Veterans After a Positive Alcohol Screen by Marianne Pugatch, Dominic Hodgkin, Michelle Gibson, Christopher Miller and Grace Chang in Substance Use & Addiction Journal

Supplemental Material

sj-docx-2-saj-10.1177_29767342251351107 – Supplemental material for Rates and Predictors of Follow-Up Care and Treatment Initiation Among Women Veterans After a Positive Alcohol Screen

Supplemental material, sj-docx-2-saj-10.1177_29767342251351107 for Rates and Predictors of Follow-Up Care and Treatment Initiation Among Women Veterans After a Positive Alcohol Screen by Marianne Pugatch, Dominic Hodgkin, Michelle Gibson, Christopher Miller and Grace Chang in Substance Use & Addiction Journal

Footnotes

Acknowledgements

The authors thank Yvonne Anthony, PhD, Brian Harward, PhD, and Stephen A. Cohn, PhD.

Author Contributions

MP, DH, and GC conceptualized the study, research goals and aims, and the methodology. MP and DH developed the statistical design for this paper. MP had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis; she performed the statistical analysis. MP took the lead in writing the manuscript. All authors reviewed and edited versions of the manuscript and approved the submitted version.

Data Availability Statement

Due to the sensitive nature of the research questions asked and the respondent population in this study, the data remain confidential and will not be shared.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest for the research, authorship, and/or publication of this article: Dr. Pugatch is an Associate Editor for Substance Use and Addiction Journal and is remunerated for this service from AMERSA, Inc. In 2024, Dr. Pugatch served as a consultant to The Life Course Translational Research Network, University of California, Los Angeles, to provide technical expertise on a study developing the INSPIRE Interactive Video.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was primarily supported by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) of the National Institutes of Health (T32AA007567) and work supported by the Office of Academic Affiliations, United States Department of Veterans Affairs (VA), with resources and the use of facilities at the Veterans Health Administration Boston Healthcare System. The VA Boston Healthcare System of the United States Department of Veterans Affairs provided access to the dataset. Additional support was provided by the National Institute of Mental Health (NIMH) of the National Institutes of Health (T32MH018261) and the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under cooperative agreements UA6MC273 and U8DMC45901, Adolescent and Young Adult Health Research Network. The funders were not involved in the research methods or writing. The opinions or assertions herein are those of the authors and do not necessarily reflect the views of the United States Department of Veterans Affairs, the United States Government, or the National Institutes of Health.

Compliance,Ethical Standards,and Ethical Approval

The Institutional Review Boards at VA Boston and Brandeis University approved this study.

ORCID iDs

Marianne Pugatch

Dominic Hodgkin

Michelle Gibson

Christopher Miller

Grace Chang

Supplemental Material

Supplemental material for this article is available online at the SAJ website

i

Disparities according to the Institute of Medicine are differences in treatment access that are not justified by differences in health status or preferences of the groups.

ii

Veterans returning from wars in Iraq and Afghanistan are defined as those women who served during the wars in Iraq and Afghanistan [Operation Enduring Freedom and Operation Iraqi Freedom, later renamed Operation New Dawn; OEF-OIF-OND], a period in which female enrollment in the US military grew rapidly.

iii

We included inpatient and outpatient settings because the VA has mandatory national performance measures for screening for at-risk drinking in inpatient and residential units including MH and SUD programs [D. Vahey, personal communication, October 19, 2018] This is relevant because most studies in the VA examine predictors of follow-up after screening for at-risk drinking in outpatient settings other than specialty care only, which may underestimate follow-up rates after screening for at-risk drinking.³⁹

iv

The modified binge-drinking variable in this study reflects through chart review combining a women Veteran reporting either drinking at least 5 or 6 drinks in one sitting (question 2) or 6 or more drinks on one occasion in the past year “less than monthly” or more frequently (question 3), the binary dependent variable was coded as 1.³⁹

References

Hoggatt

Williams

Der-Martirosian

Yano

Washington

DL.

National prevalence and correlates of alcohol misuse in women Veterans. J Subst Abuse Treat. 2015;52:10-16.

McGuire

Alegria

Cook

Wells

Zaslavsky

AM.

Implementing the Institute of Medicine definition of disparities: an application to mental health care. Health Serv Res. 2006;41(5):1979-2005. doi:10.1111/j.1475-6773.2006.00583.x

Chen

Glass

Bensley

KMK

Goldberg

Lehavot

Williams

EC.

Racial/ethnic and gender differences in receipt of brief intervention among patients with unhealthy alcohol use in the U.S. Veterans Health Administration. J Subst Abuse Treat. 2020;119:108078. doi:10.1016/j.jsat.2020.108078

Zemore

Murphy

Mulia

, et al A moderating role for gender in racial/ethnic disparities in alcohol services utilization: results from the 2000 to 2010 National Alcohol Surveys. Alcohol Clin Exp Res. 2014;38(8):2286-2296. doi:10.1111/acer.12500

Hoggatt

Simpson

Schweizer

Drexler

Yano

EM.

Identifying women Veterans with unhealthy alcohol use using gender-tailored screening. Am J Addict. 2018;27(2):97-100. doi:10.1111/ajad.12689

Carter

Borrero

Wessel

Washington

Bean-Mayberry

Corbelli

Racial and ethnic health care disparities among women in the Veterans Affairs Healthcare System: a systematic review. Womens Health Issues. 2016;26(4):401-409. doi:10.1016/j.whi.2016.03.009

Eichler

Equity in military and Veteran research: why it is essential to integrate an intersectional sex and gender lens. J Mil Veteran Fam Health. 2021;7(s1):143-149. doi:10.3138/jmvfh-2021-0016

Dallocchio

Women Veterans: examining identity through an intersectional lens. J Mil Veteran Fam Health. 2021;7(s1):111-121. doi:10.3138/jmvfh-2021-0028

Hicken

Kravitz-Wirtz

Durkee

Jackson

JS.

Racial inequalities in health: framing future research. Soc Sci Med. 2018;199:11-18. doi:10.1016/j.socscimed.2017.12.027

10.

Larson

Russ

Kahn

, et al Health disparities: a life course health development perspective and future research directions. In: Halfon

Forrest

Lerner

Faustman

, eds. Handbook of Life Course Health Development. Springer; 2018:499-520.

11.

Williams

Mohammed

SA.

Racism and health I: pathways and scientific evidence. Am Behav Sci. 2013;57(8):1152-1173. doi:10.1177/0002764213487340

12.

Katon

Benson

Sriskantharajah

, et al Experiences of discrimination among women and gender diverse Veterans using Veterans Health Administration health care. Health Equity. 2024;8(1):692-700.

13.

Lehavot

Beckman

Chen

Simpson

Williams

EC.

Race/ethnicity and sexual orientation disparities in mental health, sexism, and social support among women Veterans. Psychol Sex Orientat Gend Divers. 2019;6(3):347-358. doi:10.1037/sgd0000333

14.

Department of Veterans Affairs and Department of Defense. Substance use disorders: VA/DoD clinical practice guidelines. 2021. Accessed January 20, 2019. http://www.healthquality.va.gov

15.

Department of Veterans Affairs and Department of Defense. Substance use disorders: VA/DoD clinical practice guidelines. 2021. Accessed March 20, 2025. http://www.healthquality.va.gov

16.

Parthasarathy

Chi

Metz

, et al Disparities in the receipt of alcohol brief intervention: the intersectionality of sex, age, and race/ethnicity. Addiction. 2023;118(7):1258-1269. doi:10.1111/add.16195

17.

Arnett

JJ.

Emerging Adulthood: The Winding Road From the Late Teens Through the Twenties. Oxford University Press; 2023.

18.

Chavez

Williams

Lapham

Bradley

KA.

Association between alcohol screening scores and alcohol-related risks among female Veterans Affairs patients. J Stud Alcohol Drugs. 2012;73(3):391-400.

19.

Substance Abuse and Mental Health Services Administration, Center for Behavioral Health Statistics and Qulaity. The TEDS Report: Age of Substance Use Initiation Among Treatment Admissions Aged 18 to 30. Rockville, MD; 2014.

20.

Gobbi

Atkin

Zytynski

, et al Association of cannabis use in adolescence and risk of depression, anxiety, and suicidality in young adulthood: a systematic review and meta-analysis. JAMA Psychiatry. 2019;76(4):426-434.

21.

UNICEF Office of Research – Innocenti. The Adolescent Brain: A Second Window of Opportunity. UNICEF Office of Research – Innocentir; 2017.

22.

Silveri

. GABAergic contributions to alcohol responsibity during adolescence: insights from preclinical and clinical studies. Pharmacol Ther. 2014;143(2):197-216. doi:10.1016/j.pharmthera.2014.03.001

23.

Levy

Youth and the opioid epidemic. Pediatrics. 2019;143(2):22-34. doi:10.1542/peds.2018-2752

24.

Arnett

JJ.

Emerging adulthood. A theory of development from the late teens through the twenties. Am Psychol. 2000;55(5):469-480.

25.

Arnett

JJ.

Emerging adulthood: what is it, and what is it good for?. Child Dev Perspect. 2007;1(2):68-73. doi:10.1111/j.1750-8606.2007.00016.x

26.

Tucker

Chandler

Witkiewitz

Epidemiology of recovery from alcohol use disorder. Alcohol Res. 2020;40(3):2. doi:10.35946/arcr.v40.3.02

27.

Halfon

Hochstein

Life course health development: an integrated framework for developing health, policy, and research. Milbank Q. 2002;80(3):433-479. doi:10.1111/1468-0009.00019

28.

Lee

CA.

The identification and management of military sexual trauma in Veterans: ask the question!. J Nurse Pract. 2021;17(5):552-557. doi:10.1016/j.nurpra.2021.01.012

29.

Combellick

Dziura

Portnoy

Mattocks

Brandt

Haskell

SG.

Trauma and sexual risk: do men and women Veterans differ?. Womens Health Issues. 2019;29:S74-S82.

30.

Ryan

McGrath

Creech

Borsari

Predicting utilization of healthcare services in the Veterans Health Administration by returning women Veterans: the role of trauma exposure and symptoms of posttraumatic stress. Psychol Serv. 2015;12(4):412.

31.

Grossbard

Hawkins

Lapham

, et al Follow-up care for alcohol misuse among OEF/OIF Veterans with and without alcohol use disorders and posttraumatic stress disorder. J Subst Abuse Treat. 2013;45(5):409-415. doi:10.1016/j.jsat.2013.04.007

32.

Hawkins

Lapham

Kivlahan

Bradley

KA.

Recognition and management of alcohol misuse in OEF/OIF and other Veterans in the VA: a cross-sectional study. Drug Alcohol Depend. 2010;109(1-3):147-153. doi:10.1016/j.drugalcdep.2009.12.025

33.

Pugatch

Chang

Garnick

, et al Rates and predictors of brief intervention for women Veterans returning from recent wars: examining gaps in service delivery for unhealthy alcohol use. J Subst Abuse Treat. 2021;123:108257. doi:10.1016/j.jsat.2020.108257

34.

Andersen

Rice

Kominski

GF.

Changing the U.S. Health Care System: Key Issues in Health Services Policy and Management. 3rd ed. Jossey-Bass; 2007.

35.

Andersen

Newman

JF.

Societal and individual determinants of medical care utilization in the United States. Milbank Mem Fund Q Health Soc. 1973;51(1):95-124.

36.

Reifman

Arnett

Colwell

MJ.

Emerging adulthood: theory, assessment and application. J Youth Dev. 2007;2(1):37-48.

37.

Bradley

Bush

Epler

, et al Two brief alcohol-screening tests from the Alcohol Use Disorders Identification Test (AUDIT): validation in a female Veterans Affairs patient population. Arch Intern Med. 2003;163(7):821-829.

38.

Bradley

DeBenedetti

Volk

Williams

Frank

Kivlahan

DR.

AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res. 2007;31(7):1208-1217. doi:10.1111/j.1530-0277.2007.00403.x

39.

Pugatch

Female Veterans Returning From Wars in Iraq and Afghanistan Who Screen Positive for Alcohol Misuse in the Veterans Healthcare Administration: Predictors of Follow-Up and Completed Referral. Doctoral dissertation. The Heller School for Social Policy and Management, Brandeis University; 2019.

40.

Gilbert

Lowenstein

Koziol-McLain

Barta

Steiner

Chart reviews in emergency medicine research: where are the methods?. Ann Emerg Med. 1996;27(3):305-308.

41.

Vassar

Holzmann

The retrospective chart review: important methodological considerations. J Educ Eval Health Prof. 2013;10:12. doi:10.3352/jeehp.2013.10.12

42.

Andersen

Davidson

, Baumeister, SE. Improving access to care in America. In: Kominski GF, ed. Changing the US Health Care System: Key Issues in Health Services Policy and Management. 3rd ed. Jossey-Bass; 2007:3-31.

43.

United States Census Bureau. 2016 American Community Survey. Accessed March 30, 2018. https://www.census.gov/acs/www/data/data-tables-and-tools/data-profiles/2016/

44.

Chung

Creswell

Bachrach

Clark

Martin

CS.

Adolescent binge drinking. Alcohol Res. 2018;39(1):5-15.

45.

White

Tapert

Shukla

SD.

Binge drinking: predictors, patterns, and consequences. Alcohol Res. 2018;39(1):1-3.

46.

Stata Corp. Stata: Release 15. Statistical Software. StataCorp LLC; 2017.

47.

Bauer

Miller

Kim

, et al Partnering with health system operations leadership to develop a controlled implementation trial. Implement Sci. 2016;11(1):22. doi:10.1186/s13012-016-0385-7

48.

Naps

Maass

Kranzler

Smith

Ingram

Oslin

DW.

Predictors of treatment referral after AUDIT-C screening for heavy drinking. Addict Disord Their Treat. 2018;17(3):124.

49.

Alegria

Canino

Ríos

, et al Mental health care for Latinos: inequalities in use of specialty mental health services among Latinos, African Americans, and non-Latino Whites. Psychiatr Serv. 2002;53(12):1547-1555.

50.

Wong

Madanay

Ozer

, et al Digital health technology to enhance adolescent and young adult clinical preventive services: affordances and challenges. J Adolesc Health. 2020;67(2S):S24-S33. doi:10.1016/j.jadohealth.2019.10.018

51.

Giovanelli

Ozer

Dahl

RE.

Leveraging technology to improve health in adolescence: a developmental science perspective. J Adolesc Health. 2020;67(2S):S7-S13. doi:10.1016/j.jadohealth.2020.02.020

52.

Ozer

Lester

JC.

Innovative digital technologies to improve adolescent and young adult health. J Adolesc Health. 2020;67(2S):S3-S3. doi:10.1016/j.jadohealth.2020.05.015

53.

Ozer

Rowe

Tebb

, et al Fostering engagement in health behavior change: iterative development of an interactive narrative environment to enhance adolescent preventive health services. J Adolesc Health. 2020;67(2S):S34-S44. doi:10.1016/j.jadohealth.2020.04.022

54.

Creech

Borsari

Alcohol use, military sexual trauma, expectancies, and coping skills in women Veterans presenting to primary care. Addict Behav. 2014;39(2):379-385. doi:10.1016/j.addbeh.2013.02.006

55.

Steinhauer

Treated like a “piece of meat”: female Veterans endure harassment at the VA. The New York Times. 2019. Accessed March 22, 2024. https://www.nytimes.com/

56.

Williams

Lapham

Rubinsky

Chavez

Berger

Bradley

KA.

Influence of a targeted performance measure for brief intervention on gender differences in receipt of brief intervention among patients with unhealthy alcohol use in the Veterans Health Administration. J Subst Abuse Treat. 2017;81:11-16. doi:10.1016/j.jsat.2017.07.009

57.

Clary

Pena

Smith

DC.

Masculinity and stigma among emerging adult military members and Veterans: implications for encouraging help-seeking. Curr Psychol. 2023;42(6):4422-4438. doi:10.1007/s12144-021-01768-7

58.

Chang

Orav

Jones

Buynitsky

Gonzalez

Wilkins-Haug

Self-reported alcohol and drug use in pregnant young women: a pilot study of associated factors and identification. J Addict Med. 2011;5(3):221-226. doi:10.1097/ADM.0b013e318214360b

59.

Paz

Massey

KP.

Health disparity among Latina women: comparison with non-Latina women. Clin Med Insights Womens Health. 2016;2016(S1):71-74. doi:10.4137/CMWH.S38488

60.

Durlak

DuPre

EP.

Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327-350. doi:10.1007/s10464-008-9165-0

61.

Meyers

Durlak

Wandersman

The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol. 2012;50(3-4):462-480. doi:10.1007/s10464-012-9522-x

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.92 MB