Development and Delivery of a Theory-Guided Prescription Stimulant Misuse and Diversion Curriculum for College Health Providers: Results From Validity,Delphi,and Pre-Post Testing

Abstract

Background:

Past studies have shown a large percentage of college health providers (CHPs) feel responsible yet unprepared to address prescription stimulant misuse (PSM) and prescription stimulant diversion (PSD). As such, the authors developed a theory-guided, universal curriculum for CHPs to more confidently address PSM and PSD among their student populations. The purpose of the current study was to refine the curriculum using a multistep approach and assess the impact on providers’ PSM and PSD knowledge, attitudes, norms, barriers, and self-efficacy.

Methods:

The initial version of the curriculum was reviewed for content validity by 2 content/curriculum experts. Next, 5 students reviewed a revised version of the curriculum for face validity. The authors further refined the curriculum using the Delphi technique; specifically, a geographically diverse sample of 28 CHPs rated the appropriateness and clarity of each topic within each objective of the curriculum. Next, staff (N = 27) at 1 student health center received the training and completed a pre-post assessment to evaluate impact on knowledge, attitudes, norms, barriers, and self-efficacy. Quantitative (eg, means, medians, paired t-tests) and qualitative feedback were reviewed to guide refinement. These activities were completed between August 2023 and August 2024.

Results:

The criteria for content and face validity were met based on the reviews by curriculum experts and students, respectively. The Delphi technique required 2 rounds. All discussion topics had median and mode scores that reflected appropriateness and clarity. Pre-post assessment of the training showed significant improvements in CHPs knowledge, norms, self-efficacy, implementation barriers, and confidence to overcome barriers related to addressing PSM and PSD.

Conclusions:

A multistep process produced a comprehensive and theory-guided curriculum. Curriculum receipt resulted in improvements in multiple domains. Next steps include using an experimental design to test the impact of an intervention based on this training on student behavior, as well as digitizing and disseminating the training.

Keywords

prescription stimulant misuse diversion curriculum college health college health providers Delphi technique

Highlights

An educational curriculum for college health providers (CHPs) to address prescription stimulant misuse (PSM) and prescription stimulant diversion (PSD) among their student populations was refined via a multistep, collaborative process, including the Delphi technique.

The revised curriculum was delivered to CHPs at 1 campus health center.

Results from pre-post assessment suggest receiving the training led to improvements in providers’ knowledge, norms, self-efficacy, barriers, and confidence to overcome barriers related to addressing PSM and PSD.

Introduction

Prescription stimulant misuse (PSM; i.e., use without a prescription, use in ways other than prescribed [e.g., in excess and/or non-orally], and/or use for nonmedical reasons) and prescription stimulant diversion (PSD; i.e., sharing, selling, or trading of prescriptions stimulants¹) are interrelated behaviors engaged in by persons with and without a prescription.^2,3 One recent study using the 2021 to 2022 National Surveys on Drug Use and Health reported that among adults with a prescription, PSM prevalence was 25.3%.⁴ In a systematic review of 111 studies including college and noncollege populations, the prevalence of PSM ranged from 2.1% to 58.7%, whereas the prevalence of PSD ranged from 0.7% to 80.2%; college students were identified as a specific population at risk for both behaviors.⁵ PSM is associated with a multitude of adverse health effects (eg, increased blood pressure, irregular heartbeat, seizure, paranoia, psychosis, dependence, and overdose)⁶; PSD may increase the probability of PSM-associated adverse effects as diversion transactions lack oversight from a health care provider.⁷ Among students who reported engaging in PSM in the Spring 2023 National College Health Assessment,⁸ 19% were classified as engaging in problematic use (eg, using a higher dosage than prescribed, using more often than prescribed, having a strong urge to use, experiencing problems because of use, failing in efforts to reduce use). Given the scope of these interrelated behaviors, and the potential for morbidity and mortality, there is a need, particularly in college environments, to ensure providers who work with college students are trained to understand and address the multifaceted etiology of PSM and PSD.

College health providers (CHPs), in general, are motivated agents of change with respect to addressing PSM and PSD. Among 139 CHPs who participated in a web-based survey, 80% “Agreed” or “Strongly agreed” that they have concerns about PSD; moreover, 100% agreed that it is their responsibility to prevent PSM.⁹ In addition, significantly more CHPs versus family physicians (76% vs 66%) reported that they should be educating all college-aged patients about PSM.⁹ However, only 25% of CHPs felt “Extremely well prepared” to discuss risk behaviors, drug abuse, medication misuse, and prevention strategies with students. Furthermore, as compared to providers classified as family physicians, CHPs felt significantly less equipped to provide patients with this education.⁹ Enhancing the confidence of CHPs to engage in these discussions via formal training is critical, as a separate study found that students with a prescription were less likely to divert when their health care provider frequently engaged in conversations about diversion (e.g., discussions about “sharing” in general, as well as the health effects of sharing¹⁰).

Existing research with pediatric providers shows both a gap in formal PSM and PSD training, as well as the potential for provider training to improve prevention and intervention efforts; these are findings that parallel and show the promise of providing training within college health settings. In 1 study of 826 providers who work with children and adolescents,¹¹ 25% did not feel adequately prepared to discuss PSM with patients and 48% did not feel adequately prepared to discuss PSD. These percentages are consistent with results showing <50% of the physicians had received formal training in PSD, and those who did were nearly 2.5 times more likely to feel adequately prepared to address PSM and PSD.¹¹ In 1 study of pediatric primary care providers, McGuier et al¹² found that promoting providers’ PSD knowledge and skill is an essential first step in getting providers to engage in strategies designed to prevent PSD. In a related randomized controlled trial, pediatric primary care providers who received a 1-hour training in PSD prevention strategies had greater levels of knowledge and skill at 12- and 18-month follow-ups than providers who did not receive the training¹³; additionally, providers who received training were more likely to engage in educational strategies designed to prevent PSD.¹³ Training that enhances provider knowledge and skill is critical, particularly within the college setting, as an additional study by Molina et al¹⁴ found that college students’ intentions to engage in PSD decreased when providers were trained in PSD prevention strategies.

In light of existing training needs and the potential impact of formal training within the college health setting, the authors developed a theory-guided curriculum to enhance providers’ understanding and ability to address PSM and PSD. While the previously discussed training developed by McGuier et al included a 1-hour training for pediatric primary care providers, and focused solely on PSD strategies for patients with an existing prescription,¹³ our curriculum was designed for CHPs working with college students who may engage in PSM or PSD with or without a prescription. Our curriculum aimed to enhance both knowledge and skill by providing a comprehensive, theory-guided understanding of PSM and PSD, as well as practice with strategies to screen for and discuss PSM and PSD with patients. To our knowledge, this is the first theory-guided training curriculum that is universal in nature and designed to assist CHPs with addressing PSM and PSD among college students. The aim of the current study was to use a collaborative and robust multistep approach to refine this curriculum, deliver the curriculum to CHPs at 1 campus, and evaluate the training’s impact on recipients’ PSM and PSD knowledge, attitudes, norms, barriers, and self-efficacy. We hypothesized that the multistep approach to development would result in a formal training experience that had acceptable levels of content validity, face validity, appropriateness and clarity, and that delivery of the training to CHPs at 1 campus health center would be associated with improvements in knowledge, attitudes, normative beliefs, and self-efficacy to address PSM and PSD.

Methods

Curriculum Development

The original version of the curriculum (PSM/D_CHP: a curriculum for CHPs to understand and address PSM and PSD) was developed by the authors following the completion of an extensive review of the foundational PSM and PSD literature; that is, the authors drew upon their own published research,^15
-20 as well as publications from additional research teams and organizations.^5,21
-26 The original curriculum included 8 objectives selected based on needs identified in the literature, a 75-slide deck, an 8-item screening tool, and placeholders for 4 training videos. The original 8 objectives were as follows: (1) develop a basic understanding of PSM; (2) develop a basic understanding of PSD; (3) clarify the relationship between PSM and PSD; (4) develop the ability to universally screen for PSM and PSD; (5) intervene on PSM; (6) intervene on PSD; (7) recognize when to refer to treatment; and (8) identify strategies to overcome implementation barriers. Objectives aiming to provide a comprehensive understanding of PSM and PSD were framed within the context of the Theory of Triadic Influence (TTI); the TTI is an ecological meta-theory that categorizes behavioral risk factors by stream of influence (ie, intrapersonal, interpersonal, and environmental) and level of causation (ie, ultimate, distal, and proximal).²⁷ The TTI is an appropriate theory both for its comprehensiveness and for its prior use in foundational PSM and PSD research.^15
-17 Objectives aiming to provide strategies for discussing PSM and PSD with patients were framed within the context of motivational interviewing (MI), which is a patient-centered conversational approach that allows for collaborative, rather than confrontational, discussions about a patient’s internal motivations to change behavior.²⁸ Prior research has shown the ability of MI to be successfully incorporated into substance use discussions with students.^28,29

Content Validity Methodology

The approach used to establish content validity parallels existing research.³⁰ In September 2023, the authors selected 2 curriculum experts from 1 college campus to evaluate the extent to which the content in each objective accurately reflected the objective. One reviewer had a doctoral degree in education with an emphasis in curriculum development, and served as a practicing health provider; the second reviewer had a Master’s degree in Public Health and experience developing and delivering lesson plans as a tactical medical instructor. The authors provided reviewers with a copy of the curriculum materials and asked them to complete a web-based assessment. A 9-point Likert scale of agreement was used to assess each objective’s content validity (1 = Strongly disagree to 9 = Strongly agree; median scores of 7 and above were considered acceptable). The authors asked reviewers to elaborate on any ratings of disagreement, as well as to provide additional suggested improvements. Descriptive statistics and a review of qualitative feedback were used by the authors to determine needed edits.

Face Validity Methodology

Upon completion of the content validity review, and the implementation of suggested modifications, an existing approach was used to establish face validity.³⁰ From September to October 2023, 3 graduate and 2 undergraduate students from 1 college campus were recruited by the authors to evaluate whether the curriculum content appears to address each learning objective; students received a $25 gift card upon completion of their review. The authors gave reviewers a copy of the curriculum and asked them to complete a web-based assessment. A 9-point Likert scale of agreement was used for each objective (1 = Strongly disagree to 9 = Strongly agree; median scores of 7 and above were considered acceptable). In the web-based assessment, reviewers were asked to elaborate on any ratings of disagreement and provide specific examples of suggested improvements. Students were then asked to review the 8-item screening tool and comment on the clarity of questions, as well as whether any items were considered offensive. Lastly, they were asked to review the general structure of the MI-guided conversations providers would have with students. The authors used descriptive statistics and qualitative feedback to determine edits.

Delphi Technique Methodology

The authors further refined the curriculum using the classic Delphi technique (October-December 2023). The Delphi technique is a consensus-seeking method that allows experts (eg, CHPs) to provide their opinions on training tools.³¹ The technique involves a series of rounds whereby experts provide ratings, receive an anonymous summary of reviews, and then engage in additional rounds of reviews until there is consensus.³¹ Although there are no set standards for sample size when using the Delphi technique, Akins et al³² used bootstrapping techniques to illustrate that a relatively small sample (~20) of experts is sufficient when conducting the technique.

The Delphi technique used in this study is presented in Figure 1. To recruit CHPs from a geographically diverse sample of campuses, the authors used resources available to American College Health Association (ACHA) members. The ACHA includes over 900 colleges and universities,³³ which vary with respect to school type (eg, public and private, 2 and 4 years), size, and location. Initially, for each of the 5 geographic ACHA regions, the authors randomly selected 5 schools and e-mailed the listed primary contact to determine interest in participating in a multiple-round review of the curriculum. To supplement recruitment efforts, the research team also recruited participants from the ACHA discussion board, a resource for members. After recruiting 28 CHPs who were actively involved in patient care, the authors initiated the Delphi technique with the aim of achieving consensus with respect to appropriateness and clarity of the curriculum material. Survey materials were developed based on the curriculum content. In the first round of the technique, CHPs rated each of the curriculum items for appropriateness (defined as when the benefits of inclusion exceed consequences of inclusion^31,34) and clarity (defined as when the language captures the intended meaning).³⁴ A 9-point Likert scale of agreement was used to score appropriateness and clarity (ie, 1 = Strongly disagree; 9 = Strongly agree), and a comment box was included for each item for CHPs to elaborate on their rating. CHPs who rated an item less than “Mildly agree” were instructed by the authors to provide additional feedback. The authors reviewed mean and median scores for each item, with scores of 7 and above serving as an indicator of appropriateness and clarity; the authors also reviewed qualitative feedback to guide improvement efforts. Four weeks after the first round, the authors provided all CHPs with a document summarizing the modifications made to the curriculum materials and why they were made; mean and median scores were not provided so as not to influence participants during the second round of scoring. Reviewers were also provided with the updated materials and invited to complete a second round of scoring for clarity and appropriateness. Participants received a $100 gift card for each round of review in which they participated. Medians, modes, and means (including percent relative improvement) between rounds were compared by the authors to help determine the need for additional rounds of review.

Figure 1.

Delphi technique used for curriculum refinement.

Training Delivery

The revised curriculum was delivered to staff at 1 campus health center. Prior to training receipt, the staff were asked to complete a web-based pre-assessment survey. The survey, developed by the research team, included 5-point Likert scales for the domains of PSM and PSD knowledge (11 items; time 1 alpha = .92), attitudes (6 items; time 1 alpha = .87), norms (6 items; time 1 alpha = .88), self-efficacy (6 items; time 1 alpha = .95), implementation barriers (10 items; time 1 alpha = .87), and confidence to overcome implementation barriers (2 items). Knowledge was included given prior research, indicating knowledge is a necessary precursor for providers to engage in PSM/PSD prevention strategies¹²; the additional measures included align with the theory of planned behavior, which has been shown to be an appropriate theory to apply when assessing health care providers’ intentions to adopt new strategies following training.^35
-37

Over the course of 2 weeks in August 2024, the staff received curriculum training. The didactic training was led by 2 of the curriculum developers: the principal investigator and the MI specialist. Specifically, one 90-minute, face-to-face session was held addressing the first 4 learning objectives; the focus of this session was to provide a theory-guided, comprehensive understanding of PSM, PSD, and their interrelated nature. Prior to the second session, participants were asked to review specific training materials (ie, a manuscript¹⁹ published by the research team that used narrative vignettes to highlight PSM and PSD scenarios; and 5 training videos created by the research team that provided examples of MI-guided prevention and intervention strategies for CHPs to use in different PSM and PSD scenarios). A second 90-minute, face-to-face session was held the following week which covered the remaining objectives; this latter session focused on how to use MI strategies to discuss PSM and PSD with students. The session included role-play scenarios that were acted out by participant volunteers, after which feedback related to adherence to MI strategies was provided. After the second session, participants were asked to complete a post-assessment (time 2) survey. A total of 27 staff members completed both assessments, with a $25 gift card provided after each assessment. A series of paired t-tests were used to examine the impact of the training; we analyzed data using Stata SE 13.1 (College Station, TX, StataCorp LP).

Results

Content and Face Validity Results

Two experts with graduate degrees and clinical experience completed the content validity review of the curriculum. They were asked to rate their level of agreement that each set of slides accurately addressed each corresponding learning objective. For objectives 1 to 6, both reviewers provided a rating of “Moderately agree” (7 on the 9-point Likert scale). For objective 7, which was related to recognizing when to refer to treatment, 1 reviewer rated a 6 (“Mildly agree”) while the second reviewer rated a 7 (“Moderately agree”). For the eighth objective (Identifying strategies to overcome implementation barriers), 1 reviewer provided a rating of “Undecided” while the second reviewer rated a 7. Overall, the curriculum demonstrated content validity, although areas for improvement were noted. Recommendations for improvement included adding more examples of prescription stimulants and creating a flow chart for the screening tool to better facilitate use by CHPs.

After incorporating modifications, 5 students completed the face validity review of the updated curriculum. For all 8 objectives, students were asked to rate their level of agreement that each set of slides accurately addressed each corresponding learning objective. The median level of agreement for all 8 objectives was a 7 or higher, reflecting moderate agreement or above (range: 7-8); as such, the curriculum demonstrated face validity. Additionally, all 5 students shared that the screening tool was clear, and the wording used was not offensive. Students also reviewed the general structure of a MI-guided conversation providers could have with students about PSM and PSD, and all reviewers responded that they felt the conversational strategies were straightforward and appropriate.

Delphi Results

The Delphi technique included 28 CHPs. The 28 CHPs represented both public (n = 13) and private (n = 15) schools, larger (>10,000 students; n = 15) and smaller (<10,000 students; n = 13) campus sizes, and schools from all 5 ACHA geographic regions. The schools of all participating CHPs were 4-year campuses, and 2 of the 28 were from minority-serving institutions. Most participants reported a background in nursing or as a medical doctor (ie, Nurse Practitioner, Registered Nurse: n = 9; Medical Doctor: n = 8; Doctor of Osteopathic medicine: n = 3). Additional backgrounds included physician assistant, pharmacist, marriage and family therapist, social worker, health educator, prevention specialist, doctor of naturopathy, and an emergency medical technician. On average, participants had worked specifically in college health care for 8.43 years (range: 0-25 years) and in health care overall for 18.35 years (range: 5-37 years). Most participants were white (n = 21) and female-identifying (n = 23), and the mean age of participants was 48.11 years.

A 2-round Delphi procedure was utilized, and all but 1 CHP participated in both rounds. Table 1 provides the median, mode, and mean scores (including percent relative improvement) between Round 1 and Round 2 for the items within each objective. For Round 1, the authors provided the CHPs an 8-objective training delivered via 76 slides and 1 sample training video. After Round 1, all discussion topics had median and mode scores of 7 and above, reflecting consensus had been reached with respect to appropriateness and clarity. Moreover, only 3 of the discussion topics out of 68 had mean scores below 7. Although these results met our initial criteria for termination (ie, acceptable ratings for clarity and appropriateness), the qualitative feedback provided in Round 1 demonstrated room for improvement (eg, adding an initial objective with more background information on prescription stimulants; clarifying slides related to health effects and warning signs; providing more instruction for the screening tool). For Round 2, the CHPs were provided with a 106-slide curriculum that had 9 objectives (ie, the new, first objective was to develop a basic understanding of prescription stimulants; see Table 1), 1 sample training video, and a formal user guide for the screening tool; the authors also provided CHPs with a document summarizing and justifying 30 major revisions. In Round 2, all discussion topics continued to have median and mode scores of 7 and above. Unlike Round 1, none of the discussion topics had mean scores below 7. Moreover, 6 out of 9 objectives had uniformly positive relative improvements in mean scores for each discussion topic from Round 1 to Round 2; 3 objectives had positive relative improvements for the majority of topics. Although the criteria for appropriateness and clarity were met, qualitative feedback were reviewed to further refine the curriculum. Products developed following the completion of the Delphi technique included a primary curriculum including 9 objectives, 124 slides, a PSM and PSD screening tool, and 5 training videos showing MI-guided discussion strategies for different PSM and PSD scenarios (training videos are included as Supplemental Material).

Table 1.

Descriptive Statistics Comparing Round 1 and Round 2 of Delphi Technique (n = 28 Geographically Diverse College Health Providers)^a.

Objective	Round 1 Median	Round 2 Median	Round 1 Mode	Round 2 Mode	Round 1 Mean	Round 2 Mean	Relative improvement in mean score (%)
Objective 1: develop a basic understanding of prescription stimulants
The prescription stimulants defined slide is appropriate	N/A	8	N/A	8	N/A	8.30	N/A
The prescription stimulants defined slide has clarity	N/A	8	N/A	8	N/A	7.63	N/A
The prescription stimulant examples slide is appropriate	8	9	8	9	7.93	8.52	7.44
The prescription stimulant examples slide has clarity	9	9	9	9	7.86	8.33	5.98
The health effects of prescription stimulants slides are appropriate	NA	8	N/A	8	N/A	7.89	N/A
The health effects of prescription stimulants slides have clarity	N/A	8	N/A	8	N/A	7.07	N/A
The proactive approaches slide is appropriate	N/A	8	N/A	8	N/A	8.00	N/A
The proactive approaches slide has clarity	N/A	8	N/A	8	N/A	7.78	N/A
Objective 2: develop a basic understanding of PSM
The PSM definition slide is appropriate	8	8	8	8	8.04	8.44	4.98
The PSM definition slide has clarity	8	9	8	9	8.11	8.41	3.7
The PSM prevalence slides are appropriate	8	8	8	8	7.82	8.33	6.52
The PSM prevalence slides have clarity	8	8	8	9	6.79	7.67	12.96
The PSM motives and characteristics slides are appropriate	8	8	8	9	8.11	8.37	3.21
The PSM motives and characteristics slides have clarity	8	8	8	9	7.89	8.00	1.39
The PSM health effects slide is appropriate	8	8	8	8	7.93	7.19	9.33 (decrease)
The PSM health effects slide has clarity	8	8	8	8	7.25	7.07	2.48 (decrease)
The PSM warning signs slides are appropriate	8	8	8	8	8.00	7.41	7.38 (decrease)
The PSM warning signs slides have clarity	8	8	8	9	7.32	7.26	0.82 (decrease)
The PSM correlates slides are appropriate	8	8	8	8	7.82	8.07	3.2
The PSM correlates slides have clarity	8	7	8	9	6.96	7.22	3.74
The case study #1 slides are appropriate	8	9	8	9	8.07	8.44	4.58
The case study #1 slides have clarity	8	8	8	9	7.93	8.33	5.04
The case study #2 slides are appropriate	8	9	8	9	8.14	8.44	3.69
The case study #2 slides have clarity	8	8	8	9	7.79	8.19	5.13
Objective 3: develop a basic understanding of PSD
The PSD defined slides are appropriate	8	9	8	9	8.25	8.48	2.79
The PSD defined slides have clarity	8	8	8	8	8.07	8.48	5.08
The PSD prevalence slides are appropriate	8	8	8	8	7.68	8.48	10.42
The PSD prevalence slides have clarity	8	8	8	8	7.11	7.89	10.97
The reasons for PSD slide is appropriate	8	8	8	9	8.04	8.37	4.10
The reasons for PSD slide has clarity	8	8	8	8	7.71	8.22	6.61
The PSD legal and academic consequences slide is appropriate	9	8	9	8	7.71	8.41	9.08
The PSD legal and academic consequences slide has clarity	9	8	9	8	7.64	8.26	8.12
The harms of PSD slide is appropriate	8	8	8	8	7.75	8.37	8
The harms of PSD slide has clarity	8	8	8	9	7.39	7.93	7.31
The PSD warning signs slide is appropriate	8	8	8	9	7.82	8.41	7.54
The PSD warning signs slide has clarity	8	8	8	9	7.39	7.96	7.71
The PSD correlates slide is appropriate	8	8	8	8	7.96	8.22	3.27
The PSD correlates slide has clarity	8	8	8	9	7.07	7.37	4.24
The case study #3 slides are appropriate	8	8	8	8	8.04	8.44	4.98
The case study #3 slides have clarity	9	8	9	8	7.89	8.19	3.80
The case study #4 slides are appropriate	8	8	8	8	8.07	8.48	5.08
The case study #4 slides have clarity	8	8	8	8	7.93	8.15	2.77
Objective 4: clarify the relationship between PSM and PSD
The interrelated nature of PSM and PSD slides are appropriate	8	9	8	9	8.07	8.44	4.58
The interrelated nature of PSM and PSD slides have clarity	8	8	8	9	7.75	8.22	6.06
Objective 5: develop the ability to universally screen for PSM and PSD
The SBI key points slides are appropriate	8	9	8	9	8.18	8.26	0.98
The SBI key points slides have clarity	8	8	8	8	7.82	8.07	3.20
The SBI implementation slides are appropriate	8	8	8	8	7.86	8.22	4.58
The SBI implementation slides have clarity	8	8	8	8	7.33	7.88	7.5
The SBI PSM/PSD screener tool slides are appropriate	9	8	9	8	7.50	8.36	11.47
The SBI PSM/PSD screener tool slides have clarity	8	8	8	8	7.54	7.92	5.04
Objective 6 (intervene on PSM) and Objective 7 (intervene on PSD)
The When to use SBI on PSM behaviors slide is appropriate	8	8	8	8	7.36	7.89	7.2
The When to use SBI on PSM behaviors slide has clarity	7	8	7	8	7.36	7.70	4.62
The When to use SBI on PSD behaviors slide is appropriate	7	8	7	8	7.11	7.89	10.97
The When to use SBI on PSD behaviors slide has clarity	7	8	7	8	6.61	7.77	17.55
The How to do SBI using MI slides are appropriate	8	8	8	8	7.61	7.81	2.63
The How to do SBI using MI slides have clarity	8	8	8	8	7.36	7.26	1.36 (decrease)
The PSM/PSD SBI guided by MI slides are appropriate	8	8	8	8	7.54	7.69	1.99
The PSM/PSD SBI guided by MI slides have clarity^b	8	8	8	8	7.39	7.37	0.27 (decrease)
Objective 8: recognize when to refer to treatment
The When to refer PSM behaviors to treatment slide is appropriate	8	9	8	9	8	8.56	7
The When to refer PSM behaviors to treatment slide has clarity	8	9	8	9	7.18	8.26	15.04
The MI-guided treatment referral statements slide is appropriate	8	9	8	9	7.93	8.48	6.94
The MI-guided treatment referral statements slide has clarity	8	8	8	9	7.79	8.11	4.12
The treatment basics slide is appropriate	8	9	8	9	7.75	8.44	8.90
The treatment basics slide is has clarity	8	8	8	9	7.79	8.19	5.13
Objective 9: identify strategies to overcome implementation barriers
The Why does implementation matter slide is appropriate	8	8	8	9	7.75	8.11	4.65
The Why does implementation matter slide has clarity	8	8	8	9	7.61	8.22	8.02
The overcoming implementation barriers slides are appropriate	9	8	9	9	8.11	8.41	3.70
The overcoming implementation barriers slides have clarity	8	8	8	9	7.61	7.74	1.71
The additional barriers and strategies slide is appropriate	8	8	8	9	8.11	8.26	1.85
The additional barriers and strategies slide has clarity	8	8	8	9	7.71	8.15	5.71

Abbreviations: MI, motivational interviewing; PSD, prescription stimulant diversion; PSM, prescription stimulant misuse; SBI, screening and brief intervention.

A 9-point Likert scale of agreement was used to rate appropriateness and clarity, with higher scores representing higher levels of appropriateness and clarity (ie, 1 = Strongly disagree to 9 = Strongly agree); ^bFor this survey item, the question stated “appropriate” rather than “clarity”; however, given the structure of the survey (ie, alternating between appropriateness and clarity ratings), and the different results from the “appropriate” row above, we are confident the reviewers understood the item to mean “clarity.”

Training Delivery Results

The pre-post training assessment was completed by 27 staff members from 1 campus health center. Among participants, 22 (81.48%) identified as a woman, 10 (37.04%) identified as Latinx, and the racial breakdown included 12 (44.44%) white, 6 (22.22%) Asian, and 2 (7.41%) Black/African American participants. With respect to their roles, 12 (44.44%) were clinicians (ie, DO, MD, PA, RN, NP, or RD), 5 (18.52%) were medical assistants, 5 (18.52%) were health educators, 3 (11.11% were administrators), and 2 (7.41%) were pharmacists or pharmacy technicians. With respect to years of experience, the average number of years in health care was 16.67 (range: 0-40 years), and the average number of years in college health specifically was 8.96 (range: 0-30).

Results from the pre-post assessment showed significant (P < .05 or P < .01) improvements in all 11 indicators of knowledge, all 6 indicators of norms, all 6 indicators of self-efficacy, 2 of the barrier items (related to training time and compensation), and both items assessing confidence to overcome implementation barriers. Although there was no significant improvement in any of the items measuring attitudes, it should be noted that starting average values for the attitude items were relatively high and remained high at the post-assessment (see Table 2).

Table 2.

Results From Pre-Post Assessment of Training Implementation (N = 27 College Health Professionals From 1 Campus^a.

Area of assessment	Pre-mean (SD)	Post-mean (SD)	t
Knowledge: how knowledgeable are you as it relates to. . .
The basics of prescription stimulants (eg, examples of prescription stimulants)	2.00 (1.18)	2.56 (0.93)	−2.37*
The basics of PSM (eg, prevalence, motives, health effects, correlates, clinical presentations)	1.56 (1.09)	2.52 (0.85)	−4.74**
How to screen for PSM	0.59 (0.69)	2.26 (0.81)	−11.80**
How to intervene on PSM	0.59 (0.75)	2.18 (1.00)	−8.52**
How to refer students for PSM treatment	0.78 (1.19)	2.30 (1.14)	−6.46**
The basics of PSD (eg, prevalence, motives, correlates, consequences)	0.59 (0.89)	2.44 (1.09)	−8.25**
How to screen for PSD	0.41(0.69)	2.30 (0.95)	−10.08**
How to intervene on PSD	0.33 (0.73)	2.26 (1.02)	−9.67**
The relationship between PSM and PSD	0.63 (0.79)	2.67 (1.04)	−8.23**
Barriers to SBI implementation	0.52 (0.80)	2.37 (1.08)	−9.05**
Strategies for overcoming barriers to SBI implementation	0.33 (0.68)	2.30 (1.07)	−9.66**
Attitudes
Receiving SBI training in PSM is important	3.15 (0.66)	3.30 (0.61)	−1.07
I have a responsibility to screen my patients for PSM	2.74 (0.81)	2.70 (0.91)	0.20
Implementing SBI may help me prevent future PSM	3.00 (0.62)	3.19 (0.68)	−1.15
Receiving SBI training in PSD is important	3.22 (0.64)	3.19 (0.62)	0.27
I have a responsibility to screen my patients for PSD	2.78 (0.85)	2.67 (0.92)	0.57
Implementing SBI may help me prevent future PSD	3.04 (0.65)	3.07 (0.68)	−0.25
Norms: how many of your. . .
College health colleagues (broadly) are trained in SBI for PSM	0.89 (0.51)	1.89 (0.97)	−5.01**
Student Health Services [SHS] colleagues are trained in SBI for PSM	1.00 (0.55)	2.04 (1.06)	−4.40**
Colleagues (college health and SHS) implement SBI for PSM	0.89 (0.70)	1.85 (0.99)	−4.59**
College health colleagues (broadly) are trained in SBI for PSD	0.81 (0.56)	1.89 (1.09)	−5.60**
SHS colleagues are trained in SBI for PSD	0.81 (0.56)	2.04 (1.06)	−6.80**
Colleagues (college health and SHS) implement SBI for PSD	0.91 (0.68)	1.85 (1.10)	−4.65**
Self-efficacy: how confident are you in your ability to. . .
Screen for PSM	0.89 (0.93)	1.89 (0.93)	−4.42**
Intervene on low- and moderate-risk PSM during patient appointments using MI techniques	0.74 (0.94)	1.81 (0.96)	−4.63**
Refer high-risk PSM students to treatment	1.04 (0.98)	1.93 (1.21)	−3.52**
Screen for PSD	0.67 (0.92)	1.85 (0.99)	−4.96**
Intervene on PSD during patient appointments using MI techniques	0.59 (0.93)	1.85 (1.06)	−5.79**
Deliver prevention messages to support abstention for students who do not engage in PSM and PSD, using MI techniques	0.70 (0.95)	1.93 (1.11)	−5.50**
Barriers: At my health center, implementing SBI for PSM and PSD would be difficult due to. . .
Lack of adequate training	2.70 (0.82)	1.81 (0.79)	4.00**
Lack of time for training	2.15 (0.99)	1.93 (1.04)	0.92
Lack of compensation for training	2.11 (0.85)	1.44 (0.85)	2.87**
Lack of interest in completing training	2.00 (0.83)	1.74 (0.81)	1.37
Lack of time to screen students	2.52 (0.89)	2.41 (1.08)	0.55
Lack of time to implement brief intervention	2.22 (0.93)	2.11 (0.89)	0.44
Lack of organizational support (eg, medical director does not support SBI for PSM and PSD)	1.67 (0.96)	1.26 (0.76)	1.84
Lack of a close relationship with patient	1.70 (0.87)	1.63 (0.88)	0.34
Lack of comfort discussing substance use behaviors with a patient	1.89 (0.93)	1.74 (0.98)	0.63
Lack of confidence in ability to implement SBI	2.04 (0.85)	1.96 (0.90)	0.31
Confidence to overcome barriers: Please rate how confident you are in your ability to. . .
Overcome barriers to implement SBI for PSM	1.44 (0.89)	2.04 (1.02)	−2.41*
Overcome barriers to implement SBI for PSD	1.44 (0.89)	2.00 (1.00)	−2.37*

Abbreviations: MI, motivational interviewing; PSD, prescription stimulant diversion; PSM, prescription stimulant misuse; SBI, screening and brief intervention.

A 5-point (range: 0-4) Likert scale was used for each area of assessment (Knowledge: Not at all knowledgeable to Extremely knowledgeable; Attitudes: Strongly disagree to Strongly agree; Norms: None to All; Self-Efficacy: Not at all confident to Completely confident; Implementation barriers: Strongly disagree to Strongly agree; Confidence to overcome barriers: Not at all confident to Completely confident).

p < .05. **p < .01.

Discussions

PSM and PSD remain prevalent and interrelated behaviors on college campuses, and CHPs have been recognized as motivated but undertrained to address PSM and PSD. Accordingly, the aims of this study were to refine a theory-guided PSM and PSD curriculum and assess the impact of curriculum delivery on a sample of CHPs knowledge, attitudes, norms, self-efficacy, and related outcomes. The curriculum was reviewed for content validity by 2 content/curriculum experts, for face validity by 5 students at 1 campus, and for appropriateness and clarity via a 2-round Delphi technique that included 28 CHPs from geographically diverse campuses. Upon completion of each step of the development process, the authors used a combination of quantitative and qualitative feedback to revise the curriculum material. As hypothesized, the different iterations of the curriculum were well-rated by curriculum experts (content validity), students (face validity), and practicing clinicians (Delphi technique). Assessment of pre-post measures from training recipients showed significant improvements in indicators of knowledge, norms, self-efficacy, barriers, and confidence to overcome barriers. These findings are particularly noteworthy as knowledge has been shown to not only be lacking among providers motivated to address PSM and PSD,⁹ but also knowledge has been shown to be necessary to address these behaviors.¹³ That norms, self-efficacy, barriers, and confidence to overcome barriers were positively impacted is also encouraging, as past studies with providers have shown these variables are associated with successful adoption of new health-related strategies upon completion of trainings.^35
-37 Attitudes was the only indicator not positively impacted as hypothesized; however, ratings of these items (e.g., receiving training in PSM is important; receiving training in PSD is important; I have a responsibility to screen my patients for PSM) started high and remained high; this finding is also in alignment with past research about CHPs motivation to address PSM and PSD.⁹ Overall, the extensive efforts related to curriculum development resulted in training materials that addressed previously established deficits in CHP training.⁹

Strengths and Limitations

The study has several strengths. First, a robust review process was utilized, which incorporated multiple stakeholders. Specifically, in addition to including curriculum experts and college students in the review process, we sought out CHPs throughout the United States for their insights. Additionally, the CHPs included in the development process came from diverse backgrounds (both personally and with respect to the locations in which they practice). Their expertise helped with crafting a comprehensive curriculum that should have greater relatability. Our multistep process is also not without limitations. With respect to the content validity, face validity, and Delphi approach, participants self-selected to review the curriculum. Self-selection may be associated with more interest in the topic and therefore more positive ratings. However, we feel the potential pitfalls of self-selection were minimized given the number of recommendations for improvement provided by participants. Additionally, our training took place at 1 student health center. As such, results may not be generalizable to student health center staff across multiple campuses. Nonetheless, that improvements were seen in all areas excluding attitudes, which prior research shows is already in a favorable direction,⁹ gives confidence that the curriculum material are indeed effective at influencing precursors to successful PSM and PSD conversations by CHPs with student patients. However, it should be noted that we did not assess changes in actual ability to engage in PSM and PSD MI-guided discussions before and after training receipt. As such, any future intervention implemented based on this training should have an approval process whereby CHPs demonstrate proficient MI skill levels prior to commencing an intervention, should include MI booster trainings during intervention implementation, and should utilize a process evaluation that assesses adherence to MI strategies. Doing so should help promote both the knowledge and skill needed to implement the training material with fidelity, as a previous meta-analysis showed providers trained in MI have more MI proficiency than providers without such training.³⁸

Conclusions

Given the promising findings from this study, next steps include implementing and evaluating a screening and brief intervention based on the training. An experimental design such as a randomized control trial that includes students’ baseline and post-intervention levels of PSM and PSD, CHP intervention fidelity and performance monitoring, and feedback from both students and CHPs on feasibility and acceptability should prove insightful. Should an intervention show an impact on student PSM and PSD behaviors, we recommend digitizing and disseminating the curriculum that was created. The training can then be integrated into existing medical school curricula and/or as part of continuing education/continuing medical education training for practicing professionals.

Supplemental Material

sj-docx-1-saj-10.1177_29767342251355128 – Supplemental material for Development and Delivery of a Theory-Guided Prescription Stimulant Misuse and Diversion Curriculum for College Health Providers: Results From Validity, Delphi, and Pre-Post Testing

Supplemental material, sj-docx-1-saj-10.1177_29767342251355128 for Development and Delivery of a Theory-Guided Prescription Stimulant Misuse and Diversion Curriculum for College Health Providers: Results From Validity, Delphi, and Pre-Post Testing by Maysaa Chaalan, Sarah Iglesias, Shelby Samuelson, Liz Barnett, Alison Looby and Niloofar Bavarian in Substance Use & Addiction Journal

Footnotes

Acknowledgements

The authors wish to thank the 2 content experts (content validity phase), 5 students (face validity phase), and 28 college health providers (Delphi technique phase) who reviewed the curriculum discussed in this manuscript. The authors also wish to thank the student health staff who participated in the pilot training, as well as Adebimpe Akinwalere, Valentine Narksuwan, and Evie Ruiz for their contributions. Previous presentations: A poster presentation including some of the results presented in this manuscript were given at the 2024 and 2025 CSULB Week of Research—Student Research Symposiums.

Author Contributions

All authors contributed to curriculum development efforts. All authors contributed to the methods and analyses. NB wrote the first draft of the manuscript and all authors reviewed, edited, and approved the final version.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The findings reported here, including findings from the Delphi study, are based on research funded by the National Institute of Drug Abuse, National Institutes of Health, under 5R34DA056596-02. The Delphi study was conducted by an interdisciplinary team with representatives from Public Health, Psychology, Medicine, and Health Promotion. There was no methodological consulting.

Compliance,Ethical Standards,and Ethical Approval

Exempt Review Approval was granted on April 23, 2023, by the CSULB IRB, Office of Research and Economic Development (reference number: 2051600-1).

ORCID iD

Niloofar Bavarian

Outside Data

Not applicable.

Supplemental Material

Supplemental material for this article is available online at the SAJ website

References

McCabe

Teter

Boyd

CJ.

Medical use, illicit use and diversion of prescription stimulant medication. J Psychoactive Drugs. 2006;38(1):43-56. doi:10.1080/02791072.2006.10399827

Garcia

Valencia

Diaz Roldan

, et al. Prescription stimulant misuse and diversion events among college students: a qualitative study. J Prev. 2022;43(1):49-66. doi:10.1007/s10935-021-00654-z

Lam

Figueroa

Yomogida

Bavarian

Prescription stimulant diversion on a college campus: intrapersonal, interpersonal and environmental correlates. J Drug Issues. 2020;50(3):329-340. doi:10.1177/0022042620917103

Han

Jones

Volkow

, et al. Prescription stimulant use, misuse, and use disorder among US adults aged 18 to 64 years. JAMA Psychiatry. 2025;82(6):572-581. doi:10.1001/jamapsychiatry.2025.0054

Faraone

Rostain

Montano

Mason

Antshel

Newcorn

JH.

Systematic review: nonmedical use of prescription stimulants: risk factors, outcomes, and risk reduction strategies. J Am Acad Child Adolesc Psychiatry. 2020;59(1):100-112. doi:10.1016/j.jaac.2019.06.012

Drugabuse.gov. Prescription stimulants. 2018. Accessed June 6, 2023. https://www.drugabuse.gov/publications/drugfacts/prescription-stimulants

Goldsworthy

Schwartz

Mayhorn

CB.

Beyond abuse and exposure: framing the impact of prescription-medication sharing. Am J Public Health. 2008;98(6):1115-1121. doi:10.2105/AJPH.2007.123257

American College Health Association. American College Health Association-National College Health Assessment III: Undergraduate Student Reference Group Data Report Spring 2023. Silver Spring, MD: American College Health Association; 2023.

Loskutova

Waterman

Callen

Staton

Bullard

Shields

Knowledge, attitudes, and practice patterns of health professionals toward medical and non-medical stimulant use by young adults. J Am Board Fam Med. 2020;33(1):59-70. doi:10.3122/jabfm.2020.01.190071

10.

DeSantis

Anthony

Cohen

EL.

Illegal college ADHD stimulant distributors: characteristics and potential areas of intervention. Subst Use Misuse. 2013;48(6):446-456. doi:10.3109/10826084.2013.778281

11.

Colaneri

Keim

Adesman

Physician training and qualification to educate patients on attention-deficit/hyperactivity disorder stimulant diversion and misuse. J Child Adolesc Psychopharmacol. 2018;28(8):554-561. doi:10.1089/cap.2017.0066

12.

McGuier

Kolko

Joseph

, et al. Use of stimulant diversion prevention strategies in pediatric primary care and associations with provider characteristics. J Adolesc Health. 2021;68(4):808-815. doi:10.1016/j.jadohealth.2020.12.006

13.

McGuier

Kolko

Pedersen

, et al. Effects of training on use of stimulant diversion prevention strategies by pediatric primary care providers: results from a cluster-randomized trial. Prev Sci. 2022;23(7):1299-1307. doi:10.1007/s11121-022-01411-2

14.

Molina

BSG

Kipp

Joseph

, et al. Stimulant diversion risk among college students treated for ADHD: primary care provider prevention training. Acad Pediatr. 2020;20(1):119-127. doi:10.1016/j.acap.2019.06.002

15.

Bavarian

Flay

Ketcham

Smit

Illicit use of prescription stimulants in a college student sample: a theory-guided analysis. Drug Alcohol Depend. 2013;132(3):665-673. doi:10.1016/j.drugalcdep.2013.04.024

16.

Bavarian

Flay

Ketcham

Smit

The illicit use of prescription stimulants on college campuses: a theory-guided systematic review. Health Educ Behav. 2015;42(6):719-729. doi:10.1177/1090198115580576

17.

Bavarian

Flay

Ketcham

, et al. Using structural equation modeling to understand prescription stimulant misuse: a test of the Theory of Triadic Influence. Drug Alcohol Depend. 2014;138:193-201. doi:10.1016/j.drugalcdep.2014.02.700

18.

Looby

Beyer

Zimmerman

Non-medical prescription stimulant use: investigating modifiable risk factors. Addict Res Theory. 2015; 27;23(2):143-147. doi:10.3109/16066359.2014.946411

19.

Valencia

Garcia

Diaz Roldan

, et al. Using narrative vignettes to understand the misuse and diversion of stimulants among college students with and without a prescription. Emerg Adulthood. 2022;10(6):1420-1429. doi:10.1177/21676968221116544

20.

Looby

De Young

Earleywine

Challenging expectancies to prevent nonmedical prescription stimulant use: a randomized, controlled trial. Drug Alcohol Depend. 2013;132(1-2):362-368. doi:10.1016/j.drugalcdep.2013.03.003

21.

Arria

Caldeira

O’Grady

Vincent

Johnson

Wish

. Nonmedical use of prescription stimulants among college students: associations with attention-deficit-hyperactivity disorder and polydrug use. Pharmacotherapy. 2008;28(2):156-169. doi:10.1592/phco.28.2.156

22.

Edinoff

Nix

McNeil

, et al. Prescription stimulants in college and medical students: a narrative review of misuse, cognitive impact, and adverse effects. Psychiatry Int. 2022; 3(3):221-235. doi:10.3390/psychiatryint3030018

23.

Garnier

Arria

Caldeira

Vincent

O’Grady

Wish

. Sharing and selling of prescription medications in a college student sample. J Clin Psychiatry. 2010;71(3):262-269. doi:10.4088/jcp.09m05189ecr

24.

McCabe

West

Teter

Boyd

CJ.

Trends in medical use, diversion, and nonmedical use of prescription medications among college students from 2003 to 2013: connecting the dots. Addict Behav. 2014;39(7):1176-1182. doi:10.1016/j.addbeh.2014.03.008

25.

Substance Abuse and Mental Health Services Administration (SAMHSA). Prescription Stimulant Misuse and Prevention Among Youth and Young Adults. Publication No PEP21-06-01-003. National Mental Health and Substance Use Policy Laboratory, Substance Abuse and Mental Health Services Administration; 2021.

26.

Egan

Reboussin

Blocker

Wolfson

Sutfin

EL.

Simultaneous use of non-medical ADHD prescription stimulants and alcohol among undergraduate students. Drug Alcohol Depend. 2013;131(1-2):71-77. doi:10.1016/j.drugalcdep.2012.12.004

27.

Flay

Snyder

Petraitis

. The theory of triadic influence. In: DiClemente

Crosby

Kegler

, eds. Emerging Theories in Health Promotion Practice and Research. 2nd ed. Jossey-Bass/Wiley; 2009:451-510.

28.

Smedslund

Berg

Hammerstrøm

, et al. Motivational interviewing for substance abuse. Campbell Syst Rev. 2011;7(1):1-26. doi:10.4073/csr.2011.6

29.

Logan

Kilmer

King

Larimer

ME.

Alcohol interventions for mandated students: behavioral outcomes from a randomized controlled pilot study. J Stud Alcohol Drugs. 2015;76(1):31-37. doi:10.15288/jsad.2015.76.31

30.

Bavarian

Flay

Ketcham

Smit

Development and psychometric properties of a theory-guided prescription stimulant misuse questionnaire for college students. Subst Use Misuse. 2013;48(6):457-469. doi:10.3109/10826084.2013.778283

31.

Fatani

Bakke

Halpape

D’Eon

El-Aneed

Development and validation of patient-community pharmacist encounter toolkit regarding substance misuse: Delphi procedure. J Am Pharm Assoc. 2022;62(1):176-186. doi:10.1016/j.japh.2021.08.018

32.

Akins

Tolson

Cole

BR.

Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol. 2005;5(1):37. doi:10.1186/1471-2288-5-37

33.

ACHA. About our members. 2025. Accessed June 9, 2023. https://www.acha.org/about/about-our-members/

34.

Fitch

Bernstein María

Aguilar

, et al. The RAND/UCLA appropriateness method user’s manual. RAND. Accessed October 9, 2023.https://www.rand.org/content/dam/rand/pubs/monograph_reports/2011/MR1269.pdf

35.

Ryba

Brothers

Andersen

BL.

Predicting implementation of an empirically supported treatment for cancer patients using the theory of planned behavior. Health Psychol. 2019;38(12):1075. doi:10.1037/hea0000794

36.

Appleby

BE.

Implementing guideline-checklists: evaluating health care providers intentional behaviour using an extended model of the theory of planned behaviour. J Eval Clin Pract. 2019;25(4):664-675. doi:10.1111/jep.13075

37.

Ingersoll

Straiton

Casagrande

Pickard

Community providers’ intentions to use a parent-mediated intervention for children with ASD following training: an application of the theory of planned behavior. BMC Res Notes. 2018;11:1-6. doi:10.1186/s13104-018-3879-3

38.

de Roten

Zimmermann

Ortega

Despland

. Meta-analysis of the effects of MI training on clinicians’ behavior. J Subst Abuse Treat. 2013;45(2):155-162. doi:10.1016/j.jsat.2013.02.006

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB