Longitudinal evaluation of the Teen Outreach Programme: Impacts of a health promotion programme on risky sexual behaviours

Abstract

Objective:

This study evaluated the longitudinal impact of the Teen Outreach Programme (TOP) on adolescent pregnancy, sexual behaviour, risky sex and sexual intentions in non-metropolitan high schools.

Design:

Pair-matched, cluster randomised controlled trial.

Setting:

Non-metropolitan public high schools in Florida, USA.

Methods:

TOP was evaluated against the standard school health curriculum content. Young people (N = 3,836) enrolled in 26 schools were surveyed at baseline (2012), immediate post-programme (2013) and at 1-year follow-up (2014). Most youth were 14 years old and in ninth grade at baseline. Missing data were imputed using a sequential imputation procedure and analysis took place using generalised linear mixed-effects models with logit link function.

Results:

Post-programme, young people in the treatment group were less likely to report ever having sex, recent sex, recent sex without a condom or intention to have sex in the next year. At 1-year follow-up, programme participants were less likely to report ever having been pregnant or having got someone pregnant. When examining treatment effects by gender, statistically significant differences were found only for female programme participants.

Conclusion:

Although many protective factors dissipated by second follow-up and were primarily found among female programme participants, TOP has the potential to decrease risky sexual behaviour.

Keywords

Adolescent pregnancy prevention randomised controlled trial reproductive health sexual health

Introduction

Although the teenage pregnancy rate in the USA has declined substantially over the past 20 years (Finer and Zolna, 2016), it continues to exceed the rate of many other developed countries. In 2013, approximately 5% of teenagers aged 15–19 years became pregnant (Kost et al., 2017), a figure 7 times higher than Switzerland, the country with the lowest rate (Sedgh et al., 2015). Teenage pregnancy is associated with limited educational achievement, poor economic conditions and adverse health effects on the children of teenage parents (Hoffman and Maynard, 2008; Ng and Kaye, 2012). The USA also fares poorly on rates of sexually transmitted infections (STIs), for example, the prevalence of Chlamydia is twice the rate of the Netherlands (Alford and Hauser, 2011).

The state of Florida, where the randomised control trial (RCT) described in this article was carried out, has particularly poor adolescent sexual health outcomes compared to other states; it has the 19th highest teenage pregnancy rate (60 per 1000 females aged 15–19) (Kost and Henshaw, 2014) and high rates of STIs, including Chlamydia, gonorrhoea and syphilis, (Centers for Disease Control and Prevention [CDC]) National Centre for HIV, STD and TB Prevention (NCHSTP), Division of STD/HIV Prevention, 2015. In addition, according to CDC surveillance data from 2014, Florida had the 4th highest diagnosis rate for the human immunodeficiency virus (HIV) for young people 13–24 years old (CDC, 2016).

School-based health interventions hold promise as an avenue to influence young people’s sexual behaviours (Shackleton et al., 2016; Teitler and Weiss, 2000; Wilson et al., 2008), but the long-term effects of such interventions are not well understood. In this article, we present findings from a longitudinal evaluation study of Wyman’s Teen Outreach Programme (TOP), a positive youth development (PYD) intervention aimed at reducing teenage pregnancy and improving academic outcomes. Specifically, we address the longitudinal impact of TOP as compared to business-as-usual (i.e. regularly scheduled health classes) on the following six sexual behaviours/outcomes:

1) Ever having had sexual intercourse;

2) Having sexual intercourse in the past 3 months;

3) Having sexual intercourse in the last 3 months without a condom;

4) Ever having been pregnant (or gotten someone pregnant);

5) Intention to have sexual intercourse in the next year;

6) Intention to have sexual intercourse in the next year without using a condom.

Intervention

In 2010, the US Department of Health and Human Services Office of Adolescent Health funded 17 independent replications of TOP as part of the Teen Pregnancy Prevention (TPP) programme (Kappeler and Farb, 2014). The Florida Department of Health (FL DOH) led one of the five school-based RCT replication trials (Francis et al., 2016), and the University of South Florida College of Public Health was selected to provide external programme evaluation. The programme was delivered in school classrooms by facilitators – not by the teachers. Programme facilitators were employees of the Florida Department of Health at the county level; they were trained and annually recertified to deliver the TOP. Wyman’s¹ TOP Level 2 curriculum (suitable for 14-year-olds) was facilitated in school classrooms as supplementary education, that is, in addition to business-as-usual programming. The programme was implemented in the following classes: Health Opportunities through Physical Education (HOPE), HOPE/Physical Education (PE) or Personal Fitness classes. These classes were chosen to maximise programme reach, as 9th-grade students typically enrol in these classes and in each school, and these class types are required for graduation. Each intervention school had only one facilitator.

TOP is a widely replicated evidence-based PYD programme (Goesling et al., 2014) that utilises (1) weekly curriculum sessions, (2) community service learning (CSL), and (3) positive adult guidance and support to reduce the risk of adolescent pregnancy, school dropout, and course failure (Wyman Centre, 2016). Rather than focusing on deficits, programmes using the PYD paradigm emphasise internal and external assets in order to build positive attributes among youth for risk reduction (Gavin et al., 2010; Roth and Brooks-Gunn, 2003). Core elements of effective PYD programmes include a structured curriculum, programme delivery lasting 9 months or longer, attention to programme quality and implementation fidelity (Catalano et al., 2004).

TOP has shown previous success reducing teenage pregnancy (Allen and Philliber, 1991; Allen et al., 1994, 1997), academic suspension (Allen and Philliber, 1991; Allen et al., 1990, 1994, 1997), and course failure (Allen and Philliber, 1991; Allen et al., 1990, 1994). To achieve fidelity, the intended programme dosage for each participant is a minimum of 25 weekly sessions, sessions over the span of 9 months, and at least 20 hours of CSL over 9 months (Wyman Center, 2010).

Methods

Evaluation design and setting

This evaluation study used a pair-matched cluster randomised controlled trial (RCT) design to assess the impact of the TOP in schools in non-metropolitan Florida counties (Walsh-Buhi et al., 2016). Counties were considered for the evaluation if they had rates poorer than the state average for at least one of the following: (1) birth rate per female population aged 15–19 years, (2) repeat birth rate per female population aged 15–19 years, (3) combined Chlamydia and gonorrhoea rates per female population aged 15–19 years, (4) high school dropout rates, (5) graduation rates, and (6) out-of-school suspension rates. Based on these criteria, 30 countries were deemed eligible.

Florida Department of Health staff contacted the eligible county health departments and school districts to determine if they had the capacity or interest in participating in the evaluation. Of the 30 counties approached, 15 counties initially agreed to participate. Reasons for non-participation at the county-level included the school district declined to participate (n = 8; by superintendent decision or school board vote), county health department or school district decided to pursue alternative funding or intervention types (n = 5), or school district decision to only approve the placement of interventions in non-traditional (alternative) school sites (n = 2). Three counties were later determined to be ineligible because the classes chosen for implementation were only one semester long. These counties were excluded in an effort to standardise the programme duration at 9 months, over which participants would receive a minimum of 25 required lessons and 20 CSL project hours.

We conducted a power analysis prior to data collection and recalculated the power based on the actual sample size after data had been collected. We used methods outlined by Raudenbush for cluster randomised trials (Raudenbush, 1997) and the Optimal Design Software (Spybrook et al., 2011) to perform these calculations based on two-tailed tests with significance level of .05 and intraclass correlation coefficient of .01. For analyses involving 26 schools, we estimated that we have sufficient statistical power (80%) to detect a difference of 2.8 percentage points for pregnancy and 6.4 percentage points for sex initiation.

All traditional public schools within the 12 selected counties were eligible to participate if they offered year-long health or physical education courses, agreed to random assignment to treatment or control conditions, and agreed to survey administration at multiple data collection points. Participating schools were matched into pairs based on geographic proximity (within county matches were prioritised, which occurred in 11 of 13 pairs; one of the remaining pairs occurred in neighbouring counties), course type offered (i.e. HOPE, HOPE/PE), course scheduling (i.e. block or non-block schedule), and student population size. Then, matched pairs were randomly assigned to either receive TOP (the intervention condition, as a supplement to course content) or conduct business-as-usual health classes (the control condition).

Despite Florida statutes that mandate educational standards, variation exists between the curriculum offered by Florida counties. The 2011 Florida Statutes Title XLVII (K-20 Education Code 1003.42) describes required instruction, including standards for ‘comprehensive health education’ which addresses:

… concepts of community health; consumer health; environmental health; family life, including an awareness of the benefits of sexual abstinence as the expected standard and the consequences of teenage pregnancy; mental and emotional health; injury prevention and safety; Internet safety; nutrition; personal health; prevention and control of disease; and substance use and abuse.

Although the statute emphasises abstinence as the ‘expected standard’, it is important to note that this does not preclude discussion beyond abstinence. In fact, 15 Florida school districts adopted a more comprehensive reproductive and sexual health curriculum that requires ‘evidence-based, medically accurate, age-appropriate information including the health benefits and side effects of contraception and condoms’ in 2017 (Florida Department of Education, 2019), which was several years after our evaluation. Regardless of the reproductive and sexual health education policies in each school district, Florida statutes also protect parent’s right to request in writing that their child be exempted from any coursework related to ‘reproductive health or any disease, including HIV, its symptoms, development and treatment’ without penalty. Furthermore, regional differences in how sexuality education is carried out have been observed, with teachers in North Florida being more likely to teach abstinence than Central and South Florida (Bandiera et al., 2008). However, one of the strengths of TOP curriculum is that the programme – which focuses on PYD – is flexible and that no specific lesson (including any of the three sexuality lessons) is required (Wyman Centre, 2010).

In autumn 2012, a total of 28 high schools in 12 Florida counties were enrolled in the study. All traditional high schools in these counties participated in the evaluation. In 2013, one school chose to not continue participation in the study; this school and its matched pair were removed from analyses. Of the 26 remaining schools, 13 received TOP and 13 received business-as-usual. This resulted in 70 classes in the treatment arm and 84 classes in the control arm.

Data collection

This study offers a longitudinal analysis of youth enrolled in TOP beginning in fall 2012. Assessments occurred at 3 separate time points: Baseline (prior to programme implementation, early autumn 2012), First Follow-Up (immediate post intervention, late spring 2013), and Second Follow-Up (10–12 months post-programme implementation, late spring 2014). Young people were eligible to participate if they were (1) enrolled in a course selected for evaluation, (2) proficient in English and (3) capable of independently taking a paper and pencil survey. A total of 4,789 young people were eligible to participate in the study, with 2,198 being assigned to treatment and 2,590 youth assigned to the comparison condition.

All study procedures and data collection instruments were reviewed and approved by the Florida Department of Health Institutional Review Board (Protocol H11180). In line with institutional review board approval, parental consent was conducted through a passive process, allowing parents and guardians to opt their children out of the evaluation. Opt-out forms were sent home with youth. If the parent or legal guardian did not want their child to participate in the evaluation, then the parent/legal guardian needed to fill out the opt-out form and return to the school. Prior to survey administration, an assent form was given to each youth. If the youth did not assent, then they did not complete the survey. Data collection occurred within schools. Students who were not present in school to complete the second follow-up survey were contacted by telephone and email, as available from their study records. Study enrollment from baseline through second follow-up are shown in Figure 1. In two of the treatment schools, multiple eligible classes occurred during the same class period. Since each school had only one facilitator, random subsampling occurred in these two treatment schools and their matched pairs. In addition, one of the larger control schools had so many eligible classes that data collection in all classes was not feasible. Random subsampling occurred in this control school. In total, random subsampling occurred in 5 (two intervention and three control) of the 26 schools included in the analysis.

Figure 1.

CONSORT Diagram for Teen Outreach Programme Youth.

Measures

TOP was designated as an evidence-based intervention because of its impacts on teen pregnancy; thus, the outcomes of interest in the current study were as follows: ever had sex (ever sex), ever having been pregnant or caused a pregnancy (ever pregnant), sexual intercourse in the past 3 months (recent sex), and condom use in the past 3 months (recent condom use). Sexual intention outcomes were also measured, including intention to have sexual intercourse within the next year (intend sex) and intention to use a condom within the next year (intend sex without condom). Sexual intention questions were reported on a Likert-type scale, but dichotomised for analysis. A detailed description of the outcomes, related survey question and response options are summarised in Table 1. Demographic characteristics and school clustering were also measured. All measures were self-reported and were included at each time point of the study; first follow-up and second follow-up results are reported. To measure fidelity, data collection included attendance records, TOP facilitator curriculum logs, and TOP CSL records.

Table 1.

Definition of outcomes included in analysis.

Variable name	Response options	Recoded	Question
Ever sex	0 = ‘No’ 1 = ‘Yes’	n/a	Have you ever had sexual intercourse?
Ever pregnant	0 = ‘No’ 1 = ‘Yes’	n/a	To the best of your knowledge, have you ever been pregnant or gotten someone pregnant, even if no child was born?
Recent sex	0 = ‘No’ 1 = ‘Yes’	n/a	In the past 3 months, have you had sexual intercourse, even once?
Recent condom use	0 = ‘No’ 1 = ‘Yes’	n/a	In the past 3 months, have you had sexual intercourse without using a condom, even once?
Intend sex	4 = ‘No, definitely not’ 3 = ‘No, probably not’	0 = ‘No’	Do you intend to have sexual intercourse in the next year?
Intend sex	2 = ‘Yes, probably’ 1 = ‘Yes, definitely’	1 = ‘Yes’	Do you intend to have sexual intercourse in the next year?
Intend sex without a condom	4 = ‘No, definitely not’ 3 = ‘No, probably not’	0 = ‘No’	If you have sexual intercourse in the next year, do you intend to use a condom? (Reverse coded; calculated odds represents odds of intent to have sex without a condom)
Intend sex without a condom	2 = ‘Yes, probably’ 1 = ‘Yes, definitely’	1 = ‘Yes’

Analytic methods

The primary research question in this study, pertaining to the effect of treatment on sexual outcomes, behaviours and intentions over time, was addressed using generalised linear mixed effects models with a logit link function (Breslow and Clayton, 1993). Three levels (i.e. students within classrooms, nested within schools in the same county) of random effects were included to account for clustering effects. Repeated measures were calculated across the three time points (baseline, first follow-up, and second follow-up) for each individual, and the random variation of each observation was taken into account. Models with an additional level of random effects at the school level were also fitted. However, many of these models showed problems with convergence due to the school level clustering effects being too small. The estimates from these models were also very similar to models without school level random effects, thus this additional level of clustering was not considered in the final models. All models included fixed effects for demographic variables (gender, age group, and race/ethnicity) and the treatment condition. In addition, separate outcomes analyses were performed for young women and young men. Randomisation block indicators, that is, school pair, were also included in the models except for those using outcomes corresponding to question ‘Have you ever been pregnant or gotten someone pregnant?’, which has a very low prevalence rate in some of the randomisation blocks. For those models, blocks were collapsed into more inclusive categories based on the number of students and geographic setting of each school. All models were fitted using the SAS GLIMMIX procedure.

Missing data were imputed using a sequential multiple imputation (MI) procedure that accounted for the dependency structure across all 3 time points (Schafer, 1997; Van Buuren, 2007). Missing response values were imputed based on the variables of school identifier, age, race/ethnicity, and gender. A total of 40 imputation data sets were computed separately for control and treatment groups. Model estimates from the imputed data sets were combined using the SAS MIANALYZE procedure, providing 99% efficiency relative to a full sample analysis. There were several types of missing data that MI addressed. Examples include the following scenarios: a respondent might have completed the baseline survey, but did not complete the first follow-up survey; a respondent may have completed the baseline and first follow-up surveys, but did not complete a second follow-up survey, and so on. Using MI, the final analytical sample was 3,836. The numbers presented in the CONSORT diagram (Figure 1) were the number of participants that completed a survey at each time point. Therefore, the analytic sample presented in Table 2 varies from the survey response numbers reported in Figure 1.

Table 2.

Baseline demographic characteristics of the TOP study sample.^a

	Both genders		Female		Male
	Treatment	Control	Treatment	Control	Treatment	Control
	(n = 1,744)	(n = 2,092)	(n = 833)	(n = 1,055)	(n = 911)	(n = 1,037)
Gender
Males	52.2%	49.6%
Females	47.8%	50.4%
Age group
⩽14	63.2%	60.4%	65.7%	64.1%	60.9%	56.6%
15	22.3%	25.7%	19.9%	22.8%	24.5%	28.5%
⩾16	14.5%	14.0%	14.4%	13.1%	14.6%	14.9%
Race/ethnicity
Black	9.4%	8.8%	9.6%	8.8%	9.1%	8.9%
Hispanic/Latino	21.9%	20.1%	23.2%	19.3%	20.8%	20.8%
White	60.4%	62.1%	60.0%	62.5%	60.8%	61.6%
Other	8.3%	9.0%	7.2%	9.4%	9.3%	8.7%

Differences between treatment groups at baseline were assessed with a logistic mixed effects model for binary outcomes and with a generalised logistic mixed effects model for categorical outcomes with treatment and block modelled as fixed effects and school as random effects. There were no significant differences in baseline characteristics between treatment and control groups (p < 0.05).

In previous publications from this evaluation, different methodological approaches yielded slightly different analytic samples. In the publication by Walsh-Buhi et al. (2016), the sample size included 28 schools as opposed to 26 schools reported here. In the publication by Francis et al. (2016), the analytic sample only included youth with non-missing outcome data, that is, MI was not conducted.

Equivalence of baseline demographic variables between control and treatment groups was assessed on the pre-imputation sample for those young people with complete demographic information at baseline. Baseline risk equivalence was determined using all available pre-imputation baseline data for each risk variable. Logistic mixed-effects model was used to calculate differences for binary outcomes and generalised linear mixed effects models were used to determine differences for categorical outcomes.

Equivalence was also tested for young people who, after completing the baseline survey, did or did not complete a follow-up survey. For both follow-up surveys, young people who were present for the surveys were significantly more likely to be younger, female, and more likely to report being White or ‘other’ rather than Black compared to young people who were not present. Young people who were present were also less likely to answer ‘yes’ to both the ever had sex question and the ever been pregnant or got someone pregnant question at baseline.

Results

Sample characteristics

Of the 3,836 young people in the analytic sample, 1,888 were female (49.2%) and 1,948 were male (50.8%). There were small differences in demographic characteristics (gender, age, race/ethnicity) between treatment groups, but none were statistically significant (Table 2). In the control group, 50.4% of young people were female, whereas 47.8% of treatment youth were female. The majority of young people in the sample were in the youngest age group (⩽14 years), followed by youth 15 years of age, and 16 years of age or older. Slightly more young people aged 14 years and younger were present in the treatment group relative to the control. Most of the young people in the sample identified as White or Hispanic and approximately equal percentages identified as Black or ‘other’. The female and male subsets showed demographic characteristics mostly similar to the total sample, although females tended to be younger.

Implementation findings

TOP was implemented in 70 individual classes. Of the 70 classes, 51 classes (73%) offered at least 25 weekly sessions, as prescribed (range: 23–57 sessions). Relatively few classes—8 (or 11%)—received at least 20 CSL project hours, as prescribed. No classes received programming over a span of 9 months. Measures for implementation fidelity were limited because of incomplete attendance data, facilitator logs, and CSL records. Thus, a dosage analysis could not be conducted.

Sexual behavioural characteristics at baseline

When we examined our analytical sample for baseline equivalence, we found that there were no statistically significant differences in baseline sexual health risk between the treatment and control groups (Table 3). Approximately 16% of participants reported having sex in the past 3 months at baseline. Almost 1 in 10 young people reported having sex without a condom in that same time period. Approximately 30% of participants intended to have sex in the next year and between 7% and 8% reported an intention to have sex without a condom in the next year. While there were no significant differences in baseline risk measures between treatment and control groups for either gender, there was a significant difference between males and females in the intention to have sex at baseline, with females being less likely to intend to have sex (OR: 0.40, 95% confidence interval [CI]: [0.34, 0.46], p < 0.0001). We controlled for all baseline differences in the outcomes analysis.

Table 3.

Baseline risk comparison with percentage of individuals answering ‘yes’ at baseline based on all available data for each risk variable.^a

	Both genders		Females		Males
	Treatment	Control	Treatment	Control	Treatment	Control
Ever sex
% Yes	23.7%	24.9%	20.9%	22.8%	26.4%	27.1%
Total N	1,592	1,905	775	979	817	926
Ever pregnant
% Yes	1.8%	1.2%	1.6%	1.2%	2.1%	1.2%
Total N	1,684	2,001	817	1,018	867	983
Recent sex
% Yes	16.4%	16.6%	15.4%	16.1%	17.3%	17.1%
Total N	1,627	1,952	787	1,007	840	945
Risky sex (full sample)
% Yes	9.6%	9.4%	10.1%	9.9%	9.1%	8.8%
Total N	1,614	1,932	781	999	833	933
Intend sex
% Yes	33.0%	33.6%	23.1%	24.2%	42.6%	43.7%
Total N	1,522	1,840	752	948	770	892
Intend risky sex
% Yes	7.0%	8.1%	8.8%	8.6%	5.7%	7.6%
Total N	885	1,106	362	464	523	642

Differences between treatment groups at baseline were assessed with a logistic mixed effects model with treatment and block modelled as fixed effects and school as random effects. There were no statistically significant differences (p < 0.05) observed between treatment and control groups at baseline.

Outcomes

At first follow-up (immediately post-programme) in the combined gender group, programme participants were less likely to report: ever having sex (OR: 0.72, 95% CI: [0.54,0.96]), having sex in the past 3 months (OR: 0.70, 95% CI: [0.57,0.87]), having sex without a condom in the past 3 months (OR: 0.71, 95% CI: [0.58,0.96]) or intention to have sex in the next year (OR: 0.81, 95% CI: [0.66,1.00]). No treatment effect was observed on intention to have sex and intention to not use a condom for the combined gender group at first follow-up. Treatment effect on pregnancy was not observed in any group at first follow-up. Female treatment youth at first follow-up were less likely to report recent sex (OR: 0.65, 95% CI: [0.47, 0.90]), and having recent sex without a condom (OR: 0.58, 95% CI: [0.40, 0.84]). No treatment effect was found at first follow-up for female youth for the outcomes of intention to have sex or intention to not use a condom. No significant treatment effects were found in males for first follow-up.

At second follow-up (1 year after programme completion), there were statistically significant protective effects of the treatment on the odds of reporting ever having been pregnant or having gotten someone pregnant in the combined gender group (OR: 0.55, 95% CI: [0.36, 0.85]) and among females (OR: 0.42, 95% CI: [0.23, 0.76]). No significant difference between treatment and control groups were found at second follow-up for the outcomes sex in the past 3 months, sex without a condom in the past 3 months, intention to have sex, and intention to not use a condom. We did find an effect of the treatment on ever having sex among males at second follow-up, but not at first follow-up. However, the magnitude of the effect of treatment on ever having sex among males was similar across time points (First follow-up OR: 0.71, 95% CI: [0.48, 1.06]; Second follow-up OR: 0.71, [0.51, 0.99]), suggesting some consistency over time. Overall, compared to the first follow-up, at second follow-up, the effects of the treatment were not as large, and most were not statistically significantly protective (Figure 2).

Figure 2.

Summary of the effects of the treatment on sexual behaviour outcomes at first and second follow-up.

Discussion

The results from our longitudinal follow-up of a cohort of Florida youth who participated in this RCT of an evidence-based PYD sexual health programme (TOP) revealed several notable findings. First, it was clear that protective behaviours were present in the treatment group at first follow-up, when compared to the control group. Specifically, young people in the treatment group were less likely to report having sex in the past 3 months, and those who did have sex in the past 3 months were more likely to use a condom. However, these protective effects related to the delay in sexual activity and increased condom use dissipate by the second follow-up point in this study, even among females; at second follow-up, there was not a statistically significant difference between treatment and control youth in these self-reported behaviours selected for measurement in this study.

As Francis et al. (2016) have highlighted, TOP’s evidence-based designation is attributable to a single RCT conducted in 1997 with high school students (n = 695) in 25 nationwide sites. Allen et al. (1997) reported a lower risks of school suspension, course failure, and pregnancy. Study participants received well over the prescribed 20 hours of CSL (mean 45.8 hours; SD = 44.6, median 35 hours of service). In contrast, the FL DOH TOP replication faced significant barriers in achieving fidelity for several programme components. For programme dosage, we found that 73% of classes in intervention schools achieved 25 lessons, although all fell short of the full 9-month time frame for the implementation of TOP (Daley et al., 2015). With regards to CSL, only 11% of classes were able to reach the 20-hour CSL benchmark, with an average of 14.9 CSL hours reported per class (Daley et al., 2015). Based on focus groups with TOP facilitators and observations of evaluation staff, barriers to achieving fidelity included not being able to implement for a full 9 months due to time needed for various testing and the difficulty of obtaining permission for off-site service activities. One effect of these barriers was that more of the CSL projects conducted by programme youth were ‘indirect’ service’, that is, students did not directly engage with any community member or population, but instead created items in the classroom (cards or craft items) or raised funds while on school grounds for the populations they hoped to serve. This lack of interaction between students and their project target population contrasts with direct service – typically more intensive and of higher quality, with an opportunity for young people to see the reception and outcome of their efforts – which early studies of TOP found to be more effective in contributing to the programme’s success (Allen et al., 1990, 1994).

Although Allen and colleagues implemented TOP with fidelity, like our study they also found that teenage pregnancy rates were reduced for adolescent females who participated in the programme though that same impact (causing pregnancy) was not observed for adolescent males (Allen et al., 1997). Other differences between that study and the one presented in this paper include setting (traditional school setting vs after school programming) and gender distribution (the Allen et al., 1997 study was overwhelmingly female [86% and 83.3% for treatment and control, respectively]). Investigators, administrators and policy makers weighing the use of TOP should consider challenges related to achieving fidelity in school-settings, as well as the impact of CSL on PYD.

However, the delay in sexual behaviour among youth in the treatment group may have had some protective impact, given the significantly lower reports of ever having been pregnant among females in the treatment group at second follow-up, as compared to females in the control group. We know that most young people will transition from never having had sex to ever having had sex at some point by the age of 17–19 years old, with almost one-third of young people having sex by the age of 15–16 years old (Finer and Philbin, 2013). Our findings show that the delay in having sex among females in the treatment group in Year 1 may have translated into a lower likelihood of having been pregnant in Year 2. Other benefits of that delay could also include lower likelihood of having a STI (Epstein et al., 2014; Reese et al., 2014), although that outcome was not measured in this study. Moreover, other studies have demonstrated additional benefits of delaying sexual intercourse among early adolescents. Youth who have their first sexual intercourse during early adolescence are at greater risk of experiencing depression and lowered self-esteem, exchanging sex for money and having lower odds of romantic relationship satisfaction in early adulthood (Armour and Haynie, 2007; Lara and Abdo, 2016; Reese et al., 2014).

Limitations and strengths

This study is not without its limitations. The data were collected among Florida youth in non-metropolitan counties, making generalisability to young people outside of Florida difficult. Nonetheless, Florida is the third largest US state and possesses a racially and ethnically diverse population allowing for the analysis of health outcomes among diverse groups.

The design of this study did not include randomisation to different levels of programme dosage or intensity (i.e. number of programme offered). Thus, we have limited clarity regarding the threshold of programme intensity required for longitudinal effects. In addition, the outcome data were self-reported by adolescents which – due to social desirability bias – may have underreported behaviours. Furthermore, counties were chosen if they had rates poorer than the state average for at least one of the following: (1) birth rate per female population aged 15–19 years, (2) repeat birth rate per female population aged 15–19 years, (3) combined Chlamydia and gonorrhoea rates per female population aged 15–19 years, (4) high school dropout rates, (5) graduation rates, and (6) out-of-school suspension rates. Counties where rates of these indicators were less than the state average may not experience the same risk reduction in outcomes as was demonstrated in our evaluation. Despite attempts to reach all the young people enrolled in the evaluation, many participants were lost to follow-up. To address this, we used MI to account for any bias that might exist in measuring treatment effect because of missing data (Schafer, 1997).

TOP was originally intended to be an afterschool programme (Allen and Philliber, 1991). For this evaluation, it was adapted to be implemented within the existing school structure. System factors such as time, resources and competing priorities led to the implementation fidelity issues that this evaluation experienced. An example of this was our inability to implement TOP for the recommended 9-month period due to the large number of testing days and holidays. Structural constraints also existed including limitations on resources the schools made available for implementation (e.g. classroom space) and restrictions on transportation to out-of-school activities. This caused CSL activities to be limited in terms of content and in the amount of time available to complete them. As a result of the difficulties we encountered completing the intended programme dosage, TOP may require additional adaptations to overcome the unique barriers of the school system.

Nonetheless, the study has strengths. It offers a good example of a well-designed and well-powered evaluation conducted with programme implementation at the community level. To date, it is among the largest randomised controlled trials of teenage pregnancy prevention programmes, and it is one of the few studies that examine programme effects longitudinally. Other longitudinal studies of sexual risk reduction among youth have reported similar diminished effects over time (Stanton et al., 2006); some suggest that booster sessions and the inclusion of an intervention component for parents may be an avenue for maintaining positive impacts longer post-intervention (Wang et al., 2014).

Based on the findings of this research, it is possible that PYD programmes such as the TOP, especially those targeting female youth, have the potential to result in decreased pregnancies from a delay in the onset of sexual behaviour and a decrease in sex without a condom. These recommendations are tempered by the findings of a recent systematic review of school-based pregnancy reduction programmes (Marseille et al., 2018), which found mixed results in several studies that could be considered PYD interventions, including previous evaluations of TOP – Allen et al. (1994, 1997). Because the positive impacts of TOP seemed to deteriorate over time, future research could aim to include longer follow-up periods to determine if these delays do indeed result in decreased pregnancies. Even if the immediate positive effects dissipate over time, the potential for delayed onset of pregnancy may have lifelong implications.

Footnotes

Acknowledgements

The authors thank the Florida Department of Health, and in particular Tiffane Evans and Shay Chapman, for their leadership and collaboration. We also thank the many evaluation data collectors, Teen Outreach Programme facilitators and the students who participated in this project.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was conducted under Grant Number TP1AH000017-01 from the US Office of Adolescent Health (OAH), US Department of Health and Human Services (HHS). The study described in this manuscript is registered on clinicaltrials.gov under the trial number NCT02519530.

Notes

ORCID iD

Helen Mahony

References

Alford

Hauser

(2011) Adolescent Sexual Health in Europe and the United States. Washington, DC: Advocates for Youth.

Allen

Philliber

(1991) Evaluating Why and How the Teen Outreach Program Works: Years 3–5 of the Teen Outreach National Replication (1986/87–1988/89). New York: Association of Junior Leagues International.

Allen

Kuperminc

Philliber

, et al. (1994) Programmatic prevention of adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in the Teen Outreach Program. American Journal of Community Psychology 22: 617–638.

Allen

Philliber

Hoggson

(1990) School-based prevention of teen-age pregnancy and school dropout: Process evaluation of the national replication of the Teen Outreach Program. American Journal of Community Psychology 18: 505–524.

Allen

Philliber

Herrling

, et al. (1997) Preventing teen pregnancy and academic failure: Experimental evaluation of a developmentally based approach. Child Development 68: 729–742.

Armour

Haynie

(2007) Adolescent sexual debut and later delinquency. Journal of Youth and Adolescence 36: 141–152.

Bandiera

Jeffries

IV Dodge

, et al. (2008) Regional differences in sexuality education on a state level: The case of Florida. Sex Education 8: 451–463.

Breslow

Clayton

(1993) Approximate inference in generalized linear mixed models. Journal of the American Statistical Association 88: 9–25.

Catalano

Berglund

Ryan

JAM

, et al. (2004) Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Prevention & Treatment 5: 15.

10.

Centers for Disease Control and Prevention (CDC) (2016) NCHHSTP AtlasPlus. HIV surveillance in adolescents and young adults. Available at: https://www.cdc.gov/nchhstp/atlas/slides/hiv-slideset.html

11.

Centers for Disease Control and Prevention (CDC) National Center for HIV, STD and TB Prevention (NCHSTP), Division of STD/HIV Prevention (2015) Sexually transmitted disease morbidity for selected STDs by age, race/ethnicity and gender 1996–2014, CDC WONDER Online Database. Available at: http://wonder.cdc.gov/std-v2011-race-age.html

12.

Daley

Buhi

Wang

, et al. (2015) Evaluation of Wyman’s Teen Outreach Program® in Florida: Final Impact Report for Florida Department of Health. Findings from the Replication of an Evidence-Based Teen Pregnancy Prevention Program. Washington, DC: Office of Adolescent Health.

13.

Epstein

Bailey

Manhart

, et al. (2014) Understanding the link between early sexual initiation and later sexually transmitted infection: Test and replication in two longitudinal studies. Journal of Adolescent Health 54: 435–441.e432.

14.

Finer

Philbin

(2013) Sexual initiation, contraceptive use, and pregnancy among young adolescents. Pediatrics 131: 886–891.

15.

Finer

Zolna

(2016) Declines in unintended pregnancy in the United States, 2008–2011. New England Journal of Medicine 374: 843–852.

16.

Florida Department of Education (2019) Statutes, policies & guidelines: State statutes. Available at: http://www.fldoe.org/schools/healthy-schools/sexual-edu/policies.stml

17.

Francis

Philliber

Walsh-Buhi

, et al. (2016) Scalability of an evidence-based adolescent pregnancy prevention program: New evidence from 5 cluster-randomized evaluations of the Teen Outreach Program. American Journal of Public Health 106: S32–S38.

18.

Gavin

Catalano

David-Ferdon

, et al. (2010) A review of positive youth development programs that promote adolescent sexual and reproductive health. Journal of Adolescent Health 46: S75–S91.

19.

Goesling

Colman

Trenholm

, et al. (2014) Programs to reduce teen pregnancy, sexually transmitted infections, and associated sexual risk behaviors: A systematic review. Journal of Adolescent Health 54: 499–507.

20.

Hoffman

Maynard

(2008) Kids Having Kids: Economic Costs and Social Consequences of Teen Pregnancy. Washington, DC: The Urban Institute.

21.

Kappeler

Farb

(2014) Historical context for the creation of the Office of Adolescent Health and the Teen Pregnancy Prevention Program. Journal of Adolescent Health 54: S3–S9.

22.

Kost

Henshaw

(2014) US Teenage Pregnancies, Births and Abortions, 2010: National and State Trends by Age, Race and Ethnicity. New York: Guttmacher Institute.

23.

Kost

Maddow-Zimet

Arpaia

(2017) Pregnancies, births and abortions among adolescents and young women in the United States, 2013: National and state trends by age, race and ethnicity. Available at: https://www.guttmacher.org/report/us-adolescent-pregnancy-trends-2013

24.

Lara

LAS

Abdo

CHN

(2016) Age at time of initial sexual intercourse and health of adolescent girls. Journal of Pediatric and Adolescent Gynecology 29: 417–423.

25.

Marseille

Mirzazadeh

Biggs

, et al. (2018) Effectiveness of school-based teen pregnancy prevention programs in the USA: A systematic review and meta-analysis. Prevention Science 19: 468–489.

26.

Kaye

(2012) Why It Matters: Teen Childbearing, Education, and Economic Wellbeing. Washington, DC: The National Campaign to Prevent Teen and Unplanned Pregnancy.

27.

Raudenbush

(1997) Statistical analysis and optimal design for cluster randomized trials. Psychological Methods 2: 173–185.

28.

Reese

Choukas-Bradley

Herring

, et al. (2014) Correlates of adolescent and young adult sexual initiation patterns. Perspectives on Sexual and Reproductive Health 46: 211–221.

29.

Roth

Brooks-Gunn

(2003) What exactly is a youth development program? Answers from research and practice. Applied Developmental Science 7: 94–111.

30.

Schafer

(1997) Analysis of Incomplete Multivariate Data. Boca Raton, FL: Chapman & Hall/CRC Press.

31.

Sedgh

Finer

Bankole

, et al. (2015) Adolescent pregnancy, birth, and abortion rates across countries: Levels and recent trends. Journal of Adolescent Health 56: 223–230.

32.

Shackleton

Jamal

Viner

, et al. (2016) School-based interventions going beyond health education to promote adolescent health: Systematic review of reviews. Journal of Adolescent Health 58: 382–396.

33.

Spybrook

Raudenbush

Congdon

, et al. (2011) Optimal Design Software for Multi-Level and Longitudinal Research (Version 3.01) [Software]. Ann Arbor, MI: University of Michigan Press.

34.

Stanton

Harris

Cottrell

, et al. (2006) Trial of an urban adolescent sexual risk-reduction intervention for rural youth: A promising but imperfect fit. Journal of Adolescent Health 38: 55.

35.

Teitler

Weiss

(2000) Effects of neighborhood and school environments on transitions to first sexual intercourse. Sociology of Education 73: 112–132.

36.

Van Buuren

(2007) Multiple imputation of discrete and continuous data by fully conditional specification. Statistical Methods in Medical Research 16: 219–242.

37.

Walsh-Buhi

Marhekfa

Wang

, et al. (2016) The impact of the Teen Outreach Program on sexual intentions and behaviors. Journal of Adolescent Health 59: 283–290.

38.

Wang

Stanton

Deveaux

, et al. (2014) The impact of parent involvement in an effective adolescent risk reduction intervention on sexual risk communication and adolescent outcomes. AIDS Education & Prevention 26: 500–520.

39.

Wilson

Deane

Marshall

, et al. (2008) Reducing adolescents’ perceived barriers to treatment and increasing help-seeking intentions: Effects of classroom presentations by general practitioners. Journal of Youth and Adolescence 37: 1257–1269.

40.

Wyman Center (2010) Following the TOP Approach: Wyman’s Teen Outreach Program Facilitator Training Participant’s Guide. Eureka, MO: Wyman Center, Inc.

41.

Wyman Center (2016) Wyman’s Teen Outreach Program (TOP). Available at: http://teenoutreachprogram.com/