Abstract
BACKGROUND
Sexual Assault among Adolescents and College Students
Sexual assault is a significant problem among adolescents and college students in the United States. Findings from the Campus Sexual Assault study estimated that 15.9% of college women had experienced attempted or completed sexual assault (i.e., unwanted sexual contact that could include sexual touching, oral sex, intercourse, anal sex, or penetration with a finger or object) prior to entering college and 19% had experienced attempted or completed sexual assault since entering college (Krebs, Lindquist, Warner, Fisher, & Martin, 2009). These rates are problematic, as sexual assault in adolescence and/or young adulthood is associated with numerous adverse outcomes, including repeated victimization, depressive symptomology, heavy drinking, and suicidal ideation (Cortens, Eckenrode, & Rothman, 2013; Cui, Ueno, Gordon, & Fincham, 2013; Halpern, Spriggs, Martin, & Kupper, 2009). Importantly, there is evidence indicating experiences of sexual assault during these two life phases are related, as victimization and perpetration during adolescence are, respectively, associated with increased risk of victimization and perpetration during young adulthood (Cui, Ueno, Gordon, & Fincham, 2013). Thus, early prevention efforts are of paramount importance.
Reviews of research on the effectiveness of programs designed to prevent sexual assault among adolescents and college students have noted both a dearth of high-quality studies, such as randomized controlled trials (RCTs), and minimal evidence that these prevention programs have meaningful effects on young people's behavior (DeGue, Valle, Holt, Massetti, Matjasko, & Tharp, 2014; De Koker, Mathews, Zuch, Bastien, & Mason-Jones, 2014). Concerning the latter point, evaluations of such programs tend to measure attitudinal outcomes (e.g., rape supportive attitudes, rape myth acceptance) more frequently than behavioral outcomes (e.g., perpetration or victimization) (Anderson & Whiston, 2005; Cornelius & Resseguie, 2007; DeGue et al., 2014). Additionally, findings from a meta-analysis of studies assessing outcomes of college sexual assault prevention programs suggested that effects are larger for attitudinal outcomes than for the actual incidence of sexual assault (Anderson & Whiston, 2005).
The Bystander Approach
Given this paucity of evidence of behavior change, it is imperative that researchers identify effective strategies for preventing sexual assault among adolescents and young adults. One promising strategy is the implementation of bystander programs, which encourage young people to intervene when witnessing incidents or warning signs of sexual assault. The strength of the bystander model lies in its emphasis on the role of peers in the prevention of violence. Peers are a salient influence on young people's intimate relationships (Adelman & Kil, 2007; Giordano, 2003). In some respects, this can be detrimental, as having friends involved in violent intimate relationships (i.e., characterized by sexual or physical violence) is a risk factor for becoming both a perpetrator and victim of violence (Arriaga & Foshee, 2004; Foshee, Benefield, Ennett, Bauman, & Suchindran, 2004; Foshee, Linder, MacDougall, & Bangdiwala, 2001; Foshee, Reyes, & Ennett, 2010; McCauley et al., 2013). However, peers can also have a positive impact on intimate relationships.
Young victims and perpetrators of violence are often reluctant to divulge their experience or to seek help (especially from adults), but when they do seek help they often seek it from their peers (Ashley & Foshee, 2005; Black, Tolman, Callahan, Saunders, & Weisz, 2008; Molidor & Tolman, 1998; Weisz, Tolman, Callahan, Saunders, & Black, 2007). This indicates that victims trust their peers to provide a valuable source of support after an assault has occurred. Importantly, it also suggests that peers have the potential to play a pivotal role in the prevention of sexual assault by intervening when they witness its warning signs. In fact, in a contemporary “hookup culture” adolescents and young adults are more likely to meet and socialize in groups than they are to date in pairs and, thus, warning signs of assault are frequently exhibited in communal spaces (Bogle, 2007; 2008; Molidor & Tolman, 1998). Thus, the social nature of intimate relationships during these life stages can make peers pivotal actors in the prevention of sexual assault.
However, the potential for peer intervention can be undermined by a general “bystander effect” that diffuses responsibility for action in group settings (Darley & Latane, 1968). To intervene as a witness to sexual assault, individuals must notice the event (or its warning signs), define the event as warranting action/intervention, take responsibility for acting (i.e., feel a sense of personal duty), and demonstrate a sufficient level of self-efficacy (i.e., perceived competence to successfully intervene) (Latane & Darley, 1969). Studies have indicated that, as witnesses to sexual assault, young people often fail to meet these criteria (Banyard, 2008; Bennett, Banyard, & Garnhart, 2014; Burn, 2009; Casey & Ohler, 2012; Exner & Cummings, 2011; McCauley et al., 2013; McMahon, 2010; Noonan & Charles, 2009), with males being less likely than females to intervene (Banyard, 2008; Burn, 2009; Edwards, Rodenhizer-Stampfli, & Eckstein, 2015; Exner & Cummings, 2011; McMahon, 2010).
Thus, bystander programs seek to sensitize young people to warning signs of sexual assault, create attitudinal change that fosters bystander responsibility for intervening (e.g., creating empathy for victims), and build requisite skills/tactics for taking action (Banyard, 2011; Banyard, Plante, & Moynihan, 2004; Burn, 2009; McMahon & Banyard, 2012). Many of these programs are implemented with large groups of adolescents or college students in the format of a single training/education session (e.g., as part of college orientation). However, some programs use broader implementation strategies, such as advertising campaigns where signs are posted across college campuses to encourage students to act when witnessing signs of violence.
How the Intervention Might Work
By treating young people as potential allies in preventing sexual assault, bystander programs have the potential to be less threatening than traditional sexual assault prevention programs, which tend to approach young people as either potential perpetrators or victims of sexual violence (Burn, 2009; Messner, 2015; [Jackson] Katz, 1995). Instead of placing emphasis on how young people may modify their individual behavior to either respect the sexual boundaries of others or reduce their personal risk for being sexually assaulted, bystander programs aim to foster prerequisite knowledge and skills for intervening on behalf of victims. Thus, by treating young people as part of the solution to sexual assault, rather than part of the problem, bystander programs limit the risk of defensiveness or backlash among participants (e.g., decreased empathy for victims, increased rape myth acceptance) (Banyard et al., 2004; Katz, 1995).
As outlined by Burn (2009) bystander programs are designed to promote the following prerequisites for intervention: noticing an event, identifying a situation as warranting intervention, taking responsibility for acting, and deciding how to help. This often involves educating young people about what constitutes sexual assault, portraying victims as worthy of assistance, and building skills necessary to intervene (e.g., providing strategies for what to do and say). Although most bystander programs share the common goal of promoting such prerequisites for intervention, their specific program content and framing of sexual assault varies.
Research has indicated that, relative to males, females are overwhelmingly the victims of sexual assault (Foshee, 1996; Gressard, Swahn, & Tharp, 2015; Harned, 2001; Howard, Wang, & Yan, 2007). Thus, the earliest bystander programs tended to apply a gendered perspective to the prevention of sexual assault among adolescents and college students. For example, Katz (1995) developed the Mentors in Violence Prevention Program (MVP) with the goal of inspiring male college athletes to challenge sociocultural definitions of masculinity that equate men's strength with dominance over women. At the time of its inception, MVP was unique in its explicit focus on masculinity as well as its nonthreatening “bystander” approach that encouraged young men to intervene when witnessing acts (or warning signs) of violence against women. As Katz explained, MVP reduces young men's defensiveness to violence prevention efforts by focusing on men as potential bystanders to violence, rather than potential perpetrators of violence. In addition to reducing men's defensiveness to intervention efforts, this bystander approach emphasizes the point that “when men don't speak up or take action in the face of other men's abusive behavior toward women, that constitutes implicit consent of such behavior” (Katz, 1995, p. 168).
Since the inception of MVP a number of programs have emerged to address barriers to bystander intervention among adolescents and college students. While they all share the common goal of inspiring bystanders to act in ways that prevent sexual assault, these programs exhibit a great deal of variation in scope pertaining to their target bystander populations (i.e., males and/or females, secondary school or college students), sex of victims, and gendered versus gender-neutral approach. For example, some programs use a gendered approach by (1) critiquing gender norms that can promote violence against women and (2) encouraging males to intervene on behalf of female victims (e.g., MVP, see Katz, 1995). Others use a gender-neutral approach to build a sense of community responsibility to intervene on behalf of both male and female victims of sexual assault (e.g., Bringing in the Bystander, see Banyard, Moynihan, & Crossman, 2009; Banyard, Moynihan, & Plante, 2007). One of the major differences between gendered and gender-neutral bystander programs is that the former places socio-cultural forces, such as gender norms, at the center of discussions of violence whereas the latter places the bystander, and his/her individual cognitive processes when encountering violence, at the center of discussions of violence ([Jackson] Katz, Hesiterkamp, & Fleming, 2011; Messner, 2015).
Comparing the effects of gendered and gender-neutral programs has the potential to identify important determinants of the success of bystander programs. Although there is no empirical examination of the different effects of these programs, there are theoretical reasons to believe that each has the potential to be successful under certain conditions. Specifically, gendered approaches to bystander education programs may be better suited to target socio-cultural facilitators of sexual assault against women and address different patterns of bystander behaviors exhibited by males and females (Banyard, 2008; Burn, 2009; Exner & Cummings, 2011; Katz et al., 2011; [Jennifer] Katz, 2015; [Jennifer] Katz, Colbert, & Colangelo, 2015; McCauley et al., 2013; McMahon, 2010; Messner, 2015). On the other hand, gender-neutral programs have the benefit of deflecting the criticism that prevention programs utilizing a gendered approach are inherently anti-male (Katz et al., 2011; Messner, 2015). Avoidance of such criticism is paramount to the success of sexual assault prevention programs. This is because adolescents and young adults who are coming of age in a “post-feminist” era are likely to reject gendered explanations of sexual assault and, instead, respond more positively to gender-neutral programs that use inclusive language that can be applied to a broad range of victims and perpetrators (Barreto & Ellemers, 2005; Kettrey, 2016; Swim, Aikin, Hall, & Hunter, 1995).
Why it is Important to do the Review
Illustrating the perceived benefits of bystander programs, the 2013 Campus Sexual Violence Elimination (SaVE) Act requires post-secondary educational institutions participating in Title IX financial aid programs to provide incoming college students with primary prevention and awareness programs addressing sexual violence. The Campus SaVE Act mandates that these programs include a component on bystander intervention. Currently, there is no comparable legislation regarding sexual assault among adolescents (e.g., mandating bystander programs in secondary schools). This is an unfortunate oversight, as adolescents who experience sexual assault are at an increased risk of repeated victimization in young adulthood (Cui et al., 2013). Thus, the implementation of bystander programs in secondary schools not only has the potential to reduce sexual assault among adolescents, but may also have the long-term potential to reduce sexual assault on college campuses.
Findings from this systematic review will provide valuable evidence of the extent to which bystander programs, as mandated by the Campus SaVE Act, are effective in preventing sexual assault among college students. Additionally, by examining effects of these programs among adolescents, this review will provide educators and policy makers with information for determining whether such programs should be widely implemented in secondary schools.
Currently, there are no Campbell or Cochrane Collaboration Reviews evaluating the effects of bystander programs on sexual assault among adolescents and/or college students. Of modest relevance to the proposed review, the Campbell and Cochrane Collaboration libraries include meta-analyses of the effects of more general programs (not bystander programs) designed to prevent or reduce relationship/dating violence among adolescents and/or young adults (De La Rue, Polanin, Espelage, & Pigott, 2014; Fellmeth, Heffernan, Nurse, Habibula, & Sethi, 2013). Both of these reviews reported violence outcomes as aggregate measures that do not distinguish sexual violence from other forms of violence. Although they each found some evidence of significant effects on knowledge or attitudes pertinent to violence, neither found evidence of significant effects on young people's behavior (i.e., rates of perpetration or victimization).
Two reviews published outside of the Campbell and Cochrane Collaboration libraries are of closer relevance to this review. These include a meta-analysis of the effects of bystander programs on sexual assault on college campuses (Katz & Moore, 2013) and a narrative review of studies examining the effects of bystander programs on dating and sexual violence among adolescents and young adults (Storer, Casey, & Herrenkohl, 2015).
In what they called an “initial” meta-analysis of experimental and quasi-experimental studies (published through 2011), Katz & Moore (2013) found moderate effects of bystander programs on participants’ self-efficacy and intentions to intervene, and small (but significant) effects on bystander behavior, rape-supportive attitudes, and rape proclivity (but not perpetration). Effects were generally stronger among younger samples and samples containing a higher percentage of males. The stronger effect for younger participants (i.e., younger college students) suggests such programs may be particularly effective with adolescents.
In a narrative review of studies examining the effects of bystander programs on dating violence and sexual assault among adolescents and young adults, Storer et al. (2015) highlighted positive effects on bystander self-efficacy and intentions, but noted less evidence of positive effects on actual bystander behavior or of effects on perpetration of violence.
While informative, each of these reviews has limitations. Katz & Moore's (2013) meta-analysis focused exclusively on sexual assault on college campuses and did not examine effects of such programs among adolescents. Although Storer et al. (2015) focused on studies examining violence among both adolescents and young adults, their sample was limited in that it was exclusively comprised of peer-reviewed articles (i.e., the sample explicitly excluded theses, dissertations, and other gray literature) and the authors specified no research design criteria for inclusion (i.e., the sample included low-quality studies such as those utilizing single group pre- and post-test designs). Importantly, Storer et al. reported no meta-analytic findings. Thus, to our knowledge there are currently no existing meta-analyses examining the effects of bystander programs on attitudes and behaviors regarding sexual assault among both college students and adolescents. Additionally, Katz & Moore's early meta-analysis only included studies published/reported through 2o11 (two years prior to the 2013 Campus SaVE Act) and did not evaluate program content as a moderator.
Our review will examine the effects of bystander programs on attitudes (i.e., perceptions of violence/victims, self-efficacy to intervene, intentions to intervene) and behaviors (i.e., actual intervention behavior, perpetration) regarding sexual assault among adolescents and college students. Importantly, it will present meta-analytic findings to quantitatively assess the influence of moderators (e.g., gender composition of sample, mean age of sample, education level of sample, single- or mixed- sex implementation, gendered content of program, fraternity/sorority membership, athletic team membership) on the effects of bystander programs.
Given that the 2013 Campus SaVE Act mandates bystander education as a component of primary prevention of sexual assault on college campuses, findings will provide vital assessment of legislation outcomes. Specifically, findings will (1) provide an updated analysis of the effects of bystander programs, (2) examine previously unanalyzed moderators of the effectiveness of bystander programs (e.g., content), and (3) evaluate the effects of bystander programs among adolescents. This latter contribution has the potential to inform directions for future policy and/or legislation pertinent to prevention efforts in secondary schools. This is of vital importance in breaking patterns of sexual assault, considering that victimization and perpetration in adolescence are associated with victimization and perpetration in young adulthood (Cui et al., 2013).
OBJECTIVES
The overall objective of this systematic review and meta-analysis is to examine what effects bystander programs have on preventing sexual assault among adolescents and college students. More specifically, and given the study designs that will be included, this review will address three objectives:
The first objective is to assess the overall effects (including adverse effects), and the variability of the effects, of bystander programs on adolescents’ and college students’ attitudes and behaviors regarding sexual assault. This includes general attitudes toward violence and victims, prerequisite skills and knowledge to intervene, self-efficacy to intervene, intentions/willingness to intervene when witnessing signs of sexual assault, actual intervention behavior, and perpetration of sexual assault. This information will allow practitioners and policy makers to determine whether bystander programs, which are mandated by the Campus SaVE Act, are effective in preventing sexual assault among adolescents and college students. Furthermore, the examination of adverse effects will provide decision-makers with useful information about any backlash effects of bystander programs (e.g., decreased victim empathy or increased rape myth acceptance). This is important, as the non-threatening approach of bystander programs is believed to present minimal risk of backlash effects (Banyard et al., 2004; Katz, 1995).
The second objective is to explore the comparative effectiveness of bystander programs for different profiles of participants (e.g., mean age of the sample, education level of the sample, proportion of males/females in the sample, proportion of fraternity/sorority members in the sample, proportion of athletic team members in the sample). This information will allow practitioners and policy makers to determine which groups of young people might benefit most from bystander programs.
The third objective is to explore the comparative effectiveness of different bystander programs in terms of gendered content and approach (e.g., conceptualizing sexual assault as a gendered or gender-neutral problem, mixed- or single-sex group implementation). This information will allow practitioners and policy makers to identify characteristics of the most effective bystander programs.
METHODOLOGY
Criteria for including and excluding studies
Types of study designs
To be eligible for inclusion in the review, studies must use an experimental or controlled quasi-experimental research design to compare an intervention group (e.g., students assigned to complete a bystander program) with a comparison group (e.g., students not assigned to complete a bystander program). We limit our review to such studies because these typically present lower risk of bias relative to other research designs (e.g., natural experiments, single group designs). More specifically, we plan to include the following designs: Randomized controlled trials: Studies in which individuals, classrooms, schools, or other groups are randomly assigned to intervention and comparison conditions. Quasi-randomized controlled trials: Studies where assignment to conditions is quasi-random, for example, by birth date, date of week, student identification number, month, or some other alternation method. Controlled quasi-experimental designs: Studies where participants are not assigned to conditions randomly or quasi-randomly (e.g., participants self-select into groups). Given the potential selection biases inherent in these controlled quasi-experimental design, we will only include those that also meet one of the following criteria: . Regression discontinuity designs: Studies that use a cutoff on a forcing variable to assign participants to intervention and comparison groups, and assess program impacts around the cutoff of the forcing variable. . Studies that use propensity score or other matching procedures to create a matched sample of participants in the intervention and comparison groups. To be eligible for inclusion, these studies must also provide enough statistical information to permit estimation of pretest effect sizes for the matched groups. . For studies where participants in the intervention and comparison groups are not matched, enough statistical information must be reported that will permit us to estimate pretest effect sizes for at least one outcome measure.
Consistent with Campbell Collaboration policies and procedures, studies using experimental and quasi-experimental research designs will be synthesized separately in the meta-analyses. This is because experimental studies present the lowest risk of bias (Higgins & Green 2011). Furthermore, we plan to collect extensive data on the risk of bias and study quality of all eligible studies, which we will attend to carefully when interpreting the findings from the systematic review and meta-analyses (described in greater detail below).
Types of participants
The review will focus on studies that examine outcomes of bystander programs that target sexual assault and are implemented with adolescents and/or college students in educational settings. Eligible participants include adolescents enrolled in grades 7 through 12 and college students enrolled in any type of undergraduate postsecondary educational institution.
Eligible participant populations will include studies that report on general samples of adolescents and/or college students as well as studies using specialized samples such as those primarily consisting of college athletes, fraternity/sorority members, and single-sex samples. Study samples primarily consisting of post-graduate students will be ineligible for inclusion; the mean age of samples may be no greater then 25 to be included in the review.
Types of interventions
Eligible intervention programs are those that approach participants as allies in preventing and/or alleviating sexual assault among adolescents and/or college students. Some part of the program must focus on ways that cultivate a willingness for a person to respond to others who are at risk for sexual assault. All delivery formats are eligible for inclusion (e.g., in-person training sessions, video programs, web-based training, advertising/poster campaigns). There are no treatment duration criteria for inclusion.
Studies that report bystander outcomes but do not meet the aforementioned intervention criteria are not eligible for inclusion. Additionally, studies that assess outcomes of programs that aim to facilitate pro-social bystander behavior, but that do not explicitly include a component addressing sexual assault (e.g., programs to prevent bullying) are not eligible.
Eligible comparison groups must receive no intervention services targeting bystander attitudes/behavior or sexual assault. Thus, treatment-treatment studies that compare individuals assigned to a bystander program with individuals assigned to a general sexual assault prevention program are not eligible for inclusion. However, eligible comparison groups may receive a sham or attention treatment that is expected to have no effect on bystander outcomes or attitudes/behaviors regarding sexual assault.
Types of outcome measures
The review will include studies that measure the effects of bystander programs on at least one of the following primary outcome domains: General attitudes toward sexual assault and victims (e.g., victim empathy, rape myth acceptance). Prerequisite skills and knowledge for bystander intervention as defined by Burn (2009) (e.g., noticing sexual assault or its warning signs, identifying a situation as appropriate for intervention, taking responsibility for acting/intervening, knowing strategies for helping/intervening). Self-efficacy with regard to bystander intervention (e.g., respondents’ confidence in their ability to intervene). Intentions to intervene when witnessing instances or warning signs of sexual assault. Actual intervention behavior when witnessing instances or warning signs of sexual assault. Perpetration of sexual assault (i.e., rates of perpetration among individuals assigned to the treatment or control group of a study).
Any outcome falling in these domains will be eligible for inclusion. This includes outcomes measured with any form of assessment (e.g., self-report, official/administrative report, observation, etc.) that can be summarized by any type of quantitative score (e.g., percentage, continuous variable, count variable, categorical variable, etc.). In the event that a particular study includes multiple measures of a single construct (e.g., two different scales measuring rape myth acceptance) we will code each eligible outcome and synthesize these using the methods described later in this protocol (see “Criteria for determination of independent findings” below).
Depending on directionality, these outcomes capture both intended and adverse effects (e.g., increases or decreases in victim empathy, pro-social bystander behavior, etc.) that are important to adolescents, college students, and decision-makers alike.
Duration of follow-up
Studies reporting follow-ups of any duration are eligible for inclusion. When studies report more than one follow-up wave, each wave will be coded and identified by its reported duration. As described in more detail below, follow-ups of similar durations will be analyzed together.
Types of settings
The review will focus on studies that examine outcomes of bystander programs that target sexual assault and are implemented with adolescents and/or college students in educational settings. Eligible educational settings include secondary schools (i.e., grades 7-12) and colleges or universities. Studies that assess bystander programs implemented with adolescents and young adults outside of educational institutions (e.g., in community settings, military settings) are ineligible for inclusion in the review.
Search strategy
We will identify candidate studies through searches of electronic databases, relevant academic journals, and gray literature sources. We will also contact leading authors and experts on bystander programs to identify any current/ongoing research that might be eligible for the review. Additionally, we will screen the bibliographies of eligible studies and relevant reviews to identify additional candidate studies. We will conduct forward citation searches (searches for reports citing eligible studies) using the website Google Scholar, as this database produces similar results to other search engines (e.g., Web of Science; Tanner-Smith & Polanin, 2015) and is also more likely to locate existing grey literature.
Once candidate studies have been identified each will be entered into the meta-analysis database as a separate record. Two reviewers will then independently screen each study title and abstract and record their eligibility recommendation (i.e., ineligible or eligible for full-text screening) into the pertinent database record. Disagreements between reviewers will be resolved by consensus and the final abstract screening decision will be recorded in the database. Potentially eligible studies will then be retrieved in full text and these full texts will be reviewed for eligibility, again using two independent reviewers who will record their eligibility recommendation (and, when applicable, rationale for an ineligibility recommendation) into the pertinent database record. Disagreements between reviewers will again be resolved via discussion and consensus and the final eligibility decision will be recorded in the database. If we cannot determine eligibility due to missing information in a report, we will contact study authors for this information.
Throughout the search and screening process we will maintain a document that includes the number of unique candidate studies identified through various sources (e.g., electronic database searches, academic journal searches, and gray literature searches). We will use the information in this record to create a PRISMA flow chart that reports the screening process (Moher et al., 2015). Additionally, we will use the final screening decisions recorded in the meta-analysis database to create a table that lists studies excluded during the full-text screening phase along with the rationale for each exclusion decision.
Electronic Databases
The prevention of sexual assault among college students and adolescents is a topic that spans multiple disciplines (e.g., sociology, psychology, education, public health). Thus, we will search a variety of databases that are relevant to these fields. Search terms will vary by database, but will generally include two blocks of terms and appropriate Boolean or proximity operators, if allowed: blocks will include terms that address the intervention and outcomes. The following are electronic databases (hosts) we plan to search: Cochrane Central Register of Controlled Trials (CENTRAL) Cochrane Database of Abstracts of Reviews of Effects (DARE) Education Resources Information Center (ERIC, via ProQuest) Education Database (via ProQuest) International Bibliography of the Social Sciences (IBSS, via ProQuest) PsycINFO (via ProQuest) PsychARTICLES (via Proquest) PubMed Social Services Abstracts (via Proquest) Sociological Abstracts (via Proquest)
The strategy for searching electronic databases will involve the use of search terms specific to the types of interventions and outcomes eligible for inclusion. Search terms for types of interventions will include general terms for bystander programs as well as names of specific bystander programs (e.g., Mentors in Violence Prevention). Search terms for types of outcomes will include terms that are specific to measures of sexual violence (e.g., sexual assault) as well as more general terms that have the potential to identify studies that measure physical and/or sexual violence. Due to the overwhelming focus of bystander programs on adolescents and college students (aside from a few implementations with military samples) search terms will not limit initial results by the age or general characteristics of the target population.
The search terms and strategy for PsycINFO via ProQuest are as follows (to be modified for other databases):
(AB,TI(“bystander”)) AND (AB,TI(“education” OR “program” OR “training” OR “intervention” OR “behavior” OR “attitude” OR “intention” OR “efficacy” OR “prosocial” OR “pro-social” OR “empowered” OR “Bringing in the Bystander” OR “Green Dot” OR “Step Up” OR “Mentors in Violence Prevention” OR “MVP” OR “Know Your Power” OR “Hollaback” OR “Circle of 6” OR “That's Not Cool” OR “Red Flag Campaign” OR “Where Do You Stand” OR “White Ribbon Campaign” OR “Men Can Stop Rape” OR “The Men's Program” OR “The Women's Program” OR “The Men's Project” OR “Coaching Boys into Men” OR “Campus Violence Prevention Program” OR “Real Men Respect” OR “Speak Up Speak Out” OR “How to Help a Sexual Assault Survivor” OR “Inter ACT”)) AND (AB,TI(“sexual assault” OR “rape” OR “violence” OR “victimization”))
Searches of Academic Journals
We will search the tables of contents of current issues of journals where a large proportion of eligible studies are published. Preliminary searches of the literature suggest the following journals may be promising sources of eligible studies: Journal of Adolescent Health, Journal of Community Psychology, Journal of Family Violence, Journal of Interpersonal Violence, Psychology of Violence, Violence Against Women, and Violence and Victims.
Gray Literature Sources
We will conduct gray literature searches to identify unpublished studies that meet inclusion criteria. This will include searching electronic databases that catalog dissertations and theses, searching conference proceedings, and searching websites with content relevant to sexual assault and/or violence against women. The following are gray literature sources we plan to search: ProQuest Dissertations and Theses Global Clinical Trials Register (https://clinicaltrials.gov) End Violence Against Women International - conference proceedings (http://www.evawintl.org/conferences.aspx) National Sexual Violence Resource Center website (nsvrc.org) National Online Resource Center on Violence Against Women website (VAWnet.org) Us Department of Justice Office on Violence Against Women website (www.justice.gov/ovw) Center for Changing our Campus Culture website (www.changingourcampusculture.org)
Description of methods used in primary research
Based on preliminary searches of the literature, we anticipate that most included studies will employ randomized control trial (RCT) designs in which participants are randomly assigned to an intervention or comparison group – either individually or in clusters. Examples of clusters might include classrooms, groups of incoming college students attending orientation, fraternities/sororities, athletic teams, etc.
In a well-cited example of an RCT with individual-level assignment Banyard, Moynihan, and Plante (2007) recruited undergraduates through flyers posted around campus. They then randomly assigned participants to one of three groups: (1) a no-treatment control that only completed pre- and post- intervention surveys, (2) a treatment group that was assigned to complete one 90-minute bystander program session, or (3) a treatment group that was assigned to complete three 90-minute bystander program sessions. Note that, for the purposes of this review, we will handle such multiple-arm studies by contrasting each treatment group with the comparison group in a series of sensitivity analyses (described in greater detail below). That is, we do not plan to compare multiple treatment groups to one another.
In an example of an RCT with cluster assignment, Miller et al. (2012) randomly assigned 16 high schools to either (1) a treatment condition in which coaches were trained to lead athletes in eleven brief (10-15 minute) discussions on respect and violence prevention or (2) a comparison condition in which coaches were instructed to interact with athletes in their usual fashion.
Although we expect most included studies to be RCTs, we also anticipate that some studies conducted after the institution of the 2013 Campus SaVE Act will use controlled quasi-experimental designs. The Campus SaVE Act mandates that all post-secondary educational institutions participating in Title IV financial aid programs implement primary prevention/awareness programs that address sexual violence (and that they include a bystander component). Thus, we suspect that many studies conducted since 2013 will summarize findings from routine programs (e.g., programs incorporated into college orientation for incoming students) rather than programs implemented strictly for research purposes. Due to both logistical and ethical issues, these routine programs are unlikely to involve randomization of participants. Instead, we expect many of these studies will involve non-random assignment of participants, but report will pre-intervention measures of outcomes that may be used to measure equivalence between the treatment and comparison groups.
Criteria for determination of independent findings
Throughout the screening and coding processes, we will flag and set aside reports for determination of independence if they meet one or more of the following criteria: They share one or more common author with another report. They summarize findings from similar datasets (e.g., data collected on the same college campus). They report the same grant number or funding source.
If we determine that the results of two or more reports are related, we will assign the reports the same study number and will code them concurrently as a single “study.” If it is unclear whether two or more reports are independent we will contact the authors for clarification.
Within each independent study, we also anticipate that most studies will report on multiple outcome measures at multiple follow-up points. We will group coded effect sizes together according to the construct that they represent (i.e., attitudes toward sexual assault and victims, prerequisite skills and knowledge to intervene, self-efficacy to intervene, intentions to intervene, actual intervention behavior, perpetration of sexual assault) and the follow-up assessment period (e.g., 0-3 months, 3-6 months, and 6+ month follow-up). All quantitative syntheses (meta-analyses) will then synthesize findings separately by outcome construct and follow-up time points, to ensure the statistical independence of the effect sizes included in any given analysis.
We will handle within-study effect size dependency (e.g., more than one effect size reported per outcome construct/follow-up time point within a single study) differently depending on how many studies are eligible for the review. Specifically, robust variance estimation, a method for handling dependent effect sizes, is only appropriate in cases with larger samples. We use a cut-off sample size of 10, as recommended by Tanner-Smith and Tipton (2014) for determining the appropriateness of robust variance estimation. In the event that there are less than 10 studies that report multiple outcomes within the same construct category (e.g., two measures of rape myth acceptance within each study), we will only include one outcome per study for that construct. We will select the most similar outcomes for synthesis within a construct category. In the event that there are ten or more studies that report multiple outcomes within the same construct we will use robust variance estimation to handle statistical dependencies in the synthesis (Hedges, Tipton, & Johnson, 2010; Tanner-Smith & Tipton, 2014; Tipton, 2013).
Details of study coding categories
Two reviewers will independently double-code all included studies, using a piloted codebook (see Appendix for a draft codebook). Coding disagreements will be resolved via discussion and consensus. If data needed to calculate an effect size are missing from a report, we will contact the primary study authors for this information.
The primary categories for coding are as follows: participant demographics and characteristics (e.g., age, gender, education level, race/ethnicity, athletic team membership, fraternity/sorority membership); intervention setting (e.g., state, country, secondary or post-secondary institution, mixed- or single-sex group); study characteristics (e.g., attrition, duration of follow-up, study design, participant dose, sample N); outcome construct (e.g., type, description of measure); and outcome results (e.g., timing at measurement, baseline and follow-up means and standard deviations or proportions).
We will assess risk of bias in included studies using the Cochrane risk of bias tools for randomized studies (Higgins & Green, 2011) and non-randomized studies (Sterne et al., 2016). The Cochrane risk of bias tool for randomized studies assesses risk of bias in the following domains: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other bias. These domains are each rated as low, unclear, or high risk of bias.
The Cochrane risk of bias tool for non-randomized studies, ROBINS-I, requires that at the protocol stage of the review, two sets of items are determined: (1) confounding areas that are expected to be relevant to all or most studies in the review and (2) co-interventions that could be different between intervention groups with the potential to differentially impact outcomes. For this review, we anticipate the following confounding factors, which will be coded in the risk of bias assessments: gender, fraternity/sorority membership, athletic team membership, pre-intervention attitudes (e.g., victim empathy, rape myth acceptance), pre-intervention bystander measures (e.g., efficacy, intentions, behavior), and prior sexual assault victimization. We anticipate the following co-interventions to have a potential impact on outcomes, so they will also be coded in the risk of bias assessments: general sexual assault prevention programs, dating violence prevention programs, and general bystander programs (not explicitly targeting sexual violence).
Statistical procedures and conventions
Effect Sizes. Outcomes for included studies will likely be reported as both binary measures (e.g., proportion of participants who report bystander intervention behavior) and continuous measures (e.g., scores on rape myth acceptance scales). For binary measures, we will report effect sizes in the form of odds ratios (OR). For continuous measures, we will report effect sizes as standardized mean differences (SMD), adjusted for small sample size (Hedges’ g). We will code OR effect sizes such that positive values (i.e., greater than 1) indicate greater odds of a beneficial outcome for the treatment group (relative to the comparison group), whereas negative values (i.e., less than 1) indicate lower odds of success for the treatment group (i.e., an adverse effect). We will code SMD effect sizes such that positive values (i.e., greater than 0) indicate a beneficial outcome for the treatment group. In the event that similar outcome constructs are represented as both binary and continuous measures across or within studies (e.g., proportion of respondents who report bystander intervention behavior, mean number of bystander intervention behaviors reported) we will assume that these measures reflect different underlying constructs. Thus, we will synthesize the OR and SMD effect sizes separately.
Additionally, to minimize any potential bias in the meta-analysis results due to effect size outliers, prior to conducting the meta-analysis, we will Winsorize effect sizes or study sample sizes that lie three or more standard deviations from the mean (Lipsey & Wilson, 2001).
Model. We will conduct meta-analyses using random-effects inverse variance weights and will report 95% confidence intervals along with all mean effect sizes. We will conduct and report all meta-analyses separately for RCTs and non-RCTs. Additionally, we will conduct all syntheses separately by outcome domain (i.e., attitudes toward sexual assault and victims, prerequisite skills and knowledge to intervene, bystander self-efficacy, bystander intentions, bystander behavior, perpetration) and follow-up timing. We will display all syntheses using forest plots.
Assessment of Heterogeneity and Moderator Analysis. We will assess and report heterogeneity using the χ2 statistic and it's corresponding p value, I2, and τ2. If at least 10 studies are included in any given meta-analysis, we will then use mixed-effect meta-regression models to conduct moderator analyses. Proposed moderators include the following: (1) gendered content of the program; (2) mixed- or single-sex group implementation; (3) gender composition of the sample; (4) education level of the sample (i.e., secondary school or college students); (5) mean age of the sample; (6) proportion of fraternity/sorority members in the sample; (7) and proportion of athletic team members in the sample.
Sensitivity Analyses. We will use the contour enhanced funnel plot (Palmer, Peters, Sutton, & Moreno, 2008), Egger's regression test (Egger, Smith, Schneider, & Minder, 1997), and a trim and fill analysis (Duval & Tweedie, 2000) to evaluate the potential of publication bias or small study bias in the included studies.
We will also conduct sensitivity analyses to examine whether study-level attrition and high risk of bias (for each domain assessed with the risk of bias tools) are associated with effect size magnitude, using mixed-effects meta-regression models. Additionally, we will conduct sensitivity analyses that remove any Winsorized effect sizes or sample sizes from the meta-analysis, to assess whether this method for handling outliers may have substantively altered the review findings.
Data Management and Software
We will use the following software throughout the review process: FileMaker Pro (literature screening and data extraction), Microsoft Excel, Microsoft Word, and R (synthesis, generation of figures including forest plots).
Treatment of qualitative research
Qualitative research is not eligible for inclusion in this review.
Footnotes
REVIEW AUTHORS
ROLES AND RESPONSIBLIITIES
Members of the research team responsible for core areas of the review are as follows: Content: Heather Hensman Kettrey Systematic review methods: Heather Hensman Kettrey, Emily Tanner-Smith Statistical analysis: Heather Hensman Kettrey, Emily Tanner-Smith Information retrieval: Heather Hensman Kettrey
The research team at Vanderbilt University's Peabody Research Institute (PRI) that will undertake this project includes the lead report author, co-author, and a research assistant. Dr. Kettrey, the lead review author, will coordinate the review team and assume responsibility for the implementation of the project throughout its duration. Specific tasks include compiling the sample of research reports, creating the database, coding studies, coding program training materials (i.e., program content variables), analyzing data, and preparing the Campbell Review. Dr. Tanner-Smith, review co-author, will provide methodological guidance and mentorship to Dr. Kettrey throughout all phases of data collection and analysis (i.e., sampling, database construction, coding, data analysis, and preparing the Campbell Review).
The bibliographic information found in a thorough search for meta-analysis requires careful screening, first of abstracts, then retrieval and further screening of candidate full-text reports. The studies found eligible for the meta-analysis must then be carefully read and coded into an electronic database. To manage this labor-intensive portion of the work, one research assistant (TBD) will devote time to the project. This person will be selected from among the staff members of PRI who are familiar with meta-analysis, thus, minimizing the necessary training effort.
SOURCES OF SUPPORT
This review is supported by a grant from the Campbell Collaboration (CSR1.60).
DECLARATIONS OF INTEREST
The authors have no conflicts of interest to report.
PRELIMINARY TIMEFRAME
July 2016 (Completed) Submitted Title Registration to Campbell Collaboration (Draft submitted with grant application, 3/29/16; Revision submitted 7/1/16)
August 2016 (Completed) Submitted Protocol to Campbell Collaboration (Draft submitted 8/29/16)
September –October 2016 (Completed) Drafted Coding Manual Constructed Data Management System
November 2016 (Completed) Conducted Literature Search Screened Abstracts
December 2016 – January 2017 (Completed) Retrieved Literature Screened Candidate Studies for Eligibility Began Data Extraction and Coding
February 2017 – March 2017 (Completed) Revised Protocol and Coding Manual per Campbell Reviewers’ Comments (Received reviews 2/21/17) Submitted Revised Protocol to Campbell Collaboration (Submitted on 3/31/17)
April 2017 – June 2017 Update Literature Search and Screening per Revised Protocol Complete Data Extraction and Coding (Includes re-coding previously completed studies per revised protocol)
July 2017 Data Analysis
August 2017 Prepare Final Review
September 2017 Submit Completed Review to Campbell Collaboration by 9/29/17 per Campbell Collaboration Grant CSR1.60.
October 2017, or as needed Prepare and Submit Review Revisions to Campbell Collaboration
PLANS FOR UPDATING THE REVIEW
Heather Hensman Kettrey anticipates updating the review at least once every five years, pending continual accrual of sufficient research on the topic. If she is unable to assume this responsibility she will transfer responsibility to another qualified researcher – with approval from the Campbell Collaboration.
AUTHOR DECLARATION
ELIGIBILITY CRITERIA
FULL-TEXT CODING
Intervention and Comparison Groups
Create one record in this database for each of the intervention and/or comparison groups you selected earlier for coding. For example, studies with a single intervention group and a single comparison group will have two records in this section of the database.
Number each group consecutively within a study, starting with 1.
Select the type of group you are coding. Intervention group Control/comparison group
What type of services does this group receive? Focal/primary bystander intervention program. There may be several focal programs in a study, as when two different types of programs are compared, both of which are expected to be effective. Active treatment that is not a bystander program or IPV/sexual violence prevention program. This is a group that receives a sham treatment (e.g., watches a video on nature, receives nutrition information, diet intervention) intended to take the same duration as the focal intervention program, but does not involve any active bystander or IPV/sexual violence prevention/education components. Inactive comparison condition. This is a group that receives no prevention program and gets only assessments. Active business as usual. This is a group that receives “usual” active treatment (e.g., school-based sex education, college orientation that contains some sex education) that may be effective in preventing sexual violence but is not the focal treatment of the study. This treatment must be limited to services that the group would receive whether or not the research study was implemented (e.g., mandated college orientation). Programs that focus specifically on bystander behaviors or IPV/sexual assault are not eligible controls; however, sex education programs that may contain limited information on violence are eligible. Other.
Program name. Write in the program name or label for this group. Please use the exact name the study authors report (many of the programs will be branded – Bringing in the Bystander, Mentors in Violence Prevention).
Program description. Write in a brief description of the treatment this group receives. As much as possible, quote or give a close paraphrase of the relevant descriptive text in the study report; always include page numbers from the report when appropriate. It is acceptable to copy and paste directly from the article as long as you include the information in quotations and provide a page number for the quotation.
Enter the percent of participants who identify as male in this group. Use -9 for cannot tell.
Enter the percent of participants who identify as female in this group. Use -9 for cannot tell.
Enter the percent of participants who identify as transgender in this group. Use -9 for cannot tell.
Enter the percent of White participants in this group. Use -9 for cannot tell.
Enter the percent of Non-White participants in this group. Use -9 for cannot tell.
Enter the percent of Black participants in this group. Use -9 for cannot tell.
Enter the percent of Hispanic participants in this group. Use -9 for cannot tell.
Enter the percent of Asian participants in this group. Use -9 for cannot tell.
Enter the average age of the group using number of years. Use -9 for cannot tell.
Enter the age range of the group using “XX-XX” format. Use -9 for cannot tell.
What is the grade/educational level for the majority of the group? 7th-8th grade 9th-10th grade 11th-12th grade First year college Second year college Third year college Fourth year college Other/Cannot tell
Enter the percent of participants in this group who reported being a member of a fraternity. Use -9 for cannot tell.
Enter the percent of participants in this group who reported being a member of a sorority. Use -9 for cannot tell.
Enter the percent of participants in this group who reported being a member of a fraternity or sorority. Note that this item is of value when the study authors do not break apart fraternity and sorority membership. Use -9 for cannot tell.
Enter the percent of participants in this group who reported being a member of a school athletic team. Use -9 for cannot tell.
Enter the mean rape myth acceptance score for participants in this group at baseline. Use -9 for cannot tell.
Enter the mean victim empathy score for participants in this group at baseline. Use -9 for cannot tell.
Enter the percent of participants in this group who reported being a victim of sexual assault (by any type of perpetrator). Use -9 for cannot tell.
Enter the percent of participants in this group who reported being a victim of physical assault perpetrated by a partner (i.e., hookup, date, relationship). Use -9 for cannot tell.
Enter the percent of participants in this group who report knowing someone who has been a victim of IPV and/or sexual assault. Use -9 for cannot tell.
Enter the mean bystander efficacy score for participants in this group at baseline. Use -9 for cannot tell.
Enter the percent of participants in this group who reported confidence in their ability to intervene when witnessing violence or its warning signs (measure at baseline). Use -9 for cannot tell.
Enter the mean bystander intention score for participants in this group at baseline. Use -9 for cannot tell.
Enter the percent of participants in this group who reported intentions to intervene when witnessing violence or its warning signs (as measured at baseline). Use -9 for cannot tell.
Enter the mean number of times participants in this group reported engaging in pro-social bystander behavior (i.e., intervening when witnessing any type of violence or its warning signs) at baseline. Use -9 for cannot tell.
Enter the percent of participants in this group who reported engaging in pros-social bystander behavior (i.e., intervening when witnessing any type of violence or its warning signs) at baseline. Use -9 for cannot tell.
To what extent did the intervention focus on bystander behavior? Bystander intervention was the main focus of the program. General sexual assault prevention program with some bystander content. Note that sexual assault programs should not include content on physical violence (these will likely constitute IPV prevention programs). General IPV prevention program with some bystander content. Note that IPV prevention programs can focus on physical violence exclusively or on both physical and sexual violence. Programs focusing exclusively on sexual violence should be coded as sexual assault prevention programs. Other/Cannot tell. Please bring these to the attention of the PI, as they may not be eligible.
What was the sex composition of the intervention group (in delivery setting)? All females. All males. Single-Sex groups for mixed-sex sample. Mixed sex. Cannot tell. NA – Individual Implementation.
How did the program conceptualize the sex of perpetrators of IPV/sexual assault? Male perpetrators only. Female perpetrators only. Gender-neutral. Generic/ambiguous portrayal of sex or explicitly highlighting an equal possibility of male and female perpetrators. Mostly male perpetrators. Mostly female perpetrators.
How did the program conceptualize the sex of victims of IPV/sexual assault? Male victims only. Female victims only. Gender-neutral. Generic/ambiguous portrayal of sex or explicitly highlighting an equal possibility of male and female victims. Mostly male victims. Mostly female victims.
Describe any intervention program content that addressed gender norms, rape myths, or other cultural norms as a root of IPV and/or sexual assault (or as an influence on bystander behavior). Where possible, please indicate whether gender/cultural norm content applies to sexual or physical violence. If a specific (branded) program is implemented (and a treatment manual or other program materials may be available) please clearly indicate this, as we may wish to obtain these materials for further coding.
In what format was the program typically delivered? In-person. This involves some type of presentation or instruction delivered by one or more facilitators. Video. This includes videos that are viewed on televisions or large screens as well as on computers (by groups or individual participants). Web/computer delivered. This applies to non-interactive presentations that are viewed online as well as interactive computer-based programs. It excludes programs that primarily consist of participants viewing videos on computers (these should be coded as videos). Ad/poster campaign. This applies to programs that are designed to promote awareness and contain minimal content (i.e., posters or flyers distributed across campus, brief emails sent to a listserv). Other/cannot tell.
In what context was the program typically delivered? Individual with facilitator Small groups (<10) with facilitator Large group or whole classroom with facilitator Individual alone (e.g., self-guided) Other/cannot tell
Who typically delivered the program? Teachers or school administrators Athletic coaches Peer educators Medical professionals (e.g., doctors, nurses, clinic staff) Campus resource center staff (e.g., women's center, sexual assault center, LGBTQ student center) Non-campus community agency (e.g., domestic violence shelter staff/volunteers) Mixed (no predominant type) NA – Self-guided/implemented Other/cannot tell
Monitoring of program implementation. Was the implementation of the program monitored by the author/researcher or program personnel to assess whether it was delivered as intended? Yes, but no indication of feedback to facilitators. Do not infer that monitoring happened. Select “yes” ONLY if the report specifically indicates that implementation was monitored. Yes, with indication that facilitators received feedback. Do not infer that feedback was provided. Select “yes” ONLY if the report specifically indicates that feedback was provided. No indication that service delivery was monitored.
Implementation quality. Based on evidence or author acknowledgment, was there any uncontrolled variation or degradation in implementation or delivery of the program, e.g., high dropouts, erratic attendance, low compliance, program not delivered as intended, wide differences between settings or facilitators, etc. Note that this question has to do with variation in program delivery, not research contact. That is, there is no “dropout” if all participants complete treatment, even if some fail to complete the outcome measures. Yes Possible No, apparently implemented as intended
Implementation fidelity. Provide a description of any other implementation fidelity measures or assessments, including page numbers where appropriate.
Approximate (or exact) frequency of contact between participant and facilitator or program activity. This refers only to the element of program that is different from what the control group receives. Single-day program Less than weekly One to two times a week Three to four times a week Daily contact Cannot tell
Duration of program in WEEKS. Approximate (or exact) mean number of weeks for the period over which participants received the program, from first to last intervention contact, excluding follow-ups designated as such. Divide days by 7; multiply months by 4.3; multiply years by 52;
Duration of program as received in CONTACT HOURS. Approximate (or exact) mean number of contact hours for the period over which the participants received the program, from first program contact to last contact, excluding follow-ups designated as such. If dosage rates are provided multiply intended duration by dosage rate. Code -9 if cannot tell.
Facilitator training, preparation, or qualifications. Describe any information provided about the intervention facilitators’ training, level of preparation, or qualifications required for delivery. Provide page numbers.
Incentives for recruitment or participation. Describe any incentives for participant recruitment and/or participation. Provide specific information about incentives (including dollar amounts), when available. Provide page numbers.
Outcomes
Create one record for each dependent variable that you will be coding. For example, if the study measures bystander intentions and bystander behavior, you will have two dependent variable records. This is different from the number of times a dependent variable is measured in a study. For example, if the study measures bystander intentions before and after the intervention, you will have only one record here – for the bystander intention measure (but you will have two effect sizes for this outcome measure: one at pretest and one at posttest).
Variable number. This number is an identification number for the dependent variable you are coding. Each dependent variable is numbered consecutively, within the study you are coding so that each has a unique VarNo for that study. If there is only one dependent measure for this study, you will create only one record in this worksheet, and the variable number will be 1. If there are three dependent measures, they will be numbered 1, 2, and 3.
Enter a label for the variable you are coding. When possible, use the exact label that the study report authors use.
Write in a brief description of the measure you are coding. This should include the authors’ label for this variable (e.g., rape myth acceptance, bystander efficacy, etc.), the instrument, the direction of scoring (e.g., lower scores are better), and information about what is being measured. Quote or closely paraphrase the description that is provided in the original report.
For group equivalence variables make sure the label describes successes in terms of bystander attitudes and behaviors (e.g., females, younger participants, participants in lower education/grade levels, participants who have been IPV/sexual assault victims, participants who know an IPV/sexual assault victim, participants with lower rape myth acceptance or greater victim empathy). When coding race always default to white v. non-white. Code non-white as a success.
What type of measure are you coding?
Intervention – sexual violence – female victim Intervention – sexual violence – male victim Intervention – sexual violence – unknown victim sex Intervention – physical violence – female victim Intervention – physical violence – male victim Intervention – physical violence – unknown victim sex Intervention – unclear violence type – female victim Intervention – unclear violence type – male victim Intervention – unclear violence type – unknown victim sex
Bystander efficacy Bystander intentions
Sexual Assault Perpetration – female victim Sexual Assault Perpetration – male victim Sexual Assault Perpetration – unknown victim sex Physical IPV Perpetration – female victim Physical IPV Perpetration – male victim Physical IPV Perpetration – unknown victim sex IPV Perpetration – unclear violence type – female victim IPV Perpetration – unclear violence type – male victim IPV Perpetration – unclear violence type – unknown victim sex
Victim empathy Rape myth acceptance
Sexual Assault Victimization Physical IPV Victimization Know a victim of IPV or sexual assault
Age Sex Race/ethnicity Grade/education level Other predictor of bystander behavior or IPV/sexual assault
Noticing a sexual assault or its warning signs Identifying a situation as appropriate for intervention Taking responsibility for acting/intervening Knowing strategies for helping/intervening
Type of data collection used for outcome measure. Pencil & paper questionnaire Online/computer assisted questionnaire In-person interview Phone interview Other Cannot tell
Number of Days: Enter the number of days over which outcome was counted. Enter -8 for lifetime measures (e.g., “ever” experienced sexual assault). Enter -9 if cannot tell. Multiply months by 30 (e.g., enter 3 months as 90 days).
For cluster randomized trials, please enter the intraclass correlation coefficient (ICC) for each outcome variable coded. Code -8 for not applicable and -9 for cannot tell.
Effect Sizes
Although this is the final section of coding, it is a good idea to identify at least one codable effect size before you start coding a study, because studies that appear eligible frequently end up presenting data that cannot be coded into an effect size.
Report ID for this effect size. Indicate the report number (e.g., 22.02) for the report in which you found the information for this effect size. This is important so that we can find the source information for the effect sizes later on, if necessary, and is especially important for studies with multiple reports.
Page number for this effect size. Indicate the page number of the report identified above on which you found the effect size data. If you used data from two different pages, you can type in both, but use a comma or dash between the page numbers.
Type of effect size you are coding. Pretest and Posttest Group equivalence
There are 3 types of effect sizes that can be coded: pretest, posttest, and group equivalence (or baseline similarity) effect sizes. They are defined as follows:
Select the groups being compared in this effect size. Always select the focal prevention program to be ‘group 1.’
Select the dependent variable for this effect size.
Timing of measurement. Approximate (or exact) number of weeks
It is now time to identify the data you will use to calculate the effect size and to calculate the effect size yourself if necessary.
You need to determine what effect size format you will use for each effect size calculation. There are two general formats you can use, each with its own section in FileMaker: Compute ES from means, sds, variances, test statistics, etc. Compute ES from frequencies, proportions, contingency tables, odds, odds ratios, etc.
Also note that within each of the above effect size formats, effect sizes can be calculated from a variety of statistical estimates; to determine which data you should use for effect size calculation, please refer to the following guidelines in order of preference: Compute ES from regression coefficients with statistical controls for pretest measures and other potential confounding measures at baseline. Compute ES from univariate descriptive statistics (means, sds, frequencies, proportions). Compute ES from test statistics (t, F, Chi square). If significance tests statistics are unavailable or unusable but p-values and degrees of freedom (df) are available, determine the corresponding value of the test statistic (e.g., t, chi-square) and compute ES as if that value had been reported. If you encounter these types of data, please see the PI.
Note that if the authors present both covariate adjusted and unadjusted means, you should use the covariate adjusted ones. If adjusted standard deviations are presented, however, they should not be used.
Which group is favored?
For intervention-control comparisons, the intervention group is favored when it does “better” than the comparison group. The comparison group is favored when it does “better” than the intervention group.
For group equivalence comparisons, the intervention group is favored when it has a higher probability of pro-social bystander attitudes/behaviors than the comparison group
Remember that you cannot rely on simple numerical values to determine which group is favored. For example, a researcher might assess rape myth acceptance, and report this as a score. A lower score is better than a higher score, so in this case a lower number, rather than a higher one, indicates a more favorable outcome.
Sometimes it may be difficult to tell which group is better off because a study uses multi-item measures in which it is unclear whether a high score or a low score is more favorable. In these situations, a thorough reading of the text from the results and discussions sections usually can bring to light the direction of effect – e.g., the authors will often state verbally which group did better on the measure you are coding, even when it is not clear in the data table.
Note that if you cannot determine which group has done better, you will not be able to calculate a numeric effect size. (You will still be able to create an effect size record—just not a numeric effect size.)
Select the group that has done “better”: Intervention Comparison Neither, exactly equal Cannot tell
Effect size derived from what type of statistics? N successful/unsuccessful (frequencies) Proportion successful/unsuccessful (percentage successful or not) Means and SDs; means and variances; means and standard errors Independent t-test Chi-square statistic (1 degree of freedom) Effect sizes as reported directly in the study Other statistical approximation
For this effect size, did you use adjusted data (e.g., covariate adjusted means) or unadjusted data? If both unadjusted and adjusted data are presented, you should use the adjusted data for the group means or mean difference, but use unadjusted standard deviations or variances. Adjusted data are most frequently presented as part of an analysis of covariance (ANCOVA). The covariate is often either the pretest or some personal characteristic such as socioeconomic status. Unadjusted data Pretest adjusted data (or other baseline measure of an outcome variable construct) Data adjusted on some variable other than the pretest (e.g., socioeconomic status) Data adjusted on pretest plus some other variables
Assigned N for the intervention group
Assigned N for the comparison group
Observed N for the intervention group
Observed N for the comparison group
Mean for intervention group
Mean for comparison group
Standard deviation for intervention group
Standard deviation for comparison group
N successful for intervention group
N successful for comparison group
N failed for intervention group
N failed for comparison group
Independent t-value
2 (df=1)
Standardized mean difference (SMD) reported by authors
Odds ratio (OR) reported by authors
Standard error (for SMD or OR) reported by authors
Effect size value- standardized mean difference
Effect size value- odds ratio
Remember that you cannot rely on simple numerical values to determine which group has done better. For intervention-control comparisons, a positive effect size should indicate that the intervention group did “better” on the outcome measure than the comparison group, while a negative effect size indicates that the comparison group did “better” than the intervention group, and a zero effect size means that the two groups are exactly equal on the measure.
You must make sure that the sign of the effect size matches the way we think about direction, such that the effect size is positive when the intervention group (or posttest) is better and negative when the comparison group (or pretest) is better.
Effect sizes can range anywhere from around –3 to +3. However, you will most commonly see effect sizes in the –1 to +1 range.
Note: If the authors report an effect size, include that in your coding and use it for the final effect size value if no other information is reported. However, if the authors also include enough information to calculate the effect size, always calculate your own and report it in addition to that reported in the study.
Any problems coding this effect size?