Abstract

II. Background for the Review
Introduction
The largest waves of young people in history will soon transition into adulthood. Understanding the most effective approaches for reaching out to adolescents 1 —a population of over 1.2 billion (UNFPA, 2003)—is a critical challenge that merits global attention. The period of adolescence is particularly important given its instrumental role in the development of habits and competencies that will affect young people's wellbeing and resilience throughout their lives. Adolescence is also a vulnerable time in which emotions and risk-taking tendencies are amplified (Call et al., 2002; Dahl, 2004; Rutter, 2001; World Bank, 2006).
This systematic review aims to increase empirical understanding of the use of youth empowerment as a strategy for developing psychosocial assets among adolescents. Despite the increasing popularity of involving young people in the processes that affect their lives and communities, little is known about the demonstrated impacts that such participatory programming has on young people (Gray & Hayes, 2008; Zeldin, 2000).
Nevertheless, youth empowerment has been promoted internationally. The African Union, European Union, United Nations, World Bank, numerous national governments (e.g., United Kingdom's Youth Matters), and the philanthropic community are only a few examples of prominent institutions to have explicitly endorsed strategies to increase participation of young people in policy and programming (AU, 2006; EU, 1999; UN, 2005; World Bank, 2006; UKDCSF, 2005; Rosen & Maureen, 2001).
To some extent, the argument for youth empowerment is based on rights (Freeman, 2005). The United Nations Convention on the Rights of the Child (UNCRC), which assures children the right to be heard and form their own views (Article Twelve), commonly encapsulates the ‘rights approach’. This view emphasizes redistribution of power given a perceived injustice embedded in inabilities of young people to exercise their own voice and influence in matters that affect them.
Another approach espouses an instrumental argument. This perspective frames youth empowerment not as a right to be protected, but as a modality for improving youths' developmental outcomes and strengthening institutions and communities by way of young people's contributions (Altman & Feighery, 2004; Jennings, 2006; Suleiman, et al., 2006). The rights-based argument for youth empowerment involves a philosophical and political debate. The instrumental argument—i.e., ‘empowerment leads to positive developmental outcomes’—is an evaluative question of causality, which a systematic review of impact studies is better suited to address.
Concepts and Definitions
Common examples of YEPs are found in particular youth councils, teen centers, community-based participatory research programs, social action and advocacy groups, peer education models, and informal and non-formal education programs that regularly integrate youth participation in program decision-making, as stated above. Structurally, this participation within programs usually takes the form of advisory councils, committees, youth on boards, workgroups, or staff positions. Sometimes, young people and adults serve together in formal leadership capacities such as committees; other times, membership is reserved exclusively for youths with adults acting in more of a supportive role.
Youth empowerment involves a collective, democratic, and pro-social process of engagement, which implies group interaction (Cargo, et al., 2003; Jennings, 2006). Consequently, exclusively one-to-one youth development interventions, such as most mentoring schemes, would not meet inclusion criteria.
Like YEPs, many non-empowerment-based out-of-school programs involve structured activities and safe spaces during hours that adolescents need them most. They do not qualify as YEPs, however, if youths are not systematically involved with program decision-making. Some peer education models, for example, may only activate adolescents in content delivery rather than shaping program planning and implementation (Shiner, 1999).
Often, youth centers and after-school program schedules, objectives, and activities are adult-driven. Youths may be asked for their input or sporadically involved in programmatic decision-making, but if their involvement is not structured so as to ensure opportunity for real influence and regular participation in programmatic decision-making processes, the intervention is not empowerment-based.
While some formal education systems also employ increasingly participatory approaches (Hannam, 2001), this review will focus on youth empowerment initiatives outside of formal schooling. An analysis of effectiveness evidence and unique implementation issues for youth empowerment within formal education would be a valuable undertaking meriting a separate review.
Exponents generally agree that PYD involves an asset-based approach to working with youth, which emphasizes the presence of positive environments, pro-social activities, and supportive relationships in young people's lives. PYD links the process of a youth's involvement in the intervention with developmental outcomes (Roth & Brooks-Gunn, 2003a).
While some definitions of PYD explicitly stress the participation of young people in organizational decision-making and program design as a critical feature in the PYD process (NCFY, 2007; Small & Memmo, 2004; USDHHS, 2001; Youth Development Strategy Aotearoa, 2002), others do not (Catalano, et al., 2004; National Research Council, 2002; Roth, 2004). The PYD movement represents a broader trend towards strength-based approaches to youth and children of which YEPs represent one facet. In other words, while all YEPs are necessarily PYD, not all PYD programs are necessarily youth empowerment.
young people and adults share decision-making young people lead and initiate action adult-initiated, shared decisions with young people young people consulted and informed young people assigned and informed young people tokenized young people are decoration young people are manipulated
This review aims to capture evaluations of programs that fall within the first three degrees of participation, which integrate adolescents into program decision-making.
Moreover, the term youth empowerment—at least in United States literature—is arguably more closely associated with process constructs that better incorporate the supports, resources, and opportunities necessary to make participation meaningful rather than nominal (see, for example, Blanchard, et al., 1996; Brown, 2009; Cargo et al., 2003; Chinman & Linnery, 1998). The authors do not assert that youth empowerment should replace parallel vocabulary, and the review's search terms will be sensitive to the diverse intervention terms used to describe YEPs.
Theory of Change
Youth Empowerment Programs aspire to develop critical psychosocial assets among participating youths. Supportive relationships, affirming environments, skill-building, and the integrated experiences of ownership, challenge, and mastery associated with meaningful youth participation are hypothesized to impact strength-based competencies and behaviors among youth participants.
In turn, youth development literature expects that these assets serve as pathways to distal indicators of success and wellbeing (e.g., academic achievement) and as protective factors against consequences of social exclusion (e.g., antisocial behavior). This basic theory of change is illustrated in figure 1.1. The outcomes of interest for this review are discussed below (measures are discussed in the Methodology section).
Positive youth development frameworks for programming goals, such as the Five C's (Roth & Brooks-Gunn 2003) and the Forty Developmental Assets (Search Institute, 2008) have helped set the stage for an attention to strength-based outcomes in YEPs' theory of change. The Search Institute defines developmental assets as relationships, opportunities and personal qualities that young people need to avoid risks and to thrive. The OECD's establishment of Key Competencies reinforced concern for developmental assets (OECD, 2005). While the first OECD competency category is largely technology and knowledge-based, the remaining two categories—‘interacting in heterogeneous groups’ and ‘acting autonomously’—outline critical competencies included in or closely related to the developmental outcomes described below.
While self-efficacy and self-esteem are distinct concepts (Gilad, et al., 2004), Judge and colleagues (2002) have demonstrated that the two traits are highly related and the combination of the two can yield better prediction, for example, of job satisfaction and performance (Judge, et al., 2002). Moreover, the frequent co-presence of the two constructs as suggested outcomes in literature involving youth empowerment reinforces the sensibility of pairing self-efficacy and self-esteem as primary outcomes from an evaluative perspective (Anderson & Sandmann, 2009; Jennings, 2006; Oliver, et al., 2006; Roth & Brooks-Gunn, 2003b; Sinclair, 2000).
High self-efficacy 2 has been shown to predict better performance in work, academics, and sports; increased happiness, job satisfaction, and persistence; improved safe sex practices; and successful smoking cessation and prevention (de Vries, et al., 1988; Judge & Bono, 2001; Kalichman & Nachimson, 1999; Martin & Gill, 1991; Multon, et al., 1991; Natvig, et al., 2003). A meta-analysis conducted by Stajkovic and Luthans (1998) found that self-efficacy accounted for a 28% improvement in work-related performance. Lower self-efficacy, on the other hand, appears to predict higher levels of depression among young people (Bandura 1999) and is associated with higher alcohol use (Taylor, 2000).
Evidence suggests that high self-esteem is related to high social support and resilience (Dumont & Provost, 1999; Hoffman, et al., 1988) whereas low self-esteem is related to depression, anxiety, and suicidal ideation (Overholser, et al., 1995; Rosenberg, et al., 1995). Boden and colleagues (2008) found self-esteem to be an important “risk marker variable, with low self-esteem being associated with a range of negative outcomes”, and they found high self-esteem at age 15 to be a significant predictor of life satisfaction and peer attachment at ages 18, 21, and 25. Research by Baldwin & Hoffman (2002) indicates that self-esteem changes dramatically during adolescence, emphasizing the special importance of interventions that foster higher and more stable self-regard through this volatile life period.
Youth empowerment and youth development literature conceptually links developmental assets (e.g., self-efficacy and self-esteem) as proximal outcomes to distal performance and behavioral outcomes (e.g., academic performance and antisocial behavior), suggesting causal interactions between variables. Testing this mediator relationship through a multivariate model would require a large number of studies that report the full covariance matrix for primary data. At this time, preliminary searches by the authors indicate that the review will not likely identify an adequate base of included studies to enable such testing. Should a sufficient number of studies be identified through the searches, such testing will be conducted.
Notably, these outcomes only consider effects on participating adolescents. Given the social action and community-integrated nature of many YEPs, the literature frequently suggests that the programs may also affect positive impacts on organizations, communities, and societies as a result of youth action (Jennings, 2006; Kirby & Bryson, 2002; Nygreen, et al., 2006). While not the subject of this review, the evidence for these assertions should be critically reviewed and analyzed in future research.
Personal Assets
Forging social supports and positive connections between youths and their peers, communities, teachers, and families constitutes a central pillar of YEP models (Jennings, 2006; Kirby & Bryson, 2002; Roth & Brooks-Gunn, 2003a; Villarruel, et al., 2003), pp. 329. Such relationships are frequently discussed as both a key process component as well as an expected outcome of successful YEPs. The roles of social supports and positive connections as protective factors and predictors of various aspects of performance (e.g., in school) and wellbeing (e.g., in health indicators) are well documented (Blum, et al., 2003; Hirsch & DuBois, 1992; Hurrelmann, 1990; Witherspoon, et al., 2009).
Social skills “include a range of verbal and non-verbal responses that influence the perception and response of other people during social interactions” (Spence, 2003). As young people actively contribute to decision-making and implementation of social initiatives, it is expected that they acquire and develop transferable skills through such characteristic experiences. Literature concerning youth participation frequently makes this claim (Kirby & Bryson, 2002; pp. 25; Roth & Brooks-Gunn, 2003b; Wilson, et al., 2006). Social skills fit into the higher-order construct of social competence given increasing priority in social policy internationally (Schoon, 2009).
Emotional Intelligence (EI) consists of domains related to aspects of processing, understanding, and managing emotions. Gundlach and colleagues (2003) conceptually argue that increasing EI may act as an important pathway to increasing self-efficacy, reinforcing an important role that some believe EI may play in YEPs' theory of change (Barber, 2007). While a controversial construct (Brody, 2004; MacCann, et al., 2003; Mayer & Cobb, 2000; Qualter, et al., 2007), studies have identified significant associations between EI and academic achievement 3 (Parker et al., 2004), student retention (Parker, et al., 2006), smoking (Trinidad, et al., 2005), and deviant behavior (Petrides, et al., 2004).
Related to other outcomes such as self-efficacy, social skills, and emotional intelligence, the empowerment process may help youth develop stronger assets in coping and problem-solving. Particularly those YEPs that involve adolescent populations especially vulnerable to situations of conflict and stressful life events may prioritize youth empowerment as a strategy to cultivate young people's abilities to effectively navigate difficult situations. Empirical studies indicate a positive relationship between social problem-solving and psychological adjustment and social competence (Heppner & Anderson, 1985; Hopper & Kirschenbaum, 1985). Furthermore, adeptness in problem-solving skills is frequently highlighted as a valued characteristic of organizational and community leaders (Mumford, et al., 2000).
While this review does not limit itself to YEPs centered on social action activities, many if not most YEPs do heavily incorporate community engagement and social advocacy themes into their programming. Therefore, strengthening civic engagement among young people is a key driver for many YEPs (Jennings, 2006). The United Kingdom, for example, has a considerable history of local youth councils for which youth empowerment is often a vehicle for activating young people in neighborhoods and public action (Matthews, 2001). Civic engagement can be expressed in many ways, including volunteering, membership in civil society clubs and organizations, beliefs concerning the importance of civic engagement, expectations of future community involvement, and voting and political participation.
Academic Performance
Several of the aforementioned primary and secondary outcomes (e.g., self-efficacy and emotional intelligence) have been empirically shown to predict academic performance. Through strengthening these proximal outcomes concerning young people's psychosocial assets, YEPs may have indirect impacts on academic performance via improvements in psychosocial outcomes that mediate academic outcomes. YEPs may also directly affect academic performance when educational goals and activities are integrated in particular interventions. Academic performance can refer to standardized test scores, completion, and grades.
Antisocial Behavior
YEPs approach young people from a strengths-based perspective that translates to a primary focus on recognizing and enhancing youths' personal assets. As stated by Roth and Brooks-Gunn (2003a), “The goals of youth development programs promote positive development, even when seeking to prevent problem behaviors.” As the statement by Roth and Brooks-Gunn implies, however, the focus of YEPs on young people's strengths does not necessarily preclude YEPs from having an impact on antisocial behaviors of frequent concern to communities and policy-makers.
Research by Oman and colleagues (Aspy et al., 2004; Oman, et al., 2004a; Oman et al., 2004b), among others, indicates that enhancing adolescents' assets can be an effective course for reducing problem behaviors. By enhancing psychosocial protective factors, engaging youth in constructive activities after school hours (those of highest delinquency rates), and strengthening young people's stake in their environments, proponents contend that YEPs may be at least as effective as those directly and primarily aimed at curtailing antisocial behavior.

Youth empowerment programs—basic theory of change diagram
Considering Harm and Null Results
While the literature on youth empowerment overwhelmingly assumes positive benefits, it is possible that YEPs may be ineffective or even backfire. Some research, for instance, has shown that programs that aggregate deviant youth together—even if for the purpose of positive interactions—can unintentionally reinforce deviant behavior (Dishion, et al., 1999). Other studies have found that higher hostility has correlated with higher self-esteem scores, and offenders have reported higher emotional intelligence scores than non-offenders (Baumeister, et al., 1996; Hemmati, et al., 2004). While these studies do not establish causality between self-esteem and hostility or emotional intelligence and offending, they leave the possibility nonetheless that developing such assets may have unintended consequences for antisocial behavior. The modalities of YEPs may well curtail any such unfavorable effects, but certainty requires a robust evidence base.
Moreover, YEPs might unsuccessfully strive to improve developmental assets. Programs focused on increasing the role of young people as leaders in program decision-making might in practice downplay or neglect valuable expertise of adult youth workers. By consequence, programmatic decisions made by youths could lead to null or negative intervention effects. Attempts at empowerment might ultimately reinforce traditional power relationships, enter adolescents into challenges that leave them feeling inadequate and disillusioned, or elevate some youths over others, circumventing the growth potential of many participants.
Notably, positive youth development places a heavy emphasis on human resources, which can drive up the cost of youth interventions 4 . The expenditures of the Quantum Opportunities Program (QOP), for example, a multi-component, multi-stage youth development intervention for youths from welfare dependent families, are more than $10,000 USD per participant over four years of involvement 5 (AYPF, 1994).
On the other hand, if YEPs can circumvent the long-term consequences of social behavioral problems, the cost-savings to tax payers may be substantial (Scott, et al., 2001). Whether YEPs show effective or ineffective results, the economic implications underscore the need for rigorous evaluation to ensure that resources are invested in interventions and practices that produce intended outcomes for intended populations.
Recent Reviews of Evidence
No systematic review, to the authors' knowledge, has been published that specifically addresses the impacts of youth empowerment. This section discusses two reviews synthesizing research for positive youth development and empowerment broadly, which provide relevant insights informing the planned review.
The Catalano and colleagues (2004) review on positive youth development (PYD) was commissioned by the US Department of Health & Human Services, completed in 2002, and conducted by a team of researchers at the University of Washington (Catalano, et al., 2004). The review ultimately included twenty-five program evaluations, and findings indicate promising results for strength-based programs serving youth and children.
The study, however, did not report a systematic search strategy, extend beyond the United States, nor prospectively state sought-after outcomes. It did not include evaluations revealing null effects or significant effects that did not favor the intervention. The last characteristic is particularly concerning given that past research has demonstrated unintended harm caused by some well-intended youth interventions, which underscores the importance of understanding the consequences–positive, negative, or neutral–of youth development programs (Arnold & Hughes, 1999).
The review's definition of PYD is very broad, including any intervention that meets at least one of fifteen constructs 6 ; none of which stipulated involvement of young people in program decisions or design. The broad inclusion criteria for PYD programs coupled with a lack of predetermined outcomes largely explain why so many evaluations were included in the review. In contrast to the Catalano and colleagues review, the forthcoming review will focus on youth empowerment, reduce program heterogeneity, and minimize the chance of spurious conclusions due to a lack of predetermined outcomes.
The Wallerstein (2006) review, conducted for the World Health Organization, was the only review identified by the authors that directly addressed empowerment strategies. The review explores empowerment broadly for all age groups and with an interest in health outcomes. Wallerstein gives a useful overview of various themes within the broader empowerment movement and gives a framework for empowerment that includes multiple levels of outcomes. The broad, international review of outcomes linked to empowerment offers valuable context for this protocol. The resulting framework includes a heavy emphasis on self-efficacy, community engagement, and social bonding, which reinforce their importance as outcomes of interest in the planned review.
A brief section of the review is devoted to youth empowerment. The author, however, does not discuss the quality of the evidence behind listed outcomes linked to youth empowerment, nor is it clear that the studies actually evaluate a YEP as defined by this review, versus less participatory youth development programs. The planned review will employ a search strategy for empowerment studies specific to youth and controlled impact evaluations.
The Wallerstein review was not conducted according to systematic procedures, no specific outcomes were identified for study inclusion, virtually all types of studies were acceptable for assessing effectiveness with no distinction was made between study designs in discussing findings, and again there is no indication that the review made an effort towards including null or harmful effects. The author defends the review's approach with a dismissal of the use of systematic review methods and experimental designs to assess empowerment programs. As the present protocol runs contrary to this perspective, the authors briefly address assertions stated by Wallerstein:
“Empowerment projects by their nature are complex and do not easily fit into an experimental design…causal relationships may be too complex to uncover within a changing social environment.” While many experimental designs in practice are not sufficiently sophisticated to isolate the effects of different program components within an intervention, they are still capable of determining the impacts that participation in YEPs has on youths for outcomes of concern. The Zief and colleagues (2006) Campbell Collaboration review, for example, assesses the evidence for after-school programs, which can be comparably complex and heterogeneous. Conceivably, moreover, a controlled trial could compare youth empowerment with an active comparison in which adolescents participate in a non-participatory though otherwise similar intervention (e.g., youth-driven smoking cessation campaign versus didactic smoking cessation education classes). In other cases, complementary process and qualitative studies can help clarify active ingredients within multi-component programs (Miller et al., 2003; Morton, 2009).
“Expectations for individual health status changes may be unrealistic in short time frames…” For this reason, it is important that program evaluations delineate clear theories of change. While ambitious health changes may indeed take much time, proximal psychosocial outcomes mentioned by Wallerstein and earlier in this protocol may arise in shorter time periods. Such short-term impacts have been documented by other evaluations of positive youth development programs broadly (Catalano et al., 2004; Tierney et al, 2000).
“Evaluation should…use a mix of methodologies and designs – rather than randomized control trials…” The author appears to treat mixed methods and RCTs as mutually exclusive. To the contrary, experimental designs of social interventions are often used within a mixed methods approach and are most useful when complemented by other forms of research addressing process and implementation (Mayo-Wilson, 2007; Oakley et al., 2006). Experimental designs of social interventions are increasingly an area of interest for participatory research approaches as well (Jones et al., 2008). Experimental and quasi-experimental trials are best suited to answer a specific type of research question concerning causality and effectiveness. This is not the only type of question of concern for youth empowerment, but it is the primary focus of the planned review.
III. Objectives of the Review
This review aims to systematically investigate and summarize the state of the evidence on the impacts of YEPs on adolescents' self-efficacy and self-esteem. Specifically, this review endeavors to address the following questions in order to contribute to the body of evidence available to stakeholders and researchers so as to improve services and supports for young people:
The expectation of policy-makers, funders, and community organizations to actively involve young people in program design and implementation is becoming increasingly popular. This review works towards a better understanding of the measured merits behind that option. If evidence allows, the review further aims to advance knowledge with respect to the decisions within youth empowerment (e.g. how youth empowerment works best and for whom).
IV. Methodology
Study Inclusion Criteria
This review will include evaluation studies of youth programs and councils outside of formal education that regularly involve young people in program planning and implementation, primarily include adolescents (over 75%), meet certain methodological standards, and measure at least one outcome of interest.
Delivery may take place in community-based or school-based settings so long as the intervention occurs regularly and outside of formal education. Interventions primarily within formal education, juvenile justice, residential programs, therapeutic interventions, class-based or university activities, conferences, or workshops will not be included. Includable programs must convene regularly (i.e., not a one-off event).
While both specific and general self-efficacy measures will be included in this review, they will be distinguished in the analysis, and only comparable measures will be aggregated. Popular examples of GSE measures include Schwarzer et al.'s (1995) and Chen et al.'s (2001) general self-efficacy scales, which consist of 8 to 10 items. Both instruments have been validated internationally and with adolescents (Scholz, et al., 2002).
Search Procedures and Results
This investigation will involve a thorough, highly sensitive search for evaluations completed or in progress. The search strategy includes multiple electronic databases and direct, international outreach to professional networks.
Population:
(young OR youth* OR child* OR teen* OR adolescen* OR minors OR school ADJ student* OR boy* OR girl* OR NEETs OR NEET OR 14-19 OR school ADJ student*).ab,ti.
AND
Intervention:
(pyd OR cyd OR empowerment OR youth ADJ engag* OR volunteerism OR volunteering OR youth ADJ advocacy OR youth ADJ activism OR youth ADJ development OR youth ADJ leader* OR youth ADJ inclusion OR community ADJ service OR after ADJ school OR afterschool OR youth ADJ1 decision-making OR youth ADJ driven OR youth ADJ run OR youth ADJ adult ADJ partnership* OR youth/adult ADJ partnership* OR youth-adult ADJ partnership* OR youth ADJ action OR youth ADJ1 involvement OR youth ADJ participation OR young ADJ people* ADJ participation OR youth ADJ led OR peer ADJ education OR peer ADJ led OR peer ADJ participation OR youth ADJ voice OR service ADJ learning OR youth ADJ council* OR teen ADJ council* OR non-formal ADJ education OR nonformal ADJ education OR informal ADJ education OR teen ADJ cent* OR youth ADJ cent* OR participatory ADJ research).ab,ti.
AND
Methods:
(control* OR random* OR trial* OR effectiveness OR efficacy OR compar* OR clinical* OR experiment* OR impact ADJ evaluation OR impact ADJ study OR impact ADJ assessment OR outcome ADJ evaluation OR outcome ADJ study OR outcome ADJ assessment).af.
Additionally, eleven web-based publication databases specific to youth and family services will be searched with varying search strategies depending on the confines of each database. These include Chapin Hall (University of Chicago), Out-of-School Time Program Research & Evaluation Database (Harvard Family Research Project), Innovation Center, National Clearinghouse on Families & Youth (US Administration of Children & Families), Public/Private Ventures, Search Institute, the UNICEF Evaluation and Research Database (ERD), the Australian Clearinghouse for Youth Studies (ACYS), National Council for Voluntary Youth Services (NCVYS) Publications, the UK DCSF Inclusion Development Programme (IDP) Publication Catalogue, and the World Bank Poverty Impact Evaluations Database.
Screen of Studies for Inclusion in the Review
Both authors will review all citations and discuss and resolve issues concerning study inclusion and exclusion. The screening guide for citations and abstracts is provided in Appendix I. If the abstract is rejected by both authors according to any of the six screening criteria, it will be automatically excluded. If the article is not initially rejected according to any of the initial screening criteria, then the full article will be retrieved for further analysis, in order to determine if all six screening criteria are verified for inclusion criteria.
Judging study quality
A systematic approach to assessing study quality on the basis of predetermined criteria was developed by the reviewer. Forty-one characteristics of study design and reporting will be used to appraise study quality, and four standards in particular are considered priorities for judging study quality. These standards include evidence of (a) no significant control group contamination, (b) no significant overall study attrition nor differential attrition that would bias the results, (c) appropriate statistical measures used for analyses, and (d) primary outcomes having been measured at follow-up for all available sample members, thereby meeting the qualification of ‘intention-to treat’, not ‘treatment-on-treated’, analysis. The checklist used to appraise study quality is included in Appendix IV. The checklist is intended as a discussion guide to facilitate meaningful analysis of the quality of included studies; it is not part of the inclusion criteria.
Data Analysis
For continuous outcomes, Hedges g will be calculated using means and Standard Deviations (SDs) when these values are available from published reports or from the study authors. Unlike Cohen's d, Hedges g corrects for small sample size and may give a more conservative estimate of variance.
When means and SDs are not available, the authors will calculate Hedges g using other available statistics, for example an F-test and p-value or t-test and p-value. When data are presented in several forms that could be used to calculate an effect size, we will select the least form that is closest to the raw data. That is, when mean changes are reported in addition to ANOVAs, we will select mean change scores. When means and SDs are not available and Hedges g is calculated using other statistics, we will note this in the text.
Both primary outcomes (self-efficacy and self-esteem) would seem to involve effect sizes based on mean differences only. For most secondary outcomes, this would also be the case. However, there are smaller likelihoods for some possible effect sizes based on incidence rates to arise in a couple of the secondary outcomes—namely, antisocial behavior (if measured, for example, by occurrences of delinquency) or academic performance (if measured, for example, by occurrences of graduation). That the incidence-based effect sizes will show up is probably unlikely, but, if they do, they will be sufficiently different in nature from other outcomes that they would require separate analyses.
Program covariates will include duration and frequency of intervention, presence of a civic engagement component (i.e., volunteering, service-learning, advocacy, or public awareness), existence of a training or preparation component prior to or in tandem with leadership opportunities, and levels at which young people are involved in program decision-making. These program qualities all have important implications for the theoretical literature regarding which aspects of the various approaches to youth empowerment serve as active ingredients for achieving positive outcomes (Billig, et al., 2005; Catalano et al., 2004; Chinman & Linnery, 1998; Jennings, 2006; WHO, 1999).
The reviewers will conduct separate analyses for each type of primary and secondary outcome. As stated previously, this review is not likely to collect a sufficient number of studies and data in order to statistically analyze the relationship(s) between primary and secondary outcomes, though the literature suggests that they may in fact be related. Separately analyses will be conducted for active comparison and no-treatment comparison groups.
Measures that assess the same basic outcome (e.g., self-efficacy) will be analyzed together, but subgroup analyses will be conducted for measures assessing distinct concepts within the basic outcome (e.g., academic-specific self-efficacy vs. general self-efficacy). Measures that assess the same concept but with different instruments will be analyzed together (e.g., general self-efficacy measured by different scoring instruments).
Treatment of Qualitative Research and Process Studies
Qualitative studies can contribute usefully to a more holistic understanding of youth empowerment. This review, however, centers on a research question concerning effectiveness as assessed by impact evaluation designs capable of establishing a credible counterfactual. As such, we will maintain a focus on controlled trials for the purposes of this review, though we will discuss any qualitative process or implementation research associated with included studies (as outlined in section 20.2.3 of the Cochrane Handbook; Higgins & Green, 2009).
A qualitative systematic review exploring process, mechanisms, and perceptions underlying youth empowerment would indeed be a valuable enterprise for C2, but it would be a distinct and considerable undertaking meriting unique criteria, methods, and a separate review altogether (section 20.3.2.5 in the Cochrane Handbook; Higgins & Green, 2009; Dixon-Woods et al., 2006; Jones, 2004).
Process and implementation studies play a vital role in making sense of the results of an impact evaluation and therefore should ideally accompany any trial measuring the effectiveness of a social intervention (Mayo-Wilson, 2007; Oakley et al., 2006). This is particularly important for YEPs, which rely heavily on process and can vary significantly in implementation. Process studies can involve a combination of qualitative and quantitative methods assessing programming aspects such as program quality; levels of youth engagement, participation, and satisfaction; fidelity to intervention manuals, curricula, or plans; program environment; nature of youth-adult and peer dynamics; and program ‘dosage’ and consistency. While accompanying process studies are not required for study inclusion, they will be discussed in relation to study quality, making sense of impact study results, and implications for YEPs.
V. Anticipated Timeframe
VI. Plans for Updating the Review
There are no present plans for updating the review, but, funding permitting, updates will be conducted as needed.
VII. Acknowledgements
The inspiration for and development of this review is owed to many colleagues, both researchers and practitioners. In particular, we thank our colleagues affiliated with the Centre for Evidence-Based Intervention at the University of Oxford for their guidance with employing the strongest strategies for systematic reviews and informing the theoretical underpinnings of this research.
VIII. Statement Concerning Conflict of Interest
Morton has worked with community organizations to develop YEPs. Both authors are currently working on a randomized controlled trial of a youth empowerment program in Jordan.
Footnotes
Appendix I
Appendix II
Appendix III
Appendix IV
1
This review follows the definition of adolescence as ages 10–19 as used by the World Health Organization, United Nations, and World Bank. The authors use adolescents, youths, and young people interchangeably throughout the protocol unless otherwise stated. The aforementioned institutions technically define youth as ages 15–24 and young people ages 10–24. Ages 10–14 constitute early adolescence; ages 15–19 make up late adolescence.
2
Notably, literature on self-efficacy frequently delineates between general and specific self-efficacy. Both aspects of self-efficacy are includable in this review, and the distinction is further discussed in the Measures section.
3
Other studies have demonstrated contradictory evidence of EI's relation to academic achievement (e.g., O'Connor & Little, 2003).
4
Research by The Finance Project that surveyed fourteen youth empowerment programs in the US found that staff salaries and benefits accounted for an average 54% of intervention cost (Gray & Hayes, 2008).
5
QOP was designed as a multiyear intervention.
6
Constructs included the following fifteen objectives: Promotes bonding, fosters resilience, promotes social competence, promotes emotional competence, promotes cognitive competence, promotes behavioral competence, promotes moral competence, fosters self-determination, fosters spirituality, fosters self-efficacy, fosters clear and positive identity, fosters belief in the future, provides recognition for positive behavior, provides opportunities for pro-social involvement, and fosters pro-social norms.
7
Notably, while both specific and general self-efficacy and self-esteem measures will be eligible for inclusion in this review, some prominent youth development scholarship appears to favor general measures. Roth & Brooks-Gunn's (2003) Five C's Framework, for example, focuses on “[a]n internal sense of overall positive self-worth and self-efficacy; one's global self-regard, as opposed to domain specific beliefs” (Lerner et al., 2005).
8
Variations of Boolean operators, wildcard symbols, and field indexes will be used depending on the nature of the specific database.
