PROTOCOL: Interventions to Improve Labour Market Outcomes of Youth: A Systematic Review of Active Labour Market Programmes

1.3.1 Training and skills development

Education and skills are considered a core factor in determining the chances of young people in the labour market (Biavaschi et al., 2012). Skill training programmes are therefore the most widely used labour market intervention for young people worldwide and are increasingly delivered as a complement to other labour market measures (Betcherman et al., 2007; Fares & Puerto, 2009). The objective of skill training programmes is to develop employment-relevant skills of jobseekers. Broadly speaking, employability skills refer to a set of job-specific technical skills, but also include non-technical, soft (or behavioural) skills, such as self-management, teamwork and communication. Increasingly, employers across the world are placing higher value on these non-technical skills than on technical competencies (Cunningham, Sanchez-Puerta & Wuermli, 2010; Youth Employment Network & IYF, 2009).

In this analysis, we classify training programmes according to the skill-set which they target (Table 1):

First, training programmes that address a lack of trade- or job-specific technical skills demanded by employers. Such skills range from manual skills to computer literacy. Technical skill training programmes often include an on-the-job training component in order to increase practical work experience (i.e. by placing participants in internships, workplace training or apprenticeship schemes).

Technical training programmes are popular in development cooperation because many developing countries experience a skills mismatch between their labour force and emerging segments of their economies. However, pure training programmes have not proven very successful in many contexts (Betcherman et al., 2004). Therefore, most recent programmes tend to combine skills training with other types of interventions; for example, on-the-job training or employment services (Cunningham et al., 2010; Fares & Puerto, 2009). An example of a comprehensive skills training programme is provided in Box 1.

A number of conditions determine whether skills training programmes are successful in bringing additional youth into work – most notably, consistency between the skills offered by a training programme and those demanded by the market. To this end, some programmes introduce a market-based (or bottom-up) approach in programme design. The application of this approach enables training curricula and programme components to respond much more effectively to the needs of employers (in both private and public sectors) and communities in a demand-driven fashion.

Furthermore, the success of all these interventions relies on the assumption that the (correct) target group participates in the training and that the training is appropriate and conducted in a way which actually augments the skill sets that are relevant to the labour market. Finally, a crucial element may be the award of a legitimate certificate on successful completion of a programme to prove the acquisition of increased knowledge and skills to potential employers in the job market.

1.3.2 Entrepreneurship promotion

Entrepreneurship promotion programmes aim to lower the barriers and costs for young unemployed and underemployed planning to establish or maintain a business. Since the scope of formal wage employment is often limited in developing countries, increasing (formal) self-employment among the labour force is considered an important anti-poverty strategy (Gindling & Newhouse, 2012). Because self-employed and small-scale entrepreneurs often face numerous internal and external constraints, a multitude of measures exist to support the process.

Access to capital is often a primary constraint for young entrepreneurs. Schoof (2006) identifies a number of constraints to accessing start-up finance. These range from lack of personal savings and resources to lack of securities and credibility, insufficient business experience and skills, strict credit-scoring methodologies and regulations, among others. Accordingly, many entrepreneurship programmes address the lack of access to (affordable) finance faced by young entrepreneurs. We disaggregated such programmes into three types:

First, those providing or facilitating access to credit (including microfinance programmes).

ALMPs that facilitate access to finance often provide technical advice and imply setting partnerships and capacity building schemes with (and for) microfinance institutions (MFIs) and banks.

In addition to access to finance, there are programmes that offer business advisory services and mentoring for soon-to-be or already self-employed youth. Finally, some interventions try to reduce the barriers to business creation by assisting prospective entrepreneurs to enter markets or existing value chains. The above-described interventions and their underlying theory of change are shown in Table 2. We included training related ALMPs, which deliver the skills relevant for starting or maintaining a business (such as management skills, leadership skill and financial literacy), described in section 1.3.1 above.

Many entrepreneurship programmes take a comprehensive approach; for example, combining access to credit with business skills training or post-programme consultation (i.e. mentoring and coaching). Most microfranchise programmes follow this pattern – delivering finance, business networks and mentoring services.

Primarily, entrepreneurship programmes increase employment through their direct effect on the soon-to-be self-employed participant. The assumption is that beneficiaries actually plan to set up a new business after receiving credit and/or training (i.e. that targeted and trained individuals have been appropriately selected for the programme) and that they would not have done so without the intervention.

In order to generate additional jobs, entrepreneurship programmes have to assume that the intervention leads to either (i) increased marginal productivity of the input labour or (ii) increased output and profits resulting in additional investments and labour demand. To achieve this end, the training must suit the context and knowledge of the participants. Beneficiaries then have to apply the training or credit to their business and thereby increase performance and competitiveness.2 Whether an entrepreneur will finally hire additional workers may also depend on the macroeconomic and labour market environment. Box 2 describes the programme Start and Improve Your Business (SIYB), a widely used and adapted entrepreneurship training package designed by the ILO and tailored for youth.

1.3.3 Employment services

Employment service programmes are generally based on the (matching and) intermediation approach to active labour market policy. Intervention types within employment services are shown in Table 3. Job-placement programmes acknowledge the existence of information asymmetries in the labour market. Hence, these programmes aim to improve the job-matching process by providing information and support to both sides of the labour market. On the one hand, they inform young jobseekers about suitable job opportunities (which is of particular relevance to youth who have only recently entered the labour market and are experiencing difficulties in marketing themselves or lack the knowledge and networks to find job openings); and, on the other hand, they provide information to potential employers about available and unemployed youth. The underlying idea is to facilitate the matching of employment opportunities with jobseekers while reducing the costs and risks to employers connected with recruiting young people.

The second type of intervention, job-search assistance services, include job-search training, educational or career guidance, counselling and monitoring programmes. Such programmes primarily address disadvantaged or demotivated youth who are disconnected from the labour market. Their primary aim is to improve the intensity, motivation and effectiveness of job-searches by participants. Mentoring programmes are also provided to youth who are not currently unemployed but are in education or have just entered the labour market (post-placement support). Accordingly, in some circumstances, mentors encourage mentees to stay in education or in on-the–job training. In many countries, employment agencies adopt a case-management approach (identifying barriers to employment, designing individual action plans, referring jobseekers to interventions and monitoring job-search activity), which has been argued to be the most effective method of providing these services (Walther & Pohl, 2005).

While in some countries public employment agencies continue being the main providers of employment services, some others have moved into subcontracting, opening an important role for private employment agencies to address mismatches and information failures in the labour market. Box 3 illustrates a subcontracting model applied by a French public employment agency to facilitate counselling and job-placement for educated youth.

There is indication that involvement in employment services (and in ALMPs in general) creates a stigmatizing effect on participants (Boone & van Ours, 2004, and Kluve, Lehmann & Schmidt, 1999). Addressing this adverse effect is a prior condition for successful implementation. To this end, job-placement and job-search assistance programmes are often connected to financial incentives for jobseekers and/or employers. For example, such schemes may involve the imposition of sanctions on the unemployed for non-compliance with the intervention. Similarly, marketing of unemployed youth may be combined with the offer of short-term subsidies to employers.

1.3.4 Subsidized employment

Insufficient labour demand is one of the main constraints faced by young job market entrants – particularly in developing economies. Subsidized employment interventions comprise two main areas: wage subsidies and labour-intensive public employment programmes (Table 4), both of which are designed to increase the job and training opportunities available to young unemployed. The main aim of both types of intervention is to ensure that individuals who do not find a job on the regular labour market remain integrated and connected to economic and social life. To that end, such programmes offer short-term interventions but primarily work towards longer term labour market effects.

Wage subsidies come in numerous forms and can be provided through various mechanisms, ranging from direct transfers (vouchers) to reductions in social security contributions and tax credits.

First, employer-side subsidies reduce the financial costs or risks associated with not knowing the productivity of the person to be employed. As with employment services, this is a scheme which is particularly relevant to youth entering the labour market for the first time, and whose (perceived) marginal productivity may be below market wages. Employer-side subsidies may also serve to lower the costs to employers of providing on-the-job youth training. Such training subsidies offer the possibility of expanding the number of work-based training places for disadvantaged young people.

Second, employee-side subsidies promote labour supply through increasing the returns from employment and hence increasing incentives to seek and retain employment. While it is believed that employer-side subsidies may also encourage more active job-search (because youths believe they will be able to find work), providing employee-side earning supplements may permit better targeting of specific socio-demographic groups. Furthermore, whereas employer-side subsidies tackle a lack of labour demand, employee-side subsidies may be more appropriate in countries facing labour supply constraints, for example due to reservation wages.

It is important to acknowledge the limited use and evidence of wage subsidies in developing countries. Almeida et al. (2014) detail the results of experimental and quasi-experimental impact evaluations around the world. Most evidence comes from the US with rather mixed results concerning the effectiveness of wage subsidies as tools for fostering job creation. The UK programme, New Deal for Young People, demonstrated the subsidy's ability to lower unemployment among youth relative to other available interventions, including subsidized education and training. The programme is explained in Box 4. While evidence in developing countries is more limited, it is also mixed, showing negative impacts of wage subsidies in Jordan (Groh, Krishnan, McKenzie & Vishwanath, 2012) but positive and sustained impacts in South Africa (Levinsohn et al., 2014).

The second type of labour market intervention analysed in this category is labour-intensive public employment programmes. Public employment programmes are the umbrella for public works programmes and employment guarantee schemes. Despite the strong association of these programmes with infrastructure and construction works, there is a growing trend towards works in the social sector, environmental services and multi-sectoral, community driven programmes (Lieuw-Kie-Song, Philip, Tsukamoto & Van Imschoot, 2010).

In this type of intervention, basic social income recipients are recruited for public jobs and receive a small earning supplement to their unemployment assistance. Programmes usually target unskilled, disadvantaged or long-term unemployed workers with the aim of keeping them in contact with the labour market and mitigating the depreciation of human capital during periods of unemployment.

While public employment programmes have been often recommended as a measure in times of crises (e.g. seasonal shocks or economic recession)6 they are increasingly used as a regular component of wider employment policies (Lieuw-Kie-Song et al., 2010). In addition, they have become popular as a mechanism for addressing youth unemployment (Grosh, del Ninno, Tesliuc & Ouerghi, 2008), serving both as an introduction to the world of employment and as a tool to maintain social integration. This is particularly relevant for youth service programmes, in which youth can “play an active role in community and national development while learning new skills, increasing their employability, and contributing to their overall personal development” (Cunningham et al., 2008).

Most wage subsidies and public employment programmes are designed to support employment only in the short or medium term. A positive effect on final outcomes is only attainable if the work experience and training received during the period of subsidized work also improves the longer term employment prospects of participants. For this reason, (i) wage subsidies are often granted to firms that agree to provide additional training to subsidized employees (i.e. in connection with apprenticeship schemes)7 and (ii) public employment programmes are often paired with exit strategies, such as skills training or entrepreneurship.

3.1.1 Population

The ALMPs considered in the study (i) target the unemployed or those with low levels of skills or limited work experience or who are generally disadvantaged in the labour market and (ii) aim to promote employment and/or earnings/wage growth among the target population, rather than simply providing income support (Heckman et al., 1999). In terms of age groups, the study focuses on programmes that are designed for – or target primarily – young men and women aged between 15 and 35.

3.1.2 Intervention

Eligible programmes considered for analysis, as defined in Table 6, include training and skills development, entrepreneurship promotion, employment services and subsidized employment.

3.1.3 Comparison

Eligible comparison groups include those which receive no intervention, another youth employment intervention or are due to receive the intervention in a pipeline design. In general, studies will focus on the binary case, where two treatment states exist: exposure to treatment and non-exposure (control).

Since the extension of the binary model to multi-valued treatments is straightforward, corresponding methods have been developed. The case of a randomized control trial (RCT) with several “arms” (or treatment states) is particularly simple. Hence, we will include studies with multi-valued treatments. We will code relative effects (e.g. effects that are measured relative to an alternative treatment) for those studies that also include a non-exposure comparison group. We will code relative effects separately for studies which include a non-exposure (control) state since the interpretation of results that compare two actual treatments is different, in particular in relation to other studies' findings. The comparison group must either be constructed using an experimental design or using one of the quasi-experimental methods discussed in section 3.1.5.

3.1.4 Outcomes

The study identified the following primary outcomes of interest, which will be the focus of the analysis and provide the framework for potential policy and programme recommendations:

Employment outcomes, including employment, unemployment, participation rates, hours worked, unemployment duration and quality of employment. 10

The systematic review will include studies that measure change in at least one of these outcomes among intervention participants and relative to non-intervention participants based on a counterfactual analysis. When available, the analysis will include effects on population sub-groups outlined by age cohorts, gender, education level, income level and location, among other dimensions (see coding tool for a full set of combinations in section 6.6). No restriction is placed on the duration of follow-up data.

The review also includes outcomes which are measured conditional on some other outcome. Commonly, the review covers studies estimating the effect of treatment on wages by either (i) setting all wages of unemployed individuals to zero or (ii) excluding all unemployed individuals from the treatment and control groups. The latter would be the treatment effect on wages conditional on being employed. The review treats such conditional treatment effects as a subgroup analysis. Generally, unconditional treatment effects are considered to be the more policy-relevant estimate and are the basis for the review's analysis.

The review will not include studies that focus only on intermediary outcomes without measuring impact on the primary outcomes listed above. The review will consider net employment or general equilibrium effects as outcomes of interest if a study also measures the above described primary outcomes and meets the study inclusion criteria.

3.1.5 Study designs

The key conceptual point is that both exposure to treatment (factual) and non-exposure (control or counterfactual) states are defined for the individual unit, along with realizations of the outcome variable for each state. The causal model, known as the Potential Outcome Model, is as follows:

▪

Treatment indicator: D (=1 if participating; =0 if not participating). In general, D = {0,1,…,T}, most applications focus on the binary case, D = {0,1}, i.e. an exposure state and a non-exposure state.

The potential outcome model in the binary case is therefore:

Y = Y ₀ if D = 0,

This illustrates the missing data problem that impact evaluation research faces (in the binary case):

The model has several important implications. First, the causal effect at the individual level Δ = Y ₁ - Y ₀ cannot be observed. Hence, we need an adequate parameter to summarize individual causal effects, and empirically impact evaluations will focus on the estimation of average causal effects.

Various types of averaging are possible in this regard, and the most commonly used evaluation parameters are the Average Treatment Effect (ATE) and, especially, the Average Treatment Effect on the Treated (ATET):

ATE: E(Δ) = E(Y ₁ - Y ₀) = E(Y ₁) - E(Y ₀)

where E(.) denotes the expectations operator, and the second term in the last equation is the counterfactual (= the average outcome of the participants in the case of non-participation). This counterfactual E(Y ₀ |D = 1) is, in econometric terms, not identified. A parameter is identified if it can be estimated with ever-increasing precision as the sample size increases. This, however, is the case for the first term E(Y ₁ |D = 1), which is identified from observable data and could be estimated with perfect precision if N were infinitely large.

The evaluation problem is therefore: what assumption allows the unobservable counterfactual average to be replaced by an alternative, identified population average? Such an identifying assumption, if correct, allows for construction of a counterfactual situation and identifies the population parameter. Note that, due to the unobservable nature of the counterfactual, every impact evaluation requires one or more identifying assumptions. These must be justified and made plausible by the researcher, since they cannot be statistically tested (at least not fully), hence cannot be right or wrong a priori nor proven right or wrong a posteriori.

This - simple and well-known - delineation of the causal model underlying impact evaluations is explained here, because it facilitates the description of the various methods that the component studies of this systematic review use. These methods are competently and comprehensively reviewed in several articles and books, such that a detailed explanation is beyond the scope of this protocol and, in fact, not required to explain the review's procedure. Important resources in this regard include, for instance, articles focusing predominantly on the econometrics of impact evaluation (e.g. Heckman et al., 1999, Imbens & Wooldridge, 2009) and several (hand-) books providing guidance for practitioners (e.g. Gertler et al., 2011, Khandker et al., 2010). It is worth noting that many of the empirical methods for causal analysis now commonly used in evaluation research have been developed explicitly for the purpose of evaluating training and other employment programmes (Imbens & Wooldridge, 2009).

The key virtue of formulating the problem of causal inference using Rubin's causal model is that it makes apparent the central role of the relationship between treatment assignment and potential outcomes. Moreover, this allows the set of available methods to be correspondingly classified into three groups, following Imbens and Wooldridge (2009):

1. Causal inference based on RCTs

We will consider these three categories and the methods they comprise in turn.

Randomized experiments: The most straightforward case for analysis occurs when assignment to treatment is randomized (in a controlled way by an experimenter) and, thus, independent of covariates X as well as the potential outcomes Y. In such classical RCTs it is relatively easy to obtain estimators for the average effect of the treatment, using for example the simple difference-in-means by treatment status. Randomized experiments have been used in the evaluation of labour market programmes since the 1970s (starting in the US), with some increasing trend over the last decade, although still not on a very large scale. In recent years, RCTs have increasingly been used in development economics – analysing very diverse questions – and we expect to identify a fair number of experiments addressing the type of intervention that is of interest to this systematic review. Still, it is likely that the majority of relevant impact evaluations will not be based on RCTs.

These three categories comprise the impact evaluation methods that are relevant for this systematic review. We note, however, that caution will be needed when synthesizing effect sizes from these different studies, due to potential problems relating to comparability of different treatment effect estimators (see also section 3.4) and partial versus bivariate effect sizes (Keef & Roberts, 2004; Becker and Wu, 2007). As noted below, we will therefore collect data on, and assess sensitivity of findings to, the treatment effect estimator and use of adjusted analysis.

3.2.1 Method to find and select relevant studies

The search for relevant literature is based on a variety of sources in order to ensure that published and unpublished studies (“grey literature”) relevant to the review question are included in the search process. The search process includes (i) a primary search - searching of general and specialized databases - and (ii) a complementary search - hand-searching of relevant websites, literature snowballing and contacting experts. Through this comprehensive search process, the review seeks to arrive at a comprehensive and unbiased set of studies. To this end, country restrictions will not be applied to the search and selection process. The search will include search terms in English, Spanish, French, German and Portuguese, but there will not be any language restrictions in the selection process.

In accordance with the review's focus on contemporary research, the selection process will focus on studies published since 1990. An appropriate filter to identify such studies will be applied to the search where possible. The screening and study classification process will be based on the inclusion and exclusion criteria (see section 3.1). The study search and selection process will follow the procedure defined in Figure 2.

OPEN IN VIEWER

2

Search and selection process

3.2.2 Data sources and search strategies