PROTOCOL: Post-basic Technical and Vocational Education and Training (TVET) Interventions to Improve Employability and Employment of TVET Graduates in Low- and Middle-income Countries

The Problem, Condition or Issue

In the twenty-first century, both developed and developing nations are faced with the demands of a rapidly changing, more globally competitive world. Major forces are driving change in the world of work, including advances in information and communication technology (ICT), the introduction of new manufacturing processes, increased economic integration between countries and increased competition due to trade liberalisation. The impact of economic globalisation has been uneven, however. Whilst some developing countries, particularly China and India, have considerably improved their standing in the global economy, many non-globalising developing countries have fared much worse; many seeing an expansion of the informal economy, characterised by a reliance on unskilled work, combined with stagnation in the formal economy. Recent development progress in education means there are more skilled workers in the world than available prospects. Simultaneously, global unemployment is on the increase. Recent shocks provoked by the international financial crisis exposed severe weaknesses within the global economic system which rapidly spread to the employment sector, triggering a global jobs crisis. The global economy has substantially reduced its capacity to add new jobs.

Youth have been particularly hard hit by the economic crisis. The youth unemployment rate rose sharply during this period, from 11.6 to 12.7 per cent, and, in the absence of economic growth and development, this pattern is likely to continue (ILO, 2011). In 2011, 74.8 million youth aged 15–24 were unemployed; globally, young people are nearly three times as likely as adults to be unemployed (ILO, 2012). There is significant regional variation in youth unemployment. Countries of the Middle East, Africa, South Asia and Latin America are particularly affected (UN, 2012; UN/DESA, 2011). Furthermore, worldwide, many young people are underemployed (in part-time, temporary or low productivity jobs), and/or work in low quality, poorly paid jobs. The vast majority of the world's youth work in the informal economy, many under poor working conditions or in hazardous forms of work. In many countries, young women are much more likely to be unemployed than young men (UN, 2012). The marginalisation of women in employment and training is a relevant issue globally given the potential impact on human capital, but particularly in those countries in which women constitute the majority of the population (Misola, 2010). Other groups of young people more prone to unemployment and underemployment include youth with disabilities, those affected by HIV/AIDS, indigenous youth, demobilised young soldiers, and young migrant workers. Many developing countries, particularly those in Sub-Saharan Africa, Southern Asia, the Middle East and the Pacific Islands, are also experiencing a ‘youth bulge’ (that is, have two-thirds of their populations under the age of 30), a demographic shift which compounds what are already severely limited opportunities for integrating youth into the labour market. One billion young people will reach employment age within the next decade (ILO, 2012).

Rising levels of youth unemployment and underemployment impose heavy costs on individuals and their families, society and the economy. Both prolonged absence from the labour market and poor quality/low productivity jobs contribute to high levels of poverty. Over 40% of all young people live on less than $2 a day; in developing countries, youth are disproportionately among the working poor (ILO, 2012). This enormous unlocked potential represents a substantial loss of opportunities for economic growth. Increasing numbers of youth are moving to urban areas in search of employment (many cities in the developing south lack the infrastructure and resources to support large bursts of population growth). There are also concerns that rising levels of youth un/underemployment, and the social exclusion which results from prolonged frustration in the search for status and livelihood, may be a source of social and political instability and conflict, often in already unstable countries.

The labour productivity gap between developing and developed regions, although decreasing, continues to be significant (ILO, 2012). Education and training are widely perceived to be relevant to debates about productivity and competitiveness, with increasing emphasis being given to work- and skills-based solutions to economic competition and poverty. Following a decline in interest from the mid-1990s to the mid-2000s, technical and vocational education and training (TVET) has returned to the agenda of governments and donor agencies internationally, particularly in sub-Saharan Africa and South Asia (King and Palmer, 2010). The political and policy communities in many low-and middle-income countries remain attracted by the assumed link between TVET and a reduction in unemployment, through its equipping of individuals with relevant skills and knowledge, thus enabling them to respond to employment opportunities (World Bank, 2007). The latest UNESCO Global Monitoring Report highlights the policy importance now being placed on higher-order skills and the central place they play in the global knowledge-based economy, with regard to poverty reduction, economic growth and social stability (UNESCO, 2010a). This shift in priorities is exemplified by the theme of the 2012 Education for All (EFA) Global Monitoring Report (due to be published in September 2012), which will focus on TVET by exploring the links between skills programmes and employment (UNESCO, 2010b). TVET has become a key area for investment in developing countries and many initiatives have been implemented to address unemployment issues and improve economic growth. Local and national governments, private organisations and companies, national and international nongovernmental organisations (such as the Asian Development Bank, the International Labour Organisation, and the World Bank) and, on a more personal level, trainees themselves, have all made varying levels of investments in TVET programmes. For instance: in the Philippines around $200 million was invested by the government in TVET in 2002 (Péano et al, 2008); in Indonesia about $80 million dollars was invested by the Asian Development Bank and $35 million by the government in 2008 (ADB, 2008); in Uganda, participation in TVET programmes has increased by almost 50% within a decade (IMF, 2010).

The Intervention

There is no universally accepted definition of technical and vocational education and training (TVET). As a field, it is continually changing, usually in response to the demands made upon it (Maclean and Wilson, 2009). Broadly defined, TVET is concerned with the acquisition of knowledge and skills for the world of work. Here, we follow UNESCO's definition of TVET as ‘....a comprehensive term referring to those aspects of the educational process involving, in addition to general education, the study of technologies and related sciences, and the acquisition of practical skills, attitudes, understanding and knowledge relating to occupants in various sectors of economic and social life’ (UNESCO, 2010b). Various terms are used to describe the diverse elements of the field that are now conceived as comprising TVET, many of them specific to particular geographical areas (for example, in the United States, the current term is career and technical education). A great diversity of TVET models can be found worldwide, with no internationally accepted set of definitions of the different types that can be distinguished. For the purposes of this systematic review, the following definitions have been used:

Technical education: theoretical vocational preparation of students for jobs involving applied science and modern technology; compared to vocation education (which focuses on the actual attainment of proficiency in manual skills), technical education emphasises the understanding of basic principles of science and mathematics and their practical applications; delivered at (usually) upper secondary and lower tertiary levels to prepare students for occupations that are classified above the skilled crafts but below the scientific or engineering professions.

Vocational education: organised activities designed to bring about learning as preparation for jobs in designated (manual or practical) trades or occupations; traditionally non-theoretical and focused on the actual attainment of proficiency in manual skills; usually considered part of the formal education system and thereby falling under the responsibility of the Ministry/Department of Education.

Vocational training: prepares learners for jobs that are related to a specific trade or occupation, but (compared to vocational education) is better linked to the labour market and employment development system, and therefore usually falls under the responsibility of the Ministry/Department of Labour/Employment.

On-the-job training: workplace-based training that uses real jobs as a basis for instruction and for practical purposes.

Apprenticeship training: combines on-the-job training for a highly skilled craft or trade (from someone who is already a skilled leader in the field) with academic/theoretical instruction; also called dual-training programmes.

The organisation of TVET varies widely, both between and within countries. Brief overviews of three relevant TVET programmes in low- and middle-income countries follow.

Jóvenes en Acción (Youth in Action) was a Colombian training programme funded by the national government, and financed with loans from the World Bank and the Inter-American Development Bank. The programme was aimed at socio-economically disadvantaged young people in seven cities across the country with the aim of helping people who were hardest hit by the recession of 1998. It reached 80,000 young people (approximately 50 percent of the target population) and was given to various cohorts over a period of four years (2002-2005). Participants underwent a total of 6 months of training, formed of 3 months classroom-based training (various manual and administrative courses were offered) and 3 months unpaid on-the-job training. Training took place at private and non-profit training centres and legally registered companies. Private training institutions had to participate in a bidding process to be able to participate in the programme. Source: Attanasio et al. (2008, 2009, 2011)

The ongoing Technical and Vocational Vouchers Program (TVVP), which began in Kenya in 2008, offers young people the opportunity to undertake vocational and technical training through awarded vouchers which completely, or almost completely, cover the costs of training courses. Participants can use the vouchers to undertake training in either public or private institutions, on courses such as motor-vehicle mechanics and hairdressing. (NB: The impacts of participating in training are examined.) Source: Hicks et al. (2011)

Retraining programmes for laid-off workers (all ages) implemented following new national policy measures in China have been evaluated. The training was provided primarily by city labour bureaus, but also by private sector institutions (including city and district employment training centres, colleges, universities, and secondary technical schools). The quality of programmes varied across training institutions. Duration of training averaged one to three months. Most trainees contributed to the cost of their training. Courses on offer (some of which accommodated labour market demand for certain skills) included computing, cooking, beauty, massage and hair cutting, sewing, toy making, management, repairs and driving. Source: Bidani et al. (2002, 2005, 2009)

How the Intervention Might Work

This logic model provides a simple representation of the relationships among (i) the resources that are invested; (ii) the activities that take place; and (iii) the benefits or changes that result, as a sequence of events.

Employability refers to a person's capability of gaining initial employment, maintaining employment (including the ability to make transitions between jobs and roles within the same organisation to meet new job requirements) and/or obtaining new employment if required (Hillage & Pollard, 1998). It is therefore a concept that can be applied to both employed people seeking alternative jobs or promotion and unemployed people seeking work. The concept of employability has become a cornerstone of labour market policies and employment strategies internationally, with many policy makers viewing the development of individual employability as a crucial step towards improving access to employment and as a means of offering workers the opportunity to develop the skills allowing self-sufficiently within the labour market (McQuaid & Lindsay, 2005). Particularly outside policy circles, there is increasing recognition that employability is dependent not only on individual characteristics but also the environmental, social and economic context in which work is sought.

For all interventions, whether job creation is additional or not is a key issue. Most projects will have both positive and negative impacts. All of these should be taken into account in order to assess the net difference that results from the TVET intervention, over and above what would have taken place anyway - i.e., estimates of gross employment outcomes should be adjusted to take into account displacement and substitution effects.

•

Displacement effects: for example, where the setting up of new businesses has displaced less productive informal enterprises;

Whilst recognising that determining the ‘additionality’ of any employment effects is methodologically very challenging, it is important to be aware that even if effective in terms of employment outcomes, a TVET programme might not generate any additional employment if substitution or displacement effects are present. In both instances, TVET might only be affecting who gets employed, not the level of employment.

Why it is Important to do the Review

Despite considerable international evidence, the body of literature taking stock of the evaluation evidence on TVET and young people is relatively small. In many of the existing reviews, evaluations of training and retraining are presented alongside other typical active labour market programmes (ALMPs), such as employment services, public works, wage and employment subsidies and self-employment assistance. A minority extend previous reviews by explicitly considering the impacts for young people and/or for populations in developing and transition countries. A discussion of some of these reviews follows.

Kluve and Schmidt (2002) compared the results of a sample of European impact evaluations of ALMPs implemented between 1983 and 1999, and compared their results to the U.S. programmes previously studied by Heckman et al. (1999). Their analysis suggests mixed programme effects across categories of intervention and target population. Young workers were found to be the most difficult group to assist among the unemployed. Kluve (2006) followed this up with a meta-analysis of European ALMPs in the later 1990s and 2000s. More recently, Card et al. (2010) present the results of a meta-analysis of evaluations of ALMP impacts from 97 studies conducted between 1995 and 2007 (the vast majority set in high income countries). The sample is derived from responses to a survey of academic researchers affiliated with the Institute for the Study of Labour (IZA) and the National Bureau of Economic Research (NBER). The authors report that, when comparing across different participant groups, programmes for youths are less likely to yield positive impacts than untargeted programmes, although in contrast to some earlier reviews they find no large or systematic differences by gender. Within-country, cross-programme comparisons were undertaken by Greenberg et al. (2003) who synthesised findings from 15 publicly-funded training programmes in the United States to measure programme effects on participants' earnings. Results of their meta-analysis suggest highly heterogeneous earning effects among assisted groups. The overall training effect on youth was negligible, but some control variables showed small positive effects: (i) across training types, classroom skills training courses yielded consistently better effects than on-the-job training, while (ii) gender and race controls suggested lower effectiveness of training for white and female beneficiaries than for all other participants. A global review of skills development and transition to work (Van Adams, 2007) reports positive findings from evaluations of TVET programmes for youth, although again these findings are mostly from advanced countries.

There have been a number of reviews based on programmes in the World Bank's ‘Youth Employment Inventory’ database (a global inventory of more than 400 projects to support young workers in over 90 countries) including those focused specifically on young people and/or developing countries (see, for example, Betcherman et al., 2004, 2007; Fares and Puerto, 2009; Katz, 2008; Puerto, 2007; Stavreska, 2009). Betcherman et al. (2007) summarised information on a large number of international programmes supporting young people in their early years in the labour market. Although the largest concentration of included interventions is from OECD countries, there are also substantial numbers of programmes introduced in the largely middle-income countries of Eastern Europe and Central Asia, Latin America and the Caribbean. They found that training is the dominant form of intervention used to help young people improve their employment situation. For training as a whole, mixed results are reported, although their analysis shows an increasing incidence of positive impacts from programmes that offer multiple services, i.e., combinations of vocational training, job search assistance, entrepreneurial services, and so forth. Training programmes for youth seem to have a more positive impact in developing counties than in developed countries. More recently, Angel-Urdinola et al. (2010) analysed the main design features of non-publicly provided ALMPs in Arab-Mediterranean Countries, with a specific focus on initiatives targeted at youth. Benchmarked against international best practices, assessment of the programmes covered in the inventory reveals that the majority lack the necessary mix of design features that make such programmes effective.

Although there is growing consensus that TVET is important for economic growth and social cohesion, it is still not clear who should fund, provide and regulate it, or who should take it. Collecting evidence from studies that have analysed these issues is crucial for purposes of policy making. Since most prior reviews have focused on high-income countries and/or adults of all ages, there are clear grounds for concentrating this review solely on the effects of TVET programmes on youth in low- and middle-income countries. There is also motivation for this systematic review from a methodological perspective. Many of the existing reviews, although synthesising evidence from programmes which appear relevant to our review, are not based on a systematic search and several use a ‘vote-counting’ approach to synthesis. These are problems which this review will aim to remedy; in so doing, adding value to the existing body of research on this topic.

Criteria for Including Studies in the Review

Studies must meet the following eligibility criteria to be included in the systematic review.

Search Strategy

A comprehensive search strategy will be used to search the international research literature for qualifying studies. Different types of sources will be searched, including sources with a particular focus on low- and middle-income countries (some of which were sourced from the Cochrane EPOC Group's list of sources relevant to LMICs:

http://epocoslo.cochrane.org/lmic-databases). A number of European-focused sources have been included to assist the capture of relevant literature from ‘transition economies’ (countries in Central and Eastern Europe and the Former Soviet Union) and Turkey (see Appendix 1 for the World Bank list of low- and middle-income economies, grouped by region). The use of a wide range of sources is intended to capture both academic and ‘grey’ literature and reduce the omission of relevant studies, to ensure that our search is as unbiased as possible.

Description of Methods Used in Primary Research

Studies to include in the review will employ experimental or quasi-experimental research designs that compare outcomes for an intervention group to those for a control or comparison group. The following two studies exemplify the methods likely to meet the eligibility criteria for the review.

Attanasio et al. (2008, 2009, 2011) used a randomised design in which the availability of training was randomly assigned among those who chose to apply for training (using a centralised information system). Randomisation took place within institutions. The data set included treatment and control groups for each of the seven cities covered by the programme. Baseline and one follow-up survey were completed. The authors report that there was close to full compliance with random assignment and programme effects were first estimated using simple comparison of weighted means. To increase the precision of the estimates, they then included controls for observable characteristics (pre-treatment characteristics and course fixed effects) to help control for remaining baseline differences.

The study conducted by Bidani et al. (2002, 2004, 2005, 2009) was based on a quasi-experimental design whereby participants and comparison groups were selected after the programme had commenced. The study used a survey designed and implemented by the Chinese Institute of Labor Studies and the World Bank. Respondents to the survey were selected from lists of laid-off workers who had received training (treatment group) and laid-off workers who had not received the training (control group) from lists provided by the Labor Bureaus from two large cities, as well as local training institutions. One post-test measurement was taken. Adjustments were made in the estimation process to account for the differences in the characteristics of the participant and comparison groups. Impact estimates were computed by several different estimators: ordinary least squares (OLS)/linear probability model; probit model; propensity score matching (nearest neighbour; five nearest neighbours; kernel; local linear regression).

Criteria for Determination of Independent Findings

Sometimes, several different reports (i.e., publications) about a single study exist. In such cases, the most relevant report should be identified (for example, the publication containing the most complete data set) and used as the main record. The coding, however, should be done from the full set of relevant reports (to be identified before beginning to code). There are also occasions where a single report describes more than one study. In these cases, each study should be coded separately [as if they had come from separate report]. Efforts will be made to identify all affiliations between studies/reports before coding commences.

Information on study sample sizes, intervention details, grant numbers, and so on, will be used to identify multiple reports from single studies, and multiple studies in single reports. The authors of the reports will be contacted if it is unclear whether reports and studies provide independent findings.

We have specified two general construct categories: employment and employability. We plan to extract effects sizes for all relevant outcomes and follow-ups measured within the same study. These will be separated according to (i) the general constructs they represent (employment, employability) and (ii) the key outcome areas represented by these general constructs. It is not felt to be appropriate to calculate an overall effect size for employment and employability. Rather, we anticipate conducting separate meta-analyses of the key outcome areas (i.e., change in employment status, income level, qualifications, job performance and so on). We expect to encounter statistical dependence among study outcomes in these key areas. For example, some studies may report two measures of job mobility or have measured changes in income level in different ways. Dependence may also occur when primary research studies present comparisons of multiple treatment groups to a common control group (when only a single outcome is observed).

In cases where a single study provides more than one effect size for a particular outcome construct, to ensure that each study only contributes one data point to the analysis for each outcome (i.e., to ensure independency of the findings) we will adopt the approach of either dropping or combining outcomes. Dropping outcomes: If there are only a few instances of a single study measuring an outcome construct in more than one way, we will select the construct that is most similar to those used by other studies in that category and retain only that particular effect size in the analysis. Combining outcomes: For construct categories where single studies measuring an outcome construct in more than one way is relatively common, all the effect sizes will be retained in the analysis and we will use the technique developed by Hedges et al. (2010) to estimate robust standard errors that account for the statistical dependencies.

In cases where multiple measurements were assessed in a single study over time, we aim to analyse the outcomes by the duration of each of the follow-up periods (e.g., outcomes at six months, two years, etc.). If a sufficient number of such studies are available, we will also analyse outcomes by investigating the change in effect size over time. However, in the event that synthesising effect sizes separately at different points of duration is not feasible (e.g., not all studies may use common follow-up durations) we will form reasonable ranges of follow-up duration (e.g., short term 1-3 months, long term 9-12 months, etc.) rather than discrete follow-up duration time points.

Details of Study Coding Categories

Two reviewers will independently evaluate each study to capture both substantive and methodological characteristics. A draft version of sections A-E of the coding tool (data extraction form) is included in Appendix 3. Eligible studies will be coded on features such as year of publication, details of any linked publications, and so forth (section B), variables related to the characteristics of the study samples (section C), the nature of the intervention and its implementation (section D), and the study methods (section E). The coding process will also incorporate an assessment of the methodological quality and relevance of each study (section F). The different dimensions likely to be considered here include: study design (i.e., was the allocation mechanism or method of analysis able to control for selection bias), confounding (i.e., was the method of assignment and/or method of analysis executed adequate to ensure comparability of groups throughout the study and prevent confounding), contamination (spillovers and crossovers), performance bias, detection bias, construct validity, analysis and outcome reporting biases, statistical conclusion bias and external validity. We will draw upon the GRADE guidelines for grading the quality of evidence and strength of recommendations for practice and policy (Balshem et al., 2011). The statistical findings will be extracted (and effect sizes calculated) in section G of the tool.

We will collect data, if there is any, on net employment. Although net employment effects (i.e., those that take into account displacement and substitution effects) are seldom likely to be reported as an outcome, this issue has implication for interpreting programme effectiveness. Therefore, for each study we will record whether this issue has been addressed in any way, and/or whether the training took place in saturated/unsaturated markets, and so forth.

To enhance the usability of findings, we plan to draw upon an innovative approach to assessing applicability and transferability developed by Wang et al. (2005): where ‘applicability’ is defined as the extent to which an intervention process could be implemented in another setting, and transferability as the extent to which the measured effectiveness of an applicable intervention could be achieved in another setting. A range of intervention process and contextual factors will be collected accordingly.

We have used information in the primary study reports identified so far to identify potential effect size moderators; these include gender, education level, work experience, length of current employment status, type of TVET, duration of training, programme location (urban/rural), sector/industry, programme voluntariness and labour market conditions. We will assess the extent to which these moderating factors may be associated with heterogeneity in measured effects between studies.

The coding tool is based on previous EPPI-Centre tools, but has been modified where appropriate in accordance with Campbell guidelines. It also draws on previous work by Wilson et al. (2010). Reviewers will enter data directly into the EPPI-Reviewer 4 database, with the help of the coding manual where necessary. Piloting of the coding tool will be undertaken by members of the review team who will work independently on a random sample of eligible studies before meeting to compare their decisions. Reviewers will be retrained on any coding items that show discrepancies during this process and the coding manual adapted accordingly. This process will be repeated until a very high level of consistency in reviewers' application of the codes is achieved (at which point the tool will be finalised). The remaining studies will be double-coded. Different combinations of two reviewers (KH/CB, JT/CB, JT/KH) will independently extract information from each study report and then come together to compare their decisions (all sections of the tool, except H). Any uncertainties and discrepancies will be resolved by discussion, further review of the respective study reports and, where necessary, consultations with a third reviewer (JT or MN). Section H, which relates to the statistical findings of the primary studies and the calculation of effect sizes, will be completed by two reviewers (JT and MN): again, each study will be double-coded. Where additional guidance on statistical issues is required, we will draw on the expertise of Alison O'Mara-Eves and James Thomas at the EPPI-Centre.

The reviewers will attempt to contact the authors of studies that are missing key data that is essential for the review. Where relevant studies have been published in languages other than English, authors and funding sources will be contacted regarding the availability of translated versions. Should these be unavailable, we will seek to identify (through Campbell) additional reviewers who can use the coding tool to extract the relevant information.

Statistical Procedures and Conventions

Treatment of Qualitative Research

Qualitative research will not be included in this review.