PROTOCOL: Work programmes for welfare recipients

Abstract

Linked article:

Background

The problem: Long-term welfare recipiency is a source of vast costs for society, and welfare recipients are over represented regarding economic, mental and social problems. Data from 2002 show that in the UK 21 percent of household income came from social benefits [National Statistics 2002]. In 1998 the benefit recipiency ratio varied between approximately 20 percent in Spain and the United States and 57 percent in Belgium [OECD website]. In 1995, the US federal government spent 1,505 billion dollars on social welfare under public programs [U.S. Census]. This is an enormous burden on society and it undermines social norms of self-sufficiency. On an individual level it causes poverty and social exclusion, reduced well-being and quality of life, and it threatens social identity. Welfare recipiency therefore receives increasing interest by policy makers and politicians [Lødemel 2001] [Martin 1998]. In most developed countries, there is some kind of unemployment benefit, but this is only accessible to people who have had some earlier employment, and even then these benefits are usually time limited. Most developed countries have some kind of welfare benefit or social assistance for unemployed people who are not entitled to unemployment benefit.

Rationale for intervention: In order to reduce the extent of long-term welfare recipiency, there is increasing consensus among policy- and decision makers in developed countries that people should, as a rule not passively receive benefits if they have some ability to work. It is believed that some kind of work-related activation will help welfare recipients develop the skills needed to enter the regular job market. Therefore, many countries demand that (some) receivers of welfare benefits do some kind of work. These programmes are often labelled “workfare” or “welfare-to-work”. Other aims include increasing quality of life, self-confidence, skills and work morale. Thus, for some persons the goal is to obtain work, but for others the goals are more modest like social inclusion, quality of life, etc.

Levitas [1998] distinguished among several discourses about persons outside the labour market. The moral underclass discourse (MUD) implies a view of social assistance recipients as lazy. If the welfare benefits are too generous, they will have no motivation to seek employment. This discourse is closely tied to a paternalistic discourse (PAD) in which the welfare bureaucracy must impose strict work requirements [Dahl, 2003]. It also blames the welfare recipiency on the individual client. In Europe, there has been more recognition of structural causes of welfare recipiency, such as lack of jobs. The state has an obligation to provide jobs, and the clients are expected to take the available jobs. So the relationship between the individual and the state is one of reciprocity. Levitas labelled this the social integrationist discourse. Seen from this perspective, the compulsory work programs, sanctioning, and time limits on cash assistance that are studied in the present review, imply that the problem of long-term social recipiency is described using the MUD and the solution is described using the PAD [Dahl, 2003].

There are at least three theories about mediating variables: (1) maintenance and teaching of skills (vocational and social); (2) strengthening of motivation and work morale; and (3) strengthening of self-efficacy. It is difficult to compare effects of interventions across different countries. Generally, effects vary not only with the nature of the intervention, but also who participate, the conditions of the local labour market, and how other welfare institutions are shaped and organised.

Aspects of programmes: Lødemel and Trickey (2001) listed three basic elements of ‘workfare’ – that it is compulsory, that it is primarily about work, and that it is essentially about politics tied to the lowest tier of public income support. The programmes vary on several dimensions: (1) They are either administered by the state or by the municipalities, (2) jobs are either in the public or the private sector, (3) individuals are either directly placed or the placement is preceded by some kind of training period, (4) the main aims are to obtain competitive work or to enhance quality of life and social integration.

Rationale for this review: We are not aware of any systematic reviews of the effects of work programmes for welfare recipients worldwide. Some overviews of controlled trials have been performed (mostly in the USA) [Fischer 1995] [Michalopoulos 2000]. In Europe, a recent collaborative overview [Cornwell 2002] was performed in six European countries (Denmark, France, Germany, the Netherlands, Norway, and the UK). The results were not directly comparable because the countries have different ways of organising their social services, and they had somewhat different methodological approaches. The group of individuals on welfare differs among countries. In the USA, welfare is mostly provided to poor families with children. In 1996, the Temporary Assistance for Needy Families (TANF) program replaced the Aid to Families with Dependent Children (AFDC) as the nation's main welfare program. Most TANF recipients are single mothers, but there are also some families with two unemployed parents. Some countries (like the USA and the UK) do not have a sharp distinction between unemployment benefit and social assistance benefit, while other countries (for instance Norway and Denmark) distinguish between individuals who are entitled to unemployment benefit if they have been employed in the past, and individuals who are only entitled to social assistance. In these countries, all citizens have the right to receive social assistance benefit if they have no other source of income.

Objectives of this review

(1)

To study the effects of work programmes for welfare recipients on work status, earnings, welfare receipt, and total income

(2)

To explore subgroup differences. The review will try to identify approaches that appear to be more or less effective overall and, to the exent that adequate data are available, for subgroups.

(3)

To identify any adverse effects of such programmes (e.g. displacement or negative effect in income or work status).

Methods

Criteria for inclusion and exclusion of studies for this review

Study design

Included designs

Studies using the following designs are eligible for a quantitative review on the effects and impacts of work programmes for welfare recipients:

Randomised controlled trials (RCTs)

Cluster randomised trials

Quasi randomised trials

Excluded designs:

Studies using the following designs will be collected, listed, and described in an explorative review of associations among participation in work programmes and various outcomes:

Non-randomised concurrently controlled studies (two groups assigned by the investigators and dependent variables measured on both groups before and after the intervention)

Cohort studies (two groups defined by the interventions and dependent variables measured on both groups before and after the intervention).

We will not restrict country of publication. There will be no language restrictions.

Participants

Welfare recipients who are potentially able to take part in some kind of work. We will exclude persons entitled to unemployment benefit and persons with pensions of any kind. If only some participants in the study are welfare recipients, the study is only included if at least 50% receive welfare benefit.

Interventions

Interventions intended to help welfare recipients become self-sufficient typically come as “packages” involving several components. Therefore, it is usually not possible to measure the effect of work experience apart from the effect of the other components of the program. We include interventions that offer time limited work experience alone or together with other components, such as job searching, education, job clubs, or parental training. We exclude studies where the program does not offer time limited work experience. The intervention may or may not be compulsory.

The term “time limited work experience” needs some clarification. First, the work experience is not a steady job. The duration of the work must be explicitly stated. This duration will be recorded for each study. Second, the work must not involve a job acquired through channels external to the program under evaluation. In other words, the job must be program generated.

Control/comparison groups

The main control comparison would be a group of persons receiving ordinary (passive) social economic assistance. If possible, the relative efficacy would be considered by comparing two different work programmes.

Outcome measures

All outcomes will involve a comparison between different groups. The comparisons will be either post intervention measures or change scores (difference between post- and pre scores). Accepted sources of outcome data will be register data, recordings by program personnel or employers and self-reports. Each data source has strengths and limits. Whereas register data are relatively free from recall- and expectancy biases, they will miss e. g. employment in jobs that are not reported to the authorities. The type of data source will be recorded for each study.

PRIMARY OUTCOMES

(1)

Work status (a)

proportion of attendants who obtain competitive work (= work with standard wages and which anyone can apply for).

(b)

duration of employment during a given time interval.

(c)

elapsed time until obtained work

(d)

duration on welfare benefit during a given time interval (as above)

(2)

Earnings (a)

mean or median earnings at different time points

(b)

total individual earnings (e.g., income + social insurance + social assistance + welfare benefit)

(c)

total household earnings

SECONDARY OUTCOMES

(3)

Skills and satisfaction (a)

quality of life

(b)

social skills

(c)

self confidence

(d)

social activity, participation and integration

Adverse effects (a)

crime

(b)

drug and alcohol use

(c)

displacement (when program participants get jobs, this worsens the chances of non-participants to get jobs)

(d)

decrease in combined income

(e)

other

Follow-up durations

Because duration of follow-up is expected to vary across studies, the exact duration of follow-up will be recorded for each study. After data collection is finished, duration times can be analysed based on the available data. They can for example be grouped into short, medium and long follow-up durations.

Search strategy for identification of relevant studies

search strategy for electronic database searches:

We will search the following databases: C2-SPECTR, CENTRAL, MEDLINE, EMBASE, PsycINFO, Sociological Abstracts, Cinahl, Caredata, BIBSYS, SIGLE (for grey literature), IBSS, and Social Science Citation Index, with no restriction on publication date.

We will use the same text words across all databases and use the specialised controlled vocabularies for each database. The search strategy will be iterative, i.e. new search terms will surface as the reports accumulate.

The following is the search strategy that will be used to search Sociological Abstract:

Social Security/

Social Welfare/

Social Support/

Benefits/

Welfare Services/

Welfare Recipients/

Financial Support/

(social adj1 assistance$).tw.

(social adj1 securit$).tw.

(social adj1 welfare).tw.

(social adj1 allowance$).tw.

(insurance$ adj1 benefit$).tw.

(social adj1 benefit$).tw.

(welfare adj1 benefit$).tw.

(welfare adj1 payment$).tw.

(welfare adj1 recipient$).tw.

(welfare adj1 support$).tw.

(economic adj1 support$).tw.

(public adj1 assistance$).tw.

(public adj1 support$).tw.

(financial adj1 support$).tw.

(welfare adj1 service$).tw.

(direct$ adj1 payment$).tw.

tanf.tw.

afdc.tw.

temporary assistance to needy families.tw.

aid to families with dependent children.tw.

eitc.tw.

earned income tax credit.tw.

food stamps.tw.

(general adj1 assistance).tw.

(cash adj1 assistance).tw.

(income adj1 assistance).tw.

wic.tw.

(special supplemental food program for women infants and children).tw.

or/1-35

Vocational Rehabilitation/

Workfare/

Employee Assistance Programs/

Job Training/

Employability/

(vocation$ adj1 rehab$).tw.

(occupation$ adj1 rehab$).tw.

employability.tw.

(subsidi?ed adj1 employment).tw.

(employ$ adj1 incentive$).tw.

(employ$ adj1 program$).tw.

(employ$ adj1 scheme$).tw.

(employ$ adj1 training).tw.

(support$ adj1 employ$).tw.

(employ$ adj1 rehab$).tw.

(target$ adj1 employ$).tw.

(subsidi?ed adj1 job$).tw.

(job$ adj1 incentive$).tw.

(job$ adj1 program$).tw.

(job$ adj1 scheme$).tw.

(job$ adj1 training).tw.

(job$ adj1 creation$).tw.

(support$ adj1 job$).tw.

(job$ adj1 rehab$).tw.

(job$ adj1 search$).tw.

(job$ adj1 applicat$).tw.

(subsidi?ed adj1 work).tw.

(work$ adj1 incentive$).tw.

(work adj1 program$).tw.

(work adj1 scheme$).tw.

(work adj1 training).tw.

(support$ adj1 work$).tw.

(work$ adj1 rehab$).tw.

(work adj1 approach$).tw.

(work$ adj1 relief).tw.

(training adj1 program$).tw.

human capital development.tw.

hcd.tw.

WIN.tw.

JOBS.tw.

(job opportunity and basic skills program).tw.

(employment adj1 initiative$).tw.

(employment adj1 experience$).tw.

(employment adj1 experiment$).tw.

(work$ adj1 initiative$).tw.

(work$ adj1 experience$).tw.

(work$ adj1 experiment$).tw.

(GAIN adj1 program$).tw.

(independence$ adj1 program$).tw.

(independence$ adj1 demonstration$).tw.

FTP.tw.

family transition program$.tw.

FIP.tw.

family investment program$.tw.

(welfare adj1 restructuring).tw.

(welfare adj1 reform).tw.

(ABC adj1 program$).tw.

(better chance adj1 independence program$).tw.

or/37-94

36 and 95

welfare to work.tw.

workfare$.tw.

or/96-98

Other sources of information:

In addition, references from included primary reports and relevant reviews will be scanned. Authors of included studies and other potential experts in the field will be contacted.

The following websites will be searched:

Abt Associates (USA)

(http://www.abtassociates.com)

Department for Work and Pensions Social Research Branch (UK)

(http://www.dwp.gov.uk/asd/asd5/index.html)

Joseph Rowntree Foundation (UK)

(http://www.jrf.org.uk/knowledge/findings/)

Manpower Demonstration Research Corporation (USA)

(http://www.mdrc.org)

Mathematica Policy Research (USA)

(http://www.mathematica-mpr.com/)

National Centre for Social Research (UK)

(http://www.natcen.ac.uk/natcen/pages/op_employment.htm)

National Institute for Social Work (UK)

(http://www.nisw.org.uk/about.html)

RAND Corporation, Social Welfare (USA)

(http://www.rand.org/research_areas/population/)

Regard (UK)

(http://www.regard.ac.uk/regard/home/index_html?)

Social Work Research Centre (UK)

(http://www.stir.ac.uk/Departments/HumanSciences/AppSocSci/swrc/researchreports.htm)

The United States General Accounting Office (USA)

(http://www.gao.gov/)

Urban Institute (USA)

(http://www.urban.org/)

The study bibliographic information will be registered in Reference Manager.

Criteria for evaluating eligibility of retrieved studies

Two reviewers will independently scan the abstracts and titles of retrieved reports for eligibility, according to the inclusion criteria above. Full copies of all those deemed eligible by one of the reviewers will be retrieved for closer examination (item 3 below). Consensus will be reached by discussion and consultation with a third reviewer, if necessary. All studies which initially appeared to meet the inclusion criteria but, based on the full text reports, do not meet the inclusion criteria will be detailed in the table of excluded studies (item 7 below).

1. Potentially relevant studies identified and screened for retrieval (n = …)

2. Studies excluded with reasons (n = …)

3. Studies retrieved for more detailed evaluation (n = …)

4. Studies excluded with reasons (n = …)

5. Potentially appropriate studies to be included in the systematic review (n = …)

6. Studies included in the systematic review (n = …)

7. Studies excluded from the systematic review with reasons (n = …)

(Adapted from the QUOROM flowchart [Moher 1999])

Quality assessments

We will assess components that contribute to the measured effectiveness of interventions. Two reviewers will independently assign each selected study to quality categories described below. Uncertainty or disagreement is solved by discussion with a third reviewer.

Generation of allocation sequence

MET = Resulting sequences are unpredictable (explicitly stated use of either computer-generated random numbers, table of random numbers, drawing lots or envelopes, coin tossing, shuffling cards, or throwing dice).

UNCLEAR = Vague statement that the study was randomised but not describing the generation of the allocation sequence or statement(s) indicating that random allocation was used in some but not all cases.

NOT MET = Explicit statement that the study was not randomised OR explicit description of inadequate generation of sequence, (e.g., using case record numbers, alternation, date of admission, date of birth).

Concealment of allocation sequence

MET = Participants and investigators cannot foresee assignment, e.g. central randomisation performed at a site remote from trial location; or use of sequentially numbered, sealed, opaque envelopes).

UNCLEAR = Vague statement that the study was randomised but not describing the concealment of the allocation sequence.

NOT MET = Explicit statement that allocation was not concealed OR statement indicating that participants or investigators can foresee upcoming assignment (e. g., open allocation schedule, unsealed or non-opaque envelopes).

Control of initial difference in prognostic factors between groups

In a properly randomised study, all initial differences between groups will be caused by chance. This applies to all prognostic variables, both known and unknown. But in non-randomised designs, there may be important initial differences between groups. These differences can be systematic, and they can appear in unmeasured variables as well as in the measured ones. It is generally possible to control for the latter but not the former. Matching can be used before the intervention to make groups more similar, and regression methods can be used after the intervention to control for initial differences, but all these methods may introduce bias in the results [ Deeks 2003 ].

Studies in which both generation and concealment of allocation sequence are MET, will be coded as MET below.

MET = Control for one or more prognostic factors. Also score MET when there is no control for prognostic factors because there was no imbalance in measured variables.

UNCLEAR = Sufficient information could not be obtained.

NOT MET = Imbalance in prognostic factors and failure to control for this imbalance.

Prevention of Performance Bias

MET = Other interventions avoided or used similarly across comparison groups.

UNCLEAR = Use of other interventions not reported and cannot be verified by contacting the investigators.

NOT MET = Dissimilar use of other interventions across comparison groups, i. e. differences in the care provided to the participants in the comparison groups other than the intervention under investigation.

Prevention of Detection Bias

MET = Assessor unaware of the assigned treatment when collecting outcome measures. Also score as met if outcome is questionnaire data or register data.

UNCLEAR = Blinding of assessor not reported and cannot be verified by contacting investigators.

NOT MET = Assessor aware of the assigned treatment when collecting outcome measures.

Prevention of Attrition Bias

MET = Losses to follow up less than or equal to 20% and equally distributed between comparison groups (proportion of total loss to follow-up equal to or less than 60% in group with the highest loss to follow-up).

UNCLEAR = Losses to follow up not reported.

NOT MET = Losses to follow up greater than 20% or not equally distributed between comparison groups.

Intention-to-treat

MET = Intention to treat analysis performed or possible with data provided.

UNCLEAR = Intention to treat not reported, and could not be undertaken by contacting the investigators.

NOT MET = Intention to treat analyses not done and not possible for reviewers to calculate independently.

Data management and extraction

Two reviewers will independently extract information from the full text report on study characteristics using a specially designed pretested data extraction form. For cases in which outcome information is missing from the original reports, attempts will be made to retrieve the necessary data for the analysis from the original investigators. Inter-rater agreement (i. e. coding reliability) will be assessed and reported as Kappas. The rate of agreement will be reported separately for all items to avoid inflation with study characteristics that generally achieve perfect agreement (e. g. year of publication). Disagreements will be resolved by meeting and discussing coded items. Data will be entered into Comprehensive Meta-Analysis [Borenstein & Rothman, 1999].

Data synthesis

Continuous data

Continuous measures (e.g., earnings) will be calculated as weighted mean differences if all outcomes are reported on the same scale of measurement and standardised mean differences when results are reported on different scales. We will report the 95% confidence intervals for all of the above.

Discrete outcomes

We will express binary outcome measures (e.g., employed/unemployed) as risk ratios (relative risks). In some cases, it might also be feasible to report the number needed to treat (NNT).

Identifying heterogeneity

We will use the following methods to assess heterogeneity (1)

Common sense (e. g. are the interventions, participants or outcomes so different that they cannot be combined?) This will be based on a synthesis of the process elements.

(2)

Chi-square test for heterogeneity (p<0.10) and I-Squared.

(3)

Visual examination of graphs for outliers and between study differences.

Exploring heterogeneity

If heterogeneity is judged to be large on the basis of (1)-(3), we will examine potential sources using the following steps:

Subgroup analysis

Meta-regression

Sensitivity analysis

Subgroup analysis

We will consider subgroup analysis across the following factors: unemployment level in the study area, placement in the private or public sector, direct placement or prevocational training, type, intensity or length/period of the intervention.

Meta-regression

If useful, and after consultation with a statistician, we will conduct meta-regression to look at the relation of size of effect to characteristics of the trials.

Sensitivity Analysis

Sensitivity analysis will be used to evaluate whether the pooled effect sizes are robust across components of methodological quality. For methodological quality, we will consider sensitivity analysis for each major component of the quality checklists.

Fixed vs. Random Effects Models

We will use random effects models when heterogeneity cannot be explained by subgroup analysis or meta-regression, and when we also want to be able to generalise the findings to a universe of similar (unobserved) studies as the obtained sample. Fixed effects models will be used either if heterogeneity is ignorable or when we only want to summarise the data for the obtained studies (i.e. we do not aim to generalise).

Criteria for determination of independent findings

In many instances, several different outcome data are measured on the same subjects in the primary studies (e.g. employment status and earnings). Sometimes the same outcome is measured at multiple points in time. Because these data are from the same sample of participants, and, therefore, are not independent estimates of treatment effect, we will analyse the data in such a way that any one analysis will contain a single outcome from a single point in time.

Details of study coding categories

Study characteristics: Country (or countries) of origin, year of publication, publication type, trial quality (see above), employment rate in the area.

Design: randomised controlled trial, non-randomised controlled study, prospective cohort study.

Participants: age, socio-economic status, ethnicity, gender, education level, number of children, age of youngest child, previous work experience, welfare history.

Intervention: employment-focused or education-focused, job search first or varied first activity, level of enforcement, duration of intervention, extent of work (e.g. working hours per week), proportion in intervention group and comparison group that participated in unpaid work, funding agent and implementing agent.

Statistical procedures and conventions

Results will be analysed using Comprehensive Meta-Analysis software [Borenstein & Rothman, 1999]. Prognostic variables will be used in analyses if there are less than or equal to 20% missing data. Graphical presentations of effects will be produced using Comprehensive Meta-Analysis.

Timeframe

Time Period	Benchmarks
February 2003	First submission of this protocol
March - May 2003	Searches for published and unpublished studies. Retrieval into RefMan 10 database Pilot testing of inclusion criteria
June – September 2003	Scanning of titles and abstracts of search hits.
October 2003	Resubmission of this protocol
October 2003-March 2004	Reading of retrieved reports. Final inclusion/exclusion. Extraction of data. Statistical analysis
December 2003	Second resubmission of this protocol
March 2004	Third resubmission of this protocol
March – December 2004	Preparation of report
July 2004	Fourth resubmission of this protocol
December 2004	Submission of review to Campbell Social Welfare Review Group

Plans for updating the review

Searches will be performed every two years after publication of the full review. Results of searches will be published and classified either as (1) search performed [date] but no new studies found, (2) minor update: new studies found, but conclusions are unchanged, or (3) major update: new studies found and consumers who read the previous version are advised to read the whole update.

Potential conflicts of interest

None known.

Footnotes

National Statistics

()

OECD Website ()

US Census Bureau

References

Borenstein

Rothstein

(1999). Comprehensive meta-analysis. A computer program for research synthesis. Englewood, NJ: Biostat.

Cornwell

Enjolras

Fraisse

Lødemel

Olsen

Rosdahl

Smith

Spies

Stafford

Trickey

Voges

Walker

Vannevjen

Weise

. Workfare in six European nations. Findings from evaluations and recommendations for future development. 2002.

Dahl

(2003). Does ‘workfare’ work? - the Norwegian experience. International Journal of Social Welfare, 12, 274–288.

Deeks

Dinnes

D'Amico

Sowden

Sakarovitch

Song

Petticrew

Altman

, 2003, Evaluating non-randomised intervention studies: Health Technology Assessment, v. 7, p. 1–188.

Fischer

, 1995, Job Training as a Means to “Ending Welfare as We Know It”: A Meta-Analysis of U.S. Welfare Employment Program Effects: Dissertation Abstracts International, v. 1995, 56, 1, July, 371-A–372-A.

Levitas

(1998). The inclusive society? Social exclusion and New Labour. Hampshire: Macmillan Press.

Lødemel

Trickey

, 2001, An offer you can't refuse. Workfare in international perspective, Bristol, Policy Press.

Martin

. What works among active labour market policies: evidence from OECD countries' experiences. 35, 1–33. 1998. Directorate for Education, Employment, Labour and Social Affairs, Employment, Labour and Social Affairs Committee, OECD. Labour market and social policy - occasional papers.

Michalopoulos

Schwartz

Adams-Ciardullo

. National evaluation of welfare-to-work strategies. What works best for whom: Impacts of 20 welfare-to-work programs by subgroup. 2000. U.S. Department of Health and Human Services, Administration for Children and Families, Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Education, Office of the Under Secretary, Office of Vocational and Adult Education.

10.

Moher

Cook

D J

Eastwood

Olkin

Rennie

Stroup

D F

, 1999, Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement: Lancet, v. 354, p. 1896–1900.