Abstract
BACKGROUND
The Problem, Condition or Issue
Research suggests that social cohesion facilitates a wide /range of economic, social and political outcomes (Prewitt, Mackie and Habermann, 2014). For example, Knack and Keefer (1997) and Zak and Knack (2001) find that a country's level of trust is correlated to its rate of growth. Cross-country studies using pooled data from both high- and low- and middle-income countries (L&MICs) highlight the importance of social cohesion for the building of quality institutions, which in turn affect economic development and income inequality (Easterly, Ritzan and Woolcock, 2006). Conversely, cross-country evidence suggests that societies with fragile public institutions and fractioned societies respond worse to economic shock than those with high quality institutions and more social cohesion (Rodrik, 1999). Moreover, based on theoretical arguments and empirical evidence from both high-income countries (HICs) and L&MICs, Portes and Vickstrom (2011) argue that trust in political and civic institutions is necessary for democracy to work.
Social solidarity and social cohesion as concepts can be traced back to Durkheim's classical work, where organic solidarity is understood as an outcome of social interactions generated by the division of labour in modern societies (Durkheim, 1893). The concept of social cohesion has received increased attention from academics (Sen, 1999; Easterly, Ritzan and Woolcock, 2006) and multilateral actors (World Bank, European Union, OECD, Canadian Heritage or ECLAC) in recent decades. Social cohesion here is the interdependence among individuals, which occurs as a result of these interactions.
Social solidarity, social cohesion and social capital are multidimensional. In the contemporary debate they are related to political, institutional and interpersonal trust or stigma (as the opposite feeling to trust), social connectedness, civic engagement and collective action (King et al., 2010). Definitions of social cohesion place different emphasis on shared values, inequalities and civic engagement (Colletta and Cullen, 2000; Green, Janmaat and Han, 2009; Portes and Vickstrom, 2011). In all cases, however, social solidarity/cohesion entails cooperative behaviour among individuals and collectives, with a sense of value-based commitment (Wilde, 2007; Thome, 1999) and mutual esteem (Honneth, 2007; Wilde, 2007). 1
Recent scholarship on social cohesion also highlights the role of norms that are understood and accepted by citizens, and enforced by specialized agencies. For instance, Portes and Vicsktrom (2011) argue that in modern societies, trust depends on universalistic rules and on the capacity of institutions to enforce their observance. Here we use social solidarity and social cohesion interchangeably 2 , defining them as the attitudes and actions of individuals and groups towards other individuals, groups and institutions reflecting a cooperative disposition, commitment and esteem.
The Intervention
Social solidarity relies on strong social norms and state action to function (Green, Janmaat and Han, 2009; Portes and Vickstrom, 2011). Levels of social cohesion may change over time, including as an intended or unintended effect of government programmes. In this review we focus on the effects of social assistance / welfare provision on social cohesion (Green, Janmaat and Han, 2009; Larsen, 2013).
Cash transfer programmes are designed to provide social assistance and alleviate poverty and they remain a significant component of welfare policies in many HICs (Barrientos, 2013). In L&MICs these interventions have historically been marginal, but in recent decades have become increasingly widespread. Typically, households or individuals receive cash provided through a direct transfer by the government or a running NGO. In some cases, cash transfers are complemented by in-kind transfers, workfare or educational interventions.
Programmes can be universal, but are more often targeted at specific population or socio-economic groups, often at those experiencing higher levels of deprivation. Universal cash transfer programmes (such as the Alaska Dividend) are delivered to all households in a certain country, whereas targeted programmes (such as Progresa/Oportunidades in Mexico) are provided for specific population groups, with beneficiaries usually selected on the basis of proxy-means or means tests (Makandawire, 2005). In practice, there are many intermediate situations and many programmes are situated on a continuum between universal and targeted cash transfers with different emphasis (Esping-Andersen, 2005). This is the case with programmes directed at specific population groups, such as the case of child benefits in the UK or Sweden.
Cash transfer programmes can be unconditional or conditional when they require the completion of certain household behaviors (Barrientos, 2013). In HICs, conditionalities most commonly refer to employment (Makandawire, 2005). For instance, the family allowances programme Temporary Assistance for Needy Families (TANF) in the USA conditions recipients to find a job within two years of receiving the transfer and participating in meetings and workfare programmes for at least 30 hours a week (Loprest, 2012). In L&MICs, conditional cash transfers (CCTs) are often offered to households on the condition that they comply with certain requirements, often children's school attendance, in some cases mothers attending health clinics at regular intervals, or other such requirements (Fiszbein et al., 2009).
Sometimes, cash transfer programmes can be integrated into more general initiatives involving complementary interventions aimed at improving education, health or social participation outcomes. Many cash transfer programs explicitly aim to contribute to the improvement of social ties by strengthening informal networks and promoting effective access to other public programmes (Attanasio, Reyes and Pellarano, 2009). Programmes may have specific educational and training components designed to foster these outcomes.
We restrict the interventions of interest to public or NGO run programmes that aim to deliver cash to households or individuals in vulnerable or deprived population groups. We exclude student loans and unemployment benefits.
How the Intervention Might Work
Social protection can affect social solidarity either directly or indirectly, with its intended outcomes affecting factors including poverty and inequality. To assess potential links between social protection and social solidarity, we consider three main dimensions or groups of social cohesion outcomes: i) interpersonal trust, attitudes toward other persons and stigma; ii) political and institutional trust; iii) social connectedness, civic engagement and collective action.
As suggested by figures 1 and 2, cash transfers can affect these dimensions through several pathways. Intended and unintended effects on social cohesion outcomes may be related to programme design and implementation, particularly in the case of targeted benefits. We discuss targeted and universal programmes separately as the mechanisms at may are different.
Universal cash transfers and social solidarity: outcomes and moderators
Targeted cash transfers and social solidarity: outcomes and moderators
a) Universal programs
Figure 1 depicts the potential pathways between universal programmes and social cohesion outcomes. It has been argued that universal programmes favour social cohesion as they may generate a sense of equal treatment. People's perceptions of equal treatment might strengthen interpersonal (both within and between groups) and political and institutional trust (Kumlin and Rothstein, 2005; Watson, 2014). Hence, receiving a direct transfer might lead to an increase in trust in all its forms (Figure 1). Stigma, which we use to be the opposite feeling to interpersonal trust, might also be low as all individuals will be entitled to receiving the benefit (Van Parijs, 2004).
Another strand of the literature argues that universal programmes, and particularly the welfare state, might substitute existing bonds among individuals and erode civil society cohesion (Cohen and Arato, 2000). In this case, social participation and associational life might be discouraged as individuals find that their needs are already provided by the state (Figure 1). The effects therefore would exhibit the opposite signs.
A final pathway relates to the political economy of the benefit scheme is depicted at the bottom of Figure 1. Universal schemes involve significant budget allocations that are mainly funded through taxes. If taxpayers perceive that they are contributing to inequality and poverty reduction, and they agree on how the government allocates social spending, they will support the welfare regime (Green, Janmaat and Han, 2009). However, if tax payers perceive that public cash transfers are being used to consume “bads” or contribute to the reduction of work effort, they might lose trust in institutions and other population groups (Alesina and La Ferrara, 2002). In this case taxpayers may favour political programmes that promote budget cuts, placing the long-run sustainability of the transfer scheme at risk.
As the causal links hypothesized in the previous paragraphs are conflicting in terms of the final effects of cash transfers on social cohesion, we assume potential causality from cash transfers to social cohesion, but the sign and size of the effect cannot be predicted ex-ante.
b) Targeted programs
Targeted cash transfer programmes can affect social solidarity outcomes in different ways (Figure 2). Effects can arise from the targeting process itself (b.1, King et al., 2010) or as a consequence of the degree to which the main purposes of the intervention are accomplished (b.2). As targeted programmes create a sharp division between population groups, the final impacts on social cohesion outcomes might be different among beneficiaries, non-eligible vulnerable populations and middle and higher socio-economic strata.
b.1) Targeting mechanisms
Levinson and Raharda (2004) point out that being selected to receive a periodic transfer can make individuals or households feel more confident about the future and strengthen links with the state and community members. Among the eligible population, this can lead to a positive effect in interpersonal, institutional trust and a predisposition to collective action as a direct effect of being selected as programme beneficiary. However, depending on the target population and on the targeting process, there may be differing effects on intra-group trust and inter-group trust.
Different reactions might emerge among the vulnerable, middle and upper income strata groups. Depending on the prevalent attitudes towards the causes of poverty, leaving vulnerable groups out of the target population might generate resentment for being excluded from the benefits. Alternatively, they might be proud of not being labelled as individuals or households in need (potentially a stigmatized position) or they might support the transfer scheme. In the first case, social cohesion would be likely negatively affected as inter-group trust and political trust might fall, probably also provoking this group's withdrawal from collective action or generating intra-group solidarity. The latter effect might generate a reduction in social cohesion. Among the middle and upper socio-economic strata, effects will depend on their prevalent attitudes towards poverty and their perception of the outcomes of the intervention. This point will be re-addressed at the end of this section.
In all cases, the process of publicly identifying a certain population as ‘poor’ might create or foster stigma, which has been acknowledged as one of the main social and psychological costs of targeting (Coady, Grosh and Hoddinot, 2004). The degree of stigma associated with being a beneficiary of a certain programme depends on how the previous culture of the country perceives the causes of poverty (individual or social) and pre-existent stigma towards the poor (Rainwater, 1982); inequality levels; preferences for redistribution; and specific programme features (Coady, Grosh and Hoddinot, 2004; Stuber and Schlesinger, 2006).
The specific forms of public outreach, application, and selection processes can make participation more or less prone to stigma. 3 If programme take-up is reduced due to stigma, a proportion of the eligible population might not be receiving the benefit and thus the intended outcomes of the intervention might be at risk or effects could be reduced compared to what's expected. This also implies that individuals that should be treated similarly ex-ante, would receive unequal treatment. However, if the programme is successful in promoting self-esteem and self-efficacy, it might counteract stigma (Stuber and Schlesinger, 2006) and influence social cohesion positively.
While the success of the process cannot be attributed to a certain targeting method and highly depends on the particular institutional setting in which the programme is launched (Coady, Grosh and Hoddinot, 2004), the implications of the process on social cohesion might vary. Targeting methods can be broadly classified into three main groups: individual/household assessments (means tests, proxy-means tests and community based targeting), categorical targeting (for example, based on demographic attributes such as age, ethnicity, gender, work status, disability or geographical criteria) and self-selection (Coady, Grosh and Hoddinot, 2004).
Some authors point out that group and area targeting might generate less suspicion of the whole process, but that it may stigmatize a particular group, reducing inter-group trust and increasing intra-group trust (King et al., 2010). In turn, this can redirect the purpose and scope of collective action and even foster conflict.
Meanwhile in means and proxy means-tested benefits each case must be tested individually, and suspicions of cheating, arbitrariness and discrimination can be raised (Kumlin and Rothstein, 2005). 4 If this perception is generalized, trust in institutions may decline among the three groups (eligible, vulnerable and middle and upper sectors), and interpersonal trust might be at risk. However, if the process is perceived as transparent and fair, confidence in institutions might remain unchanged or even increase.
In community targeting, the government allows the community to select programme beneficiaries. In this case, previous information on the situation of each household and poverty concepts that prevail in the community might be at play (Coady, Grosh and Hoddinot, 2004). Alatas (2012) carried out a RCT in Indonesian villages comparing the performance of proxy-means-tests and community targeting, and they found that poverty effects were not statistically different between the two methods, although in the case of community targeting satisfaction and legitimacy among beneficiaries were higher.
b.2) Programme effects
As in the case of universal programmes, social solidarity outcomes can also be affected through the direct effects on the outcomes of interest of the intervention. For example, if income, educational attainment or health and nutritional status of beneficiary households increase, self-respect and attitudes towards the community and public institutions might also be modified. Many cash transfers programmes include complementary interventions such as orientation and support meetings and workshops might also affect social solidarity related outcomes by strengthening bonds among beneficiaries and promoting social participation.
Both in HICs and L&MICs, targeted cash transfer programmes increasingly include conditionalities in their design. There is an on-going debate about their potential effects on democratic engagement, social capital and trust. While some authors strongly argue that conditional programmes depress political and civic participation (Bruch et al., 2010), others argue that this result depends on specific features of programme design (Watson, 2014). At the same time, conditionalities can reinforce programme outcomes and thus can indirectly affect social solidarity outcomes (Watson, 2014).
Social interactions might also be affected by being a programme beneficiary, generating potential changes in reference groups. Beside the potential effects generated by increased or reduced stigma, which were discussed in section b.1, if the programme is effective and direct and indirect outcomes are improved, it might have consequences on exposure to new interactions (for example through work, education or programme related activities) or due to the indirect effects of increased self-esteem.
The political support of targeted benefit schemes can be eroded or fostered in many ways. If the programme is mainly funded through taxes, unintended effects might have relevant consequences on its long-run support. Meanwhile, it is likely discontent would not be the same if the programme would be based on external resources, as has been the case with some cash transfer programmes in Latin America and Africa (Fiszbein, Schady and Ferreira, 2009).
If taxpayers perceive that their money is used efficiently and poverty and inequality are reduced, they might support the programme and their interpersonal and intra-group trust might grow. In the opposite case, however, they could lose trust in institutions and might try to ban the program. Unintended programme effects, such as decreased labour effort or increased informality, can contribute to the perception that beneficiaries are an undeserving population, which can lead to changes in the political support for these policies.
Why it is important to do the review
Cash transfer programmes are a relevant component of anti-poverty policies and are widely implemented both in HICs and L&MICs. In L&MICs, these interventions have been growing in the last decades. In South Africa, social cash transfers expanded in the last decade, in the form of a broad pension scheme; Bostwana, Namibia, Mauritio and Mozambique have also implemented social cash transfer programmes with wide coverage (Hailu and Vera Soares, 2008; ILO, 2016). Meanwhile, over 30 countries have conditional cash transfer schemes; in Latin America they cover approximately 129 million beneficiaries and 18 countries (Fiszbein et al., 2009). In Asia, there are large cash transfer programmes, both conditional and unconditional (Ellis, Devereux and White, 2006; Handayani and Burkley, 2009). In more economically developed states, conditional and unconditional cash benefits are a relevant component of most social protection systems (Currie and Gavhari, 2008).
Social cohesion favours many economic and social outcomes. However, empirical studies have found a decline in social cohesion in developed countries, except for the Nordic ones (Green, Janmaat and Han, 2009). 5 Chan et al. (2006) argue that politicians and policymakers worldwide are increasingly aware that public disenchantment with democratic politics and global economic restructuring pose new challenges to promote trust and solidarity. In L&MICs, fragile democracies and institutions and strong ethnic, economic and social disparities can create structural barriers for social cohesion, which is a permanent concern and challenge for the design of public policies (see, for example, Sojo and Uthoff, 2007 and Ocampo, 2006 for the case of Latin America, and King et al., 2010 for Africa).
There is an on-going debate about the effects of social protection programmes on social cohesion outcomes. Kassenboehmer and Haisken-DeNew (2009) suggest welfare payments generates stigma among beneficiaries and social jealousy among non-recipients in Germany. Similarly, a study of six Latin American capital cities find that welfare programmes had a negative effect on social ties via stigmatization of beneficiaries (Chong et al., 2009). 6 An emergency transfer plan in rural Zimbabwe had negative effects on social cohesion between villages because of the targeting process, although the sense of interpersonal trust among beneficiaries within villages was increased (Kardan et al., 2010). Finally, Vera Soares et al. (2010) find no effects on social participation in the Paraguayan transfer programme Tekoporá, while in Colombia, effects of CCTs on willingness to invest in collective projects and social participation were positive (Attanasio, Reyes and Pellerano, 2009).
The results of this systematic review can also contribute to highlight design elements that can increase programme take-up and avoid stigma. Although this issue has been mainly addressed in the literature on economically developed countries, it is relevant also for L&MICs, as it can enhance policy effectiveness. For instance, in the case of the US, between one and two thirds of Americans forego programmes they are entitled to (Stuber and Schlesinger, 2006; Moffitt, 1983). In their analysis for the European Union, Hernanz, Malherbert and Pellizzari (2004) show that take-up rates span from 40 to 80 per cent in the case of social assistance. All studies point out stigma and information barriers as relevant explanatory factors.
To our knowledge there are no systematic reviews assessing the effects of cash transfers on social solidarity. King et al. (2010) provide a related review as they are also focusing on social cohesion outcomes. However, they assess the impact of community-driven development interventions and curriculum interventions on social cohesion in sub-Saharan Africa. While community driven development interventions may involve direct transfers, they are delivered to the community rather than to individuals.
The effects of conditional and unconditional cash transfer programmes on income, education, health and labour outcomes have been assessed in a number of systematic reviews, but their effects beyond these more conventional outcomes are not clear (Boullion and Tejerina, 2007; Baird et al., 2013; Leroy et al., 2009; Gaarder et al., 2010; Fiszbein, Schady and Ferreira, 2009). To what extent do these interventions foster interpersonal trust, social bonds, esteem for different life styles and other forms of social solidarity? What if these programmes are strengthening human capital accumulation but are undermining other collective assets and thus generating, for example, stigma and social conflict? Answering these questions can contribute to improving the design of these interventions to ensure that, at the very least, such programmes do not have negative effects on social solidarity. The results of this study can also contribute to highlighting design elements that can increase programme uptake and avoid stigma.
OBJECTIVES
The aim of this review is to assess whether direct transfers provided to households or individuals damage or foster social solidarity. Do these interventions foster interpersonal trust, social bonds, esteem for different life styles and other forms of social solidarity, or are they rather strengthening human capital accumulation whilst undermining other collective assets? The primary objective of this review is therefore to identify, appraise and synthesise evidence on the
To address these objectives, we will attempt to answer the following research questions: What are the effects of direct transfer programmes on social solidarity and related outcomes? Do the effects of direct transfers vary with context, programme design (e.g. conditionalities), demographic categories, and complementary interventions? What are the factors (barriers and facilitators) that may influence the effects of cash transfers on social solidarity related outcomes?
METHODOLOGY
Given our overarching concern, a programme ‘works’ when it delivers cash transfers (and possibly associated services) and simultaneously promotes (or at least does not undermine) social solidarity. The evidence to answer questions (i) and (ii) must come from quantitative studies based on randomised controlled trials or quasi-experimental methods with a controlled comparison. If data permit, we will carry out a quantitative synthesis based on meta-analysis, grouping studies as described later in this document. To answer question (iii), we will rely on a broader set of sources including; eg. survey data on programme beneficiaries and/or policy makers, programme documentation and qualitative evaluations, and studies on beneficiaries and non-eligible populations, provided they identify mechanisms that link the relevant social programmes to solidarity outcomes.
The review will follow the Campbell guidelines for conducting and reporting systematic reviews (The Campbell Collaboration, 2014; Hammerstrøm et al., 2010). It will also dig deeper into processes and programme design features when addressing review question (iii). In what follows, we discuss the characteristics of the studies relevant to the review, the criteria for inclusion and exclusion of studies, our search strategy for finding eligible studies, data extraction and study coding, risk of bias, synthesis procedures and statistical analysis, and the treatment of qualitative research.
Criteria for including studies in the review
Population
We will include studies for any population as long as they are recipients of cash transfer programs, or the corresponding comparison group (Table 1). We will include studies conducted in any country. The review does not exclude individuals and/or groups on the basis of their nationality, countries of residence, or any classification of countries, such as HICs vs L&MICs, nor based on urban vs. rural setting within countries. These inclusion criteria are broad, and moderator analyses will be conducted according to population characteristics.
Inclusion criteria by SR component
Interventions
We will include programmes that provide conditional or unconditional direct cash transfers to households or individuals periodically and for at least six months. We will assess both targeted and universal programmes. 7 Specifically, we will include studies where the interventions are universal benefits, pensions, child benefits or general social programs, both as the only programme component and when they are embedded within a broader intervention.
As we want to assess the effect of cash transfers delivered to households that do not have a highly directed purpose, we will exclude public works or unemployment benefits, student loans, as well as vouchers (e.g. educational) or implicit subsidies (e.g. subsidized electric power tariffs). We will include both government- and NGO-run programmes.
Comparison
Eligible comparison groups include an untreated group compared directly, and can also include a group treated with a different intervention (such as conditional versus unconditional, cash versus in-kind).
Outcomes
The key programme outcome of interest is social solidarity, defined as the attitudes and actions of individuals and groups towards other individuals, groups and institutions reflecting a cooperative disposition, commitment and esteem. To be included, studies need to provide a measure of at least one of the primary outcomes described below. Within each dimension, objective and subjective indicators can capture behaviour, attitudes, social ties, and experiences (Prewitt et al., 2014). Table 2 includes the main variables, units of analysis, types of indicator and data sources. Considering that results can be sensitive to these aspects of measurement, in our analysis we will consider the latter two elements as moderators. We will map the different ways in which included studies assess our outcomes of interest within each outcome group and we will group the similar ones together. In what follows, there is a brief description of the most commonly used variables used for measurement within each outcome group.
Dimensions and variables currently used in social cohesion quantitative measurement
a) Interpersonal trust, attitudes towards others and stigma:
Usually, interpersonal trust is measured on the basis of a standard survey question asking ‘Generally speaking, would you say that most people can be trusted or that you can't be too careful in dealing with people?’ Although the question is very general, validation studies find out that answers are usually associated to unknown people or strangers (Uslaner, 2002; Naef and Schupp, 2009) or trustworthiness (Glaeser et al., 2000), being, in this way, a proxy for intergroup trust. Evidence from field experiments suggests that it captures interpersonal trust adequately (Bellemare and Kroeger, 2007). Trust in a particular group (one´s own group, welfare recipients, etc.) might be measured on the same question applied to a certain community.
Additionally, trust can be measured in an experimental setting based on the trust game, originally developed by Berg, Dickhaut and McCabe (1995) and later modified and adapted by many authors (see, for example, Sapienza, Toldra and Zingales, 2007). For instance, and in close connection to the questions raised in this review, Chong, Ñopo and Ríos (2009) implement a version of the trust game, in order to assess the effects of welfare programmes in four Latin American cities.
Following the seminal work of Moffitt (1983), in many economic studies, stigma is most often modelled as a fixed cost of being on welfare and it is estimated indirectly. However, there are specific survey instruments to measure stigma. For instance, to assess the effect of participating in TANF on stigma, Stuber and Kronebusch (2006) develop a scale based on subjective questions. 8 (pp.938).9. There are also survey questions specifically designed to capture attitudes toward poverty, distinguishing social and individual responsibilities. Recently, the Oxford Poverty and Human Development Initiative has developed survey instruments to measure shame and humiliation (OPHI, 2016).
b) Political and institutional trust: This group of outcomes is usually measured using the survey question: “I am going to read you a list of (country)… institutions. Please tell me how much confidence you have in each one: a great deal, quite a lot, some or very little?” The answer includes a wide set of institutions, that are considered separately: parliament, political parties, the President, government, Social security or welfare institutions, the military, the police, criminal justice system, the church or organized religion, trade-unions., etc. (Prewitt et al., 2014).
c) Social connectedness, civic engagement and collective action: This group comprises many different measures of activities and attitudes. Usually, surveys gather data on social connectedness based on the number and diversity of friends, frequency of contact with friends, family and (own and different) group members, and mode of contact (face to face or virtual and remote).
To measure civic engagement, survey questions include activities such as participating in community life through elections, attending public meetings, volunteering, participating in a wide set of political and civic organizations and joining in community projects (Prewitt et al., 2014). These activities might occur at different levels; neighbourhood and local, national or international.
There is also extensive experimental literature that captures propensities to collective action (Karlan, 2005). The voluntary contributions mechanism, a public goods game developed by Marwell and Ames (1979), has been widely used and adapted in the experimental literature on assessing trust and cooperation. For instance, Attanasio, Pellerano and Polonia (2009) measure the effect of the Colombian cash transfer program, Familias en Accion on cooperation, based on an adaptation of this game.
Barriers and facilitators
Our review also seeks to uncover barriers and facilitators whose presence dampens or augments the effects of direct transfers on social solidarity. It will examine studies seeking insights about why interventions might cause a loss or gain in solidarity, and under what conditions. For included programmes, we will analyze the direct effects on the explicit outcomes of the intervention.
Study designs
The component of the review seeking to answer questions (i) and (ii) will include primary studies that conduct a quantitative or mixed-methods impact evaluation, based on principles of experimental or quasi experimental designs that allow for causal inference. Specifically, we will include: Studies where participants are randomly assigned to the treatment and comparison groups (experimental study designs); Studies where assignment to the treatment and comparison group is based on other known allocation rules, including a threshold on a continuous variable (regression discontinuity designs) or exogenous geographical variation in the treatment allocation (natural experiments); Studies with non-random assignment to the treatment and comparison group, provided they include pre- and post-test measures of the outcome variables of interest, to ensure equity between groups on the baseline measure, and use appropriate methods to control for selection bias and confounding factors, such as statistical matching (e.g. propensity score matching, or covariate matching), regression adjustment (e.g. difference-in-differences, and single difference regression analysis, instrumental variables, and ‘Heckman’ selection models).
For the multi-methods attempt to address question (iii) we will include studies and documents that are related to the interventions studied in the included impact evaluations and meet at least one of the following criteria: (1) Collect primary data using qualitative methods of data collection and analysis, and report some information on all of the following: the research question, procedures for collecting data, sampling and recruitment, and at least two sample characteristics; (2) Collect and analyse descriptive quantitative primary data and report some information on all of the following: the research question, procedures for collecting data, sampling and recruitment, and at least two sample characteristics; (3) Process evaluations assessing whether a policy is being implemented as intended, analyzing primary or secondary data; this may include collecting information from different stakeholders to cover subjective issues, such as perceptions of intervention success or more objective issues, such as how an intervention was operationalized; (4) Project documents providing information about planned, on-going or completed interventions. They may describe the background and design of an intervention, or the resources available for a project for instance. As such, these documents do not typically include much analysis of primary evidence, but they provide factual information about interventions. The purpose of including them in our review is to ensure we have sufficient information about the context and interventions of included studies.
Other inclusion/exclusion criteria
The publication time for primary studies is from 1990 to 2015, with no cut-off for the date of the intervention. We will include primary studies in English, Spanish, Portuguese, French and Italian. Specialized sites and bibliographic databases will be searched in those languages.
Search Strategy
We will follow the usual strategies to minimize the chances of omission, and base our search strategy on Campbell's guidelines (Hammerstrøm et al., 2010). We will investigate several sources of bibliographic materials and/or citations, including grey literature, seeking to minimize the risk of bias that could be associated with a specific type of publication.
Electronic searches
We will undertake a comprehensive search of academic databases, citations database, indexed working papers series, and a broad range of search engines and repositories of grey literature. Table 1 (Annex) identifies the sources that will be investigated. They, as other parts of the Search Strategy, were discussed and agreed upon with an information specialist recommended by 3ie.
These databases provide adequate coverage of the relevant sources, and access to title and abstract of potential studies, for many types of documents (journal article, working paper, government documents). All searches will be stored to ensure replicability and references will be downloaded to the EPPI Reviewer 4 reference management software. A dedicated electronic record of candidate primary studies and decisions on their inclusion (and reasons for rejections) will be kept as a component of the broader database. The generic elements of the search strategy will be adapted to fit the requirements and constraints of the electronic databases included in the search. Thus, where appropriate, thesaurus terms will be used in addition to natural language terms in those databases where both can be searched.
The keywords used in the search strategy can be grouped into two main categories (Higgins and Green, 2008): outcomes of interest (e.g., social cohesion; social capital) interventions of interest (e.g., conditional cash transfer);
A broad range of free-text terms is included in each category of search terms (see Table 2). For the two components of the review addressing questions (i) and (ii) we will run searches combining descriptors from groups 1 and 2. A draft of the search strategy is included in Annex (Table 2). This strategy will be adapted to fit all the electronic databases included in the search and where appropriate, thesaurus terms will be used in addition to natural language. Additionally, programme related documents are to be considered for answering question (iii). Once we define the studies included to answer questions i) and ii) we will identify project documents and process evaluations, and will contact authors and implementing agencies in order to access project documentation.
Other searches
We will hand search the most recent volumes of particularly relevant serial publications, including the Journal of Development Economics, Journal of Development Studies, Journal of Development Effectiveness, all the journals of the American Economic Association, Economic Development and Cultural Change, Quarterly Journal of Economics, World Development, World Bank Economic Review, International Journal of Social Welfare and Comparative Political Studies.
To ensure comprehensive coverage, we will contact researchers, research centers, agencies and foundations on the basis of an assessment of who has been publishing extensively and who receives funding or is contracted to work on social programme implementation and evaluation.
We also expect to identify relevant items and authors via “snowballing” techniques (Petticrew and Roberts, 2006: Waddington et al., 2012.a): We will use included studies (or reviews) to identify other studies (or reference authors). On a parallel track, we will search for articles that cite a key reference (the “pearl”), and we will also identify keywords to complement our initial list, from looking up the key words of included articles, and searching for other papers that use the same terms.
All records identified through this search will be assessed for eligibility, according to the inclusion and exclusion criteria presented in section III.1, keeping primary studies only. This process will be carried out independently by two researchers, who will examine titles, keywords and abstracts. In case of disagreement, a third independent reviewer will be consulted. A sample of excluded studies (5%) will be examined by a third reviewer as a consistency check.
Studies meeting the inclusion criteria at this stage, will be coded as “eligible first step”. We will obtain the full text papers of those studies that meet the eligibility criteria in the first phase and those for which abstract and title screening was inconclusive. This full text review will also be done independently by two researchers and a third one consulted in cases of disagreement.
Selected studied will be labelled as “included” and moved to “data extraction” stage, while those not meeting the eligibility criteria will be marked as “excluded”, along with the reason for exclusion.
Data extraction
The data to answer the review questions will be extracted from the studies using an instrument organized in twelve sections (see Table 3, Annex) covering full bibliographic details and miscellaneous information; data relevant to addressing questions (i) and (ii), and information addressing question (iii). We will use the EPPI Reviewer software to manage the screening and data gathering processes. Each record will include information on source, eligibility, methods and study type, specific intervention details, outcomes, results, programme beneficiaries, participants and baseline data, duration of the study and potential moderators (Higgins and Green, 2008).
List of Search terms
Data from primary studies will be extracted independently by two researchers, to minimize errors and reduce potential biases. Disagreements will be resolved in the same manner as in the screening process. However, if disagreements cannot be solved, after consulting a third researcher, study authors will be contacted. If the disagreement persists, it will be reported in the review. The same procedure will be followed when significant data is missing from the full text item. Authors will contact authors of other studies to ask for specific pieces of information, when unable to extract them from available studies. Alternatively, this information could be obtained from related publications or carrying out specific internet searches.
Risk of bias and critical appraisal
We will appraise all studies based on two separate checklists. To assess the risk of bias in randomized control trials and quasi-experiments we will carefully scrutinize studies and apply the checklist presented in Table 4 (see Annex). This checklist is an adaptation based on the guidelines provided by CBEP (2010) checklist, EPOC (2014) and the IDCG toolkit. Although available tools usually rank RCT over quasi-experimental designs, we will be carefully assessing the particular circumstances of the impact evaluation implementation, as severe caveats can emerge in many stages of the evaluation process (CBEP, 2010; EPOC, 2014).
Specifically. we will be assessing the overall design (sampling units, sample size, based on the power of the study to capture effects on a particular outcome), the equivalence of the control and intervention groups (baseline characteristics, contamination, outcome data collection), the study´s outcome measures (validity, blindness in data collection process, duration of effects) and the reporting of effects (percentage of outcomes in which results are reported). On this basis, we will classify the risk of bias of the selected studies in three groups: low, high and unable to classify (Waddington et al., 2012.b). Following Wilen et al. (2012), we will separately classify risk of bias linked to conflict of interest, creating three groups (high risk, unclear, low risk).
We will use a checklist based on the Critical Appraisal Skills Programme tool (CASP, 2006) to appraise qualitative studies on views and attitudes included to address question iii (Snilsveit et al., 2012; Waddington et al., 2012.b). In this way, we will be able to judge reporting, data collection, presentation, analysis and conclusions drawn. The checklist is included in Table 5 (see Annex).
We will report the results of the risk of bias/critical appraisal assessment and conduct when possible sensitivity analysis according to risk of bias/critical appraisal status.
Description of methods used in primary research
The literature features a variety of impact evaluation designs, from treated-only, perception studies, to quasi-experimental designs and possibly RCTs. In the quantitative synthesis, we will include studies using methods based on the definition of a control and a treatment group and using a suitable econometric methodology for assessing causality (i.e.: differences in differences, regression discontinuity, propensity score matching or instrumental variables).
The studies found in our exploratory mapping of the literature to date are mainly quasi-experiments, and the econometric methods most frequently used are differences in differences, instrumental variables and propensity score matching (Vera et al., 2010; Attanasio et al., 2009; Stuber and Kronebusch, 2004; Stuber and Kronebusch, 2006). For example, Attanasio et al., 2009 analyse the effects on trust and collective action of Familias en Accion, a Colombian conditional cash transfer programme that also includes a social component. They base their assessment on a behavioural game, played by 38 groups coming from two similar neighbourhoods in Cartagena. Controlling for observable differences and using an instrumental variables strategy, they find that receiving the programme for two years increases in 11 per cent de probability of individuals to contribute in a public project. They also check the trust measures obtained from the game against more conventional ones coming from a survey questionnaire and they find low correlations.
To address question iii) we will include quantitative and qualitative studies assessing the links among direct transfers and social solidarity outcomes, based on statistical analysis of opinion and other surveys. 10 Views and qualitative studies will include any studies with these characteristics assessing the included programmes, according to question i). For example, in their study of the Mchinji programme in Malawi, Miller et al. (2008) provide an interesting example for assessing questions ii) and iii) (conditional on including relevant quantitative studies on the programme in question 1i). Based on focus groups evidence, as well as interviews to key informants and analysis of reports, processes and monitoring tools, they conclude that, in spite of improvements in terms of objective outcomes, the targeting mechanism generates jealously and conflict among communities, eroding social cohesion. 11
Criteria for determination of independent findings
To ensure that pooled impact estimates for each intervention type and outcome/domain are built from statistically independent findings, we will only include independent estimates of effect in any single meta-analysis. We will identify and address potential dependent effect sizes as follows.
If we have several publications reporting on the same study, we will use effect sizes from the most recent publication. In cases where we identify several studies using the same data set or if studies present results for a certain outcome using several model specifications, we will include the one with the lowest risk of bias.
For studies with outcome measures at different time points we will map all follow up periods and combine studies measuring similar follow up periods, including only one follow up point per study per meta-analysis. If multiple follow-up time points exist we will aim to address time trends, depending on what has been measured in the included studies.
If studies include multiple outcome measures to assess related outcome constructs, we will be computing effect sizes for each one separately to capture the sensitivity of the results to different operationalizations. We will map out the measures in the included studies to get an overview of the data, in order to compare measures and identify groups of measures that might be appropriate to combine, considering both the outcome constructs and sources of dependency, following a similar procedure as Baird et al. (2013). A final source of dependency is studies with multiple treatment arms representing separate treatment constructs with only one control group. In such cases, we will calculate the effect size for treatment A versus control and treatment B versus control and include in separate meta-analyses according to the treatment construct. If the treatments provide variations of the same treatment construct, we will calculate the weighted mean and standard deviation for each treatment arm before calculating the effect size for the merged group versus control group, following the procedures outlined in Borenstein et al. (2009, chapter 25).
Statistical procedures and conventions
Measurement of treatment effects
For continuous outcomes, we will compute mean differences and standardized mean differences (adjusting for sample size), d, and we will, also, report the standard errors and 95 per cent confidence intervals, based on the formulae provided in CIDCG (2016). For dichotomous or categorical outcomes, we will compute odds ratios CIDCG (2016), OR, also reporting the standard errors and a 95 per cent confidence intervals (Borenstein et al., 2011). Treatment effects will be computed consistently as the ratio among treated and control group observations.
Most of the outcome variables in included studies will be dichotomous or categorical variables. Although we will be computing separate ES for each variable, to ensure comparability, if necessary we will use the following formula to convert standardized mean differences to odds ratios (Borenstein, 2009):
And to compute the variance:
For each outcome variable, we will provide an interpretation in terms of effect size (sign and magnitude). If necessary, the corresponding formulae will be used to infer the relevant estimator for computation of effect size.
Unit of analysis issues
We will assess studies for unit of analysis errors, where the unit of the treatment is different to the unit of analysis, without taking account of clustering in the analysis (The Campbell Collaboration, 2014). If unit of analysis errors exist we will attempt to correct for this using the corresponding statistical corrections, using reported (or requesting to the authors) on level and number of clusters and intra-group and between group variance (Hedges, 2009).
Missing data
Where included studies do not provide the data required to calculate effect sizes, we will attempt to contact the authors of the primary studies to request this missing information. Formulae will be used where necessary to extract or impute effect sizes based on other commonly reported statistics (Lipsey and Wilson, 2001).
Methods of synthesis
To address questions 1 and 2 we will synthesise evidence on the effects of direct transfers on social solidarity outcomes. We will synthesise studies using meta-analysis, conditional on the number of comparable effect estimates available from the included studies. Following Wilson, Weisburd and McClure (2011) we will carry out a meta-analysis if we find two or more studies, each with a computable effect size of a common outcome construct (potentially measured in different ways), and similar comparison condition.
If we identify a sufficient number of studies for meta-analysis we will synthesise studies using inverse-variance meta-analysis using use Stata software for this purpose (Stata Corporation, College Station, TX, USA). We expect a degree of heterogeneity among outcomes, populations and interventions to be significant, so we will use a random effects model if meta-analysis is performed.
We will assess heterogeneity of effect sizes graphically, and test for heterogeneity formally by calculating the Q-statistic. Heterogeneity will be estimated using the I-squared and the Tau2 to provide an overall estimate of the amount of variability in the distribution of the true effect sizes (Borenstein et al., 2009).
If we do not identify enough studies that are sufficiently similar for meta-analysis to be meaningful, we will report standardised effect sizes for all relevant outcomes. We will present effect sizes in tables and forest plots (without a pooled effect size) as appropriate, and report on the magnitude and variance of effects organised by intervention and outcome.
We also aim at exploring sources of heterogeneity based on moderator analysis, including extrinsic, methodological and substantive moderators (Lipsey, 2009). In the first group we will include a group of variables that do not refer to the research design but they may be correlated with the results of the study: date of publication, publication type (peer reviewed journal, grey literature, thesis, book), funding and existence of conflict of interest. Among the second group, we will include variables that reflect study design and refer to the identification strategy (RCT, Regression Discontinuity, Instrumental Variables, Difference in Difference, Propensity Score Matching). We will also explore including dummy variables reflecting risk of bias group and the metrics of the outcome variable in case we combine indicators using different scales.
Finally, we will include the following substantive moderators: programme design features (such as existence of complementary interventions, whether the programme is conditional or unconditional, target population, targeting method) and sample characteristics that the literature identifies as being correlated to our outcomes of interest (as age, gender, location, geographic area and ethnicity). For example, Stuber and (2006; 2004) identify that, in the US, black populations are more prone to welfare stigma and there is a high degree of variation in stigma by states. Alesina and La Ferrrara (2000, 2002) find that social and political participation, as well as trust, show a gradient by ethnicity and gender. Considering the heterogeneity of the countries we will be including in the review, we will also include geographical location as a moderator variable. Table 6 (see Annex) contains the full list of moderator variables. If we do not have sufficient studies to conduct a statistical moderator analysis, we will explore the role of moderators narratively.
We will conduct sensitivity analysis according to categories of risk of bias, study design (experimental and quasi-experimental, adjusted and unadjusted effect sizes) and treatment effect (for example, intention to treat, average treatment effect on the treated, local average treatment effect).
To assess publication biases, we will use funnel plots, regression based tests and qualitative sub-group analyses.
Treatment of qualitative research
To answer our third question on barriers and facilitators, we will carry out a narrative synthesis based on descriptive quantitative studies and qualitative research results. This will contribute to the understanding of the mechanisms that researchers identify as actually causing changes in social solidarity.
Specifically, we will develop a thematic synthesis of the main findings of the included studies (Noyes and Lewin, 2006; Pawson et al., 2004; Snilstveit et al., 2012). On the basis of the full tests will be identifying themes in regard to views and perceptions from beneficiaries and non-beneficiaries. However, the methods used to carry out the synthesis will heavily depend on the type of studies obtained. We will be using the qualitative analysis software provided by EPPI reviewer.
External validity
We will explore external validity on the basis of the subgroup analysis, the inclusion of moderators and the context characteristics in which interventions and studies were carried out. In this way, we will be also able to identify for which subgroups evidence is scarce.
Footnotes
I. SOURCES OF SUPPORT
This research is supported by a grant awarded by Comision Sectorial de Investigación Científica (CSIC) of the Universidad de la República, to the group Ethics, Justice and Economics.
II. DECLARATIONS OF INTEREST
Andrea Vigorito participated in a primary study that might be potentially included in the review.
III. REVIEW AUTHORS
IV. ROLES AND RESPONSIBLIITIES
Please give brief description of content and methodological expertise within the review team. The recommended optimal review team composition includes at least one person on the review team who has content expertise, at least one person who has methodological expertise and at least one person who has statistical expertise. It is also recommended to have one person with information retrieval expertise.
One of our team members, Andrés Rius, has participated in the following systematic review: Aboal D, Noya N, Rius 
A systematic review on the evidence of the impact on investment rates of changes in the enforcement of contracts. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.
This work was funded by DFID and the review team received technical advice from EPPI-Centre and 3ie experts. Gustavo Pereira is a philosopher that has published many books and articles addressing topics connected to solidarity, social justice and public policies. Martín Leites has a strong statistical background and is currently participating in an impact evaluation exercise. Gonzalo Salas and Andrea Vigorito carried out several impact evaluations and have been working for many years in topics related to inequality, poverty and social policies.
Responsible co-authors by dimension:
PRELIMINARY TIMEFRAME
Approximate date for submission of the systematic review (please note this should be no longer than 2 years after protocol approval. If the review is not submitted by then, the review area may be opened up for other authors). September 2015
PLANS FOR UPDATING THE REVIEW
We will be updating the review five years after completion.
AUTHORS’ RESPONSIBILITIES
By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.
A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.
You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.
PUBLICATION IN THE CAMPBELL LIBRARY
The support of the Campbell Collaboration and the relevant Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review and subsequent updates in the Campbell Library. Concurrent publication in other journals is encouraged. However, a Campbell systematic review should be published either before, or at the same time as, its publication in other journals. Authors should not publish Campbell reviews in journals before they are ready for publication in the Campbell Library. Authors should remember to include a statement mentioning the published Campbell review in any non-Campbell publications of the review.
I understand the commitment required to undertake a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:
Form completed by: Martín Leites, Andrés Rius, Gonzalo Salas, Andrea Vigorito
Date: 9 August 2016
In Honneth's view, solidarity is achieved when individuals understand that they are “esteemed” by all citizens to the same degree and that different ways of life are tolerated and respected.
In this sense, the idea of solidarity is closely related to “recognition”.
‘Social cohesion’ as used in this paragraph follows the terminology of international organizations.
Recent sociological literature keeps the expression ‘social solidarity’ in reference to Durkheim's original work and reserves the term social cohesion for present definitions. Jamaat et al. (2009) argue that ‘social cohesion’ has been used in the Anglo-Saxon literature as a translation for the French solidarité social. Meanwhile, in the philosophical literature, critical theory refers to solidarity in very close association to Durkheim and the sociological tradition, and thus in very close connection to social cohesion, whereas in pure ethical reflection, solidarity is understood as a disposition to reciprocal or asymmetrical aid to others. In the latter case, cohesion results from an individual or social ethical virtue.
3
Coady, cite the examples of cash transfer programmes in Armenia and Jamaica as highly stigmatizing examples as they strongly appealed to self-selection and were very open and oriented towards promoting enrollment and acceptability. Sometimes, lists of beneficiaries are published in order to avoid leakage.
4
argue that in means-tested programmes, as applicants and public employees develop more proximate relations, people form their views on the system from particular experiences, including their own. For instance, if providing deceiving information to obtain a certain benefit is perceived as widespread, people will wonder why should other people be trusted in general? This will affect programme applicants only, reinforcing the different varieties of trust or discrediting them.
5
These authors carry out an empirical study of the evolution of social cohesion in EU states, operationalizing it as social and political trust, tolerance and perception of social conflict. They find different trends, with countries in which social cohesion has declined in the last decades as the UK, and others as the Nordic countries increased cohesion. In their empirical analysis of the determinants of trust for the US, Alesina and La Ferrara (2002) find that the recent history of traumatic experiences, belonging to a historically discriminated group, being economically unsuccessful in terms of education and income, and living in a mixed or highly unequal community, all reduce trust.
6
They do not investigate specific programmes but assess the effects of welfare programmes in general.
7
As defined above, targeted programmes select beneficiaries (usually the poorest in a given population; for example, single mothers with young children with incomes below a threshold) whereas universal programmes are available to all citizens within wide categories, even if they are justified on poverty alleviation grounds (such as an old-age pension programme available to all).
8
As mentioned before, these authors separate identity and institutional stigma; in the first group they use the following battery of questions: I worry that being on welfare would make me lazy; The average women on welfare has too many kids; Once people start to receive welfare they usually stay on it for more than 2 years; Many people in my neighbourhood on welfare are lazy; Many people on welfare drink too much; Many people in this country on welfare are lazy. In the second group, they include the following statements: the application process for welfare is humiliating; Many people are treated poorly when they apply for welfare; When applying for welfare, you have to answer unfair questions about your personal life; When participating in welfare, the rules take away your personal freedom; Many people on welfare do not want other people to know they are on welfare; There are a lot of people in this country who do not respect a person on welfare.
10
For example, non-experimental attempts to measure attitudes or behaviours associated with the notion of solidarity, and their change, in jurisdictions where there are or there has been active transfer programs.
11
The study also includes a quantitative impact evaluation, which will be considered in the analysis of review questions 1 and 2. In this description we focus on the qualitative components of the study.