Abstract

2. Background
1.1 The problem
In many places of the world, women remain at a disadvantage to men. Women globally earn 24 per cent less than men for doing the same work and 75 per cent of women's work in developing countries is in the informal sector (World Economic Forum, 2015). Gender inequality in access to economic opportunities, a lack of voice and participation as well as limited opportunities to accumulate wealth could perpetuate vulnerability to poverty among women, limit human capital accumulation in the new generation and restrict economic growth.
With this view, the UN Millennium Development Goals (MDGs) and the Sustainable Development Goals (SDGs) have recognized that improving female economic empowerment is not only valuable in its own right but is also a key element to end poverty and boost economic prosperity (UN, 2000; UN, 2015). Based on this premise, international organizations, local governments, non-profit organizations and private companies have engaged in various initiatives that aim to decrease gender inequality and foster female empowerment. For instance, in 2016 the Bill and Melinda Gates Foundation contributed $80 million US dollars to help accelerate the progress of women and girls around the world.
Empowerment in a broader sense is defined as the expansion of people's (both men and women) ability to make strategic choices in a context where this ability was previously denied to them (Kabeer, 2012). To the extent that freedom of choice depends on economic, social, political and psychological aspects, empowerment is a multidimensional concept. Yet, one aspect that is considered to be crucial in increasing the ability of individuals to control their lives and exert influence in society is economic empowerment – defined as increased access to - and control over - resources and economic lives.
Given the importance of economic freedom to promote development and growth, this systematic review focuses on interventions that aim at fostering female economic empowerment. There is a wide range of interventions that have been developed to foster female economic empowerment. A useful classification is provided by Malhotra and Schuler (2005) who distinguish three dimensions of economic empowerment; namely, at the household level, community and national level. Household economic empowerment is related to the control of household income. Therefore, this dimension considers interventions that foster gender equity in rights to own and inherit land. At the community level, it refers to access to employment, credit, representation in trade associations and access to markets. It considers for example initiatives that give in-kind grants to women to establish an agricultural or micro-entrepreneurial activity, initiatives that give women access to labour markets by providing child or care for the elderly, or by opening short-term employment possibilities for women. At the national level, the focus is on female representation in high-paying jobs, the proportion of female CEOs and the representation of women's economic interests in macroeconomic policies and federal budgets. In this review, we focus on interventions that foster female economic empowerment at the community level and consider their impact on human development.
Human development is defined as the process of enlarging people's freedoms and opportunities and improving their well-being. Traditionally, this term has been associated with increases in per-capita income or the growth rate. Yet, recognizing that the quality of life not only depends on a monetary dimension, a more recent approach is the capabilities approach that considers the ability that an individual has to freely choose the life they want to live (Nussbaum and Sen, 1993). This approach recognizes that there are certain functional capabilities that allow a decent quality of life (Nussbaum, 2000). Following this conceptualization, and an approach similar to the UN human development reports, we consider two dimensions of human development. First, we consider a dimension of wealth and take into account household income-per-capita, assets and access to basic services. Second, we consider three capabilities: i) the capability to live a life of normal length with good physical and mental health, protected from violence and enjoying recreational activities; ii) the capability to think, reason and engage in critical reflection and iii) the capability to control one's environment by participating in decisions in the household and in the community.
While previous evidence suggests that diverse interventions have been successful in reducing gender inequality and fostering female empowerment (Dekker, 2013; Buvinic, et. al., 2013), there is relatively little evidence of their impact on human development. The evidence on the effects of female economic empowerment on development is relatively scarce (Morrison et al., 2010; Duflo, 2012 and Bandiera and Natraj, 2013). We build on previous work by doing a systematic review of studies from developing countries that consider a variety of interventions that aim at providing women access to income generation alternatives and their impact on human development (i.e. support for agricultural and micro-entrepreneurial activities, access to labour market and formal financial markets).
1.2 The Intervention
Programs that focus on female economic empowerment are very diverse both in terms of the intervention and the dimension of empowerment that they address (Buvinic, et. al., 2013). We adopt the conceptualization of Malhotra and Schuler (2005) and classify interventions according to the context in which they take place. Table 1 presents the indicators of female economic empowerment in each context and classifies the interventions according to whether they are covered or not in the present systematic review.
Female Economic Empowerment
In this review, we only consider interventions that aim to increase female economic empowerment at the community level by providing alternatives for income generation. We consider interventions that: Foster female productivity in agricultural activities. We focus on interventions that a) give women access to inputs and new technologies (i.e. improved seeds, fertilizer, farmer field schools), b) support access to final markets (i.e. promote farmer associations, or provide information on prices) and c) provide subsidies to connect to existing infrastructure (i.e. electricity, irrigation). Promote urban or rural micro-enterprises by a) providing subsidies for female entrepreneurs (i.e. soft credit, start-up grants, in-kind grants) and b) promote female business networks. Promote financial access by a) giving access to formal financial markets (i.e. credit, savings and insurance) and b) support access to informal financial markets (i.e. promote micro-savings, microcredit or micro-insurance) Promote female labour force participation. We consider interventions that a) provide job placements through internships, subsidies to employers and changes in the hiring process (i.e. affirmative action, anonymous hiring), b) provide childcare and care for the elderly, c) allow flexible working hours and working from home, d) provide economic incentives for women to work (i.e. tax deductions, subsidized employment) and e) foster equal rights for male and female employees.
In this review, we focus on interventions that attempt to directly reduce or eliminate the challenges that women face, which prevent them from participating in economic activities. Therefore, we do not consider interventions that could indirectly affect female labour force participation as conditional cash transfers (Leroy et al. 2009 and Yoong et al. 2012; Kabeer et al.2012); education, vocational and financial literacy, entrepreneurship training programs —including field schools (Miller et al. 2014; Tripney and Hombrados, 2013, Woodruff and McKenzie, 2014; Chinen et al. 2016); diverse types of information campaigns (i.e. Aker, 2010); interventions as certification to firms for good practice in hiring and retaining female employees, consulting and mentorship programs for women (Feigenberg, et al., 2013; Field, et al., 2016) or interventions that aim to reduce domestic violence against women (Day, 2009).
Finally, in the analysis, we do not consider interventions that aim to empower women at the national and multinational level by increasing representation among CEOs or decision-making positions in government, through political reservations or within the judiciary system (Dahlerup, 2013; Schmitt, 2015). We also do not consider interventions that aim to give women access to resources in the household. We, therefore, exclude interventions that give men and women equal rights under the law (i.e. land rights and land titling, inheritance laws and law on divorce settlements: marital property, child support, custody and alimony).
1.3 How the intervention might work
Improving the economic position of women by providing them with access to income generating opportunities at the community level is likely to reduce poverty and generate greater human development within the household (Morrison et al., 2010). Annex 1 presents the theory of change that provides a base for our analysis.
The main objective of female economic empowerment interventions is to increase the possibility for women to generate income. Income in the hand of women can affect human development by different channels. First, the most direct effect of female economic empowerment is that this will increase household's income allowing higher consumption which will be reflected in increased well-being. Second, it is expected economic opportunities for women will affect female's bargaining power within the household (Allendorf, 2007; Doss, 2013). This is expected to improve health outcomes of women (Beegle et al., 2001; Li and Wu, 2011), reduce intimate partner violence (Panda and Agarwal, 2005 and Rao, 1997) and have positive intergenerational effects. Increase female bargaining power has been shown to be associated with a larger share of budget used on food and on children's education and health (Quisumbing and Maluccio, 2003; Duflo and Udry, 2004; Doss, 2006; Robinson, 2012). Evidence also shows that income in the hands of women is associated with increased intergenerational welfare (Schultz, 2002; Rubalcava et al, 2009). Moreover, income in the hands of women has been shown to increase investment in girls, reducing future gender inequality (Duflo, 2003). The third indirect channel by which female economic empowerment could affect human development is related to fertility decisions. As the opportunity cost of time for women increases, it is expected that fertility rate drops and that income per capita, savings and asset accumulation increase (Schuler and Hashemi, 1994; Dupas and Robinson, 2009).
Interventions that provide financial access are expected to relax credit constraints that households face to borrow capital for their enterprises. This would be reflected in more investments in enterprises, adoption of new technologies and in higher productivity (Morrison et al., 2010). On the other hand, access to credit is expected to reduce the negative effect of unanticipated shocks reducing households’ vulnerability to poverty. Interventions that foster saving groups are also expected to lead to an increase in asset accumulation and have a positive impact on children's well-being (Pitt et al., 2003, Cheston, 2006, Dupas and Robinson, 2013).
While female economic empowerment could lead to a decrease in poverty and higher human development, it is not clear if this is the case. There are many contextual factors that may limit the possibility of an impact on human development. Even if women have access to resources this does not necessarily guarantee their control over their use. Social barriers might prevent women from controlling these resources. For example, social norms on the acceptable role of women in society might limit the mobility of women and the possibilities to participate in economic activities. Moreover, low initial bargaining power within the household might imply that men control the resources generated by women. For instance, Goldstein and Uldry (2008) show that males with higher status in Ghana were able to exploit this to their advantage using fallowing land. On the other hand, the systematic review by Vasseen et al. (2012) on the impacts of microfinance on a woman's control over spending, shows that the effect is relatively low and not significant, which is partly attributed to the demand of the social network including the husband.
Assuming that it is possible to enforce a woman's control over resources so that they make investment decisions, it is not clear if they would engage in more productive activities than men. Women might specialize in less profitable crops, such as food crops, foregoing the income generating opportunities stemming from cash crops (Croppenstedt et al. 2013). Alternatively, they could specialize in less productive enterprises that give them more flexibility to combine household responsibilities and working hours (Amin, 2010). Second, women might invest less in high-yield but riskier technologies. Udry (1996) shows that men are more likely to use agricultural inputs than women, which translates to differences in productivity. Similarly, experimental evidence shows that micro-loans directed at women were not associated with larger effects than those given to men (Karlan and Zinman, 2011). Different studies show that the return of in-kind grants is larger for male than female owned business (De Mel, et al., 2008 and Fafchamps et. al, 2011). Therefore, interventions that support female business have been criticized in that they enable less successful enterprises to survive longer (Woodruff and McKenzie, 2013). A critical aspect might be the lack of access to marketing opportunities.
Interventions that foster female labour force participation, through the provision of childcare or childcare subsidies, can also generate positive spill-over effects on children's nutrition and development (Ruel et al.,2006; Attanasio& Vera-Hernandez, 2004; Berlinski&Galiani, 2007). Another problem is that these interventions can increase employment, but at a lower salary (Todd, 2013) or generate negative spill-over effects on the jobs and wages of the nonbeneficiary population (Betcherman et al., 2004). A critical assumption of programs that rely on the promotion of female labour force participation is that they would not generate a negative effect on non-beneficiaries. Hence, it is important to try to capture general equilibrium effects. When available, we will extract information on the change in male employment that can be attributed to the intervention. Finally, the sustainability of these interventions is questioned as discrimination against women might continue to persist once the incentives have been removed (Groh et al 2012).
It is possible that female economic empowerment increases tensions within the household and the community. For example, women could be subject to the triple burden of work as they need to contribute income to the household and still be responsible for housework and childcare. This can lead to more stress and conflict within the household. (Ahmed 2005; Ahmed & Chowdhury 2001). As women gain access to income-generating alternatives, they may reduce their time with their children, resulting in worse performance of their children at school and hence an increased likelihood of their children dropping out of school, involvement in gangs or switching/transferring responsibilities of housework to younger daughters (Ruhm, 2008). As working mothers have less time available, they could switch to processed foods of high caloric value leading to nutrition problems (Cawley and Liu, 2012).
Increased access to labour market opportunities could also lead to an increase in domestic violence (Heath, 2014), though this correlation might disappear once endogeneity is controlled for (Lenze and Klasen, 2016). Participation in economic activities outside of the home could also make women more vulnerable to crime or could expose women to environmental hazards at work (Amaral et al. 2015).
In summary, the interventions for female economic empowerment that we consider in this review are expected to affect human development by the following three main channels. First, these interventions increase the opportunities for women to have economic independence by enabling them to generate their own income. Second, they generate a direct income effect in the form of higher household income. Third, they reduce the vulnerability of the households by reducing the variability of their income, either by the provision of insurance or by decreasing the covariance of a household's income.
However, various factors may prevent these effects from occurring. While it is difficult to capture these factors, we can use the proxies: region of analysis, religious beliefs and a measure of female participation in a particular sector before the intervention. To capture the effect of female control over resources, we use as an indicator for female bargaining power. Yet, this factor is considered in the analysis only if we have measures of human development.
A second factor that might restrict the success of the support programs for agriculture and entrepreneurship is financial access. Therefore, one needs to consider the financial situation of the beneficiaries before the program began, distance to financial institutions and the perceived financial access.
A third condition for the program to be successful is that the support programs lead to productive investments with good market access under reasonably good economic conditions. Hence, in the case of agricultural support programs, we need to consider if the beneficiaries invested in food or cash crops. Second, if they invested additional resources (i.e. inputs). Third, we need to consider how adverse effects might have affected the program (i.e. weather shocks, price shocks etc.) and lastly, consider if they work in a cooperative group or not because it may facilitate market access.
Finally, programs could have a positive effect reducing a household's vulnerability to poverty by providing more stable income. One channel by which this might occur is by decreasing the impact of negative shocks and the degree of covariance of a household's income. To the extent that this information is available, we would include it in the analysis. An intervention that is expected to last only a couple of months is expected to generate lower effects than long-term interventions. Therefore, we would need to consider the effect of the expected duration of the program in addition to the duration of exposure to an intervention.
1.4 Why it is important to do the review
Our motivation to undertake a systematic review is to enhance academic knowledge and provide useful insight for policy makers both at the country and international level. Many studies address the question of what works to promote female empowerment as well as female economic empowerment. For example, Brody et al. (2013) consider the impact of self-help groups; Woodruff and McKenzie (2013) business training programs; Cho and Honorati (2014) entrepreneurship programs; Bandiera et al. (2013) capital, training or a combination of both; Mehra et al (2013) financial services; Rodgers and Menon (2013) land rights; Doss et al (2013) and Knowles (2013) agricultural productivity; Todd (2013) employability and quality of work; Katz (2013) young employment; and Vaessan et al (2014) reviews the effects of microcredit. In our review, we pursue an additional step and assess which interventions of female empowerment have an impact on human development.
Previous studies have addressed the relation between gender equity and development (Duflo, 2012; Morrisson et. al, 2010) However, no study has used a compact measure of outcomes such as a human development indicator. In our analysis, we consider an aggregated measure of human development that has four dimensions: household income and assets, health outcomes, educational outcomes and attitudes towards gender equity. Our measure of outcome considers the aggregated effect at the household level.
Furthermore, we contribute to this line of research by focusing on Randomized Control Trials- RCTs and quasi-experiments that use relevant econometrics to explain the causal impact of an intervention. We pursue the use of a robust methodological approach which other reviews are unable to provide (with a few exceptions such as Brody et al., 2013; and Rodgers and Menon, 2013). For the search of interventions in developing countries, we follow a specific time frame and include studies that have been published between 1990 and 2017. In addition, the risk of selection bias is addressed by including different languages (English, French, German, Spanish), working papers as well as reports from particular programs in our search.
Our focus is to provide feasible policy relevant conclusions of the evaluated interventions. Female empowerment and gender equality is not only an important aim of the UNDP to achieve the Millennium Development goals (UNDP, 2014), but it has also been on the policy agenda of developing nations. Large scale programs, such as the Mahila Samakhiya in India, Business Development Schemes in Kenya, Uganda and Tanzania, are examples of the steps undertaken by various governments to improve female empowerment and development.
Through our review, we attempt to compare the relative impact of interventions across different interventions, studies and contexts. Depending on data availability, we will include information on the cost of an intervention, which could provide a base for comparison on the cost-effectiveness of the different interventions. In the absence of data, we will favour simple interventions that can be easily implemented, taken to scale and sustained even after the implementer has left. We build on the works of Buvinic et al. (2013): “Roadmap for Promoting Women's Economic Empowerment” and Dekker (2013): “Promoting gender equity and female empowerment.” While these studies consider the effectiveness of interventions on female economic empowerment (measured as increased productivity, wages, income and assets on the hand of women), in this review we consider the impact of these interventions on a very different of dimensions related to economic development.
3. Objectives
Our research objective is to provide a systematic review of the impact of the most common interventions used to promote female economic empowerment on human development indicators in low- and middle-income countries. We focus on interventions that give women access to income generating opportunities in the following aspects of human development: agriculture, the support for female entrepreneurship activities, and the promotion of female labour force participation and the ease of financial access.
We address the following questions:
Q1. What is the effect of interventions at the community level that aim to promote female economic empowerment on a household's human development in terms of income, health, education and attitudes?
Q2. What are the pathways through which these interventions affect these dimensions of human development? Is this due to changes in female bargaining power, a change in fertility decisions, changes in expenditure, and a change in the investment in girls?
Q3. How are institutional environments, such as social barriers to women in a society, low initial levels of female bargaining power, lack of productive alternatives, religion, ethnicity, social networks, social norms and culture, associated with the effectiveness of the different interventions?
4. Methodology
3.1 Criteria for Including and Excluding Studies
Following Petticrew and Roberts (2006), we use the PICO model (Population, Intervention, Comparison and Outcome model) to define the inclusion and exclusion criteria.
3.1.1 Types of Study Designs
We only consider quantitative studies that aim to evaluate the causal effect of an intervention on the outcome. In particular, we focus on studies that use either experimental or quasi-experimental approaches of impact evaluation. We incorporate quasi-randomized approaches because studies that claim to use randomized assignment of treatment may actually use a non-randomized method. Whether a study uses genuine randomization will be reflected in our risk of bias assessment. In the case of randomized or quasi-randomized controlled trials, the evaluation of the effect is undertaken by comparing the outcome with the control group and by using an appropriate methodology.
In the absence of randomization, the main challenge is that the treatment can be endogenous, i.e. it can be confounded with unobservable factors correlated with both the treatment and the outcome. There is large heterogeneity in the methods used to control for potential endogeneity. We restrict our analysis to methods that are in theory able to address unobservable confounding factors. These methods include regression discontinuity design (RDD), instrumental variable (IV), difference-in-differences (DID) and interrupted time series (ITS). Some of these methods may use estimators that are not comparable with RCTs – this can be assessed at the synthesis stage in the sensitivity analysis. We exclude studies that are unable to address the issue of unobservable confounders in evaluating the causal effect of the intervention.
We will also exclude studies that: Do not contain any of the interventions to promote female economic empowerment or do not contain any of the indicators of human development. Quantitative studies without any type of observable comparator (for example, time or control group) and credible methods of correcting for selection bias as outlined above. Qualitative studies that do not employ the defined methodologies listed above or that do not draw from direct observation or direct reports from program participants. Although some of the outcomes we are interested in can be better captured if qualitative data is considered in addition to quantitative data, analysing qualitative data is beyond the scope of this study. Yet the clear limitation is that we might be unable to capture the adverse effects of the interventions. Future work should focus on this point. We only include studies published in the period from 1990 to 2017. Studies that were not published within this time frame are excluded.
3.1.2. Types of participants
We include interventions where the participants are women and girls (irrespective of age) from low- and middle-income countries as defined by the World Bank categorization at the time the data was collected. Thus, we exclude studies of interventions in high-income countries. We also include studies where men participate and have the potential to promote female economic empowerment and where this effect can be identified.
3.1.3. Types of interventions
We focus on the following kinds of interventions, which aim to promote female economic empowerment at the community level: . . . .
Level of Intervention:
Programs that aim at addressing female economic empowerment at the community level are eligible. However, household level interventions, such as a change in land use rights and inheritance laws, are not eligible. Interventions that consider a broader base, such as increasing the share of women CEOs or representation of female interests in macroeconomic policies, are not eligible for inclusion.
3.1.4. Comparisons
We include clearly defined comparison groups, including a control group, which did not receive an intervention, waiting lists, business as usual, paired matching and before and after comparisons. We exclude studies that compare outcomes with an alternative intervention. Comparison observations can be measured contemporaneously among separate groups or non-contemporaneously among the same group over time.
3.1.5 Types of outcome measures
a. Direct outcomes on human development indicators
We consider primary outcomes at the household level in four domains of human development: health, education, financial (income, assets) and attitudes towards gender equity. These outcomes are to be observed for the different interventions included in the analysis. Below you will find a list of the different domains captured in the analysis with some examples of the metrics used: Health domain (e.g. Anthropometrics of individuals living in the household, diseases/sickness reported after the intervention, dietary information, number of meals per day) Education domain (e.g. school enrolment and school attendance, years of schooling, level of education, degree of studies, number of years of experience in the work place) Wealth domain (e.g. financial assets, ownership of physical assets and land) Income generated by women (e.g. female wages, number of hours worked, type of occupation of the female, sector of occupation: formal or informal, type of productive investments: food vs. cash crops, manufacture, etc.) Household income using proxy variables (e.g. yields or productivity, use of innovative marketing strategies, use of registrations and accounting in enterprises, use of credit and savings accounts, investments, membership in associations) Attitudes towards gender equity (e.g. household decision-making index and attitudes towards gender equity) Studies that measure outcomes at the community level are not eligible. Moreover, we aggregate outcomes at the household level; hence we do not consider the individual outcome on children.
b. Secondary outcomes
Studies that consider intermediate outcomes (e.g. change in female bargaining power), but which do not consider the impact on human development indicators are excluded. The intermediate outcomes we consider are: Variability in income can be measured by the number of times in which a household did not have sufficient food and the number of unemployed household members. Female bargaining power (e.g. proportion of income contributed by the women in the household, access to family resources, decision power of women in consumption decision, control over income). Fertility rate (i.e. ideal number of children by women and men, use of family planning) Investment in girls (relative number of years of education of girls vs. boys, relative gender indicators of height for age, gender gaps in educational achievements by age) Household consumption (e.g. share of expenditure in education, health and food consumption). In addition, to the extent possible we capture the use of preventive healthcare measures such as vaccinations and the consumption of processed food. To capture the potential effects on the triple burden of work, we consider time use in different activities (e.g. work, housework, leisure, child rearing).
3.2. Search strategy
3.2.1. Electronic searches
The search strategy aims to find both published and unpublished studies. A three-step search strategy will be utilized in this review. In the first stage, studies will be screened for the text in the title and abstract and the keywords in the article description. We will search in the following databases: 3ie database of impact evaluations Campbell data base of impact evaluations Econlit SSRN EconPapers ScholarGoogle Proquest - IBSS, ASSIA, Social Sciences Database, Education Collection. Ebsco Discovery – Repec& World Bank E-library, WHO Global Health Library World bank e-library Research Gate.
The second phase will consist of backward and forward citations, where reference lists of included studies will be searched and reviewed for additional studies using Google Scholar. We will use citations from the following papers and systematic reviews:
Lastly, in an effort to capture studies that are still unpublished, we will send emails to the leading authors in the field of gender and development requesting unpublished working papers. A record will be maintained describing the databases searched, the keywords used and search results from each search engine.
3.2.2. Other searches
We will also do searches in key development economic journals including: Journal of Development Effectiveness Journal of Development Economics International Journal of Sustainable Development Indian Growth and Development Review Journal of International Development Economic Development and Cultural Change Feminist Economics and World Development Journal of Development Studies
Multilateral organizations working on development issues would also be consulted. Among these we will focus on: United Nations Development Fund for Women United Nations Development Programme United Nations Children's Fund African Development Bank Asian Development Bank United Kingdom's Department for International Development United States Agency for International Development World Bank International Fund for Agricultural Development Inter-American Development Bank International Labour Organization
3.2.3 Search Terms
For searching the above databases, our team will decide on search terms for each section of the systematic review and try test-searches before finalizing key-words. An example of the search terms used in Econlit and ERIC are presented in appendix 2. This search will be modified to fit the search format for each database.
3.2.4 Search Query
The following search terms are used to search the selected databases and were modified to fit the search format for each database. For an example see appendix 2.
3.3. Data Collection and Analysis
3.3.1. Selection of Studies
We will follow the usual procedure of unbiased and systematic data collection and analysis. Identified studies will be entered in a Zotero database that includes the abstract and link to the paper. This database will be exported to JabRef. This will generate a master database that includes all papers, articles and reports. After automatic elimination of duplicates, we will create a database with the remaining studies for title and abstract screening. Title and Abstract screening will be carried out by one researcher. To avoid exclusion of any potentially relevant study, at this stage we shall include a study if there is confusion about its relevance.
Criteria for exclusion will be reported in this stage. We will include studies that: Consider the impact of a relevant intervention for female economic empowerment Use adequate methods to control for endogeneity and draw causal inference Refer to at least one of the relevant outcomes Refer to a middle-income or low-income country Rely on original empirical analysis on quantitative evidence
Studies that refer to the impact of female empowerment on diverse outcomes but that do not use original data would be classified as Priority 2 and excluded. However, these studies will be revised to identify additional relevant studies. Studies that use macro-economic evidence or focus on theoretical models alone will be classified and excluded from the revision.
The studies that are selected for full text screening will be stored in another database and distributed to two authors to determine eligibility. We will keep records of the reasons for excluding the study from the review. We will solve disagreements on inclusion via discussion or by a third independent reviewer.
3.3.2. Data Extraction and Management
The documents selected for data extraction will be randomly allocated to two team members who would work independently extracting information from each study. Both team members will use a pre-piloted data extraction form. Disagreements in coding will be resolved through discussion. If no agreement can be reached, a third independent member of the team will be present to resolve the disagreement. In appendix 3, we have included the data extraction form that we plan to use. This form was piloted with five studies to test for the consistency and the objectivity of the form. It includes basic information on the study, intervention type, information about the target groups and its demographics, information on sample sizes, outcome variables, detailed information on treatment and comparison groups, relevant statistics, information on the size of the effects and moderator variables.
We will collect information on moderators or institutional factors that take into consideration the heterogeneous effects of the interventions for different subgroups. As moderator variables we consider: Social barriers to female empowerment. This is captured by the geographical location of the program (country, urban/rural area), religion (other stated identities), and perceptions of the role of women in society from social value surveys (before the intervention). Male control over resources is captured by questions related to female bargaining power before the intervention was implemented and, if available, the rate of domestic violence before the program was implemented. Ideally, we would like to consider heterogeneous effects according to male control over resources. Credit constraints are captured by the financial situation of the beneficiaries before the program began, distance to financial institutions, perceived financial access, use of formal and informal financial services before the intervention. The existence of productive investments and market access is captured by the type of productive investments (food vs. cash crops), membership in groups or associations and the use of modern technologies. For programs in urban areas, we will extract information on the type of work generated (i.e. formal/informal, skill/unskilled, temporary/permanent, health risks). Negative shocks. To capture the effect of adverse shocks we will use information on self-reported shocks (i.e. weather shocks, price shocks, etc.) and support from an individual's social network (i.e. receive help/offers help to family and friends). We would also consider heterogeneous effects by the wealth level of the household. To do this we want to use information prior to the program on levels of schooling, income group or poverty level.
3.3.3. Assessment of risk of bias in included studies
Two independent authors will assess the quantitative study rigor of eligible studies using a published list of criteria, developed by 3ie, to assess the risk of bias in social experiments and quasi-experiments (Hombrados and Waddington 2012). In appendix 4, we have included a form for assessing the risk of bias. Here we try to make sure that the studies have been conducted in a high-quality manner and that reporting of all details and statistics, as well as any shortcomings, are included in the paper.
The critical appraisal tool will assess the likely risk of the following biases:
Confounding. Was the identification method free from any sources of bias due to confounding or were sources of bias adequately corrected for with an appropriate method of analysis? Separate evaluation questions would be used for each of the identification methods (Randomization, Regression Discontinuity Design, Instrumental Variable, and Difference in Difference, Propensity Score matching, and before and after comparison).
Sample selection bias. Is there a differential selection of participants in the study groups at the baseline (censored data) or follow-up (attrition)?
Spill-overs, cross-overs and contamination. Was the study adequately protected against spill-overs, cross-overs and contamination?
Outcome reporting. Was the study free from selective outcome reporting?
Analysis reporting. Was the study free from selective analysis reporting?
Performance bias. Was the process of being observed free from motivation bias?
Other risks of bias. Is the study free from other sources of bias?
We will judge whether a study is subject to a high/medium/low risk of bias for each of these bias categories using the following decision rules:
We will follow Brody et al. (2015) and critically appraise the studies according to the likely risk of bias. We will assess the risk of bias among several domains using the decision rules in the IDCG risk of bias tool. For further details, please see section 7.4 in the Appendix. In addition, the following classifications will be made according to their respective definitions.
We will group question/answer combinations into low, high and medium risk of bias. For example, in question 1, an answer “Yes” will be coded L for a low risk of bias, an answer “No” will be coded as H for a high risk of bias and an answer “Unclear” will be coded as “M” for a medium risk of bias. Then we would give values to L, H, and M – 0, 1, and 0.5 respectively. For each study we will calculate the total value and the closer the total number is to 0, the lower the risk of bias is. Examples of low, high and medium risk of bias are given below:
Low risk of bias: appropriate and clearly described selection of participants, measurement of exposure and outcome variables, use of analytical methods to test for the initial differences between the treatment and control groups; low risks of spill-overs or contamination; low risk of outcome and analysis reporting bias.
Medium risk of bias: inappropriate or unclear use of one of the following: selection of participants, measurement of exposure and outcome variables, use of design or analytical methods to control confounding, assessment of spill-over or contamination risks; medium risk of outcome and analysis reporting bias.
High risk of bias: inappropriate use of two or more of the following: selection of participants, measurement of exposure and outcome variables, use of design or analytical methods to control confounding, assessment of spill-over or contamination risks, high risk of outcome or analysis reporting bias.
Unclear risk of bias: unclear description of any of the following: selection of participants, measurements of exposure and outcome, study design or analytic methods to control for confounding, assessment of spill-over or contamination risks.
We will report the risk of assessment bias for each included study conducting sensitivity analyses by the overall risk of classification bias and, where sufficient studies are available, for each risk of domain bias.
3.3.4. Measures of treatment effect
Here again we will follow standard procedures in systematic reviews (Higgins and Green (2011) and, where possible, we will calculate the standardized mean differences (SMDs) for continuous outcome variables and the odds ratios (ORs) for dichotomous outcome variables.Treatment effects will be calculated as the ratio of, or difference between, treatment and control observations in a consistent way, such that outcome measures are comparable across studies. SMDs and Odds Ratios will be converted to SMD using appropriated formulas. Where it is not possible to calculate SMDs, we will calculate odds ratios, which measure the ratio for the odds of success in the intervention group relative to the odds of success in the comparison group.
3.3.5. Methods for handling dependent effect sizes
During the data extraction process, we will keep records of the paper, data sets and interventions used in the analysis. In case there are multiple papers or studies looking at the same intervention, we plan to include an aggregate effect per category (income, health, education, attitudes). Where studies report multiple effect sizes by sub-group, we will report data in separate analyses or compile estimates prior to meta-analysis by calculating a single sample-weighted average effect size for each study, using the appropriate formulas to recalculate variances and standard errors and making covariance assumptions as necessary (as per Borenstein et al., 2009 and the Cochrane Handbook Chapter 16). We will also differentiate papers from studies and interventions, so that the analysis accounts for multiple papers from the same underlying study/dataset and multiple studies from the same program/intervention.
We will attempt to do a meta-analysis for the studies with comparable variables. A single meta-analysis will include one estimate per study. The unit of analysis will be the study, but if there are multiple studies of the same program, then the unit of analysis will be the program itself. This is because the program level unit of analysis may be a more policy relevant metric. If we have sufficient effect estimates, then we intend to carry out robust meta-regression (using Stata command robumeta), rather than synthetic effects which are rather data limiting. If the effects are reported by groups, estimates will be pooled to calculate a single effect size for each study using the appropriate econometric methods. In regards to heterogeneous effects, we will standardize the unit of measure or make use of methods such as stratification, fixed effects and random effects models (Veroniki et al. (2015).
3.3.6. Unit of Analysis Issues
We shall correct the standard errors of the effect sizes by taking into account any clustering in the study design. We shall follow Waddington et al. (2012) to incorporate these methods.
3.4. Statistical Procedures and Conventions
This section describes the methodology that we plan to follow to address the three research questions mentioned earlier. Specifically, for the first question, we aim to conduct meta-analysis and meta-regression to evaluate the effect of female empowerment on human development. The second question is basically to analyze the intermediate outcomes to identify the channels through which the intervention may affect human development. For this mediator analysis related to the second question, we will conduct meta-analysis and meta-regressions wherever possible considering data availability on mediator variables. If sufficient data is not available, then for both the first and second questions we will complement the quantitative synthesis with a narrative analysis of the effects. Our third question will be addressed through moderator analysis, sub-group analysis, and meta-regressions when possible. The following sub-sections elaborate on these methods further.
3.4.1. Quantitative Synthesis
We will present the synthesis of the evidence from the included studies through narrative and statistical analysis of comparable effect sizes using meta-analysis. Meta-analysis is useful in synthesizing quantitative evidence from multiple studies as it considers the statistical power of the effect estimated in these various studies. We will first categorize the studies based on type, since we consider multiple interventions and outcomes in our systematic review. We can calculate the standardized mean differences, or odds ratios, which are appropriate for comparison across similar types of treatment effects.
However, even within the scope of randomized control trials, different studies may estimate various types of treatment effects, e.g. the average treatment effect, intent to treat effect, local average treatment effect etc. These different types of treatment effects may not be comparable as they are specific to the case and context. Therefore, we will attempt to use similar treatment effects or convert treatment effects into comparable measures (Duvendack et al 2012), or test for the differences across treatment effects using moderator analysis. In bivariate meta-analysis, we will pool across studies where: the effect sizes can be computed for comparison the outcome measures are sufficiently similar
These results will be presented using conventional tools, such as forest plots, and when possible, will be presented using inverse-variance weighted meta-analysis. We will use STATA for this purpose. In the case where meta-analysis is not possible due to the reasons mentioned above, we shall use narrative synthesis including discussions on the sample size and magnitude of the effects.
3.4.1.1. Assessment of Heterogeneity
When meta-analysis is possible, we shall test for heterogeneity across studies using the I-squared statistic that measures the percentage of variability across studies that is not due to sampling error but rather to differences in the study population, the intervention and its implementation. As suggested by Borenstein et al (2009), we will follow the rule of thumb that if the I-squared statistic has a threshold of 75 percent, then there is high heterogeneity, and if it is 50 percent then the extent of heterogeneity is low. We can also use the Q-statistic for statistical heterogeneity in the outcome variables, and the tau-squared statistic when random effects are used.
If heterogeneity is present, then we shall investigate what factors explain it by conducting moderator analysis, including sub-group meta-analysis and meta-regression if possible. Otherwise we shall discuss potential factors behind the heterogeneity through a narrative method. The moderator variables will include type of intervention (agricultural, financial access, self-entrepreneur, access to labour market and their sub-categories), geographical factors (location: rural/urban, distance to workplace, etc.), identity of the household (e.g. religion), baseline level of education, poverty level, household size, membership in group association, type of work (formal/informal, skilled/unskilled, temporary/permanent) etc.
3.4.1.2. Sensitivity Analysis
To check if the results are sensitive to the quality of the data and approaches to analysis, we shall report sub-group analysis based on the study design, and would carry out a weighted ANOVA and meta-regression according to the overall risk of bias classification, and if possible, risk of bias status for each category. If these methods are not implementable, then we shall analyze studies separately based on their study design and use a narrative method to analyse the methodological factors that might moderate the size of the effect. Also this will involve describing studies based on design and the risk of the different categories of bias.
We shall use funnel plots, qualitative assessment and sub-group analysis comparing published versus unpublished studies to assess potential publication bias.
5. Timeline
Title: 02/05/12
Protocol: 30/09/16
Draft Report: 30/09/17
Final Report: 01/12/17
Policy brief and short summary: 01/01/18
6. Acknowledgements
We would like to thank the Department for International Development (DFID), the Hewlett Foundation and the International Development Research Centre (IDRC) for funding this research project. We would also like to acknowledge the support of our advisory group and research assistants.
Footnotes
9. Appendix 1: Theory of Change
10. Review authors
| Name: Name: Marcela Ibanez |
| Title: Professor Development Economics |
| Affiliation: University of Goettingen |
| Address: PlatzGöttingenSieben 5 |
| City, State, Province or County: Göttingen, Niedersachsen |
| Post code:37073 |
| Country: Germany |
| Phone: +49 551 21662 |
| Email: |
|
|
| Name: Sarah Khan |
| Title: Postdoctoral researcher |
| Affiliation: University of Goettingen |
| Address: PlatzGöttingenSieben 5 |
| City, State, Province or County: Göttingen, Niedersachsen |
| Post code:37073 |
| Country: Germany |
| Email: |
| Name: Anna Minasyan |
| Title: Postdoctoral researcher |
| Affiliation: University of Göttingen |
| Address: Platz Goettingen Sieben 5 |
| City, State, Province or County: Göttingen, Niedersachsen |
| Post code:37073 |
| Country: Germany |
| Email: |
| Name: Soham Sahoo |
| Title: Postdoctoral researcher |
| Affiliation: University of Goettingen |
| Address: PlatzGöttingenSieben 5 |
| City, State, Province or County: Göttingen, Niedersachsen |
| Post code:37073 |
| Country: Germany |
| Email: |
| Name: Pooja Balasubramanian |
| Title: Doctoral researcher |
| Affiliation: University of Goettingen |
| Address: PlatzGöttingenSieben 5 |
| City, State, Province or County: Göttingen, Niedersachsen |
| Post code:37073 |
| Country: Germany |
| Email: |
11. Roles and responsibilities
12. Sources of support
External Sources
Funders: the Department for International Development (DFID), the Hewlett Foundation and the International Development Research Centre (IDRC).
Internal Sources
The University of Goettingen, the Research Centre on Poverty, Equity and Growth in Developing and Transition Countries.
13. Declarations of interest
We all have participated in research that is related to this research question in some way, but if any publications from our own work are determined to be eligible for inclusion in the study, we will have an independent evaluator assess the quality of the study.
14. Approximate date for submission of the systematic review.
Date you plan to submit a draft protocol: September 1, 2016 Date you plan to submit a draft review: December 1, 2017
15. Plans for updating the review
The authors agree to update the review once sufficient studies and funding becomes available.
