PROTOCOL: Effects of Payment for Environmental Services and Decentralized Forest Management on Deforestation and Poverty in Low- and Middle-Income Countries: a Systematic Review

Abstract

BACKGROUND

Description of the problem

This systematic review will study the impact low- and middle-income countries of payment for environmental services (PES) and decentralized forest management (DFM) programs on deforestation and poverty in forest communities in low- and middle-income countries. Such programs find wide application around the world as part of government strategies to manage forest loss and climate change. PES programs allow for direct exchange between those demanding “environmental services” such as protection or rehabilitation of natural forests and those in a position to provide them locally (Forest Trends, Katoomba Group and UNEP, 2008; Wunder, 2005). DFM programs relocate decision-making authority on forest use in the direction of forest communities, although the extent to which this empowers forest community members depends on the institutional context (Tacconi, 2007). PES and DFM exist alongside “community-based forest management” and “protected areas” (that is, parks and reserves) as core components of government and privately-led forest management efforts around the world (Angelsen, 2009). Governments have applied PES and DFM strategies domestically for decades to manage forests and prevent irredeemable loss of valuable endemic forest resources.

Both PES and DFM have recently been considered in the context of global efforts to manage climate change. Since 2007, governments have coordinated such efforts through the Reducing Emissions from Deforestation and Forest Degradation (REDD) initiative, which consists of inter-governmental framework agreements to facilitate the protection of forests around the world. The goal of REDD is both to reduce carbon emissions resulting directly from deforestation (accounting for about 17 per cent of emissions and arising from processes such as peat soils decay or burning) and to preserve natural forests as carbon sinks so as to mitigate the effect of other carbon emissions on climate change (Intergovernmental Panel on Climate Change, 2007). In an essay that helped to inspire the REDD initiative, Santilli et al. (2005) noted the importance of forest protection for climate change, indicating that “current annual rates of tropical deforestation from Brazil and Indonesia alone would equal four-fifths of the emissions reductions gained by implementing the Kyoto Protocol in its first commitment period [i.e., 2008-2012]” (267). Gullison et al. (2007) estimate that “[r]educing deforestation rates by 50% by 2050 and then maintaining them at this level until 2100” would have a net impact of reducing global carbon emissions by an amount “equivalent to nearly 6 years of recent annual fossil fuel emissions” (985). Furthermore, Gullison et al. estimate that per unit costs of emissions reduction through forest protection may be substantially less than other approaches to emissions reduction. This review contributes to current knowledge on effective natural forest management strategies.

An important goal of this review is to inform the debate on how welfare concerns should be included in the analysis of conservation strategies. It is natural to consider adverse welfare impacts of PES or DFM. Beyond that, the issue of whether PES or DFM mechanisms should attend to poverty is debated. For PES, there could be risks to first-order conservation objectives if PES is allowed to deviate from a simple transaction for environmental services. Pagiola et al. (2005) argue that coupling poverty goals with environmental protection goals in conservation programming may be inefficient for reaching either type of goal, and that in many instances the two objectives are orthogonal to each other, if not in direct conflict. For example, it is not apparent that poorer members of forest edge communities, who stand to gain the most from poverty alleviation programming, are necessarily the ones who constitute the greatest deforestation threat. Such individuals may have relatively little means or incentive to engage in deforestation relative to larger scale farmers or logging operations. If this were the case, then making a priority of poverty alleviation in forest edge communities would imply that programs are not targeting those whose change in behavior has the biggest conservation payoff (Wunder, 2005, p.12-14). Political imperatives to attend to distributional issues may thus undermine PES as a conservation instrument. For DFM, while some research demonstrates that local administration may be better for administering policies affecting welfare of the poor (Bardhan and Mookherjee, 2005), it is not clear that this necessarily puts it at a comparative advantage in forest management.

There are potential moral and practical counterarguments to this conclusion however (Agrawal and Benson, 2011; Porras, 2010). A moral counterargument is based on the fact that forest communities may be vulnerable to livelihood disruption due to forest protection interventions. The point is especially relevant for tropical forests. Tropical forests are appealing as targets for conservation for global climate change mitigation because of their high carbon storage density and lower local-level implementation and (in absolute terms) opportunity costs of conservation; however, tropical forests also host communities exhibiting high levels of poverty (Deveny et al., 2009; Kremen et al., 2000; Van Kooten and Sohngen, 2007). This motivates an imperative to tie PES directly to poverty relief to compensate for livelihood disruption, especially when forest protection initiatives limit poor community members' ability to exploit resources for productive purposes, whether by themselves or as hired labor (Angelsen and Wunder, 2003; Chomitz, 2007, Ch. 3; Edwards et al., 2011).

A practical counterargument is that the political sustainability of conservation strategies may be enhanced when conservation programs are shown to be coincident with poverty alleviation. If protection goals are not legitimized in this way, we might expect high potential for local level subversion, if not open confrontation. For PES arrangements, this undermines the credibility that service commitments can actually be implemented on the ground. For DFM, this raises the risk that state-mandated conservation goals will be ignored, as local officials may have neither the capacity nor incentive to implement them. Indeed, subversion risks have been demonstrated by rampant increases in deforestation in protected areas after DFM in Indonesia (Burgess et al., 2011), in the substantial discounts applied to offsets in PES schemes based on services from developing countries (Conte and Kotchen, 2010), and the regularity with which local corruption has lead to higher rates of deforestation following DFM around the world, as discussed by Ostrom (1990, p. 23). Such political concerns are especially salient in government-mediated service arrangements. When host governments offer environmental service contracts, there is the potential for abuse of power, as with a case in Uganda documented by Oxfam (Grainger and Geary, 2011). If only wealthier members of forest communities may be in a position to supply environmental services, PES may exacerbate local inequality both by increasing the wealth of service providers and reducing employment opportunities in enterprises involving deforestation (Wunder, 2005). Poverty alleviation might be an appropriate condition to minimize risks of such harm and the potential for hostilities between poor forest community members on the one hand and governments or wealthier members of forest communities on the other (Mapedza, 2006). To the contrary of the efficiency argument, poverty alleviation may be essential for realizing the potential of PES and DFM programs. Such benefits may arise through the conduct of the programs themselves or as add-ons to the conservation-focused aspects of the programs. As Pagiola et al (2004) demonstrate, the extent of poverty reduction benefits will depend on how well the program design captures potential synergies.

Theory alone cannot settle such a debate: evidence should be marshalled to assess the relative merits of the efficiency claim versus the counterarguments discussed above. This would involve synthesizing evidence on (i) how PES and DFM programs affect local poverty and in turn (ii) what consequences these poverty effects have for success in forest protection. The theory of change that our review will test is one that hypothesizes poverty impacts as mediators of environmental impacts. We also analyze any incidental poverty impacts that might be associated with PES and other decentralized forest management arrangements.

Description of the intervention

We evaluate the current evidence base on PES and DFM programs and their impact on both deforestation and forest community welfare, focusing in particular on poverty impacts for forest communities. The reason for assessing PES and DFM side-by-side is driven by two considerations. First is the need to fill particular gaps in the current literature. As we note below, both protected areas and community based forest management are the subject of other reviews. Therefore, this review is meant to complement those other reviews as part of a general assessment of major strategies currently employed to manage natural forests in low- and middle-income countries. Second, as our theoretical discussion in section 1.3 indicates, many of the same institutional and economic-contextual factors would seem to be relevant as moderators of the effectiveness of both PES and DFM. For this reason, a joint analysis allows us to make a more general statement about how institutional and economic-contextual factors moderate the possibility of effective forest management.

It is useful to clarify what we mean by “PES program” and “DFM program” interventions for the purposes of this review. At the most basic level, PES (also called “payment for ecological services” or “reward for environmental/ecological services”) refers to voluntary supply of a well-defined environmental service (for example, maintenance of forest density in a designated area) in exchange for payment or other reward from a buyer entity. Our conception of PES thus encompasses what is sometimes referred to as “rewards” for environmental services or “compensation” for environmental services. Whether PES should be defined to include additional provisions is subject to some debate. Wunder (2005), in a sophisticated treatment of this definitional issue, defines an “ideal” PES program as one that involves (i) such a voluntary exchange as well as (ii) the payment being issued conditional on delivery of the environmental service (as opposed, say, to being issued prior to and in expectation of delivery of the service) and (iii) the buyer entity being the immediate users of the environmental services. Such an ideal form of PES is appealing in theory, as it would seem to define an “incentive compatible,” and therefore sustainable, approach to conservation.

But we feel that this ideal form is ill suited to a review of PES. As Wunder (2005) and Wunder, Engel, and Pagiola (2008) note, such ideal type programming is extremely rare in developing countries. Conventional economic theory provides possible explanations for this. To the extent that any local demand for environmental services exists (and it is not clear that it always will), the benefits from such services have the quality of public goods (Samuelson, 1954). Therefore, any single beneficiary would have an incentive to free ride on others' purchase of environmental services, introducing the potential for market failure (Salzman, 2010). Furthermore, if demand originates predominately among non-local or foreign entities, transaction costs may make direct contracting with local services providers impractical, again undermining the potential for market formation. Thus, government or NGO intervention is likely to be required to overcome market failure risks and organize the purchase of environmental services in developing country contexts. In that case, Wunder's condition (iii) that buyer entities are the immediate users of the environmental services may not be considered to hold (indeed, Wunder, Engel, and Pagiola (2008) explicitly rule out government-managed PES as satisfying the ideal type). That being the case, governments or NGOs may find conditional payment to be sub-optimal in satisfying their manifold goals. For example, an NGO may have a goal of building trust with forest communities, in which case the NGO may find more appealing a “gift exchange” approach (Akerlof, 1982), where at least partial payment or administration of benefits is issued prior to service delivery, with the expectation that recipients will reciprocate by providing environmental services.

Taking this into account and in a manner consistent with the studies reviewed by Wunder, Engel, and Pagiola (2008), we use the broad definition of PES as “voluntary supply of a well-defined environmental service in exchange for payment from a buyer entity,” where payments need not be monetary but may come in the form of other material benefits and the “environmental service” must involve the maintenance or rehabilitation of natural forests. “PES programs,” then, are actions undertaken by corporate or government entities to facilitate PES by establishing necessary legal frameworks (for example, by demarcating property rights) or connecting potential “buyers” to potential providers of environmental services. While PES refers to a type of exchange that may be realized at any time, PES programs have clear start dates that allow in principle for an evaluation of their impacts.

PES programs are currently underway in many countries with large forest areas, including Malawi, Indonesia, Mexico, Costa Rica, Bolivia, Malawi, Brazil, Madagascar, Mozambique, China, Honduras, Nicaragua, Cameroon, and Vietnam. PES programs have become considerably common in recent years. An example is given by The and Ngoc (2006), who describe a Vietnamese pilot PES scheme in which two groups of farmers (45 in 2003 and 41 in 2004) were paid VND 140,000 per hectare per year (approximately USD 7.00). These funds came from a government reserve set aside for investment in the protection and development of forests. This project covered a total of 206 hectares. Each farmer was paid in three installments: 50 per cent of the total when the contract was signed, 30 per cent after six months, and 20 per cent after a year. Any farmer who failed to keep his contract was required to pay the money back, along with a fine. If a farmer violated his contract, they would not be permitted to log their land until all money was paid back. The Vietnam program focuses on small-scale farmers with an average of 1.6 hectares of forestland. In other cases, such as in Costa Rica, PES participants include not only individual farmers, but for-profit companies, conservation/development associations, groups of farmers, and entire indigenous communities (Porras, 2010). In Costa Rica, contracts can cover 300 (or sometimes 600) hectares—a much larger scale than Vietnam—and are renewable every five years. The money for each contract comes from the National Forestry Fund (FONAFIFO). For individual farmers, most contracts were signed for 30-100 hectares of land. Costa Rica has PES policies specifically directed at poorer farmers, but overall the system is aimed at anyone who owns forestland. In evaluating the impact of PES programs, such differences in their design must be taken into account.

Our definition of DFM follows Irawan and Tacconi (2009), defining it as the de jure transfer of control of state-managed forest resources from central government to local authorities. Forest management refers to decision-making over designating existing state-owned forest areas as protected versus available for conversion, timber harvesting, or other types of forest resource exploitation. The focus on de jure, rather than de facto, conditions is justified by the desire to study manipulable policy instruments.

DFM is similar in certain respects to “community-based forest management” (CBFM), which involves the transfer of control of forest resources to local communities through statutory recognition of communities' rights to manage forests (Agrawal and Angelsen, 2009). Whether DFM should be considered as distinct from CBFM is subject to some debate. Lawry (2012), for example, conceptualizes a spectrum of strategies of “forest governance devolution,” which ranges from strategies involving no new property rights to forest communities (for example, revenue sharing models) to strategies involving the statutory recognition of forest communities' customary title to forest lands. Lawry thus proposes that forest governance strategies should be evaluated in terms of variation along this spectrum. While we agree that this approach is appealing conceptually, we also agree with Tacconi (2007) that DFM, in terms of deconcentration (or devolution) of government power, constitutes a distinct strategy relevant in many parts of the developing world. DFM and CBFM may coincide, however, if local authorities that receive authorization to manage forests happen to be the leadership of forest communities. Such would be the case when forest management is devolved to local authorities in autonomous indigenous areas for example. As discussed below, CBFM strategies, unlike DFM strategies as far as we can tell, have already been subject to systematic review (Bowler et al., 2010). What makes the most sense at this point is to evaluate the evidence base on DFM systematically in anticipation of a synthesis that then compares results across strategies and highlights important points of intersection.

A “DFM program”, then, refers to significant de jure transfer of management responsibilities for natural forests, including authority to grant concessions or establish use restrictions, from centralized to local authorities. Thus, DFM programs have clear start dates that allow in principle for evaluation of their impacts.

DFM programs are also in effect all over the world, including Bolivia, India, Uganda, Mali, Ethiopia, Guatemala, the Philippines, Honduras, Kenya, and Nepal. For example, in Uganda, a 1987 law transferred control of forestry resources to local institutions (Banana et al., 2002). The law created five tiers of decentralized institutions: villages, parishes, sub-counties, municipalities, and districts. Another example comes from Guatemala, where the Forestry Law of 1996 decentralized forest regulation to municipal governments (Holder & Chase, 2011). Along with the devolution of forest authority, the Forestry Law also required that municipal governments provide technical advice to local users and assist the central government's forest authority to enforce natural forestry laws. Such differences in institutional arrangements and complementary programming need to be taken into account as sources of heterogeneity in the impact of DFM.

How the intervention might work

Figure 1 presents a schematic representation of the theory of change that we will evaluate with this review. The causal arrows in the diagram are not meant to characterize all conceivable causal relationships, just the ones that we seek to test. We have drawn a causal arrow that flows from poverty to deforestation, and not the other way around. This does not mean that we assume no effects of deforestation on poverty. It is meant to clarify the particular mediating relationship that interests us in this review. The debate that we seek to comment on is the one discussing whether conservation programs should incorporate poverty alleviation policies in order to boost conservation effectiveness, rather than the other way around.

Illustrating the theory of change

We embed the causal relationships between PES/DFM and poverty/deforestation in Ostrom (2007)'s generic analytic framework for conservation dynamics, which defines the context of environmental interventions in terms of the resource system, governance system, resource units, and resource users. In this review, the governance systems and resource users are the key areas of contextual variation that may moderate PES and DFM effects, while resource system (forest systems in developing countries) and resource units (forested land) are assumed to be of secondary concern once we condition on governance systems and resource users, with the latter understood as being potential agents of deforestation.¹

The two most basic hypotheses that we wish to test are as follows: •

H1a & H1b: (a) PES and (b) DFM reduce deforestation rates.

•

H2a & H2b: (a) PES and (b) DFM have non-negative impacts on local poverty levels.

The focus on non-negative, as opposed to “positive” impacts per se, reflects a primary concern in many evaluation studies, which is to ensure that policy interventions do no incidental harm in association with ultimate goals, which in this case are taken to be reductions in deforestation.

These hypotheses reflect the policy interests motivating this review, although by no means do they reflect a consensus of opinion on how PES or DFM may work. The justification for PES, ostensibly, is that without intervention, benefits of forest protection are external to those who would contribute to deforestation. PES programs thus harness and redirect the value of such externalities in the form of payments to those who would otherwise contribute to deforestation (Wunder, 2005; Angelsen 2010). In principle, a PES arrangement operates as a standard performance-based contract, whereby upon performance, in terms of forest protection or rehabilitation, payments are issued (Ferraro, 2011). Material benefits to program participants follow naturally, whether as a result of direct payment to individuals or provision of other material rewards (for example, infrastructure) to groups in exchange for service provision. A number of conditions are necessary for such an arrangement to work. The payments must be targeted toward those whose activities significantly affect deforestation rates, the payments must be sufficient to overcome opportunity costs of conservation, and the “sellers” must be induced to carry out the conservation service rather than pocketing the payment and continuing with deforestation (Ferraro, 2011). These conditions may fail if institutional conditions are not right. Those designing the program may have inadequate knowledge or capacity to target properly or to set appropriate payment levels.² Constraints on PES buyers' ability to monitor and sanction may allow would-be sellers of conservations services to get away with pocketing benefits without actually reducing deforestation. These institutional conditions are captured by the “governance system.” If the PES buyer is a government agency, then their ability to monitor and sanction will depend on local public administration, law enforcement capacity, as well as levels of corruption. Similarly, if the PES buyer is a private entity, these same institutional factors will determine the private entity's ability to hold non-compliance to account. In addition, the nature of the poverty alleviation impacts will depend on the targeting of the program and whether local institutions represent the interest of the poor and therefore help to ensure that benefits accrue to the poor (Corbera et al., 2008). Average incomes may rise, for example, but these gains may be concentrated among the non-poor, in which case poverty levels may be unchanged (or even increase if cessation of deforestation either reduces demand for labor among the poor or otherwise infringes on livelihoods of the poor).

The ostensible justification for DFM rests on one two possible accounts. The first account emphasizes administrative rationality: local authorities are assumed to be (i) more aware of local conditions, meaning that conservation or sustainable forest use policies can be more efficiently designed and executed, and (ii) more responsive to forest communities' interests, which in turn are assumed to prefer maintaining forests (Tacconi, 2007). The combination of these conditions is expected to result in both improved consumption for forest communities and reduced deforestation. The second account emphasizes incentives for sustainable resource management (Gibson et al., 2000; Lawry, 2012). In developing countries, national level administration of forests typically involves designation of national reserves whose use is controlled by national authorities. Forest and forest-edge communities are granted no rights to forest properties. In a situation of lax enforcement by national authorities, the incentives are for private actors to exploit the window of opportunity to extract forest resources as much as possible in anticipation of national authorities eventually enforcing use restrictions or granting concessions to firms. Decentralization typically involves (i) transferring management rights to local authorities, who are in a better position to monitor usage restrictions, and (ii) allowing these local authorities to tax concessions, thereby generating revenue for investing in local development. Under decentralization, local authorities may now have an incentive to manage forest resources sustainably to secure a steadier revenue stream. The result should be less forest degradation and reduction in levels of poverty.

The assumptions underlying these “ideal type” accounts may be questioned, however. When authority is devolved to local districts, decisions about conservation must be made with respect to local, within-district needs and conditions. If such devolution means an end to access to resources outside the district, then local authorities may find it necessary to open up natural resources such as forests to exploitation in order to satisfy material needs. Burgess et al. (2011) found this to be the case as new districts were formed as part of the decentralization process in Indonesia. A recent example comes with the creation of the new province, North Kalimantan, in Borneo, Indonesia. North Kalimantan was carved out of East Kalimantan province. Because North Kalimantan will now be cut off from East Kalimantan's extractive industry revenues, there is fear that the North Kalimantan leadership will seek to exploit forest areas by allowing mining, clearing for agriculture, and logging.³ In addition, by carving up territories into separately administered districts, consequences of environmental degradation that were once “internalized” by local authorities are made “external.” For example, Lipscomb and Mobarak (2011) found that when territories in Brazil surrounding rivers were carved into separately administered counties, the amount of pollution that upstream users sent downstream increased. This was because the consequences of pollution downstream were no longer of concern to authorities managing upstream parts of the rivers.

Furthermore, local authorities may have inadequate capacity to acquire the necessary knowledge. Assuming forest communities do prefer to reduce rates of deforestation, responsiveness to such interests depends on the extent to which local authorities owe their jobs to forest communities via elections or other channels of influence and whether organizations are present that advocate for local conservation interests (Kauneckis and Andersson, 2009). But forest communities may actually prefer more rapid deforestation, in which case accountability would work against conservation. Thus, the effects of DFM on deforestation are likely to depend on levels of public administration capacity, corruption, and the interaction of local democratic accountability and opportunity costs of conservation borne by forest communities.

We can state these points from the paragraphs above in terms of moderator hypotheses. The hypotheses are as follows: •

H3a & H3b: (a) PES and (b) DFM deforestation reduction impacts are positively moderated by the level of local public administration and legal enforcement capacity.

•

H4a & H4b: (a) PES and (b) DFM deforestation reduction impacts are negatively moderated by the level of corruption in government.

•

H5: Given that the opportunity costs of conservation borne by forest communities are sufficiently low, DFM deforestation reduction impacts are positively moderated by the level of local democratic accountability.

Section 1.1 described positions in the debate over the possible mediating role of poverty conditions for deforestation outcomes. The skeptical take is that the two dimensions are orthogonal or even conflicting. The alternative position is that the “legitimacy effects” of poverty conditions affect deforestation rates. This alternative position postulates that members of forest communities interpret the “legitimacy” of a program in terms of whether material benefits tend to flow to those in need or not. For PES, the viciousness of the dilemma of attending to distributional concerns versus targeting major agents of deforestation likely depends on levels of local inequality in terms of holdings and vulnerability due to cessation of deforesting activities (part of the “resource users” context) as well as the political position of those who stand to lose from cessation of deforesting activities relative to those who stand to gain from PES. Thus, we have the following hypotheses: •

H6a & H6b: (a) PES and (b) DFM deforestation reduction impact is positively mediated by poverty conditions.

•

H7: PES deforestation reduction impact is negatively moderated by prevailing levels of local inequality in holdings of forested property and vulnerability due to cessation of deforesting activities.

In testing these hypotheses, we will have to control for variations in the design features of PES and DFM programs. As discussed in section 1.2, PES programs vary by the size and terms of the payments offered. DFM programs vary in the extent to which local authorities are granted rights to offer concessions, tax such concessions, or sell property rights to forested lands. Such differences across programs will reflect policy-makers' adaptation to contextual factors. In our analysis, we will study how contextual variables, and in particular the moderating factors discussed above, affect the PES and DFM design. To the extent that our data will allow, we will assess how different combinations of moderating factors and program designs compare in terms of their deforestation reduction and poverty alleviation impacts (Fisher et al., 2010).

As mentioned above, we consider PES and DFM side-by-side for reasons that have more to do with gaps in the current literature than because of conceptual overlap between the two forest protection strategies. Clearly the two strategies are not substitutes—it is quite conceivable that the same land and individuals could be subject to both PES and DFM at the same time. How might the two strategies complement each other? DFM programs may involve both the transfer of forest management responsibilities to local authorities as well as the conditional transfer of resources to such authorities, with conditionality based on forest protection outcomes. Such an arrangement would function, effectively, as a PES program that recognizes local authorities, as opposed to individual landholders, as service providers. It would presumably abide by the same dynamics as those described above for PES programs targeting individuals. Publicly administered PES programs may be run through local authorities. If so, then we would expect the effectiveness of such a program, relative to a nationally administered program, to be moderated by same set of factors that we have identified above: local administration capacity, corruption, local opportunity costs of conservation, and democratic accountability. If the local context presents conditions favorable to DFM on these variables, then we should expect such a context to be especially favorable to an integrated DFM and PES strategy.

The “project-by-project” synthesis approach taken in most systematic reviews risks violating the considerations about “leakage” that informed the establishment of REDD (Santilli et al. 2005; Alix-Garcia et al., 2012). That is, a project-by-project approach runs the risk of misstating global impact if it fails to account for the possibility of leakage whereby, for example, deforestation reduction in one area is offset by increases in another. With the possibility of leakage, we cannot judge what would be the effect of global application of a program through simple comparisons of areas that received the program versus those that did not. It is possible that global application of a program could have zero effect, but partial application could induce significant displacement of deforestation and therefore give the appearance of a positive effect. For these reasons, in our assessment of the evidence, we will determine the extent to which global or leakage invulnerable baselines are used to assess the possibility of leakage that could undermine inferences about global effects from project-specific impact estimates.

Why it is important to do this review

While the environmental science is clear in characterizing the potential gains from forest conservation (Santilli et al., 2005; Gullison et al., 2007), it remains for social scientists to provide insights into how institutions and incentives may be arranged to realize such potential (Gibson et al., 2000). Furthermore, as Ferraro et al. (2011) and Ferraro (2011) explain quite convincingly, there is a need to move toward credible estimation of the demonstrated impact of conservation programs rather than to continue with approaches that estimate the worthiness of programs on the basis of elicited valuations of environmental services combined with hypothetical projections of the services that a program is supposed to deliver. Realized impacts may depart substantially from hypothetical projections, in which case the latter are not a reliable guide for policy. PES and DFM are among the most prominent national-level institutional and incentive-based approaches to forest conservation (Angelsen, 2009). At the same time, the discussion in the preceding section makes clear that their likely impact cannot be deduced from theory alone. With respect to conservation per se, in theory, PES and DFM have the potential to make forest protection more incentive compatible and efficient (Angelsen, 2009; Gibson et al., 2000). However, evidence from case studies of PES and DFM programs is inconclusive about the effectiveness of such programs for forest protection; this may reflect how the implicit theories used to design PES and DFM programs have failed to account for local structural and institutional context (Angelsen, 2010; Ferraro et al., 2012; Gibson et al., 2000; Tacconi, 2007; Wunder, 2005).

A few recent studies have had similar goals as this review, and so it is important to clarify our added contribution. Bowler et al. (2010) have produced a high quality systematic review of CBFM interventions, which is part of the reason that we have decided to apply a tight definition of DFM that highlights differences with CBFM. Other recent studies with similar substantive goals as ours, but that differ in terms of their methodologies, include the following: •

Pattanayak et al. (2010) review theoretical motivations for forest-oriented PES and findings from eight quasi-experimental studies and 18 case studies, but do not meet the search and synthesis standards of a systematic review;

•

Wunder et al. (2008) review evidence on distributive effects of forest-oriented PES programs from case studies, but do not provide quantitative synthesis;

•

The volume edited by Angelsen (2009) contains chapters that describe varieties of forest conservation policies, including forest-oriented PES and DFM programs, but these reviews do not meet the search and synthesis standards of a systematic review;

•

Kauneckis and Andersson (2009) conduct a cross-national study of forest-oriented DFM impacts but only for Latin America.

Our contribution above and beyond these studies is the application of systematic review methodology to the current evidence base covering all regions of the developing world. Of course, our ability to generate insights that go beyond the existing reviews depends on whether the current evidence base permits more systematic synthesis than what has already been done. This is something that we can assess only after conducting our search of the literature. Our review also complements reviews by Geldman et al. (2011) and Pullin et al. (2011), which focus on the deforestation impacts of protected areas, an intervention type that is distinct from PES and DFM.

OBJECTIVES OF THE REVIEW

The overall objective of this review is to evaluate the conservation and poverty impacts of PES and DFM programs and to assess the extent to which poverty impacts mediate conservation impacts. Doing so is important for moving the debate outlined in section 1.1 beyond theoretical discussions and into better-informed, evidence-based discussion (assuming relevant evidence can be found). More specifically, we seek to test the hypotheses set forth in section 1.3 above, with hypotheses H1a, H1b, H2a, and H2b being of primary interest, and the rest being of secondary interest. Our strategy for selecting studies will be targeted toward testing the four primary hypotheses as rigorously as possible. Table 1 relates each hypothesis to the types of evidence we will need. Such an assessment of impacts does not necessarily provide the basis for a full cost-benefit analysis of PES or DFM programs. We acknowledge this limitation and propose that follow-up work should focus on filling in the cost side of the equation.

Table 1

Questions and types of evidence needed for the review

Hypothesis	Type of evidence
Main Hypotheses • H1a & H1b: (a) PES and (b) DFM reduce deforestation rates. • H2a & H2b: (a) PES and (b) DFM have non-negative impact on local poverty levels.	Quantitative data on forest conservation and host community poverty outcomes for sites with PES or DFM and sites that constitute a plausible counterfactual. Qualitative accounts of whether the interventions operated as planned.
Moderator Hypotheses • H3a & H3b: (a) PES and (b) DFM deforestation reduction impacts are positively moderated by the level of local state capacity. • H4a & H4b: (a) PES and (b) DFM deforestation reduction impacts are negatively moderated by the level of corruption in government. • H5: Given that opportunity costs of conservation borne by forest communities are sufficiently low, DFM deforestation reduction impacts are positively moderated by the level of local democratic accountability. • H7: PES deforestation reduction impact is negatively moderated by the level of local inequality.	Quantitative measures of local state capacity, corruption, local democratic accountability, opportunity costs of conservation borne by forest communities, and local inequality for each study to assess covariation between these measures on the one hand and deforestation and poverty on the other. Qualitative accounts of how issues related to local state capacity, corruption, local democratic accountability, or inequality affected the functioning and effectiveness of given PES or DFM programs.
Mediator Hypothesis • H6a & H6b: (a) PES and (b) DFM deforestation reduction impact is positively mediated by poverty conditions.	Quantitative estimates of both poverty and deforestation impacts from PES or DFM for at least a subset of cases to assess covariation between the two types of impact. Qualitative accounts of whether poverty benefits (disruption) contributed to compliance (non-compliance) and effective (ineffective) functioning of PES or DFM programs.

Hypothesis

Type of evidence

Main Hypotheses •

H1a & H1b: (a) PES and (b) DFM reduce deforestation rates.

•

H2a & H2b: (a) PES and (b) DFM have non-negative impact on local poverty levels.

Quantitative data on forest conservation and host community poverty outcomes for sites with PES or DFM and sites that constitute a plausible counterfactual.

Qualitative accounts of whether the interventions operated as planned.

Moderator Hypotheses •

H3a & H3b: (a) PES and (b) DFM deforestation reduction impacts are positively moderated by the level of local state capacity.

•

H4a & H4b: (a) PES and (b) DFM deforestation reduction impacts are negatively moderated by the level of corruption in government.

•

H5: Given that opportunity costs of conservation borne by forest communities are sufficiently low, DFM deforestation reduction impacts are positively moderated by the level of local democratic accountability.

•

H7: PES deforestation reduction impact is negatively moderated by the level of local inequality.

Quantitative measures of local state capacity, corruption, local democratic accountability, opportunity costs of conservation borne by forest communities, and local inequality for each study to assess covariation between these measures on the one hand and deforestation and poverty on the other.

Qualitative accounts of how issues related to local state capacity, corruption, local democratic accountability, or inequality affected the functioning and effectiveness of given PES or DFM programs.

Mediator Hypothesis •

H6a & H6b: (a) PES and (b) DFM deforestation reduction impact is positively mediated by poverty conditions.

Quantitative estimates of both poverty and deforestation impacts from PES or DFM for at least a subset of cases to assess covariation between the two types of impact.

Qualitative accounts of whether poverty benefits (disruption) contributed to compliance (non-compliance) and effective (ineffective) functioning of PES or DFM programs.

METHODOLOGY

I. Criteria for including studies in the review [PICOS]

Our “PICOS” criteria are summarized in Table 2. Details are given in the following subsections.

Participants

This review will only include studies that focus on either (i) deforestation outcomes in forest areas in developing countries or (ii) poverty conditions of forest-dwelling and forest-edge populations in developing countries, where “forest” is defined as per the FAO Global Forest Resources Assessment, and “developing countries” are those classified as lower income, lower middle income, or upper middle income by the World Bank in the year of the initiation of the program under study.

Interventions

The review will include studies of (i) PES or (ii) DFM interventions.

Following the discussion in section 1.2, the requirements for a program to be considered a PES program are that there is a clear start date when either payments or rewards are themselves offered to individual or corporate property holders to maintain or rehabilitate (for example, via planting endemic species) natural forests, or institutions are established to facilitate such offers. We allow for those offering the rewards (the “buyer entity”) to be either public or private actors, and we allow for payments to be made in a manner that is either conditional or unconditional on the fulfillment of the prescribed maintenance or rehabilitation. These differences will be noted in our characterization of the design of each PES program.

For a program to be considered a DFM program, de jure responsibility for managing natural forest resources must pass from centralized to local authorities and this responsibility must grant local authorities the right to grant concessions or establish use restrictions. We will be flexible as to the precise level of administration to which responsibilities are passed as well as whether or not the decentralization also involves granting local authorities the right to sell or transfer titles to forested lands. These differences will be noted in our characterization of each DFM program.

Comparisons

Quantitative comparisons will be between areas hosting PES or DFM programs and areas that do not host such programs but that provide a basis for a credible, contemporaneous counterfactual. Such comparisons will nonetheless differ in the nature of the counterfactuals. For example, in the review of CBFM by Bowler et al. (2010), comparisons were made to forest areas under state management, unspecified management, protected areas, private or commercial land, semi-government management, traditional management, or no management. We anticipate similar comparisons in the studies that we analyze. We will collect information on the type of the counterfactual comparators and assess the contribution of comparator type to effect heterogeneity.

Outcomes

Outcomes of interest are (i) deforestation or (ii) poverty conditions of forest-dwelling communities. As Bowler et al. (2010) discovered, researchers vary in the precise metric that they use for deforestation impacts. The varieties of deforestation measures in their study included tree/stem density for endemic species, tree basal area for endemic species, natural forest land cover, height/size of endemic tree species, perceptions of natural forest conditions, and biomass/volume of natural forest; such deforestation outcome data are collected from either on-the-ground point samples or by remote sensing with satellite or flyover imagery (West, 2009; Achard and Hansen, 2013). Replacement natural forests by plantation and other forms degradation will be considered as deforestation.

Poverty impacts will be assessed in terms of impacts on consumption, income, or income potential for members of forest communities residing below or just above the consumption-based, two-dollar per day purchasing power parity absolute poverty line (Ravallion et al., 1991). Such outcomes are typically assessed using household economic surveys or administrative data on consumption, food security, employment, or access to productive assets (Deaton, 1997).

Our assessment of poverty impacts will have to account for any impacts on in- or out-migration into the areas under study. If a program causes outmigration among the most poor, then the resulting poverty level in the area may be less than was the case before the program. However, it would be inappropriate to take this to mean that the program helped to alleviate poverty. To the extent that is possible, we will note any recorded effects on in- and out-migration. We will most likely not be able to adjust poverty impact estimates to account for such migration effects, however we can use such effects to qualify findings on poverty impacts.

We will be particularly interested in identifying unintended impacts of forest conservation programs on local poverty conditions. We will take note of whether studies account for various types of spill-over effects, such as deforestation “leakage”. Failure to account for spill-over may result in a biased interpretation of the impact of a program.

Study Types

Table 1 above sketches out the types of quantitative data and qualitative evidence we seek for this review. We prioritize identifying rigorous studies that address hypotheses H1a, H1b, H2a, and H2b. For quantitative synthesis, we will consider well-designed experimental and quasi-experimental studies that use robust methods to construct counterfactual comparisons across intervention and non-intervention (or control) states.

We include in the quantitative synthesis only (a) randomized experiments or (b) quasi-experimental studies that employ strategies for causal identification with clearly delineated treated and control areas and use some method for removing biases due to non-random assignment of treatment. Such methods include: regression adjustment, difference-indifferences estimation, instrumental variables regression, fixed effects regression, regression discontinuity, matching, or inverse-propensity-weighted estimation. While application of such a method is sufficient for inclusion in our study, we appreciate that not all studies apply methods for causal identification with equal rigor and will assess the quality of all included studies (see below). Table 3 below lists studies that satisfy our quantitative synthesis inclusion criteria for either PES interventions or DFM interventions. Quantitative studies that will be excluded will be those that fail to establish a credible contemporaneous counter-factual. We will exclude studies that rely exclusively on before-after comparisons, as well as controlled-before-after studies that fail to adopt any of the above mentioned methods of analysis to correct for selection bias and confounding. Qualitative data will be used in the synthesis to provide descriptions and context for interventions that are included in the quantitative synthesis. Such data will be drawn from the quantitative studies themselves as well as qualitative studies that cover the same programs or settings as the quantitative studies. We will seek out such qualitative studies once the set of interventions for which we have quantitative date has been determined.

Table 2

PICOS inclusion criteria

Type	Criteria
Participants	Forest areas or forest communities in developing countries.
Interventions	PES or DFM programs.
Comparisons	“PES versus no PES contemporaneous counterfactual” or “DFM versus no DFM contemporaneous counterfactual”
Outcomes	Deforestation or poverty among forest communities.
Study types	Quantitative studies providing a robust counterfactual via randomized experiment or quasi-experiment or qualitative study with clear research objectives, original analysis, explanation of methods, and links to scholarly literature.

Table 3

Candidate studies for inclusion for quantitative synthesis

Intervention type	Study reference	Hypotheses addressed
PES	Alix-Garcia, J., E. Shapiro, & K. Sims. (2011). Forest Conservation and Slippage: Evidence from Mexico's National Payments for Ecosystem Services Program. Land Economics (Forthcoming).	H1a, H3a, H4a, H6a, H7
PES	The, B.D., & H.B. Ngoc. (2008). Payments for Environmental Services in Vietnam: An Empirical Experiment in Sustainable Forest Management. ASEAN Economic Bulletin. 25 (1): 48-59.	H1a, H2a,H3a, H4a, H6a, H7
DFM	Andersson, K., & C. Gibson. (2006). Decentralized Governance and Environmental Change: Local Institutional Moderation of Deforestation in Bolivia. Journal of Policy Analysis and Management. 26 (1): 99-123.	H1b, H3b, H4b, H5
DFM	Jagger, P. (2008). Forest Incomes after Uganda's Forest Sector Reform: Are the Rural Poor Gaining? CGIAR Collective Action and Property Rights Working Paper. 92.	H2b, H3b, H4b, H5

II. Search methods for identification of studies

Electronic searches

Our search criteria were developed after initial scoping exercises with a Campbell Collaboration information retrieval specialist. We will search the set of databases, specialist websites, and search engines that Bowler et al. (2010, pp. 55-56) searched as well as others identified to possibility contain relevant content.⁴ Our list of sources is given in section 8.1 below. Search terms will include the following: •

(“pay*” OR “reward*” OR “incentiv*” OR “compensat*”) ◦

AND (“forest” OR “deforest*” OR “ecol*” OR “ecos*” OR “environment*” OR “conservation”)

•

(“decentrali*” OR “co-manag*”) ◦

AND (“forest” OR “deforest*” OR “ecol*” OR “ecos*” OR “environment*” OR “conservation”)

To these keywords we will also apply a lower- or lower-middle-income filter based on the Cochrane EPOC filters (http://epocoslo.cochrane.org/lmic-filters). Some of the databases considered (for example, IDEAS, RUPES, and JSTOR) include search results for non-English language studies even when using English search terms and keywords. Relevance of such results will be reviewed by native language speakers (the authors are able to cover French, Spanish, German, Bahasa Indonesia).

III. Data collection and analysis

Selection of studies

The review team will apply the PICOS inclusion criteria listed in Table 2 in three stages: first to titles to remove spurious citations, then to abstracts, and finally to full texts. For all stages, we will maintain an account of the number of studies excluded, and the reasons for exclusion, by tracking references in an Endnote database. In the full text stage, excluded studies will be tagged in terms of the PICOS criteria that were violated. All screening will be done by two independent reviewers from the research team, with disagreements resolved by a third reviewer from the team. To ensure consistency in selection procedures, multiple reviewers will review a sample (of 50, for example) of citations and consistency will be assessed (using, for example, a kappa score of 0.60 to determine whether search rules are sufficiently clear). If necessary the PICOS criteria will be discussed to ensure adequate consistency.

Data extraction and management

For studies eligible for either qualitative or quantitative inclusion, we will collect data on the factors listed in Annex I. As in the comparative study of CBFM and protected areas by Porter-Bolland et al. (2011), data will be entered on a “case” basis.⁵ That is, if a study contains impact estimates from two different forest areas, each forest area will receive its own entry. These data will be double-entered into Microsoft Excel by the review team. While it would be ideal to have data on moderator variables measured at the level of the regions in which the programs under study are applied, we do not anticipate that such data will be available. Therefore, we will obtain quantitative data on the moderator variables using the relevant country-level indicators from the World Bank Governance Indicators.

Assessment of risk of bias in included studies

Risk of bias will be assessed using the IDCG Risk of Bias Tool (version March 2012). This tool collects information on the following: 1

Potential for selection bias due to non-random assignment, non-exogenous source of quasi-experimental variation in assignment, no adjustment for differences in baseline measurements,

Potential for spill-over, non-intervention based differences in treatment, or other types of interference across intervention and non-intervention units,

Selective outcome and analysis reporting based on systematic differences between reported and unreported findings,

Potential bias due to non-compliance, attrition, or otherwise missing data,

And other sources of bias.

For each study, a value of either “high risk of bias”, “low risk” or “unclear risk” will be assigned for each of these five domains. A summary judgment of the within-study risk of bias will be carried out using the criteria provided in Higgins and Green (2011, Table 8.7.a). These results will be recorded in a table that will be presented as part of the review.

Measures of treatment effect

Our preference is to use treatment effects measured as differences in means in terms of natural, substantively meaningful units. For example, for deforestation, common units for effects include changes in endemic species tree density or per cent natural forest coverage; for poverty, common units for effect include changes in poverty rates, monthly income, or employment rates. If possible, we will use effects measured on these natural scales as the outcomes in the meta-analysis. Using substantively meaningful units allows for a readily interpretable synthesis and avoids the problem of standardizing relative to standard deviation units, which may vary from study to study even if the estimated effect on the scale of the natural units is the same (Greenland and O'Rourke, 2008, p. 681).

We anticipate that many studies will report effects as regression coefficients on treatment indicator variables; such coefficients estimate differences in means (possibly differences in covariate adjusted means). While ratios of means have some advantages over differences in means in that they can be interpreted in a unit-free manner (Hedges et al., 1999; Friedrich et al., 2008), computing such ratios requires a control group mean, and we do not anticipate that such information will be available for all studies, although we will contact study authors to see if they may be furnished. In case they are, we will include ratios of means in an appendix.

Only when comparable constructs are compared in different, non-translatable units will we measure treatment effects as standardized mean differences (Hedges g), following the Campbell Collaboration (2011, Appendix 2) guidance and the approach of Bowler et al. (2010). All effect estimates will be reported alongside 95 per cent confidence intervals. As per Campbell Collaboration (2011, 6-7), sub-group estimates will be aggregated using sample-weighted averages within each intervention-outcome group. When multiple estimates are presented, the estimate that is deemed to have minimal risk of bias will be reported. Treatment effect estimates on comparable outcomes will be displayed together using forest plots.

Studies may differ in the period of assessment between the beginning of the intervention and time of data collection. In this case, it will be important to account for such timescale differences. Our approach will be to do so using meta-regressions, where outcomes measured in the manner above are regressed on an interval variable indicating number of years since the onset of the program as well as years squared to capture either diminishing or increasing returns. We will display scatter plots with these regression fits. We prefer such an approach to standardizing in terms of arithmetic or geometric rates of change, because such standardization is intended for processes exhibiting constant (either linear or geometric, respectively) period-by-period change. We do not expect that the programs in question will exhibit constant rates of change. Nonetheless, in the appendix we will report effects in terms of arithmetic and geometric rates of change to allow for comparability to other studies that employ such standardization (for example, Bowler et al., 2010, p. 19).

Unit of analysis issues

When the unit of analysis is at a lower level of aggregation than assignment units, the review will follow the guidance of the Campbell Collaboration (2011, 7-8) to adjust confidence intervals based on the intra-class correlation (ICC) adjustment to standard error estimates.⁶ Plausible ICC values will be derived from available development survey data.

Dealing with missing data and incomplete data

When studies do not report on endpoint or intermediate outcomes, the study authors will be contacted to determine whether such outcome data do in fact exist and so estimates could be produced. If so, we will seek to obtain effect estimates on these outcomes either from the study authors or using the raw study data to compute them directly.

IV. Data synthesis

Quantitative Synthesis

Our quantitative synthesis is guided by the hypotheses listed in Table 1. The “main hypotheses” (H1a, H1b, H2a, and H2b) require a synthesis of basic effect estimates on deforestation and poverty levels. For each hypothesis, our ability to produce such synthetic estimates depends on the following conditions being met (adapted from Wilson et al., 2011): i.

two or more studies meeting the quantitative inclusion criteria, with

ii.

effect sizes for common outcome constructs, and

iii.

effects measured against similar comparators.

If these conditions are met, the first type of analysis we will perform will be to test each of the main hypotheses by computing an “overall” average effect for each hypothesis. We will compute this overall average effect as a weighted average that accounts for the imprecision of each effect estimate. The estimate will be constructed using a random effects model fit via empirical Bayes in the metafor package for R (Viechtbauer, 2010). If we have a set of effects that measure effects for common outcome constructs but cannot be converted to a common scale, we will test these hypotheses by simply computing a p-value for a global one-sided significant test based on study p-values, as described by Poole et al. (2006, p. 372). All hypothesis tests will be carried out at an alpha level of 0.05, although when possible we will report model estimates of average effects with appropriate confidence intervals.

The moderator hypotheses require that we study how effect estimates covary with measures of the moderator variables. For sets of estimates that allow for meta-regression, we will test the moderator hypotheses by fitting meta-regressions based on the following mixed-effects model:

τ_{i} = α + X_{i}^{'} β + u_{i},

where Ä_i is the true effect for case i, α is a global intercept, X_i is a vector of moderators coded in such as way that the hypotheses to be tests show imply a positive association with Ä_i, β are the coefficients for the moderators, and we assume that the error

u_{i} \sim N (0, σ^{2})

.⁷ In estimating the model, we use estimates for the true effect,

{\overset{\circ}{τ}}_{i}

, as the outcomes of the regression, weighting the contribution case i by the inverse of the sum of estimated variance of

{\overset{\circ}{τ}}_{i}

and an estimate for σ_i. We employ the empirical Bayes estimator provided in the metafor package for R (Viechtbauer, 2010). We will test each of the hypotheses based on the coefficient estimates for the moderators, using one-sided tests of the null hypotheses versus β > 0 at 90 per cent confidence. (The low confidence level is fixed to account for the fact that the number of cases is likely to be small.) We will first consider each moderating factor on its own in the regression specification, and then, degrees of freedom permitting, we will check for robustness by fitting a model that includes all moderators together. If our set of effects does not allow for meta-regression, we will test the moderator hypotheses by computing p-values for global one-sided significant tests based on study p-values within strata defined by values of the moderators (Poole et al., 2006, p. 372).

A compelling test of the mediator hypothesis would require that we have a set of cases that present effect estimates on both deforestation and poverty outcomes and that these estimates can be aggregated across studies. Given such a set of cases, we would test the mediator hypothesis using the following meta-regression,

τ_{if} = γ + κ τ_{ip} + e_{i},

where Ä_if is the estimate of the deforestation reduction effects, Ä_ip is the estimate of the poverty alleviation effect, γ and κ are regression coefficients, and we assume

e_{i} \sim N (0, ψ^{2})

. The test of the hypothesis would be a one-sided test of the null hypothesis versus κ > 0 at 90 per cent confidence. Rejection would imply a positive association, as hypothesized. However, this can only be taken as suggestive evidence of a mediation effect, given that mediation effects per se are not generally identified (Bullock et al., 2010). This parametric estimate will be accompanied by a non-parametric, rank-based test as a robustness check. The test will use Kendall's correlation coefficient as a test statistic, constructing a p-value through permutation under the sharp null of zero correlation (Sprent and Smeeton, 2007, pp. 289-291). If such paired deforestation-poverty effects data are not available, then we will have to use less direct methods using either p-values for one sided tests of direct effects, applying the same methods for global one-sided significance tests described above, or most crudely, simply testing for significant association in a two-by-two table that shows counts of studies with positive effect estimates on deforestation reduction and poverty alleviation.

In addition to the synthesis that tests the hypotheses in Table 1, we will conduct and report the results of an assessment of publication bias via a funnel plot and funnel plot regression (Egger et al., 1997). The discussion of publication bias will also contain any findings regarding selective outcome reporting or other within-study reporting biases that we detect. Of course, our ability to detect such reporting biases within a study will be limited by the fact that few studies are expected to have preregistered protocols against which to compare outcomes and tests that are actually reported in the write-up.

In the event that we find studies that meet the standards for inclusion in the quantitative synthesis but do not present effect estimates that can be synthesized based on the criteria specified above, we will present tables of effect estimates and provide a narrative discussion of trends in the size and direction of the effects reported by the studies. The narrative discussion will highlight issues related to modes of measurement, nature of comparators, as well as moderator conditions that should be taken into account when comparing the different effect estimates.

In case we do not find an adequate number of studies that meet our inclusion criteria for quantitative synthesis, this part of the review will focus on a critical assessment of methods that have been employed and provide concrete recommendations for how rigorous and comparable evidence might be generated in future research.

Studies may differ in the time between program initiation and outcome measurement. We will make note of such differences and control for them as best as we can when comparing results across studies.

Qualitative Synthesis

We will extract qualitative data from both the included quantitative studies as well as qualitative studies the cover the same types of programs and contexts (defined by our moderator variables) as our quantitative studies. We will use such qualitative data to establish that conditions recorded in quantitative data are being interpreted correctly and that hypothesized, but difficult to measure, chains of events do in fact occur in linking explanatory factors to outcomes (Collier, 2011; Vajja and White, 2008).

External Validity

We will evaluate external validity in three ways. First, we will simply report the distribution of the studies over the moderating factors listed above. This will provide a picture of the range of contexts from which our data have been gathered. Second, external validity issues are a function of the degree of effect heterogeneity and the homogeneity test that we propose will provide a basic measure of the degree of such heterogeneity. Third, assessing external validity involves assessing how moderating factors co-vary with effect estimates (Rubin, 1992; Imbens, 2010). The moderator analysis that we proposed to test hypotheses H3 through H7 will serve this purpose. In addition, we will include an appendix of estimates of the bivariate relationships between effect estimates and each of the moderating factors as a reference for researchers interesting in further pursuing external validity issues.

TIMELINE

The deliverable deadlines are as follows: •

31 January 2012: Title registration submitted

•

21 May 2013: Final protocol submitted

•

15 July 2013: Draft review submitted

•

15 October 2013: Final review submitted

ACKNOWLEDGEMENTS

We thank 3ie for support as well as Michele de Nevers. John Eyers, Ilaria Firmian, Steven Lawry, Carlos Manuel Rodriguez, Jesus Quintana, Juan Robalino, Birte Snilstveit, David Young, four anonymous reviewers, and seminar participants at the 2013 American Economic Association annual meeting and 2012 3ie Systematic Review Colloquium in Dhaka for helpful guidance. Reema Badrinarayan, Harini Dedhia, Justin Epstein, Shomik Ghosh, Johan Gorr, and Zihao Su provided excellent research assistance.

Footnotes

SOURCES OF SUPPORT

The review is sponsored by a grant from 3ie.

DECLARATIONS OF INTEREST

The research team identifies no potential conflicts of interest, whether due to financial interests, prior publications, or any other source.

REVIEW AUTHORS

Name: Cyrus Samii

Title: Assistant Professor, Wilf Family Department of Politics

Affiliation: New York University

Address: 19 W. 4th St, Office 424

City, State, Province or County: New York

Postal Code:

Country: USA

Phone: 212-992-8079

Email: cds2083@nyu.edu

Name: Matthew Lisiecki

Title: Junior Research Scientist, Wilf Family Department of Politics

Affiliation: New York University

Address:

City, State, Province or County:

Postal Code:

Country: USA

Phone:

Mobile:

Email:

Name: Parashar Kulkarni

Title: PhD Candidate, Wilf Family Department of Politics

Affiliation: New York University

Address:

City, State, Province or County:

Postal Code:

Country: India/ USA

Phone:

Mobile:

Email:

Name: Laura Paler

Title: Assistant Professor, Department of Political Science

Affiliation: University of Pittsburgh

Address:

City, State, Province or County:

Postal Code:

Country: USA

Phone:

Mobile:

Email:

Name: Larry Chavis

Title: Assistant Professor, Kenan-Flagler School of Business

Affiliation: University of North Carolina, Chapel Hill

Address:

City, State, Province or County:

Postal Code:

Country: USA

Phone:

Mobile:

Email:

ROLES AND RESPONSIBLIITIES

•

Cyrus Samii will be responsible for drafting the review, statistical analysis, and overall project management.

•

Matthew Lisiecki will be responsible for drafting the review, qualitative synthesis, and management of the search and data extraction process.

•

Parashar Kulkarni will be responsible for revision and quality assurance of the review.

•

Laura Paler will be responsible for revision and quality assurance of the review.

•

Larry Chavis will be responsible for revision and quality assurance of the review.

APPENDICES

ANNEXES

1

To some extent this is merely a semantic point: clearly aspects of the resource system will influence both deforestation rates and poverty conditions and will also affect the likely impact of PES or DFM programs. For example, whether there are highly valuable timbers or whether there are mining opportunities. However, resource systems factors such as these operate through resource users' opportunity costs. Our argument is that conditional on opportunity costs, timber values and mining opportunities per se are of secondary importance. On the basis of economic theory, opportunity costs provide a sufficient statistic for incorporating such resource system factors into the analysis.

2

Indeed, as Boerner and Wunder (2008) and Greig-Gran (2008) demonstrate, proper valuation of opportunity costs of forest conservation is not a trivial undertaking. For example, in valuing the opportunity costs in two sites in Brazil, Boerner and Wunder combine production and price data with forest loss projections to derive valuations that could be used to scale payments. Along similar lines, further problematize opportunity cost analyses by pointing out that some opportunity costs may be illegal or based on perception that are not rooted in formal markets (e.g., slash and burn farming), making it especially difficult to establish clear opportunity cost benchmarks.

3

See Diana Parker, “Indonesia's East Kalimantan loses forest area to new province,” Mongabay.com, February 24, 2013.

4

We apply the same strategy of reviewing only the first 100 hits for search engine results (but not database results), given that search engines typically return many thousands of results.

5

While we find that Porter-Bolland et al.'s “case” approach is well motivated, we do hope to go beyond what they do in terms of ensuring that estimates of forest impact are based on credible counter-factual comparisons. As far as we can tell from their write up, no such exclusion criteria were applied in their synthesis.

6

These are technical considerations that boil down to the following intuitive logic: when a cluster of areas is subject to a single program treatment, then it is not the case that each area contributes an independent piece of information about the effect of such a program. For this reason, the types of adjustments described in this paragraph are necessary if the analysis was done in a way that had mistakenly assumed that each area contributed an independent piece of information.

7

Note that normality in this case is motivated by the fact that the outcomes of the meta-regressions are estimated effects, which by central limit theorem results are expected to obey normality no matter what the underlying distribution of the outcome data. The homoscedasticity assumption is based on the presumption that the moderators included account for all systematic heterogeneity and that the inverse variance of weighting accounts for contributions to the variance of

τ_{i}

due to estimation error.

References

Achard

Hansen

M.C.

, eds. (2013). Global Forest Monitoring from Earth Observation. Boca Raton, FL: CRC Press.

Agrawal

Angelsen

(2009). Using Community Forest Management to Achieve REDD+ Goals. In Angelsen

, ed., Realising REDD+: National Strategy and Policy Options. Bogor: CIFOR.

Agrawal

Benson

C.S.

(2011). Common Property Theory and Resource Governance Institutions: Strengthening Explanations of Multiple Outcomes. Environmental Conservation 38(2), 199–210.

Akerlof

(1982). Labor Contracts as Partial Gift Exchange. Quarterly Journal of Economics. 97:543–569.

Alix-Garcia

Shapiro

E.N.

Sims

K.R.E.

(2012). Forest Conservation and Slippage: Evidence from Mexico's National Payments for Ecosystem Services Program. Land Economics, 88(4), 613–638.

Angelsen

ed. (2009). Realising REDD+: National Strategy and Policy Options. Bogor: CIFOR.

Angelsen

(2010). Policies for Reduced Deforestation and Their Impact on Agricultural Production. Proceedings of the National Academy of Sciences, 107(46), 19639–19644.

Angelsen

Wunder

(2003). Exploring the Forest-Poverty Link: Key Concepts, Issues, and Research Implications. CIFOR Occasional Paper, 40.

Banana

A.Y.

Gombya-Ssembajjwe

Bahati

(2002). Decentralization of Forestry Resources in Uganda: Realities or Rhetoric? Makerere University. Kampala, Uganda.

10.

Bardhan

Mookherjee

. 2005. Decentralizing Antipoverty Program Delivery in Developing Countries. Journal of Public Economics, 89(4), 675–704.

11.

Boerner

Wunder

(2008). Paying for Avoided Deforestation in the Brazilian Amazon: From Cost Assessment to Scheme Design. International Forestry Review 10(3), 496–511.

12.

Bowler

Buyung-Ali

Healey

J.R.

Jones

J.P.G.

Knight

Pullin

A.S.

(2010). The Evidence Base for Community Forest Management for Supplying Global Environmental Benefits and Improving Local Welfare: Systematic Review.

13.

Collaboration for Environmental Evidence Review 08-011 (SR48). Environmental Evidence: www.environmentalevidence.org/SR48.html

14.

Bullock

J.G.

Green

D.P.

S.E.

(2010). Yes, But What's the Mechanism? (Don't Expect and Easy Answer). Journal of Personality and Social Psychology, 98(4), 550–558.

15.

Burgess

Hansen

Olken

Potapov

Sieber

(2011). The Political Economy of Deforestation in the Tropics. Working Paper, London School of Economics.

16.

Campbell Collaboration International Development Coordinating Group (2011). Systematic Review Protocol and Review Guidelines. London: Campbell Collaboration.

17.

Chomitz

Kenneth M.

(2007). At Loggerheads? Agricultural Expansion, Poverty Reduction, and Environment in the Tropical Forests. Washington, DC: The World Bank.

18.

Cochrane Effective Practice and Organization of Care Group (2009). Risk of Bias. London: Cochrane Collaboration.

19.

Collier

(2011). Understanding Process Tracing. PS: Political Science and Politics. 44(4):823–830.

20.

Conte

Kotchen

(2010). Explaining the Price of Voluntary Carbon Offsets. Climate Change Economics. 1 (2).

21.

Corbera

Gonzalez Soberanis

Brown

(2009). Institutional Dimensions of Payments for Ecosystem Services: An Analysis of Mexico's Carbon Forestry Programme. Ecological Economics 68, 743–761.

22.

Deaton

(1997). Analysis of Household Surveys: A Microeconometric Approach to Development Policy. Baltimore, MD: Johns Hopkins University Press.

23.

Deveny

Nackoney

Purvis

(2009). Forest Carbon Index: The Geography of Forests in Climate Solutions. Washington DC: Resources for the Future.

24.

Edwards

Hunter

Stephan

Phamtran

Margules

Nurse

(2011). Linking Poverty Alleviation to Ecosystem Service Payments in Asia-Pacific: A Call to Action. Melbourne: The Foundation for Development Cooperation.

25.

Egger

et al. (1997). Bias in Meta-Analysis Detected by a Simple Graphical Test. British Medical Journal. 315(7109), 629–634.

26.

Ferraro

P.J.

(2011). The Future of Payments for Environmental Services. Conservation Biology 25(6), 1134–1138.

27.

Ferraro

P.J.

Lawlor

Mullan

K.L.

Pattanayak

S.K.

(2011). Forest Figures: Ecosystem Services and Policy Evaluation in Developing Countries. Review of Environmental Economics and Policy 6(1), 20–44.

28.

Fisher

Kulindwa

Mwanyoka

Turner

R.K.

Burgess

N.D.

(2010). Common Pool Resource Management and PES: Lessons and Constraints for Water PES in Tanzania. Ecological Economics 69: 1253–1261.

29.

Forest Trends, The Katoomba Group, & UNEP . (2008). Payments for Ecosystem Services: Getting Started. Nairobi: Forest Trends and the Katoomba Group.

30.

Friedrich

J.O.

Adhikari

N.K.J.

Beyene

(2008), The Ratio of Means Method as an Alternative to Mean Differences for Analyzing Continuous Outcome Variables in Meta-Analysis: A Simulation Study. BMC Medical Research Methodology. 8(32), (online access)

31.

Geldman

Barnes

Coad

Craigie

Hockings

Burgess

(2011). Effectiveness of Terrestrial Protected Areas In Maintaining Biodiversity and Reducing Habitat Loss. Collaboration for Environmental Evidence.

32.

Gibson

C.C.

McKean

M.A.

Ostrom

, eds. (2000). People and Forests: Communities, Institutions, and Governance. Cambridge, MA: MIT Press.

33.

Grainger

Geary

(2011). The new forests company and its Uganda plantations. Oxford: Oxfam International.

34.

Greenland

O'Rourke

(2008). Meta-Analysis. In. Rothman

K.J.

Greenland

Lash

T.L.

, eds. Modern Epidemiology, Third Edition. Philadelphia: Lippincott, Williams & Wilkins.

35.

Gregersen

El Lakany

Karsenty

White

(2010). Does the Opportunity Cost Approach Indicate the Real Cost of REDD+? Washington, DC: Rights and Resources Initiative.

36.

Grieg-Gran

(2008). The Cost of Avoiding Deforestation. London: International Institute for Environment and Development.

37.

Gullison

R.E.

; Frumhoff

P.C.

; Canade

J.G.

; Field

C.B.

; Nepstad

D.C.

; Hayhoe

; Avissr

; Curran

L.M.

; Friedlingstein

; Jones

C.D.

Nobre

(2007). Tropical Forests and Climate Policy. Science 316(5827):985–986.

38.

Hedges

L.V.

Gurevitch

Curtis

P.S.

(1999). The Meta-Analysis of Response Ratios in Experimental Ecology. Ecology, 80(4), 1150–1156.

39.

Higgins

J. P. T.

Green

(2011). Cochrane Handbook for Systematic Review of Interventions. London: Cochrane Collaboration.

40.

Holder

C.D.

Chase

(2011). The role of remittances and decentralization of forest management in the sustainability of a municipal-communal pine forest in eastern Guatemala. Environment, Development, and Sustainability, 14, 25–43.

41.

Imbens

G.W.

(2010). Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009). Journal of Economic Literature, 48(2), 399–423.

42.

Intergovernmental Panel on Climate Change . (2007). Climate Change 2007: Synthesis Report. Adopted at the Intergovernmental Panel on Climate Change Plenary XXVII, Valencia.

43.

Irawan

Tacconi

(2009). Reducing Emissions from Deforestation and Forest Degredation (REDD) and decentralized forest management. International Forestry Review, 11(4), 427–438.

44.

Kauneckis

Andersson

(2009). Making Decentralization Work: A Cross-national Examination of Local Governments and Natural Resource Governance in Latin America. Studies in Comparative Development, 44, 23–46.

45.

Kremen

Niles

Dalton

Daily

Ehlrich

Fay

Grewal

, and Guillery

. (2000). Economic Incentives for Rainforest Conservation Across Scales. Science. 288(5472):1828–1832.

46.

Kuper

Reeves

Albert

Hodges

B. D.

(2008). Why use theories in qualitative research?. British Medical Journal, 337, 631–634.

47.

Lawry

(2012). Devolution of Forest Rights and Sustainable Forest Management, Volumes I and II. Washington, DC: USAID.

48.

Lipscomb

Mobarak

A. M.

(2011). Decentralization and the Political Economy of Water Pollution: Evidence from the Re-drawing of County Border in Brazil. Unpublished manuscript, University of Notre Dame and Yale University.

49.

Mapedza

(2006). Compromised Co-Management, Compromised Outcomes: Experiences from a Zimbabwean Forest. Africa Development, 31(2), 123–146.

50.

Ostrom

(1990). Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge: Cambridge University Press.

51.

Ostrom

(2007). A Diagnostic Approach for Going Beyond Panaceas. Proceedings of the National Academy of Sciences, 104(39), 15181–15187.

52.

Pagiola

Arcenas

Platais

(2004). Can Payments for Environmental Services Help Reduce Poverty? An Exploration of the Issues and Evidence to Data from Latin America. World Development 33(2), 237–253.

53.

Pagiola

Arcenas

Platais

(2005). Can Payments for Environmental Services Help Reduce Poverty? An Exploration of Issues and the Evidence to Date from Latin America. World Development, 33(2), 237–253.

54.

Pattanayak

S. K.

Wunder

Ferraro

P. J.

(2010). Show Me the Money: Do Payments Supply Environmental Services in Developing Countries?. Review of Environmental Economics and Policy, 4(2), 254–274.

55.

Poole

Greenland

Luetters

Kelsey

J.L.

Mezei

(2006). Socioeconomic Status and Childhood Leukaemia: A Review. International Journal of Epidemiology, 35, 370–384.

56.

Porras

(2010). Fair and Green? Social Impacts of Payments for Environmental Services in Costa Rica. London: International Institute for Environment and Development.

57.

Porter-Bolland

Ellis

E.A.

Guariguata

M.R.

Ruiz-Mallen

Negrete-Yankelevich

Reyes-Garcia

(2011). Community Managed Forests and Forest Protected Areas: An Assessment of Their Conservation Effectiveness across the Tropics. Forest Ecology and Management. (Online early access).

58.

Pullin

Bangpan

Dalrymple

Dickson

Healey

Hockley

Jones

Knight

Oliver

(2011). Human Well-being Impacts of Terrestrial Protected Areas. Collaboration for Environmental Evidence.

59.

Ravallion

Datt

van de Walle

(1991). Quantifying Absolute Poverty in the Developing World. Review of Income and Wealth, 37(4), 345–361.

60.

Rubin

D.B.

(1992). Meta-analysis: Literature Synthesis or Effect-Size Surface Estimation? Journal of Educational and Behavioral Statistics, 17(4), 363–374.

61.

Salzman

(2010). Designing Payments for Ecosystem Services. Working Paper, PERC Policy Series, No. 48.

62.

Samuelson

(1954). The Pure Theory of Public Expenditure. Review of Economics and Statistics, 36(4), 387–389.

63.

Santilli

Moutinho

Schwartzman

Nepstad

Curran

Nobre

(2005). Tropical Deforestation and the Kyoto Protocol. Climactic Change, 71, 267–276.

64.

Spencer

Ritchie

O'Connor

(2003). Analysis. In Ritchie

Lewis

(Eds.), Qualitative Research Practice: A Guide for Social Science Students and Researchers. Boca Raton: Sage.

65.

Sprent

Smeeton

N.C.

(2007). Applied Nonparametric Statistical Methods, Fourth Edition. Boca Raton, FL: Chapman and Hall.

66.

Tacconi

(2007). Decentralization, Forests and Livelihoods: Theory and Narrative. Global Environmental Change, 17, 338–348.

67.

The

B.D.

Ngoc

H.B.

(2006). Payments For Environmental Services In Vietnam: Assessing An Economic Approach To Sustainable Forest Management. Research Report – Economy and Environment Program for Southeast Asia. Singapore.

68.

Vajja

White

(2008). Can the World Bank Build Social Capital? The Experience of Social Funds in Malawi and Zambia. Journal of Development Studies, 44(8), 1145–1168.

69.

Van Kooten

Sohngen

(2007). Economics of Forest Ecosystem Carbon Sinks: A Review. International Review of Environmental and Resource Economics, 1(3), 237–269.

70.

Viechtbauer

(2010). Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software, 36(3), (online access).

71.

Waddington

Snilsvelt

Vojtkova

Hombrados

(2012). Farmer Field Schools for improving farming practices and farmer outcomes in low- and middle-income countries: a systematic review. Campbell Collaboration.

72.

West

P.W.

(2009). Tree and Forest Measurement, Second Edition. Heidelberg: Springer-Verlag.

73.

Wilson

D.B.

Weisburd

McClure

(2011). Use of DNA testing in police investigative work for increasing offender identification, arrest, conviction and case clearance. Campbell Systematic Reviews 2011(7).

74.

Wunder

(2005). Payments for environmental services: Some nuts and bolts. CIFOR Occasional Paper, 42.

75.

Wunder

Engel

Pagiola

(2008). Taking stock: A comparative analysis of payments for environmental services programs in developed and developing countries. Ecological Economics, 65, 834–852.

PROTOCOL: Effects of Payment for Environmental Services and Decentralized Forest Management on Deforestation and Poverty in Low- and Middle-Income Countries: a Systematic Review

Abstract

BACKGROUND

Description of the problem

Description of the intervention

How the intervention might work

Why it is important to do this review

OBJECTIVES OF THE REVIEW

METHODOLOGY

I. Criteria for including studies in the review [PICOS]

Participants

Interventions

Comparisons

Outcomes

Study Types

II. Search methods for identification of studies

Electronic searches

Other Searches

III. Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Measures of treatment effect

Unit of analysis issues

Dealing with missing data and incomplete data

IV. Data synthesis

Quantitative Synthesis

Qualitative Synthesis

External Validity

TIMELINE

ACKNOWLEDGEMENTS

Footnotes

SOURCES OF SUPPORT

DECLARATIONS OF INTEREST

REVIEW AUTHORS

ROLES AND RESPONSIBLIITIES

APPENDICES

ANNEXES

1

2

3

4

5

6

7

References