PROTOCOL: Participation,inclusion,transparency and accountability (PITA) to improve public services in low- and middle-income countries: a systematic review

The problem of unaccountable government systems and poor service delivery

The sustainability of global development investments depends on strong institutions, citizen engagement, accountable governments, and equitable economic growth (World Bank, 2017). Goal number 16 of the Sustainable Development Goals explicitly recognises the importance of the development of effective, accountable and transparent institutions at all levels, and of ensuring responsive, inclusive, participatory and representative decision-making at all levels (UNDP, 2016). In the Paris Declaration on Aid Effectiveness, donor and partner countries committed to improving their mutual accountability and transparency in the use of development resources, with partner countries further committing to systematically involve diverse stakeholders in national development priority setting processes (OECD 2005). Many development challenges, such as poor service delivery, corruption and slow growth, persist because of the political context around them; they are as much about power dynamics as they are technical challenges.

Improving the governance of public institutions and service delivery has long been a central tenet of strategies for achieving or supporting development; World Bank World Development Reports since the late 1990s have included elements of improving governance as central to their theories of change (Grindle, 2004). In the decades since, mainstream approaches to realising good governance have shifted in focus, away from privatisation of service delivery and towards a focus on increasing the engagement of constituents, particularly vulnerable groups, with public institutions and service providers in such ways to increase the effectiveness, appropriateness, and quality of service delivery. The 2004 World Development Report (WDR) highlighted the insight that public spending on service delivery in developing countries often primarily reached the better-off minority of citizens; for example, in India, curative health subsidies were primarily going to the richest 20 per cent of the population, who received three times the subsidies of the poorest 20 per cent (World Bank, 2004). This insight remains pertinent. For example, a recent evaluation of an e-governance intervention in India that aimed to improve transparency in a fiscal transfer system for a social benefits programme suggested that while the intervention was successful at reducing leakages, the savings did not translate into improved outcomes for beneficiaries (Banerjee et al., 2017). One of the authors later posited that this may have been because the intervention did not empower the ultimate beneficiaries to ensure that financial gains from reduced corruption were converted into increased outcomes for the poor (Page and Pande, 2018).

There are many definitions of governance. For the purposes of this review, we use the recent definition employed by the World Bank, where governance is defined as “the process through which state and non-state actors interact to design and implement policies within a given set of formal and informal rules that shape and are shaped by power” (World Bank, 2017). Where characteristics of good governance are weak or absent from public processes and service delivery, the effectiveness and sustainability of development interventions is likely to suffer (World Bank, 2016). Barriers to access to public services for vulnerable groups exacerbate inequality, with potential long-term repercussions for a society's development (Easterly, 2007). Fraud and corruption are pervasive across low- and middle-income countries, and the negative consequences on quality of life and core development outcomes are well documented (Svensson, 2005; Molina et al., 2016). Where state and public actors cannot be effectively held accountable, a culture of impunity develops that normalises fraud and rent-seeking practices. The World Bank's World Development Report 2017 highlighted key repercussions of power asymmetries, including: exclusion of large portions of society from services, institutions or resources, which is correlated with violent conflict: elite and/or interest-group capture of policies in order to serve interests, resulting in poor targeting and ineffective or inappropriate policies, which can lead to poor or stagnant growth, condemning economies to an under-developed state; and clientelism, which often leads to rent-seeking and poor service delivery, which have long-term repercussions on societies’ growth (World Bank, 2017).

Despite the decades of acknowledgement of the importance of good governance, progress has been slow; the Worldwide Governance Indicators show limited to none or even negative progress on key governance indicators amongst aggregates of low and lower-middle income countries from 2006 to 2016 (World Bank, 2018). The repercussions of continued governance failures are high, and well documented; for example, in Nigeria, unabated corruption led to the squandering of billions of dollars by the National Petroleum Company, jeopardizing the country's long-term growth potential and financial stability (World Bank, 2017).

Approaches to improve governance have generally either focused on mechanisms to strengthen the effectiveness and institutionalisation of public institutions, or on external pressures to improve service delivery despite weak institutions. While each approach has yielded valuable insights, translating insights from theory into practice has been challenging. There is some evidence that at times, failures could be due to an over-emphasis of the demand side of governance by service users, citizens and civil society, which ignores the constraints faced on the supply side by politicians, bureaucrats and service providers (Brinkerhoff and Wetterberg, 2015), or of the power of information (Wibbels and Keohane, 2018). More recently, insights are emerging into the value of system-based approaches that look at both the supply and demand sides of governance as actors in a single system, drawing on power analyses and social network theories (Mcloughlin and Batley, 2012; Fox, 2014; Halloran, 2015; Wibbels and Keohane, 2018).

USAID's Democracy, Human Rights and Governance (DRC) Center identified participation, accountability, transparency and inclusion (PITA) as critical principles that could be incorporated into interventions within and across sectors to improve development outcomes, and in line with the Doing Development Differently global initiative (USAID, 2016). We define participation as efforts to involve citizens in the design, monitoring and delivery of policy and programmes upstream (Quick and Feldman, 2011). Transparency is a “characteristic of governments, companies, organisations and individuals of being open in the clear disclosure of information rules, plans, processes and actions” (Transparency International, 2009: 44). Accountability is the concept that individuals, agencies and organisations are held responsible for executing their powers according to a certain standard downstream (McGee and Gaventa, 2011). Finally, inclusion means a particular focus on marginalised and vulnerable citizens in policy and programming upstream or downstream (Quick and Feldman, ibid).

Interventions promoting participation, inclusion, transparency and accountability

Mapping the universe of PITA interventions

A recent evidence gap map (EGM) on interventions for state-society relations highlighted a number of interventions to improve governance (Phillips et al., 2017). These were broadly grouped into interventions for inclusive political processes and leadership (e.g. community-driven development, electoral monitoring, and quotas for women and minority representation in political institutions), and interventions for responsive and accountable institutions and service delivery (e.g. audits, land reform and public servant performance incentives).

Drawing on Phillips et al. (2017), and also insights from the literature, we theorise good governance can come about through sustained improvements across three domains: within the political system; within the management and administration of public sector offices and institutions; and in the ways in which public officials and service providers engage with service users (external engagement). In this framing, good governance interventions attempt to influence the social contract that mediates the relationships between government and citizens, regarding who has access to what power and in return for what accountability for service provision, through three accountability domains:

Influencing how the broader political system functions: The broader political system dictates access to and contestability of the policy arena (World Bank, 2017). It further comprises the checks and balances, or “sideways accountability” between institutions. Increasingly, good governance interventions seek to influence how this system functions, rather than the specific form it takes (World Bank, 2017).

Influencing how a specific public service or institution's system functions internally: Many good governance interventions aim to improve service delivery through the institutionalisation of public services and institutions. These interventions foster “internal accountability” of institutions, and include, but are not limited to, strengthening human resources management, systems of upwards accountability of staff and management or between different levels of government, and supply chains for infrastructure, goods, and financial flows (Finan, Olken and Pande, 2015).

Influencing how a specific public service or institution engages externally with constituents: These interventions aim to mediate the ways that citizens engage with government and public service providers outside of the “long route” of electoral processes (World Bank, 2004). They work to improve service delivery through “external accountability”, by increasing the engagement between service providers and service users to improve the responsiveness and effectiveness of public services.

Many good governance interventions are designed to improve service delivery for citizens. This is often done through interventions that embody one or multiple PITA characteristics, which seek to address power dynamics between the state, civil society and citizens to make service delivery more effective and equitable (USAID, 2016). PITA characteristics influence the functioning of the social contract and its systems throughout each of the three accountability domains, and thus, good governance interventions may target one or more of these (Figure 1). For example, within the political system domain, the PITA characteristics have a direct impact on who has access to the electoral systems and who can contest the policy arena. Elected officials must exercise some basic level of downwards accountability towards the constituents who elected them (or, in non-democratic states, who grant them legitimacy), and sideways accountability to their fellow statesmen through the checks and balances built into the system. PITA interventions targeting this domain tend to focus on creating a fair system. Within the internal system domain, PITA interventions tend to focus on creating an efficient system, such as through improving the upwards accountability of officials and service providers to management, or through improving the relevance of service provision at local levels through decentralisation. Finally, in the external engagement domain, the PITA characteristics of a service or institution mediate the means through which it engages with citizens, civil society, and business/interest groups. These PITA interventions aim to address a more diverse set of system attributes, primarily the relevance, effectiveness and inclusivity of the service delivery system, and are further differentiated from those in the previous domains through their reliance on soft power.

OPEN IN VIEWER

1

PITA interventions throughout the three domains of good governance

The effectiveness of interventions that target the PITA characteristics within one domain will be mediated by the context of the other domains as well, the power relations and constraints, and also by other interventions aiming to improve good governance and service delivery, particularly those that target service delivery supply chains. There is increasing scholarship that suggests that while interventions improving the PITA characteristics of public services and institutions, particularly in the external engagement domain, may be necessary for achieving sustainable improvements in service delivery and a stable social contract, they may not be sufficient (e-Pact Consortium, 2016). On the other hand, while interventions that target strengthening PITA characteristics within internal institutional systems may be sufficient for improving governance within the system, the impact of those governance improvements may not reach the ultimate beneficiaries (citizens/service users) without the incorporation of interventions strengthening the system's external PITA characteristics (Page and Pande, 2018).

While recognising the interactions of PITA interventions across each domain and with complementary good governance and service delivery initiatives, it has been pointed out that to attempt to cover the entirety of good governance interventions in a single review would be “exceedingly ambitious” (Sáez, 2013). Thus, this review will analyse the value-add of interventions in the third domain, external PITA interventions targeting public service and institution engagement with citizens.

The focus of the review on PITA for external engagement with service users

External PITA interventions can be implemented as stand-alone interventions or as part of a larger programme working to strengthen governance and service delivery. They may be implemented either on the supply or demand side of service delivery, or may target both simultaneously, such as a public audit process that trains community members on tools to hold public officials accountable, and works with public officials to increase their understanding of the importance of downwards accountability and transparency. As in that example, the interventions may strengthen one or multiple PITA characteristics of the ways public services and institutions engage with their constituents.

For the purposes of this review, the definitions of PITA have been further operationalised as follows (Figure 2):

Participation: The intervention promotes or formalises continuous citizen input in the design and implementation of public services, processes or policies. ∘

Participation interventions create specific opportunities or processes for citizens to provide meaningful input into public policy or strategy design and planning. An example of a participation intervention is the introduction of participatory budgeting so that citizens may directly contribute to the development of a budget proposal (Touchton & Wampler, 2014). A community-level example could be the creation and capacity building of a representative community-based natural resource management committee that is mandated to develop and monitor locally agreed standards and regulations for the use of common property.

Transparency: The intervention involves the disclosure and/or dissemination of information (rules, plans, processes, prices and actions) regarding public services or institutions, with the explicit aim of changing the way that citizens and service providers or public officials interact and the power relations between service providers and users. ∘

An example of a transparency intervention is local clinics posting information about patient rights, service fees and standards, and budget execution (USAID, 2016), which restricts the scope for service providers to charge bribes..

Accountability: The intervention encompasses monitoring and soft/social accountability mechanisms to encourage or actively hold individuals, public service providers and institutions responsible for executing their powers and mandates according to a certain standard. ∘

Accountability interventions create opportunities or processes for constituents to hold government and public service providers accountable. An example is a project to encourage and build the capacity of civil society to hold government accountable for the sustainable and equitable management of natural resources (USAID, 2016), or a citizen report card intervention, in which a community group is taught the quality standards to which they are entitled and how to monitor the quality and performance of service delivery, and then to work with the service providers to address any identified issues through a mutually agreed action plan.

Inclusion: The intervention includes particular strategies to promote the opportunities and capacities of marginalised and vulnerable groups such as women, ethnic minorities or lesbian, gay bisexual, transgender and intersex (LGBTI) people to engage with the management of public institutions and service providers. Hence, we define inclusion specifically as a component of an intervention that targets a change in participation, transparency or accountability. ∘

An example of an intervention to promote inclusion is ensuring that a certain proportion of places in a community governance group are reserved for women (Humphreys et al., 2012).

OPEN IN VIEWER

2

Operationalising external PITA into intervention categories

The intervention categories are described in more detail in (Table 3) in the Methodology section below. While most interventions contribute primarily to a single PITA mechanism as described above, there is often significant interplay between the PITA characteristics to which an intervention contributes. Though efforts have been made to make the definitions mutually exclusive, a single intervention may contribute to strengthening multiple PITA characteristics. For example, a public audit may improve the community's understanding of services the government can and cannot provide (contributing to transparency), while further allowing them an opportunity to pressure and apply informal sanctions to public officials should the audit uncover discrepancies in spending (contributing to accountability). More generally, interplay at outcome level often exists within transparency and accountability dimensions. For example, interventions to improve access to information about users’ rights to services (transparency) may aim, at an intermediate outcome level, to improve the way a service is governed (accountability).

As noted above, according to the definition used in this review, eligible interventions that are designed to improve the access of a marginalised group of citizens (inclusion) to a decision-making process aim, at an intermediate outcome level, to improve the group's input into the process by providing increased opportunities for consultation (participation), or other aspects of monitoring service delivery (transparency and accountability). Indeed, the assignment of other intervention sub-types will vary depending on their particular iteration within a project. For example, officials may mandate a health care committee to increase access to health services for marginalised community members through community campaigns (contributing to inclusion) and/or to undertake health service monitoring (contributing to accountability). Thus, the ultimate assignment of a study looking at a health care committee intervention will depend on the structure and mandate of the committees within that project.

How PITA interventions might work

We present a model showing an indicative, stylised theory of change for how the intervention may work (Figure 3). This preliminary theory of change developed for this systematic review draws on insights from the literature and programmatic best practices. In particular, the framework builds on the 2004 WDR (World Bank, 2004) theory of change, which articulated the importance of pro-poor governance practices that actively engage end users for effective outcomes, and Rahman and Robinson (2006) who articulated the importance of local ownership and long-term support. The assumptions and moderating factors draw on insights from Fox (2014), Page and Pande (2018), and the 2017 WDR (World Bank, 2017), among others.

OPEN IN VIEWER

3

Intervention theory of change

This review does not take a “rights-based approach” that views improvements in PITA characteristics as the end objective. While recognising the value of PITA characteristics in and of themselves, the focus of this review is on the value-add they bring to improving development outcomes through improved service delivery. It is unlikely that any single theory of change would be able to capture all the ways different PITA interventions affect good governance outcomes and service delivery. Thus, this review, at this protocol stage, attempts instead to hypothesise the main causal pathways through which PITA interventions targeting public services’ and institutions’ external engagement with citizens lead to improved development outcomes. Some interventions may contribute to all the pathways; others may only contribute to one. Similarly, as the focus is on the external engagement processes, while some interventions may have effects on other domains of the social contract such as electoral systems, they are not captured in this theory of change. The theory of change will be built out and developed in more detail through the evidence synthesis.

We note here a useful distinction between the demand and supply side of governance. Implementers may target stakeholders on the demand-side of governance, such as through efforts to improve the capacity of civil society to monitor government service delivery, or the supply-side, such as by training public officials on pro-poor development planning. Other interventions may be geared to affecting both demand- and supply-sides, such as a participatory budgeting process in which government officials are trained on the value of participatory budgeting, while community members are trained and supported to participate in the process.

The indicative theory of change presents the hypothesised causal chain for PITA interventions, from changed opportunities and capacities, followed by behavioural changes on both the supply and demand side of governance, ultimately leading to improved service delivery performance and enhanced quality of life outcomes for citizens.

Beginning with the intervention on the left hand side, the figure follows a primary causal chain, with immediate, intermediate and endpoint outcomes indicated in boxes, and key assumptions in bubbles. The theory of change starts with critical assumptions of the design, inception and implementation phases: first, that the intervention designed is relevant and addresses an identified local need; second, during inception that wider community acceptance for the intervention has been sought and received from key social, religious and political leaders; and finally, that community mobilisation activities are undertaken during implementation. Similar to how the quality of PITA characteristics in the public planning and service delivery spheres contributes to strengthening the corresponding development outcomes, the strength and quality of the PITA characteristics of the intervention itself are suggested to contribute to its efficacy.

The exact form of the intervention (Block 1) will vary widely, yet the majority aim to create an enabling environment for increased and mutually empowering interactions between service providers and citizens through changes to their knowledge, attitude and practices (KAPs). On the demand side, this may include efforts to improve citizens’ knowledge of the services to which they are entitled; their capacity to demand those services through key tools; and/or their sense of self-efficacy and empowerment to do so effectively. An intervention focused on a technical skill such as participatory budgeting may start with capacity building on budgeting processes and the role that citizens can play; interventions that aim to increase inclusion of marginalised groups often start with community campaigns to raise awareness amongst the target households.

On the supply side, either in addition to demand-side efforts or independently, interventions aim to strengthen openness from and active engagement with supply-side stakeholders in efforts to improve service delivery. These may target the actors implementing or managing the service in question, but also other key stakeholders in the community and throughout the system. Seeking and attaining community acceptance prior to implementation is a widely-applied best practice for ensuring that development projects do no harm and that they will have sufficient buy-in from the community to be successful. There is some evidence that suggests that this may be particularly critical for PITA interventions. Securing buy-in from stakeholders at the point of intervention, upstream and downstream along the service delivery / good governance supply chain may create an enabling environment for PITA interventions to successfully navigate the social network and power differences within which the intervention is implemented (Mcloughlin and Batley, 2012).

Different tools implemented as elements of PITA interventions may require different conditions to be effective; the synthesis process will aim to identify and these out. The context will also influence what works, how; for example, if a political player can increase his or her own personal power through framing an improvement in service delivery as a personal “win,” then she or he may be more motivated to work for its improvement (e-Pact, 2016).

The first immediate outcome (Block 2) posits that through engaging in the PITA interventions, citizens will increase their engagement with State and public service officials. This is often an explicit aim of PITA interventions, as it is a critical precursor to the higher-level outcomes. Through the increased engagement, the next level of change (Block 3) posits that citizens develop a better understanding of processes, services, and the constraints faced by service providers, while simultaneously, service providers gain a deeper understanding of the needs of their constituents and appreciation for the engagement process.

In subtle ways, these changes reflect renegotiations of power relations between the State, civil society and citizens, mitigating the power imbalances. This happens as the PITA interventions shift the dynamics of power by drawing on collective and representative voice.

Participation interventions address power relations by building in meaningful opportunities for citizens to provide input over the direction of policies that affect them and the supply of services they rely on. For example, participatory development planning interventions enable community groups to develop their own plans and priorities, which are then consolidated into a coherent local development plan, which then feeds up into higher-level plans. This ultra-decentralised decision-making ensures that the priorities of target communities are at the forefront of any final government development plan.

Power relations are dynamic; they can change quickly, both for the better and worse, and gains are not necessarily secure. A key assumption here is that supply-side actors are fully engaged throughout the process; otherwise, the attempts to increase soft power by citizens may be seen as confrontational rather than collaborative, which could de-incentivise service providers from the process to avoid being seen to give up any of their power (World Bank 2004). Where PITA processes are seen as collaborative, they can be mutually empowering, creating changes in the interactions between state and society that simultaneously give citizens greater input into the provision of the services they rely on, and strengthen the standing of the service providers in the community (Fox, 2014).

Where interventions are unsuccessful at building coalitions to facilitate an enabling environment for change, they may not be successful at changing power relations, as actors may adapt to new systems (Halloran, 2015). For example, though advancements in the field of information and communications technology (ICT) offer exciting possibilities for strengthening external PITA characteristics, a change in technology that is not complemented by supporting interventions that create an enabling environment may fall flat (Hogge, 2010).

As the power relations are shifting and engagement is increasing, a core intermediate outcome of the PITA interventions will emerge (Block 4): public service delivery will improve in efficiency, effectiveness, and equity. Once public officials and service providers are taking into account the input of community members, the selection and targeting of services will improve. This will improve the effectiveness and appropriateness of public service delivery. Inclusion interventions improve the equality of service provision, as they increase access to services and processes for the most vulnerable community members. Transparency initiatives increase the efficiency of public service delivery, as they streamline costs and processing times, and make it harder for politicians and officials to demand inflated payments for services. Finally, accountability initiatives can have direct benefits to the performance of public service delivery, as citizen feedback mechanisms such as Public Audits end with joint workshops between the service provider, citizen representatives, and other key stakeholders to come up with an actionable plan to which all parties can be held to account for how they will address the major issues identified and improve service delivery.

The key assumption here is that institutions have the capacity to respond to priorities requested and issues raised by constituents. This is a critical assumption, because in its absence, the interventions risk doing harm by having a negative consequence on perceptions of State effectiveness resulting from raised and then unmet expectations. For example, the 2017 WDR highlights the risk that investments in service provider capacities may not be enough to improve service delivery, if power relations within the institution are not addressed (World Bank, 2017). Further, depending on the structure of the intervention, improvements may be related to a one-off change in the situation that is not sustained; many PITA interventions are designed as experiments, whose study design may capture short-term gains that revert back to the baseline conditions with time. Fung et al. suggest that transparency interventions contribute to improvements only when the information provided becomes embedded in the decision-making process (2005).

In some cases, PITA interventions, particularly those that focus on improving access to services for marginalised groups (I), may not lead to the active, empowered engagement between citizens and service providers that leads to mitigated power differences and improved services. However, they could still lead to increased use of public services, particularly amongst vulnerable populations (Block 5). This comes about as a direct result of Inclusion and Transparency (information dissemination) interventions, but also through the other PITA interventions; as communities are mobilised to engage with their local government and services, they become more invested in the services that they are attempting to improve. And thus, they are more likely to take advantage of those services, as they understand the importance of ensuring high quality service provision for themselves and their families. However, increased access for marginalised groups is not a given outcome of PITA interventions; there likely needs to be concerted, targeted efforts to reach and engage these groups in order for the impacts to reach them (E-Pact Consortium, 2016). Similarly, interventions targeting services where changes are relatively immediate and visible may be more likely to encourage buy-in and support from supply-side actors (ibid.).

The joint effects of changes from Blocks 4 and 5 lead to improved development outcomes for citizens (Block 6). These will vary by sector targeted, and by project duration. It is increasingly thought that PITA interventions, through the immediate outcomes increasing engagement with government and public officials and mitigating the power differences, can have a positive impact on perceptions of state effectiveness when services and development outcomes improve as a result (World Bank, 2017) (Block 7). As citizens engage with public processes and services, they learn more about the constraints under which these institutions operate. As they see increased responsiveness of public officials, and subsequent real improvements in the quality of services they receive, their perceptions of State effectiveness and legitimacy will increase. This is particularly critical in fragile and post-conflict Sates, where the State may still be vying with other actors for legitimacy over governing and control.

The social contract, wherein the state and citizens negotiate the granting of power and legitimacy in return for service provision, is a complex system that is being constantly renegotiated as actors at various levels within and outside of the government interact and negotiate for their own individual power and accountability, within frameworks of collective power and accountability. Thus, it is both the context within which good governance interventions are implemented, and also at the heart of the systems and services that said interventions aim to improve. There is some evidence that the relationship between politics and service delivery is two-way: the political context influences service delivery, but service delivery also influences politics, particularly state legitimacy (Mcloughlin and Batley, 2012). Thus, all good governance interventions in some way influence the social contract between the state, its institutions and service providers, and the citizenry; on the other hand, good governance failures often stem from fragmented social contracts (World Bank, 2017). This perspective also draws on the ideas of a social accountability ecosystem developed by Halloran (2015).

In the majority of PITA programmes, the PITA characteristics interventions are add-ons to core interventions and outcomes in a public service sector, such as health or agriculture. In the long run, all three intermediate outcomes contribute to sustaining social and economic outcomes of communities (Block 8). The key assumption here is that sustained support is provided to the institutions or service providers charged with maintaining the implementation of the PITA intervention, such that it becomes institutionalised. As noted above, power differences are dynamic and constantly evolving. Thus, a short-term project may well change outcomes in the short-term, but without proper support those gains may easily be lost. Thus, sustainability mechanisms must be built into the PITA programme design, to support communities and service providers on the road to sustainable participatory, inclusive, transparent and accountable public service delivery and public management.

The context in which this theory of change, or elements of the same, are implemented has strong ramifications on the ways in which the interventions must be designed, implemented and supported in order to ensure success. Common key characteristics are noted in the box below the theory of change, alongside the key stakeholders from both the supply and demand side of governance. Governance programmes are generally implemented in resource-poor contexts, where there are entrenched problems around low levels of education and capacity, high turnover amongst public officials, and endemic corruption. Target communities are frequently difficult to access, either due to remoteness and extreme weather, or to conflict and insecurity. It is precisely because of these challenges that governance interventions are so strongly needed in such areas, but they must be taken into account during the design phase to ensure risks are appropriately mitigated. These factors breed vicious cycles of weak public service supply, which leads to weak demand, which in turn facilitates weaker public financial management, and so on. In an ideal world, the PITA interventions would create a virtuous circle of active community engagement in their government and service provision.

Interventions tailored to the specific context in which they are implemented, that target both the demand and supply sides of good governance, are more likely to be successful, particularly when the interventions are supplemented by complementary ones that target the technical side of service delivery and/or service delivery supply chains. For example, in the Philippines, a project focusing on improving access and quality of maternal and child health and family planning included social accountability mechanisms in the form of Quality Assurance Partnership Committees, which Brinkerhoff and Wetterberg (2015) argue led to more effective service delivery that improved the client-focus of providers and increased service use.

Additional moderating factors that will have a strong influence on project results include top-down political will; key to ensuring that local government officials and service providers have the capacity to implement the changes they agree to with their constituents is having the support of the higher levels of government, which can ensure that funds are appropriately allocated. Political will further influences the sustainability of the results, and the possibility of a change in administration poses a risk to programmes that may be cut due to high association with the outgoing regime.

It is important at this point to also highlight two broad issues which determine the effectiveness of programmes, relating to intervention design and implementation fidelity. There are two main reasons why we might not expect to see the intended impacts of a programme implemented in the ‘real world’ (Bamberger et al., 2010). The first is that the programme design is inappropriate – that is the underlying mechanisms that drive change are not appropriate for the context in which the programme is based, or for particular groups of participants in that context (Pawson, 2006). According to van der Knaap et al. (2007: 3), “mechanisms are the engines behind behavior, which are often not immediately recognizable… They [include] people's efforts to give way to group pressure (groupthink), people's efforts to be status-congruent with others or to avoid or reduce cognitive dissonances, or people's desire to be an early adopter of an innovation. [T]he action of mechanisms to some extent depends on the context in which they are used… Behavioral change is achieved through this context”.

An example would be a community driven development programme that is supposed to rely on community participation to foster social cohesion, but is unable to support the appropriate level of participation, and therefore cohesion, because people are not comfortable speaking in public meetings due to elite capture (Snilstveit, 2012; White et al., 2018). Similarly, interventions to decentralise decision making in schools are less likely to be effective in low income, low education contexts where communities have low status relative to school staff (Carr-Hill et al., 2016). Another example would be a women's empowerment programme which is ineffective in reaching a particular sub-group of participants (e.g. women from Muslim households) because it does not take into consideration the need to involve community leaders in design of the programme targeting strategy.

Such “failure mechanisms” will vary based on intervention design and targets; for example, in some cultures, traditional community leaders may be critical stakeholders to engage in interventions seeking to change the equity of or access to services, despite the disconnect between their de facto and de jure power – but only depending on the service targeted. Baldwin and Raffler (2016) argued that traditional leaders are often highly socially accountable for public services such as conflict resolution or natural resource management, but less so for services such as education or health care. In that case, an intervention targeting equitable access to and use of public land may fail if it does not engage traditional leaders, but a similar intervention simply targeting equitable access to and use of health services may still be successful. Failures may also come in the form of unintended consequences; for example, Chong et al. (2014) found that increasing the dissemination of corruption information to voters in Mexico decreased support not only for exposed corrupt politicians, but also for all political parties, and led to a decrease in voter turnout.

The second reason is due to implementation failures for a programme that otherwise (in theory) would be effective in the implementation context. Examples would be technical and logistical problems relating to project delivery (e.g. inadequate training and support to practitioners); weaknesses in implementer systems (e.g. human resource, financial or monitoring); or due to external factors (e.g. conflict, natural disasters).

Criteria for including and excluding studies

The study inclusion and exclusion criteria are summarised in Table 1.

Description of methods used in primary research

We will follow the inclusion and exclusion criteria outlined in the previous section to include studies in the systematic review. We anticipate that we will identify a mixture of experimental and quasi-experimental studies. Examples of studies included and excluded are given in Appendix 2, and include the following:

Finkel (2012), “The VOICE Impact Evaluation in the Democratic Republic of the Congo Final Report”. This study uses an experimental design and is an example of a PITA intervention using information dissemination around on-going decentralisation in the Democratic Republic of Congo

In addition, interventions conducted in schools and education facilities, or education outcomes measured in otherwise eligible interventions, will be excluded. Two examples are:

Aturupane et al. (2014), “An Assessment of the Impacts of Sri Lanka's Programme for School Improvement and School Report Card Programme on Students’ Academic Progress”.

Search strategy: Studies to address review question 1

We developed the systematic search strategy in consultation with an information specialist (John Eyers) to cover comprehensively the published and unpublished literature, following systematic search guidelines in Kugley et al. (2017)). We also drew upon, and expanded, the search terms used in the evidence map by Philips et al. (2017) and harvested terms from the papers included in that map that were eligible for inclusion in our review. To reduce the potential for publication bias, the search will include both academic databases as well specialist organisational websites, websites of bilateral and multilateral agencies and repositories of impact evaluations in international development. The full list of databases is presented below. The substantive scope of this review is cross-sectoral and therefore in addition to general sources of social science research, we will search several sector specific databases, for example databases of health, governance and public management. We will search for studies published in 2000 or after up until 2018.

Search terms for the academic databases can be found in Appendix 1. Separate search strings were developed for the two academic health databases to capitalise on MeSH terms, to remove non-health related terms and add some specific health-related intervention terms (Medline and Global Health). The search strings combine specific intervention terms, study design terms and terms for low-and middle-income countries.

A simplified series of search strings was developed for searching the grey literature, wherein the search engines are not as sophisticated as the academic databases and cannot handle the same detailed strategy. Due to the broad scope of the review, and in order to ensure the grey literature search was exhaustive, a series of PITA search strings were developed. These focused on PITA terms such as participatory or participation. An intervention-based strategy, more similar to the academic database strategy, was piloted, but discarded due to the number of individual searches per site that were required for an exhaustive search, rendering it inefficient. Population and study type terms were not included, because the advanced search options within the grey literature search engines were not sophisticated enough to allow for an “or” limiter for each LMIC and methodology. The broad study type term “impact evaluation” was added alongside each search to improve the relevance of results.

Targeted search for studies to address review question 4

In order to answer question 4 relating to programme design, implementation, mechanisms and context, we will attempt to identify programme and project documents associated with the programmes in the impact studies identified in the first stage of the search. We will do this by undertaking a targeted search for programme names and authors using Google Scholar. Evidence on context and mechanisms will be collected from any studies eligible for research questions 1-4. Programme mechanisms may be suggested by study authors or identified by the review team. In addition, we may impute contextual information not provided in included studies using international data, for example the World Development Indicators (World Bank) or the “Polity IV” governance index (Marshall et al. 2011).

Studies to address review question 5

We will incorporate economic evaluations and cost data that are provided in the included impact studies.

Screening

All search results will be imported into EPPI-Reviewer 4 and duplicates removed. All studies will be double screened in EPPI reviewer 4 against the review inclusion criteria by two independent reviewers using information available in the title and abstract. Any disagreements will be resolved by a third reviewer. If a title and abstract does not present enough information to definitively include or exclude a study, it will be included for full-text screening.

At the title and abstract stage, we intend to use innovative text mining technologies to reduce the initial screening workload (O'Mara-Eves et al., 2015). We will use two functions in EPPI Reviewer 4 to do this: the priority-screening function and inclusion/ exclusion classifier (Thomas et al., 2011; O'Mara-Eves et al., ibid). The priority screening function can be used at the title and abstract screening stage to prioritise the items most likely to be ‘includes’ based on previously included documents. This will involve double screening a random test set of 100 citations to train the priority screening function, which will learn to identify relevant records based on key-words in the title and abstract of the included and excluded studies. All team members will be involved at this stage of screening. The function continues to learn as screening progresses. Using priority screening in this way allows for the identification of includable records at an earlier stage in the review process so that work can begin earlier on full-text screening and data extraction. We will also use the priority screening function to develop a classifier that will classify studies into groups based on their probability of inclusion in the review. We will conduct piloting and verification, but we anticipate excluding studies with a low probability of inclusion (<10% probability of inclusion) automatically from the review. We would screen a random sample of the automatically excluded studies as a check on accuracy of the function.

Studies included for full-text screening will be double screened by two independent reviewers. Disagreements on inclusion or exclusion will be resolved by discussion and the input of a third reviewer if necessary.

Screening of studies intended to address research question 4 will take place in a second stage of screening in EPPI reviewer. Studies will be assessed for relevance, that is, whether they cover one of the programmes covered by the included impact evaluations, and whether they meet the minimum quality requirements, described in the section below on assessment of quality in qualitative studies and process evaluations.

Criteria for determination of independent findings

We will report data according to the intervention that the evidence was based on. We will attempt to avoid double-counting of evidence and synthesis of dependent findings from multiple studies in any single analysis by linking papers prior to analysis, and conducting sensitivity analysis as necessary. Where information is collected on the same programme for different outcomes at the same or different periods of time, we will extract information on the full range of outcomes over time. Where multiple outcomes are reported from different specifications, we will select the specification according to likely lowest risk of bias in attributing impact, for example the most appropriately specified outcomes equation. Where studies reported multiple outcome sub-groups for the same outcome construct (e.g. studies reporting simple, intermediate and complex knowledge), we may calculate a “synthetic effect size”. Where studies report multiple outcomes or evidence according to sub-groups of participants, we will report data on relevant sub-groups separately. Further information on criteria for determining independent effect sizes is presented below.

Data extraction and coding procedures

We will extract the following descriptive, methodological, qualitative and quantitative data from each included study using a standardised data extraction form (provisional form provided in appendix 2):

Descriptive data including authors, publication date and status as well as other information to characterise the study including country, type of intervention and outcome, population, context, type of intervention.

We will extract quantitative data for outcomes analysis using Excel. We will extract descriptive, methodological and qualitative data using KoBo Toolbox. Descriptive and qualitative data will be single coded by one reviewer and checked by a second reviewer.

Critical appraisal

For critical appraisals, we will report the full results for each study included in the final review.

Effect size calculation

An effect size expresses the magnitude or strength of the relationship of interest (Borenstein et al., 2009; Valentine et al., 2015). To address questions 1, 2 and 3, we will extract data from each individual study to calculate standardised effect sizes for cross-study comparison where possible. For continuous outcomes comparing group means in a treatment and control group, we will calculate the standardised mean difference (SMDs), or Cohen's d, its variance and standard error using formulae provided in Borenstein et al. (2009). An SMD is difference in means between the treatment and control groups divided by the pooled standard deviation of the outcome measure. Cohen's d can be biased in cases where sample sizes are small. Therefore, in all cases we will simply adjust d using Hedges’ method, adjusting Cohen's d to Hedges g using the following formula (Ellis, 2010):

g ≅ d ( 1 − 3 4 ( n 1 + n 2 ) − 9 )

Formulas for effect size calculations will be used depending on data provided in included studies. For example, for studies reporting means (X) and pooled standard deviation (SD) for treatment (T) and control or comparison (C) at follow up only:

d = x T p + 1 − x C p + 1 S D

If the study does not report the pooled standard deviation, it is possible to calculate it using the following formula:

S D p + 1 = ( n T p + 1 − 1 ) S D T p + 1 2 + ( n C p + 1 − 1 ) S D C p + 1 2 n T p + 1 + n C p + 1 − 2

Where the intervention is expected to change the standard deviation of the outcome variable, we will use the standard deviation of the control group only.

For studies reporting means (

X _

d = Δ X _ p + 1 − Δ X _ p S D p + 1

For studies reporting mean differences (

Δ X _

d = Δ X _ p + 1 S D p + 1 = X _ T p + 1 − X _ C p + 1 S D p + 1

For studies reporting mean differences between treatment and control, standard error (SE) and sample size (n):

d = Δ X _ p + 1 S E n

For studies reporting regression results, we will follow the approach suggested by Keef & Roberts (2004) using the regression coefficient and the pooled standard deviation of the outcome. Where the pooled standard deviation of the outcome is unavailable, we will use regression coefficients and standard errors or t-statistics to do the following, where sample size information is available in each group:

d = t 1 n T + 1 n C

d = 2 t N V a r d = 4 N + d 2 4 N

We will calculate the t-statistic (t)by dividing the coefficient by the standard error. If the authors only report confidence intervals and no standard error we will calculate the standard error from the confidence intervals. If the study does not report the standard error, but report t we will extract and use this as reported by the authors. In cases in which significance levels are reported rather than t or se(b), then t will be imputed as follows:

Prob > 0.1: t=0.5

Where outcomes are reported in proportions of individuals, we will calculate the Cox-transformed log odds ratio effect size (Sanchez-Meca et al., 2003):

d = l n ( O R ) 1.65

Where outcomes are reported based on proportions of events or days, we will use the standardised proportion difference effect size:

d = p T + p C S D ( p )

S D ( p ) = p ( 1 − p )

p = n T p T + n C p c n T + n C

In all cases after synthesis, we will convert pooled effect sizes to commonly used metrics such as percentage changes and mean differences in outcomes metrics typically used (e.g. weight in kg).

Unit of analysis

Unit of analysis errors can arise when the unit of allocation of a treatment is different to the unit of analysis of the study, and this is not accounted for in the analysis. We will assess studies for unit of analysis errors (The Campbell Collaboration, 2014). If unit of analysis errors exist, we will correct for this by adjusting the standard errors (Hedges, 2009, Higgins and Green, 2011, Waddington et al., 2012):

S E ( d ) ′ = S E ( d ) * 1 + ( m − 1 ) c

where m is the average number of observations per cluster and c is the intra-cluster correlation coefficient. Where included studies use robust Huber-White standard errors to correct for clustering, we will calculate the standard error of d by dividing d by the t-statistic on the coefficient of interest.

Methods of synthesis: Review questions 1, 2 and 3

Once we have identified all included studies we will map out all interventions, PITA characteristics, sectors and outcome measures provided in the included studies to determine how we will synthesise outcomes. We will only conduct meta-analysis of studies which we assess to be sufficiently similar. The inclusion criteria for the review are broad and we anticipate including studies that report on a diverse set of interventions, sectors and outcomes. It is therefore difficult to predict how meta-analysis will be used in the review prospectively. However, minimum criteria will be to only combine studies using meta-analysis when we identify two or more effect sizes using a similar outcome construct and where the comparison group state is judged to be similar across the two, similar to the approach taken by Wilson et al. (2011). We provisionally suggest that we combine studies in the same bivariate analysis when they evaluate the same intervention type, for example information dissemination intervention or community based monitoring. We provisionally also suggest that we will attempt to run separate bivariate analyses by the PITA characteristics the intervention incorporates. Multivariate analyses can take into account multiple interventions and PITA characteristics as moderator variables, or examine the correlation between outcomes along the causal chain (mediator analysis). Hence, we will assess the extent to which we can model the heterogeneity in intervention and comparator using moderator analysis and meta-regression. Where there are too few studies, or included studies are considered too heterogeneous in terms of interventions or outcomes, we will present a discussion of individual effect sizes along the causal chain. As heterogeneity exists in theory due to the variety of interventions and contexts included, we will use inverse-variance weighted, random effects meta-analytic models (Higgins & Green, 2011).

We will either use Stata's metan and metareg commands (Sterne et al., 2008), or the metafor package in R software to conduct the meta-analysis (R Development Core Team, 2008; Viechtbauer, 2010).

We will conduct separate analyses for the major outcome groups:

Service user / citizen (demand-side behaviours).

We will categorise outcomes that we identify into immediate outcomes (participation of service users and responses of service providers) under blocks 2 and 3 of the theory of change), intermediate outcomes (blocks 4 and 5) and endpoint stages (blocks 6, 7 and 8). We will subsequently analyse outcomes at each stage along the causal chain. We will also conduct moderator analysis to explore heterogeneity in the included studies. If feasible, we will use multiple meta-regression to explore the association between the moderator variables and the outcomes of interest (Borenstein et al., 2009). We will use sub-group analysis to explore heterogeneity by different treatment sub-groups.

Where sufficient observations exist, we will undertake moderator analysis by the following groups of variables:

Methodology: study design, risk of bias status, timing of evaluation (follow-up length).

We will also collect qualitative information from studies about the intervention and how engages with PITA characteristics. This information may subsequently be coded quantitatively to be used in moderator analysis, or used to classify intervention mechanisms in synthesis or in the further development of intervention causal chains. These characteristics may include: intervention objectives (to change processes, behaviours or both); whether interventions are strategic (complex, adaptable strategy to realise change) or tactical (tool-based); the source of intervention (local, NGO, government or researcher-led); the scale of the intervention (pilot experiment versus adoption of formal policy/law); use of demand side target group (e.g. households or citizen groups) and/or supply side (front line service delivery agents, local, regional or national service delivery managers, national government); agent on whom the onus for change is placed (unorganised citizens, community groups, NGO, government).

Assessment of heterogeneity

We will assess heterogeneity by calculating the Q-statistic, I², and Tau² to provide an estimate of the amount of variability in the distribution of the true effect sizes (Borenstein et al., 2009). We will complement this with assessment of heterogeneity of effect sizes graphically using forest plots. We will explore heterogeneity using moderator analysis in bivariate and, where possible, multivariate meta-regression specifications.

Methods of synthesis: Review question 4

In the context of ‘real world’ programmes, we are often concerned about project design and implementation fidelity as the principal reasons why programme evaluations show limited impacts, which is partly why advocates of mixed-methods evaluation approaches recommend collecting implementation process data (e.g. Bamberger et al., 2010; Snilstveit, 2012). We will use a thematic synthesis approach to extract information from project design and implementation documents and included impact studies on context, implementation and mechanisms. We will draw on the theory of change to do this as well as key frameworks for understanding why governance change may or may not happen, such as the Kaleidoscope Model (Resnick et al, 2015), which identifies key determinants of political change that could be influenced by PITA interventions, such as the relative power of proponents versus opponents in the adoption phase of the policy cycle. Finally, these will be compared with the effect sizes for each study to attempt to draw some conclusions about the extent to which the identified themes are systematically associated with outcomes.

Realist synthesis highlights variation in programme design in explaining differences in outcomes across contexts (Pawson, 2006). Realists argue that the effectiveness of a programme depends on the combined action of the behavioural mechanisms underlying it and the context in which it takes place. Behavioural mechanisms operate through the values, beliefs and past experiences of individuals in the social system. Thus, factors such as interpersonal networks and individual agency are important in the adoption and rejection of an intervention. The action of mechanisms depends in part on the context in which they are used. Behaviour change is achieved via the entire system of social relationships (the context) and, therefore, an intervention geared towards the achievement of behaviour change must be aligned with the context in which it is used (Waddington et al., 2009). The approach that draws these concepts together is called context-mechanism-outcome (CMO) synthesis. There are different ways of conducting CMO synthesis including iterations of a causal model (e.g. theory of change diagram) (Waddington et al., 2014), CMO tables (Petrosino et al., 2012) and qualitative comparative analysis (QCA) (Ton et al., 2017). QCA articulates the associations between empirical effects and context and mechanism conditions drawing on “truth-tables” which articulate all possible instances of conditions and show which cases share the same combination of conditions.

As CMO is largely an iterative process, the full list of CMO codes for analysis will be developed through the synthesis. Potential themes and context factors for investigation include: the enabling conditions that allow for project success; and systemic and social levels targeted by the intervention. Key enabling conditions could include whether the intervention is designed to build off of and work within local systems of power relations and social norms that uphold the social contract between the State and society (as in Halloran's “accountability ecosystem”, 2015), or the political salience of the public service targeted (Mcloughlin and Batley, 2012). Where key enabling conditions are already in place, an intervention effectively designed may be successfully implemented in isolation; where key conditions are missing, the intervention design may need to be adjusted or expanded to include complementary interventions that seek to strengthen the enabling environment. For example, an intervention seeking to build transparency and accountability through open data interventions may need to build a coalition of support that engages people at the point in the system targeted for data release, upstream, downstream, and externally to create an environment in which data is provided, demanded, and used (Hogge, 2010). These enabling conditions may change depending on context factors such as the target level of the intervention- whether it targets service delivery at community, sub-national, or national level (E-Pact Consortium, 2016) or whether the external stakeholders it seeks to engage are organised civil society or interest groups, marginalised or vulnerable groups, or citizens and service users more broadly (McGee and Gaventa, 2010). We also plan to conduct more detailed analysis of whether the bottleneck for good governance is likely to be properly identified as resting with citizens (e.g. lack of organization, lack of knowledge/capacity), with the system (e.g. lack of opportunities for citizens to engage), or with individual service providers (e.g. power relations, corruption). Data will be collected qualitatively from studies and then grouped using thematic synthesis or content analysis, and presented in tabular format.

Van der Knapp et al. (2006) is possibly the first example of a systematic review that explicitly incorporates context-mechanism-outcome synthesis. These authors indicate that the CMO synthesis is undertaken after the systematic review and meta-analysis. The broad approach is as follows:

Information on possible programme mechanisms is collected from studies during the coding phase. Van der Knapp indicate that one way of doing this is to search included studies for information about how or why the intervention is supposed to work, stating that “The focus in such a classification can be on behavioral and social ‘cogs and wheels’ of the intervention… but could also include administrative or legal mechanisms.” (p.6). The review team also anticipates identifying and coding mechanisms associated with particular interventions or PITA.

We will explore appropriate methods to link the meta-analysis with context and mechanism information, such as by using QCA (Befani, 2016). We note that the application of QCA is limited by the number of included studies, their comparability and the completeness of reporting within them, hence the application of QCA might be difficult in this review. We therefore also plan to use tabular reporting by intervention and further iterations of the theory of change by PITA groups.

Methods of synthesis: Review question 5

We will draw on standard approaches to synthesise economic appraisal evidence depending on the available cost evidence (Shemilt et al., 2011; Shemilt et al., 2008). If we are unable to identify sufficient data to calculate cost-effectiveness ratios for example, we will simply report the cost data that we identify in tables alongside effect sizes.