PROTOCOL: Administrative Reforms in the Public Sector and their Impact on the level of Corruption: A Systematic Review

Administrative corruption: the problem of this review

The problem this review will address is administrative corruption in the public sector.

This systematic review will refer to the administrative corruption in the public sector as the abuse of public office or public role for private gain at the implementation end of public policies or procedures. By public office, we refer to people belonging to any kind of public institutions (e.g. schools, hospitals) in addition to governmental ones, with the exclusion of politicians.

This review will consider the following main corrupt acts: bribery of public officials; embezzlement; misappropriation or other diversion of property by a public official (theft of state assets or diversion of state revenues), nepotism, as well as a range of corrupt behaviors such as trading in influence, abuse of function, illicit enrichment. Some corruption-related concepts will also be included in the review such as fraud and extortion. Fraud is a broader legal and popular term that covers both bribery and embezzlement. Extortion is a form of corruption often seen in cases when state officials return preferential business opportunities and freedom from taxation. According to OECD (2011), asking for a bribe becomes extortion “when this demand is accompanied by threats that endanger the personal integrity or the life of the private actors involved.”

Furthermore, absenteeism as a serious form of corruption in specific sectors, such as education and health (Transparency International 2013) will also be included in this review. Indeed, “stealing time” (e.g. not show up to work) while performing a public office is considered a serious form of diversion or theft of state assets (time in this case) (Hanna 2011: 8).

Although administrative corruption is often defined and studied together and as opposed to political corruption, this systematic review will exclude political corruption. We deliberately separate politics from administration. Political corruption affects the formulation of policies, it involves political decision-makers and happens when politicians who are entitled to make the laws (the rulers) are themselves corrupt. In political corruption, laws and regulations are abused and even tailored to fit the interest of the rulers. Political corruption takes place at the high levels of the political system and it has political repercussions (Amundsen, 1999). Administrative corruption occurs at any level of authority in the public administration and affects the implementation of policies. The review will not examine the acts of political corruption mainly because we believe that the mechanisms and the functioning of political corruption (i.e. manipulations of rules of voting systems) are significantly different and more complex that those related to administrative corruption. We thus believe that political corruption would need an ad-hoc separate review.

Anti-corruption interventions/reforms implemented at the administrative level: the scope of this review

The aim of this systematic review is to identify the impact of the existing interventions or reforms implemented at the administrative level in order to curb the types of administrative corruption mentioned above.

The majority of administrative interventions and reforms in the public sector have been developed to curb public corruption at the state administration level (World Bank 1997, USAID 2009: 4, European Commission 2014^b). Several anti-corruption programs have been, indeed, developed both at the domestic and international level. In the words of Gounev (2012) “significant funds are being allocated by the EU cohesion policy to the strengthening of administrative capacity at all levels, including regionally, especially in less developed regions and newer EU Member States. The added administrative efficiency that should result will reduce actual levels of corruption and consequently the pressure on personnel to become corrupt. Once administrative efficiency has been improved, additional specific anti-corruption measures can be added.”

The issue so far has been that, in spite of the large amount of anti-corruption programs implemented (cfr. European Commission 2014^b), the literature bringing them into line is most of the times an overview and compilations of general findings, leading to conclusions which might be too general and vague to inspire policy-makers. In addition, evaluations are more easily available on outcomes of higher sanctions and intensified controls, than on administrative reforms which try to reduce the “market” for corruption by making public bureaucracies more efficient and “user-friendly”. Though there has been a noticeable trend of developing significant anti-corruption programs helping to understand the causes and capture the indicators of corruption, there has been little systematic research into the impact of these anti-corruption efforts in reducing corruption. As a consequence, there has been a serious lack of comprehensive assessment of the implementation and the efficiency of anti-corruption measures in different settings.

Given the need for such evaluation, we propose a systematic review of all studies that have examined administrative anti-corruption reforms as well as their effects on the level of corruption. As formulated by Johnsøn & Søreide (2013: 1), “producing evidence that anti-corruption interventions have an impact in reducing corruption is a relatively new area for research and evaluation”. This is the contribution this systematic review is aiming at.

Why the focus on administrative corruption and why it is a target for administrative anti-corruption interventions/reforms?

In order to keep the systematic review manageable, it only covers administrative corruption. Along with this primary reason, the focus on administrative corruption and the administrative anti-corruption reforms is explained as follows: (1)

The importance of the parties involved in administrative corruption

The reason we focus on the public sector corruption is that the state plays a decisive role and it is reflected in most definitions of corruption. Nevertheless, administrative corruption also exists within and between private businesses, within non-governmental organizations, and between individuals in their personal dealings. However, for this review, when the act of corruption does not imply or emphasize a state-relationship, it will be excluded from the review. The term “public sector” used for the review does not exclude the private sector or private companies which might be equally corruptible with serious public flow-on effect. For this review, we take into consideration the counterparts/protagonists to the corrupt officials who can be any non-governmental, nonpublic individual, or the general public. In sum, acts of corruption with no state components or public office involvements will be excluded from the review.

This leads us to another significant reason to focus on administration corruption. As a matter of fact, whenever administrative corruption happens, it does involve numerous protagonists of the society, namely bureaucratic elites, public officials and interested individuals or parties. Administrative corruption involves appointed bureaucrats and public administration staff at the central or subnational levels. As advanced by Johnston (2011), administrative corruption implies that within public bureaucracies, abuse of roles, powers or resources have been found and it has been initiated by staff officials, their superiors or the agency clients. Although anchored in large and centralized governments, protagonists of administrative corruption might also be individuals or institutions from elsewhere in the public sector (Johnston, 2011).

Examples of the types of possible interventions

Many important organizations and actors are behind the administrative anti-corruption reforms. Interventions seeking to reduce the discretionary power and monopoly of the power by the government officials, improvement of law enforcement, civil service reform, increase of transparency and improvement of citizen participation (UNDP, 1997). A genuine reform of bureaucracy by reducing the incentives and opportunities of corruption is one of the anti-corruption interventions proposed by UNDP.

Efforts and reforms of anti-corruption are deployed in all sectors of education, environment, energy, health, information technology and transport (USAID, 2011). In 2007, for instance, a program called “program of collective action” comprising the directions and principles to counter corruption was developed by OECD.

One of the anti-corruption reforms suggested by Schleifer and Vishny (1993; 610) is to produce competition between bureaucrats in the provision of government goods, which will drive bribes down to zero. This arrangement has been introduced in many agencies of the US government, in particular the passport office. It seems that creating competition in the provision of government goods might increase theft from the government but at the same time reduces bribes.

Decentralization is another example of anti-corruption reforms. When applying a decentralization policy, the responsibility for the implementation of a given policy passes from a higher level of government to a lower one (Hanna 2011: 9).

The use of technology and electronic payments can help to bypass various lengthy bureaucratic procedures and to avoid direct contact with civil servants. This can reduce the opportunity for bribes (Hanna 2011).

In the case of Italy, the new anticorruption law ratified in November 2012 was specifically addressed to the reduction of administrative corruption through the increase of transparency and dissemination of information by public administration. Within the three-year national anti-corruption and integrity action plan addressed to all administration bodies, the following main areas of administrative anti-corruption reforms have been considered for intervention:

Reduce the likelihood of corruption

The following are some examples of practical interventions which have been developed within each of these areas:

Public e-Procurement to manage public bid processes online and increase transparency (e.g. Acquistiinretepa.it) and shifts of management staff.

The types of interventions vary also on the basis of the country where they are developed. For example, Public Expenditure Tracking Surveys (PETS) are considered among the few methods having a positive impact on corruption in service delivery in developing countries with a weak system of governance (Sundet 2007, p. 2). The application of PETS in Uganda, for example, shows that the flow of funds improved dramatically, from 13 percent on average reaching schools in 1991-95 to around 80 percent in early 2001 (Reinikka and Svensson 2003). However, this method was not successful in other countries with similar characteristics (e.g. in Tanzania) (Sundet 2007).

3.1.1. Type of study designs

Studies eligible for inclusion in this review must quantitatively assess the effects of the administrative reforms on the level of corruption in a given context. The included studies have to quantitatively measure the intervention either as a dichotomous, ordinal or scaled variable.

To be included in this review, studies should ideally use randomised controlled trials (RCT's), or any quasi-experimental method as study design. Therefore, we will include in this review RCTs, cluster RCTs, quasi-RCTs, pretest-posttest two groups studies (with a non-equivalent control group) and interrupted time series studies. The latter must have a clearly identifiable time point at which the intervention occurred and must include information for at least three time points before and after the intervention. Given the limited number of studies in this field of research using any control groups, we will include also studies in which neither a control group nor a random assignment is present, such as pretest-posttest one group studies and cross-sectional studies like surveys (suchas the study of Yongqiang, 2011).

Both the definition of corruption and the criteria of inclusion are deliberately broad because we are concerned about the limited number of possibly eligible studies. If a reasonable number of studies (more than twenty) can be identified and located we shall consider restricting the review to those studies meeting the highest methodological standards. At the documentary stage, however, it seems safer to include as many studies as possible.

Anyway, we will include only those studies which clearly describe their research design and clearly report the change in the level of administrative corruption due to a specific intervention.

3.1.2. Types of participants

The review will include studies developed in low, middle and high income countries (defined according to the World Bank classification method). For studies to be included they will need to collect and report on data at an individual or aggregated level.

If the outcomes of interest are measured at an aggregate level, the units of analysis will be any geographic place (e.g. community, city, province, state, region, or country) and/or governmental agencies, public and private companies within a developed or developing country. If the outcomes of interest are measured at an individual level, the unit of analysis will be the individual (e.g. civil servants, representatives of governmental agencies, public and private companies).

In particular, the review will include studies focusing on the following units of analysis:

Geographical areas (countries, regions, provinces, local municipalities),

The type of unit of analysis depends on the different studies' sources of data. For example, studies considering the level of corruption as measured by administrative statistics, which have specific geographical areas as unit of analysis, will consider as population a number of provinces, local municipalities, or countries. On the other side, studies analyzing corruption as measured by population surveys or self-report studies, will consider a population of individuals or companies, which are the unit of analysis of such investigations.

The above-mentioned units of analysis will be analyzed separately, in order to highlight how the impact of administrative measures on corruption differs across them.

3.1.3. Types of interventions

The types of interventions of this review, or the independent variables, are the administrative anti-corruption reforms.

As mentioned in Chapter “Background”, Section 1.1, “Administrative corruption: the problem of the review”, administrative corruption in this review is intended as “the abuse or misuse of public office for private gain” and includes acts of bribery, fraud, extortion, embezzlement, misappropriation or other diversion of property by a public official, theft of state assets or diversion of state revenues, absenteeism, extortion, nepotism. Therefore, anti-corruption interventions or reforms which are not specifically meant to curb or prevent these administrative corrupt acts will be excluded from this review.

This systematic review will include administrative anti-corruption reforms without any geographic and temporal limitations. The study of Hanna (2011: p. 17) demonstrated that temporal exclusion criteria do not have much impact on the review's findings because the study of anti-corruption strategies has only taken off in recent years.

Furthermore, the administrative anti-corruption reforms considered for the review need to concern reforms belonging to at least one of the following six main categories (see definitions under section 1.3):

Economic

Studies included in this review should clearly distinguish the types and characteristics of the interventions (administrative reforms) according to the above-mentioned criteria.

The intervention has to be quantitatively measured. In particular, it could be measured either as a dichotomous variable (e.g. presence / absence of a specific legislative reforms), ordinal variable (e.g. long-term implementation of a specific legislative reform / medium-term implementation / short-term implementation), or as a scaled variable (e.g. increased amount of civil servants' salaries).

3.1.4. Types of outcome measures

This review will include studies focused on interventions which have direct effects on the level of administrative corruption, and which produce outcomes directly related to our definition of administrative corruption (mentioned in Section 1.1, p. 5) ³ . We are aware that, besides more direct impacts (such as the reduction in the level of specific forms of corruption), interventions might also have indirect effects such as improved public integrity, improved quality or quantity of specific public services (Bjorkmann and Svensson 2009). Usually the latter are a consequence of the former and they are identifiable only in the long-term period. It can also happen that interventions originally addressed to the improvement of a specific public service, had an impact also on the level of corruption.

Furthermore, we understand that the outcome of specific anti-corruption reforms might have a wider positive displacement than simply reduce the level of corruption. However, the purpose of our review is to mainly focus on interventions with direct effects on the level of administrative corruption. Such decision will help us not to get lost in other types of outcome indicators, such as public service quality or access to public service. Nevertheless, to not deny the relevance of these further outcomes, we would mention and classify them within the review when it clearly appears they are direct consequences of the selected interventions. Therefore, the type of outcome or the dependent variable of this review is the level of administrative corruption ⁴ .

The dependent variable should be measured quantitatively. It means that the assessment of the outcomes of the interventions (i.e. levels of corruption before and after the intervention) should be quantitatively measured. The source of quantitative data to measure the level of administrative corruption could be:

criminal justice statistics, i.e. number of cases of corruption reported to the police and recorded by the police in a given context, the number of cases of corruption prosecuted.

Studies including at least one of the above-mentioned measures of corruption will be included in the systematic review.

3.1.5. Exclusion criteria

As articulated in Section 1.1., studies focused on political corruption are excluded from the review.

Furthermore, as mentioned in Section 1.1 and in Section 3.1.2., studies which do not examine administrative corruption offences and reforms with the involvement of at least one party as the state or public office are excluded from the review (private on private corruption).

Studies not using a quantitative method of assessment of the outcome will be excluded (see below for further details).

Qualitative and purely descriptive studies on anti-corruption strategies, as well as those aiming at testing the causes and consequences of corruption are excluded from this review. Indeed, even if these investigations may be interesting to understand the origins of corruption and their interrelation with the social and economic environment of a given country, they will not be included in this review because they do not measure the effects of anti-corruption reforms.

3.1.6. Examples of studies eligible for inclusion

On the basis of the abovementioned criteria, this section provides some examples of studies eligible for inclusion in this review.

1. Treisman, D. (2000). Decentralization and the quality of government.

This study statistically demonstrates how the degree of political decentralization in a country can affect the quality of its government and the level of perceived corruption. The analyses of the data on up to 154 countries suggest that states which have more tiers of government tend to have higher perceived corruption.

2. Steves, F. and Rousso, A. (2003). Anti-corruption programmes in post-communist transition countries and changes in the business environment, 1999-2002. European Bank for Reconstruction and Development Working Paper No. 85.

Using the results of a large survey of firms across 24 post-communist transition countries, this paper shows that omnibus anti-corruption programmes and membership in international anti-corruption conventions have not led to reductions in the level of either administrative corruption or state capture, between 1999 and 2002. Moreover, it finds out that the implementation of anti-corruption programmes is positively correlated with the perception of corruption.

3. Olken, B., A. (2005). Monitoring corruption: evidence from a field experiment in Indonesia. Working paper 11753. National Bureau of Economic Research. Cambridge, MA.

This study provides an example of ex-post cost-benefit analysis. The study quantifies in monetary terms both the costs and the benefits of an anti-corruption intervention, after its implementation. In particular, the author of this paper uses a randomized filed experiment to examine different approaches to reducing corruption across 600 Indonesian village road projects. He demonstrates that traditional top-down monitor can reduce corruption. In particular, he finds out that “announcing an increased probability of government audit, from a baseline of 4% to 100% reduced the missing expenditures (measured as discrepancies between official project costs and an independent engineer's estimate of costs) by about 8 percentage points”. On the other side he discovers that “increasing ‘grass-roots’ participation in the monitoring process only reduced missing wages, with no effects on missing material expenditure”. Overall, the net social benefits of this type of intervention were US$250 per village, 150% more than the costs of the intervention (Johnson 2014: 6).

4. Yongqiang, N. (2011). Government Intervention, Perceived Benefit, and Bribery of Firms in Transitional China. Journal of Business Ethics (2011) 104:175-184. Springer.

This study interview 600 business graduates (EMBA and MBA), working in different Chinese companies, in order to understand if there are any relationship between government intervention (in the forms of licenses, quotas, permits, approvals, authorizations, franchise assignments, etc.) and bribery/corruption, and if this relationship was somehow mediated by the firm's perceived benefit it can get from the corrupt practice. The results show that the government intervention causes bribery/corruption indeed, but a firm's perceived benefit fully mediates this relationship.

3.3.1. Selection of studies

All potential studies identified through the searches will be stored into reference management software and duplicates removed.

Two independent review authors will start from reading the titles and abstracts of identified studies in order to determine their eligibility against the inclusion/exclusion criteria. When a title or abstract cannot be rejected with certainty, the full text of the article will be reviewed. In case of disagreement between the two review authors at any stage, this will be resolved by discussion, and if necessary, consultation with a third review author.

The included studies will be then organised into categories based on their methodological approach, type of interventions, types of administrative corruption acts, geographical region and other factors.

Studies that appear to be relevant but do not meet the inclusion criteria will be listed in the table of excluded studies with reasons given for exclusion.

3.3.2. Data extraction and management

Data on the identified studies will be extracted independently by two review authors on the basis of the template provided by the Cochrane Public Health Group.

In particular, the following information will be stored:

year of publication;

In case of disagreement about whether or not a study should be included, the lead author (Professor Martin Killias) will act as arbitrator.

Data will be entered into Review Manager (RevMan 5) software and checked for accuracy.

A coding sheet with a description of the data collected is included in the appendixes.

3.3.3. Missing data

In case the selected studies do not report sufficient data to calculate effect sizes, we plan to contact the authors of the studies and ask for them. In case the authors will or could not provide the data, we will calculate response ratios effect size. The response ratio provides a measure of the relative change in an outcome caused by an intervention. Besides some limitations (see Borenstein et al. 2011), the response ratio offers good possibilities for estimation.

3.3.4. Description of methods used in the component studies

Studies eligible for inclusion in this review must quantitatively assess the effects of the administrative reforms on the level of corruption in a given context.

The method used by the studies covered by this review should ideally be based on randomised control trials (RCT's), such as the study of Olken (2005) (see summary on page 24).

Randomization and field experiments can be considered the most reliable methods to evaluate anti-corruption initiatives, because they control well for bias, but they are not the most used. Indeed, it is often difficult to randomly establish a control group and to obtain support by decision makers or donor organizations (Johnsøn & Søreide 2013: 18). Therefore, quasi-experimental methods will also be included in the review. According to Johnsøn & Søreide (2013: 20) they are most applicable for evaluating large interventions targeting a large number of people or units. Eligible study designs include randomized controlled trials, natural experiments, interrupted time-series designs, and any other quasi-experimental design with or without control group (e.g. pretest-posttest two groups studies and pretest-posttest one group studies), as well as cross-sectional studies like surveys (e.g. Yongqiang, 2011).

So far our review shows a prevalence of studies based on cross-sectional studies using ordinary least square multiple regression analyses, such as the one of Yongqiang (2011) estimating the effect of a government intervention on Chinese firms' bribery, the one of Treisman (2000) studying the impact of political decentralization on perceived corruption, and the study of Steves and Rousso (2003) focusing on the relationship between anti-corruption programmes and the level of administrative corruption.

3.3.5. Criteria for determination of independent findings

There may be different sources of non-independence of findings, the most important regards the following issues:

Multiple indicators of the level of corruption reported in a single study (e.g. police recorded cases, prosecuted cases). Only one will be selected for the analysis, taking into consideration the most updated data and the comparability with the results of the other studies included in the review.

3.3.6. Details of study coding categories

Preliminary coding sheets have been produced for this protocol and included in its appendixes. They illustrate the systematic method of extracting information regarding each study's characteristics, research design, participants, type of intervention, type of outcome, risk of bias, etc.

Further table will be used to summarize the results of the critical appraisal for each study.

3.4. Assessment of risk of bias in included studies

When developing a systematic review, the most difficult source of bias to control for is the low methodological quality of the selected studies (Jarde et al. 2012).

In order to ensure the selection of studies methodologically sound, the researchers will assess the overall quality of each of them on the basis of the STROBE statement (Strengthening the reporting of observational studies in epidemiology) ⁷ checklist.

In addition, the potential specific risks of bias (selection bias, performance bias, detection bias, attrition bias, reporting bias, and other potential risks) of included studies will be also assessed using the relevant Cochrane EPOC (Effective Practice and Organisation of Care) “Risk of Bias” tool ⁸ .

In particular, for studies with a separate control group (RCTs; NRCTs and CBA studies), the overall risk of bias will be estimated as low, high or unclear, on the basis of the nine standard criteria identified in the EPOC classification (1. Sequence generation, 2. Allocation concealment, 3. Blinding of participants and personnel, 4. Blinding of outcome assessment, 5. Incomplete outcome data, 6. Selective outcome reporting, 7. Other potential risks of bias). For interrupted time series (ITS) studies, the overall risk of bias will be assessed on the basis of the six standard criteria identified by EPOC (1. Independence of the intervention of secular changes; 2. Pre-specification of the shape of the intervention effect; 3. Likelihood of the intervention to affect data collection; 4. Blinding of outcome assessment; 5. Selective outcome reporting; 6. Other potential risks of bias) (see also Petticrew & Roberts 2006: 138).

It has to be mentioned that, on the basis of the preliminary literature review we have conducted, there are few true experimental or quasi-experimental studies providing data on the effects of anti-corruption reforms, and, thus, several non-experimental studies will have to be appraised before including them into the review.

Sanderson et al. (2007) highlighted that there is not a single and unanimously recognised tool for assessing the quality of non-experimental studies. Discussion on this issue was developed by Jarde et al. (2012 and 2013) in two different articles, leading to the creation of the tool Q-Coh.

Following the checklists provided by the STROBE statement, the Q-Coh tool, and the framework provided by Petti crew and Roberts (2006: 142) this review will assess the risk of bias for non-experimental studies looking at four specific items which can affect the methodological quality of a research: a)

Design of the study – The quality level of the design of non-experimental studies will be evaluated as low, high or unclear by answering the following questions: 1. What question(s) is the study aiming to answer?; 2. Was the study specifically designed with this question(s) in mind?; 3. Are the setting, locations, population and data collection method of the study carefully described? (Yes/No/DK); 4. Does the study clearly define all outcomes, predictors, potential confounders, and effect modifiers? (Yes/No/DK for each item); 5. Does the study provide sources of data and/or details of methods of measurement for each variable of interest? (Yes/No/DK); 6. Does the study explain how the study size was arrived at? (Yes/No/DK); 7. Does the study describe all statistical methods, including those used to control for confounding? (Yes/No/DK); 8. Does the study explain how missing data were addressed? (Yes/No/DK); 9. Does the study mention any effort to address potential sources of bias? (Yes/No/DK).

On the basis of these evaluations we will provide each study with an assessment of the study quality “weight”, the overall weight of evidence which the study provides. During this phase, we will also provide technical details on analytical ⁹ and statistical methods used to test the effectiveness of the anti-corruption reforms.

To obtain a preliminary and visual assessment of the risk of bias, forest plots ordered by judgements on each ‘Risk of bias’ entry will be analysed (through RevMan5). Forest plots on the risk of bias are useful to provide a visual overview both of the relative contributions of the studies at low, unclear and high risk of bias, and also of the extent of differences in intervention effect estimates between studies at low, unclear and high risk of bias.

Effect sizes belonging to studies presenting different risk of bias will not be combined together. In particular, studies with a ‘high’ or ‘unclear’ risk of bias will be kept separated from the main analysis and treated as a separated category. Ideally, studies at high or unclear risk of bias should be given reduced weight in meta-analyses, compared with studies at low risk of bias. However, formal statistical methods to combine the results of studies at high and low risk of bias are not sufficiently well developed. Therefore, the major approach is still to restrict meta analyses to studies at low (or lower) risk of bias, or to stratify studies according to risk of bias (Spiegelhalter 2003).

3.4.1. Assessment of reporting bias

In order to assess whether there is a systematic difference between reported and unreported findings, the following questions will be answered: 1. Were some outcomes, but not others, selectively reported in the study depending on the nature and direction of the results? (Yes/No). 2. Were the research findings published or non-published depending on the nature and direction of the results? (Yes/No); 3. Were the publication of the research delayed depending on the nature and direction of the results? (Yes/No); 5. Were the research findings published in journals with different ease of access or levels of indexing in standard databases, depending on the nature and direction of the results? (Yes/No).

In order to further check for publication bias, we will assess whether there are significant differences between the results of published and unpublished studies, by using funnel plots to assess the relationship between effect size and study precision and by running a sensitivity analysis to identify the level of impact caused by different effect estimates in small studies. Considering that the statistical significance of the results is often considered a requirement for publication (Gerber & Malhotra 2008a, 2008b), a large number of published studies with a p-values just below the standard 0.05 threshold can be considered a first sign of the presence of publication bias (Molina 2013: 32).

3.5.1. Effect size extraction and transformation

According to the nature of the data being collected, the comparable effect size estimates will be extracted from existing studies together with 95 percent confidence intervals. This effect size data or data that can be used to calculate a standardized effect size will be recorded in free-text format as part of the standardized coding sheet. A second reviewer will double-check the coding and data extraction for every study that contains effect size data. All relevant data will be input into Rev Man or Comprehensive Meta-Analysis software to calculate standardized effect sizes and their standard errors and the conversion between effect size types, to ensure that a common metric is used.

The different effect sizes will be then treated and calculated as following:

In case both the dependent and independent variables are dichotomous, the Odds-Ratio (OR) effect size will be calculated.

Odds-Ratio, Correlation Coefficients and Standardized Mean Effect Sizes Effect sizes will be calculated both by hand using Microsoft Excel as well as by using the David Wilson's Practical Meta-Analysis Effect Size Calculator ¹⁰ , or other similar programmes (Rev Man or Comprehensive Meta-Analysis software).

Effect sizes will be reported using forest plots.

3.5.2. Effect size combination

Following the Cochrane Handbook, the combination of intervention effect estimates across studies will be computed using the inverse variance weight method (IV). Assuming that the different studies differ for a number of reasons (e.g. study design, type of intervention, type of country covered, etc.), a random-effects meta-analysis will be developed and the variance components (τ²) calculated accordingly (method of moments estimator in RevMan).

The meta-analysis will combine only those studies focused on the same outcome, the same unit of analysis, the same type of intervention, the same type of study design, and the same type of effect size. For example, an Odds Ratio effect size in a cross-sectional study will not be included in the same analysis as a standardized mean-difference effect size in a longitudinal study. Only one effect estimate per study will be included in a single meta-analysis.

Therefore, we will calculate separate effect sizes, and report them by using forest plots, for each different group indicated in Table 2 below.

The review will also provide technical details on analytical ¹¹ and statistical methods used to test the effectiveness of the anti-corruption reforms.

3.5.3. Assessment of heterogeneity

In order to test whether the different effect sizes that are averaged into a mean value all estimate the same population effect size, we will report tests for between-studies variability ¹² of effects (Hedo-Medina et al. 2006). In particular, we will compute a Q test of homogeneity between groups and calculate the I² index to complement the results of the Q test (Higgins and Green 2011, Huedo-Medina et al. 2006).

We assume the presence of between-studies variability in our review as far as the included studies differ on a number of characteristics (e.g. geographical region, characteristics of the samples, variations in the treatment, in the design quality, in the unit of analysis, etc.) and as such assume and implement a random-effects model for meta-analysis.

Depending on the number of available studies, we will explore potential causes of heterogeneity, based on the subgroups listed in Table 2. Caution will be taken in the interpretation of results with a high level of unexplained heterogeneity.

3.5.4. Moderator analysis

In order to test whether and how variation among studies in effect size is associated with differences in key contextual factors (e.g. geographical region, country income level and types of corruption) we will run a moderator analysis.

If our moderators are categorical (e.g. low, medium and high income countries; corruption intended as bribery, diversion of public assets, etc.), we will estimate models analogous to one-way ANOVA.

Considering that we might have to deal with small sample sizes within each sub-group, if the number of cases in the sub-groups is lower than 5 we would need to use a pooled estimate of the variance component otherwise we will use separate estimates of the variance component (τ²) within sub-groups.

Even if we assume that the moderators included in our review will be mainly categorical, in case of continuous moderators and if we want to test more than one moderator (either continuous or categorical) we will use meta-regression.

3.5.5. Sensitivity analysis

In order to check for the certainty ¹³ of our findings we will perform several sensitivity analyses in our review, including: type of counterfactual, experimental vs. non-experimental studies, randomized vs non-randomised studies; relevant differences in impact measure, traditional versus meta-analyses including the risk of bias and the uncertainty about the risk of bias, etc. It is likely that other issues suitable for sensitivity analysis will be identified during the review process when the individual peculiarities of the studies under investigation are better defined. We will specify and report all of them in the last version of our review.

In case sensitivity analyses will show specific decisions or missing information strongly influencing the findings of the review, we will try to obtain extra information on the concerned studies.