The Impact of Business Support Services for Small and Medium Enterprises on Firm Performance in Low- and Middle-Income Countries: A Systematic Review

3.1.1 Types of studies

The review draws on a broad search to identify studies that relate to the interventions aimed at SMEs in LMICs.

To address the review questions (i.e. review questions i. through iv.), the review focused on quantitative analysis and included only studies that used rigorous impact evaluation in the form of experimental (randomised controlled trials, or RCTs) and quasi-experimental methods – such as regression discontinuity design (RDD), instrumental variables (IV), difference-in-differences (DID), matching on covariates, or propensity score matching (PSM), and any other studies that purported to control for selection bias (for example, Heckman two-step estimator). ⁸ Studies selected must have reported controls for the endogeneity of programme placement or self-selection into the programme. Experimental and quasi-experimental methods are widely seen as the best tools when the main objective is to estimate the causal impact of an intervention or policy (for example, see Duflo et al., 2008). When an intervention is carefully designed or the identification strategy of an observational study convincing enough, the findings on the impact of the programme or intervention are said to have internal validity, that is, one can claim that the difference in the outcomes between treatment and control groups was caused by the intervention. ⁹

This review thus only considered those studies that assessed the impact of an intervention comparing the treatment (or eligible) and the control (or comparison) groups at one or more points in time. In cases where more than two treatment phases were considered, the estimates involved comparison of the two treatments. ¹⁰ The studies considered are therefore drawn from cross-sectional and panel data datasets. Quasi-experimental studies that relied on observation data must have shown balance tests or use a matching method to control for imbalances in observed characteristics to warrant inclusion. Moreover, studies using matching methods needed to clearly state the eligibility criteria of the programme to make the case that the problem of selection bias was (mostly) due to observed characteristics. Most importantly, the studies included documented the impact of any business support service on SMEs compared to business as usual. In addition, and as noted prior, the review compared the impact of different types of business support service on firm performance.

As discussed in Waddington et al. (2012b), focusing exclusively on studies that use experimental and quasi-experimental methods may significantly restrict the studies that can be included in a review. Although this is a legitimate concern particularly if one is interested in comparing different interventions, we accepted this trade-off based on the idea that findings that do not control for selection biases may be misleading in terms of policy relevance.

3.1.2 Types of participants

This review only focuses on studies that evaluate policies aimed at supporting SMEs in LMICs (as defined by the World Bank's classification). The focus on LMICs is justified firstly because private firms in these countries tend to be more labour intensive and less innovative, and consequently are a main employer for a large proportion of the labour force (e.g. Acz and Amoros, 2008; Cravo et al., 2012). Secondly, restricting the scope to LMICs helps to identify the binding constraints that SMEs might face in similar institutional contexts, such as in a number of African settings. The term SME covers a wide range of definitions and measures that vary depending on country context and reporting methods. Some of the commonly used criteria to define and measure SMEs are the number of employees, total net assets, sales, and investment level (Ayyagari et al., 2007). The most common criterion used to classify SMEs is based on employment information, often due to data availability. The cut-off used to define SMEs is usually 250 employees. ¹¹

This review draws on this definition and considered SMEs to be firms that have up to 250 employees. We also included studies that do not provide the number of employees but use annual revenue to classify firms as SMEs instead ¹² . Other types of interventions, such as those aimed only at supporting entrepreneurship and the creation of microenterprises (for instance, microfinance ¹³ ) are not considered for this research. We make this distinction because self-employed and micro-entrepreneurs are thought to have a different nature in comparison to SMEs. ¹⁴ The former, especially in LMICs, are comprised of less productive or informal enterprises of few employees in the fringe of markets. Furthermore, these enterprises are often ineligible for those public interventions covered in this review. Thus, the common definition of SME based on number of employees fits our purpose of covering a broad set of interventions and potential relevance for African countries. ¹⁵

Though the literature recommends that synthesis is informed by the theory of change embedded in the design of an intervention (see Waddington et al., 2012b), our focus extends beyond the outcomes directly anticipated by an intervention to include unanticipated outcomes also.

3.1.3 Types of interventions

Support to SMEs is related to the dual goals of productivity growth and employment generation; the theory of change that motivates SME support services is linked to fostering institutions that enable SMEs to grow in these goals. Figure 1 provides a general illustration of the theory of change for the interventions surveyed in this review, which are detailed in Table 1.

Following the discussion in Section 1, we include the following interventions in our review: ¹⁶

Tax simplification; might be seen as an institutional improvement. The support to SMEs in this case is usually accompanied by actions that support formalisation of SMEs. Therefore, tax simplification is intended to provide incentives for informal SMEs to formalise. For instance, new legislations might establish that SMEs pay taxes based on a fixed percentage of gross revenue, usually reducing the tax burden paid by firms (e.g. Fajnzylber et al, 2011). Tax simplification incentives can also be coupled with strategies that streamlining the process of opening a business (e.g. Bruhn and Mckenzie, 2013).

Exports/Access to External Markets; defined as interventions that correct market failures such as information externalities and help SMEs overcome obstacles to exporting (Volpe and Carballo, 2010; Volpe et al., 2010; World Bank, 2010). As suggested in Section 1, this type of intervention is related to information asymmetries that prevent firms from having access to external markets. Institutions that promote exports usually offers support through the creation of export consortiums, trade promotion in international business fairs, market research, trademark development, and trade information. For instance, Weiss et al (2011) describes a public policy instruments for export promotion in Chile called Export Marketing Assistance (EMA). This initiative provides participant SMEs knowledge about external markets, specialised information and allow firms to participate in international fairs.

Support for innovation policies is based on the idea that social returns to innovation exceed private returns (Lundvall and Borras, 2005; Acs and Audretsch, 1988). Interventions designed to support innovation vary. This review will consider different types of innovation support subsidies and tax incentives, as identified in the preliminary search.

Matching grants are interventions that provide a government subsidy related to those costs firms incur with regards to training, marketing, and/or attending a trade fair.

Local production systems: defined as interventions that help individual firms benefit from agglomeration externalities and overcome the coordination failures that prevent SMEs from capturing these externalities (Schmitz 1995; Schmitz and Nadvi 1999; Giuliani et al., 2005). Arraiz et al (2013) describes a Supplier Development Program in Chile where collective action aims at establishing a long-term commercial relationships between large buying firms and their small and medium enterprise (SME) suppliers to increase competitiveness. The objective is to collectively form a mutually beneficial relationship to help firms compete more effectively in the marketplace.

Training and technical assistance: defined as interventions that provide support for employee training and technical assistance, based on the idea that skills improve employability and wages of workers and contribute to firm productivity (Attanasio et al., 2011; Rosholm et al., 2007). This type of intervention also includes consulting services and management practices such as those considered by the World Bank (2010), Bruhn et al. (2013) and Bloom et al. (2013).

SME Financing/Credit Guarantee: adverse selection and moral hazard in credit markets generate financial constraints, which in turn restrain SME activities (Beck and Demirguc-Kunt, 2006; Michelacci and Silva, 2007; Canton et al., 2012). The review will consider in this line of support, interventions that provide loans or insurance services to SMEs, such as those noted in World Bank (2010) for credit and in Oh et al. (2009) for credit guarantee schemes.

It is important to note that various sub-components of business support interventions may overlap in the review/analysis. To avoid this we developed a conceptual model to categorize interventions as accurately as possible. Whenever possible sensitivity analyses are conducted using moderator factors and/or excluding studies with high risk of bias.

3.1.4 Eligible comparison groups

Most of the papers included in this review investigating the impact of a public policy targeting SMEs compare a treated (or eligible) group with a control group (or comparison group in the case of quasi-experimental design). However, we distinguish studies that compare treatment and control (or comparison) groups from those studies that have more than two treatment arms, and further separate evidence according to intervention design. In the case of RCTs, for instance, an intervention can use a phase-in design, an encouragement design, cluster (or block) randomisation, or pure randomisation (see Duflo et al., 2008). Different designs have two implications: (1) they almost always identify different parameters - intention to treat (ITT), average treatment effect (ATE), average treatment effect on the treated (ATT), local average treatment effect (LATE) and so on; and (2) they almost always differ in terms of data collected (different take-up rates, different attrition rate, different risk of contamination bias and so on).

3.1.5 Types of outcome measures

Our review covers studies that looked at both intermediate (or secondary) outcomes (such as access to credit, formalisation and access to external markets) and final (or primary) outcomes (such as profits, employment generation, and productivity). To be included in the review the study had to report estimates to at least one final outcome. Studies that reported estimates for secondary outcomes only were excluded. ¹⁷ To understand the causal chain of each intervention, this review looked for context-specific variables that can help explain either the failure or success of an intervention.

For the purposes of this review, we defined firm performance impacts as referring to objective indicators such as revenues, profits, job creation, innovation, formalisation, number of workers trained, and access to credit. Only factual/objective measures of firm performance impacts are included: subjective measures on beliefs and perceptions are excluded.

Primary outcomes

Primary outcomes of SME support revolve around better firm performance and growth and therefore can be categorised as: firm performance (e.g. revenues and profits), employment, productivity, and labour productivity. The following are examples of studies looking at these outcomes, which we include in the review: Mano et al.'s (2012) experiment in Ghana to analyse the effect of an SME training programme on sales and profit; Benavente and Crespi's (2003) study of the effects of an association strategy on productivity in Chile; Arraiz et al.'s (2012) assessment of the effect of value chain support on sales, employment and exports in Chile; Tan's (2009) evaluation of different Chilean SMEs programmes for technical assistance, cluster programmes, technology programmes and credit programmes on sales, output, employment, wage, productivity, and exports; and Castillo et al.'s (2011) study of the effects of process and innovation support on exporting, employment, wages, and survival in Argentina.

Secondary outcomes

Secondary outcomes vary according to the type of programme, but can be broadly defined as: innovation, exports, investment, and access to credit, formalisation, and management practices.

Programmes that provide access to credit ultimately aim to increase firm resilience and survival (for instance, allowing firms to endure an economic recession) and/or seek to encourage investment. The primary intention of these interventions is thus firm survival and increases in productivity. Similarly, with SME support related to innovation, training, and the value chain, underlying assumptions hold that innovative practices, more skilled workers, and a better coordination will result in higher productivity, employment generation, and access to foreign markets. For instance, Ibarraran et al. (2009) focus on how interventions such as training programmes, access to credit, product innovation, and certification affect the productivity of SMEs in Latin American countries.

3.2.1 Electronic searches

The generalised search strategy covered as comprehensive a set of published and unpublished sources as was feasible within the period allocated. We prioritised electronic searches since regarding the interventions of interest, it was most likely that sources available electronically were reported in the formal literature on SMEs, or in the ‘grey literature’ from national and international organisations.

The first stage of the review involved a search of all published and unpublished studies likely to be relevant to our study objectives. To be included, they had to:

Report on SME support interventions of the kind detailed in the section on interventions;

Given the variety of interventions covered in this research, reference ‘snowballing’ was an effective strategy for beginning our search (Hammerstrøm et al. 2009; cited in Waddington et al., 2012). Reference snowballing consists of using existing reviews, papers, and reports to identify the set of studies to be reviewed. Our search strategy therefore drew on a first set of important studies already identified (see References, section 10). We then proceeded to conduct the electronic search as laid out in the next section.

3.2.1 Electronic searches

Databases:

3ie database of impact evaluations: http://www.3ieimpact.org

EconLit (Ovid)

ABI/INFORM Global (ProQuest)

PAIS International (http://www.csa.com/factsheets/pais-set-c.php)

Sociological Abst

Worldwide Political Science Abst (WPSA)

ASSIA

Web of Science ie ‘Web of Science – Social Sciences Citation Index‘

Business Source Premier (Ebsco)

Academic Search Complete (Ebsco)

Scopus

DAC (OECD) ¹⁹

Google Scholar: http//scholar.google.com

Journals:

Informaworld Taylor & Francis Journals Complete

Ingentaconnect.com (Ingenta)

JSTOR (All Collections)

Periodicals Archive Online (ProQuest)

Royal Society Journals

SAGE Journals Online

ScienceDirect

SpringerLink (MetaPress)

Wiley InterScience

Portals:

World Bank: http://www.worldbank.org/html/extdr/thematic.htm

IDB: www.iadb.org

AFDB: www.afdb.org

ADB: www.adb.org

UNDP: http://www.undp-povertycentre.org/

DFID: http://r4d.dfid.gov.uk/

CIDA: http://www.acdi-cida.gc.ca/reports

Search terms

Table 1 provides the list of basic search terms used to identify studies in the systematic review. Based on these terms, a detailed search strategy was set up to account for US and British English spelling, to seek for the most relevant studies and to restrict the search to LMICs. The details of the search strategies are provided in Appendix A. The search strategy was developed using the Social Science Citation Index (ISI) and Econlit databases, two of the most important databases in economics. These search strategies were adapted for other databases that allow the users to construct detailed strings of search terms that are provided in the appendix. For the 3ie database and Google, we used the search terms provided in Table 1. ²⁰ All searches strategies performed are provided in the appendices.

3.2.2 Searching other resources

Along with database searches, three research assistants undertook manual back searches in bibliographies of studies and journals identified as relevant to the review. ²¹ Given that the search focuses on LMICs, we also contacted experts in the field for recommendations on studies as well as addressing under-researched aspects of the interventions of interest. In addition, we contacted authors to obtain more information pertaining to the interventions they studied. The review covers studies published in English, Spanish, and Portuguese. ²²

3.3.1 Selection of studies

The selection of studies followed the search method described above. The search and selection of studies were done as follows:

Two Principal Investigators (Pis), with support of John Eyers of 3ie, searched all the relevant electronic platforms and downloaded 9,475 papers using a RefWorks account. Additionally, the two PIs snowballed papers and books and downloaded a further 17 papers. After dropping duplicates, the list was reduced to 5,785 papers.

Whenever necessary, the PIs discussed and agreed on which papers to drop based on the detailed ‘filters’ outlined in the protocol.

3.3.2 Data extraction and management

The list of information extracted from the papers is shown in the study protocol (Gonzalez et al. 2014). The papers were tabulated in an Excel sheet and all relevant data were then uploaded to and analysed in Stata.

3.3.3 Assessment of risk of bias in included studies

To assess risk of bias in RCTs and quasi-experimental studies we used the 3ie risk of bias tool. Three researchers contributed to the risk of bias assessment. Two researchers worked on the extraction of the data and decisions on risk of bias, with disagreements resolved by the PI. Appendix B presents the criteria used to check whether the studies addressed risk of bias. To rank the studies we followed the same approach used by Baird et al. (2013) based on Hombrados and Waddington (2012), who divided studies into three groups: Low, Medium, and High risk of bias. The criteria used are simple and consist of answering YES, UNCLEAR, or NO for key questions in five categories that could bias results:

Low Risk of Bias: If ‘YES’ for at least four issues listed under potential sources of bias.

The five categories are as follows:

Selection bias and confounding: This has to do with the identification strategy used in the study. In other words, we checked whether the identification strategy employed in the study convincingly addressed sources of selection bias. This category is classified in each paper as ‘NO’, ‘UNCLEAR’ or YES' depending on the method of analysis as described in Hombrados and Waddington (2012) and Baird et al. (2013).

The results for the risk of bias assessment are provided in Section 4.2.

3.3.4 Measures of treatment effect

The treatment variables test the effect of a particular intervention, such as a component of a more comprehensive programme, the effect of a package composed of multiple components (for instance, matching grants programmes can include subsidised credit for technology adoption or upgrade, and some type of technical assistance) or the effect of one programme against other. For cases testing a particular intervention, the test compares the treatment group against (presumably) a pure control whereas for packages the test was made either against a pure control (effect of the package), or against a control group that were offered access to some components of the package (for instance, package against technical assistance), or similarly, comparisons of two separate interventions.

The effect of the interventions were tested on primary and secondary outcomes.

Primary Outcomes:

Employment creation

Secondary Outcomes

Access to credit

Under ‘firm performance’ we grouped various outcomes such as sales, sales growth profits, production, value added, assets, and total factor productivity. ²⁴ For ‘employment’ we grouped paid workers, new workers, workers recruited, and employment rate. ‘Innovation’ encompasses all types of investments for research and development (R&D), new products, and patents. Our measure of labour productivity grouped studies that reported sales per worker, profit per worker, revenue per worker, and R&D per worker.

To compare effect sizes across studies we used two standardised measures. For binary outcome variables we computed risk ratio (RR), and for continuous variables we used standardised mean differences (SMD). In most of the cases, the standard deviation of the whole sample (pooled standard deviation or ‘pooled_sd‘) was not reported and we therefore made some assumptions in order to compute the SMD and its standard error (SE). For instance, in a couple of studies that reported the effects of different interventions in a long set of intermediary and final outcomes, the descriptive statistics showed the comparison of means between treated and comparison groups, yet only the difference in means and the t-statistic for the difference was noted. The means and standard deviation for each group were not reported. In this case, we made the assumption that the standard deviation is the same in the treatment and control samples and that the covariance of the outcome variable Y between both groups is zero. ²⁵

Although this assumption might be considered plausible in RCTs where the randomisation is at individual level and sample sizes are similar for the treatment and control groups, it is stronger in the context of quasi-experimental studies, particularly where sample size is relatively small and numbers of observations differ sharply between treated and comparison groups. In these cases, we assumed that the standard deviation was the same regardless of the selection process and the sample size in each group.

Whenever studies provided the sample size for the treatment and control groups at the baseline, SMD was computed using the following formulae:

SMD = treatment effect/pooled_sd

For cases where pooled_sd is not available we used the following:

SMD = t * [ ( Nt + Nc ) / Nt * Nc ) ]

For studies that used small samples we corrected SMD using the following correction (see Waddington et al. 2012): ²⁷

SMD corrected = SMD * { 1 - 3 / [ 4 * ( Nt + Nc - 2 ) - 1 ] } .

We computed RR as follows (see Waddington et al. 2012):

RR = [ Mean ( YC ) + β ] / Mean ( YC ) ,

The computation of SE of the effect sizes also requires some assumptions, particularly for RR. As discussed in Waddington et al. (2012), the SE of the error term in the regression model is the preferred option to compute RR (or SMD). In most cases this was not available, thus we used the standard deviation of the outcome among control units at the baseline. We used the following formulae to compute SE(SMD) and SE(RR):

SE ( SMD ) = [ ( Nt + Nc ) / Nt * Nc ) + SMD 2 / 2 * ( Nt + Nc ) ] 1 / 2 SE ( RR ) = σ * { 1 / Nt * [ Mean ( Y C ) + β ] + 1 / Nc * ( Mean ( Y C ) ) } ,

Finally, we made an assumption regarding sample size when this was not provided for each group separately. In cases where only the whole sample was reported, we arbitrarily split the sample equally between treated and control units.

3.3.5 Unit of analysis issues

Most of the studies use data at firm level with the great majority coming from administrative data, such as census data about formal firms or large samples of firms. ²⁸ In one study where the intervention took place at municipal level, authors clustered SE accordingly.

3.3.6 Dealing with dependent effect sizes

For our meta-analyses, the unit of analysis was the study. Nonetheless, several studies performed more than one estimate for the same outcome. For example, in some cases studies report on different interventions, and in others different specifications are tested and therefore there is a need to synthesise several estimates for the same intervention (say, matching grant) and outcomes (say, employment). When a study covered more than one treatment (say, matching grants and technical assistance), and provided estimates for each treatment separately and also for what some studies defined as ‘any programme’ – in this case the treatment dummy is defined as one if a firm is supported by at least one of the two interventions (either matching grants or technical assistance) and zero otherwise (as in Hong Tan, 2011; Lopez-Acevedo et al., 2011) –, we used the latter estimates to compute overall effect size across different interventions. ²⁹

When such ‘synthetic effect’ is not provided, we determined it by taking a simple average of the ES across different interventions per outcome per study (Lipsey and Wilson, 2001). In such cases, the variance of different effect sizes was computed assuming zero covariance because in most cases overlap was limited, that is, firms either participated into a programme or another. ³⁰ Averaging out across standardised ES provided in the same study was necessary to generate one overall ES per outcome per study so we could carry out meta-analysis pooling together different business support programmes.

We estimated synthetic effects in two other cases. First, because the outcome ‘firm performance’ encompasses different measures such as revenue, sales and profits, and in some cases there estimates are provided for each separately in the same study, we had to compute a synthetic effect for those cases as well. Second, some studies reported average effects in different points in time (e.g. short and medium run effects). We computed the synthetic effect for those cases, averaging the effects across time. In both cases, we assumed covariance equals to zero. ³¹

We also performed subgroup analysis looking at some interventions separately. Our review reports on a relatively high number of studies looking at the effect of matching grants on firms' outcomes. In cases where the same study tested the impact of more than one intervention (for example, matching grants and technical assistance), we first averaged the ES for matching grants and technical assistance separately and then took a simple average to obtain an overall ES per outcome per study. As before, this was made to estimate an overall standardized ES across different intervention and again we computed the variance assuming covariance between effect sizes as zero ³² . For interventions covering for at least two studies, standardised ES are reported separately as well as each programme or intervention being analysed- in this case, matching grants and technical assistance.

When sample sizes and treatment effects for subgroups are available, we computed summary effects as a weighted average of the effects sizes. As before, we also computed the variance by assuming covariance between the ES equals zero because this seems to be a plausible assumption for cases where there overlap between subgroups is inexistent or small, that is, where the ES are plausibly independent.

3.3.7 Dealing with missing data

We contacted study authors to ask for missing information, such as descriptive statistics at the baseline (mean, standard deviation and sample size and intra-cluster correlation when it applies), and received quick feedback in most cases. Unfortunately, the quality of data presented varies considerably across studies. In many cases, we had to make assumptions in order to compute SMD, RR, and the SE, for instance ³³ :

When sample size was not provided for the treatment and control groups separately, we arbitrarily split the sample equally;

3.3.8 Assessment of heterogeneity

We reported forest plot and heterogeneity measures, such as the Chi-squared test for heterogeneity (which captures within-study variance), the I-squared statistic, which we interpret as the proportion of total variance across the observed effects explained by between study variance, and τ², an estimate for the variance of the ‘true effect size’ (see Borenstein et al. 2009). ³⁴

We also considered the factors explaining heterogeneity through moderator analysis in the meta-regressions that include intervention design parameters as independent variables. To address the likelihood of limited evidence on intervention design, the review collected data on all final and intermediate outcomes, although it was restricted to studies which reported final outcomes, because this enabled us to better analyse the causal chain.

3.3.9 Assessment of reporting biases

To check for publication bias, we obtain the funnel plots using the metafunnel and metabias commands in Stata as well as Egger's (1997) simple meta-regression test.

3.3.10 Data synthesis

Most of our studies use quasi-experimental methods to estimate the causal effect of a programme. Most estimate the average treatment effect on the treated (ATT), but few estimate the LATE instead. As discussed in Duvendack et al. (2012), there is not a consensus of whether meta-analysis should be performed for quasi-experimental studies. In this review we decided to use meta-analysis to have the ‘big picture’ of the impact of interventions aimed at SMEs. However, in face of the challenges in practice and decisions made, we argue that these results should be treated with care.

After obtaining the effect sizes and their respective SE per outcome per study, we computed forest plots using the Stata command metan. The overall effect was computed assuming a random effects (RE) model. A RE model assumes there might be different ES underlying different studies and interventions, and that the total variance for these should account for between-studies variance (see Borenstein et al. 2009). We also report the confidence interval for each overall estimate and its p-value to assess statistical significance.

3.3.11 Subgroup analysis and investigation of heterogeneity

We provide synthesised ES for three primary outcomes – (1) firm performance; (2) employment; and (3) labour productivity. For four secondary outcomes – (a) exports, (b) investment, (c) innovation, and (d) formalisation rate – we show the forest plots with individual estimates since we did not systematised review studies looking specifically at those outcomes. This analysis is complemented with meta-regressions (metareg command in Stata) controlling for some moderating factors, such as region fixed effects, firm size, and risk of bias. ³⁵ These moderator variables were identified in the study protocol (Gonzalez et al. 2014). We decided to present forest plots only for outcomes that had at least four ES. For outcomes with two or three observations we present random effects estimates using bivariate meta-regression only.

3.3.12 Sensitivity analysis

Given the relatively small number of studies that looked at the impact of the same (or similar) intervention on the same outcomes and the low number of studies with low risk of bias, we conducted the sensitivity analysis dropping studies that stand out visually as clear outliers and, whenever possible, looking at the effects of interventions separately. In the meta-regression analysis we were able to explore moderator factors, including risk of bias and study design, more successfully and provide estimates for individual interventions. ³⁶

4.1.1 Results of the search

The initial search retrieved 9,475 studies. After dropping duplicates, the list dropped to 5,785 papers. The systematic review approach used detailed search codes to retrieve papers analysing the effect of SME support programmes from the following platforms: ISI, ECONLIT, ABI, PROQUEST and SCOPUS. In addition to searching online platforms, the two Pis snowballed key papers and books and added other 17 studies to the list. Although this review covers only studies that used experimental or quasi-experimental methods, our search strategy did not filter them according to the methods used.

The final list of studies from searching online platform was therefore examined with all filters outlined in the review protocol, which assessed the impact of an SME intervention using rigorous evaluation methods. With that in mind, three research assistants double-screened abstracts of 5,785 studies. A preliminary final list had 63 studies. It was noted that the great majority either did not use quantitative methods to assess the impact of an intervention, did not use a rigorous method to address selection problems, or looked at interventions targeting micro-entrepreneurs (21 cases). The PIs decided to exclude six studies that looked exclusively at intermediate outcomes – such as formalisation rate and number of new firms – and different versions of the same study and unpublished versions of published studies.

In the end, the team came up with a list of 40 studies (23 from the search in the online platforms and 17 from snowballing). Figure 2 illustrates this procedure. For the meta analysis we had to exclude four studies because we were unable to compute a standardised effect size and/or its standard error. The empirical analysis therefore included 36 studies and 72 ES per intervention-outcome combination. The large number of ES is due to the fact that a few studies tested the impact of several interventions together and then separately on the same outcomes, and some randomised controlled trials tested the effect of more than one treatment arm.

OPEN IN VIEWER

2

- PRISMA flow diagram showing study selection

4.1.2 Included studies

This review investigates the impact of a diverse array of SME support, as discussed in Section 1.3. The types of support include: matching grants/credit, innovation support, support to exports, tax simplification, training, and local production systems. Most of the papers included in this review measured the impact of a SME support intervention by more than one outcome at firm and employee levels (Figure 3). This section presents a brief analysis of each paper included in this review to provide qualitative discussion of specific results by each type of intervention.

OPEN IN VIEWER

3

– Percentage of Reported Outcomes

According to Figure 3, five outcomes stand out: firm performance (27.8 per cent of the ES), employment (20.1 per cent of the ES), exports (15.3 per cent of ES), labour productivity (11.1 per cent of the ES), and investment and innovation (8.3 per cent of the ES each). The firm performance outcome groups the following individual variables: sales, sales growth profits, production, value added, assets, and total factor productivity. Because few studies report on the same type of outcome (e.g. profits) we took the decision to group these outcomes, which arguably measure similar constructs, together to maximise statistical power. ³⁹

Figure 4 shows the cumulative number of studies produced between 2003 and 2014.

OPEN IN VIEWER

4

– Cumulative number of studies per year

Between 2003 and 2010 there were only 16 studies using experimental or quasi-experimental techniques to assess the impact of different business support to SMEs. Between 2011 and 2014 that number more than doubled. As noted in Figure 5, the evidence comes from 18 countries, most of which are in the Latin American region and five are in African countries.

OPEN IN VIEWER

5

– Number of studies per country

Table 2 summarises the findings for each study (which are presented in detail in Appendix D). Most of studies use quasi-experimental methods and seven studies use experimental design (Atkin et al., 2014; Bruhn et al., 2012; de Giorgi and Rahman, 2013; Karlan et al., 2014; de Mel et al., 2012; McKenzie and Sakho, 2007), including one which was excluded from the meta-analysis because we were unable to calculate the effect size (Mano et al., 2012). The most commonly evaluated intervention category was matching grants (8 studies) and export promotion (8 studies), followed by innovation programmes (7 studies), tax simplification (6 studies) and training interventions (6 studies). Some of the less researched interventions include access to credit (4 studies), local productive systems (3 studies) and formalisation (3 studies). Two studies report on clusters of interventions. Fifteen studies focused on the manufacturing sector, while thirteen included all sectors and the remaining twelve focused on other sectors (agriculture, construction, textile, tailoring) or a combination of sectors. The studies display a large range of sample sizes; as low as 167 total observations from a managerial training programme in Ghana (Mano et al., 2012), to over 1.6 million observations from data assessing business registration regulations in Mexico (Bruhn, 2011).

4.1.3 Excluded studies

The papers selected from those retrieved by the search codes were carefully screened based on their abstracts and selected to be included in the systematic review. The full revision of these selected papers deemed 21 studies ineligible as they looked at interventions targeting microentreprises, which are not included in our review, for example: De Mel et al. (2009, 2010, 2012, 2013a, 2013b), Fafchamps et al. (2011), Valdivia (2011) and Stewart et al. (2012). The review excluded studies that looked at the impact of an intervention only on intermediary outcomes (such as formalisation rate): Monteiro and Assuncao (JDE, 2012) and Andrade, Bruhn and McKenzie (2013). Studies that looked at impact of programmes that we did not consider a public intervention targeted exclusively to SMEs were dropped (Bah et al., 2011). Studies that looked at the impact of export zones, such as Cirera et al. (2011) and Cirera et al. (2013), were dropped. Finally, studies (RCTs) that did not clearly test a public policy and that was conducted with rural firms only such as Gine and Mansuri (2011) were not included in the review.

4.2.1 Results of the risk of bias assessment

The assessment of the risk of bias is important to identify issues that might influence the estimated coefficient of studies and thus might have an impact on the results of this systematic review. This report uses the risk of bias tool, based on Hombrados and Waddington (2012), as described in section 3.3 to rank the studies and check whether they addressed the risk of bias. Additionally, we followed the strategy used by Baird et al. (2013) and provide an additional aggregated classification of risk of bias.

Table 3 presents the summary of aggregated results from the risk of bias assessment. The risk of bias results for each paper is presented in Appendix C.

Selection bias and confounders: Only 2 out of the 40 reports (5.0 per cent) completely address this issue. This is partly due to the fact that for some categories of quasi-experimental design (PSM, OLS, DID) the best possible ranking is “unclear” for selection bias and confounders, and most of the papers' approaches correspond to these methodologies.

Following Baird et al. (2013), using the above categories, we categorise the reports as low, medium or high risk of bias in Part B of Table 3. Only five per cent of the reports (2 studies) are categorized as low risk, 33 per cent (13 studies) as medium risk and 65 per cent (25 studies) as high risk. Since most of the reports presented quasi-experimental designs, it was especially challenging to find those that discuss all relevant features of the approach. This was especially true for the PSM methods, for which the most challenging requirement was the Rosenbaum test for hidden bias (which was not presented by any of the papers), followed by the lack of a test for equality in means of covariates between treatment and control groups after matching.

The overall results indicate that there is a huge heterogeneity in the potential for bias but most papers are classified as medium risk of bias. This result is hugely influenced by the assessment of the spill-overs, cross-overs and contamination category of the risk of bias tool. From the 40 reviewed, given the characteristics of SME support, most studies were unable to ensure that there is no spill-over or contamination of the treatment. As all SMEs are part of the whole economy in a particular region, general equilibrium effects are likely. The individual firm-level treatment is likely to produce spill-overs within the economy which are not controlled for.

4.3.1 Quantitative synthesis40

This section discusses the meta-analysis and meta-regression estimates. Forest plots are provided for interventions investigated in at least 4 studies. We complement this analysis discussing meta-regression estimates for individual interventions. ⁴¹ Because the business support interventions analysed in this review were envisaged to improve firms' indicators, positive average effect sizes therefore represents positive effects. Thus, average overall ES that lies on the right hand side of a zero solid line in the forest plots indicates positive effect on both primary and secondary outcomes.

Primary Outcomes

1. Firm performance

We found that several studies looked at a myriad of outcomes related to firm performance such as profits, revenues, sales, assets, and so on. We thus grouped them under an outcome named ‘firm performance’ to be able to say something about the impact of different interventions on firms.

Our review found 20 ES related to firm performance (see Figure 6 below) across different interventions. Although the interventions may consider different group of firms (e.g. sector and size) and aim to tackle different market failures, we believe that providing an overall picture of the interventions covered in the review can still be relevant for high-level policy making. ⁴² Figure 6 reports the standardised ES (SMD) of each study and the overall average across interventions.

OPEN IN VIEWER

6

– Forest Plot – All interventions: Firm Performance

On average, interventions aimed at improving firm performance had a positive effect of 0.15 standard deviations. The effect is statistically significant at 1 per cent (p-value = 0.000) with a 95 per cent confidence interval (95% CI) of (0.08, 0.22). It is worth noting that most of the estimates (10 out of 20) come from interventions that took place in Latin American countries. Five estimates are from African countries. Also interesting is the relatively small heterogeneity between studies. As indicated by the homogeneity test statistics (I-squared = 92.8%, tau-squared = 0.0196) there is an indication of high heterogeneity across studies. This measure captures the degree of inconsistency in the studies' results (Higgins et al. 2003).

Since our review included 13 studies that examined the impact of matching grants programmes and nine that investigated the impact of export promotion programmes, our data allows us to look at the effect of these two interventions on firms' performance in isolation. Figure 7 shows that the effect of MG on firm performance is similar but not significant in statistical terms (SMD = 0.13, 95% CI of (-0.04, 0.30). The assessment of homogeneity suggest a large degree of heterogeneity across studies (I-squared = 96.5%, tau-squared = 0.064). However, as discussed below, the effect becomes identical to that obtained with all interventions pooled together once we drop one outlier study from the analysis. For support to exports programmes, we found zero effect on firm performance with the 95 per cent CI of (-0.08, 0.09) as shown in Figure 8. The assessment of homogeneity suggests that there is no between-study heterogeneity (I-squared = 0.0%, tau-squared = 0.000).

OPEN IN VIEWER

7

– Forest Plot – Matching Grants: Firm Performance

OPEN IN VIEWER

8

– Forest Plot –Support to export programmes: Firm Performance

The impact of MG on firm performance is interesting and could have at least two possible interpretations. First, it could be argued that business support of any sort works as subsidies (‘free money’) that end up favouring firms that would actually be able to carry on without any injection of public resources, i.e. a picking the winners argument. On the other hand, one could take this result as an indication that SME interventions of any sort are key to SMEs needing a ‘nudge’ to increase performance (or survive). In order to shed light on these two competing views, in the section below we look at the effect of MG on secondary outcomes, such as investment. In the meta-regression analysis we also approach this issue indirectly by looking at whether firm size influences the result.

As mentioned in section 5.4, some studies were not included in the meta-analysis as we were unable to compute either the standardised effect sizes or the adjusted standard errors. Despite the fact that standardised effect sizes or the adjusted standard errors could not be calculated, these studies also provided results on the impact of SME support programmes on firm performance and indicated the same effect of SME support programmes on firm performance as suggested in Figure 6. Mano et al. (2012) studies the impact of business consulting in the form of basic managerial training by doing an RCT in Suame Magazine, an industrial area consisting of metal workshops and enterprises in Kumasi, the second largest city in Ghana. The data collected comprised 167 firms, 60 in the control group (of which 53 were randomly selected; the other seven had been promised a place in the programme) between November 2007 and November 2008. The study collected data related to outcomes such as sales revenue, value added and gross profit. The results suggest that participation in the programme improves gross profit and value added of the firms that participated in the experiment. Another study not included in the meta-analysis and provide results on firm performance is Benavente et al. (2007). They analyse the effectiveness of the Chilean Technology Development Fund (TDF), the FONTEC programme. The authors adopt difference-in-differences and results suggest that the programme found a positive impact on sales.

2. Employment

The meta-analysis for employment outcomes included 15 effect sizes (see Figure 9 below). Although most of the evidence comes from Latin America, the figure suggests that different types of business support for SMEs help create jobs in almost all the countries considered. On average, programmes targeted at SMEs tend to help with employment creation. The overall effect is equal to 0.15 standard deviations (average SMD = 0.15). The effect is significant at 6 per cent (p-value = 0.057) with 95 per cent CI of (-0.00, 0.30). The values of I-squared statistic (99.2%) and tau-squared (0.081), though, indicates a high estimated between-study variability. This result is consistent with the common-sense view that SMEs may be an important source for job creation but the study also highlights that there is considerable variation in the effectiveness of different SME-support programmes on employment generation.

OPEN IN VIEWER

9

– Forest Plot – All interventions: Employment Creation

OPEN IN VIEWER

10

– Forest Plot – Matching grants: Employment Creation

Some of the studies that were not included in the meta-analysis because we were unable to compute either the standardised effect sizes or the adjusted standard errors present results on employment. Benavente et al. (2007) that uses difference-in-differences to analyse the FONTEC programme found a positive impact on employment. Corseuil and de Moura (2011) uses regression discontinuity design to assess the effect of the introduction of the SIMPLES legislation on manufacturing employment generation and the results show that SIMPLES has a positive impact on the creation of new manufacturing jobs in Brazil. Similarly, Kalume et al. (2013) evaluate the impact of Super Simples Nacional using the difference-in-difference estimator, the results indicate that the programme contributed to the definitive restart of activities for the inactive ones or the opening of new firms, thus generating jobs.

3. Labour productivity

The meta-analysis for labour productivity includes eight effect sizes. The evidence comes almost exclusively from countries in Latin America (see Figure 11). The overall effect size is 0.04, but it is statistically insignificant (p-value = 0.36) with a CI of (-0.05, 0.13). The assessment of homogeneity indicates a large degree of between-study variability (I-squared statistic = 88.7%, tau-squared = 0.0117), indicating that the pooled effect estimate needs to be interpreted with caution. The meta-analysis includes one study with a negative statistically significant effect, two studies with statistically insignificant effects and 5 studies with positive statistically significant effects indicating the potential for business support services to be both successful and to have potentially adverse effects on labour productivity. When we look at the effect of matching grants only we find a small negative effect that is not statistically different from zero (-0.02 SD, 95% CI = -0.15, 0.10) – see Figure 12. Again, the assessment of homogeneity indicates a large degree of between-study variability (I-squared = 94.1%, tau-squared = 0.02).

OPEN IN VIEWER

11

– Forest Plot – All interventions: Labour Productivity

OPEN IN VIEWER

12

– Forest Plot – Matching grants: Labour Productivity

Secondary Outcomes

I. Exports

Figure 13 shows the distribution of SMDs of interventions that, among other things, aimed to help firms access external markets (exports). These interventions include export promotion programmes as well as matching grants that were envisaged to help firms access external markets. Most of the studies show a small and statistically insignificant effect, ranging from SMD = 0.02 (95% CI = 0.00, 0.04) to SMD = 0.037 (95% CI = -0.15, 0.89), with an outlier evaluation of a programme in Chile reporting an SMD of 4.4 (95% CI = 4.3, 4.4). Figure 14 shows that the effects of programmes conceived with the purpose to spur exports. Again, there are some positive but very small non-statistically significant effects on exports, ranging from 0.02 (95% CI = 0.00, 0.04) to 0.037 (95% CI - -0.015, 0.89).

OPEN IN VIEWER

13

– Forest Plot – All interventions: Exports

OPEN IN VIEWER

14

– Forest Plot – Support to export programmes: Exports

II. Innovation

Figure 15 shows the forest plot for innovation supports. The review found six ES for interventions aimed at helping SMEs to innovate. The effect sizes range from SMD = 0.00 (95% CI = -0.02, 0.02) to SMD = 0.45 (95% CI = 0.16, 0.75). Most of the studies find very small effects and those that found positive effects are imprecisely estimated. This result may go against a prevalent view that argues that SMEs do not innovate. It is also important to bear in mind that we are pooling together different programmes envisaged as helping SMEs to expand their production frontier through innovation. Thus, one should read this result carefully. This is especially important given that the overall estimates synthesise studies that use different definitions and measurements of innovation, different firm sizes, and study different country/institutional contexts.

OPEN IN VIEWER

15

– Forest Plot – All interventions: Innovation

When attention is turned to MG interventions only, Figure 16 shows a similar pattern, that is, no effect on innovation across most included studies, with effect sizes ranging from SMD = 0.00 (95% CI: -0.02, 0.02) to SMD = 0.11 (95% CI: -0.11, 0.35).

OPEN IN VIEWER

16

– Forest Plot – Matching grants: Innovation

The study of Benavente et al. (2007), not included in the meta-analysis because we were unable to compute either the standardised effect sizes or the adjusted standard errors present results on employment, evaluated the Chilean Technology Development Fund (TDF), the FONTEC programme. It suggests that that FONTEC's subsides promote technological upgrades and process innovations, rather than radical product innovations.

III. Investment

The average effects of business support on firms' investment are shown in Figure 17. Again, most of the effects are small and not statistically significant, while two studies showing positive and statistically significant effects for innovation programmes in Mexico (SMD = 0.22, 95% CI = 0.14, 0.29) and Vietnam (SMD = 0.23, 95% CI = 0.20, 0.25).

OPEN IN VIEWER

17

– Forest Plot – All interventions: Investment

Figure 18 shows the forest plot for MG only. Two studies have a positive but not statistically significant effect and one study has a positive statistically effect with SMD = 0.23 (95% CI = 0.20, 0.25).

OPEN IN VIEWER

18

– Forest Plot – Matching grants: Investment

4.3.2 Sensitivity Analysis

This section first reports the effects for primary outcomes dropping studies that stand out as clear outliers in the forest plots based on a pre-determined definition discussed above (see footnote 35), then provides meta-regression with the following moderator variables: a dummy variable identifying Latin American countries (LAC), a dummy variable identifying African countries (Africa), a continuous variable that inform the size of a firm in terms of number of employees, a dummy variable for moderate or high risk of bias (RoB), a binary indicator for the method used (1 if RCT and *** if quasi-experimental - QE), and the secondary (intermediary) outcomes – investment, innovation and exports.

Forest Plots

A. Primary Outcomes

Figures 19 to 21 show the forest plots for primary outcomes firms' performance, employment and labour productivity respectively. Dropping the study by Duque and Muñoz (2011) reduces the magnitude of the overall effect size on firms' performance to 0.13 SD. The 95 per cent CI of (0.06, 0.20) remains almost the same. Excluding the outlier improves I-squared statistics only slightly (from 92.8% to 92.1%).

OPEN IN VIEWER

19

– Forest Plot – All interventions: Firm Performance – Dropping outliers

Figure 20 shows that the overall effect of business support on employment after the exclusion of Duque and Mufioz (2011). The average effect size is 0.15 SD (with 95% CI of 0.08, 0.22). The result is now highly statistically significant (p-value = 0.000). With the exclusion of the outliers there is also a gain in terms of consistency between studies' findings. Despite still being relatively high, the I-squared statistic drops from 99.1 per cent to 92.8 per cent. The Tau-squared statistic also reduces sharply to 0.013 (compared to 0.081).

OPEN IN VIEWER

20

– Forest Plot – All interventions: Employment Creation – Dropping outliers

Figure 21 shows an overall standardised effect size of 0.11 with a 95 per cent CI of (0.08 and 0.15) for labour productivity once the study of Duque and Mufioz (2011) is excluded. The difference is huge compared with the previous result showed in Figure 11. It is worth noting the gain in precision due to the fall in between studies variance (Tau-squared statistic of 0.0006, I-squared of 31.3%).

OPEN IN VIEWER

21

– Forest Plot – All interventions: Labour Productivity - Dropping outliers

Figure 22 shows that excluding the outlier studies – Duque and Munoz (2011) and Hong Tan (2011) – results in a positive and statistically significant (p-value = 0.000) effect of MG on firms' performance. The standardised average effect is 0.15 (95% CI = 0.08, 0.22). The heterogeneity is remains moderate with the I-squared statistic of 52.8 per cent and the Tau-squared statistic close to zero (0.004).

OPEN IN VIEWER

22

– Forest Plot – Matching grants: Firms' Performance – Dropping outliers

Figures 23 and 24 summarise the effect of MG on employment and labour productivity respectively. With exclusion of the outlier (Duque and Muñoz, 2011) the overall impact of MG on employment becomes positive 0.14 SD with a 95 per cent CI of (0.03, 0.24) – and statistically significant at 1 per cent (p-value of 0.01). The I-square (93.8%) and Tau-squared (0.018) statistics indicate that removing outliers does not result in a significant reduction in studies' heterogeneity.

OPEN IN VIEWER

23

– Forest Plot – Matching grants: Job Creation – Dropping outliers

Figure 24 shows that the effect of MG on labour productivity remains indistinguishable from zero following exclusion of the outlier (Duque and Mufioz, 2011). The overall average standardised effect is now positive (0.05 of a SD, 95% CI: -0.05, 0.15) though not statistically significant (p-value = 0.31). There is a very slight gain in terms of consistency across studies' findings though a large degree of between-study heterogeneity remains. The I-squared statistic is 90.7 per cent compared to 94.1 per cent in Figure 12.

OPEN IN VIEWER

24

– Forest Plot – Matching grants: Labour Productivity – Dropping outliers

Meta-Regression

The analysis here concentrates on cases where an outcome has at least two reports. Where few ES per outcome (less than four) are available we were unable to control for moderator variables. Thus, only random effect estimates are shown. All the analyses below are conducted after excluding outliers.

A. Primary Outcomes

Table 4 shows the coefficients for meta-regression. The first row shows the random effects estimate without controlling for any moderator factor. The coefficients are identical to those reported in the forest plot once outliers are excluded. The first row shows the RE estimate without controlling for any moderator factor. These estimates correspond to the overall mean effect as showed in the forest plots. We then estimate meta-regression controlling for each moderator factor in separated regressions. We had to estimate each regression one-by-one due to insufficient sample size. We report the coefficient for the constant (RE when the dummy variable takes the value of zero) and the coefficient of the moderator variable in all cases. To indicate whether the coefficient is statistically significant we used p-values.

OPEN IN VIEWER

Table 4

Meta-Regression for Primary Outcomes (excluding outliers)

	Firms Performance	Employment Creation	Labour Productivity
RE estimate – no controls	0.13***	0.15***	0.11***
p-value	0.000	0.001	0.001
N	19	13	7
Moderator variables (Control variables)
Constant	0.10**	0.19***	0.14**
p-value	0.036	0.01	0.014
LAC fixed effect (1 if LAC; 0 otherwise)	0.057	-0.06	-0.03
p-value	0.35	0.43	0.48
N	19	13	7
Constant	0.15***	0.15***	Na
p-value	0.000	0.002
Africa fixed effect (1 if Africa; 0 otherwise)	-0.10	-0.03	Na
p-value	0.18	0.82
N	19	13
Constant	0.16***	0.21***	0.13
p-value	0.000	0.004	0.11
Firm size (continuous variable)	-0.001*	-0.001*	-0.0003
p-value	0.06	0.15	0.70
N	19	13	7
Constant	0.09**	0.07	0.11**
p-value	0.047	0.116	0.027
Risk of bias (1 for moderate or high RoB; 0 for low RoB)	0.09**	0.07	0.11**
p-value	0.047	0.116	0.027
N	19	13	7
Constant	0.14***	0.16***	Na
p-value	0.000	0.002
Method (1 if RCTs; 0 if QE)	0.14***	0.16***	Na
p-value	0.000	0.002
N	19	13

Note: ***, **, * Statistically significant at 1, 5 and 10 percent respectively.

Given the small sample of studies, these estimates are underpowered. The lack of statistically significance should not mean that these factors are unimportant. The magnitude of the effect size and its sign can be informative in such context.

First, the coefficient of the dummy variable for LAC is positive but statistically insignificant. The estimate indicates that business support services implemented in LAC is associated, on average, with higher effects on firm performance. However, for the other two outcomes we observe the opposite, that business support services implemented in LAC are associated, on average, with lower effects on employment creation and labour productivity, by 0.06 of a SD and to 0.03 of a SD respectively. As before, the estimates are not significant in statistical terms. We have insufficient data to explore this issue further, but it could be that business support to SMEs in LAC are more capital intensive and therefore less likely to create jobs.

The estimate for the ‘Africa’ dummy indicates that SME support programmes in Africa are associated with a lower pooled effect on firm performance, but is only marginally associated with lower effect on employment creation. The differences between estimates on firm performance in LAC and Africa regions could be suggesting that, on average, business support to SMEs is more labour intensive in African countries. One cannot be assertive, but this could be reflecting differences in skills of the work force in both regions.

The size of firms may play a role in the main findings. As can be seen in the table, the random effects estimate increases in all three cases once we control for firm size, suggesting that larger firms are associated with larger impacts. The relationship might not be linear though. ⁴³ Figure 25 shows the histogram for this variable.

OPEN IN VIEWER

25

– Histogram for Average Firm Size

The figure highlights that most of the firms assessed in the studies covered by this review have fewer than 100 employees. A high percentage (25%) has no more than 10 employees (first bar). For studies covering African countries, the median size of firms is 93 and the mean is 83. This indicates that there is a larger proportion of small firms studied in Africa given the left-skewed distribution.

Table 4 shows the random effects estimates once risk of bias is controlled for. Because the dummy risk of bias takes the value of 1 for studies with a high risk of bias, the significant reduction in the magnitude of the effects indicates that high-risk studies tend to show more positive results on firms' performance than studies with low or moderate level of bias. The same holds for employment creation, but not for labour productivity. In fact, once a dummy for risk of bias is added to the model, the effect on employment turns statistically insignificant. One could interpret these results as a signal that the most rigorous studies have not found effects of business interventions on these firms' performance and employment creation, and therefore with so few good studies out there any conclusion regarding the effect of such interventions is still premature.

Finally, the coefficient of the dummy variable that informs the method used (one for RCT and zero for quasi-experimental methods), suggests that the RCTs included in this review were less likely to find positive effects on firms' performance and employment creation. We believe that this might be in part due to the scales of the programmes evaluated. Studies using quasi-experimental methods usually rely on administrative datasets with thousands of observations whereas RCTs might test programmes in their pilot stages.

Table 5 replicates the exercise only for MG interventions.

OPEN IN VIEWER

Table 5

Meta-Regression for Primary Outcomes Matching Grants (Exclude Outliers)

	Firms Performance	Employment Creation	Labour Productivity
RE estimate – no controls	0.15**	0.13*	0.052
p-value	0.012	0.083	0.33
N	7	7	5
Moderator variables (Control variables)
Constant	0.11*	0.13	0.14
p-value	0.095	0.305	0.244
LAC fixed effect (1 if LAC; 0 otherwise)	0.10	0.13	0.14
p-value	0.40	0.305	0.244
N	7	7	5
Constant	0.17**	0.17**	Na
p-value	0.000	0.029	Na
Africa fixed effect (1 if Africa; 0 otherwise)	-0.27**	0.17**	Na
p-value	0.03	0.029	Na
N	7	7	Na
Constant	0.17*	0.27*	0.24
p-value	0.084	0.053	0.113
Firm size (continuous variable)	-0.001	0.27*	0.24
p-value	0.37	0.053	0.113
N	7	7	5
Constant	0.15	0.015	0.068
p-value	0.131	0.33	0.501
Risk of bias (1 for moderate and high risk of bias; 0 for low)	-0.01	0.015	0.068
p-value	0.94	0.33	0.501
N	7	7	5
Constant	0.16***	0.20**	Na
p-value	0.002	0.018	Na
Method (1 if RCTs; 0 if QE)	-0.23	0.20**	Na
p-value	0.27	0.018	Na
N	7	7	Na
Constant	0.15**	0.16*	0.10*
p-value	0.012	0.074	0.047
Export (continuous variable)	2.23**	2.86	-2.85**
p-value	0.02	0.11	0.012
N	7	7	5
Constant	0.06	0.13	0.06
p-value	0.48	0.16	0.37
Innovation (continuous variable)	6.32	8.23	-1.85
p-value	0.15	0.23	0.59
N	7	7	5
Constant	0.08	0.17**	0.025
p-value	0.36	0.027	0.67
Investment (continuous variable)	-0.92	-2.99	8.00
p-value	0.35	0.01	0.52
N	7	7	5

Note: ***, **, * Statistically significant at l, 5 and 10 per cent respectively.

The results for firm performance are qualitatively similar to those presented in Table 4, but few estimates stand out interestingly. First, the coefficient of the dummy ‘Africa’ is large and negative in the first column, suggesting that MG programmes in Africa is associated with worse performance of firms.

On the other hand, the coefficient for Africa region is positive and relatively large for employment creation. This suggests that MG in African countries were more likely to create jobs. This is consistent with the hypothesis that African firms' production function may be more labour intensive (than LAC, for instance), and that they likely work at relatively low scale hence the scope to grow through addition of labour inputs.

As expected, the coefficient for size of firms is positive and large. This might be picking a mechanical effect since firms' size is measured as number of employees. This would explain the relatively large effect on labour productivity as well.

MG programmes that aimed at improving firms' capacity to export and innovate showed positive effects on firms' performance and employment creation, but negative on labour productivity. This result is a bit puzzling and we interpret it as an indication that firms targeted by the type of interventions covered in this review were likely facing some constraint to increase output beyond the variable cost associated with extra hired labour. This could also reflect some distortion in case an intervention somehow incentivised firms to create jobs (e.g. unpaid jobs through employment of family members) through different forms of subsidies (e.g. wage subsidy).

Finally, the coefficient for the variable ‘investment’ was negative for employment creation. Our interpretation is that the investment made by these firms was toward addition of capital goods.

In a nutshell, these findings suggest that matching grants serve different firm composition and business purposes. Export-oriented firms for example need to become more efficient to be able to compete in the external market while labour intensive firms may use matching grants to hire extra labour.

B. Individual Interventions

Table 6 shows random effects estimates for individual interventions. The table reports the coefficient, t-statistic, p-value and number of studies (reports) for each primary outcome. As can be seen, when we look at interventions individually we can see how little we still know about the impact of each of these policies. In many cases there are only two reports per outcome.

Since the sample size is small in all cases, the estimates lack power. So, as before, we concentrate on the magnitude of the effect sizes that are statistically significant. The overall picture suggests that most interventions may affect outcomes positively. Disregarding issues such as risk of bias, the first column suggests that tax simplification and matching grants programmes seem to be the most significantly effective to improve firms' performance indicators and to create jobs. In contrast, technical assistance does appear to lead to big effects for firm performance, employment and labour productivity in magnitude although never statistically significantly (probably due to the small number of studies which have assessed these programmes).

OPEN IN VIEWER

Table 6

Meta-Regressionfor Individual Interventions

	Firm Performance	Employment	Labour Productivity
Technical assistance	0.27	0.14	0.12
p-value	0.3	0.19	0.49
# of studies	2	2	2
Training	0.08	0.07
p-value	0.43	0.51
# of studies	3	2
Cluster	0.09	0.04	0.06
p-value	0.28	0.42	0.48
# of studies	2	2	2
Support to Export	-0.004
p-value	0.93
# of studies	4
Innovation	0.023	-0.004	-0.04
p-value	0.225	0.91	0.55
# of studies	2	2	2
Tax Simplification	0.28**	0.18
p-value	0.047	0.37
# of studies	3	2
Matching Grants	0.15**	0.13*	0.052
p-value	0.012	0.083	0.33
# of studies	9	9	6

Note: ***, **, * Statistically significant at l, 5 and 10 per cent respectively.

4.3.3 Publication bias

This section uses funnel plots and Egger's tests to check whether there is any indication of publication bias. Figure 26 (below) plots the effect size (SMD) on the horizontal axis and the standard error of the effect size (SE SMD) on the vertical axis. The solid line crosses the horizontal axis at the overall average fixed effect estimate. Although most of the dots (studies) are spread around the solid line and within the triangle area (95% CI), there are quite a few cases of studies on the right side of the triangle area, which are not symmetrically represented on the left side.

OPEN IN VIEWER

26

– Funnel Plot for Firm Performance

These studies report positive effects and seem to have mixed level of precision. We also performed Egger's test for publication bias using the metabias command in Stata. The first column in table 7 shows the results for the outcome ‘firms' performance‘. The coefficient of the variable bias is positive but only statistically significant at 11 per cent (p-value = 0.104). According to our interpretation, the funnel plot and Egger's test might indicate some publication bias towards studies showing positive effects of business support on SMEs performance indicators.

The funnel plot for employment outcome is shown in Figure 27.

OPEN IN VIEWER

27

– Funnel Plot for Employment Generation

Most of the dots are scattered on the top and outside the 95 per cent CI. The solid line crosses the horizontal axis at the fixed effect estimate. Note how different the fixed effect estimate is when compared with the random effects estimate reported in the forest plots. Egger's test is shown in the second column of Table 8. As can be seen, there is an indication of publication bias towards positive results. The coefficient of the variable bias is positive (7.14) and statistically significant at 9 per cent (p-value = 0.084) for employment creation.

Figure 28 shows the funnel plot for labour productivity. The figure shows most of the dots concentrated on the top, on the positive quadrant and within the 95 CI interval. The Egger's test in the third column of Table 8 shows that the coefficient for the variable bias is negative and statistically insignificant. We observe a very similar pattern for MG programmes as is shown in Table 8.

OPEN IN VIEWER

28

– Funnel Plot for Labour Productivity

It is worth mentioning that this conclusion could be affected by the four studies that could not be included in these empirical tests. These conclusions would be reinforced by the results of the excluded studies as three of them – Benavente et al. (2007), Mano et al. (2012) and Corseuil and de Moura (2011) – found positive effects on jobs creation, two – Benavente et al. (2007) and Mano et al. (2012) – found positive effect on firms' performance, and one – Benavente et al. (2007) – also found positive effects on innovation and exports ⁴⁴ . We therefore interpret these findings as not providing evidence for publication bias for firms' performance and labour productivity outcomes, but providing evidence of possible bias for employment creation outcomes.

Figures 29 to 31 present the funnel plots for the same outcomes but only for MG interventions whereas Egger's test is showed in Table 9. The findings with respect to possible bias have the same interpretation as the findings for interventions overall: findings provide evidence of publication bias for employment creation outcomes but we are not able to conclude there is evidence for publication bias for firms' performance and labour productivity outcomes.

OPEN IN VIEWER

29

– Funnel Plot for Matching Grants: Firm Productivity

OPEN IN VIEWER

30

– Funnel Plot for Matching Grants: Employment Generation

OPEN IN VIEWER

31

– Funnel Plot for Matching Grants: Labour Productivity