PROTOCOL: Effectiveness of road safety interventions: An evidence and gap map

1.1 The problem

The World Health Organization (WHO) released its World Report on Road Traffic Injury Prevention in 2004 (Peden et al., 2004). This report focused on road traffic injuries (RTI) and fatalities as a worldwide health problem and included a summary of the known risk factors associated with road traffic crashes and possible countermeasures that should be put in place to control the problem. It also pointed out that without new or improved interventions, RTI will be the third leading cause of death by the year 2020. The publication of this report spurred some national and international agencies and civil society groups to give more attention to the problem of road safety and a number of resolutions have been passed by the United Nations General Assembly, World Health Assembly and the Executive Board of the WHO (WHO, 2004, 2009, 2011, 2016).

As a follow up, two Global Ministerial Conferences on Road Safety have been held in Moscow, Russia (19–20 November 2009), and Brasilia, Brazil (18–19 November 2015). And the third one is scheduled in February 2010 in Stockholm, Sweden. At the close of the conference in Brasilia, the 2,200 delegates adopted the Brasilia Declaration on Road Safety through which they agreed ways to halve road traffic deaths by the end of this decade—a key milestone within the new Sustainable Development Goal (SDG) target 3.6 (Brasilia Declaration, 2015). This target has not been achieved and the total number of RTI and fatalities are still increasing in most countries of the world.

The WHO has also released four Global Status Reports on Road Safety in 2009, 2013, 2015 and 2018. The final one being the Global Status Report on Road Safety 2018 (WHO, 2018). These reports give a broad assessment of the status of road safety in ∼180 countries. The data were obtained from national governments using a standardized survey form. The GSRRS18 shows that the overall global road traffic fatality rate is 17.4 per 100,000, but there is a great disparity by income and regions. Low- and middle-income countries (LMIC) are reported to have the highest annual road traffic fatality rates, at 24.1 per 100 000, while the rate in high-income countries (HIC) is lowest at 9.2 per 100,000, and that over half of those who die in road traffic crashes are pedestrians, bicyclists and users of motorized two-wheelers (MTW).

In LMIC traffic injuries have been steadily rising, and now rank in the top 10 causes of death (WHO, 2018). Over 50% of the victims are pedestrians, bicyclists and motorcyclists and RTI has become the leading cause of death for young adults in most countries. While most HIC have well-established road safety policies, many LMICs are in the process of establishing national regulatory agencies and sustainable funding streams to support large scale interventions that systematically address risky behaviours and the safety characteristics of vehicle and road infrastructure.

LMIC are also growing their rural road and highway infrastructure. National governments and international development agencies consider the expansion of road infrastructure a key strategy for economic and social development. In the last two decades, China has built a highway system that rivals that of the United States, with plans of substantial expansion (Xu & Nakajima, 2017; Yan, 2011). In India, the rapid growth of the highway infrastructure is currently underway because insufficient road transport is viewed as a key impediment to industrial growth (Ghani, Goswami, & Kerr, 2013). Africa, where most people do not have access to all-weather roads, plans to expand its road network by 6–10 times by 2040 (Programme for Infrastructure Development in Africa, 2013). This increase in road infrastructure and number of vehicles is likely to result in an increase in RTI rates unless accompanied by appropriate evidence-based road safety interventions.

The Decade of Action for Road Safety 2011–2020, officially proclaimed by the UN General Assembly in March 2010, seeks to save millions of lives by building road safety management capacity; improving the safety of road infrastructure; further developing the safety of vehicles; enhancing the behaviour of road users; and improving post-crash response. Several national and international initiatives have been taken over the past decade to promote and fund road safety initiatives around the world. While there is reasonable agreement internationally on safer designs of motor vehicles (except locally produced vehicles like three-wheeled scooter taxis, tuk-tuks, jeepneys, etc.), there is a lack of evidence-based interventions in road and infrastructure design, police enforcement and post-crash care (Davey & Freeman, 2011; Hauer, 2019; Wilson & Gangathimmaiah, 2017).

1.2 Impact of RTI on society

A very large number of high income countries have been estimating the costs of road traffic crashes over the past three decades. The methods used and costs allocated have generated a great deal of discussion and debate, in particular, because of the difficulty of putting monetary values on pain and suffering. A study undertaken by the European Federation of Road Traffic Victims on the impact of road death and injury in collaboration with the Commission for European Union gives the following qualitative conclusions regarding the effect of road traffic crashes on victims (Haegi, 1995).

Physical and mental impairment through road traffic injury can have long-term effects which deny victims the ability to maintain their standard of living.

We have quoted from this report extensively because it is important to note that economic costing of human tragedies can only be used as an inefficient tool to understand the partial costs of the problem.

1.3 Scope of the evidence and gap map (EGM)

Road safety interventions are described as the interaction of three components in the light of Haddon's matrix (Haddon, 1968): human, vehicle and environment separated into pre-crash, crash and post-crash phases. This has been strengthened further over the past two decades by focussing on the Safe Systems Approach which also includes strengthening of institutions and other road safety policy measures (Bliss & Breen, 2009) The interventions included in the EGM here will focus mainly on their impact on road traffic-related injuries as its primary outcome and selected safe road-use practises as its intermediate outcome. In the late 1960s and early 1970s, there was a significant paradigm shift in the development of countermeasures to reduce motor vehicle crash injuries and deaths. Prior to this time countermeasures were largely based on folklore and were not evaluated; the new paradigm envisaged a much broader range of countermeasures and emphasized the importance of evaluating them. There are several modes of transport which are widely used in LMICs such as motorcyclists, three-wheelers, but very limited literature is available on effectiveness of various interventions on road safety with heterogeneous composition of traffic. The EGM will cover all aspects of road safety interventions except car-design. Safety standards for car design including crash worthiness standards have been evolving since 1970s. The UNECE WP.29 (United Nations Economic Commission for Europe, 2012) vehicle safety standards and New Car Assessment Programme (NCAP; Hobbs and McDonough, 1998) are also attempting to harmonize vehicle standards internationally. There is general consensus on technologies which work and don't work internationally and car designs around the world are converging to similar international standards. Safer car designs are also influenced by the market because of car safety ratings announced by agencies like NCAP. However, infrastructure design, safety policies and enforcement are not subject to market mechanisms in the same way. Hence, car design interventions have been excluded from EGM.

The scope of this EGM is to cover relevant studies in the road safety sector from all countries and present the effectiveness of interventions in terms of mainly traffic crash injuries as its outcome.

The interventions adopted in this EGM are classified into five broad categories: Human factors, vehicle factors and protective devices, road design, infrastructure and traffic control, post-crash pre-hospital care and legal and institutional framework. They are described as follows:

Human factors: These cover all interventions including any factor or road user behaviour that leads to occurrence or consequence of RTI.

1.4 Conceptual framework of the EGM

The conceptual framework links road safety interventions with the outcomes and impacts along the causal chain (Figure 1). The conceptual framework shows the causal chain through which the inputs are turned into final societal impacts, through activities, outputs and outcomes.

OPEN IN VIEWER

1

Conceptual framework for the EGM.

It is to be noted that the links in the causal chain are not automatic. For example, the use of police enforcement to reduce road user risk and behaviour would also need legal and institutional framework to reduce health and traffic crashes in order to reduce injuries and fatalities, thus improving health and resulting in less morbidity and mortality.

A scientific approach for controlling RTI evolved in HIC over the last 50 years or so. The main concept behind the evolving strategies rested on the understanding that we must move away from finding fault with victims and instituting retribution systems to a more reasoned approach that deals with systemic improvements and finding solutions which by and large do not put an extra burden on road users. It was understood that RTI result from a complex interaction of sociological, psychological, physical and technological phenomena and since injuries result from an exchange of energy between the environment and the human body, it is possible to develop safety policies and strategies in a scientific and comprehensive manner (Bliss & Breen, 2009; Haddon, 1980; Peden et al., 2004). This will require a shift to an approach with a result focus which aims to eliminate road deaths and serious injuries by adopting interventions that are more likely to succeed in aiming for Vision Zero (Svensson, 2018).

The intervention categories in this EGM are adapted from the Haddon's Matrix which presents the typology of road traffic crashes in time and space (Haddon, 1972). The time category is divided into three mutually exclusive categories comprising interventions that prevent the occurrence of a crash, prevent or reduce the severity of injuries during a crash, and reduce the possibility of a negative outcome after the crash. The space categories include those interventions that are associated with human beings, vehicles and rest of the environment. The last category includes the road infrastructure, legislation, policing and role of institutions. The systems approach as detailed by Bliss and Breen (2009) further elaborates the role of different institutions and inputs included in the environment category in Haddon's Matrix. For the purposes of the EGM these interventions have been clubbed into five groups: (a) human factors, (b) vehicle factors and protective devices, (c) road design, infrastructure and traffic control, (d) post-crash pre-hospital care, and (e) legal and institutional framework, and associated sub-categories.

One of the reasons that road safety interventions were not particularly successful in the first 60 years of motorization (1910–1970) in any part of the world was because policy makers and researchers could not measure the effectiveness of various interventions in terms of fatalities, injuries or crashes prevented. Subsequent research indicated that some interventions may have been based on theoretical beliefs of their effectiveness but were not so in real use and others may produce behaviour change but not found to be effective in the real world of traffic. For example, recent systematic reviews indicate that the following interventions do not result in reducing the adverse effect of road traffic crashes:

Advanced training in trauma life support for ambulance crews (Jayaraman, Sethi, & Wong, 2014).

Researchers in the field of traffic safety have been aware of the existence of counterintuitive results in their area of work more than four decades. The fact that many interventions do not result in reductions in RTI is mainly because a large number of studies only measure intermediate outcomes like change in behaviour or knowledge and not the actual results in the field. For example, Duperrex et al. (2002) point out that “Pedestrian safety education can result in improvement in children's knowledge and can change observed road crossing behaviour, but whether this reduces the risk of pedestrian motor vehicle collision and injury occurrence is unknown”. Hauer (2019) goes further and states that “Issues of concern are the lightness with which decisions affecting road-user safety can be based on opinion that is unsupported by evidence…that questionable results can be given a ring of consensual truth, and that the questions which research asks and what findings get published are at times influenced by external interest”.

In this EGM importance has been given to primary outcomes as there is little agreement among researchers on the relationship between behavioural/knowledge changes and reductions in traffic crashes. Therefore, studies are included only if the outcome is measured in changes in RTI (fatal, injury only) or use of protective devices, changes in vehicle speeds or drinking and driving.

In a recent paper Hauer (2019) states:

Over the past two decades or so progress has been made towards evidence-based practice. Research and researchers provided valuable tools for practitioners to use. But much of practice is still opinion-based and the role of research remains ambiguous. The first step towards reform is to rethink and then to revamp the research-practice relationship. The reformed relationship should be endowed with a purposeful structure, one that cures what dysfunction there is and promotes the generation of trustworthy evidence.

Unfortunately, in the absence of local research on road-related safety interventions, roads and highways in LMIC are being designed to safety standards of HIC without an adequate understanding of the evidence base of existing standards. For example, it is generally accepted that traffic calming measures like chicanes, road narrowing and roundabouts are effective in reducing RTI (Bunn et al., 2003). However, effectiveness of some of these measures in LMICs is not known as vehicles like MTW may not be affected. Even in HIC, many standards for vehicles, roads and policing activities are being promoted without the availability of adequate scientific evidence regarding their effectiveness (Elvik, 2017; Hauer, 2019). Many systematic reviews also point out the fact that evidence for road safety interventions may be available for HICs, but the same is lacking from LMICs (Mulvaney et al., 2015; Roberts & Kwan, 2003).

The SDG target 3.6 for road safety aims to halve the number of global deaths and injuries from road traffic accidents by 2020. However, considering the example of WHO European region, despite a decrease of 8.1% in road traffic deaths between 2010 and 2013, the region may not be able to fulfil this SDG 2020 target to halve the road traffic fatalities (Jackisch, Sethi, Mitis, Szymañski, & Arra, 2015; WHO, 2018)

The SDG target 11.2 seeks to provide access to safe, affordable, accessible and sustainable transport systems for all, improving road safety, notably by expanding public transport, with special attention to the needs of those in vulnerable situations, women, children, people with disabilities and older people.

In order to come closer to accomplishing these targets, it is important to allocate resources to promote interventions that are effective in achieving outcomes in the context of road safety. A mapping will provide a comprehensive overview of existing knowledge in the area of road safety and its effectiveness across the world. The map will guide programme managers to high quality evidence and inform targeted commissioning of future research.

1.6 Existing EGMs and relevant systematic reviews

A map of evidence maps conducted in LMIC identified no EGM conducted around transportation and other adaptive measures (Phillips et al., 2017). This EGM being a global focus aims to include studies on the effectiveness of road safety interventions from countries in all continents. The EGM strives to capture relevant studies conducted and broaden the included interventions and outcomes to better reflect the state of evidence in road safety in 2019.

The following systematic reviews and synthesis studies will be incorporated in the EGM after being subjected to inclusion/exclusion criteria.

3.1 Defining EGMs

An EGM aims to provide a visual overview of thematic collections of what we know and do not know about the effectiveness of interventions (Snilstveit, Vojtkova, Bhavsar, Stevenson, & Gaarder, 2016).

3.2 EGM framework

The framework for this EGM was constructed through a review of strategy and policy documents and discussion through external consultations:

1.

An Advisory Group specially nominated for this project

2.

Discussion with ICoRSI

INDEPENDENT COUNCIL FOR ROAD SAFETY INTERNATIONAL (for more information refer https://www.icorsi.org/).

Council Members.

3.

Reference to reviews/meta-analysis in The Handbook of Road Safety Measures and Safety Cube Decision support system.

The identification of primary studies and systematic reviews to be included will be through a comprehensive search of previously published and unpublished literature irrespective of whether the study is completed or ongoing. This, in turn, will also help identify and hence fill the existing gaps in the evidence map.

The final EGM will have a structured framework of interventions and outcomes relevant to road safety with various filters in a user-friendly way.

Key features include: 1.

The EGM will contain all the relevant evidence from SRs and primary studies and provide access to user-friendly summaries and appraisals of those studies.

2.

The EGM will show where completed and, through the inclusion of trial registries, on-going primary studies have been conducted.

3.

The EGM will highlight absolute gaps (lack of studies for particular interventions/outcomes) and synthesis gaps (where there is a density of primary studies but lack of high-quality SRs or an update of existing SRs)

3.2.1 Intervention

The EGM will include any intervention aiming to reduce RTI and fatalities as an objective excluding the effectiveness of car design intervention in the vehicle factors and protective devise category. All the interventions possible in road safety have been categorized in groups and sub-groups as shown in Table 1.

The outcomes are listed in outcome domains and each domain has a number of sub-domains (Table 2). These may be modified during the piloting stage.

We will be excluding any study that includes only vehicular (including bicycles) damage/crash but do not report any related fatalities and injuries.

3.3.1 Types of study designs

The EGM will include impact evaluations and systematic reviews of the effectiveness of interventions. Impact evaluations are defined as intervention evaluations or field experiments that use quantitative approaches applied to experimental or observational data to measure the effect of an intervention relative to a counterfactual representing what would have happened to the same group in absence of that intervention. Impact evaluations may also test different intervention designs. We will include both completed and on-going impact evaluations and systematic reviews; to capture the latter, we will include prospective study records in trial registries or protocols when available.

Details of study designs:

(a)

Prospective studies allocating the participants to the intervention using randomized or quasi-randomized mechanisms at individual or cluster levels. i.

Randomized control trial (RCT) with assignment at individual or cluster level (e.g., clustering at market, round-about, etc.).

ii.

Quasi-RCT using a quasi-random method of prospective assignment (e.g., alternation of clusters).

(b)

Non-randomized designs with selection on unobservables: i.

Natural experiments using methods such as regression discontinuity (RD).

ii.

Panel data or pseudo-panels with analysis to account for time-invariant unobservables (e.g. difference-in-difference (DID), or fixed- or random-effects models).

iii.

Cross-sectional studies using multi-stage or multivariate approaches to account for unobservables (e.g. instrumental variable, IV, or Heckman two-step estimation approaches).

(c)

Nonrandomized designs with selection on observables: i.

Controlled before and after studies with an intervention and comparison group using methods to match individuals and groups statistically (e.g., PSM) or control for observable confounding in adjusted regression.

(d)

Studies explicitly described as systematic reviews and that describe methods used for search, data collection, and synthesis.

We will include impact evaluations where the comparison/control group receive no intervention (standard road safety intervention), a different intervention (e.g. police enforcement), a placebo or the study employs a pipeline (wait-list) approach.

All theoretical, modelling or laboratory studies would be excluded. Additionally, all studies using self-reporting and simulations will also be excluded.

3.3.2 Treatment of qualitative research

We do not plan to include qualitative research.

3.3.3 Types of settings

We will include all studies conducted of interventions aiming to have an impact on traffic crashes. We will exclude studies focusing on car design effectiveness in road safety the reasons for which are explained in scope of the EGM.

3.3.4 Status of studies

We will search for and include completed primary studies and systematic reviews and on-going systematic reviews. We will not exclude any studies based on language or publication status or publication date.

4.1 Unit of analyses

The unit of analysis is the paper. Where multiple papers exist on the same study (e.g. a working paper and a published version), the most recent open access version will be included in the evidence map. If the versions report on different outcomes, an older version will be included for the outcomes not covered in later versions.

We shall code where papers come from the same study (i.e. the same data set by the same research team). It is common in public health to publish multiple papers from the same study, reporting different outcomes or different sub-populations in separate papers.

4.2 Planned analyses

The analysis shall present tabulations and graphs by the primary and secondary dimensions, as well as time trends. Evidence gaps shall be identified and discussed.

4.3 Presentation

The EGM will have two primary dimensions: interventions (rows) and outcomes (columns). Additional dimensions will be:

1.

Income: Low-income countries, lower-middle income countries, upper-middle income countries, high-income countries (according to World bank 2020 fiscal year classification).

2.

Region: South Asia, Sub-Saharan Africa, East Asia and Pacific, Europe and Central Asia, Latin America and Caribbean, Middle East and North Africa, North America (according to World bank classification).

3.

Systematic review quality: low, moderate, high.

4.

Type of primary study: RCT, non-RCT, Observational study.

5.

Status of study (completed, ongoing).

In the hard copy of the EGM, multiple 2 × 2 representations of the EGM will be reported. In the online version, selected additional dimensions will be possible to use as a filter (Appendix A). The online version will include references to included studies and brief summaries of each study (for impact evaluation studies) or plain language summary (for systematic reviews) provided for it.

5 STAKEHOLDER ENGAGEMENT

The framework presented here has been developed through the following process:

Stage 1: Initial framework to be constructed through review of strategy and policy documents, and discussions through external consultations through (a) consultation with Advisory Group specially nominated for this project and (b) discussion with ICoRSI Council Members, (c) reference from The Handbook of Road Safety Measures (Elvik and Vaa, 2004).

Stage 2: Piloting framework with 30 included studies. The framework will be finalized once the first 30 studies are coded. The protocol will be revised at this point.