PROTOCOL: Evidence and gap map protocol: Understanding pathways between agriculture and nutrition: An evidence and gap map of tools,metrics and methods developed and applied in the last 10 years

1.1 The problem, condition or issue

The global food price crises of 2007–2008 and 2010–2011 drew attention to the need for addressing the underlying determinants of malnutrition in low- and middle-income countries (LMICs; Brinkman, de Pee, Sanogo, Subran, & Bloem, 2010; Webb, 2010). Specifically, as the primary source of food and income in LMICs, agriculture received renewed focus. Making agriculture work for nutrition—nutrition-sensitive agriculture—has climbed the international development agenda (Ruel, Alderman, & Maternal and Child Nutrition Study Group, 2013). More recently, given the sharp increase in diet-related chronic diseases underpinned by overweight and obesity in LMICs and the threats of climate change to diets, attention has expanded to leverage food systems to optimize nutrition, health and environmental outcomes (Johnston, Fanzo, & Cogill, 2014). Donors, researchers and implementers mobilized research agendas to invest in understanding how to strengthen agriculture and food systems to realize nutrition outcomes sustainably.

Progress in this field in the last decade included three key developments:

Development of conceptual frameworks to aid the investigation of agriculture-food system and nutrition linkages, highlighting multiple direct and indirect complex pathways (Global Panel, 2015; Hawkes, Turner, & Waage, 2012; Johnston et al., 2014; Kadiyala, Harris, Headey, Yosef, & Gillespie, 2014; Lock et al., 2010; Masters et al., 2018).

These efforts led to widespread recognition of inadequate tools, methods and metrics to study the direct, indirect and dynamic relationships between in agriculture-food systems and nutrition outcomes. There have been several calls to accelerate the development of innovative tools, methods and metrics to underpin the development of a robust scientific evidence base needed to guide policy investments in agriculture-food systems for improved nutrition and health. In response to this demand, several projects and programmes were launched specifically to develop new research methods, including the DFID-funded Innovative Metrics and Methods for Agriculture and Nutrition Actions (IMMANA) programme (Innovative Methods and Metrics for Agriculture and Nutrition Actions [IMMANA], 2018).

Research undertaken under IMMANA and others (CGIAR, 2018; Global Dietary Database [GDD], 2014; International Dietary Data Expansion Project [INDDEX], 2018; Sustainable and Healthy Food Systems [SHEFs], 2018) have built on existing theoretical underpinnings and have helped to refine hypothesized pathways, illuminating additional aspects and dynamics between agriculture or food systems and nutrition outcomes, such as food environments, environmental factors and food safety. The Food and Agriculture Organization (FAO) of the United Nations (UN) adopted the High-Level Panel of Experts (HPLE) definition of a food system: “all the elements (environment, people, inputs, processes, infrastructures, institutions, etc.) and activities that relate to the production, processing, distribution, preparation and consumption of food, and the output of these activities, including socio-economic and environmental outcomes” (HLPE, 2017). Agriculture and health are part of the broader milieu of a food system. As such, there has been a proliferation of innovations in programme design and implementation, as well as in metrics and methods and their application. While the body of evidence on effectiveness of food systems to improve nutrition has been recently summarized (Ruel et al., 2018), the portfolio of new methods and metrics has not. It is now necessary to take stock of these developments and plan for the future to support the production of effective and relevant research.

1.2 Scope of the evidence and gap maps (EGM)

Recognition of the multiple pathways through which nutrition impact is achieved brought about new conceptual frameworks, and along with them, new thinking in how to measure the complexity and dynamism within these systems. The innovations that emerged range from new technology to new indices to the application of methods from other fields. New metrics and methods have been developed throughout the pathways (household production, decision-making, income, etc.) linking agriculture and nutrition.

In a standard effectiveness map, the row headings are interventions, and the column headings outcomes. In this map, those thematic pathways or domains will be considered the “interventions” through which nutrition is improved. We consider our “outcome” to be tools, metrics or methodological innovations, which are the columns of this map. Some innovations have been widely adopted across settings, and others are still in development. Therefore, each example innovation will be mapped using the studies that pertain to the innovation. Innovations have taken place at every level of measurement (individual, household, national, etc.) and correspond to certain cross-cutting themes. These additional aspects will be coded internally on the map.

As an example of technology application at an individual and household level, researchers have utilized accelerometers to measure calorie expenditure in new ways to address intra-household food allocation and gender roles (Zanello, Srinivasan, & Nkegbe, 2017). At a community level, researchers have employed wearable cameras and GIS technology to map changing food environments in urban areas (Schrempft, van Jaarsveld, & Fisher, 2017). A methodological innovation at a sub-national level has been to use Bayesian theory and decision-analysis for making policy that affects nutrition (Yet et al., 2016). More thorough data collection and new indices to capture prices of nutritious foods in markets at the regional level have led to better estimates of cost of nutritious diets in Ghana (Masters et al., 2018). New innovation in this field also includes tools to conceptualize and operationalize food systems, including how to frame and measure cost-effectiveness of complex interventions, which have a range of outcomes (Masters et al., 2018). In this EGM, the columns will be types of innovation or novel application.

The aim of the gap map is to articulate and summarize the innovation in tools, metrics and methods that have been created and applied to understand food systems and agriculture-nutrition linkages in the last ten years. We have chosen the ten-year period based on the focus on and funding for agriculture-nutrition linkages that emerged following the global food price crisis in 2007–2008, as well as wanting to focus on new innovations, which, by definition, would no longer be novel if developed more than a decade ago. We also aim to highlight gaps and opportunities for future development.

1.3 Conceptual framework of the EGM

Although the intervention-outcome framework is most common for maps on effectiveness studies, this framework will be organized differently. We will take an approach that considers tools, metrics and methods (types of innovation/application in the columns) for agriculture-food systems-nutrition research (thematic domain in the rows). The map will be organized around a combination of conceptual frameworks that include the definition of food systems offered by the HPLE report on nutrition (HLPE, 2017), predefined pathways to improved nutrition (Global Panel, 2015; Hawkes et al., 2012; Herforth, Nicolò, Veillerette, & Dufour, 2016; Kadiyala et al., 2014), as well as additional themes that have been identified as more research is being undertaken on this topic (Grace et al., 2018; Masters, 2016; SHEFs, 2018) (Appendix A).

These conceptual frameworks (illustrated in Appendix A) overlap a great deal. For instance, each highlights the role of on-farm production as a means for direct consumption as well as a potential income source, both which influence food availability and diet quality, and thus contribute to nutrition outcomes. Frameworks 1 (Kadiyala et al., 2014), 2 (Hawkes et al., 2012) and 3 (Herforth et al., 2016) are very similar—in fact the most substantive difference is the visual organization of components. These frameworks include aspects of women's time, income and employment. The same three highlight interacting aspects of care, education or knowledge, as well as overall health as drivers of nutrition outcomes.

Each framework also has differences, both in how it is visualized and the content. Each represents “indirect” determinants of nutrition outcomes, such as the role of climate, the environment, policy, governance and culture, inequity, and so forth, but some are shown as an outside layer of influence, whereas some of these are considered within the central framework. For instance, Framework 4 shows the interactions between environment or sustainability aspects and food production, highlighting that human health should always be balanced against planetary health, since they are symbiotic in the long run (Tuomisto et al., 2017). Several highlight important domains that are not equally represented on the other. Frameworks 5 (Masters, 2016) and 6 (Global Panel, 2015) propose the most current thinking about markets and the economic role of nutrition. These support the idea that production will lead to consumption only where, when and for whom markets are missing. The Global Panel Metrics and Methods Framework (Framework 6) puts the food environment as the central milieu into which other dynamics feed, and diet diversity, adequacy and safety as general by-products of that food environment. In contrast, Framework 2 (Hawkes et al., 2012) specifically articulates the subsequent layers of the food environment that progressively lead to nutritional status. We will use all frameworks generally to ensure that the EGM is comprehensive and that the domains within conceptual pathways in agriculture to nutrition literature are represented and categorized logically, while maintaining iterative methods of refining the domains based on search results.

1.4 Why it is important to develop the EGM?

Governments, non-government donors, implementing agencies and academia have all made significant investments, both intellectually and financially, in improving agriculture or food systems for nutrition outcomes. This investment has gone beyond scholarship and documentation and taken the form of application and innovation of tools, metrics and methods at every level. Stakeholders have called for a synthesis project on this topic in order to visualize the current portfolio of these developments, strategically plan the next wave of investment and shape the next generation of agriculture-nutrition research.

1.5 Existing EGMs and/or relevant systematic reviews

There are no current gap maps on the topic of metrics and methods on the topic of agriculture and food systems for nutrition (or to improve nutrition outcomes). Some mapping exercises have been undertaken on pathways between agriculture and nutrition, namely the 2012 LCIRAH “Current and planned research on agriculture for improved nutrition: a mapping and a gap analysis”, which led the way to the IMMANA programme (IMMANA, 2018). The FAO Compendium of nutrition-sensitive indicators also summarizes the most well-established indicators on the subject (Herforth et al., 2016), but does not fully capture innovative tools and methodologies, as well as metrics that are in development currently. Furthermore, to our knowledge, none of these synthesis projects have been systematic or published as a formal EGM, and overall there have been no EGM of tools, metrics, or methodologies; rather most existing gap maps focus on effectiveness studies.

3.1 Defining EGM

All previous EGMs published to date have been compilations of effectiveness studies, therefore this EGM will be novel in many aspects. We are not aware of any protocols or published EGMs on which to model this project. There are several synthesis reports on this topic (Hawkes et al., 2012; Herforth et al., 2016), but as mentioned previously, none of them are current, systematic or are formal EGMs. We will use published, well-established conceptual frameworks to define thematic domains of agriculture-to-nutrition in order to categorize the identified tools, metrics and methods. Therefore, our “intervention” will be each broad domain on the food systems or agriculture to nutrition pathway. The columns of our map, (outcome in effectiveness maps) will be each item of innovation (tool, metric or method) that has been developed or applied to capture or measure these domains.

While “innovation” is hard to categorically define, we will take a pragmatic, data-driven approach to selecting what is new, novel or innovative in this dynamic field. By a data-driven approach, we mean that we will use existing knowledge and data to inform the process, both through the published record and expert consultation. We will adopt several strategies and guidelines in selecting new, novel or innovative tools, metrics or methods:

●

Limit the search to work published after 2008.

●

Identify completely new tools, metrics or methods that were introduced after 2008 with no previous iterations.

●

Identify tools, metrics or methods that existed prior to 2008 but have been significantly revised or modified since. As a “significant” change or modification is difficult to determine, we will rely on the group or authors' own assertions and explanations, and make an expert judgement as a group when unclear.

●

Identify new, novel or innovative applications of existing tools and methods. This will mostly entail applying these cross-disciplinarily. This will be the most difficult aspect of “newness” to define, and we therefore will also rely on the authors' description and justification, and secondarily make a collective expert decision.

Because some of the tools, metrics and methods are in their infancy, while others are globally adopted and have become standard practice, we will code each study based on the current (e.g. in 2019) stage of development of the innovation. We will further code and categorize innovations by several thematic filters (e.g., gender, equity, economics, technology, private-sector engagement, conflict or political fragility), geographical application, and level of measurement.

3.2 EGM framework

The “intervention” (rows), will be defined using agriculture-to-nutrition conceptual frameworks mentioned previously, divided into “domains” of influence on agriculture and food systems or nutrition, such as household or on-farm production, food policy and governance, or food environments and markets. The columns, or categories of innovations/applications, will be grouped by different classifications of tools (technology application and instruments to capture data on a range of agriculture-nutrition topics), metrics (new indices and measures to quantify agriculture-nutrition linkages) and methods (research design and analytical approaches applied to agriculture and nutrition research). We will code each study related to an item and group the items iteratively once all items have been mapped. We have chosen to do this since some well-adopted innovations will have many papers that use the tool, metric or method, while others will have only a few, and some will apply the tool, metric or method in different ways. Users will be able to see each individual item as well as grouping by tool, metric or method.

Traditional EGMs include a quality assessment of each item, such as a risk of bias rating, which are not designed to evaluate tools, methods or metrics. In place of a quality assessment, this EGM will summarize the stage of development or application (explained below). We will add filter codes for certain cross-cutting themes such as gender or private-sector engagement. It will also categorize the measurement level (individual, household, district, national, etc.), and setting or geographical application (Asia, Africa, global, etc.). We also may add other filters as the search progresses. The framework structure is shown in Appendix B.

3.3 Population

This map will only include tools, metrics and methods that have been applied to agriculture-nutrition pathways in any country at any level: individual, household, community or district, sub-national, national and global.

3.4 Intervention

The problem we are considering is any domain that exists on the conceptual pathway between agriculture and/or food systems and nutrition outcomes. These domains have been grouped (through using frameworks and extensive rounds of expert consultation) by broad themes around food production, food safety, value-chains, markets and food environments, food policy and governance, environment and climate, among others. We have organized the domains around broad themes in order to group items with minimal double- or triple-coding, but we do envision that some items will appear in more than one domain. Whenever possible we will select a “primary” domain and use the filters and codes to indicate other aspects of the tools that are cross-cutting, such as gender, technology or economics.

The first column lists 12 broad domains, and the second column are examples of what types of work would fit in to these domains. All included tools, metrics or methods must explicitly relate to either agriculture/food systems, or to nutrition. Any tools, metrics and methods that are not related to either agriculture/food systems or nutrition will be excluded. Most of the domains could be measured at various levels (individual, national, global, etc.). We will not differentiate in the domains, but rather in the internal coding of each tool, metric or method innovation. As the initial results are identified, we may refine these domains and add sub-domains using an iterative methodology.

Specifically, we propose the following overarching categories based on the conceptual frameworks discussed previously (Table 1).

The domains of food safety, food environments and economic evaluations each have systematic reviews on methods and metrics. We will include the items that are included in those reviews and also use these items to test the framework.

The primary “outcome”, of the gap map (i.e., the columns in the map) is the innovative item (tool, metric or method) created and applied to studying and describing the broad agriculture to nutrition pathways. We will define “tool” as is a vehicle or an aid to collect information and data (e.g., a survey module to collect data required to compute an index or a piece of technology). “Metrics” will be defined as the parameters (measures) or indices used for measurement, comparison or tracking performance (e.g., disability adjusted life years; household dietary diversity score and Women's Empowerment in Agriculture Index [WEAI]). We define “method” as the process and approach involved in a systematic inquiry of relationships between agriculture, nutrition and health and generally refer to study design or application of an analytical method to this topic (e.g., impact evaluations using various types of counterfactuals, pathway analyses, decision analyses).

Several types of “innovation” are described above. We will identify each tool metric or method and group them together once identified. Some methods or tools will have slight variation in their application or analysis, but we will group these logically together as an item if the construction of the item is similar. We will include all of the studies on the map.

Each item might have multiple innovative components that fit distinctly within the categories of tools, metrics or methods. We propose that these will not be exclusive, rather the appropriate component of a paper or project will be categorized accordingly, and the total number of items summarized separately. In order to determine what is substantial enough for inclusion as a separate item (rather than a much smaller exploratory innovation and application), we will use the published or unpublished study objectives, considering only primary and secondary objectives. We will group tools, metrics and methods into the following categories, with some illustrative examples in the right-hand column (Table 2).