Needs and Challenges Related to Multilevel Interventions: Physical Activity Examples

Abstract

The theme of this commentary is to discuss the need for, and challenges of, conceptualizing, implementing, and evaluating multilevel health behavior change interventions. Ecological models, recommendations from national and international authoritative groups, and growing evidence all support the need for interventions designed to change multiple levels of influence, including individuals, social environments, organizations, built environments, and policies, to achieve population improvements in public health. Though multilevel interventions are becoming more common in practice, they are still under-used, and research on multilevel interventions is relatively rare. Drawing on examples from physical activity, several types of evidence are summarized to demonstrate that multilevel interventions are feasible and can be effective. Serious challenges to implementing and evaluating multilevel interventions include the need for teams with diverse expertise, lack of control over intervention implementation, unpredictability of timelines, managing complex teams over extended periods, and need to apply unfamiliar and less-rigorous study designs and methods. Recommendations are offered for changes in training, funding priorities, and academic incentives that could lead to more and better multilevel interventions.

Keywords

ecological model exercise health behavior health education health promotion intervention natural experiment quasi-experimental

In April 2018, I was honored to receive the Elizabeth Fries Health Education Award at the SOPHE (Society for Public Health Education) Conference in Columbus, OH. I am humbled to be named along with the previous winners of this prize, all of whom I greatly admire. I express particular gratitude to David Sleet and Karen Glanz who nominated me, both of whom are previous awardees. Many thanks to Victor Bovbjerg for representing the Fries family and for his gracious and unexpected comments that his late wife Elizabeth Fries found some of my work to be useful. That connection with Elizabeth made the award even more meaningful.

One of the responsibilities, and pleasures, attached to the award is to present the Elizabeth Fries Award Lecture at SOPHE. The theme of my talk was about the need for, and challenges of, conceptualizing, implementing, and evaluating multilevel health behavior change interventions. Health behavior research and intervention has evolved from a primary focus on health education, to a broader approach to health promotion, to the current emphasis on environmental, policy, systems, and multilevel interventions guided by ecological models. This evolution is seen more clearly in population health research and applications compared with the clinical treatment realm.

Because each health behavior field has evolved differently, I focus this commentary on the physical activity field, with some examples from the closely related field of obesity prevention. The goals are to (a) summarize the rationale for multilevel interventions and their promise for improved effectiveness, (b) identify authoritative recommendations that call for multilevel interventions, (c) summarize evidence that supports multilevel interventions, (d) describe challenges of studying and implementing multilevel interventions, and (e) offer recommendations for advancing practice and research.

Rationale and Background for Multilevel Interventions

The rationale for multilevel interventions is easy to understand. Because health behaviors have complex etiologies, with multiple levels of influence, it is not surprising that single-level interventions often are not sufficient. Educational and behavior-change skills training targeting individuals has limitations regarding reach, effect sizes, and maintenance of change (Sallis et al., 2006). Social and physical environments, policies, and economics often provide powerful opposition to healthful behaviors, for example, negative social norms about exercise, lack of parks in some neighborhoods, weak physical education policies, and free parking for cars. Though he is a thought leader for individually targeted interventions, Albert Bandura (1986) concluded, “When environmental conditions exercise powerful constraints on behavior, they emerge as the overriding determinants.” (Thanks to Dr. Neville Owen for sharing this quote.) In such situations, individuals can be expected to struggle to overcome powerful barriers, even with the benefit of theory-based interventions. Changing only environments and policies also cannot be considered a panacea. Policy changes (strong physical education requirements) without supportive environments (encouragement from school principals, sufficient activity equipment) are unlikely to be effective. Environmental changes, such as a new walking and bicycling trail, may not be apparent to nearby residents.

Multilevel interventions can be designed to work synergistically to overcome limitations of strategies targeting any single level of influence. I have long recommended making environment and policy changes first, so educational and motivational interventions take place in more supportive environments with fewer barriers to change. For example, strong physical education policies should be supported with activity equipment (physical environment), teacher training and incentives (social environment), and parental education (individual/family). A new walking and bicycling trail should be promoted with media stories (individual, social environment), partnerships with local community organizations (social environment), events on the trail (social environment), signage directing people to the trail (physical environment), and safety improvements to connecting roads so it is safer to walk and bicycle to the trail (physical environment).

Multilevel interventions are not new to public health. Multilevel ecological models have been used in public health research and practice for many years, though their influence appears to be increasing (Sallis & Owen, 2015). The Ottawa Charter for Health Promotion explicitly calls for multilevel interventions (World Health Organization, 1986), and this foundational document appears to have had a major influence on shaping public health practice to be more oriented to multilevel conceptualizations and interventions. The widely used PRECEDE–PROCEED model instructs practitioners to develop multilevel interventions (Green & Kreuter, 1999). My personal observation is that multilevel interventions have advanced more quickly in practice than in research.

Evidence Supporting Multilevel Interventions to Promote Physical Activity

Multilevel interventions have been implemented on a large scale in the tobacco control field for decades, with key components being smoking cessation programs (individual), media campaigns (social environment), restricted access such as removing cigarette machines (physical environment), and taxation (policy; Warner & Mendez, 2010). More recently, multilevel obesity prevention strategies have been recommended and supported by the U.S. Institute of Medicine (Kohl & Cook, 2013; Koplan, Liverman, & Kraak, 2005), the U.S. Surgeon General (U.S. Department of Health and Human Services, 2001), and the World Health Organization (2004). Though the obesity prevention strategies all include physical activity interventions, I focus here on evidence specific to physical activity.

Indirect evidence about the promise of multilevel intervention comes from studies of correlates and determinants of physical activity. A review of reviews indicated there are consistent correlates of physical activity among people across the life span from the individual (e.g., self-efficacy), interpersonal (e.g., social support), and environmental (e.g., walkability, park access) levels of influence (Bauman et al., 2012). This evidence supports the value of ecological models and suggests interventions targeting mediators at multiple levels of influence are consistent with this evidence.

Additional indirect evidence comes from studies showing interactions of psychosocial and built environment variables in explaining variance in physical activity. The patterns of interactions appear to vary by age of participants, suggesting that physical activity multilevel interventions may need to be structured differently, depending on the target population. Ding et al. (2012) found several interactions explaining walking for leisure among adults aged 20 to 64 years that all had the same pattern. Activity-supportive environmental conditions, such as having a nearby park or good aesthetics, seemed to be most helpful for people with unfavorable psychosocial conditions, such as many perceived barriers or low enjoyment of activity. For younger adults, improving built environments might help those less-inclined to be active to overcome psychological barriers. Carlson et al. (2012) found that walking was highest for older adults when they had both supportive environments, such as walkability and sidewalks, and favorable psychosocial variables, such as social support and self-efficacy. This finding seems to imply that the combination of improving environments and individual interventions may be most effective for older adults. This indirect correlational evidence helps generate hypotheses about the nature of multilevel interventions that can be evaluated in further studies.

I am not aware of any randomized studies that directly compare single-level versus multilevel physical activity interventions. Such studies would be difficult to implement. But there is suggestive evidence that multilevel interventions tend to more effective. An intriguing pattern of results has been found in the bicycling intervention literature. Two review papers concluded there is no convincing evidence that single interventions are effective in increasing bicycling (Pucher, Dill, & Handy, 2010; Yang, Sahlqvist, McMinn, Griffin, & Ogilvie, 2010). However, a series of 14 case studies led to a strikingly different conclusion (Pucher et al., 2010). Regardless of initial prevalence of trips by bicycle, cities on several continents that implemented numerous strategies over years reported substantial increases in bicycling. The sole exception was a Dutch city that had 40% of trips by bicycle at baseline. Transportation interventions are typically driven by a multilevel model known as the “5 Es,” so the cities implemented multilevel interventions. The “Es” are engineering (physical environment, such as protected bike facilities, bike parking), enforcement (social environment, such as penalties for drivers who collide with bicyclists), education (individual level, such as school safety programs and driver education), encouragement (individual and social environment level, such as incentives and workplace programs), and evaluation.

Safe Routes to School interventions are funded by transportation departments and also apply the “5 Es” model. One study evaluated several hundred schools in five states that were funded through the federal Safe Routes to School program (Stewart, Moudon, & Claybrooke, 2014). The study was a pre–post assessment of schools with grants that included physical environment changes, such as improving sidewalks and street crossings, usually along with other strategies. The percentage of students walking to school on a given day increased from 10% to 14%. Though they did not conduct statistical tests, this appears to be a positive outcome for the multilevel interventions.

Three evaluations of city-wide multilevel physical activity interventions show the feasibility of a range of intervention approaches. Similar interventions with a theme of “10,000 Steps” were implemented and evaluated in Rockhampton, Australia (Brown, Mummery, Eakin, & Schofield, 2006), and Ghent, Belgium (De Cocker, De Bourdeaudhuij, Brown, & Cardon, 2007). Individual interventions included informational campaigns in local media and widespread distributions of pedometers for personal feedback. Social and organizational interventions targeted health professionals, workplaces, schools, and organizations serving older adults. Environmental approaches were modest and focused on placing extensive promotional signage throughout the communities as well as street signs indicating the number of steps to nearby destinations. Both studies included comparisons to neighboring cities over 1 year. In Rockhampton, the intervention yielded significant increases in steps among women (Brown et al., 2006), and in Ghent, population-wide increases were documented, compared with the control city (De Cocker et al., 2007). A third study in Liverpool, England, evaluated an even more ambitious 5-year community intervention led by a multisectoral coalition of health and sport governmental agencies and nonprofit organization (Dawson, Huikuri, & Armada, 2015). Intervention strategies were mainly implemented by participating organizations and included a variety of communication modes targeting individuals, organizational changes to prioritize physical activity promotion, increased availability of physical activity programs throughout the city, and improvements to sidewalks, parks, and sports facilities. There was no comparison city, so the evaluation focused on process evaluation and examination of pre–post changes in physical activity among various demographic and neighborhood subgroups based on repeated surveys. Many organizational and environmental changes were documented, and there was suggestive evidence of increased physical activity, especially among residents of disadvantaged neighborhoods where intervention efforts were concentrated.

As a final example of evidence supporting multilevel interventions, consider a review of 43 trials of obesity prevention interventions conducted in early care and education settings (Ward et al., 2017). All interventions had multiple components that were coded for intervention “strength,” with higher scores for more components and for environment and policy changes that were relatively permanent, and lower scores for educational and short-term strategies. Across the studies intervention strength scores were correlated greater than 0.30 with anthropometric improvements. More explicit support for multilevel interventions was provided by a finding that the strength of parent engagement strategies added to the effectiveness of the other intervention components.

Though the evidence that illustrates the promise of multilevel interventions for physical activity promotion is admittedly indirect, it provides support for the principle of ecological models that multilevel interventions can be expected to be more effective than single-level interventions. I assert that current evidence justifies further study, ideally using more sophisticated designs. However, there are significant barriers to both implementing and evaluating multilevel interventions.

Challenges in Implementing and Studying Multilevel Interventions

Commonly used ecological models of behavior, recommendations from national and international authoritative organizations, and mounting though indirect evidence all point to the promise of multilevel interventions to improve physical activity on a population-wide basis. However, the significant challenges of implementing and evaluating multilevel physical activity interventions should be understood. Here is an incomplete list of challenges to consider.

Expertise

The U.S. National Physical Activity Plan is organized around eight societal sectors, including media, transportation and land use, business and industry, and faith communities (U.S. National Physical Activity Plan, 2018). There are recommended strategies and tactics for each sector, and most of the sector plans envision multilevel interventions. Thus, the Plan is a suite of multilevel interventions. Each sector plan calls for distinct expertise from professionals who are knowledgeable about the sector. To implement any sector plan, expertise from multiple disciplines would likely be needed. Developing, coordinating, and sustaining such multidisciplinary partnerships is a challenge, especially when there is a limited history of the disciplines working together. From a research point of view, a new field has emerged called “Science of Team Science,” with the goal of conducting research that will contribute to improved functioning of such interdisciplinary teams (Hall, Vogel, & Croyle, in press). A similar research effort to improve the functioning of multisector teams implementing multilevel interventions might also be valuable, and this agenda is compatible with Implementation Science (Glasgow et al., 2012).

Efforts have been underway since the beginning of the 21st century to engage a wide range of disciplines in active living research, with some evidence of progress (Barker & Gutman, 2014; Sallis et al., 2014). However, continued efforts are needed to support interdisciplinary and multisector research and practice, because the professionals and government agencies responsible for designing and managing common settings for physical activity, such as cities, transportation systems, parks, and schools, are not obligated to consider health effects of their decisions or involve health professionals in decision making (Dannenberg, Frumkin, & Jackson, 2011).

Timing and Predictability

Multilevel interventions can span a wide range of strategies, including electronic, social, and mass media; group programs; organizational changes; built environment changes; and policy adoption and implementation. Each level of implementation operates on different timelines. Media interventions can be planned and implemented relatively quickly, group programs can be planned and implemented over a period of months, but built environment and policy interventions generally require years to plan and implement.

There are parallel differences in the predictability of the timing. Though there are uncertainties in media interest in physical activity stories and implementation of group behavior change interventions, those uncertainties are dwarfed by environment and policy change efforts. Political processes leading to policy change are notoriously unpredictable and outside the control of health professionals and investigators. Policy adoption may or may not be followed by its implementation, or policies may be implemented in such a way that they widen health inequalities. Even relatively simple environmental changes such as building sidewalks or a trail, or renovating a park, require planning processes, agency and government approval, funding, and construction time.

The disparities in timing and unpredictability of the change processes create obvious challenges to the coordination of multilevel interventions, even those conducted within a single sector. What group has the expertise to coordinate such a complex, long-term effort? What leadership group would have the stability of funding needed to maintain an advocacy and oversight effort for many years? An argument can be made that multilevel, multisector interventions to promote physical activity are simply infeasible. A counterargument is that there are many reasons to believe that such interventions are essential for controlling the pandemic of inactivity (Kohl et al., 2012). Moreover, the preceding section illustrates that complex long-term interventions have been implemented, with evidence of effectiveness.

Study Designs and Methods

Researchers interested in multilevel interventions must go beyond their comfort zone of randomized trials. Lack of control over environment and policy intervention levels usually prevents randomization of units, such as communities, to conditions. Thus, less rigorous study and evaluation designs are both necessary and appropriate (Sallis, Story, & Lou, 2009), such as natural experiments with comparison conditions (Brown et al., 2006; De Cocker et al., 2007), process evaluations (Dawson et al., 2015), and multiple case studies (Pucher et al., 2010). Studies with such designs are usually given low quality scores in rating systems used in meta-analyses, and it may be difficult to publish papers with these designs in leading journals. These challenges may discourage investigators from pursuing evaluations of multilevel interventions out of concern about hampering their career advancement or even doubts about the value of studies based on designs with weak evidence of causality.

My own perspective is that public health researchers have an obligation to study the most important questions using the most appropriate designs and methods. That is, our research agendas should be driven by the most important questions, not by methodology. If we avoid studying questions that cannot answered with randomized trials, this retards progress in public health, especially when the intervention strategies judged to have the most public health potential and recommended by the leading public health agencies cannot realistically be evaluated with the most rigorous designs.

Several study designs and methods have been recommended that are appropriate for complex, long-term, community-wide, policy and environment, and multilevel, multisector interventions (Shadish & Galindo, 2010). Though none of the controlled or uncontrolled studies in the preceding section can be considered definitive, a convergence of results from pragmatic evaluations can build confidence in the utility of a general approach, such as multilevel interventions. It is the nature of community-wide interventions that strategies are tailored to the needs, desires, and opportunities of each community, so validation of specific and fully replicable interventions cannot be expected. But the Rockhampton (Brown et al., 2006) and Ghent (De Cocker et al., 2007) examples demonstrate replicability of general principles of multilevel intervention across very different settings, as do the 14 case studies of multilevel, multi-year bicycle interventions (Pucher et al., 2010). The Liverpool evaluation provides process evaluation results that can help improve the practice of future multilevel, multisector interventions, despite limitations of the outcome evaluation (Dawson et al., 2015). There is a growing literature on appropriate study designs and evaluation methods for complex interventions that extend more traditional quasi-experimental evaluation approaches, such as regression discontinuity designs, interrupted time series, and propensity score analysis (Shadish & Galindo, 2010).

The challenges of implementing and evaluating multilevel interventions are formidable. Their sheer complexity and unpredictability make them seem infeasible. The demands on leadership teams to manage complex multisectoral partnerships over time are substantial. There may be a poor fit with funding timelines. Evaluations may not meet criteria for high quality. But to solve the public health problems related to physical inactivity, it may be necessary to take on these challenges, even if the timeline is long, multisector partnerships are difficult to organize and maintain, and evaluations may not produce definitive results. Across studies, the weight of the evidence could be sufficient to advance public health science and practice, while informing policy decisions.

Recommendations for Advancing Multilevel Intervention Research and Practice

Many interventions conducted in practice are multilevel, and these should be evaluated. Community-based multilevel interventions have the great benefit of providing evidence from the real world, complex and unpredictable as it may be. There are great opportunities for practitioners and researchers to collaborate in generating practice-based evidence that may be more useful to practitioners and policy makers than more-controlled but less-realistic studies. Even when there is desire and capacity to evaluate real-world interventions, identifying funding sources is a major challenge.

Because of the nature of environment and policy changes, they cannot be randomly assigned to affect individuals. Thus, either complex cluster designs (i.e., randomizing neighborhoods) or simple pre–post designs with or without comparison conditions must be used. It would be helpful if funding agencies would set aside funds for the complex studies needed to evaluate multilevel interventions.

Because of the challenges of implementing and evaluating multilevel interventions, more opportunities for training should be developed during graduate school and for continuing education. Planning multilevel interventions should be incorporated into health promotion courses. Students in multiple disciplines should be provided internships with public health departments so trainees can gain experience planning and implementing multilevel and multisector interventions. Continuing education workshops and certification courses covering such topics as planning multilevel interventions, developing multisector collaborations, managing multisector coalitions, and conducting process and outcome evaluations of complex interventions would be valuable.

Conclusions

Ecological models, recommendations from many authoritative groups, growing evidence, and necessities of practice all support the need to implement well-planned multilevel interventions. The challenges to implementing and evaluating multilevel interventions are considerable, but the need to implement the most promising strategies for reducing the physical inactivity pandemic must be a public health priority. Multilevel interventions to promote physical activity are no longer theoretical. They are being implemented by public health, city planning, transportation, parks, and education departments, usually in collaboration with a variety of nonprofit groups, and often supported by foundations (Bors et al., 2009). There is mounting evidence that multilevel interventions can be effective, but investigators and practitioners need to improve the evidence through large field trials, evaluations of natural experiments, and application of newer approaches to quasi-experimental evaluations. Practitioners and researchers should be trained in multilevel and multisector interventions both during and after formal graduate programs.

Multilevel interventions are a logical next step in the evolution of health promotion practice and research. But we must realize that progress may require changes in training, funding priorities, support for multisector collaboration, incentives for investigators, and training to overcome the major challenges. Though the present article focuses on physical activity promotion, some of the points made are likely to generalize to other health behaviors, just as the physical activity field has benefitted from the lessons of decades of multilevel tobacco control efforts (Green et al., 2006).

Footnotes

Acknowledgements

I express my admiration for the foresight of Jim and the late Sarah Fries on establishing the James F. and Sarah T. Fries Foundation as well as the Elizabeth Fries Health Education Award. Very special thanks to David Sleet and Karen Glanz for nominating me for this honor. It was a treat to have David Sleet, Victor Bovbjerg, and Larry Green introduce the award and myself. I appreciate the guidance from Diana Robelotto from the CDC Foundation and Elaine Auld from SOPHE for making the arrangements for the award and lecture.

Author’s Note

The Elizabeth Fries Health Education Award Lecture on which this article is based was presented at the 69th Annual Meeting of the Society for Public Health Education on April 5, 2018, in Columbus, Ohio.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Preparation of this article was partially supported by the James F. and Sarah T. Fries Foundation, the CDC Foundation, and the Robert Wood Johnson Foundation. The funders had no involvement in the preparation of this article.

References

Bandura

(1986). Social foundations of thought and action. New York, NY: Prentice-Hall.

Barker

D. C.

Gutman

M. A.

(2014). Evaluation of Active Living Research: Ten years of progress in building a new field. American Journal of Preventive Medicine, 46, 208-215.

Bauman

A. E.

Reis

R. S.

Sallis

J. F.

Wells

J. C.

Loos

R. J.

Martin

B. W.

, on behalf of the Lancet Physical Activity Series Working Group. (2012). Correlates of physical activity: Why are some people physically active and others not? Lancet, 380, 258-271.

Bors

Dessauer

Bell

Wilkerson

Lee

Strunk

(2009). The Active Living by Design National Program: Community initiatives and lessons learned. American Journal of Preventive Medicine, 37(6 Suppl. 2), S313-S321.

Brown

W. J.

Mummery

Eakin

Schofield

(2006). 10,000 Steps Rockhampton: Evaluation of a whole community approach to improving population levels of physical activity. Journal of Physical Activity and Health, 3(1), 1-14.

Carlson

J. A.

Sallis

J. F.

Conway

T. L.

Saelens

B. E.

Frank

L. D.

Kerr

. . . King

A. C.

(2012). Interactions between psychosocial and built environment factors in explaining older adults’ physical activity. Preventive Medicine, 54, 68-73.

Dannenberg

Frumkin

Jackson

R. J.

(Eds.). (2011). Making healthy places: Designing and building for health, well-being, and sustainability. Washington, DC: Island Press.

Dawson

Huikuri

Armada

(2015). Liverpool Active City 2005-2010: Increasing population physical activity levels through intersectoral action. Journal of Physical Activity and Health, 12, 749-755.

De Cocker

K. A.

De Bourdeaudhuij

I. M.

Brown

W. J.

Cardon

G. M

. (2007). Effects of “10,000 steps Ghent”: A whole-community intervention. American Journal of Preventive Medicine, 33, 455-463.

10.

Ding

Sallis

J. F.

Conway

T. L.

Saelens

B. E.

Frank

L. D.

Cain

K. L.

Slymen

D. J.

(2012). Interactive effects of built environment and psychosocial attributes on physical activity: A test of ecological models. Annals of Behavioral Medicine, 44, 365-374.

11.

Glasgow

R. E.

Vinson

Chambers

Khoury

M. J.

Kaplan

R. M.

Hunter

(2012). National Institutes of Health approaches to dissemination and implementation science: Current and future directions. American Journal of Public Health, 102, 1274-1281.

12.

Green

L. W.

Kreuter

M. W.

(1999). Health promotion planning: An educational and ecological approach. Mountain View, CA: Mayfield.

13.

Green

L. W.

Orleans

C. T.

Ottoson

J. M.

Cameron

Pierce

J. P.

Bettinghaus

E. P.

(2006). Inferring strategies for disseminating physical activity policies, programs, and practices from the successes of tobacco control. American Journal of Preventive Medicine, 31(Suppl. 4), S66-S81.

14.

Hall

K. L.

Vogel

A. L.

Croyle

R. T.

(Eds.) (in press). Strategies for team science success: Handbook of evidence-based principles for cross-disciplinary science and practical lessons learned from health researchers. New York, NY: Springer.

15.

Kohl

H. W.

III Cook

H. D.

(2013). Educating the student body: Taking physical activity and physical education to school. Washington, DC: National Academies Press.

16.

Kohl

H. W.

3rd Craig

C. L.

Lambert

E. V.

Inoue

Alkandari

J. R.

Leetongin

, . . . Lancet Physical Activity Series Working Group. (2012). The pandemic of physical inactivity: Global action for public health. Lancet, 380, 294-305.

17.

Koplan

J. P.

Liverman

C. T.

Kraak

V. I.

(Eds.). (2005). Preventing childhood obesity: Health in the balance. Washington, DC: National Academies Press.

18.

Pucher

Dill

Handy

(2010). Infrastructure, programs, and policies to increase bicycling: An international review. Preventive Medicine, 50(Suppl. 1), S106-S125.

19.

Sallis

J. F.

Cervero

R. B.

Ascher

Henderson

K. A.

Kraft

M. K.

Kerr

(2006). An ecological approach to creating more physically active communities. Annual Review of Public Health, 27, 297-322.

20.

Sallis

J. F.

Cutter

C. L.

Lou

Spoon

Wilson

A. L.

Ding

. . . Orleans

C. T.

(2014). Active Living Research: Creating and using evidence to support childhood obesity prevention. American Journal of Preventive Medicine, 46, 195-207.

21.

Sallis

J. F.

Owen

(2015). Ecological models of health behavior. In Glanz

Rimer

Viswanath

(Eds.), Health behavior: Theory, research & practice (5th ed., pp. 43-64). San Francisco, CA: Jossey-Bass/Pfeiffer.

22.

Sallis

J. F.

Story

Lou

(2009). Study designs and analytic strategies for environmental and policy research on obesity, physical activity, and diet: Recommendations from a meeting of experts. American Journal of Preventive Medicine, 36(2 Suppl. 2), S72-S77.

23.

Shadish

W. R.

Galindo

(2010). Quasi-experimental design. In Weiner

E. B.

Craighead

W. E.

(Eds.), The Corsini encyclopedia of psychology (Vol. 3, pp. 1406-1408). New York, NY: Wiley.

24.

Stewart

Moudon

A. V.

Claybrooke

(2014). Multistate evaluation of safe routes to school programs. American Journal of Health Promotion, 28(Suppl. 3), S89-S96.

25.

Ward

D. S.

Welker

Choate

Henderson

K. E.

Lott

Tovar

. . . Sallis

J. F.

(2017). Strength of obesity prevention interventions in early care and education settings: A systematic review. Preventive Medicine, 95, S37-S52.

26.

Warner

K. E.

Mendez

(2010). Tobacco control policy in developed countries: Yesterday, today, and tomorrow. Nicotine & Tobacco Research, 12, 876-887.

27.

U.S. Department of Health and Human Services. (2001). The Surgeon General’s call to action to prevent and decrease overweight and obesity. Washington, DC: U.S. Government Printing Office.

28.

U.S. National Physical Activity Plan. (2018). Retrieved from http://www.physicalactivityplan.org/index.html

29.

World Health Organization. (1986). Ottawa charter for health promotion. Copenhagen, Denmark: Author.

30.

World Health Organization. (2004). Global strategy on diet, physical activity and health. Geneva, Switzerland: Author. Retrieved from http://apps.who.int/gb/ebwha/pdf_files/WHA57/A57_R17-en.pdf

31.

Yang

Sahlqvist

McMinn

Griffin

S. J.

Ogilvie

(2010). Interventions to promote cycling: Systematic review. British Medical Journal, 341, c5293.