Local Data for Action: Statewide Dissemination of School Wellness Policy Evaluations in Wisconsin

Background

Schools are key environments that promote children’s health through various mechanisms, including district school wellness policies (SWPs). School districts participating in the National School Lunch Program or the School Breakfast Program are required to develop a SWP governing obesity prevention best practices such as nutrition education, physical education and activity, nutrition guidelines for school meals and competitive foods, wellness promotion and marketing to students, and a triennial SWP assessment (U.S. Department of Agriculture [USDA], 2016). In Wisconsin, the average SWP quality is low, characterized by vague policy language (Weymouth et al., 2019), demonstrating the importance of improving the quality of SWPs statewide.

While many studies have evaluated SWP quality (M. J. Cox et al., 2016; Hood et al., 2013; Meendering et al., 2016; Piekarz-Porter et al., 2017; Schwartz et al., 2012; Taber et al., 2012; Weymouth et al., 2019), little is known about the process of disseminating tailored SWP evaluation results to school districts statewide. Indeed, within public health there is a research to practice translation gap (Green et al., 2009), partly due to unsuccessful dissemination (Brownson et al., 2018). Studies suggest that school systems want local data for action. For example, school district superintendents have reported a need for better access to SWP evaluation tools and resources (Asada et al., 2021). In California, school board members reported that success stories from other districts, best practice information, and sample policies would be helpful to improve physical activity-related policies (L. Cox et al., 2011). Few studies document dissemination efforts of SWP evaluations, although for research to be actionable, it must be translated for local audiences.

In general, dissemination efforts to influence school policy have been successful. For example, one study noted that report cards highlighting physical activity indicators sent to a broad physical activity policy audience, including school stakeholders, was an effective knowledge translation strategy (Colley et al., 2012). In a similar study, school-specific reports featured research findings from a student attitude survey and an environmental scan that supported schools in enhancing their evidence-based physical activity programming and policies (Leatherdale et al., 2009). To influence tobacco-free school policies in South Dakota, researchers successfully disseminated tailored policy evaluation reports including strengths and weaknesses of the school’s tobacco-free policy and recommendations for policy improvement (Fahrenwald et al., 2013). While these results present the success of dissemination efforts within the broad field of student health, limited research exists that details the dissemination processes of translating SWP research to actionable practice guidelines.

This study describes the statewide dissemination of SWP evaluation results tailored to individual Wisconsin public school districts. As part of a larger obesity prevention initiative, researchers with the Wisconsin Health Atlas conducted two waves of statewide SWP evaluations (Joyner et al., 2020) to document the quality of Wisconsin districts’ school board approved SWPs before and after the Healthy Hunger Free Kids (HHFK) Act Final Rule (USDA, 2016) was published in July 2016. Wave I of SWP collection and evaluation occurred between May 2016 and April 2017 and included SWPs that were published prior to July 2016. Wave II occurred from September 2017 through May 2018 and included SWPs that were published after July 2016. SWP quality was evaluated using the WellSAT 2.0 (Schwartz et al., 2009), a quantitative coding scheme for assessing the quality of SWPs developed by the University of Connecticut Rudd Center. Beyond documenting changes to the quality of Wisconsin SWPs before and after the HHFK Final Rule was published, the goal of the Wisconsin Health Atlas SWP evaluation project was to promote policy quality improvement by sharing actionable, local data and best practices with school districts. To ensure that districts could access and interpret their local SWP data, the SWP evaluation project prioritized translation and dissemination through tailored district data reports and a website that included local, regional, and statewide interactive data dashboards.

Aims

The aims of the current study are to describe the process of disseminating tailored SWP data reports and interactive data dashboards to Wisconsin public school districts and to answer the following research questions:

Method

Development of the SWP Data Report

After completing Wave II of policy collection and evaluation (Joyner et al., 2020), we utilized Lavis’s framework for knowledge transfer (Lavis et al., 2003) to guide the development and dissemination of district-specific SWP data reports. Lavis’ framework offers the following questions: (1) What should be transferred to decision makers? (2) To whom should research knowledge be transferred? (3) By whom? (4) How? and (5) With what effect? During the dissemination planning period, we leveraged established partnerships with Team Nutrition at the Wisconsin Department of Public Instruction (WI DPI) and with a statewide network of partners working on healthy eating and physical activity in schools to contribute to the credibility of our message. Throughout Waves I and II, all 442 public districts were asked to supply a school board approved SWP (Joyner et al., 2020). The report’s target audience encompassed SWP stakeholders at the 232 school districts who supplied SWPs during Wave II of policy collection. The WI DPI Team Nutrition advised that in Wisconsin, these stakeholders vary among districts and could include district superintendents, nutrition directors, school nurses, wellness coordinators, business managers, and school board members. To inform our dissemination activities, we created a dissemination plan as seen in Table 1.

OPEN IN VIEWER

Table 1

SWP Evaluation Project Dissemination Plan

Component	Descriptions for SWP evaluation project
Target audience	Wellness stakeholders involved with the SWP revision process.
Goal	Provide SWP stakeholders with actionable and accessible SWP data, including key improvement areas and information on best practices, to support Wisconsin school districts in improving the quality of their SWP.
Primary message	Understand district’s SWP quality, identify improvement areas, review recommendations, and access the interactive data dashboards to support the quality improvement of the SWP.
Secondary message	Utilize this data report as a base for your triennial assessment.
Audience engagement	Formative feedback requests via needs survey and usability survey; tailor data reports, dashboards, and communication for each district.
Messengers	Wisconsin Health Atlas; WI DPI Team Nutrition.
Materials	Electronic data report; hard copy data report; interactive data dashboard.
Channels	Email; U.S. mail; WI DPI Team Nutrition communications.
Timing	Formative feedback request Summer and Fall of 2018; official data reports and dashboards dissemination in March 2019.
Evaluation	Follow-up survey seeking information on the utilization of the tailored data reports and data dashboards; reach of electronic data report, hard copy data report, and data dashboards.

Note. Components adapted from Lavis et al. (2003) framework for knowledge transfer. SWP = school wellness policy; WI DPI = Wisconsin Department of Public Instruction.

To further understand the needs of our target audience, we crafted an electronic, six-item survey seeking feedback about the types of data and information that would be most useful for school districts. The needs survey was included in a May 2018 Wisconsin Team Nutrition newsletter emailed to the Team Nutrition’s statewide school nutrition email list, consisting of 597 school nutrition professionals statewide. The anonymous responses (n = 20) from this survey indicated that the most valuable topics to include in the report were the following: identifiable areas for policy improvement, links to online policy improvement resources, individual district data on federal mandate adherence, and comparison of district’s evaluation results to the statewide average. The needs survey was used to support our report design and is beyond the scope of this article but is available on request to the corresponding author.

The SWP data report template was created in July 2018 and included the topics identified in the target audience needs survey, as well as the HHFK Act Final Rule requirements and local Wisconsin success stories that highlighted districts who made sustainable improvements to their SWP. To test the usability of the information presented in the report, an electronic, six-item survey was distributed to 16 school wellness partners in the statewide network in October 2018, with a goal of receiving eight (50%) responses. The school wellness partners were suggested by the network coordinator as their respective districts differed in locality and size. We received anonymous surveys from nine (56%) school wellness partners. The data visualizations and report content were updated following the survey to reflect the formative feedback. The report template was finalized in January 2019, and the 232 district-specific reports were finalized in March 2019. The interactive data dashboards featuring additional district-specific, regional, and state-level data were finalized in February 2019.

Results

Tailored SWP Data Reports: Final Product

Emphasizing the translation of technical language for accessibility and relevancy, the data report included personalized policy improvement recommendations, examples of how to use the report for action, links to policy improvement online resources, and information about the HHFK Act Final Rule federal mandates and the obesity prevention best practices. Selected pages of the report are found in Figure 1. The full report template is available on request to the corresponding author. The report contents included (1) an executive summary explaining the statewide SWP evaluation study, (2) district-specific data from the policy evaluation study including the district’s adherence to the HHFK Act Final Rule federal mandates compared with the statewide average, (3) the district’s requirement of obesity prevention practices compared with the statewide average, (4) a unique code to enter in the Wisconsin Health Atlas data dashboards to explore additional district-specific, regional-, and state-level data; and (5) detailed success story narratives from four Wisconsin districts that were identified through our Wave I to Wave II data analysis as having made significant improvements to their SWPs. The success story narratives presented in the report were derived from semistructured interviews conducted by the authors and included descriptions of their local mechanisms for policy improvement, images, and quotes from the identified districts who agreed to be featured (Joyner et al., 2020).

OPEN IN VIEWER

Figure 1

Example Pages of School Wellness Policy (SWP) Data Report Featuring the Cover, the District’s Adherence to Final Rule Mandates, and the District’s Policy Quality Compared With the Statewide Average

Interactive SWP Data Dashboards: Final Product

The Wisconsin Health Atlas website (www.wihealthatlas.org/lwp) features interactive data dashboards as companions to the SWP data reports, presenting additional data and information not found in the tailored report. Selected data dashboards are found in Figure 2. The dashboards include the following research translation components:

Individual district data, including policy quality change over time, visualized in Tableau Desktop 2019.1 (Tableau Software, Seattle, WA)

Regional and statewide data, visualized in Tableau Desktop 2019.1

Best obesity prevention practices information, example policy language from Wisconsin districts, and links to additional SWP improvement resources

An interactive map of the success story districts created in Story Map Tour feature embedded in ArcGIS Online (ESRI, Redlands, California)

Wisconsin Health Atlas information

OPEN IN VIEWER

Figure 2

Example Data Dashboards Featuring District-Level, Regional-Level, and State-Level Data Available on the Wisconsin Health Atlas Website

Reach of Electronic and Hard Copy Tailored Data Reports

The dissemination email containing the SWP data reports was successfully delivered to all 232 districts. Characteristics for districts that received a report are described in Table 2. Among the 232 districts, 213 districts (92%) opened their tailored dissemination email. Five hard copies of the tailored reports were mailed to each district superintendent, yielding a total of 1,160 paper reports mailed across the state.

OPEN IN VIEWER

Table 2

Characteristics of Wisconsin School Districts That Received a Tailored SWP Data Report (n = 232) and Answered a Follow-Up Survey (n = 47)

Characteristic	Statewide (n = 442)		Received report (n = 232)		Answered survey (n = 47)
	n	%	n	%	n	%
District size (Enrollment)
Small (1–2,499)	362	81.9	185	79.7	39	83.0
Medium (2,500–9,999)	71	16.1	44	19.0	7	14.9
Large (10,000+)	9	2.0	3	1.3	1	2.1
District type
K–12	372	84.2	201	86.6	42	89.4
K–8	52	11.8	27	11.6	5	10.6
Other	18	4.1	4	1.7	0	0.0
Locale
City	38	8.6	7	3.0	1	2.1
Suburb	78	17.7	46	19.8	5	10.6
Town	95	21.5	55	23.7	17	36.2
Rural	231	52.3	124	53.5	24	51.1

Note. District size was determined using enrollment data for the 2017–2018 school year from the WISEdash Wisconsin Education Statistics portal. One school district was missing enrollment size in that data set, and enrollment was taken from the October 2017 Enrollment and Participation Reports (https://dpi.wi.gov/school-nutrition/program-statistics). Data on urban/rural classification is from the 2018–2019 Wisconsin School District Locale Codes, Congressional District Number, County Code Directory (https://dpi.wi.gov/wise/federal-reporting). SWP = school wellness policy.

Evaluation of Data Reports and Dashboards: Follow-Up Survey

Approximately 20.3% (n = 47) of districts who received a data report responded to the utilization follow-up survey. Respondents’ roles included school nurses (38.3%), nutrition directors (19.1%), wellness coordinators (12.8%), district superintendents (10.6%), business managers (6.4%), physical education instructors (4.3%), teachers (4.3%), principals (2.1%), and school board members (2.1%). Characteristics of the districts who responded to the follow-up survey are described in Table 2.

Most respondents (74.5%) had viewed their district’s tailored data report. All five respondents who indicated their role as a district superintendent confirmed that they received and viewed the mailed, hard copy report. Among respondents who indicated they had not viewed their report, lack of time was the most common reason (50.0%) followed by irrelevant content (25.0%) and other (25.0%).

Most respondents (81.9%) agreed that their district’s SWP data report would help their district take action to improve their SWP. Most respondents (86.4%) indicated they had used the information in the report or were planning on using the information in the report. Respondents who indicated they had used the report at the time of the follow-up survey were asked how they had used their report and could select more than one response option. Respondents reported using the report to identify areas for policy improvement (50.0%), to include in their triennial assessment (50.0%), to bring to a school board meeting (25.0%), to start a wellness committee in their district (25.0%), and other (25.0%). In an open-ended question, one district shared how their superintendent had used the report as a starting block to revamp their SWP. Respondents who indicated they had viewed their report were asked about the usefulness of the different report components, as displayed in Figure 3.

Over two thirds of the respondents (68.8%) had not viewed the Wisconsin Health Atlas SWP interactive data dashboards. Lack of time (54.5%) and being unaware of the resource (22.7%) were the leading factors for not viewing the data dashboards. Displayed in Figure 3, respondents who indicated they had viewed the data dashboards were asked about the usefulness of the different dashboard components

OPEN IN VIEWER

Figure 3

Mean Usefulness Rating of Report and Dashboard Components Among Follow-Up Survey Respondents Who Viewed the Print or Electronic Report (n = 35) or the Interactive Dashboard (n = 15)

Discussion

With 232 tailored SWP data reports disseminated to Wisconsin school districts, this study describes the statewide dissemination effort to support SWP change by using local data for action. The three dissemination materials include emailed electronic data reports, mailed hard copy data reports, and the interactive data dashboards. Each of the 232 school districts received a tailored report specifically for their district’s SWP including their policy’s evaluation data and personalized recommendations for policy improvement. Most districts that received a tailored report were small, K–12 districts in rural or town areas, similar to the statewide district characteristics.

Districts that responded to the follow-up survey tended to be K–12, small districts in rural or town areas, which is similar to the characteristics of all districts that received a tailored data report. Overall, the feedback about the data report from districts was positive. Most districts who responded to the survey agreed that the information presented in the report will help their district take action to improve their SWP and indicated they either have used the report or are planning on using the report to facilitate policy change. Districts who responded to the survey indicated that the most useful components of the report were the district comparison to the statewide average, the information about the federal mandate, and their district’s local data. Among respondents who used the report at the time of follow-up, the most common activities included identifying areas for policy improvement and using the report to support their triennial assessment, suggesting that districts value the technical support in completing their SWP evaluations.

To facilitate the use of research findings, Brownson et al. (2018), Butler et al. (2010), and Green et al. (2014) emphasize the importance of building relationships with the target audience, considering the needs of the target audience, and shaping the dissemination product to be most useful to the target audience. In our study, we embedded feedback from school wellness stakeholders early and throughout the dissemination planning process. This valuable feedback shaped the information, data visualizations, and recommendations in our reports, which may have contributed to the positive feedback regarding utilization of our findings.

Aligning with the credible messenger component of Lavis’s framework (Lavis et al., 2003), working with the WI DPI gave our communication efforts credibility and may have increased engagement both with our evaluation efforts and with the dissemination of our results. As part of the HHFK Act Final Rule, school districts must evaluate progress on their SWP and complete a triennial assessment (USDA, 2016). The WI DPI assured school districts that our data reports could be used to help meet this requirement, which may have further encouraged districts to engage with our data reports. Nearly all districts had opened their email containing their tailored data report and most respondents of the utilization follow-up survey had viewed their electronic report, indicating that email was an effective way to reach districts and school wellness leaders. To further encourage districts to open the dissemination email and view their tailored data report, it may have been beneficial to build relationships with additional credible messengers, such as the local School Nutrition Association or the local Association of School Nurses.

While we do not know the full extent of use of the mailed paper reports, all five district superintendents that responded to our utilization follow-up survey indicated they had received and viewed the hard copies. In contrast, although we saw an increase in views of the Wisconsin Health Atlas SWP website, only a third of respondents indicated that they had used the data dashboards and resources on the website. Because of the limitations in time, resources, and technology that some school districts face, our dissemination of hard copy reports may have encouraged engagement with our evaluation results that would have not occurred with exclusively electronic reports and electronic dashboards. This highlights the importance of providing dissemination materials in multiple formats, echoing several studies that recommend developing a suite of dissemination products to promote the use of research findings (Brownson et al., 2018; Butler et al., 2010; Dobbins et al., 2007).

The most common roles of the follow-up survey respondents were school nurses, nutrition directors, wellness coordinators, and district superintendents, indicating that school wellness leaders may emerge from different professions. In the future, it may be effective to mail the reports directly to school nurses as school nurses were the largest survey respondent demographic. To influence school health policies, researchers should collaborate with diverse school wellness stakeholders and encourage policy change by disseminating actionable data, reliable information, and specific recommendations to multiple stakeholders within a district. This type of effective dissemination strategy typically requires a great time commitment to ensure the materials and tactics are accessible and relevant for the target audience (Lavis et al., 2003). In our study, the Wisconsin Health Atlas had the staff-time and funding to successfully plan and execute the statewide dissemination strategy.

There were several limitations of our study. First, our results are specific to Wisconsin public school districts and not generalizable to other states or private schools. Second, website user spike could have also been due to other sources directing individuals to the Wisconsin Health Atlas website, but the lack of hits to the SWP data dashboards in the days leading up to the dissemination and the spike on dissemination day lends credence to the effectiveness of our dissemination efforts in attracting users to the website. Third, the responses for the utilization follow-up survey may reflect nonresponse bias as districts that did not view their SWP data report may have opted to not respond to the survey. Finally, the follow-up survey response rate is low but similar to rates reported in other studies that describe emailed surveys to school district staff (L. Cox et al., 2011; Madariaga et al., 2017; Mansfield & Savaiano, 2018).

Implications for Research and Practice

With a goal of supporting school districts in improving the quality of their SWPs to prevent childhood obesity, this study offers insight into a practical SWP evaluation dissemination strategy for research organizations. Emphasizing the practicality of research translation tools, this study adds to the current literature that suggests districts value receiving local data in an easy to understand format, tailored recommendations, and policy evaluation technical support. To encourage research translation, organizations can prioritize dissemination early in their projects and develop relationships with credible organizations and potential stakeholders. The process should include collaborating with the target audience to understand the types of data, visualizations, and communication channels that are most useful for local policy improvement. Although our study is only representative of Wisconsin public districts, other states can adapt our process to disseminate SWP data to districts statewide. Future steps include disseminating the regional-level data to state-level policy makers. Further research is needed to elucidate how local data for action contributes to long-term organizational policy changes.