Abstract
High-impact, transformative educational practices change the way students see themselves and others, as well as impart knowledge. Practices from the technology industry may offer innovative strategies for fostering transformational learning experiences. We developed and implemented two innovation techniques—Hackathon and Innovation Time Off (ITO)—in a graduate course on social psychology and public health nutrition. The Hackathon occurred in the sixth and seventh weeks of the course; the last 4 weeks provided 10% of class time for ITO projects. All enrolled students participated in the pilot study (n = 6; M age = 27.5 years; 83% female; 67% White) and completed reflection papers during the final exam period. Student learning was assessed from these reflection papers using a rubric for transformative versus nontransformative changes in five areas: confidence, pride, skills, perspective, and identity. Student responses revealed transformative changes in perspective (n = 7). Additionally, nontransformative changes were found in confidence (n = 1), identity (n = 2), perspective (n = 4), and skills (n = 9). This pilot work suggests that the Hackathon and ITO contributed to perceived skill-building in problem-solving and teamwork, and the Hackathon may have led to transformative changes in perspective; ITO may not be appropriate for learners who need structure; high confidence remained unchanged; and these strategies are likely to be feasible and replicable.
Transformational learning has been defined as learning that “changes the way people see themselves and their world” (Baumgartner, 2002 p. 16). High-impact, transformative educational practices are those that spark self-reflection, broaden frames of reference, soften rigid assumptions, and connect with students from diverse backgrounds (Dirkx et al., 2006; Kuh, 2008; Mezirow, 2003). Transformative teaching approaches can promote student retention of material, active engagement, and greater persistence (Slavich & Zimbardo, 2012). Specific strategies for transformative learning include problem-based or collaborative learning (e.g., team-based learning), flipped classrooms, structured self-reflection, cultural experiences, and activities requiring evaluation of personal assumptions (Enkhtur & Yamamoto, 2017; Kasworm & Bowles, 2012; Loyens et al., 2015; Slavich & Zimbardo, 2012). These are thought to enhance transformative learning by providing opportunities to practice collaborative critical thinking and communication skills, problem solving, and self-awareness development (Roberts et al., 2018; Slavich & Zimbardo, 2012; White & Nitkin, 2014). Many of the strategies studied have emerged from contemporary educational settings—K–12, higher education, and adult learning. However, little research has examined the efficacy of strategies developed outside of these fields. In particular, the field of technology, which demands a focus on innovation and collaborative problem solving, may offer new strategies for generating transformative learning.
Two specific innovation practices used in the technology field that may be well-suited to adaptation for the classroom are (a) Hackathons and (b) Innovation Time Off (ITO; Pope-Ruark, 2016). Hackathons can be defined as groups of people coming together for creative problem solving during a short period of time (MIT Hacking Medicine, 2016; Tauberer, 2014). While hackathons originated around computer programming problems, they have expanded to university settings for use addressing a range of topics. For example, Brown Hack Health 2019 brought together physicians, engineers, artists, and others around the theme of mental health (Wang, 2019). The winning solutions included a blanket-like covering for elderly patients that could address side-effects of dementia and using technology (e.g., Alexa or social sites) to ease symptoms of depression or other mental illness (Wang, 2019). ITO is time at work that is “free” for pursuing projects of personal interest. Google began this practice in 2004, allowing employees 20% of their time to work on projects of their choosing that they thought would benefit the organization (Page & Brin, 2004). It is credited with innovations such as Gmail (Lovejoy, 2013). Hackathons and ITO have been proposed as potential sources of high-impact learning in higher-education because they create opportunities for focused, immersive engagement; self-directed learning; and play (Pope-Ruark, 2016). To the authors’ knowledge, no evidence is available to date about whether these approaches lead to transformative learning in the college or graduate classroom, though interest in exploring these strategies is growing (e.g., Pollock & Avi Brooks, 2017). Studies are needed to develop and adapt these methods for the classroom setting.
This pilot incorporated these two innovation techniques into a higher education classroom. The overarching aim was to begin to explore the use of hackathons and ITO for fostering skill development and transformative learning experiences. Thus, this article focuses on the course components most relevant to those activities.
Design and Implementation of the Hackathon and Innovation Time Off
The framework for the course, called Applying Social Psychological Principles and Technology Innovation Methods to Address Public Health Nutrition Problems, was social psychology as applied to public health issues. The main learning objective relevant to the Hackathon and ITO project was the following: generate insights about public health problems, including clarifying the scope of a problem and proposing a novel solution.
The course was developed and taught at the Friedman School of Nutrition Science and Policy at Tufts University, which is located in Boston, MA, and offers master’s and doctoral programs in the social and biological sciences. It was offered in the spring semester of 2018 and enrolled six master’s students who all completed the course. Their mean age was 27.5 years (range 24–31 years), one was male and five were female, and two were Asian and four were White. Within the Friedman School of Nutrition Science and Policy, five of the students were enrolled in master’s programs in Nutrition Interventions, Communication, and Behavior Change and one was enrolled in Biochemical and Molecular Nutrition. Students self-enrolled based on interest and schedule availability. The course met for 3 hours weekly for 13 weeks. Students were told about the study on the first day of the course, and it was emphasized that their decision to participate was voluntary and would not affect their grade or any requirements of the course. All enrolled students provided consent to participate. All methods were approved by the Social, Behavioral and Educational Research Institutional Review Board at Tufts University.
The first 5 weeks of the course were focused on content from social psychology. Implementation of the innovation strategies began on the sixth week. This class was designed to teach students about problem-solving strategies and how to apply them. Students were taught three problem-solving processes: Design Sprint (i.e., Map the problem, Sketch multiple solutions, Decide one to move forward, Prototype creation, and Test with a real audience; Knapp et al., 2016), Design Thinking (Empathize, Define, Ideate, Prototype, Test; Hasso Plattner Institute of Design at Stanford University, 2017), and WRAP (Widen your options, Reality test your assumptions, Attain distance, and Prepare to be wrong; Heath & Heath, 2013). As there was overlap among these processes, students were also taught how to integrate them (see Table 1).
Timeline and Integration of Design Spring, Design Thinking, and WRAP Processes With Community Engagement.
The next two classes encompassed the Hackathon. As described in the syllabus (see Table 2), students were assigned to small teams. As a kick-off, guest speakers from a community public health organization posed a problem to the class. Students were expected to work on the problem for the intervening week. In the second Hackathon class period, teams gave a short report of their solution to the group.
Syllabus Descriptions of Hackathon and Innovation Time Off (ITO) projects.
In the kick-off class, staff from the Boston Public Health Commission (BPHC) presented a public health problem along with a consultant from Head Start. The issue presented to the students was the suboptimal health status and health behaviors of early childhood education teachers (e.g., Head Start teachers, daycare teachers). After the problem was presented, both Hackathon teams had 1 week to work on solutions to this issue. They were encouraged to utilize the consultant to ask questions, gain insights, and to obtain feedback on potential solutions. The consultant also helped connect students with other staff at local Head Start centers for additional insights and feedback. Boundaries were established in advance in terms of the number of emails per day and the times of day that a response could be expected; the Head Start consultant received an honorarium of $500 for her participation. Students emailed four to five periodic summary reports to the instructors and interacted with the consultant as they moved through the problem-solving steps. As an example, an initial update was as follows: Team 1: Our first meeting focused on mapping out and defining the problem. We took into consideration key actors as well as brainstormed some end goals. So far after narrowing it down we determined that our end goal will be: To increase the efforts of childcare centers to improve health and wellness of their educators.
During the second Hackathon class, teams presented their solutions and received feedback from instructors, peers, the BPHC staff, and the Head Start consultant. Team 1 proposed an educational event designed to enlist champions and eventually spur changes to the benefits package of the early childhood educators. Team 2 proposed implementing and conducting a pilot study of a wellness program for the early childhood educators and other staff that was modeled after a current program for Boston Public Schools personnel. The day after the second Hackathon class, a rough report was due that documented the team’s process and solution. The instructors evaluated the Hackathon based on the rubric (see the appendix).
ITO was implemented during the last 4 weeks of the course. Students were allotted 20 minutes or approximately 10% of the 180-minute class time for their projects. They were referred to the syllabus description to frame their thinking (see Table 2). They were also told that they were not expected to spend time outside of class on their projects, but if they chose to do so it should be limited to 1 to 2 hours per week. On the last day of class, students gave a formal presentation of their projects and submitted a written report.
All six students chose to use their ITO time on independent projects. Although we described the ITO in class and offered an opportunity to ask questions, we found that students had some initial difficulty with the open-ended nature of the project. We clarified expectations by meeting individually with two of the students, and as a result of these meetings, sent an email with feedback applicable to the whole class and further clarified the assignment in the following class. By the second week of ITO, all students had identified a problem to address, and no further clarification about the project was needed. Student projects are described in Table 3. The same evaluation criteria were used as for the Hackathon.
Innovation Time Off (ITO) Projects.
Implementation of both the Hackathon and ITO proceeded in accordance with a priori plans, as indicated in the syllabus. There were no substantial deviations to the procedures or timeline. The only minor alteration was the need to further clarify the ITO as described.
Assessment of Transformative Learning
Student learning experiences were evaluated using the rubric from Springfield et al. (2015) as applied to reflection papers that students completed during the final exam period. In the Springfield rubric, nontransformative changes are defined as being quantitative, that is, changes in extent or amount, while transformative changes are defined as being qualitative, that is, students are in some way fundamentally different than they were before: “The student recognizes that change occurred AND the change has substantially altered them in some way”; “I used to . . . but now I . . .” (Springfield et al., 2015, p. 74). The rubric also defines five types of changes, each of which may be transformative or nontransformative. These are confidence, or belief in one’s ability to use a skill; pride in accomplishment; skills, which could include any type of skill learned in the course; perspective, or how a student understands or perceives other people or situations; and identity, or how they understand or see themselves. In the reflection papers, students received the following prompts, which were adapted from Springfield et al. (2015):
Consider your experience in this course this semester. Please describe it in one to two paragraphs.
After you’ve done that, use the questions below as a guide to write more about aspects of your experience. The questions are a guide but are not an exhaustive list. If different issues spring to mind, please feel free to describe those as well. Did you have any a-ha moments this semester? What does it mean to work toward improvements in public health nutrition? Have your views have changed compared to the start of the semester? If so, how? What does it mean to be a productive member of a team? Have your views changed compared to the start of the semester? If so, how? What do you see as the role of reflection or self-reflection in your profession moving ahead? Did you learn anything that surprised you?
A priori, based on the goals of the course, we expected changes in confidence, skills, and perspective. We did not necessarily expect changes in identity in adult learners within the timeframe. The course was not specifically designed to increase pride in accomplishment, which typically results from praise feedback. To code the reflections, a research assistant (TF) created a data shell in Microsoft Excel with all student responses. The two instructors (ES and SF) and the research assistant independently coded the type of change (i.e., confidence, pride, skills, perspective, identity) and whether it was transformative or nontransformative, and entered any comments. At the completion of coding, all three met to resolve any discrepancies through discussion and to determine a consensus code.
Reflection papers revealed few student-reported instances of changes in confidence (n = 1) and identity (n = 2; Figure 1). As expected, no changes in pride were found. Students described nontransformative changes in skills (n = 9), mainly in problem solving and working effectively in a team.

Frequency of student-reported changes in confidence, pride, skills, perspective, and identity by transformation status.
I really liked learning about problem-solving. I think that is one of the biggest things I learned from this class; how to go about solving a problem in a structured and productive process. (Student reflection paper)
There were 11 changes in perspective, and seven of these were transformative. Transformative change occurred for five of the six students, with two students describing two instances. Students attributed changes in perspective to both the Hackathon and other course activities.
I think I have a new appreciation for just how many facets there are when thinking about behavior change, public health nutrition, and improvements in our systems. The Hackathon was the best experience that highlighted just how many levels you would need to go through to influence healthy behavior. (Student reflection paper)
Lessons Learned
This was a pilot study and findings rest on data from a small class of six highly motivated graduate students. Lessons learned are meant to provide additional insight on this limited experience and to suggest hypotheses for future work.
The Hackathon and ITO contributed to perceived skill-building. In their reflection papers, students attributed an increase in problem-solving skills to both activities and an increase in teamwork skills to the Hackathon. Both skills are important in public health and many other disciplines, and on that basis, these strategies are worthy of further consideration for incorporation into the classroom.
The Hackathon contributed to transformative changes in perspective. Students attributed transformative changes in perspective to the Hackathon, but not to ITO. There are several possible reasons for this. In the Hackathon, community engagement and cultural inclusion were emphasized, compared with the ITO which allowed students to pursue an idea within their own interests. Also, working in a Hackathon team may have forced students to self-reflect when their suggestions were questioned by others, which would not occur with the ITO project. Furthermore, there is some literature that links team-based learning among nurses and midwives with higher engagement and development of the skills required for transformational learning (Dearnley et al., 2018). Intensive group work has also been used in the health sciences and social work to spur students to engage with one another while grappling with how the problems in their field interact with society (Green, 2019; Roberts et al., 2018). This suggests that teamwork combined with problem-solving as in the Hackathon may be an important route for bringing about transformative learning experiences, and this could be further explored in larger studies.
ITO may not be appropriate for all learners. The Hackathon had more structure and the nature of the assignment seemed clear from the start. The purpose of the ITO was less apparent, and our students felt some initial discomfort with the open-ended nature of it, even after having learned and applied the problem-solving strategies for the Hackathon. They were more comfortable after we provided additional guidance. This suggests that ITO may not be effective for students who require a high level of structure.
There was little change in confidence. Our second-year master’s students likely had a high level of confidence on entering the course. The impact of the Hackathon and ITO on change in confidence should continue to be explored, both pre–post (rather than retrospectively) and with other student populations.
These strategies are likely to be feasible and replicable. Factors that would likely affect the ease of implementing Hackathons and ITO projects in other courses include availability of funding, stakeholder gains, and time (upfront preparation and in-class). Although the consultant was paid, she initially agreed to participate without pay. Consultants may also be willing to participate for a smaller stipend that could potentially be covered by departmental funds. Our BPHC partners and the consultant all found the experience worthwhile and enjoyed it. As instructors, we had limited personal experiences with Hackathon procedures and no experience with ITO, and were able to construct a process based on publicly available resources (Hasso Plattner Institute of Design at Stanford University, 2017; Heath & Heath, 2013; Knapp, 2017; Knapp et al., 2016; MIT Hacking Medicine, 2016; Tauberer, 2014). Overall, the Hackathon did not require excessive preparation compared to other teaching activities. Planning for the Hackathon involved setting aside the course time for the practice session, kick-off and wrap-up; creating the Process map (Table 1); and coordinating with public health professionals to present a problem and with a target audience member to provide feedback. The ITO required only setting aside in-class course time and responding to questions. Another feasibility consideration is the ability to preserve and prioritize the ITO. This was challenging and took extra planning for the other elements of the course and careful attention to time during class. We did not feel that ITO had a detrimental impact on instructional time.
Conclusion
This is the first published evaluation of a Hackathon and ITO in the education context. Results indicate feasibility and suggest the promise of these approaches, particularly the Hackathon, for perceived skill-building and transformative changes in perspective. Future research is needed to further refine these strategies, better understand underlying mechanisms, and fully evaluate their potential.
Footnotes
Appendix
Grading Rubric for Hackathon From Syllabus.
| Components | Expected level of competence (“check-plus”) | Moving toward expected level of competence (“check”) | Below expected level of competence (“check-minus”) |
|---|---|---|---|
| Timely submission of assignment | Report is submitted on time. | Report is submitted late. | Report is submitted very late (one day or more after due). |
| Style guidelines | Writing is clear and succinct. Demonstrates care in writing, which may be informal, but thoughtful. The memo has a smooth flow of ideas and is well organized and logical. | Writing lacks clarity in some places. Flow of ideas could use some improvement. | Writing lacks clarity throughout. Flow of ideas is rough. Memo is not well organized. |
| Demonstrates inquiry into problem and effort toward devising a solution | Demonstrates clear implementation of the design process and serious contemplation of the best solution, synthesized from the process. Shows original thought. | Demonstrates that the design process was followed and the proposed solution shows some integration of the process. | Design process was cursorily included, but report gives little indication that it was followed very carefully or that the proposed solution aligns with the results of the process. |
Acknowledgements
We would like to acknowledge the students who participated in this experimental course; Alison Brown for her work to create the case studies; Mark Kennedy and Sonia Carter of the Boston Public Health Commission and Kelley Bennett from Head Start–Mattapan.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by a Tufts Innovates grant from Office of the Provost at Tufts University, awarded to authors Sara Folta and Eleanor Shonkoff.
