Evaluation of a Joint Health Communication Campaign Simulation between an Ad/PR Capstone Course and a Health Communication Class

Abstract

A quasi-experimental study was conducted to assess attainment of assignment objectives plus course-level cognitive and affective learning for a joint simulation between an advertising/public relations capstone course and a health communication class. Two treatment and two control sections of the capstone course were compared over a period of two years. In comparison to the control condition, students in the treatment sections reported themselves more likely to continue to stay abreast of information about the subject matter covered in the course than students in the control condition, and viewed their campaigns as of higher quality. Differences with respect to confidence about ability to produce health communication campaigns in the real world, as well anticipation that they would talk about the topic after the semester was over, approached significance. No differences in cognitive learning emerged.

Just more than a decade ago Coombs and Rybacki (1999) called for increased pedagogical research in the field of public relations. Noting that the educational process typically is divided into four major components - outcomes, curriculum, pedagogy and assessment - they asserted that only three of these had received attention in the field. “Pedagogy, the delivery of course content,” they stated, “is the forgotten element in discussions of assessing public relations education” (p. 56). Coombs and Rybacki based this call on their analysis of data collected at a summer conference on public relations education conducted by the National Communication Association in 1998. At that time they concluded that although pedagogical practices in the field placed a laudable emphasis on active learning, empirical research on the effectiveness of pedagogical approaches was sparse. This paper responds to Coombs and Rybacki by reporting the results of an evaluation of a problem-based learning approach to a health communication campaign assignment, jointly conducted by an advertising/public relations capstone course and a health communication class.

Problem-Based Learning

History of PBL

One pedagogical model that has gained attention in recent years in some marketing programs is problem-based learning (PBL). Originally formalized in medical schools in the 1950s and 1960s, PBL was a reaction to the prevailing practice of filling students like a bucket with a broad range of knowledge from which they were expected to retrieve the appropriate cup or spoonful when a specific case called for it. In contrast, PBL presented students with complex historical cases typical of real patients, which students then worked in teams to solve. By forcing students to struggle with multistate, complex cases, PBL was assumed to provide a memorable context for learning not only the primary subject matter, but also a range of “soft” skills, such as research, teamwork and written and oral communication. Use of PBL spread to other science education contexts such as engineering, and finally into the humanities and social sciences (Allen, Donham & Bernhardt, 2011). Some (Wee, Alexandria, Kek & Kelley, 2003) have argued that PBL approaches, in combination with team-based learning models (Chad, 2012), are better suited than traditional approaches to prepare graduates for careers in communication, especially in graduate-level courses.

Evaluation of PBL in medical schools has yielded mixed results, with some meta-analyses suggesting that PBL is generally superior to traditional models (see Vernon & Blake, 1993) and other reviews indicating it has modest or no beneficial effect on student learning of course content (see Albanese & Mitchell, 1993). Recent and more nuanced assessments, though, suggest that students with PBL experience may be advantaged when it comes to linking concepts and principles to applications (Gijbels, Dochy, Van den Bossche & Segers, 2005) and critical thinking (Tiwari, Lai, So & Yuen, 2006). Students themselves tend to report positive attitudes toward PBL, citing it as challenging and motivating (Vernon & Blake, 1993). Less research is available about the effectiveness of PBL in undergraduate classrooms, or in fields outside of medicine. However, pedagogies that encourage activity and student involvement generally are believed to increase both cognitive and affective learning (Lubbers & Gorcyca, 1997).

Role of faculty

PBL shifts the role of instructors from expert sources of information to facilitators of problem solving. The key task of faculty is to construct problems that pose cognitive challenges for students but within which instructors can provide monitoring, feedback and, when necessary, intervention. In undergraduate classes, PBL can take a wide variety of forms. One of the most popular is simulation or “participating in a very real learning experience that closely resembles an actual setting” (Clapper, 2010, p. 39). Simulations can range from use of computer modeling in business courses to role-plays with standardized patients in medical and nursing education. Proponents point to the active learning, emotions and reflections characteristic of simulations as key to developing critical thinking skills required for the workplace (Stewart, Williams, Smith-Gratto, Black & Kane, 2011).

Use of PBL in public relations/advertising

The teamwork skills required for simulations also can be valuable for undergraduates making the transition from classwork to the team atmosphere in the professional world of advertising, public relations and communication. Advertising or public relations capstone courses are a natural place for integrating such experience, as they typically demand hands-on work that is executed in teams. Resulting portfolios are useful in assessment of program outcomes but are also helpful to students in job interviews.

Ad/PR Capstone Course Design

Value can be added to the curriculum by enabling students to gain experience working with specific topic areas outside of their own major, to simulate collaboration with other departments in larger organizations. However, credit hour limits created obstacles to adding new and specific courses to the curriculum. This was the situation at a Florida institution two years ago when the instructor of a new undergraduate health communication course in the interpersonal/organizational specialization area approached the Ad/PR specialization area with a request for collaboration¹. Like many communication departments across the country, that university's department at the time offered a single introductory course in health communication (Query, Wright, Bylund & Mattson, 2007). This made it challenging for faculty to create experiential assignments for health communication students, because many of them did not come with the technical or design skills required to undertake actual health communication projects. Convinced that a mentored experience of health communication would enhance students’ learning (Kirschener, Sweller & Clark, 2006), and that working in teams across disciplines would benefit the Ad/PR students, the authors of this article developed a joint simulation assignment between students in one of their advertising/public relations capstone campaigns classes and students in the health communication course.

Advertising/PR is one of four majors within the Nicholson School of Communication at the University of Central Florida. The school of communication is broken into four separate “areas” – essentially, small departments – which mount the separate majors.

The Assignment

Ad/PR program capstone course description

The Ad/PR program capstone course already was based largely on a PBL approach. On the first day of class, students self-evaluate their strongest skill set out of the following possibilities: art direction (design of materials), copywriting (promotional writing of materials to consumers), research (both primary and secondary data collection), public relations (creating traditional media materials as well as social media applications) and account executive (organizing, tracking, keeping the team on task and deadlines). As students are required to have one professional internship before enrolling in the course, they have a basic understanding of these roles, but they are also given job descriptions of each. Instructors can adjust roles as strengths and weaknesses of the students become clearer throughout the term.

In a 15-week semester, the five-student teams produce three integrated campaigns that grow in intensity and quantity of materials produced. Each campaign is on a topic determined by the instructor. The first campaign is usually about something entertainment-related (for example, a garage band wanting to become famous) to sharpen students’ understanding of buzz and viral communication tactics. The second campaign is on a lesser-known actual product of the students’ choosing and folds into the viral tactics used in the first campaign's partnerships, sponsorships and other forms of promotion. The third campaign is on a public health topic.

Project grades are determined through faculty assessment of materials with a grading rubric. Scores are awarded in priority order: extensiveness of research analysis to develop strategy, adherence to strategy in execution of promotional materials, the quality of produced materials (writing quality: clear, concise, correct) and the quality of a live presentation.

Health campaign project description

The health campaign project, which is the topic of this article, replicates the typical real-life health communication design process in which public health related organizations frequently outsource creative work to public relations firms or advertising agencies (U.S. HHS, 2009). In the simulation, groups of students from the health communication class act as strategic planning teams, while groups from the Ad/PR campaigns class act as creative teams. The assignment is based on guidelines from the Centers for Disease Control and Prevention (2012), as well as prominent health communication scholars (Booth-Butterfield, 2011; Fishbern & Cappella, 2006). It leads students through the development of a mass media campaign, from inception through formative research and creative efforts, and, finally, to pretesting messages with members of the target audience. Strategic teams in the health communication class: 1) conduct a literature review; 2) identify intended audiences and audience segments; 3) conduct theory-based formative research about audience knowledge, attitudes, behavior and media use; and 4) create health communication objectives. Background research and objectives are presented to the creative teams in the Ad/PR class, which develop proposals for multimedia campaigns. After message testing by the health communication students, resulting campaigns are presented to the class as well as guests from the industry. Ad/PR students gain experience in listening, evaluating and creating materials that align with the analysis of the health communication students. Both classes develop skills in research and working in multidisciplinary teams.

Hypothesis and Research Questions

In sum, the simulation that is the subject of this report pushes the existing PBL approach of the capstone course even further toward real world experience. From the beginning it was clear that the health communication students responded positively to involvement in an extended simulation. However, it was less clear how valuable the project was to students in the Ad/PR class, given that they were already required to conduct some level of formative research about their campaigns projects. The instructors expected that because the simulation uses an assortment of methods associated with student engagement, students who experience the simulation would be more engaged than those who did not. Therefore, the following hypothesis emerged:

H1: Students in the treatment condition as compared to the control condition would report higher levels of engagement as indicated by: a) likelihood of using information in real life; b) likelihood of discussing the content after the semester is over; and c) likelihood of keeping abreast of issues addressed in the course.

Because of the paucity of literature addressing undergraduate student learning outcomes in PBL the instructors also posed two research questions:

RQ1: Would self-reported attainment of health communication campaign assignment objectives differ between the treatment and control conditions?

RQ2: Would other aspects of cognitive and affective learning of students in the course as a whole differ between the treatment and control conditions?

Method

To investigate these issues the researchers set up a quasi-experimental non-equivalent control group study. Data were collected in four sections of the course, over a period of four semesters. Two sections of the course served as the treatment condition and were given the collaborative assignment with a section of the health communication class. Two other sections served as the control condition and engaged in no collaboration. Ethical permission was obtained from the IRB at the institution. All four Ad/PR campaigns classes were taught by one instructor, and both health communication classes were taught by another instructor.

Research Sample

Students in all of the Ad/PR campaigns classes were told that the instructor wanted to gather information on strong and weak points in the class in addition to formal university evaluations. They were given a hard copy of a brief questionnaire at the end of each semester that solicited their response to the health campaigns assignment in particular, and into their overall experience in the class. They were told not to put their names on the questionnaire and offered the option of typing their responses so that the instructor would have no means of identifying them from their handwriting. A total of 67 out of 80 students enrolled in the courses completed the questionnaire.

Dependent Variables

Attainment of assignment objectives

To assess students’ self-reported attainment of relevant learning objectives for the campaign assignment students were presented with a list of statements to which they responded on 4-point scales (strongly disagree, disagree, agree, strongly agree). For example: “I think our project was based on solid primary research,” and “I feel confident about going out and creating a health campaign in the ‘real world.'” (For a full list of statements, see Table 2).

Table 1

Comparison of Treatment to Control Group Regarding Affective & Cognitive Learning Variables in the Course as a Whole

Engagement Variables	M	SD	t	P
Likelihood of using content in real life			.75	.228
Collaborative project (treatment)	6.64	1.08
Control	6.46	.76
Likelihood of discussing content after semester is over			1.48	0.72⁺
Collaborative project (treatment)	6.53	1.26
Control	6.10	1.05
Likelihood of keeping abreast of issues addressed in course			2.67	.004^**
Collaborative project (treatment)	6.40	1.17
Control	3.39	1.70
Other Affective Learning Variables ¹
Attitude toward course			.64	.265
Collaborative project (treatment)	6.35	1.36
Control	6.15	1.12
Appreciation for subject matter			.88	.379
Collaborative project (treatment)	6.35	1.10
Control	6.12	.96
Likelihood of enrolling in a similar course			1.16	.251
Collaborative project (treatment)	6.24	1.35
Control	5.080	1.64
Cognitive learning ²			-.38	.707
Collaborative project (treatment)	.57	.74
Control	.64	.75

p < .01.

p < .10 (approaching significance)

Note: p values for engagement variables (H1) based on one-tailed tests; p values for other affective learning and cognitive variables (RQ1) based on two-tailed tests.

Adapted from McCroskey, 1994, and Mottet & Richmond, 1998

Richmond, Gorham & McCroskey, 1987

Table 2

Comparison of Treatment to Control Group Regarding Assignment Objectives

	M	SD	t	P
I would give a high rating to the quality of my group's health campaign.			2.24	.029^*
Collaborative project (treatment)	3.69	.47
Control	3.34	.76
This project helped me recognize the importance of teamwork in producing good work.			–.16	.871
Collaborative project (treatment)	3.62	.55
Control	3.65	.54
I have gained an appreciation for how consumer testing could shape a message.			.13	.898
Collaborative project (treatment)	3.37	.71
Control	3.35	.69
I have a greater understanding of the research process than I did after campaign 2.			-2.16	.034^*
Collaborative project (treatment)	3.25	.67
Control	3.60	.66
I feel confidence in my ability to do secondary research (literature and Internet search).			1.62	.109
Collaborative project (treatment)	3.72	.46
Control	3.51	.56
I feel confident in my ability to do primary research (focus groups, questionnaires).			.76	.448
Collaborative project (treatment)	3.58	.50
Control	3.49	.51
I feel confident about being able to go out and create a health campaign in “the real world.”			1.90	.062⁺
Collaborative project (treatment)	3.56	.50
Control	3.29	.67

p < .05.

p < .10 (approaching significance)

Note: All p values based on 2-tailed tests.

Overall affective learning in the course

To measure affective learning in the entire course the researchers used items from McCroskey's (1994) Affective Learning Scale and the Revised Affective Learning Measure (Mottel & Richmond, 1998). The measures are seven-step semantic differential scales on which students evaluate their feelings about course content and subject matter (for example, “good-bad,” “valuable-worthless”) and likelihood of taking future courses in the content areas (for example, “likely-unlikely,” “possible-impossible”). Cronbach's alphas in the study were: likelihood of using content in real life = .95; likelihood of discussing content with others = .96; likelihood of staying abreast of issues = .97; attitude toward course = .89; appreciation for subject matter = .90; and likelihood of enrolling in course with similar content = .96, showing a high level of reliability for the whole instrument.

Overall cognitive learning in the course

The researchers used the “learning loss” measure (Richmond, Gorham & McCroskey, 1987) to assess cognitive learning. In this measure, cognitive learning is assessed by first asking students to indicate on a scale of 1 to 10 how much they are learning in the specific class. They are then asked to indicate on a scale of 1 to 10 how much they could have learned in the class if they had the ideal instructor. The score is obtained by subtracting the response for the first question from the second. Seven students rated their own instructor more highly than the ideal instructor, meaning that they did not correctly understand the learning loss measure. These responses were omitted from analysis, bringing the total down to 60.

Students also provided their name, major, current GPA and expected grade in the course. Finally, they answered an open-ended question about what aspects of the course should remain the same and which should be changed.

Results

To test for differences between the treatment and control groups a series of independent samples t-tests were performed. Results are presented in Tables 1 and 2.

H1 predicted that students in the treatment condition would report higher levels of engagement in course material than students in the control condition. Answers on all items trended toward higher scores for the treatment condition. However, only one statistically significant difference emerged: students in the collaborative project rated themselves as more likely to stay abreast of news about the issues addressed in the class than students in the control condition (t = 2.67, p = .004). It is also worth noting that a difference with respect to talking about the topic after the semester was over also approached significance. Therefore, H1 was partially supported.

RQ1 asked how self-reported cognitive learning outcomes would differ between students in the treatment and control conditions. With respect to the health communication assignment, students from the treatment group assigned a higher rating to the quality of the campaigns they created. Differences also approached significance with respect to students’ confidence about conducting health campaigns in the real world, with higher means for students in the treatment group. In contrast, students in the control condition reported gaining a greater understanding of the research process through the campaign than did students in the collaborative project. Findings are displayed in Table 2.

RQ2 asked how other aspects of cognitive and affective learning in the course as a whole would differ between treatment and control conditions. Regarding cognitive learning, no statistically significant difference emerged (see Table 1). However, given that 10% of the students did not understand the measure, the validity of the measure is open to question. With respect to the other affective learning sub-scales, no statistically significant differences were evident.

In the open-ended items many students in the collaborative project wrote positively about the value of formative research and analysis done by the health communication class. A number of these students articulated the instructors’ goal for the collaboration, for example, “I really enjoyed working with Dr._____'s class because it shows you how to work in a group setting while using real world applications. Very useful in preparing me for a career in the field.” Responses of students in the first treatment group reflected what appears to be some confusion about the role health communication students were playing. A few seemed to view the other class as more like the research department in an ad agency than a client, for example: “Getting research [from the other class] is a great starting point for finding a theme or direction for a campaign.” The researchers already were aware that some confusion had arisen in this area during the first run of the project and made changes accordingly. The second group evidenced a clearer picture of the nature of the collaboration that was being simulated.

A few students complained about lack of motivation among their partner teams. For example, one student opined: “[…] the health care class group wasn't serious and only two members from their group did anything.” However, it should be noted that students in the control group complained about their own classmates; when asked what they wished had been different about the class, a number responded simply: “My group.” Finally, several students stated they would have liked more time and/or more meetings with the health communication students. “I wish that we had a little more time,” one remarked. “There was more we wanted to do, but didn't have the time.”

Discussion

The innovation of this assignment lies in the collaboration between students in the same school of communication who possess different skills sets. The assignment is characterized by intense mentoring of student teams both in the Ad/PR class and the health communication class. As is perhaps typical when one experiments with pedagogy, results of this study indicated that the project did result in positive student outcomes, though not always precisely those the instructors had expected. The strongest outcome was that students in the treatment condition reported themselves more likely to continue to stay abreast of information about the subject matter covered in the course than students in the control condition. This may be associated with a result that emerged from the qualitative data that students in the treatment condition had a greater sense of having experienced something close to the client relationships they would have in the industry after graduation, a major advantage of PBL (Allen, Donham & Bernhardt, 2011).

No significant difference emerged between the groups, however, regarding students’ expectation that they would use information from the entire class more in the real world. This could be because most of the students in the campaigns class have goals of marketing products rather than health messages. Also, one of our goals for the Ad/PR students was to enrich their experience with market research by allowing them to interact with the “client” who had done in-depth primary and secondary research on their topic and to observe the results of message testing of their campaigns. Comments in the open-ended section suggested that they appreciated the research conducted by their peers in the other class, but quantitative measures indicated that this did not lead them to understand the research process better or to appreciate the value of message testing more. It is possible that because the “client” conducted the message testing, it may have seemed less relevant to the Ad/PR students. Literature on both PBL (Downing, Kwong, Chan, Lam & Downing, 2009) and simulations (Newberry & Collins, 2012) states that successful implementation is dependent on the scaffolding of learning and knowledge construction. Future collaborations will require a careful reevaluation of the structure of the assignment, probably allocating more time and attention to the message-testing step.

Projects that involve more than one class are complex and require an unusual amount of coordination between classes to accomplish (Newberry & Collins, 2012). One of the biggest challenges is the fact that the Ad/PR campaigns class has a lower enrollment cap than the health communication class. Economic realities at the institution have made it impossible to lower the cap on the health communication class commensurately, with the result that groups in that class are much larger. Despite this, from the instructors’ perspective, the assignment provides a simulation of client/agency interaction regarding a topic outside of the experience of our advertising students. We think the wide-ranging benefits of the exercise make it worth the effort.

References

Albanese

M. A.

, & Mitchell

(1993). Problem-based learning: A review of literature on its outcomes and implementation issues. Academic Medicine, 68 (1), 52–81.

Allen

D. E.

, Donham

R. S.

, & Bernhardt

S. A.

, (2011). Problem-based learning. New Directions for Teaching & Learning, 128, 21–29.

Booth-Butterfield

(2011). Steve's primer of practical persuasion 3.0 [Web log]. Retrieved March 1, 2013, from http://healthyinfluence.com/wordpress/steves-primer-of-practical-persuasion-3-0/

Centers for Disease Control and Prevention [CDC] (2012). CDCynergy. Retrieved March 31, 2012, from http://www.cdc.gov/healthmarketing/cdcynergy/

Chad

(2012). The use of team-based learning as an approach to increased engagement and learning for marketing students: A case study. Journal of Marketing Education, 34, 128–39.

Clapper

T. C.

(2010). Role play and simulation. Education Digest, 75 (8), 39–43.

Coombs

W. T.

, & Rybacki

(1999). Publication relations education: Where is pedagogy? Public Relations Review, 25 (1), 55–63.

Downing

, Kwong

, Chan

S.-W.

, Lam

T.-F.

, & Downing

W.-K.

(2009). Problem-based learning and the development of metacognition. Higher Education, 57 (5), 609–21.

Fishbein

, & Cappella

J. N.

, (2006). The role of theory in developing effective health communications. Journal of Communication, 56, S1–S17.

10.

Gijbels

, Dochy

, Van den Bossche

, & Segers

(2005). Effects of problem-based learning: A meta-analysis from the angle of assessment. Review of Educational Research, 75 (1), 27–61.

11.

Kirschener

P. A.

, Sweller

, & Clark

R. E.

, (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, problem-based, experiential, and inquiry-based teaching. Educational Psychology, 41, 75–86.

12.

Lubbers

C. A.

, & Gorcyca

D. A.

, (1997). Using active learning in public relations instructions: demographic predictors of faculty use. Public Relations Review, 23 (1), 67–80.

13.

Mottet

T. P.

, & Richmond

V. P.

, (1998). Newer is not necessarily better: A reexamination of affective learning measurement. Communication Research Reports, 15 (4), 370–78.

14.

Newberry

, & Collins

M. K.

, (2012). A recruiting and hiring role-play: An experiential simulation. Marketing Education Review, 22 (1), 67–71.

15.

Query

J. L.

Jr. , Wright

K. B.

, Bylund

C. L.

, & Mattson

(2007). Health communication instruction: Toward identifying common learning goals, course content, and pedagogical strategies to guide curricular development. Health Communication, 21 (2), 133–41.

16.

McCroskey

J. C.

(1994). Assessment of affect toward communication and affect toward instruction in communication. In Morreale

& Brooks

(Eds.), 1994 SCA summer conference proceedings and prepared remarks: Assessing college student competency in speech communication (pp. 56–71). Annandale, VA: Speech Communication Association.

17.

Richmond

V. P.

, Gorham

J. S.

, & McCroskey

J. C.

, (1987). The relationship between selected immediacy behaviors and cognitive learning. In McLaughlin

M. L.

(Ed.), Communication Yearbook 10, (pp. 574–90). Beverly Hills, CA: Sage.

18.

Stewart

A. C.

, Williams

, Smith-Gratto

, Black

S. S.

, & Kane

B. T.

, (2011). Examining the impact of pedagogy on student application of learning: Acquiring sharing, and using knowledge for organizational decision making. Decisions Sciences Journal of Innovative Education, 9 (1), 3–26.

19.

Tiwari

, Lai

, So

, & Yuen

(2006). Comparison of the effects of problem-based learning and lecturing on the development of students’ critical thinking. Medical Education, 40 (6), 547–54.

20.

Vernon

D. T. A.

, & Blake

R. L.

, (1993). Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine, 68 (1), 550–63.

21.

U.S. Department of Health & Human Services [HHS], National Institutes of Health [NIH], National Cancer Institute [NCI]. (2009). Making health communication programs work [The Pink Book]. Retrieved March 1, 2013, from http://www.cancer.gov/cancertopics/cancerlibrary/pinkbook/page3

22.

Wee

L. K.-N.

, Alexandria

, Kek

Y-C.

, & Kelley

C. A.

, (2003). Transforming the marketing curriculum using problem-based learning: A case study. Journal of Marketing Education, 25 (2), 150–62.