Abstract
The aim of this review was to analyze the effects of instance response systems or clickers on students’ learning in different teaching strategies. A total of 128 empirical studies were reviewed; 80% of the studies were conducted in the context of lectures or collaborative learning. Further analysis of the studies using a quasi-experimental design revealed that clicker usage in traditional lectures may enhance students’ attention and participation. However, it is not more effective than low-technology methods such as hand raising or response cards in terms of learning performance. Clickers combined with collaborative peer-aided learning have shown positive results with large effect sizes. Furthermore, incorporating clickers into innovative teaching strategies appears to be promising. Finally, the use of clickers to promote high-order thinking is discussed.
Introduction
This study discusses the effect of incorporating clicker techniques into different teaching strategies on students’ learning. The term clicker refers to a technology-enhanced pedagogical tool which assists students to express their answers to and opinions on an instructor’s questions during instruction. The literature shows that four different techniques have been applied in the classroom to enhance teacher–student and student–student interaction and to collect students’ instant responses. First of all, students use low-technology and low-cost tools such as hands, flashcards, color cards, or whiteboards to give their responses. Second, instant response systems are used. The major trigger devices are numeric keypads, and the communication technology is hard-wired equipment. Third, the technology has shifted to wireless radio frequency or infrared devices. These devices typically resemble a TV remote control, by which students can give their response to a question. The fourth technology is network-based systems and bring your own devices. For example, intelligent mobile phones can be used as answer devices, and an application can quickly scan and collect students’ answers. These techniques, except for the first kind, are referred to as clickers in this text. Previous studies have revealed the potential of clickers in education. Nevertheless, there are also studies which have found no significant differences in the effect of those using technology and those using hand raising on examination results (e.g., Stowell & Nelson, 2007). The present study reviewed articles published in the past few years and attempted to analyze the effects of various teaching methods using clickers on students’ learning and participation.
Several papers have reviewed research articles on clickers and discussed them from different viewpoints. For example, Gök (2011a) compared the fundamental differences between clickers and traditional classrooms in terms of student involvement (attendance, attention, anonymity, participation, and engagement). The researcher summarized the benefits and the challenges associated with the use of clickers. There is a considerable amount of data suggesting that students using clickers are more engaged in the concepts covered, participate more, pay more attention in class, and are more involved in class discussion. Clickers offer the opportunity to make the lecture more interactive but are not threatening.
In addition to students’ involvement, Han (2014) analyzed the effects of clickers on students’ metacognition, anxiety, understanding, and performance, as well as conceptual changes. The researcher thought that it was possible for instructors to use clickers in a didactic way, to check student attendance, and to obtain summative feedback, but students might focus on accumulating participation marks rather than paying attention to the teaching materials.
Similarly, Auras and Bix (2007) and Cardoso (2011) generalized the benefits of the incorporation of clickers. Clickers increase students’ attendance, preparation, and interactions. They yield better class material comprehension and retention of lecture materials. They help to create a collaborative learning environment, active participation of students during class, and greater student satisfaction, as well as promoting self- and peer-assessment. However, according to Hunsu, Adesope, and Bayly’s (2016) meta-analysis, the effects of clicker use were small, albeit significant, on knowledge acquisition and transfer, and insignificant on knowledge retention. The effects were negligible when the clicker questions were presented to the comparison groups.
Moreover, White, Syncox, and Alters (2011) looked into the role of incentives, reviewing four kinds of studies: using clickers for testing, where correct answers were rewarded; participation, where clickers alone could receive incentives; a combination of testing and participation; and no incentive at all. It was common that clickers were used in quizzes, which might be worth a small portion of the course grade. There were also many instructors who highlighted participation and students’ misconceptions revealed through clickers, rather than evaluating the correctness of the answers or considering clickers as tests. Furthermore, a few studies combined testing and participation by giving a major portion of the grades to correct answers and a small portion to participation. Finally, it appears that nonincentive-based use is not recommended (Beatty, 2004; Trees & Jackson, 2007; Wieman et al., 2008). On the other hand, rewards should be avoided for incorrect answers in the condition of extremely simple or trivial questions.
These researchers also proposed the challenges associated with clickers. These challenges were evaluated from the aspects of technology, research method, and learning. First, clickers are new technologies and have room for technological improvement and advancement (Gök, 2011a). For example, most systems can only capture quantitative data and limit the responses to multiple-choice or true or false questions. It should be noted that this challenge has been solved by some of the newly developed systems (e.g., the Cloud Classroom) and applications (e.g., Nearpod), which have made open-ended questions and drawings available for users. Instructors could share selected student responses or drawings with the whole class. The Cloud Classroom even allows instructors to immediately choose certain students’ text responses for the class to vote on. These functions significantly improve the teaching flow and opportunities for in-depth discussion.
Second, many research articles have discussed the key problems of clicker use including a lack of systematic research, a bias toward using anecdotal evidence, excessive focus on attitudes as opposed to learning and cognitive processes, and samples derived from limited educational settings. A limitation of these studies was that they relied exclusively on describing the application of clickers. This limitation was further complicated by the fact that the researchers played the multiple roles of teacher, researcher, and participant. The various relationships among the use of clickers and student learning have been identified but not empirically investigated in detail (Cardoso, 2011; Gök, 2011a).
Third, the use of clickers faces some cognitive restrictions. Clicker questions highlight important ideas for students. The effect may come about by directing students’ attention to certain resources or specific items during class. Moreover, questions that promote high-level cognitive thinking are omitted. The published literature on the learning effects of clickers is hampered by methodological issues that impede clear understanding of clickers’ effects on learning and memory (Han, 2014; Shapiro & Gordon, 2012).
Researchers have studied the influence of the use of clickers in class from a macro or microcosmic perspective, but there is no article reviewing the effect of different teaching strategies on students’ learning. In this article, we compare the effects of clickers on student learning with different teaching strategies and unpack the missing factors relating to student learning with clickers. The results can help us to understand the combined effects of teaching strategies and clickers and to draw practical conclusions based on empirical evidence.
Methods
Analysis Nodes.
Outcomes are double coded if multiple-outcome variables were measured.
The researchers practiced coding 12 articles to establish the interrater reliability. The agreement among them reached 90%. Then the remaining articles were divided among the four coders. Any uncertainty was marked for group discussion and decided by consensus. Finally, those studies that had experimental or quasi-experimental designs with teaching intervention and manipulated teaching strategy as an independent variable were examined for their effects. A total of 20 studies fitted the criteria. Based on the data available in the articles, effect sizes, Cohen d for t tests and ηp2 for analyses of variance, were calculated for comparison across studies.
Results and Discussion
The results indicated that participants were mostly college students (Table 1). Nearly 90% of the studies were undertaken in university classrooms, which was similar to Hunsu et al. (2016) analysis. Although clickers are becoming popular, relatively few studies have conducted research in K-12. Clicker use in K-12 encounters problems different from those in college. Primary and secondary schools may have more concerns about the technical support if remote keypads are used, and with the uncontrollable Internet use if network-based systems and bring your own devices are used. Unfortunately, there was only one study using clickers for primary and secondary school teacher professional development.
The disciplines were distributed mainly in the domains of science and engineering (50.39%) and social science (25.58%). A high percentage of the social science studies was in psychology. A few studies have applied clickers to the humanities. For example, Marlow (2010) used clickers to sustain native English undergraduate speakers’ course participation, to check their comprehension of grammar, to analyze misconceptions of grammar, to address open-ended questions, and to implement collaborative learning.
Most studies (71.88%) were carried out in North America, especially in the United States. There were merely two studies and one study, respectively, conducted in South America and Africa. A high percentage (74.22%) of studies applied remote technology. A few articles (5.47%) did not specify which clicker technologies they used. They might have conducted large-scale surveys or developed their own measurement instruments. Nevertheless, the use of applications and web-based mobile devices has been increasing in recent years. Regarding pedagogy, lectures were common, and collaborative learning, role-playing games, and flipped classrooms were observed. Concerning the outcomes, researchers have paid more attention to achievement, participation, and affect, but little attention to complex learning abilities such as critical thinking, problem solving, and metacognition ability.
Finally, among the 128 studies, 116 specified the teaching strategies, whereas 12 studies did not provide detailed descriptions of the teaching strategies. The majority adopted lectures (31.25%) or collaborative peer-aided learning (48.44%). The following sections analyze the influence of the use of clickers with different teaching strategies on students’ learning in terms of their achievement, participation, and affect, as well as their high-level cognitive thinking.
Effects of Clickers on Students’ Achievement With Different Teaching Strategies
Effects of Clickers With Different Teaching Methods on Students’ Achievement.
Lectures
Of the 20 studies, 14 were lecture based with clicker questions. However, the outcomes in terms of students’ test scores were not always positive. Some studies revealed positive outcomes that favored clicker use (Kim et al., 2015; Kvadsheim, Haugerud, Hammer, Bratterud, & Habib, 2015; Mayer et al., 2009; Morling, McAuliffe, Cohen, & DiLorenzoet, 2008), whereas others found no significant difference between the experimental group (using clickers) and the control group (no clickers, hand raising, or response cards; e.g., Grzeskowiak, Thomas, To, Reeve, & Phillips, 2015; Levesque, 2011; see more in Table 2).
In Morling et al. (2008), the instructor posted five multiple-choice, fact-based clicker questions based on the day’s required reading at the beginning of class. Extra credit was given for correct answers. Later in the lecture, the instructor would briefly elaborate on questions that most students had answered incorrectly. The students in the clicker sessions got slightly higher scores than those in the nonclicker sessions (M = 69.62–72.69 vs. 69.04–71.13; Morling et al., 2008). However, the effect size was very small, ηp2 = 0 .003, and was mainly driven by scores in two of the four exams. Likewise, Mayer et al. (2009) and Kvadsheim et al. (2015) claimed that clickers are more effective than questions without clickers or no questions, but the effect sizes (d = .06–.43) were also small. Kvadsheim et al. noted that the effect was most pronounced for exam questions adhering to the subject-matter knowledge. Moreover, the effects might be subject sensitive. According to FitzPatrick, Finn, and Campisiet (2011), clicker use improved achievement in exercise physiology and some of the topics in anatomy and physiology, whereas nonclicker use produced higher scores in human pathophysiology.
Furthermore, in several studies, using clickers did not outperform raising hands or response cards (Desrochers & Shelnutt, 2012; Dill, 2008; Fallon & Forrest, 2011; Levesque, 2011; Stowell & Nelson, 2007). Neither did remote clickers or mobile devices create a significant difference in achievement (Stowell, 2015). In fact, the first study of Anthis (2011) revealed that the use of clickers was negatively associated with the students’ examination performance. In this study, throughout the semester, students used clickers or raised their hands to practice multiple-choice questions drawn from the examinations that they took later. Based on the results of five examinations, the group that raised their hands performed better than the group that responded with clickers. The researcher suspected that the negative effects were caused by an artifact of the procedure or selection of participants and thus conducted a second study in which clickers were not used until the sixth week, and two sessions alternated clicker use and hand raising in the middle of the experiment. The second study showed no relationship between clicker use and examination scores.
Simply put, even if clickers increase engagement in lectures (Zdravkovska, Cech, Beygo, & Kackley, 2010) and enable students to compare their responses to those of their peers (Grzeskowiak, Thomas, To, Reeve, & Phillips, 2015), the learning effects are not necessarily reflected in achievement tests. More studies are needed to determine the effects of clicker use in lectures, especially the frequency, timing, and incentives of use, and whether their use matches the nature of the subject.
Peer-aided learning
Although lectures with clickers do not yield strong positive effects on participants’ achievements, several researchers found that when incorporating a proper teaching strategy with clicker questions, students’ learning process and exam performance are increased. Among the reviewed studies using a quasi-experimental design, six studies clearly stated that they applied collaborative learning or peer-aided learning with clickers. The test results showed that the combination promoted students’ understanding of knowledge. The effect sizes were generally large (Table 2). In fact, those studies that concluded a positive effect of clicker use in lectures might have partially incorporated peer discussion (e.g., FitzPatrick et al., 2011).
Peer-aided learning is a sort of collaborative learning, whereby students discuss possible answers or solutions to clicker problems. For example, Gauci, Dantas, Williams, and Kemm (2009) had students engage in peer-to-peer discussion before answering dilemmas or complex questions. They uncovered the potential of clicker use in engaging students in thoughtful consideration of difficult issues. Peer discussion and clicker questions can get students to engage more in an active learning process focusing on the application of concepts and problem solving (Gauci et al., 2009). A prominent advantage of clickers is the ability to encourage students to voice their ideas and discuss with their peers to reach an agreement (Yu, Chen, Kong, Sun, & Zheng, 2014).
According to Blasco-Arcas, Buil, Hernández-Ortega, and Sese (2013), the underlying mechanisms regarding the effectiveness of clickers in collaborative learning are that clickers promote student–student and student–teacher interaction, which leads to active collaborative learning. Consequently, engagement and performance are enhanced. They tested their model by surveying 198 undergraduate students in introductory marketing courses using clickers. The results demonstrated a significantly positive influence of interactivity on active collaborative learning and engagement, which had significant effects on learning performance. Blasco-Arcas et al. (2013) pointed out that clickers increased the interactivity within groups and between students and the teacher. By using clickers, the students perceived that their answers and opinions were taken into account by the teacher and their peers and understood better that this process helps them improve their learning performance. Thus, clickers assist active collaborative learning and engagement and lead to achievement.
Furthermore, Morice, Michinov, Delaval, Sideridou, and Ferrieres (2015) indicated that students learning in the peer instruction condition did not achieve better learning gains on quizzes than did those in the individual learning condition because in the latter condition, the students had access to online consultants to help them find the right answers to the questions. The authors proposed that the function of peer instruction might be replaced by online consultants. With an online consultant, clickers with individual learning were as effective as clickers with peer instruction. What matters might be the feedback quality, not the format. Finally, Chien, Lee, Li, and Chang (2015) found that displaying the real-time whole class responses normalized high school students’ follow-up peer discussion and resulted in lower conceptual learning outcomes. The nondisplay session performed significantly better in both post- and transfer tests. The effect sizes reached the medium level.
New Teaching Strategies With Clickers
A few but innovative studies have applied clickers to particular teaching strategies such as role-play, interrupted case teaching, and flipped classrooms. The combination of different teaching methods and clickers is helpful for researchers to explore the unique value and function of clickers in teaching, as well as the limitation and application scope of clickers. They revealed the additional advantages and potential of clickers. Future studies may investigate in-depth the effectiveness of the combination.
Role-playing game
Brouhle (2011) used a role-play game teaching method. The students identified the dilemma inherent in oligopoly market settings before the game, repeatedly competed against each other by making pricing decisions with clickers in the game, and evaluated and discussed lessons learned after the game. A key advantage of using clickers was the possibility of playing the game quickly, and hence it was able to be implemented within a single class period. The use of clickers allowed students to talk with each other anonymously. It was observed that students were highly engaged in the game and were more likely to play according to the profit-maximizing assumption. Finally, because the game could have and even expected multiple outcomes, the use of clickers was able to count all kinds of results in real time. It thus allowed the instructors to be better informed and to control the outcome of the game.
Interrupted case teaching
The high interactivity supported by clickers increases the possibility of incorporating case stories into big class sizes (Lundeberg et al., 2011). Case materials might be presented progressively, piece by piece. Students use clickers to fill in the missing parts and unfold the storyline with the help of instructors.
Flipped classrooms
Clickers can be integrated with a flipped classroom. Students view the online lecture or video clips before class and answer clicker questions. Instructors can easily find who has viewed the online materials and who has not. When students perform poorly on the clicker questions, the instructor can provide more explanations or details of the context, or conduct peer discussion. Clickers assist instructors in monitoring students’ learning through online materials and thus help them to bridge the online and in-class learning. Clickers are also used for warm-up and for maintaining a high-interaction level in the class (Morisse, 2015). The use of clickers with flipped classrooms can help students better understand concepts and develop learning skills (Flynn, 2012; Prunuske, Batzli, Howell, & Miller, 2012).
Effects of Clickers on Students’ Participation and Affection
The aforementioned effects have much to do with participation and affect in learning. About half of the reviewed studies considered attitudes, participation, and attendance as outcome variables. Clickers actively engage nearly all of the students in the class. Although students can refuse to respond with their clickers, they generally all participate (Cleary, 2008). Survey results have demonstrated that 75% of the students agreed or strongly agreed that the use of clickers allowed them to participate more openly in the class (Russell, McWilliams, Chasen, & Farley, 2011). Landrum (2013) also claimed that 83.1% of the students in his study agreed that the use of clickers made them participate more in class. The interactivity with peers and with the teacher that results from using clickers was critical for increasing engagement (Blasco-Arcas et al., 2013). Clauson, Alkhateeb, and Singh-Franco (2012) assessed the use of clickers at three campuses and found that students believed that having the system made the class better and encouraged participation.
Concerning affective outcomes, as a whole, clickers let the students learn more positively, and they had a pleasant experience in the process of learning. Clickers alter the learning environment such that students are encouraged to take the course more seriously. Instructors could evaluate students’ knowledge of the assigned readings with clicker questions. To do well, students would preview the materials carefully to identify major themes and concepts (Mollborn & Hoekstra, 2010). According to data from the student survey, 95% of students agreed or strongly agreed that the use of clickers helped them to pay attention in class (Russell et al., 2011). Moreover, using clickers, instructors could give timely feedback on each student’s answer. Chen Whittinghill, and Kadlowec (2010) revealed that the students all felt that having feedback was at least somewhat helpful to their learning. Specifically, they felt that the teacher paid attention to every student’s answer.
Furthermore, Cardoso (2011) surveyed and interviewed participants and found that learners viewed the use of the technology in a highly positive way, as they believed that clickers increased their level of participation and the general enjoyment of classes. Finally, as a result of participation, clickers could improve attendance (Auras & Bix, 2007; MacGeorge et al., 2008), as 83.1% of students agreed that use of clickers encouraged them to attend class (Landrum, 2013). Levesque (2011) pointed out that, for those semesters that clickers were used, test grades were positively correlated with attendance, r = .57–.58, p < .01, whereas for those semesters that clickers were absent, no significant correlation was found, r = .18–.25, p > .05. Clickers seemed to arouse meaningful attendance. Moreover, clickers increase the students’ level of motivation (Cardoso, 2011; Gauci et al., 2009). This may have a profound effect on students with performance-oriented goals, who are normally not as productive as those with mastery-oriented goals. Zingaro (2015) compared the effect of peer instruction with clickers on the two groups’ subject-matter interest and revealed that the instruction enhanced the interest of the students with performance-oriented goals.
Effects of Clickers on Students’ High-Level Cognitive Ability
Questioning is seen as a useful method for stimulating students’ thinking, but it is difficult to implement successful questioning in traditional lecture classes (Mayer et al., 2009). With the assistance of clickers, the questioning method can be implemented in lectures, benefitting students by stimulating and engaging their thinking, thus enhancing their comprehension of the subject and high-order cognitive ability (Lin, Liu, & Chu, 2011). In our review, three types of high-order cognitive thinking were discussed.
Critical thinking ability
Researchers generally consider that clickers have the potential to enhance critical thinking skills (Velasco & Cavdar, 2013). By engaging in complex questions, the students practiced problem solving and critical thinking (Flynn, 2011). Data from the student surveys showed that most students (69% in Hoekstra, 2015; 76% in Russell et al., 2011) agreed or strongly agreed that clickers helped them hone their critical thinking skills. Nevertheless, students’ critical thinking ability or skills were not measured in these studies. There was no direct evidence of improvement in critical thinking.
Another issue is whether the common question type, that is, multiple-choice questions, could cultivate students’ critical thinking. On the positive side, students submit numeric clicker answers simultaneously; the predominant answers are easily gauged using the histogram function of the clicker program. Critical thinking ability may be developed from seeing and analyzing the varied responses of their peers (Gök, 2011b). Specifically, if combined with productive discussion of the underlying reasons for choosing a response, students may think critically and develop empathy for alternative perspectives (Hoekstra, 2015). The negative side considers that it was difficult to stimulate higher level thinking with multiple-choice questions. Instead, multiple types of clicker questions, such as open-ended questions, ill-defined questions, past experience questions, and student-designed questions, should be used (Mollborn & Hoekstra, 2010). Further research is needed to clarify whether multiple-choice questions could improve critical thinking and how. Moreover, with the advances in machine learning algorithms, text and figure responses to other types of questions may be modeled in real time for instructors to make use of the mass responses in a large class.
Problem-solving ability
Researchers have found that using problem-based clicker questions interspersed throughout the lectures could improve problem solving in the final examination and the delayed test (Crossgrove & Curran, 2008). For example, following a lecture on a concept, Levesque (2011) gave students clicker problems that required them to have a firm understanding of the concept and to think critically to analyze the problem from different perspectives. The students solved the problems individually. Then the histogram of student responses was immediately displayed on the screen. Students explained and discussed their solutions before the correct answers were indicated. They appeared to perform well on tests that involved challenging questions. The use of clickers introduces practice and class discussion, both of which can improve problem-solving ability.
Metacognition ability
Ideally, the learning process with clickers creates opportunities for students to self-reflect and self-monitor, consequently improving their metacognitive ability. Based on qualitative analysis of observations and interviews, Brady, Seli, and Rosenthal (2013b) pointed out that clicker use improved metacognition because it promoted student reflection on concepts and levels of preparation. However, their quantitative data revealed that students’ metacognitive knowledge increased in the group using paddles, but not in the group using clickers (Brady, Seli, & Rosenthal, 2013a). Moreover, Mayer et al. (2009) argued that clickers develop students’ metacognitive skills for gauging how well they understood the lecture material and for how to answer exam-like questions in the future. Further studies may provide empirical evidence for such an argument.
Conclusion and Future Research
Based on the results, a few conclusions are drawn, and recommendations for future studies are provided. Clickers are commonly used with lectures. However, the integration of clickers alone into lectures yields trivial effects, if any, on learning outcomes. Hunsu et al.’s (2016) meta-analysis has reached the same conclusion. Our further analysis of the teaching strategies revealed that incorporating peer-aided learning with clickers has evinced more definite positive outcomes. Instructors are recommended to use cooperative learning and clickers. Moreover, based on the inconsistent results of research in lecture classes, the effectiveness of clickers is influenced by various factors. For example, Fallon and Forrest (2011) found no clear advantages of clickers in improving test scores, increasing feelings of hope, or reducing anxiety over traditional response cards. Nevertheless, they did observe some benefits of clickers such as enhancing test scores or preparedness for some students at certain points during the semester. These benefits were shown in some sessions in half of the semester, but not in the other sessions or in the other half of the semester. These findings, as well as the disagreements in the literature, suggest that additional factors such as class dynamics, the instructor’s style, clicker questions being graded or ungraded, or teaching content need to be taken into consideration before definitive conclusions can be reached regarding the effectiveness of clickers with lectures. The effects of these factors have not been well analyzed thus far. Future research may take the content as well as the contextual factors into account.
Moreover, one of the major rationales for clicker use is the increase in classroom participation and interaction, which are highly associated with students’ intention of use. However, in a class, some students may want to get correct answers, while others may not have the incentive to get the correct responses and may click randomly for each question without attempting to come up with a correct response. Researchers have not measured the intention (Perez et al., 2010) or self-monitoring of students using clickers. A combination of questionnaires, interviews, and think-aloud methods may help researchers to investigate students’ answer intention and self-regulated learning. Likewise, researchers have not evaluated students’ attitudes toward long-term use of clickers. Students may be bored if clickers are commonly and continually used in all courses and classes. Further investigation of these issues would deepen our understanding of the nature of learning in a technology-enriched environment.
Finally, most of the existing studies were conducted in North America and Europe, as also found in Hunsu et al. (2016). The geographical distribution of studies reveals a need for research in more diverse cultures. Since clickers have a lot to do with classroom interaction, their effects may be associated with culture. For example, for learning English as a second language, a study conducted with Brazilian undergraduates and young adults revealed only an ephemeral effect on achievement (Cardoso, 2011), whereas a study with Chinese undergraduate students demonstrated a significant improvement in English listening and speaking skills (Yu, Chen, Kong, Sun, & Zheng, 2014). Clickers present more effects with Chinese students perhaps due to the culture of students being less expressive and the low level of interaction in their typical classrooms. Culture could have played an essential role. More research in various cultural settings is therefore needed to clarify the role of clickers.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is supported by Ministry of Science and Technology, Republic of China and Taiwan, under grant numbers NSC 102-2628-S-011-001-MY4 and MOST104-2511-S-011 -006 -MY3.