Pausing the classroom lecture: The use of clickers to facilitate student engagement

Abstract

In a big classroom, it is not easy for instructors to be aware of whether or not all students are engaged in the lecture and who has difficulty understanding learning content. One way in which the engagement between instructor/lecturer and students in the classroom is via technology designed to facilitate this. A computer system, making use of clickers, was designed for students to do this. That is, students can request to pause the lecture when necessary. Two classes with 109 college students participated in this study. Students learned the subject matter by using the system for 2 weeks. This article describes the reasons why students choose to engage via this clicker-based system, the time when they do so and what factors influence students’ willingness to engage in this way. Furthermore, we investigated whether introverts would engage differently from the more non-introverted. The following main findings were obtained. Most students chose to engage via the clicker system. Results showed that students’ willingness to engage via this technology were because they found it difficult to remember or understand the lecture content. For instructors, there is a need to understand when, precisely, to pause the lecture so that an appropriate intervention might be made at that point. An optimal decision boundary for pausing the lecture is proposed. Finally, the article looks at whether or not the learning style of introverts influences student willingness to use such a clicker system.

Keywords

engagement report learning style optimal decision boundary trust

Engagement and learning effectiveness

Lectures are still a common teaching method in education, whether undergraduate or postgraduate. Several issues relate to this method. For example, due to the large number of students in a classroom, instructors are not always aware of whether the students are following what is going on, or whether they need the lecturer to pause for while so that there is the opportunity for an intervention of some sort, perhaps, say, the ability for students to ask questions at that point. If the instructor presents too much information at a time, it may cognitively overload students. On the other hand, if the instructor delivers learning material too slowly or the content is too easy for students, it may make students feel annoyed or bored. In both cases, students can easily be disengaged from learning but the instructor is not aware of this if there is no engagement or intervention. Thus, it is an important issue to investigate when students become disengaged in the lecture, when is a suitable time for the instructor to pause the lecture, what learning issues are being experienced by students, and what support the instructor needs to provide for students.

To increase active engagement of students in the classroom, several approaches have been proposed. One of them is to use instant response systems (IRSs) or clickers (Bachman and Bachman, 2011; Cain et al., 2009; Caldwell, 2007; Heaslip et al., 2014; Hoyt et al., 2010; Powell et al., 2011; Sevian and Robinson, 2011). Clickers allow the instructor to broadcast students questions related to the lecture and allow students or the instructor to respond. The pedagogical usefulness of this technology is reported in the literature but much of this reports how much of the control is in the hands of the instructor rather than the students.

Student engagement is defined as their investment of time and effort in learning activities that lead to success (Heaslip et al., 2014). According to Coates (2005), engagement involves not only earning the formal indicators of success, such as grades, but also understanding learning material and incorporating it in everyday life. Following this notion, student engagement can be achieved through a “student’s willingness, need, desire and compulsion to participate in and be successful in the learning process promoting higher level thinking for enduring understanding” (Bomia et al., 1997: 294). Engagement is an important factor when it comes to learning achievement (Heaslip et al., 2014). Literature suggests that learning improves when students are inquisitive, interested, or inspired (i.e. engaged). In other words, the more time and effort students spend learning a subject, the more they tend to learn about it (Gettinger and Ball, 2007). Students who are more engaged earn higher grades, score higher on tests, and show better personal adjustment to school (Quaye and Harper, 2015). On the other hand, learning tends to suffer when students feel bored, dispassionate, or disaffected (i.e. disengaged). When students are disengaged, they are passive, do not try hard, and give up easily in the face of challenges. Disengaged students can withdraw from learning opportunities (Skinner and Belmont, 1993). Xie and Salvendy (2000) introduced accumulated load which can be represented as total mental workload in a task. They also suggested that peak load is maximum value of mental workload; after a student reaches the peak load, their learning becomes inefficient. They also suggested that peak load is maximum value of mental workload; after a student reaches the peak load, their learning becomes inefficient. Therefore, it is important for educators and researchers to create conditions that foster learning engagement and make it stronger.

Based on the teaching learning and assessment taxonomy by Anderson et al. (2001), learning objectives from low to high are “remember, understand, apply, analyze, evaluate, and create.” Higher level learning objectives result from lower level prerequisite knowledge and skills. For example, students’ understanding of learning material depends on their memory capacity (Shadiev et al., 2014). When a student is not engaged in the lecture and does not understand learning material, it is likely that they would have difficulty applying new knowledge to complete assignments.

Five different levels of student engagement were identified by Schlechty (2002). Authentic Engagement is defined as students’ immersion in work that has clear meaning and immediate value to them. Ritual Compliance is when the work has little or no immediate meaning to students, but there are extrinsic outcomes of value that keep them engaged (e.g. earning grades). Passive Compliance takes place when students see little or no meaning in the assigned work but expend effort merely to avoid negative consequences. Retreatism happens when students are disengaged from assigned work and make no attempt to comply, but are not disruptive to the learning of others. Finally, Rebellion takes place when students refuse to do the assigned tasks, act disruptively, and attempt to substitute alternative activities. Schlechty (2002) outlined three categories to measure the level of engagement for an entire classroom. One is the compliant classroom category, that is, the one that can be observed in most conventional classrooms. According to Schlechty (2002), this category of engagement represents a classroom which is in order and most students study; so it would be easy to infer that learning is taking place. However, while there is a little evidence of rebellion in the compliant classroom, retreatism is a real danger as students are usually disengaged from learning in such classroom.

Facilitating student engagement

Several approaches have been proposed to promote a high level of student engagement. According to Engagement Theory (Kearsley and Schneiderman, 1999), students need to be engaged in learning activities through interaction with others (i.e. classmates or the instructor) and learning tasks. Student engagement may occur with and without the use of technology. There is also a body of knowledge related to technology assisted engagement. Shadiev et al. (2015) suggest that active notifications or alerts through communicating tools increase student engagement. Furthermore, in a blended learning environment, where computer-mediated communication tools are involved, live events (synchronous interaction or feedback) and collaboration are two important factors to engage students in learning (Shadiev et al., 2015). In light of this, applications of IRSs or clickers in education have gained some attention in recent years. For example, IRSs were introduced to increase student active engagement (Bachman and Bachman, 2011; Blasco-Arcas et al., 2013; Caldwell, 2007; Campbell and Monk, 2015; Graham et al., 2007), to enhance learning performance (Bachman and Bachman, 2011; Blasco-Arcas et al., 2013), to improve student motivation and attention (Cain et al., 2009), to facilitate participation of students (Beekes 2006; Campbell and Monk, 2015; Heaslip et al., 2014; Hoyt et al., 2010), to influence student metacognition (Brady et al., 2013) and as a mean of formative assessment (Powell et al., 2011; Sevian and Robinson, 2011).

Benefits of using IRSs in the classroom are reported. They can help create a positive learning environment (De Gagne, 2011; Kay and LeSage, 2009; Keough, 2012). That is, when IRSs are used in class, student attendance, motivation and attention levels (Cain et al., 2009), participation (Bachman and Bachman, 2011; Beekes, 2006; Powell et al., 2011; Campbell and Monk, 2015), metacognition (Brady et al., 2013) and engagement (Bachman and Bachman, 2011; Caldwell, 2007; Graham et al., 2007; Heaslip et al., 2014) can be increased. More interestingly, Graham et al. (2007) explored how useful IRSs were for engagement of reluctant students. Results show that students who are reluctant to share opinions or hesitant to ask questions in class viewed the use of IRSs as helpful whereas students who prefer classes without student participation were significantly less positive regarding helpfulness of IRSs. The use of IRSs promotes students’ level of interactivity with peers and the instructor which, in turn, influences active collaborative learning and engagement (Blasco-Arcas et al., 2013). In addition, their interaction and discussion as well as quality of their learning and performance can be improved. For example, students shift their attention and actively participate in the learning process when IRS questions are presented in class. Besides, misconceptions are easily identified from students’ feedback and the instructor can ask students to resolve their misconceptions through discussion or offer alternative explanations of concepts being incorrectly comprehended. They may also help improve the feedback process and provide effective formative assessment (De Gagne, 2011; Kay and LeSage, 2009).

IRS has effect on learning achievement. For example, Bachman and Bachman (2011), Brady et al. (2013), Caldwell (2007) and Powell et al. (2011) found that there was a difference in the exam scores between students who used IRSs and those who did not use them. On the other hand, results obtained by Hoyt et al. (2010) indicate that use of IRSs enhanced performance on the examination performance of low-achievement students only. Results obtained by Blasco-Arcas et al. (2013) show that the high level of interactivity between students and the instructor achieved with the use of IRSs and students engagement in the learning process positively influence their learning performance. Students also appreciated the anonymity afforded by IRSs (Caldwell, 2007; Campbell and Monk, 2015; Heaslip et al., 2014). Most students perceived that using IRSs increased their understanding of and ability to self-identify concept areas (Powell et al., 2011)

The use of clickers improved the quality of students’ learning experiences, particularly, in promoting more collaborative and engaging learning (Laxman, 2011). As for instructors, they perceived that IRS feedback was useful to evaluate student’s comprehension and adjust lectures accordingly (Cain et al., 2009). Sevian and Robinson (2011) suggested that using IRSs can address several issues from instructors’ experiences in the classroom, such as fully engaging all students, identifying misconceptions, assessing preparedness, gauging the understanding of concepts, eliciting discussion, and identifying students who need additional help.

IRSs have been implemented in all classrooms but more in large lecture classes, where dialogue and engagement between the instructor and students and between students is more difficult. Using these, the instructor engages students through asking questions, prompt, or immediate feedback (Bachman and Bachman, 2011; Cain et al., 2009; Caldwell, 2007; Hoyt et al., 2010; Powell et al., 2011; Sevian and Robinson, 2011). In the lecture, the instructor asks questions about the lecture content or provides prompts or feedback and then students try to respond by using IRSs. Students respond to questions via wireless keypads and feedback is immediately displayed on a whiteboard in the classroom.

Challenges for the instructor and students in using IRSs are also reported. According to Kay and LeSage (2009), the instructor needs time to create effective IRS questions and to learn and to implement IRSs. The instructor also needs to cover adequate content and be able to respond to instantaneous student feedback when IRSs are used. As for students, they need to adjust to a new method of learning, in some cases their discussion leads to confusion, and they have negative reactions to being monitored. All of these may be too challenging for relatively inexperienced instructor and students. Students in the study of Campbell and Monk (2015) and Graham et al. (2007) had negative feelings toward the use of IRSs due to some technical problems (e.g. devices did not work properly). Brief responses that can only be made using IRSs is another important issue (Campbell and Monk, 2015). Among challenges summarized by De Gagne (2011) are potential for overuse IRSs that leads to delivery of limited knowledge and frustration and overload of information. That is, the instructor teaches more concepts but not deeply, and when students’ comprehension is evaluated using IRSs, it turns out that they still had no chance to consolidate new knowledge appropriately. Another challenge is inadequate faculty development. That is, inexperienced instructors create questions that do not provoke or stimulate students’ ability to analyze and apply knowledge or instructors are not able to engage students in the learning process verbally or emotionally.

Pausing the lecture and engagement

According to Caldwell (2007) and Young et al. (2009), a student’s effective learning time is normally shorter than the length of the whole lecture. Some reasons to explain this are that students receive lessons in a teacher-centered lecture format, that little or no interaction takes place, and that students encounter difficulty in maintaining sustained concentration. Therefore, being aware of the progress of students in the lecture is very important for the instructor (Powell et al., 2011). To better understand student learning progress and problems associated with learning and engagement, the instructor needs to encourage students to actively reflect regarding their learning (Coates, 2005). If students are not given time for such reflection, it is likely that their learning progress may be worsened. Thus, the instructor needs to pause the lecture in order to provide students with sufficient time for reflection and asking questions. Pausing the lecture can also be useful to inform the instructor that learning material is being delivered too fast or it is too difficult for students to comprehend. Working memory has limited capacity and if no pauses are made at any point during the lecture, students may easily be cognitively overloaded (Hasler et al., 2007). As a result, learning performance can be negatively affected.

Another important issue to consider is when “the right time” is to pause. In a large classroom, such as the lecture hall, with more than 100 students, and with perhaps 500 in it, it is difficult for the instructor to make the decision as to when or whether they should pause the lecture when seeing students’ hands raised, or indeed when not seeing any hands raised, as students are often reluctant to raise their hands in such a “public” forum, as they do not want to look foolish or draw attention to themselves for whatever reason. Hence, a mechanism needs to be developed and applied that will help students to request a pause during the lecture (i.e. to alert the instructor that some kind of engagement is needed at that particular point) and to report issues associated with learning. Such a mechanism will enable the instructor to be aware of students’ requests and why they are making them. With current existing technology such as IRSs, students are not able to actively report important aspects of their learning to the instructor (i.e. problems associated with remembering or understanding the lecture content). In addition, current technology is not designed in a way that students can request a pause or pauses during the lecture (i.e. to alert the instructor that some kind of engagement is needed at that particular point).

During the lecture, students normally follow the pace, organization of activities, and arrangements set by the instructor. The instructor designs lesson plans and establishes learning objectives or goals. Furthermore, various instructional strategies are used by the instructor in the lecture such as implementing IRSs to pose questions on presented key concepts. Literature suggests that the instructor’s teaching style and how students perceive the instructor’s leadership and management influences student learning engagement (Quaye and Harper, 2015; Umbach and Wawrzynski, 2005). Slavich and Zimbardo (2012) also argued that a strong correlation exists between the instructor’s leadership style, the classroom climate, and students’ affective performance. Students are usually too shy to request a pause to catch up a lecture because they think it shows their inability in learning. Therefore, the trust that students have in their instructor’s ability to manage, organize, and lead the lecture might be important factors for student reflection on their learning progress to the instructor via IRSs.

Students’ learning styles also need to be considered when exploring their interaction with the instructor. Students can be extroverts or introverts (Felder and Silverman, 1988). An extrovert student is outgoing, involved in many activities, prefers to be around other people, and easily asks for help from others. In contrast, an introvert student is quiet, less active, and tends to be withdrawn from others (Cain, 2012: 11). Extroverts and introverts correspond to active and reflective learners respectively. In class, an active learner likes to join a problem-solving process, whereas a reflective learner takes time to think and reflect (Felder and Silverman, 1988).

One advantage of using IRSs for students is that pausing the lecture gives them a break from it and allows some time to reflect on its content, offering an opportunity to address misconceptions and so on (Cain et al., 2009; Caldwell, 2007). Whether for an introvert or an extrovert, a pause or pauses in the lecture will allow them to either carry out practical exercises in the classroom in order to advance their understanding or to allow some time to think about and reflect on the content. Regardless of being either an introvert or an extrovert, students use IRSs to request a pause in the lecture (i.e. to alert the instructor that some kind of engagement is needed at that particular point) when they feel that they are unable to follow the lecturer, they need time for reflection or a practical exercise, or they want to catch up on what they might have missed.

As explained earlier, current technology is not designed in a way that students can request a pause or pauses during the lecture (i.e. to alert the instructor that some kind of engagement is needed at that particular point). In light of this, a system called Pause Lecture, Instant Tutor-Tutee Match, and Attention Zone (PLITAZ) was designed. Rather than being led by the instructor/lecturer, that is, it is up to the instructor/lecturer when that pause or those pauses are made, PLITAZ allows for students to take control, that is, it changes the process from being instructor-initiated to one which is student-initiated. That is, students initiate the interaction to request to pause the lecture (i.e. to alert the instructor that some kind of engagement is needed at that particular point) and then the instructor allows students to have time for reflection, exercise, and provide instructional support. This approach is quite different from the one reported in studies on clickers.

It is essential to investigate how this PLITAZ system, one which is student-initiated in terms of engagement rather than instructor-initiated, is used when it comes to alerting the instructor that some kind of engagement is needed at that particular point in the lecture. There is a need to better understand how students (either extrovert or introvert) use such a student-initiated system in order to alert the instructor that some kind of engagement is needed at that particular point and what factors influence the usage of it (i.e. learning styles or trust in the instructor). Furthermore, there is a need to explore the optimal time for the instructor to react to their requests for engagement at that particular point or points in the lecture in order to make the decision as to when, or if, they are to pause the lecture at that particular point. Thus, the following research questions need to be addressed:

Do students alert the instructor that some kind of engagement is needed at that particular point in the lecture when they have difficulty remembering or understanding the lecture content? And what causes it to happen?

When is suitable time for the instructor to pause the lecture?

Does students’ trust in their instructor influence their willingness to make a request for a pause?

Do students, whether extrovert or introvert, make such requests to the same extent?

Method

Participants

Students from the departments of Civil Engineering (denoted as c), Landscape Architecture (denoted as i), Applied Mathematics (denoted as m), Agronomy (denoted as a), History, Geography (denoted as h), and Forestry and Natural Resource (denoted as f) who took an optional geographic information system course participated in this study. Due to a large number of enrolled students and limited number of computers in a classroom, all participants were arranged into two groups: Group A included 48 students (36 males and 12 females) and group B included 61 students (40 males and 21 females).

Learning procedure

The 2-week course included six classes (50 minutes each) that took place in a computer classroom. The main concepts of their subject matter and the major functionality contained within the software were introduced in the course. In addition, students worked with the software tools on assigned practical exercises. PLITAZ was designed using Java with client–server architecture. The system includes two interfaces, one for the instructor and one for students. At the beginning of each class, the instructor and students logged into the system through the instructor or student interface, respectively.

Each class was divided into two parts. In the first part, the instructor delivered the lecture, while in the second part, students worked on exercises to reinforce the target subject matter. In the class, the instructor referred to the textbook (around seven to nine pages) while lecturing. The lecturing content was displayed on students’ computer screens via a commercial broadcast system. During the lecture, students could request to pause the lecture (i.e. alert the instructor that some kind of engagement is needed at that particular point in the lecture) when they could not remember or understand the lecture content. For that, students had to press the “pause” button on the system. The instructor then would be notified about students’ requests.

A notification message pops up on the instructor’s system interface after a certain proportion of students (one-fifth, one-fourth, one-third, or one-half) press the “pause” button. It was possible that the instructor had finished the lecture before any messages had popped up due to the fact that the lecture time was short or the students believed that the content was easy and/or it was covered very fast. When notifications popped up, the instructor could announce a break so that students could carry out practical exercises or instead decide to continue the lecture.

Data collection and analysis

This included data on how often and how many times the students pressed the “pause” button (called “personal engagement reports”), reasons that they did so and their perceived level of understanding of the lecture content. All of these were collected using the computer system. The data related to the questionnaire in terms of learning style, trust in the instructor, and willingness to use the system were collected in traditional paper-based format. Details of the data collection are described as follows.

Personal engagement reports

The system recorded all instances when students pressed the “pause” button. Each record includes student ID and period of time (in seconds) from the beginning of the lecture until the “pause” button was pressed/a report was made.

Reasons to press the “pause” button/make personal engagement reports

The system collected students’ reasons as to why they pressed the “pause” button/made personal engagement reports. For that, after a student pressed the button, the system prompted the student with these three items: “Could not remember,” “Could not understand,” or “Other reason,” from which students were asked to select one.

Perceived level of understanding the lecture content

After the lecture, the system displayed a 5-point Likert scale, anchored by the end-points “do not understand at all” (1) and “completely understand” (5), and students were invited to select one option.

Survey of learning style, trust in the instructor, and willingness to use the system

This survey was designed following general recommendations of Dong and Hwang (2012). Participants completed it at the end of the last class, in November 2012. The survey items were categorized into the following three dimensions. Dimension 1: Q1 and Q2 were used to measure students’ learning styles, such as extrovert or introvert; Dimension 2: Q3 and Q4 were used to measure students’ trust in the instructor’s arrangement of lecturing content and its delivery; Dimension 3: Q5 and Q6 were used to measure students’ willingness to press the “pause” button when they could not remember or understand the lecture content. Responses to the items were scored using a 5-point Likert scale, anchored by the end-points “strongly agree” (1) and “strongly disagree” (5). A total of 34 students in group A and 56 students in group B completed the survey. The internal consistency of the survey was tested by employing Cronbach’s α; the value was 0.722 for the first dimension, 0.644 for the second dimension, and 0.867 for the third dimension, demonstrating satisfactory reliability of the items.

Learning style and level of students’ trust were viewed as factors that may influence student willingness to press the “pause” button. A t-test was employed to compare willingness to press the “pause” button, trust in the instructor, and learning style of students in group A and group B. Pearson correlation was conducted to explore the relationship between these three dimensions for preparing clustering. Furthermore, K-means is the simplest and most popular algorithm for classification (Jain, 2010). K-means data mining technique was adopted to cluster students according to their responses to the survey.

Results

Pressing the “pause” button

Table 1 shows students’ responses to the survey about their willingness to press the “pause” button. According to the table, most students (60%) agreed that they were willing to press the “pause” button.

Table 1.

Willingness to press the “pause” button.

	Strongly disagree	Disagree	Undecided	Agree	Strongly agree
Group A	0 (0%)	1 (3%)	15 (44%)	12 (35%)	6 (18%)
Group B	1 (2%)	1 (2%)	18 (32%)	26 (46%)	10 (18%)
Total	1 (1%)	2 (2%)	33 (37%)	38 (42%)	16 (18%)

Table 2 shows learning session profiles. To ensure enough data for analysis, data were collected from four sessions in which a proportion of students who pressed the “pause” button exceeded one-fourth of all students in group A and group B. These sessions were called 1st session (in group A), 2nd session (in group A), 3rd session (in group B), and 4th session (in group B) as shown in Tables 4 and 5. In total, 67 students pressed the “pause” button during four sessions: 14 in 1st session, 20 in 2nd session, 12 in 3rd session, and 21 in 4th session.

Table 2.

Learning session profiles.

	Group A		Group B
	1st session	2nd session	3rd session	4th session
Time span of lecture (in seconds)	777	1674	1591	1130
Number of students who attended the lecture	39	36	36	50
Number of engagement reports	14	20	12	21
Reasons for engagement reports	R(12), U(0), O(2)	R(12), U(3), O(5)	R(5), null(7)	R(11), U(4), null(6)
Number of potential engagement reports	8	4	7	13
Accumulated time span of lecture whenever a certain proportion of students had signaled to the instructor.	723 (1/5), 726 (1/4), 729 (1/3)	935 (1/5), 1645 (1/4), 1647 (1/3)	1451 (1/5), 1451 (1/4), 1452 (1/3)	1091 (1/5), 1091 (1/4), 1093 (1/3)

In all, 47 students (70.1%) claimed that the reason that they pressed the “pause” button was because they found it difficult to remember or understand the lecture content, seven students (10.4%) mentioned some other reasons, and 13 students (19.4%) did not give any reasons.

Figures 1 and 2 (the top part) show seating arrangements of students in group A and group B, which students pressed the “pause” button, and when they pressed it. Students’ seats were arranged in four columns and 16 rows. The “stu” label means that a student sat there and the number in red (shown in seconds) illustrates when a student pressed the “pause” button. In addition, the figures (the lower part) show timestamps when students pressed the “pause” button. Reasons why they pressed the “pause” button are indicated in the figures as follows: “R”—unable to remember, “U”—unable to understand, and “O”—other reasons. Several students did not give any reasons (null). Those students who had other reasons or did not give any reasons were interviewed afterwards in order to understand their motives as to why they pressed the “pause” button. The responses were as follows:

When I wanted to practice to catch up the lecture progress.

When I was tired or wanted to sleep.

I needed the instructor to explain the lecture material again.

When others pressed the pause button.

When the lecture material was boring.

Figure 1.

Engagement reports in 1st (left) and 2nd (right) sessions of group A.

Figure 2.

Engagement reports in 3rd (left) and 4th (right) sessions of group B.

Suitable time to react to the pause button having been pressed

Students could request to pause the lecture (i.e. to press the “pause” button) when they did not remember or understand its content. If the instructor pauses the lecture too soon (i.e. after noticing only a very small number of students pressing the “pause” button), the lecture would be interrupted frequently, perhaps too frequently. On the other hand, if the instructor pauses the lecture too late (i.e. after noticing a large number of students pressing the “pause” button), students will be disengaged from learning. Therefore, it is very important to know when the instructor should pause the lecture.

Observation of when students pressed the “pause” button (their “personal engagement reports”) and timestamps of when reports were made showed the time when students requested to pause the lecture. A considerable number of students pressed the “pause” button in a short time span (around 5 seconds), that is, when a quarter to one-third of students did so (see the bottom part of Figures 1 and 2). One may argue that students pressed the “pause” button because of other students sitting next to them. Several students sitting next to each other could agree to press the “pause” button at the same time. However, some instances do not confirm this argument; for example, in the 2nd session, four students who pressed the “pause” button at about the same time (timestamp: 1647 seconds) were located in different seats.

Students pressed the “pause” button due to having problems understanding or remembering the lecture content or some other reasons. If the instructor does not react in time, students’ learning and engagement can be jeopardized. Therefore, it is important, in each session, to draw an effective learning time boundary based on timestamps of when they pressed the “pause” button given the importance of the accumulated load which can be represented as total mental workload in a task. The effective learning time zone can be integrated as all students’ accumulated workload toward the learning material, as shown in Figure 3. It was found that, in each session, an approximate horizontal fit line which links timestamps between a quarter to one-third and all the following requests in the same timestamps could be drawn. The researchers defined the line as an “optimal decision boundary for pausing the lecture.”

Figure 3.

Analysis of optimal time to react to student engagement reports.

Some students perceived that they need more time to process and reflect on the lecture content in order to understand it better but did not press the “pause” button. This implies that the survey data and the pressing of the “pause” button have some sort of correlation. We called potential requests to pause the lecture as “potential engagement reports.”

Eight students in the first session, four students in the second session, seven students in the third session, and 13 students in the fourth session did not press the “pause” button; however, in the survey, they selected “do not understand” or “undecided” items related to their understanding level of the lecture content. At least one-third of students in the class are introverts (Cain, 2012). Even though almost one-third of students in each session (except the second one) needed more time to process and reflect on the lecture content, they did not press the “pause” button.

According to the data, the number of times that students pressed the “pause” button combined with the number of potential engagement reports in four sessions were of more than half of students in a group. Therefore, the optimal decision boundary shows that less than half of students in a group fell into the inefficient learning time zone, while the other half did learn effectively. If the instructor ignores the optimal decision boundary and continues lecturing, then student engagement would be at stake and their learning progress could decrease. Therefore, the optimal decision boundary appeared to be important during the lecture and it signals the instructor to pause the lecture when at least half of students in a group are unable to understand or remember the lecture content or for some other reasons.

Factors that influence students’ willingness to press the “pause” button

Results of the survey of learning style, trust in the instructor, and willingness to use the system are reported in Table 3.

Table 3.

Learning style, trust in the instructor, and willingness to use the system.

Cronbach’s α coefficient	Questions	Group A (n = 34)		Group B (n = 56)
Cronbach’s α coefficient	Questions	M	SD	M	SD
Extrovert learning style	Q1. If I cannot understand or cannot remember the lecture material, I will actively report to the instructor.	3.56	0.79	3.45	0.93
Extrovert learning style	Q2. If I encounter a practical problem, I will actively search for help from the instructor or other students.	4.24	0.61	4.02	0.86
Trust in the instructor	Q.3 Due to trust in the instructor, I will not make an engagement report in the lecture time	3.00	0.85	3.48	0.83
Trust in the instructor	Q4. I will not have an engagement report in lecture time because it is not a respectful behavior toward the instructor.	2.44	0.86	3.21	1.16
Willingness to make an engagement report	Q5. If I cannot remember the lecture material, I will use the system to make an engagement report.	3.65	0.98	3.29	0.97
Willingness to make an engagement report	Q6. If I cannot understand the lecture material, I will use the system to make an engagement report.	3.62	0.92	3.23	0.97

SD: standard deviation.

Pearson correlation analysis was employed to explore the relationship between the survey dimensions (i.e. learning style, trust in the instructor, and willingness to use the system). Results showed that, for group A, trust in the instructor has a significant negative correlation with willingness to use the system (r = −0.397*). Students in group A were clustered by K-means technique as A-1 (n = 20) and A-2 (n = 14) with the trust in the instructor and willingness to use the system (see Table 4). Analysis results showed that students in cluster A-1 had lower value of trust in the instructor (t = 7.25***; p < 0.001) and higher value of willingness to use the system (t = −4.58***; p < 0.001) compared to students in A-2.

Table 4.

Cluster with K-means in group A.

Cluster	Grade level	Department							Extrovert learning style (2~10)	Trust in the instructor (2~10)	Willingness to make an engagement report (2~10)
Cluster	Grade level	a	c	f	h	l	m	Total	Extrovert learning style (2~10)	Trust in the instructor (2~10)	Willingness to make an engagement report (2~10)
A-1 (n = 20)	2nd		11		1			12	7.65	7.30 (high)	6.35 (low)
	3rd				2	3		5
	4th		1			2		3
A-2 (n = 14)	2nd		10					10	8.00	4.64 (low)	8.57 (high)
	3rd	1				1		2
	4th			1			1	2
Independent t-test with A-1 and A-2									t = −0.83	t = 7.25***	t = −4.58***

Results also showed that, for group B, an extrovert learning style significantly correlates with willingness to use the system (r = 0.319*). Students in group B were clustered by K-means technique as B-1 (n = 33) and B-2 (n = 23) with learning style and willingness to use the system (see Table 5). After classification, students in cluster B-1 had significantly lower value pertaining to learning style (t = −5.20***; p < 0.001), and low value of willingness to use the system (t = −9.24***; p < 0.001) compared to students in cluster B-2.

Table 5.

Cluster with K-means in group B.

Cluster	Grade level	Department					Extrovert learning style (2~10)	Trust in the instructor (2~10)	Willingness to make an engagement report (2~10)
Cluster	Grade level	c	f	h	l	Total	Extrovert learning style (2~10)	Trust in the instructor (2~10)	Willingness to make an engagement report (2~10)
B-1 (n = 33)	2nd	23				23	6.70 (low)	7.09	5.33 (low)
	3rd				2	2
	4th		4	4		8
B-2 (n = 23)	2nd	15		1		16	8.57 (high)	7.30	8.22 (high)
	3rd				3	3
	4th	1	1	2		4
Independent t-test with B-1 and B-2							t = −5.20***	t = −0.53	t = −9.24***

We further investigated students’ behaviors in terms of pressing the “pause” button. For that, we derived “Expression rate of pressing the ‘pause’ button” (“making a personal engagement report”) variable (shown in percentage) from dividing the number of pressings of the “pause” button by the sum of the number of pressings of the “pause” button and the number of potential engagement reports.

Cluster A-1 students, who were less willing to use the system but had more trust in the instructor, had 70% of expression rate of pressing the “pause” button (14 actually pressed the “pause” button and 6 made potential engagement reports). Cluster A-2 students, who were more willing to use the system but had less trust in the instructor, had 89% of expression rate of pressing the “pause” button (16 actually pressed the “pause” button and 2 made potential engagement reports) as shown in Table 6.

Table 6.

Engagement reports analysis in four clusters.

	Cluster A-1	Cluster A-2	Cluster B-1	Cluster B-2
Number of personal engagement reports	14	16	20	13
Number of potential engagement reports	(6)	(2)	(12)	(7)
Expression rate of engagement reports	70%	89%	63%	65%

Cluster B-1 students, who were less willing to use the system and who were less extrovert, had 63% of expression rate of pressing the “pause” button (20 actually pressed the “pause” button and 12 made potential engagement reports). Cluster B-2 students, who were more willing to use the system and exhibited a more extrovert learning style, had 65% of expression rate of pressing the “pause” button (13 actually pressed the “pause” button and 7 made potential engagement reports) as shown in Table 6.

Conclusion

Literature suggests that in traditional classroom, students’ effective learning time is normally short (Caldwell, 2007). The main reason is that lessons are delivered in a teacher-centered format and little interaction happens in class (Young et al., 2009). As a result, students are not able to inform the instructor about their learning progress, especially when they experience some difficulties related to the lecture content. The computer system described in this article makes the instructor aware of the perceptions that students have about their learning (Powell et al., 2011) through active reflection (Coates, 2005). The system enables students to press the “pause” button when they could not remember or understand the lecture content.

Results of this study show that most students were willing to press the “pause” button when they find it difficult to remember and understand the lecture. This study analyzed time span when most students pressed the “pause” button and an optimal decision boundary for pausing the lecture was proposed. Finally, results showed factors that influenced student willingness to press the “pause” button. When students had lower level of trust in the instructor and when they were unable to follow the instructor, they were more willing to press the “pause” button. The introvert learning style influenced student willingness to press the “pause” button. Almost one-third of students did not press the “pause” button. Introvert students pressed the “pause” button fewer times/less compared to extrovert students. According to the instructor using the system, it was easy to track whether students are engaged in the lecture and most students agreed that pressing the “pause” button was very useful for learning.

Our findings are in line with research in this area. Quaye and Harper (2015), Slavich and Zimbardo (2012), and Umbach and Wawrzynski (2005) argued that students learning engagement can be influenced by how they perceive their instructor, specifically, their leadership. Therefore, students were more willing to press the “pause” button when they had a lower level of trust in the instructor, and when they were unable to follow them. In contrast to other studies on clickers in which control of class progress was led by the instructor (Bachman and Bachman, 2011; Cain et al., 2009; Caldwell, 2007; Heaslip et al., 2014; Hoyt et al., 2010; Powell et al., 2011; Sevian and Robinson, 2011), students in this study were able to reflect on their progress by pressing the “pause” button and informing the instructor that they wanted a pause or pauses in the lecture. Students learn differently due to their characteristics (Felder and Silverman, 1988) and we know that extroverts are outgoing and easily ask for assistance and that introverts are less active, do not easily ask for help and need longer time to think and reflect on learning content (Cain, 2012), so the results reflect this. Introverts pressed the “pause” button fewer times/less when they were unable to remember or understand the lecture content compared to extrovert students.

Some limitations need to be acknowledged: First, the small sample and that the study was implemented for a short period of time. Classes were on one subject matter/discipline, that is, software engineering. Therefore, in the future, more studies are required to include a bigger sample size, a longer experiment time, and different disciplines in order to make a broader generalization of the results. Another limitation relates to reliability and validity of student self-reports. That is, although the validity of self-report has been reported, external criteria were not applied to verify student self-reports. This issue should be addressed in the future. Furthermore, future study may explore deeper the relationship between student behavior to terms of pressing the “pause” button, learning progress, and learning style. For example, in this study, although about one-third of students pressed the “pause” button, a considerable number of students did not do so when they needed more time to process and reflect on the lecture content.

Based on the results, there are several implications and suggestions. First, to avoid the learners from being cognitively overloaded in the lecture, the instructor needs to implement a mechanism with which learners can press the “pause” button in order to signal that they want some kind of engagement or pause at that particular point in the lecture. Students will then be able to play a greater role in taking control of what is going on in the classroom and the instructor will have greater confidence in their decision to pause the lecture at the appropriate time (i.e. when one-third of students in the class press the “pause” button within a particular timeframe). Second, even though introverts need a pause in the lecture, they do not make their wishes known, or are more reluctant to do so, by pressing the “pause” button. Therefore, the instructor needs to consider this issue and it is suggested that the instructor should not pause the lecture too late (i.e. when more than one-half of students in the class signal their wish or need for a pause in the lecture). Finally, the difficulty level of the content in any lecture will be an important key to efficient and effective learning. Thus, the instructor needs to consider the difficulty or otherwise of the level of the material when considering when such intervention might be needed and also the nature or type of intervention to be provided.

Footnotes

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Jian-Jie Dong is a technology teacher of National Feng-Yuan Senior High School. He received PhD degree in 2014 from the Department of Computer Science and Information Engineering, National Central University, Taiwan. Address: Department of Computer Science and Information Engineering, National Central University, No.300, Jhongda Rd., Jhongli City, Taoyuan County 32001, Taiwan. [email: djj6.tw@gmail.com]

Wu-Yuin Hwang is a full Professor of Graduate School of Network Learning Technology, National Central University, Taiwan. Owing to the outstanding contributions of research in E-learning, he received the Ta-You Wu, Memorial Award from National Science Council, Taiwan, in 2005; and the honor of distinguished professor from National Central University, Taiwan, in 2010. Address: Graduate School of Network Learning Technology, National Central University, No.300, Jhongda Rd., Jhongli City, Taoyuan County 32001, Taiwan. [email: wyhwang@cc.ncu.edu.tw]

Rustam Shadiev is a Professor at the School of Education Science, Nanjing Normal University, China. His research interests include learning and instruction in online synchronous learning environment, human–computer interaction for collaboration, and speech-to-text recognition (STR) technology for learning. Address: School of Education Science, Nanjing Normal University, No. 122, Ninghai Road, Nanjing 210097, Jiangsu Province, China. [email: rustamsh@gmail.com]

Ginn-Yein Chen is a full Lecturer of the Department of Civil and Water Resources, National Chia-Yi University, Taiwan, who has written three books of geographic information system (GIS) application and has been teaching GIS education for 15 years. Address: Department of Civil and Water Resources Engineering, National Chiayi University, No.300, University Road, Chia-Yi City, Taiwan. [email: annie@mail.ncyu.edu.tw]

References

Anderson

Krathwohl

Bloom

(2001) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.

Bachman

(2011) A study of classroom response system clickers: Increasing student engagement and performance in a large undergraduate lecture class on architectural research. Journal of Interactive Learning Research 22(1): 5–21.

Beekes

(2006) The “millionaire” method for encouraging participation. Active Learning in Higher Education 7(1): 25–36.

Blasco-Arcas

Buil

Hernández-Ortega

. (2013) Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance. Computers & Education 62: 102–10.

Bomia

Beluzo

Demeester

. (1997) The Impact of Teaching Strategies on Intrinsic Motivation. Champaign, IL: ERIC Clearinghouse on Elementary and Early Childhood Education.

Brady

Seli

Rosenthal

(2013) “Clickers” and metacognition: A quasi-experimental comparative study about metacognitive self-regulation and use of electronic feedback devices. Computers & Education 65: 56–63.

Cain

Black

Rohr

(2009) An audience response System strategy to improve student motivation, attention, and feedback. American Journal of Pharmaceutical Education 73(2): 1–7.

Cain

(2012) Quiet: The Power of Introverts in a World That Can’t Stop Talking. New York: Broadway.

Caldwell

(2007) Clickers in the large classroom: Current research and best-practice tips. CBE Life Sciences Education 6(1): 9–20.

10.

Campbell

Monk

(2015) Introducing a learner response system to pre-service education students: Increasing student engagement. Active Learning in Higher Education 16: (1) 25–36.

11.

Coates

(2005) The value of student engagement for higher education quality assurance. Quality in Higher Education 11(1): 25–36.

12.

De Gagne

(2011) The impact of clickers in nursing education: A review of literature. Nurse Education Today 31(8): 34–40.

13.

Dong

Hwang

(2012) Study to minimize learning progress differences in software learning class using PLITAZ system. Educational Technology Research and Development 60(3): 501–27.

14.

Felder

Silverman

(1988) Learning and teaching styles in engineering education. Engineering Education 78(7): 674–81.

15.

Gettinger

Ball

(2007) Best practices in increasing academic engaged time. In: Thomas

Grimes

(eds) Best Practices in School Psychology V. Bethesda, MD: National Association of School Psychologists, pp. 1043–75.

16.

Graham

Tripp

Seawright

. (2007) Empowering or compelling reluctant participators using audience response systems. Active Learning in Higher Education 8(3): 233–58.

17.

Hasler

Kersten

Sweller

(2007) Learner control, cognitive load and instructional animation. Applied Cognitive Psychology 21(6): 713–29.

18.

Heaslip

Donovan

Cullen

(2014) Student response systems and learner engagement in large classes. Active Learning in Higher Education 15(1): 11–24.

19.

Hoyt

McNulty

Gruener

. (2010) An audience response system may influence student performance on anatomy examination questions. Anatomical Sciences Education 3(6): 295–9.

20.

Jain

(2010) Data clustering: 50 years beyond K-means. Pattern Recognition Letters 31(8): 651–66.

21.

Kay

LeSage

(2009) Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education 53(3): 819–27.

22.

Kearsley

Schneiderman

(1999) Engagement theory: A framework for technology-based learning and teaching. Available at: http://home.sprynet.com/~gkearsley/engage.htm (accessed 5 February 2015).

23.

Keough

(2012) Clickers in the classroom: A review and a replication. Journal of Management Education 36: 822–47.

24.

Laxman

(2011) A study on the adoption of clickers in higher education. Australasian Journal of Educational Technology 27(8): 1291–303.

25.

Powell

Straub

Rodriguez

. (2011) Using clickers in large college psychology classes: Academic achievement and perceptions. Journal of Scholarship of Teaching and Learning 11(4): 1–11.

26.

Quaye

Harper

(2015) Student Engagement in Higher Education: Theoretical Perspectives and Practical Approaches for Diverse Populations. New York: Routledge.

27.

Schlechty

(2002) Working on the Work. San Francisco, CA: Jossey-Bass.

28.

Sevian

Robinson

(2011) Clickers promote learning in all kinds of classes—Small and large, graduate and undergraduate, lecture and lab. Journal of College Science Teaching 40(3): 14–8.

29.

Shadiev

Hwang

Huang

(2015) A pilot study of facilitating cross-cultural understanding with project-based collaborative learning activity in online environment. Australasian Journal of Educational Technology 31(2): 123–39.

30.

Shadiev

Hwang

Yeh

. (2014) Effects of unidirectional vs. reciprocal teaching strategies on web-based computer programming learning. Journal of Educational Computing Research 50(1): 67–95.

31.

Skinner

Belmont

(1993) Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology 85(4): 571–81.

32.

Slavich

Zimbardo

(2012) Transformational teaching: Theoretical underpinnings, basic principles, and core methods. Educational Psychology Review 24(4): 569–608.

33.

Umbach

Wawrzynski

(2005) Faculty do matter: The role of college faculty in student learning and engagement. Research in Higher Education 46(2): 153–84.

34.

Xie

Salvendy

(2000) Prediction of mental workload in single and multiple tasks environments. International Journal of Cognitive Ergonomics 4(3): 213–42.

35.

Young

Robinson

Alberts

(2009) Students pay attention! Combating the vigilance decrement to improve learning during lectures. Active Learning in Higher Education 10(1): 41–55.