Abstract
Background
Research indicates that laboratory experiences facilitate student learning.
Objective
This study aimed to evaluate the impact of a virtual laboratory (CyberRat) on student performance in Learning and Behavior classes. Additionally, we evaluated the impact of this laboratory as a function of class modality.
Method
We incorporated CyberRat laboratories into two classes (one in-person and one asynchronous online), with two classes of the same modality serving as controls. The same instructor taught all four classes. We evaluated student performance using target exam items and class grades. We also collected self-report data regarding the perceived usefulness of CyberRat and perceptions about the class.
Results
Performance on both learning measures was higher for CyberRat classes than for control classes. This main effect was fueled by the differential benefit of CyberRat in online but not in face-to-face classes. Online students in the CyberRat condition benefited more than their in-person peers.
Conclusion
Incorporation of a virtual laboratory appears to bolster student success; however, the impact of this type of laboratory may be limited to online classes. Teaching Implications: Educators should evaluate the efficacy of virtual laboratory experiences for in-person learning because the benefit seems to lie uniquely for online classes.
Laboratory experiences are widely assumed to be crucial supplements to the classroom experience, fostering student engagement and learning; however, it is essential to test the pedagogical effectiveness of lab work (Hofstein & Lunetta, 1982, 2003; Hofstein & Mamlok-Naaman, 2007; Tobin, 1990). Across disciplines, researchers have found significant improvements in student learning in courses with laboratory experiences compared to those without laboratories (e.g., Abdrabou & Shakhatreh, 2021; Catena & Carbonneau, 2019; Kallarackal, 2021; Thieman et al., 2009). With the proliferation of online courses, virtual lab experiences in the form of computer simulations have become increasingly important pedagogical tools (Brinson, 2015). This is especially relevant because students in online courses tend to underperform compared to students in traditional, in-person courses (Francis et al., 2019; Helms, 2014; Spencer & Temple, 2021; Xu & Jaggars, 2014), and report lower satisfaction (Roach & Lemasters, 2006; Tratnik et al., 2019), although see Means et al. (2009) and Soffer and Nachmias (2018) for reports on better performance for online students. When compared to in-person labs, virtual labs can be less expensive, safer to use, more accessible to people needing accommodations and/or in different geographic locations, and potentially decrease errors that can occur when physical lab conditions are not ideal and/or lab equipment is improperly handled (Brinson, 2015; Heradio et al., 2016; Santos & Prudente, 2022; Tatli & Ayas, 2010). As with traditional in-person labs, virtual labs have a positive impact on student learning (Brinson, 2015; Heradio et al., 2016; Santos & Prudente, 2022).
Increased demand for online courses and the need for virtual lab experiences apply to most disciplines, including psychology. Within psychology, lab experiences are especially important when teaching principles of learning and behavior (Karp, 1995). Over the years, various lab experiences have been incorporated into learning and behavior courses (e.g., Epting & Green, 2011; Goodhue et al., 2019). Live rat laboratories were once the foundation for the learning and behavior curriculum; however, given costs, ethical regulations, and time/space limitations, they have become increasingly difficult to implement. As a result, virtual laboratories are considered effective alternatives for teaching principles of behavior analysis (Graham et al., 1994; Shimoff & Catania, 1995). Graf (1995), Hyten (1989), Mulick (1992), and Shimoff and Catania (1995) reviewed some early computer programs designed to teach behavior analysis principles. However, student learning outcomes when using these early programs were not assessed. Technological advances in the mid-1990s allowed the development of Sniffy the Virtual Rat, a software program that simulates the behavior of a real rat in an operant chamber (Graham et al., 1994). Assessments of learning outcomes when using Sniffy yielded contradictory findings. Venneman and Knowles (2005) found that students who used Sniffy scored significantly higher on exams than those who did not. However, Lewis (2015) found that the virtual rat lab experience did not lead to significant differences in exam scores when comparing students who used Sniffy with those who experienced human demonstrations.
Researchers have also argued the importance of studying student preferences, perceptions, and attitudes toward laboratory experiences (e.g., Elcoro & Trundle, 2013). When using a live rat lab to teach basic learning and behavior principles, Karp (1995) noted that most students rated the rat lab as very helpful in learning course concepts. Epting and Green (2011) used human operant lab exercises to teach behavior analysis principles. They found that students reported these exercises useful in understanding basic concepts and appreciating experimental procedures. Venneman and Knowles (2005) had mixed results when using Sniffy. About half of their students made positive comments regarding an increased understanding, enjoyment, and interest in concepts, as well as liking Sniffy. The other half made negative comments regarding the cost and difficulty in understanding the software.
Given the mixed findings in performance outcomes and perceptions around virtual rat labs, it is important to continue to investigate the efficacy of virtual labs in learning and behavior courses, especially with the rise in demand for online and hybrid courses. To that end, we used CyberRat (Ray, 2003), a versatile web-based system that allows students to engage in a variety of laboratory experiments. Unlike Sniffy, which uses an animated rat, CyberRat generates live-action simulations from more than 1,600 pre-recorded video clips of live rats in an operant chamber. The simulation is responsive to user interaction with the system; for instance, it is responsive to manual reinforcer delivery (Ray, 2011). Although there have been a few commentaries on CyberRat (Iverson, 2011; Phelps, 2011; Ray, 2011), to our knowledge, a comprehensive assessment of learning outcomes using CyberRat has not been conducted. Therefore, we developed a series of virtual laboratories using CyberRat and investigated its impact on learning and student perceptions in in-person and online courses. As suggested by Lewis (2015), we conducted this study with the same instructor across different semesters to minimize confounds.
Based on previous research indicating significant improvements in learning for students in courses with in-person and virtual laboratory experiences, we hypothesized that the CyberRat lab experience would improve student learning (as measured by embedded exam questions and overall course grade). In addition, given previous research findings of poorer academic performance and lower satisfaction ratings in online courses compared to in-person, we explored whether this virtual laboratory experience would have a differential impact on student learning and perceptions in online versus in-person classes.
Method
Participants
We recruited undergraduate students enrolled in upper-level learning and behavior courses at Kennesaw State University (KSU). At the beginning of each semester, the instructor briefly presented information about the study to their classes, indicating that the study's purpose was to explore whether the types of instructional activities used affect student performance. The instructor assured students that the class's content, activities, and assessments would not be impacted regardless of whether they participated. Participants provided written consent. At the end of the semester, we provided participants with detailed information regarding the study as part of the debriefing process. KSU's Institutional Review Board approved the study (approval # 20-048).
Our comparisons involved four course sections taught by the same instructor, including two in-person sections in the Fall semester of 2019 (N = 62 enrolled, 59 consented) and two online sections in the Spring semester of 2020 (N = 62 enrolled, 39 consented), for a total of 98 recruited participants. To protect confidentiality, no demographic information was collected. In Fall 2023, our sample consisted of 42.8% White, 26.4% Black/African American, 15.5% Hispanic, 5.8% Asian, and 9.5% of other/undeclared backgrounds. The average age of undergraduate students was 21.6 years, and 38.3% were first-generation students (Kennesaw State University, 2023). At the university, 50.6% of students identified as female, but the psychology major tends to have a higher percentage of female students. In the Radow College of Humanities and Social Sciences (RCHSS) at KSU, enrollment for Fall 2023 was 65.6% female (Kennesaw State University Fact Book, 2024).
Materials
CyberRat Application
CyberRat (version 5.0; Ray, 2019) is an interactive software program that provides an animal operant laboratory experience (Ray & Miraglia, 2011). It consists of a simulated operant chamber. The user can watch the rat (via interactive video clips) in real-time and provide reinforcement manually or program rapid simulations according to the parameters they set. A variety of graphs can be generated, including cumulative records of each session. Instructors can access student results through a web-based portal. Through a Teaching Incentive Grant awarded by RCHSS at KSU, all students in the treatment condition of our study were provided with free access codes to download the software.
Laboratory Exercises
We developed four laboratory exercises for the treatment condition. The labs used the CyberRat software and focused on the topics of shaping, schedules of reinforcement, extinction, and stimulus control. Prior to each lab, students received online or in-person instruction on the underlying concepts. Full labs are available for download on the Open Science Foundation platform (Mallavarapu et al., 2025).
Shaping. The objectives for the first laboratory exercise included learning how new behaviors can be acquired through the reinforcement of successive approximations. Students created a shaping plan, used reinforcement to shape a novel behavior, and interpreted cumulative records. To complete this activity, students watched a rat in real time and manually reinforced behaviors that were gradual approximations to the goal behavior of pressing the bar (e.g., proximity to the bar, wall exploration, physical contact with the bar). After completing one session with at least 10 bar presses, students ran two additional fast simulations (i.e., computer-simulated 60-min sessions without the students watching the videos or delivering reinforcement to the virtual rat) in which bar pressing was reinforced using a continuous reinforcement schedule. The students generated graphs of these sessions and completed a lab report summarizing aspects of their training sessions, reflecting on elements of their training process, and comparing response rates on the graphs.
Schedules of Reinforcement. The objectives for this laboratory included learning how responding differs between ratio and interval schedules of reinforcement and how ratio strain and schedule thinning affect behavior. In addition, the students acquired skills in yoking ratio and interval schedules for comparison purposes, comparing cumulative records under varying contingencies, and using schedule thinning to maintain responding under lean reinforcement contingencies. In the first part of the activity, the students selected a rat with a training history of bar pressing reinforced using continuous (fixed ratio 1, or FR1) reinforcement. They then ran three 30-min fast simulations under a variable ratio 5 (VR5) schedule of reinforcement and used the average time between reinforcers to program and run simulations using the yoked variable interval (VI) schedule of reinforcement. In the second part of the activity, students added a new rat with a history of continuous (FR1) reinforcement for bar pressing. They then ran a fast simulation of an FR62 schedule to demonstrate ratio strain. Then, the students were instructed to formulate and implement a plan that would result in the rat responding at a high rate with the FR62 schedule (i.e., use schedule thinning). Students summarized aspects of their training sessions, compared response patterns and rates between VR and VI schedules, explained the difference in response rates, identified ratio strain, and described their schedule thinning process.
Extinction. The objectives for the third laboratory included learning how extinction alters behavior and how different reinforcement histories can alter the extinction rate. The students analyzed and compared cumulative records of extinction. Students first used a new rat with a training history involving bar pressing reinforced on a continuous (FR1) schedule. They ran fast simulations of five 30-min extinction sessions. Then, for comparison, the students used the rat they had previously trained (through schedule thinning) to respond rapidly under FR62 contingencies and ran five (or more) 30-min extinction sessions. Students summarized aspects of their training sessions (e.g., number of bar presses), compared the graphs and extinction rates between the two rats, described how the response pattern changed across sessions, and identified evidence of an extinction burst.
Stimulus Control. The primary objective of the laboratory activity was to understand how discriminative stimuli can influence behavior. The students used discriminative stimuli and reinforcement schedules to produce differential responses and compare responses across different contingencies. Students used a rat with a training history in which bar pressing was reinforced using continuous (FR1) reinforcement. They ran fast simulations in which bar pressing was reinforced on an FR1 schedule when a green light (discriminative stimulus or SD) was present but was not reinforced when a red light (s-delta) was present. The student could choose fixed or variable schedules for the discriminative stimulus and the time interval for each light. The goal was to achieve a discrimination index of over 90%. Students graphed their training sessions. Students summarized aspects of their training sessions, reflected on their training experience, and described an example in which they could apply discrimination training to a real-world situation.
Probe Questions
We developed 16 multiple-choice questions to assess students’ knowledge of the concepts covered in the laboratories and incorporated them into course exams. The questions targeted multiple levels of the revised Bloom's Taxonomy (Anderson & Krathwohl, 2001), and we included four questions for each of the four topics (shaping, schedules of reinforcement, extinction, and stimulus control). Within each topic, one question was definitional (e.g., “______ is the process of establishing a new behavior by delivering reinforcement contingent on behaviors that are successive approximations to the target behavior.”), representing the remembering level of Bloom's Taxonomy (Anderson & Krathwohl, 2001). Two questions focused on details of the concepts or procedures (e.g., “On a cumulative record graph, ______ indicates the organism is not responding.”). These targeted the understanding level in Bloom's Taxonomy (Anderson & Krathwohl, 2001). The final question targeted the analyzing and applying levels of Bloom's Taxonomy (Anderson & Krathwohl, 2001), requiring the student to use the information in a different setting (e.g., “Lila works as a copy editor. She was typically asked to proofread approximately four articles each day. However, after her company merged with another company, her new boss increased that requirement to 20 articles each day. Lila found herself staring into space rather than proofreading articles and eventually just stopped coming to work. What behavioral phenomenon best describes Lila's lack of responding after the new work requirements?”). In summary, the probe questions were related to the four course concepts and represented the four foundational tiers in Bloom's Taxonomy.
Self-Report Surveys
All students who consented to the research were asked to complete an end-of-semester survey. Students were asked to indicate their agreement (on a 10-point scale ranging from very strongly disagree to very strongly agree) with statements that the information they learned in the course would be useful to their life, that they found the course enjoyable, that the activities and assignments in the course facilitated their learning, and that the course was interesting. Additionally, there were open-ended questions asking what students found enjoyable and not enjoyable about the course. Students in the treatment condition answered 19 additional items regarding their experience with CyberRat. Students rated these items on a 10-point scale ranging from very strongly disagree to very strongly agree. Items included that working with CyberRat was enjoyable, easy, and interesting; that using CyberRat helped them to understand the four targeted concepts; that using CyberRat required a lot of patience, was difficult, and was boring; that CyberRat was an important aspect of the course; that CyberRat helped them to understand real behaviors; that CyberRat helped them to understand cumulative records; and that they did not enjoy working with CyberRat. Lastly, students in the treatment group were asked two open-ended questions about what they enjoyed and did not enjoy about CyberRat.
Procedure
We conducted this study over two semesters. The same instructor taught four sections of an upper-level learning and behavior course – two face-to-face (F2F) sections in Fall of 2019 and two online sections in Spring 2020. Each semester, one course was randomly selected as the treatment class and the other as the control class. We provided students in the treatment sections with a semester-long subscription to the CyberRat software. The students in the treatment sections completed the CyberRat lab activities described above as homework assignments. Students in the control condition did not complete the CyberRat lab activities, nor were they provided with an alternative assignment. While the primary comparison was across sections within the same modality, the course content between the two modalities was kept as similar as possible. The online courses used webpages designed with Softchalk that contained short, voice-over lecture videos (with the same content as the in-person lectures) and interactive concept check questions and activities that replaced in-person review questions. Many in-person class activities were converted to weekly assignments in the online class. We collected academic performance data throughout the semester, recording student performance on the 16 probe questions related to topics covered in the CyberRat activities. To assess the potential for emergent or Gestalt benefits from interacting with the virtual laboratory that extend beyond the curated probe questions targeting circumscribed information contained in the laboratories, we included overall course grade as an additional measure. Furthermore, considering that retention, progression, and graduation (RPG) is a foundational metric used to measure student success, our inclusion of course grade as a dependent variable provides useful insight into course-level success as a function of the incorporation of a virtual laboratory experience. At the end of the semester, we also emailed the self-report survey to participants who had consented.
Data Analysis
Academic Performance
We calculated the number of correct probe questions answered and the final course grade (as a percentage) to evaluate student performance. The overall course grade was determined based on exams, quizzes, assignments/in-class activities, two papers, and an oral presentation. We then conducted a 2 (F2F/online) by 2 (treatment/control) analysis of variance (ANOVA) on each measure, examining both main effects and interactions. Power analyses were conducted for all inferential analyses and were found to be > .90, indicating adequate samples sizes for our analyses. Our alpha level was .05.
Self-Report Survey
Open-Ended. There were open-ended questions asking what students found enjoyable and not enjoyable about the course and CyberRat. We conducted a thematic analysis to capture the breadth and depth of the construct of “enjoyability” using a coding reliability methodology (Joffe, 2011). In this approach, a primary rater examined the raw data and developed a coding scheme based on participant responses for each question. Descriptions of themes identified are in Tables 1 and 2. Coding was performed by hand using a Microsoft Excel spreadsheet. The first two authors, both with prior experience in thematic analysis, independently scored each theme as “1” (response applied to that theme) or “0” (response did not apply to that theme). Percent agreement was then calculated for each theme. Average inter-rater reliability across the 29 themes was 97.8% (minimum 89.4%, maximum 100%). Joffe (2011) states that inter-rater reliability exceeding 75% signifies a reliable coding framework. In the case of disagreement, the results from the primary rater were used. Once the thematic analysis was complete, we calculated the percentage of student responses that fell into each theme and then compared the percentages across both modalities (online/F2F) and condition (control/treatment) using Chi-Square Tests of Independence. We set the significance level at .05 for all analyses.
Enjoyability of CyberRat: Responses to Open-Ended Items.
Note. Description of themes: Helped me learn (CyberRat helped learn course concepts); Interactiveness (interactive nature of the labs, including ability to control variables, occurring in real time, hands-on, etc.); No response (question left blank); Software usability (CyberRat software was either easy to use or difficult); Specific virtual rat (positive comments about the virtual rat); Time involved (time taken to complete the labs– either that it did not take too much time, or that it was tedious and required patience); Nothing unenjoyable (nothing unenjoyable about CyberRat labs); Did not help me learn (CyberRat did not help with learning course concepts).
Enjoyability of Learning and Behavior Class: Responses to Open-Ended Items.
Note. Description of themes: Applicable (course material was applicable to real-life); Concepts (enjoyed or did not enjoy course material in general, or a specific concept); Instruction (enjoyed or did not enjoy instructional methods – lectures, explanations of concepts, examples given, course organization/format, instructor engagement with the class, instructor relationship with students, instructor personal characteristics, pace of instruction, grading, etc.); No response (question left blank); Interactiveness (mentioned interactive, hands-on activities and demonstrations); Nothing enjoyable (did not enjoy anything about the course); Nothing unenjoyable (nothing unenjoyable about the course); Difficulty (some components of the course were too hard, too difficult, confusing, or overwhelming); Specific assignment/exam (disliked exams, quizzes, readings, assignments for a reason other than difficulty or workload); Workload (workload was too high); CyberRat (did not enjoy CyberRat); Not applicable (course material was not applicable to real-life).
Closed-Ended. For the self-report survey, we summed the scores on multiple questions to create three composite variables: Perceived Targeted Impact (questions pertaining to how the labs helped them understand the concepts covered in the labs), Perceived Global Impact (questions about CyberRat's usefulness in the course overall), and Usability (questions pertaining to the enjoyment and use of the software). We then used one-way ANOVAs (α = .05) to compare these composite variables between online and F2F students who used the CyberRat software.
Results
Performance Measures: Probe Questions and Course Grade
As aforementioned, we gathered two types of academic performance information: Correct Probe Questions (out of 16) and Course Grade (specifically, percentage). This dual approach provides information about the targeted impact of CyberRat on student performance regarding specific Learning and Behavior topics that were underscored in the laboratory activities, as evaluated by Probe Questions. Additionally, this approach provides a litmus test for the potential emergent effect of CyberRat on overall course performance, as evaluated by Course Grade. We ran a two (F2F/online) by two (treatment/control) analysis of variance (ANOVA) on each measure, examining both main effects and interactions.
Data Exclusion Criteria
Before data exclusion, 50 students were recruited in the treatment groups across Fall 2019 and Spring 2020. Data from participants in the treatment groups who did not complete all four of the CyberRat laboratories were not included in academic performance or self-report analyses. After data exclusion, data from 46 participants in the CyberRat condition were analyzed (28 F2F, 18 online asynchronous). Forty-eight students (28 F2F, 20 online asynchronous) participated in the control group across the two semesters.
Probe Questions
Probe questions were embedded in two exams at different times in the semester. Some students did not complete one or either of those exams. Data from students who did not complete all 16 probe questions were removed from this analysis. The resultant participant groups for this analysis were 45 in the treatment group (28 F2F, 17 online) and 42 in the control group (26 F2F, 16 online). We computed a between-groups two-way ANOVA incorporating the variables Modality (F2F, online asynchronous) and CyberRat (treatment, control). See Figure 1. There was no significant main effect on course modality. Online (M = 11.94, SD = 2.95) and F2F (M = 12.72, SD = 1.65) students essentially answered the same number of Probe Questions correctly, F(1, 87) = 3.04, p = .09. There was a significant main effect for CyberRat. As anticipated, students in the treatment classes scored significantly higher (M = 12.78, SD = 1.77) on the Probe Questions than their control-group peers (M = 12.05, SD = 2.64), F(1, 87) = 4.81, p = .031, η2 = 0.06. There was a significant interaction between CyberRat and Modality, F(1, 87) = 7.29, p = .008, η2 = 0.08. Broadly speaking, the impact of CyberRat depends on the course modality. Specifically, for F2F classes, there was no difference in Probe Question performance between treatment (M = 12.61, SD = 1.50) and control groups (M = 12.85, SD = 1.83); however, for online classes, there was evidenced benefit from CyberRat (M = 13.06, SD = 2.16) compared to their control group peers (M = 10.75, SD = 3.26). See Figure 1.
Probe Questions Correct as a Function of CyberRat Use and Class Modality. Note. * Significant at .05.
Course Grade
Three students in the control group withdrew from the class; therefore, they did not have a course grade. The resultant participant groups for this analysis were 46 in the treatment group (28 F2F, 18 online) and 45 in the control group (28 F2F, 17 online). We computed a between-groups two-way ANOVA incorporating the variables Modality (F2F, online asynchronous) and CyberRat (treatment, control). Please see Figure 2. There was a significant main effect for course modality. F2F students had significantly higher Course Grades (M = 86.88, SD = 6.36) than their online peers (M = 81.87, SD = 9.78), F(1, 91) = 10.24, p = .002, η2 = 0.11. There was a significant main effect for CyberRat. As anticipated, students in the treatment classes earned higher (M = 86.68, SD = 7.53) Course Grades than their control-group peers (M = 83.19, SD = 8.51), F(1, 91) = 7.89, p = .006, η2 = 0.08. A significant interaction was found between CyberRat and Modality, F(1, 91) = 6.62, p = .01, η2 = 0.07. As with Probe Question performance, the impact of CyberRat on Course Grade depended on the course modality. Specifically, there was no difference in Course Grade between treatment (M = 87.97, SD = 7.20) and control groups (M = 86.69, SD = 5.52) for F2F classes; however, there was significant benefit of CyberRat for online students (M = 86.06, SD = 8.19) compared to their control group peers (M = 77.43, SD = 9.55).
Course Grade as a Function of CyberRat Use and Class Modality. Note. * Significant at .05.
Self-Report Measures: Closed-Ended Items
As described earlier, participants in the treatment and control classes completed an end-of-semester survey designed to assess student perceptions regarding the learning and behavior course. Additionally, students in the treatment classes provided their perceptions about the utility of CyberRat for understanding the target topics, the global utility of CyberRat for learning, and the platform's usability.
Perceptions of Learning and Behavior Course
All participants were asked to complete an end-of-the-semester questionnaire with four items designed to assess overall perceptions of the learning and behavior course. Items included the usefulness and enjoyability of the course, how much the course facilitated their understanding of learning and behavior, and an evaluation of how interesting the course was. We created a composite variable consisting of these four survey items (Perceptions of Course, Cronbach's α = .97). Composite scores could range from 4 to 40, with higher scores indicating more positive perceptions of the course. F2F students (M = 35.37, SD = 8.15) and online students (M = 31.97, SD = 7.95) rated the course equally, F(1, 77) = 1.91, p = .17. Although students in the treatment group (M = 35.55, SD = 6.52) rated the course more positively than their control group peers (M = 32.78, SD = 9.43), this difference was not statistically significant, F(1, 77) = 3.52, p = .06. There was no course modality by treatment/control interaction, F(1, 77) = 0.36, p = .55.
Utility of CyberRat for Understanding the Five Targeted Topics
There were four topics covered in individual CyberRat Laboratories (Shaping, Schedules of Reinforcement, Stimulus Control, and Extinction), and the topic of Cumulative Records was a common theme among them. Students in the treatment group were asked to indicate on a scale of 1 (Very Strongly Disagree) to 10 (Very Strongly Agree) how much they agreed with “Using CyberRat helped me understand _______________” for each of the five topic areas. We created a composite variable (Perceived Targeted Impact, Cronbach's α = .95) by summing the responses of the five relevant survey items. Perceived Targeted Impact scores could range from 5 (very low utility) to 50 (very high utility). Students generally indicated that using CyberRat was useful in understanding these topics (M = 40.28, SD = 9.27). F2F students’ rating of the utility of CyberRat (M = 41.00, SD = 8.21, n = 25) did not differ from their online peers (M = 39.0, SD = 11.02, n = 15), F(1, 39) = 0.40, p = .53.
Global Utility of CyberRat
To gain a more global understanding of students’ perceptions of CyberRat's utility, we asked them to rate (on a scale of 1 to 10) CyberRat's importance for understanding learning and behavior, as well as real-world behavior, and whether future classes should continue to use it. We created a composite variable (Perceived Global Impact, Cronbach's α = .97) by summing the responses of three relevant survey items. Perceived Global Impact scores could range between 3 (very low utility) and 30 (very high utility). Students generally indicated that CyberRat was useful for these global facets (M = 23.13, SD = 7.26). F2F student ratings of the global utility of CyberRat (M = 22.76, SD = 7.25, n = 25) were not different from their online peers (M = 23.78, SD = 7.49, n = 15), F(1, 39) = 0.17, p = .69.
Perceptions About the Usability of CyberRat
Participants in the treatment group were asked seven questions on the post-test related to the usability of the application, with Cronbach's α = .86. Composite scores could range from 7 to 70, with higher scores indicating more positive perceptions about the usability of CyberRat. Composite scores consider factors such as ease of use, level of patience required, difficulty, and general like/dislike. There was no difference between usability ratings for F2F students (M = 51.08, SD = 11.74) and ratings for online students (M = 51.87, SD = 11.60), F(1, 39) = 0.04, p = .84.
Self-Report Measures: Open-Ended Items
Enjoyability of CyberRat Application
Students noted that CyberRat helped them learn (38.3%) and that they enjoyed the interactiveness of the platform (31.9%). Over 40% of students indicated that the time involved in engaging with CyberRat was not enjoyable compared with 2% who indicated that time use was enjoyable, which was a significant difference, Χ2(1) = 4.97, p = .03. Chi-squares for other factors were not statistically significant (Table 1).
Enjoyability of Learning and Behavior Class
Students indicated that they enjoyed the learning and behavior course concepts (33.70%) and their applicability to real life (37.90%). Forty-two percent of students indicated that there was “nothing unenjoyable” or did not indicate anything when asked about what was unenjoyable. Over 17% of students indicated that they did not enjoy the difficulty of the class. Interestingly, 21% of students in the Control group courses mentioned that the course was interactive, whereas 4.3% of the CyberRat students mentioned that the course was interactive, Χ2(1) = 5.91, p = .02. This chi-square did have one cell with a count less than 5 (Treatment group mentioned Interactiveness = 2; Treatment group did not mention Interactiveness = 45). Chi-squares for instruction, course concepts, or application to real life were not significant (Table 2).
Discussion
In this investigation, we provided university students in online and in-person learning and behavior courses with a virtual laboratory experience in which they investigated concepts such as shaping, schedules of reinforcement, extinction, and stimulus control by training a virtual rat. We found that engaging in the virtual laboratory activities improved learning (as measured by probe questions on exams and overall course grade) compared to classes without the virtual laboratory. However, these learning gains were evident only for students enrolled in the online course, as the F2F students in the virtual laboratory group did not outperform their control group peers. In general, students had favorable (above the midpoint) views of the learning and behavior course. However, that did not differ based on whether the students engaged in the virtual laboratory experience, nor did it differ based on class modality. Similarly, students in the course sections that used CyberRat generally had favorable (above the midpoint) views of its usefulness and usability and felt that it contributed to their learning. No differences were seen between the online and in-person courses on these measures.
Although scholars have previously described the possible benefits of using CyberRat software in educational settings (Iverson, 2011; Phelps, 2011; Ray & Miraglia, 2011), our study was the first to empirically evaluate its effectiveness in improving learning in an undergraduate learning and behavior course. Our finding that incorporating CyberRat laboratory exercises benefited student learning is consistent with previous meta-analysis findings on the effectiveness of virtual laboratories in various disciplines (Brinson, 2015; Santos & Prudente, 2022). Previous scientific evaluations of virtual rat labs (i.e., Sniffy) to enhance learning in psychology courses had yielded mixed results. Our results were consistent with those of Venneman and Knowles (2005), who found significant improvements in exam scores among students who completed the virtual laboratories. However, students in another study showed no increase in exam scores when comparing virtual laboratories with human demonstrations (Lewis, 2015).
Importantly, previous studies investigating virtual rat laboratories (Lewis, 2015; Venneman & Knowles, 2005) were conducted in F2F classrooms. Our study was the first to compare the impact of a virtual rat lab across two course modalities: Asynchronous online and F2F. We found that including the virtual lab improved learning for students in the online sections but not for F2F students. A potential explanation for this differential effect may be found in the level of student engagement and active learning between the course modalities. Compared with an asynchronous, online course, a F2F course typically provides more opportunities for interaction (e.g., student-to-instructor, peer-to-peer) and more instructor-directed active learning experiences (e.g., in-class activities). Students view F2F classes as more engaging behaviorally and emotionally than online classes (Cooper, 2018; Lee & Wong, 2024). In addition, increased engagement in F2F classes appears to be related to the elevated performance in fully in-person classes compared to online asynchronous classes (Cooper, 2018; Diong et al., 2023). Although the online sections had a variety of active learning opportunities (e.g., self-check questions and active learning activities embedded in online lessons), the level of student engagement was likely lower than their F2F peers. In the already engaging and interactive learning environment of an F2F classroom, perhaps the additional active learning experience of the laboratory exercises provided minimal benefit. However, in an online course setting with less interaction and more independent learning, the virtual laboratory may have provided a meaningful increase in active learning, resulting in improved performance. Future research could help to further examine the reasons behind these differences.
Online students had lower overall course grades than the F2F students and benefited more from the CyberRat labs. Despite these performance differences, in the closed-ended perception questions, students in these modalities rated the course similarly as it related to their enjoyment of and interest in the course, as well as the usefulness of the course. In general, students rated the course favorably, and this perception also did not vary between the CyberRat and no CyberRat sections. Thematic analysis of our open-ended questions showed that student comments were generally similar between online and F2F sections and between CyberRat and no CyberRat conditions. Students mentioned course instruction in both the enjoyable and not enjoyable categories. Students listed that they enjoyed the course concepts and applicability to real life, but did not enjoy the difficulty of the course.
The thematic analysis of qualitative responses yielded a difference in the number of responses that mentioned interactiveness (i.e., hands-on or interactive activities or class demonstrations). Overall, only 12 students (of 95) had responses that fell into this category. Contrary to expectations, more students in the control classes (n = 10) had responses that fell into this category than did the treatment classes (n = 2). Although we could not run a chi-square comparing course modality due to low expected cell counts, 10 of the 12 responses in this category were from students in F2F classes, and of those 10 students, nine were in the control condition. In-person students mentioned class participation (n = 1), hands-on or in-class activities (n = 4), interactive class demonstrations (n = 4), or just mentioned “how interactive it was” (n = 1). The two online students (one control and one treatment) whose comments fell in the interactive category mentioned “interactive and though (sic) out lecture presentations” and the interactive “Kahoots” challenges (n = 1). None of these responses mentioned CyberRat, and the activities mentioned in these responses were consistent across treatment and control course sections. For those students who completed the CyberRat laboratory, over 31.9% of responses mentioned something about the interactive nature of the labs (i.e., they could control the variables, that it was in real-time or “hands-on”) as something they enjoyed about the virtual labs. Given the small sample size, that almost all of the comments were from in-person students, and that the comments were not directly related to CyberRat, we are not confident that the difference between the control and treatment groups would hold up in a replication.
Students rated CyberRat favorably regarding its usefulness in the course overall and in learning the specific concepts targeted in the lab. These ratings did not differ by course modality. Although the online students benefited more from the virtual laboratories, they did not perceive them as more beneficial than their F2F peers. The open-ended comments reflected that many students (38.3%) perceived CyberRat to be helpful in their learning (e.g., “It made learning about shaping using reinforcements much easier because I am a visual learner”, “I like how it let us try multiple times. It was a better way to learn hands-on without being in an actual lab”). These favorable perceptions are consistent with past examinations involving virtual rats (Elcoro & Trundle, 2013; Hunt & Macaskill, 2017; Lewis, 2015). However, when virtual laboratories were directly compared with laboratories involving live rats, students rated the live rats as more enjoyable and interesting (Elcoro & Trundle, 2013) and more helpful in their learning (Hunt & Macaskill, 2017). Similarly, in comparing perceptions of students taught using human demonstrations with those who learned through a virtual rat laboratory, Lewis (2015) found that students rated the human demonstrations as more enjoyable and more helpful with the concepts. Our study's comparison group did not receive an alternative laboratory, so we cannot make these direct comparisons.
In our study, 40% of students mention the time involved as a reason they did not enjoy CyberRat (e.g., “took too long and was very repetitive for something that could have just been explained in class”), with more F2F students (53.6%) commenting negatively on the time involved compared with online students (21.1%). In an online, asynchronous course, there is likely an expectation of more self-paced time completing computer-based activities than in a F2F class, so this finding is not surprising. Students in previous assessments of virtual labs have also expressed some frustration with the labs, including calling the virtual rat “boring” (Hunt & Macaskill, 2017) or using words like “confusion,” “frustration,” and “hate” to describe the virtual rat. However, in comparing virtual to live rats, Elcoro and Trundle (2013) found that a higher percentage (75%) of students indicated that working with the live rat involved “too much” or “a lot” of patience, whereas only 42% classified the virtual rat in these categories. Training a rat (live or virtual) can certainly be a time-consuming process, so any program designed to simulate this experience can also be expected to require time and patience.
Our research design had many strengths. The instructor was the same across all sections, and within each modality, the treatment and control classes were identical, other than the addition of the CyberRat laboratory in the treatment courses. This eliminates the confound of instructor effects and other course differences, increasing confidence that the observed differences were due to the laboratory experience. Additionally, Kennesaw State University has a diverse student population (i.e., only 42.8% White; 38% first-generation students), which increases the generalizability of these results. Finally, our performance measures were course grades and exam question grades in real classes, increasing external validity.
Our study did have a few limitations. Although the course sections were randomly assigned as treatment or control, students were not randomly assigned to sections. Thus, it is possible that some differences in student performance across sections could have been due to individual differences (e.g., GPA, age, motivation). We also had relatively small sample sizes in each condition, and a lower percentage of online students consented to having their data included, but the resultant number of consenting students in each treatment condition was similar. Additionally, our online courses took place during the start of the COVID-19 pandemic in 2020, a semester in which both students and instructors faced unique challenges. However, the classes used in this study were originally scheduled to be online and asynchronous; therefore, the classes experienced minimal interruptions. Our university closed for two days, requiring the deadlines in both sections to be pushed back that week. Also, the instructor was lenient with any extension requests or make-up assignments due to the impact of the pandemic on the students. No course content was altered. The combined course average for all students who completed the course in those two sections in Spring 2020 was 79.31. The same instructor's online course average (8 sections, Fall 2020 – Spring 2024) was 78.08 (SD = 1.67), putting the course average for the Spring 2020 semester in line with students’ performance in a typical semester. Further, any impact of the pandemic likely impacted treatment and control sections equally. Finally, the CyberRat program does not capture all elements of a live animal training session. Training a live animal teaches operant conditioning concepts and other skills not covered in a virtual lab (e.g., animal husbandry). However, CyberRat provides an interactive simulation in which the users’ choices and inputs determine the behavior of the rat, thus allowing users to gain valuable experience by applying operant conditioning principles.
Conclusion
In conclusion, we found that students considered virtual laboratories involving CyberRat to be useful for their learning. This perception was supported by increased course grade and lab-related exam question performance only for online learners. This benefit to online learners is especially important given the increased demand for online courses (Garrett et al., 2023) and the mixed findings from research comparing students in online courses versus F2F courses, with several researchers finding that online students perform worse (e.g., Francis et al., 2019; Helms, 2014; Spencer & Temple, 2021; Xu & Jaggars, 2014), and some finding the opposite (e.g., Means et al., 2009; Soffer & Nachmias, 2018). Virtual laboratories offer students a hands-on, active learning experience that appears to benefit online learners differently. Future research evaluating this differential impact would be valuable in disciplines outside of psychology. It would also be beneficial to determine what specific components in a virtual laboratory are especially useful in improving student learning. In addition, it would be valuable to compare the virtual CyberRat lab with synchronous live demonstrations in both online and F2F modalities. Our findings provide a possible avenue for improving learning outcomes in online courses and can guide educators toward best practices when incorporating laboratories across modalities.
Transparency and Open Science Statement
The raw data and analysis code/syntax used in this study are not openly available for download. The list of questions and coding manuals for the qualitative analyses are not openly available for download. All of these can be obtained from the corresponding author following the completion of a privacy and fair use agreement. No aspects of the study were pre-registered. Some of the materials (that is, the labs) are openly available for download via OSF.
Footnotes
Author Note
All authors made an equal contribution to this project, and we have agreed to list our names in alphabetical order. We confirm that this research received clearance from Kennesaw State University's Institutional Review Board. We do not have any conflicts of interest to disclose.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
