Abstract
Background:
Student anxiety about statistics may lead to poorer learning outcomes.
Objective:
The purpose of this study was to evaluate an exercise designed to teach students in an introductory statistics class the principles of bivariate regression and to emphasize how statistical tools used by psychologists are also implemented in other fields.
Method:
Students used a published model on the relationship between tooth size and the length of great white sharks to estimate the length of extinct sharks and to explore factors that could affect the accuracy or validity of regression analyses. Data from an anonymous self-report scale were used to assess the activity.
Results:
More than 95% of respondents agreed or strongly agreed that the activity was engaging, approximately 95% of students agreed or strongly agreed that the activity helped them learn about factors that can lead to problems with bivariate correlation/regression, and approximately 91% of respondents correctly answered a question designed to assess basic content acquisition.
Conclusion:
Feedback data suggest that the exercise was successful in achieving its content and process learning goals.
Teaching Implications:
Implementation of similar exercises may improve student engagement and outcomes in psychology statistics courses.
Pedagogical research has consistently demonstrated that active, participatory classroom learning is superior to traditional, lecture-based teaching (Freeman et al., 2014), and is important for increasing equity in undergraduate science education by improving learning outcomes in students from underrepresented and/or marginalized backgrounds (Theobald et al., 2020). In particular, active learning helps students to develop the skills needed to implement a scientific approach in the real world (e.g., searching the scientific literature, and interpreting data), and is especially effective in teaching the process of science, as opposed to the content.
An introductory statistics course is a critical, required component in the vast majority of undergraduate psychology programs (Norcross et al., 2016). However, many students find the course to be daunting, and “statistics anxiety” is common (Chew & Dillon, 2014; Onwuegbuzie & Wilson, 2003). Although there are questions in the literature about the specificity of the construct of statistics anxiety and how to parse it from related variables (Onwuegbuzie & Wilson, 2003), it seems clear that some students experience negative responses to statistics courses (Onwuegbuzie et al., 1997) and these responses are associated with poorer performance (Chew & Dillon, 2014; Macher et al., 2012). Authors of a comprehensive literature review (Baloğlu & Zelhart, 2003) suggested that two strategies that could be effective in ameliorating these effects are to connect statistical principles with real-world problems and to use engaging and/or humorous examples and materials. Other researchers suggest that focusing on the assumptions of statistical tests and allowing students to experience their practical utility could be important in improving outcomes for all students, but may be especially helpful for individuals who are anxious about the subject (Chew & Dillon, 2014).
I designed the current activity to familiarize students with the principles of bivariate regression by presenting them with a real-world application of this technique. The subject of the exercise was deliberately outside the scope of psychology, both because of the importance of interdisciplinary learning for psychology undergraduates (Goodwin-Smith et al., 2013), and to capitalize on a topic that might be particularly novel and engaging for many students (Stucky, 1996). Specifically, the activity centered on a published bivariate regression model examining the relationship between tooth size and body length of extant great white sharks, which might also be useful in estimating the length of the extinct shark megalodon (Shimada, 2002). The megalodon is the largest known fossil shark, though importantly for this exercise, estimates of its maximal length in paleontological literature vary considerably (Shimada, 2002). Many undergraduate students are familiar with this charismatic fossil species from popular science depictions or fiction, such as the recent film The Meg (Turteltaub, 2018).
Students worked in small discussion groups to study the concept of regression through application, first by modeling the length of smaller extinct sharks using actual fossilized teeth from different species, and then repeating the process for a large megalodon using a tooth cast. The goal of the activity was to help students understand how statistical and methodological confounds (e.g., unreliable measurement) affect the reliability of bivariate regression and correlation. As such, students brainstormed and tested ideas about how these variables could affect the validity of their results and considered how they could lead to errors in real-world settings. I assessed the effectiveness of the exercise using an anonymous feedback questionnaire that measured both students’ subjective experiences and probed for content acquisition.
Method
Sample and Setting
I utilize the activity in an undergraduate introductory statistics course that typically enrolls 30 students. The course is required for the psychology major at a liberal arts college, but also attracts non-majors (the course fulfills a general education requirement) and individuals interested in pursuing careers in the health professions. The activity takes approximately 1 hr and 15 min, and follows a lecture on the basics of the bivariate linear model and an accompanying, required reading from the course textbook.
Content Learning Goals
I developed the activity to enhance students’ abilities to: A) use a bivariate regression equation to calculate a criterion variable (shark length) from a predictor variable (tooth size), B) understand how specific variables (unreliable measurement, restriction of range, outliers, and small sample sizes) can affect regression results, and C) consider how issues in experimental design might affect the validity of regression models implemented in real-world situations.
Process Learning Goals
More broadly, my goal for the activity was to improve students’ abilities to: A) work collaboratively to evaluate research and statistical design, and B) understand how research tools often used by psychologists are implemented in other scientific disciplines.
Design and Implementation
See a publically available Open Science Framework (OSF) project for detailed classroom implementation notes and examples of the PowerPoint slides that are used in this activity (Watson, 2021). The instructor first reviewed the basic linear regression model (Y = a + b[x]) and the standardized regression coefficient (β). The instructor then explained that students would consider a real-world example of what they have learned. Students each received a fossil shark tooth (purchased inexpensively online; total cost $10–20 for a class of 30) that they were allowed to keep at the end of the exercise, and worked in groups to identify their species (e.g., hammerhead, sand tiger) using a guide. After soliciting tentative identifications from students, the instructor presented a cast of a megalodon tooth (which is dramatically larger than that of other species) and an image depicting widely-varying estimates of the megalodon’s body length. Students then briefly discussed potential reasons for these discrepancies, and why the issue likely will not be resolved conclusively by finding a complete example of a megalodon (because sharks are cartilaginous and do not readily fossilize).
The instructor then introduced and explained Shimada’s (2002) regression equation, distributed printed rulers (available for free online), and asked students to use them to model the length of their shark. Students then shared their results with their group members (typical lengths were between 1–2 m) and briefly discussed any issues they encountered. The entire class then modeled the length of the megalodon using a tooth measurement recorded by the instructor (arriving at an estimate of 12.69 m). Students then evaluated how different methodological issues might affect the accuracy and reliability of their results. Specifically, they explored:
Unreliable measurement
Students switched shark teeth with a groupmate. Without looking at the other student’s initial data, they repeated the modeling process. Students shared results and the class brainstormed reasons why partners’ results generally did not match (e.g., difficulties in using the printed rulers, errors in how they measured the tooth). After defining unreliable measurement, the instructor demonstrated its effects first on simulated data sets and then using two different measurements of the megalodon tooth.
Restriction in range
Students learned the range of sharks’ lengths used to calculate the Shimada model (163–594 cm), briefly discussed why the author might have chosen this sample, and then conducted internet searches to determine that the range is indeed representative of great white shark size. They then discussed ideas about what might have happened had the author not had access to such a wide range of subjects. The instructor then defined restriction in range and demonstrated its effects on simulated data sets on the relationship between shark length and “scariness” (where the restricted data set only included small—and hence not very scary—sharks).
Outliers
Students discussed what might have happened had one shark in Shimada’s sample been much larger (or smaller) than the others. The instructor then defined outliers, and demonstrated their effects on simulated data by including or not including a lone “megalodon” in a data set.
Small sample size
Students discussed the sample size (N = 12) used in forming Shimada’s model and the overall benefits of larger sample sizes. The instructor demonstrated the effects of outliers in smaller and larger samples using simulated data. Students briefly discussed possible tensions that arise between the desire for large samples and practical concerns, such as availability of shark samples (or human research participants) and expense.
Other issues
Finally, students considered how broader issues and assumptions might affect their regression results. The instructor scaffolded the class discussion to ensure that students reflected on how researchers would have to understand and replicate the specific methodological details (e.g., Shimada’s model was based on teeth from a particular location in the mouth, while students’ shark teeth were more varied) and consider the generalizability of a sample (e.g., Shimada’s model was based on extant great white sharks, while students were modeling other, extinct species).
Assessment Questionnaire
Following the activity, students had the option to provide voluntary, anonymous, quantitative feedback to assess how much they enjoyed the exercise and their understanding of the material covered. Students provided responses on a Likert scale and one multiple choice question via the internet using SurveyMonkey. Students completed the survey
Seven statements assessed students’ experiences, including enjoyment of the activity and perceived educational value. Potential responses were strongly disagree (1), disagree (2), neither agree nor disagree (3), agree (4), and strongly agree (5). A mean rating of 4 (agree) was considered to indicate a positive result. One additional question used a multiple-choice format to probe basic content acquisition (“Which of the following is a problem for regression and correlation? A] Outliers B] Unreliable Measurement C] Restriction in Range D] Small Sample Sizes E] All of the above”). Descriptive statistics were calculated using JASP 0.12.2.
Results
The complete data set is publically available in an OSF project (Watson, 2021). Twenty-three out of 32 students (71.88%) completed the anonymous feedback survey, though one student declined to answer some questions. Student impressions were favorable (Table 1). Importantly, more than 95% of respondents agreed or strongly agreed that the activity was engaging, that the activity should be presented again in subsequent years, and that the development of other similar activities would be helpful for future students in the course. Approximately 91% of respondents reported that the exercise was more engaging than a traditional lecture.
Means and Standard Deviations for Response to Items on the Feedback Questionnaire.
Note. Responses were given on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). A score of 4 corresponded with agree. Differences in N reflect that one respondent skipped certain questions.
Approximately 95% of students agreed or strongly agreed that the activity helped them learn about factors that can lead to problems with bivariate correlation and regression and 86% reported that it helped them understand how regression modeling can be used to answer real-world questions. Twenty out of 22 respondents (90.91%) correctly answered the multiple-choice question designed to assess basic content acquisition.
The lowest score was for the statement that the activity “increased my interest/curiosity in how regression and correlation are used in different scientific disciplines.” Approximately 59% of students agreed or strongly agreed with this statement, while 36% were neutral (rated 3). Only one respondent disagreed (rated 2).
Discussion
A substantial majority of students found the activity enjoyable and engaging and agreed that it should be repeated in future years (the modal response for these questions was strongly agree). Similarly, a large majority of students reported that the activity achieved its primary goal of teaching about factors that can adversely affect bivariate correlation and regression (again, the modal response was strongly agree). Mirroring these data, more than 90% of students correctly answered the multiple-choice question on content. As such, the activity supported both its primary content and process learning goals. Students learned, but were also engaged in the process of learning. Given the potential importance of engagement in student success in statistics classes (Baloğlu & Zelhart, 2003), as well as the importance of focusing on assumptions and practical, real-world utility of analyses to reduce students’ statistics-related anxiety (Chew & Dillon, 2014), the data suggest that this activity—and others like it—could be useful additions to undergraduate statistics classes. See Watson (2021) for the complete data.
While a smaller percentage of respondents reported that the activity increased their interest in how regression and correlation are used in different scientific disciplines, given the exercise’s relatively small demands in class time, even this modest increase could be considered useful. Perhaps more importantly, a substantial majority of students reported that the activity showed them how modeling can be used to answer real-world questions, even outside of psychology. Students also reported that it would be helpful if similar activities were designed and implemented in future iterations of the course. Given the importance of training interdisciplinary thinking in the psychology major (Goodwin-Smith et al., 2013), instructors should consider the utility of mining other disciplines for source materials to create interactive, engaging active-learning opportunities for their statistics classes.
The narrative for the current activity was chosen because paleontology is frequently represented in popular science media and is familiar and engaging to many students. As another example, Grant and colleagues (2016) independently developed a similar, but more paleontology-focused exercise in which high school students estimated the size of megalodon using Shimada’s model and determination of tooth position. They found that the activity was considered interesting and informative, and showed that 3-D printed fossils can be used to trigger student interest. These two independently developed exercises show how one regression model can be tailored for activities with disparate learning goals in a variety of student populations, potentially in classes housed in disparate departments.
While the data suggest that this activity was successful, it is important to note several limitations to this preliminary report. First, given that this was a standard educational opportunity, I was not able to include a control group. It is therefore not currently possible to assess differences between students who did or did not complete the exercise, and as such, it is not possible to determine if the large percentage of students who correctly answered the multiple-choice content acquisition question directly reflects the efficacy of the exercise (learning) or other factors. Second, the exemption I applied for (and was granted by our IRB) only covered the anonymous, voluntary feedback questionnaire. It is therefore not possible to present more detailed assessment data such as exam scores or mention of the exercise on teaching evaluations. Finally, the current data represent only one relatively small group of students who self-selected to provide anonymous feedback. As such, it is perhaps best to consider this report a proof-of-concept. A replication in a larger, controlled study would be helpful. In the meantime, instructors who choose to implement or modify this activity should carefully monitor and assess learning outcomes to ensure it achieves desired pedagogical goals.
Conclusion
Preliminary data suggest that students find this active-learning exercise engaging and enjoyable, and that it could be a useful tool in helping students understand bivariate regression analyses and their application across scientific disciplines.
Footnotes
Acknowledgments
The author would like to thank Dr. Zenab Amin for editorial assistance, and two anonymous reviewers for their insightful comments and suggestions on earlier drafts of this manuscript.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
