Comparing Undergraduate Student Participation in Online Discussion Boards Hosted on Reddit and Canvas

Anonymity

Reddit users often use pseudonymous “throwaway” accounts (Leavitt, 2015; Stutzman et al., 2013), that don’t contain any personally identifying information, which can promote self-disclosure of opinions (Ma et al., 2016). Surveys suggest that some students support the idea of anonymity in discussion boards (Roberts & Rajah-Kanagasabai, 2013). In their study, Roberts and Rajah-Kanagasabai found that about one third of students preferred to post their viewpoints anonymously, many of which said they would be more likely to post and reply if they were able to do so. Although pseudonyms are available in some LMS, others, like Canvas, do not offer this feature.

Public Interaction

A second distinction is that Reddit facilitates interactions with the public. Interactions with the community may broaden a student’s learning experience by encouraging conversations with people from different backgrounds, similar to community-based learning approaches being adopted in higher education (Ibrahim, 2010). Although Reddit has some demographic biases (Barthel et al., 2018), those biases are different from those of a typical undergraduate student body (Aud et al., 2010) regarding income and education. However, interaction with the public could also be harmful; larger audiences may make students more reluctant to express themselves (Barasch & Berger, 2014).

Design

The last factor that could drive differences in engagement is design. For example, Canvas displays user-photos alongside each comment, includes more space between the username and post-text, and adds an outline around posts. Moreover, Reddit incorporates additional features, for example, the ability to both like and dislike comments and to sort posts by which are “hot” (a combination of votes and date-posted). There is reason to believe that Reddit has created these design elements to maximize engagement: Reddit is exclusively dedicated to providing discussion boards, is valued at three billion dollars, and relies on advertising as a significant source of income.

Based on this rationale, we have designed a study to evaluate whether hosting an undergraduate course’s discussion board on Reddit might increase text output, indicating increased engagement, relative to a traditional LMS. In this study, a non-randomized controlled trial was conducted on two sections of two undergraduate psychology courses. Students were assigned a minimum number of posts and replies, along with a minimum word count, but were strongly encouraged to post beyond that minimum. We hypothesized that students using Reddit would post and/or reply more often, post longer content, and report more subjective value of the discussion board exercise.

Participants

Undergraduate students at a 4-year private University were all taking part in one of two courses: Introduction to Psychobiology and Cognition (IPC) taught by ASN, and Child Development (CD) taught by JM. Overall, students were 63% female, 36% male, and 1% gender variant/non-conforming. Students in IPC were 100% freshman, and in CD, 44% freshman, 55% sophomore, 1% senior. In IPC, students were 47% White, 34% Asian, 6% Black, and 13% Mixed Race (no data for the CD course).

Students were excluded if they (1) did not agree to participate, (2) did not submit any posts or replies, or (3) elected to submit extra credit in lieu of some of their essay assignments (in IPC). Some other outlying data have been excluded as specified throughout the text. In IPC, discussion-board data from 16 and 21 students were included from the course sections using Reddit and Canvas respectively. In CD, 21 and 19 students participated using Reddit and Canvas, respectively—providing a total of 77 participants.

Assignment Instructions

In the IPC course, students were required to submit a minimum of seven posts and 15 replies. In the CD course, students were required to submit eight posts for full credit but could submit an additional 2 as extra credit—this course also required 15 responses. In both courses, posts were expected to be around 300 words, and replies “∼100 (>75)” words. However, the instructors noted in the assignment instructions, and in course, that students were strongly encouraged to go beyond the minimum. Full assignment instructions can be found in the supplementary material.

Discussion Board Survey

Each instructor administered a series of questions at the end of the semester regarding students’ experiences with each discussion board which was completed by 68 students. In the CD course, we discarded some questions as we mistakenly used an older version of the final survey. For the three questions on the value of pseudonymity and public-openness, the responses ranged from approximately neutral to maximally positive or negative. We thought that this scale would not adequately characterize how many students felt and therefore decided not to include the data from CD for these questions.

Data Analyses

The primary endpoint of this study is post number, reply number, and word count. As much of the data were not normally distributed, we used a two-way ANOVA based on ranks, with course (IPC vs. CD), and platform (Canvas vs. Reddit) as independent variables. For those analyses in which we observed an interaction between course and platform, or a main effect of platform, post hoc Wilcoxon Rank-Sum tests were conducted for each course between discussion board platform (with two tests, we used the Bonferroni adjusted p-value threshold of p = .025 [Dunn, 1961]). For effect size, an r statistic was computed from the z score (Pallant, 2010). Any main effect of course (between instructors) was ignored. In the original OSF registration, only one course was going to be used, and therefore, a Student’s t-test or an analogous non-parametric test was planned. However, as another course was included, an ANOVA was the more appropriate preliminary analysis, with post hoc Wilcoxon Rank-Sum. ¹

We conducted a follow-up analysis regarding students' attitudes toward the assignment by asking “How valuable was the discussion board for your learning of the material?” rated on a 1–5 Likert scale. These data were evaluated with ANOVA as noted above.

For the IPC course, we explored the correlation between grades and post number, reply number, and word count for each (using Spearman’s rho and the Bonferroni adjusted p-value of p = .0125 as a cutoff). One student whose exam score was >3 SDs below the mean was excluded from this analysis.

The p-value threshold for interpreting an effect as significant is specified throughout the text—some analyses relied on more stringent thresholds using a Bonferroni correction to control for the family-wise error rate. No a-priori power calculation was conducted as the sample size was determined based on course enrollment. For ANOVA tests, we calculated a post hoc statistical power of .58 assuming a medium effect size (f = 0.25). For Wilcoxon Rank-Sum tests (comparing across platform, within course), we calculated a post hoc statistical power of .42 for IPC course, and .45 for CD course, assuming a medium effect size (d = 0.50, r = .24).

Statistical analyses were carried out in JMP Pro version 15, including the “Calculate Effect Sizes v0.07.jmpaddin,” and ANOVA power calculations in g-power (Faul et al., 2009).

Anonymity

We hypothesized that those students posting in Reddit would feel less inhibited to express their perspectives and in turn be more inclined to post longer comments, replies, and reply more frequently. Yet, the data did not indicate such findings. Rather, students’ posts did not go further than the expected word count and in one of the courses, longer word counts were noted on Canvas. We speculate that since CD included more sophomores who have known each other for an additional year, students were more motivated to post beyond the minimum as a means of maintaining their already established relationships. In the surveys administered by Roberts and Rajah-Kanagasabai (2013), considering attitudes toward posting on student discussion boards, only one third of students said they would prefer to be anonymous, whereas two thirds of students said they would either prefer to be identified or had no preference. Moreover, although the comparison is indirect, outside of education, on question-and-answer sites like Quora and Yahoo answers, anonymously posted questions tend to have fewer answers and comments relative to pseudonymously posted questions (Guo & Caine, 2021), suggesting that for some people, social factors can promote engagement.

Second, although no students reported any anti-social behavior on Reddit, the fear of antisocial behavior may have had a chilling effect on speech. In general, many students are reluctant to share their viewpoints in course (Griffin & Roy, 2019) which may increase as the platform is opened to the public. Symeonides and Childs (2015) found that students who engage in online platforms to exchange viewpoints often experience difficulties in expressing thoughts clearly and compare themselves to others. This is compounded when sharing through platforms open to the public, outside the course confines. When posting on discussion boards on Canvas, students are clearly aware that professors are viewing posts and grading accordingly. There is an inclination to avoid toxic language or posts that may provoke other students, sometimes found on social media platforms such as Reddit (Mohan et al., 2017).

Assignment Format

The format of the assignment could have also affected response output. For this particular study, both instructors gave students a lot of control over what they wrote. However, research suggests that students tend to engage more openly when given concept invention questions rather than personal reflections on in-course discussions (Dalelio, 2013). For example, topics that focus on conflict or controversy, or those in which knowledge is shared, tend to promote more in-depth discussions even when occurring online (Sawyer and Benson, 2004). Instead of providing a reflective format, instructors could have posed questions for students to reflect upon and then discuss, which may have prompted longer posts and responses.

Limitations and Future Research

The study had several limitations. First, the study was grounded on a specific set of assignment instructions: a minimum post and reply count, as well as a minimum (albeit vague) word count. As has been suggested elsewhere (Thomas, 2002), it may be that students did not engage in substantive discussion well beyond the minimum because the expected word count constrained natural conversational patterns. In fact, many students have mentioned that explicit word counts and reply numbers can be disengaging (Buelow, et al., 2018).

Another reason, at least in IPC, is that posts were designed to be fairly self-contained packages, in which students came to a tentative conclusion. It may be that if students were encouraged to conclude with a question, or use some other format, again, more genuine dialogue would emerge (Champion & Gunnlaugson, 2018). However, data suggested that even when students were encouraged to end with a question, as in CD, students did not engage further than was expected.

Finally, the term “student engagement” was operationalized as post and reply number and word count. Measuring student engagement in this way is problematic because length does not necessarily imply quality or depth of thought. Future studies may more comprehensively evaluate post quality, perhaps by recruiting professors from other courses to complete blinded evaluations of each post’s quality.

Should Teachers use Reddit?

Our results were mixed—one course observed longer posts in Reddit, whereas another course observed both shorter posts and replies in Reddit. However, the use of Reddit may carry two additional costs, including (1) inconvenience and (2) distraction. On the whole, the use of Reddit as a discussion board platform may not promote student learning or engagement more than Canvas, at least given the particular assignment instructions used here. Again, we propose further research into the pros and cons of using a traditional online platform, such as Canvas, versus a platform which allows for anonymity and public-interaction. The end goal is to investigate which type of posting is more motivating and encourages further engagement, ultimately influencing deeper learning and academic success.