Abstract
It is increasingly common for journalists to use reader comments from social media as a quote or source of evidence for news. Past work on the effects of user-generated content has revealed that comments can have a host of negative effects on readers’ psychological processing of news, particularly when reader comments are hostile or uncivil. What happens, however, when online comments are moderated by a journalist? Does comment gatekeeping influence the effects of comments? Informed by the Modality-Agency-Interactivity-Navigability model and warranting theory, an experiment (N = 365) was conducted to answer these questions using a 2 (comment gatekeeping: low vs. high) × 2 (comment valence: positive vs. negative) × 2 (news involvement: low vs. high) design. Results revealed that journalistic gatekeeping of comments decreases article credibility and increases bias through the indirect pathway of source trustworthiness, regardless of news type or comment valence.
The affordances of digital media have led to a proliferation of information regarding the opinion of media consumers (Sundar, 2008b). This has led to dramatic changes in the ways that media are encountered and processed, as many types of popular content are now accompanied by a variety of cues related to audience opinion that co-occur alongside media (Metzger, 2009). These changes in the media landscape have significant consequences for the study of media effects, as metrics like the number of times a news story is “liked” or the tone of comments that appears under a news story can affect individuals’ subsequent evaluations of content (Anderson, Brossard, Scheufele, Xenos, & Ladwig, 2014; G. M. Chen & Ng, 2016; Houston, Hansen, & Nisbett, 2011). This is particularly apparent in the domain of online news, as news outlets have increasingly been asked to weight the benefits of providing online forums to readers with the possible negative effects that online comment sections can evoke, particularly when the sentiment of the discussion between readers becomes hostile (Cameron & Geidner, 2014; Lee & Jang, 2010; Winter, Brückner, & Krämer, 2015).
Accompanying the diffusion of audience feedback metrics alongside news is the increasing prevalence of news stories that integrate user-generated content alongside traditional news content. Comments and “tweets” are increasingly being used by journalists in the course of reporting, whether it be user-generated content filtered from social media or citizen video submitted by readers themselves. Common examples of this trend can also be found in realms such as celebrity news or politics, where a single social media posting by a prominent public or political figure can serve as an impetus for an entire story. Mainstream news outlets like the Washington Post even devote weekly columns to the “best” comments or Tweets of the week, further demonstrating the normalization of social media content as a journalistic source.
While the effects of comment sections have been studied by a variety of scholars in past work (Chung, Munno, & Moritz, 2015; Kim, 2015; Lee, 2012; Stavrositu & Kim, 2015), less work has considered how the effects of comments possibly vary when the comments in question have been selected by a journalist. At a broader level, it is also unclear how audiences perceive news stories that prominently feature social media content, which raises the possibility that such practices might alter the credibility of either the journalist or news article in question. The following research question thus arises: Does journalistic gatekeeping of comments affect perceptions of news credibility? If so, what are the theoretical mechanisms responsible for such effects? The following sections review relevant research on the effects of comments and news perceptions, followed by an outline of the study’s methodology, a report of results, and a discussion of theoretical and practical implications.
Literature Review
A growing body of research has examined the effects of online comments on individuals’ attitudinal and behavioral responses to media. In many cases, it appears that individuals tend to adjust their attitudes toward media to correspond with the opinion expressed by other people. For example, several studies have found that comments can affect the perceived polarity of public opinion (Anderson et al., 2014), the quality of popular programming (Cameron & Geidner, 2014; Waddell & Sundar, 2017), the success of public service announcements (Shi, Messaris, & Cappella, 2014), or the assumed effect of media on others (G. M. Chen & Ng, 2016; Veldhuis, Konijn, & Seidell, 2014) among a variety of other outcomes (Hsueh, Yogeeswaran, & Malinen, 2015; Hwang, Kim, & Huh, 2014; Li, Feng, Chen, & Bell, 2015; Rim & Song, 2016; Waddell, 2017; Waddell & Bailey, forthcoming; Waddell & Bailey, 2017; Walther, DeAndrea, Kim, & Anthony, 2010). What theoretical mechanisms are potentially responsible for such effects? One theoretical framework that offers a possible explanation for the effects of comments is the Modality-Agency-Interactivity-Navigability (MAIN) model. As Sundar (2008a, 2008b) describes, the affordances of digital media provide a range of cues that influence consumers’ evaluations of content. The MAIN model (Sundar, 2008a) theorizes that the effects of popularity cues like comments and “shares” affect individuals’ psychological state through the activation of heuristics or mental shortcuts that bias individuals’ subsequent processing of content. In the context of online comments, the MAIN model theorizes that online comments influence individuals’ evaluation of news credibility through triggering the bandwagon heuristic or the rule of thumb that if others like this content, then I should too (Sundar, Jia, Waddell, & Huang, 2015). Initial studies have found support for this theoretical pathway, such as Xu (2013) who found that the number of times an article was recommended heightened sharing intentions through the indirect pathway of bandwagon perceptions.
Given that comments tend to elicit group consistent behavior, it seems possible that the effects of comments observed in past work may be partially explained by variations in perceived bandwagon support. However, one potential moderating variable that has received relatively less attention in past work is whether the effects that comment elicit vary based on their selection or curation by an outside source. In other words, do the effects of comments vary in their strength when comments are selected by a third party, such as when journalists publish social media messages within traditional news content? The direction of effects expected to be elicited by journalistic selection of comments varies depending on the theoretical framework invoked to explain possible effects. On the one hand, the MAIN model (Sundar, 2008a; Sundar et al., 2015) theorizes that cues related to the prestige or expertise of a source can trigger the “authority heuristic,” leading to more positive assessments of content produced by the source in question. In support of this theoretical maxim, past work has found that content displayed or recommended by agents or televisions categorized as “experts” is more likely to be positively evaluated than when the same content is recommended by an agent or television described as a generalist (Koh & Sundar, 2010; Nass, Reeves, & Leshner, 1996). Given this logic, it seems possible that journalistic gatekeeping of comments may elicit a similar effect, where selective filtering of comments through placement within news may lead to increased persuasive effects elicited by comment exposure. Put simply, individuals may be more likely to go along with the comments expressed by others when the comments have been endorsed by a presumed expert, such as a journalist. Following this logic, the following hypotheses are proposed:
While the MAIN model might predict that journalistic filtering would heighten the effects that comments elicit, alternative theoretical frameworks specify that the effects of audience feedback are likely to be contingent on their perceived authenticity or ease of modifiability. According to warranting theory (Walther & Jang, 2012; Walther & Parks, 2002), the flexibility in presentation that the affordances of online media provide has been accompanied by increased concerns regarding the authenticity of online information. As a result, online information is likely to vary in its influence on individuals’ perceptions depending on the extent to which the cue has “warranting value.” The warranting value of a cue is often operationalized based on the ability of the original source to modify or selectively present the information in question (DeAndrea, 2014). For example, several studies in the domain of social media have found that comments provided by one’s “friends” on Facebook are more influential to source evaluations than information provided by the profile owner, which warranting theory ascribes to the higher warranting value of information created by others relative to the self (Walther, Van Der Heide, Hamel, & Shulman, 2009; Walther, Van Der Heide, Kim, Westerman, & Tong, 2008). Put simply, a central tenet of warranting theory is that the more control one potentially has over a source of information, the less likely it is that such cues will influence individuals’ subsequent perceptions of the source.
Applied to the context of comment selection, it seems possible that filtering of comments by journalists may lead to decreased perceptions of article credibility, regardless of whether the comment in question is positive or negative. In other words, warranting theory would predict that comments selected by journalists would be perceived as lower in warranting value than comments that were not curated by an outside source. Furthermore, one consequence of these differences in warranting value may be decreased perceptions of source credibility, which in turn might attenuate perceptions of article credibility. The opportunity to test the assumptions of theory thus arises: If journalistic gatekeeping increases the effect of comments on article credibility, then the tenets of the MAIN model will be supported. Alternatively, if journalistic filtering of comments decreases article credibility, then the assumptions of warranting theory will be supported. More formally, the following hypothesis is proposed:
Comment Effects and Issue Involvement
To further probe the possible theoretical mechanism responsible for the effects of comments, the current study also tests whether the effects of comments are moderated by variations in issue involvement. Theoretical frameworks like the MAIN model (Sundar, 2008a) are grounded in the assumptions of dual process models of persuasion (S. Chen & Chaiken, 1999; Petty & Cacioppo, 1986) that outline a variety of conditions where heuristics activated by peripheral cues are more or less likely to affect individuals’ attitudes. One such moderator is an individual’s level of involvement with a topic, which past work has often manipulated by varying the perceived relevance or importance of the stimulus (Petty, Priester, & Brinol, 2002). For example, a classic study (Petty, Cacioppo, & Schumann, 1983) found that peripheral cues (such as celebrity endorsement) increased purchase intentions only when the context of the decision decreased the perceived relevance of the product decision. By comparison, participants induced to perceive the product as high involvement were unaffected by heuristic cues, instead processing the message based on the merits of the message itself.
Following this logic, if comments operate through the activation of heuristics, then it would be expected that comment effects on credibility would be moderated by the perceived relevance or importance of the news story, with effects most likely to occur for low involvement rather than high-involvement topics. More broadly, the field of media effects has recently called for increased attempts to generalize study findings beyond specific stimuli to broader classes of media overall to improve the external validity of findings (O’Keefe, 2015). Given that past studies on comments have often demonstrated effects in specific domains, a test of comment effects across low- and high-involvement news stories offers an opportunity to delineate possible boundary conditions for comment effects. Following the tenets of dual process models of persuasion, the research question is proposed:
Method
An online experiment was conducted to test a 2 (comment valence: positive vs. negative) × 2 (comment gatekeeping: low vs. high) × 2 (issue involvement: low vs. high) between-subjects factorial design. Participants (N = 365) were asked to read a news article accompanied by positive or negative comments that appeared either in the comments section or were embedded within the article itself. Following exposure to the news article, participants completed a postexposure questionnaire that measured their bandwagon perceptions, the perceived credibility of the article, and the perceived credibility of the article’s author.
Participants
Three hundred and sixty-five participants were recruited to participate in the study using the crowdsourcing website, Amazon Mechanical Turk (Buhrmester, Kwang, & Gosling, 2011). An a priori power analysis determined that a sample size of at least 327 participants would be needed given f = .20, alpha = .05, and power = .95. Participants were compensated 50 cents for participating in the study, which was approved by the institutional review board at the university of the primary investigator. The sample was 55% female (n = 201) with a mean age of 38.83 (standard deviation [SD] = 12.60). When asked to self-report their race, 78% (n = 285) reported “white/Caucasian,” 9% (n = 34) reported “black/African American,” 5% (n = 19) reported “Hispanic/Latino/Latina,” 5% (n = 19) reported Asian/Asian American,” and 3% (n = 8) reported “biracial” or “Other.” All participants read a written description of the research and consented to participate in the study before beginning the pre-exposure questionnaire.
Stimuli
Participants were asked to read a news article on a low- or high-involvement current event. Each news article was attributed to USA Today, a relatively nonpartisan news outlet. Furthermore, the author attributed to the article was given a gender-neutral name to avoid introducing sex-related confounds.
In order to identify current events that were perceived by participants as low and high involvement, a pretest (N = 54) was conducted where participants read a series of news headlines with accompanying summaries. After each headline, participants were asked to indicate the extent to which the news topic was “relevant” and “important,” two classic indicators of perceived issue involvement. Repeated measure analyses of variance (ANOVAs) were conducted to compare participants’ evaluations of each news story, with findings revealing that each of the four news topics expected to be low involvement (e.g., “Megyn Kelly is leaving Fox News”; “Queen is feeling better, says Princess Anne”; “Minnesota fires football coach after player suspension, boycott controversy”; and “Famed Hollywood sign in Los Angeles vandalized”) scored lower on perceived relevance and importance than the four news topics expected to be high involvement (e.g., “Congress reverses ethics move after outcry”; “More cases of Russian cyberattacks come to light”; “Turkey nightclub attack: Arrests in hunt for gunman”; and “5 dead after Tornado, Severe Weather Sweeps across South”). The two stories that scored highest on perceived relevance and importance were selected for the “high-involvement” condition (“Russia” and “Congress”), while the two stories that scored the lowest on each measure were selected for the “low-involvement” condition (“Sports” and “Queen”).
Experimental Conditions
Comment valence
Participants viewed three comments that were manipulated to appear as though they were supposedly taken from Twitter. The comments expressed readers’ attitudes toward the news article, with the content of the comments remaining constant aside from the presence of adjectives intended to manipulate the perceived sentiment of the comments. For example, the positive comment condition included remarks such as “@USATODAY so important, this belongs in the news” and “@USATODAY great media coverage here,” while remarks in the negative comment included statements such as “@USATODAY who cares? this doesn’t belong in the news” or “@USATODAY typical lazy media coverage.” Participants in the control condition were exposed to a news story that was not accompanied by any comments to serve as a baseline for the positive and negative comment conditions.
Comment gatekeeping
In addition to varying the perceived sentiment of the comments, level of comment gatekeeping was manipulated by varying the location of the comments. In the “low gatekeeping” condition, participants were presented with a mock comment section (modeled after the comment section seen on USA Today) after reading the news article which featured either positive or negative comments. In the “high gatekeeping” condition, the aforementioned comments were embedded within the news article itself following the second paragraph of the article, preceded by the line “as news of this broke, members of the public took to social media to react.” The content and appearance of the comments remained constant regardless of whether the comments appeared within or after the news article.
News involvement
As mentioned above, a pretest was conducted to identify news stories that varied in their degree of involvement. Participants were thus assigned to one of the four possible news stories, which was subsequently collapsed into either a low- or high-involvement variable for subsequent analyses.
Manipulation Checks
Comment sentiment
In order to verify the efficacy of the comment valence manipulation, a single semantic differential item on a 7-point scale asked participants to identify whether the comments were “mostly positive” or “mostly negative” (M = 3.47, SD = 2.64).
Comment curation
In order to verify the efficacy of the comment gatekeeping manipulation, a single Likert-type item on a 7-point scale asked participants to identify the extent to which the comments they saw during the study were “curated” (M = 3.33, SD = 1.77).
Mediating Variables
Bandwagon perceptions
Five Likert-type items adapted from prior research (Xu, 2013) were used to measure perceived bandwagon support for the news article. Sample items included, “how likely are other people to enjoy reading the news article?” “How likely is it that other people would think the news article is valuable?” and “how likely is it that other people would rate the news article positively?” An index was formed by combining the 5 items, which was reliable (Cronbach’s α = .94; M = 4.04, SD = 1.67).
Source credibility
Two subdimensions of source credibility were adapted from prior research (McCroskey & Teven, 1999) and measured using semantic differential items on a 7-point scale: (1) source competence and (2) source trustworthiness. Six items were used to measure source competence including items such as the source was “unintelligent/intelligent,” “untrained/trained,” and “inexpert/expert.” Six items were also used to measure the source trustworthiness dimension of source credibility including items such as the source was “dishonest/honest,” “untrustworthy/trustworthy,” and “unethical/ethical.” An index was formed by combining the 6 items for each of the respective indices, both of which were reliable (source competence: Cronbach’s α = .94; M = 5.30, SD = 1.19; source trustworthiness: Cronbach’s α = .96; M = 5.19, SD = 1.22).
Dependent Variable
Article credibility
Two dimensions of article credibility were measured using validated indices from past work (Appelman & Sundar, 2015): (1) reflective indicators of credibility (factors that measure credibility directly) and (2) formative indicators of credibility (factors that contribute to perceived credibility). In terms of reflective indicators, an index of message credibility was adopted from prior research including the extent to which the article was “accurate,” “authentic,” and “believable.” These 3 items were combined to form an index, which was reliable (Cronbach’s α = .90; M = 5.45, SD = 1.28). As for formative indicators, participants were asked to indicate the extent to which the article was “newsworthy” (M = 4.82, SD = 1.97) and “biased” (M = 3.20, SD = 1.84).
Procedure
Participants began the study by reading a written description of the study, then completing a pre-exposure questionnaire that measured their demographic information. Afterward, participants were exposed to a news article which featured either positive or negative comments either embedded within the news story or appearing after the news story, which dealt with either a low- or high-involvement topic, all of which was determined by random assignment. Following the news story, participants completed a postexposure questionnaire that measured the outcome variables of interest including bandwagon perceptions, source credibility, and article credibility.
Results
Manipulation Checks
Comment sentiment
To determine the efficacy of the comment valence manipulation, a one-tailed t test with comment valence as the independent variable and perceived comment sentiment as the dependent variable was conducted. The main effect of comment valence was significant, t(265) = 25.96, p < .0001, d = 3.13, as participants in the positive comment condition (M = 5.84, standard deviation = 1.81) were more likely to perceive that the comments were “mostly positive” than participants in the negative comment condition (M = 1.37, standard deviation = .91). In sum, the manipulation was successful.
Comment curation
To determine the efficacy of the comment gatekeeping manipulation, a one-tailed t test with comment gatekeeping as the independent variable and perceived comment curation as the dependent variable was conducted. The main effect of comment gatekeeping was significant, t(264) = 2.31, p = .01, d = .29, as comments in the high gatekeeping condition were perceived as more curated (M = 3.60, SD = 1.82) than comments in the low gatekeeping condition (M = 3.10, SD = 1.68). In sum, the manipulation was successful.
Main Analyses
Hypothesis 1 predicted that negative comments (relative to positive comments) would decrease article credibility through the mediating pathway of bandwagon perceptions, while Hypothesis 3 predicted that comment gatekeeping would decrease article credibility through the mediating pathway of source credibility. Before testing mediation hypotheses directly, a series of three-way ANOVAs were conducted for each dependent variable of interest to determine whether total effects were present.
Beginning first with article credibility, a main effect of comment gatekeeping was observed, F(1, 285) = 7.06, p = .009,
Mediation Analyses
To test the mechanism through which comment valence and comment gatekeeping affect article credibility, a series of parallel mediation models were conducted using Model 4 of the PROCESS macro (Hayes, 2013) with 5,000 bootstrapped samples and 95% bias-adjusted confidence intervals (CI). Rather than probing for indirect effects in the absence of a total effect, analyses were only run for variables where a significant F value was observed in the aforementioned analyses. Beginning first with the effects of comment valence, negative comments (relative to positive comments) had an indirect effect on newsworthiness (−1.01, 95% CI [−1.32, −0.73]) through the mediating pathways of bandwagon perceptions. Specifically, negative comments decreased the perceptions of bandwagon support (β = −1.43, p < .00001), which was subsequently associated with lower levels of perceived newsworthiness (β = 0.71, p < .00001). Negative comments (relative to positive comments) also had an indirect effect on newsworthiness (−.12, 95% CI [−0.29, −0.02]) through the mediating pathway of source credibility, as negative comments decreased source credibility (β = −0.32, p = .02), which was positively correlated with newsworthiness (β = 0.39, p < .00001). No indirect effects via source trustworthiness were observed. Hypothesis 1 was partially supported.
As for the effects of comment gatekeeping, a similar series of parallel mediation models revealed that high comment gatekeeping (relative to low comment gatekeeping) had an indirect effect on article credibility (−.13, 95% CI [−0.30, −0.02]) and perceived bias (.14, 95% CI [0.02, 0.34]) through the mediating pathway of source trustworthiness. Specifically, high gatekeeping decreased perceptions of source trustworthiness (β = −0.32, p = .02), which was subsequently related to article credibility (β = −0.39, p < .00001) and bias (β = −0.45, p < .00001). No other indirect effects via either bandwagon perceptions or source credibility were observed. Hypothesis 3 was partially supported.
Summary of Results
In sum, the results of the present study reveal that comment gatekeeping of comments decreases perceptions of article credibility and increases perceived bias. Furthermore, mediation analyses reveal that the inhibitory effect of comment gatekeeping on article perceptions operates through lower perceptions of source trustworthiness. By comparison, comment valence was inconsequential to reflective indicators but did influence perceptions of article newsworthiness through the mediating pathway of bandwagon support. Finally, no evidence of moderation by news type was observed, as comment effects were relatively consistent between low- and high-involvement news stories.
Discussion
Comments from readers are no longer limited to social media feeds or comment sections at the end of news stories, as more and more journalists now integrate comments within the article itself. Past research has shown that the sentiment of comments can influence news readers’ perceptions of current events and attitudes toward news in general (e.g., Anderson et al., 2014; G. M. Chen & Ng, 2016; Houston et al., 2011). However, these studies have largely focused on comments that appear in comment sections rather than comments that were selectively filtered by an outside source. The theoretical and practical question thus arises: Does journalistic selection of comments influence the effects that comments elicit on perceptions of news and source credibility? If so, what are the possible theoretical mechanisms through which such effects occur?
The results of the current study reveal that journalistic gatekeeping of comments decreases perceptions of article credibility and increases perceptions of news bias. This inhibitory effect of comments gatekeeping occurred regardless of whether the comments included in the news story were positive or negative in valence. Such findings appear to lend support to warranting theory (DeAndrea, 2014; Walther & Parks, 2002) as an explanatory framework for comment effects. As the theory describes, the warranting value of information is contingent on the extent to which the information in question is possibly modifiable by an outside source. Building on this logic, the present work reveals that journalistic gatekeeping of comments appears to lower the warranting value of comments as reflected by lower perceptions of source and article credibility. Differences in perceived warranting value thus appear to have significant consequences for the way that news is perceived and evaluated, a novel extension of warranting theory to a previously understudied domain.
Mediation analyses further reveal that the effects of comment gatekeeping on article credibility and bias appear to be mediated by variations in source trustworthiness. Specifically, journalistic gatekeeping of comments (relative to comments that appear without moderation) decreased the perceived trustworthiness of the journalist that produced the news story, with downstream consequences for readers’ perceptions of article credibility and bias. Given these results, readers appear to perceive that journalists may be biased in their selection of online comments that are included in news, thus leading to more negative attitudes toward the journalist and the news product they created. These findings again lend support to warranting theory, while also failing to provide support to the authority heuristic of the MAIN model (Sundar, 2008a). In terms of the latter, several studies have found that content created or endorsed by “expert” sources tends to evaluated more positively than the same content attributed to a generalist source (Koh & Sundar, 2010; Nass et al., 1996). Such predictions, however, do not appear to extend to online comments, as selective filtering of comments by journalists does not appear to imbue comments with increased authority, nor are moderated comments more likely to elicit persuasive effects. In sum, this pattern of results thus provides stronger support to warranting theory than the MAIN model in the context of news and online comment gatekeeping.
The results of the present study also provide nuance to existing research on the effects of comments on article credibility, revealing that negative comments (relative to positive comments) affect both the perceived newsworthiness of online news and the competence of the journalist that produced the news. Furthermore, mediation analyses revealed that the effect of negative comments on newsworthiness was mediated by bandwagon perceptions, such that news accompanied by negative comments was less likely to be perceived as supported by others, which subsequently decreased the assumed newsworthiness of the article. Taken as a whole, these findings provide the theoretical contribution of revealing the importance of distinguishing between reflective and formative indicators of credibility when investigating the effects of comments. Specifically, our results appear to suggest that comments are more influential to formative elements that contribute to credibility (such as newsworthiness) rather than factors that are reflective of credibility. Along similar lines, comment valence also appears to have a targeted influence on specific subcomponents of source credibility, affecting the presumed competence of the journalist but not their perceived trustworthiness. Our findings thus contribute to a larger body of research that has revealed the negative effects that comments elicit on readers’ news perceptions (Kim, 2015; Lee, 2012), while also providing theoretical nuance regarding the mechanisms through which such effects occur as well as the specific subcomponents of source and message credibility more likely to be affected by online comments.
Finally, the present study also tested whether issue involvement moderated the effects of comment valence and gatekeeping, with findings revealing that the influence of comment gatekeeping and comment type was uniform across low- and high-involvement news stories. Theoretical perspectives like dual-process models of persuasion (S. Chen & Chaiken, 1999; Petty et al., 2002) suggest that heuristics only influence attitudes when involvement is low rather than high. This theoretical maxim is relevant to the current work because perspectives like the MAIN model attribute the effects of comments to the activation of heuristics (Sundar et al., 2015). Therefore, it appears surprising that while study findings reveal the evidence of mediation via bandwagon perceptions, this effect does not vary between low- and high-involvement news stories. With that said, subsequent refinements of persuasion frameworks like the elaboration likelihood model have highlighted that cues can take on multiple roles not only as a heuristic under low involvement but also as an argument under high involvement (Petty & Wegener, 1999). Applied to the study of comments, it appears that online comments are influential to multiple types of news, a finding that contributes to existing research that has typically relied upon single news exemplars to study comment effects. While theory might suggest that attitude change that results from comment persuasion may vary in its duration between low- and high-involvement news stories, our findings nonetheless show that in the short term, comments can influence readers’ attitudes toward a variety of news stories regardless of their perceived relevance or importance.
Practical Implications
The results of the present study hold a variety of possible practical implications for news outlets. First, it should be noted that news consumers appear to respond negatively to news articles that use social media content in traditional news stories because it undercuts the perceived trustworthiness of the journalist, possibly due to the perception that the small handful of comments selected by the news author were chosen in a biased or unrepresentative manner. Given this pattern of results, journalists who wish to include social media content in news stories as evidence should strive to demonstrate to readers that such comments are unmodified and/or reflective of the general sentiment on social media. At a minimum, it is important for editors and journalists alike to be aware that tweets are not perceived as “just another quote” but instead are processed and perceived differently than “person on the street” quotes or other traditional journalistic sources.
Aside from the undesired effects of comment selection, study findings also reveal that comments from other readers that are uncivil or critical in tone can also be detrimental to perceptions of article newsworthiness, both because individuals tend to “jump on the bandwagon” of negativity and because such negative comments influence the perceived competence of the journalist. As a result, among those news outlets that continue to afford readers with the opportunity to comment, journalists and editors will need to consider ways to attenuate the effect of negative comments. Given the mediating role played by bandwagon perceptions, one possibility may be the inclusion of additional audience feedback metrics that can contextualize comments in relation to actual group opinion, such as the number of times the article has been liked or viewed. As for source trustworthiness, interface cues that indicate the presumed expertise of the source, such as the number of articles they have written or their number of years of service, may also help to augment the negative effects that comments tend to elicit.
Limitations and Future Research
The results of the present study should be contextualized according to several limitations. In terms of the manipulation of comment valence, this study focused on comments that were either exclusively positive or negative. Future studies can thus build upon this study by testing the effects of comment gatekeeping when both positive and negative exemplars are included in news, which might be perceived as readers as higher in warranting value than comments that are uniform in sentiment. Second, an additional consideration is that the current study contrasted the effects of comments that appear within news to those that appear after news exposure. Although this manipulation captures several of the most common ways that online comments are encountered, a third possibility unexplored by this study is the effects of comments that precede news, such as those that accompany news teasers posted on social media. Time/order of comment exposure (before vs. during vs. after) would therefore be a possible moderating variable for future research to explore, in addition to manipulating the degree to which comments are moderated by others. Finally, while our results might suggest that comment gatekeeping by journalists within the text of news is negatively perceived, there is a variety of other mechanisms through which comments are evaluated by others, such as the number of times a comment has been liked by peers or when a comment is responded to by a journalist. Our findings cannot generalize to these other forms of comment selection, which would be fruitful avenues for future research to test and contrast with the findings of the present work.
Conclusion
Despite the potential opportunities for extension and refinement, the findings of the present work nonetheless offer several novel theoretical contributions to the study of comment effects. While user-generated content is increasingly available to journalists, it appears that journalistic gatekeeping of comments from social media can undercut the perceived credibility of the news articles that journalists create. Negative comments (whether moderated by journalists or in comment sections) can also negatively affect news perceptions, as readers adjust their opinion toward news to be consistent with the assumed majority. Whether they are selected by journalists or appear in comment sections alongside news, comments thus continue to present challenges to the perceived credibility of news.
Footnotes
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.