The comment gap: Affective publics and gatekeeping in The New York Times’ comment sections

Abstract

This study examines journalists’ gatekeeping and audiences’ participation in The New York Times’ (NYT) comment sections. The concepts of affective publics and news gap informed a qualitative content analysis guided by the questions: (1) What are the characteristics of the comments selected for the NYT Picks section? (2) What are the characteristics of the comments selected for the Reader Picks section? (3) What is the overlap between the two types of comment sections depicted in these curated lists? The analysis was conducted on a sample of best comments according to the NYT (563) and its readers (400). Findings reveal that readers and journalists value comment sections differently, only coinciding 17.2% of the time (the comment gap). Both value comment sections as safe spaces for passionate comments. However, while readers reward confrontational, direct, aligned comments, journalists prefer conciliatory, articulate, and diverse ones. Implications for gatekeeping theory and boundary work are discussed.

Keywords

Affective publics comment sections journalism qualitative methods reader comments The New York Times the news gap

Online news comment sections have historically been subjected to the hope that they would aid democracy (Coe et al., 2014; Robinson, 2015). Driven by this hope, many newspapers opened comment sections in their articles. In doing so, they awarded commenters some gatekeeping power, traditionally reserved to journalists (Santana, 2016). However, comment sections failed to live up to these ideals, due to the pervasiveness of incivility (Coe et al., 2014; Muddiman and Stroud, 2017; Santana, 2014). The perception that comment sections have failed, paired with a decline in most news outlets’ bottom lines (Williams, 2017), has forced newspapers to either remove comment sections (e.g. The Atlantic) or invest in comment moderation (e.g. The New York Times).

This work argues that comment sections are valued differently by journalists and their audiences. An analysis of comment sections in The New York Times illustrated these similarities and differences. The New York Times is suitable for this project because it displays two different curated lists of comments: one based on selections by the newspaper (NYT Picks), and the other based on votes by the readers (Reader Picks). By contrasting these perspectives, this study sheds light onto the tension between journalistic gatekeeping and audiences’ participation (Chen and Pain, 2017; Diakopoulos, 2015; Lewis, 2012; Robinson, 2015), and how this tension can lead to the expansion of journalism’s boundaries (Lewis, 2012; Robinson, 2015).

Drawing from a qualitative analysis, this study builds on past literature regarding comment sections, but moves away from the role of incivility. I apply the ideas of affective publics (Papacharissi, 2015) and the news gap (Boczkowski and Mitchelstein, 2013), to explain the difference between what makes comment sections valuable according to journalists and readers of The Times. Findings reveal that readers and journalists reward comments differently attending to the interconnected features of emotionality, formality, and diversity of viewpoints.

Hopes and fears in comment sections

The premise that citizens’ participation in public discussion is good for democracy has been prevalent since the beginning of democracy itself (Coe et al., 2014; Papacharissi, 2004). This assumption relates to Habermas’ (1989, 2001) concept of public sphere, namely that in a democracy, all citizens are free to participate in rational civil debate. As a result of discussion, public opinion is formed. Because public opinion can be used by the people to control and influence political life, it contributes to the well-being of democracy. The concept of public sphere is relevant to this study because it establishes a relationship between public political discussion and democracy and helps explain the normative ideals of online news comment sections.

Many scholars viewed the democratization of the Internet as an opportunity for the realization of the public sphere allowing for social changes (Jenkins, 2006; Rheingold, 2000; Shirky, 2008; Surowiecki, 2004). Meanwhile, other scholars like Papacharissi (2002) and Sunstein (2007) showed how some aspects of the Internet (e.g. fragmentation of political discourse) could hold back its democratic potential. However, even the more skeptical accounts recognized the ability of online tools to promote political discussion. For example, Papacharissi’s (2004) work illustrates what online spaces for political discussion could be: a real contribution to a healthy democracy through public discourse and debate.

The hopes for the democratic role of journalism and its enhancement due to online affordances are not unfounded. Previous research distinguished online newspapers in their potential to use feedback and promote audience engagement and discussion (Jankowski and Van Selm, 2000). Additionally, more recent research has focused on comment sections. For example, Ruiz et al. (2011) used a discursive ethics approach to examine comment sections of online newspapers in five different countries and assess their success in providing a public sphere. Their findings revealed that comment sections can be spaces for debate based on mutual respect and exchange of viewpoints.

Nevertheless, comment sections have repeatedly failed to live up to these ideals, leaving journalists hopeful for their potential but disappointed at reality (Santana, 2011). Considering the evidence of incivility in comment sections (Coe et al., 2014; Muddiman and Stroud, 2017; Santana, 2014), it seems that newspaper readers are using them differently than the most optimistic journalists expected. This might be unsurprising, since research has shown that journalists and audiences think of news media differently. According to Boczkowski and Mitchelstein (2013), journalists and audiences have different ideas of newsworthiness. This difference between the stories that journalists considered most newsworthy and those that were most read (or clicked on) by audiences is what they called the news gap. Thus, audiences and journalists could also have different perceptions of what matters in comment sections. Understanding the differences in their notion of what comment sections ought to be is necessary to assess the risks of removing them and the benefits of maintaining and investing on them to foster audience engagement.

A reason why one would expect journalists and readers to understand comment sections differently responds to the journalistic role of gatekeeping. Gatekeeping encompasses the selection of news content and the forces impacting that selection (Lewin, 1947; Shoemaker and Vos, 2009; White, 1950). Selection decisions are affected by the journalists’ conception of the audience’s interest and channel space restrictions (Bruns, 2003). As such, the rise of online news has changed traditional gatekeeping. Bruns (2003) argues that journalists’ gatekeeping role has relaxed, as online channels do not impose the same space restrictions and individuals can access audiences directly (e.g. citizen journalism). These changes have given way to the emergence of gatewatchers. Without requiring journalistic skills, gatewatchers’ main task is to promote news sources by directing readers to them (Bruns, 2003). According to Bruns, gatewatching also involves an active role from the readers, since, by selecting information, they become their own gatekeepers. Recent work confirms that audiences are increasingly becoming gatekeepers (Tandoc, 2018), affecting journalists’ decisions. Web analytics provide journalists with engagement measures that subsequently influence journalistic work (Ferrucci, 2020; Tandoc, 2014), although there is still some confusion surrounding audience engagement and its measures (Nelson, 2018). This work builds on gatekeeping theory because commenters challenge the journalistic power over content in comment sections (Santana, 2016).

Ultimately, the tension between journalistic control and commenters’ participation reflects a negotiation about the boundaries of journalism, namely the definition of journalism reflected in what belongs in a journalistic space and what does not (Carlson, 2015). Therefore, some scholars have considered comment sections as journalistic boundary spaces (Robinson, 2015; Wolfgang, 2018). For example, Wolfgang (2018) defines three approaches journalists can take to resolve this tension and draw journalistic boundaries. These approaches include collaborating with commenters, keeping them away from journalistic spaces, and recognizing both positive and negative aspects of comment sections. Importantly, this negotiation is imbalanced, since journalists have traditionally held the role of gatekeepers.

In addition, the claim that audiences might defy journalists’ expectations connects with Hall’s (2006) concept of agency. News readers have the agency to use comment sections in ways other than journalists prefer, although they need to be motivated to exert such agency. According to previous research, news readers’ motivation to read and write articles’ comments is driven by social interaction and emotional expression, over cognitive stimulation (Diakopoulos and Naaman, 2011; Springer et al., 2015). Furthermore, users’ agency and motivations are constrained by the interface and the comments system (e.g. moderation guidelines). Despite these limitations, Robinson (2015) found that commenters often resist the journalistic authority by adopting roles that are traditionally journalistic.

While commenters can challenge journalists in comment sections (Robinson, 2015), journalists can re-assert their gatekeeping role through moderation and engagement with comments (Chen and Pain, 2017). Newsrooms can promote desirable comments online (Ksiazek, 2015). Allowing users to flag undesirable comments, moderating comment sections, increasing the participation of journalists (Diakopoulos and Naaman, 2011), and highlighting exemplary practices (Jomini Stroud et al., 2015; Sukumaran et al., 2011) can be effective strategies. This is true in the case of online news and the NYT Picks specifically (Diakopoulos, 2015; Kolhatkar and Taboada, 2017; Park et al., 2016). Scholars have used algorithms to identify high quality comments (Diakopoulos, 2015), designed moderation tools (Park et al., 2016), and developed comment classifiers (Kolhatkar and Taboada, 2017) to aid the task of moderation. Importantly, if NYT Picks send cues of preferred use, Reader Picks could do so too.

The relevance of social and expressive motivations for commenters has been established in previous literature (Diakopoulos and Naaman, 2011; Springer et al., 2015). Nevertheless, the question of how news readers use comment sections to respond to these motivations beyond ‘just’ commenting, remains unclear. Papacharissi’s (2015) concept of affective publics provides insights regarding comment sections from the commenters’ perspective. Furthermore, this concept connects with the notion of comment sections as spaces where journalists are gatekeepers, but also where readers retain some control. Affective publics are bonded by sentiments, their will to control a narrative, and their use of media to do so (Papacharissi, 2015). Moreover, in her analysis of the use of hashtags by affective publics, Papacharissi (2016: 312) raises some central questions:

‘Beyond the fact that these publics are networked, what tonality does this modality of networking afford? What happens to the mode, the texture of expression? What do networked publics sound like and how do they talk? What are prevailing tendencies of expressivity and connectivity, as these publics attempt to effect impact, assert agency, and claim power?’

The present study considers that affective publics use comment sections to associate around sentiments and claim their agency in storytelling.

This study provides a tentative answer to Papacharissi’s (2016) questions by using the case of the Kavanaugh hearings as covered by The New York Times. Comments were examined based on their structures of feeling (emotionality), textures of expression (formality), and structures of storytelling (diversity of viewpoints). Comment sections are spaces that connect journalists and readers, where readers’ comments are allowed and controlled by journalists (Robinson, 2015; Santana, 2016). Considering this, I employed the idea of the news gap (Boczkowski and Mitchelstein, 2013) to explain the difference between what comment sections ought to be according to journalists and readers. By examining curated lists of best comments according to journalists at The Times with lists of best comments according to readers, this study’s goal is to define what journalists and readers value in comment sections. To achieve this, the analysis was guided by the following research questions:

R1: What are the characteristics of the comments selected for the NYT Picks section?

R2: What are the characteristics of the comments selected for the Reader Picks section?

R3: What is the overlap between the two types of comment sections depicted in these curated lists?

Method

I conducted a qualitative textual analysis on a set of comments in The New York Times. I used a grounded theory analytical approach to develop themes reflecting the patterns found in the comments and established connections between the commenters’ reactions and the context given by the news pieces (Glasser and Strauss, 1967). I conducted the analysis in three steps: (1) pre-coding or data immersion, (2) first cycle coding – descriptive, to understand how commenters expressed themselves – and (3) second cycle coding – pattern coding, to collapse the codes into the categories of emotionality, formality, and diversity of viewpoints (Saldaña, 2013). The analysis rendered an explanation of the kind of comments that journalists and readers value the most. Excerpts of the comments illustrate the results. The commenter’s nickname and the tab where the comment was found are provided at the end of each quote. Finally, a comparison between the two tabs illustrated the comment gap, namely the difference between what journalists and readers find most valuable in comment sections.

A comparative analysis uncovered elements in comments that were rewarded by journalists and by readers based on their structures of feeling, textures of expression, and structures of storytelling. These three concepts were used by Papacharissi (2015, 2016) to articulate her concept of affective publics. She defined structures of feeling as a collective experiential framework that allows discursive expressions of sentiment. For the purposes of this project, structures of feeling are operationalized as emotionality, namely the expression of emotional reactions. Textures of expression are defined as tonality, the mode in which affective publics talk (Papacharissi, 2016). I operationalized textures of expression as formality, comprising eloquence, coherence, length, and complexity. Finally, structures of storytelling are conceptualized by Papacharissi (2016) as allowing an affective engagement of publics that are networked around an event and want to explain it in their own terms. In this article, I refer to structures of storytelling as diversity of viewpoints, including comment alignment, pervasiveness of diversity, and inclusion of criticism.

Moderation at The New York Times

Several reasons make The Times a useful case study. First, it includes a feedback system for comments: comments highlighted by journalists, known as NYT Picks, are easily accessible in a tab in the comment section, while another tab called Reader Picks indexes the comments according to the users’ upvotes. Also, The Times is the flagship newspaper in the United States, read by elites and everyday people alike. Amidst a stagnant crisis in journalism (Williams, 2017), the newspaper managed to maintain subscriptions (Fletcher, 2018). Furthermore, while some news organizations removed their comment sections due to incivility, The Times invested resources in moderating them. Despite this investment, the number of articles with comment sections decreased due to the high cost of moderation (Etim, 2017a). To be more efficient, The Times started employing Google’s technology to help moderators (Etim, 2017b), although moderation is still manually conducted (The New York Times, 2019). In any case, since automatic moderation still follows The Times’ editorial standard, it is safe to establish that NYT Picks reflect the journalists’ preferences.

Comments in The Times are pre-moderated before they become public, which means that the newspaper exercises editorial control even on Reader Picks comments. This is important because it limits commenters’ gatekeeping even in that space. Therefore, it is important to examine the newspaper’s moderation policies. According to The New York Times (2019), accepted comments must be ‘articulate, well-informed remarks that are relevant to the article’. Criticism of The Times is welcome. However, ‘personal attacks, obscenity, vulgarity, profanity (including expletives and letters followed by dashes), commercial promotion, impersonations, incoherence, and SHOUTING’ are not tolerated. Likewise, personal attacks against the staff and unrelated criticism to the newspaper or to its moderation policies are not allowed. The moderation team explains that NYT Picks represent diverse viewpoints and highlight interesting and thoughtful comments. While the Reader Picks are ranked by number of votes, there is no evidence that order matters for NYT Picks.

The sample for this study comprised articles covering the nomination of Judge Brett Kavanaugh to become an Associate Justice of the Supreme Court of the United States. On 9 July 2018, President Donald Trump nominated Kavanaugh to replace Justice Anthony Kennedy in the Supreme Court. Before the Senate Judiciary Committee voted to confirm Kavanaugh, he was accused of sexually assaulting Dr Christine Blasey Ford. The following days saw the questioning of both Judge Kavanaugh and Dr Blasey Ford by the Senate Judiciary Committee, the opening of an FBI investigation on the case and the ultimate vote and confirmation of Kavanaugh. I assumed that a controversial political issue that was recent enough to include NYT and Reader Picks would entice readers to comment. However, the results of this study will be impacted by this choice, since online news can elicit different levels of user engagement depending on the topic (Ksiazek, 2018).

To build the dataset, I searched for ‘Kavanaugh’ under the filter ‘most relevant’ in The New York Times archive. Out of the first 150 results retrieved, 40 included comment sections with both NYT and Reader Picks. From these 40, 14 pieces were in the politics section, 24 were opinion pieces, one was from The Upshot and one was from The Reader Center. The analysis included all the NYT Picks (563) and the 10 most upvoted comments in the Reader Picks (400). Replies were included in the sample whenever they were displayed as highlighted comments in the tabs.

The comment gap: NYT and Reader Picks

Findings reveal that journalists and readers value features of comments differently. Only 17.2% of the comments were present in both curated lists. There were 11 instances in which the top three most voted stories in Reader Picks were also selected as NYT Picks, six instances in which two of the top three were, 12 in which one of the top three comments was and 11 times in which none of the most valued comments by readers were also NYT Picks (see Table 1).

Table 1.

Comment gap in top three Reader Picks.

	Instances/total sample
Three of the top three Reader Picks are also NYT Picks	11/40
Two of the top three Reader Picks are also NYT Picks	6/40
One of the top three Reader Picks is also NYT Picks	12/40
None of the top three Reader Picks are also NYT Picks	11/40

Despite these differences, the themes that emerged during the analysis were common in both lists. This is not surprising since both NYT and Reader Picks include comments authored by the newspaper’s readers. However, there were differences regarding the readers’ most voted comments and those selected as exemplary by The Times. I identified three categories that respond to features rewarded by The Times and its readers: emotionality, formality, and diversity of viewpoints. These are connected and, together, they clarify the types of comment sections that The Times and its readers strive for.

Emotionality

Comments displaying clear emotional reactions were coded under the category of emotionality. Emotions play a central role in making sense of ourselves and our place in a community (Wahl-Jorgensen, 2019). The selected case study elicited emotionally charged responses regarding partisanship and societal and personal values concerning gender equality. Readers used the comment section as an expressive outlet, which also allowed other readers to find emotional support in the Reader Picks tab. On the other side, the NYT Picks worked as an effective tool for The Times to demonstrate its readers’ engagement. Although the intensity of emotions was moderately high in both tabs, the kinds of emotions displayed in each were different.

Comments in both tabs were marked by anger, ‘an essential political emotion’ (Wahl-Jorgensen, 2019: 12) reflected in the use of absolute terms, drastic language, and occasional capital letters. Anger was elicited by not feeling represented by the justices at the Supreme Court and by a sense of unfairness, and it was directed toward the events, President Trump, or Republicans. Some commenters were upfront in talking about their feelings (‘And hey, sorry if I seem cataclysmically enraged on this issue’ – Dan Stackhouse, Reader Picks), while others used historical references and metaphors (‘Rome is burning. Stop the hearing!’ – Vote with your $’s, NYT and Reader Picks).

Numerous comments also displayed enthusiasm toward the midterm election. Like anger, enthusiasm was present in both lists to encourage voting. There was a common sense of urgency for electoral mobilization reflected in calls to action regarding voting and persuading others to vote. In NYT Picks, the emphasis was placed in hope for a power shift in government, with references to the midterm elections (‘I’m sorry to say that I am pessimistic about their self control [sic]. November may be our last hope for this experiment in government’ – Michael Roberts, NYT Picks). In Reader Picks, enthusiastic comments used bellicose terms such as ‘resistance’, ‘fighting’, ‘war’, or ‘fight back’. For instance, commenters wrote about the struggles they faced in advocating for women’s rights (‘The next generation, and especially the women, just need to step up and fight for their rights. I’m just so tired of fighting that same battle over and over again’ – AMM, Reader Picks).

The dichotomy between hope and fight reflects a broader dynamic between the tabs, namely a more conciliatory, albeit discouraging, tone in NYT Picks, and a more confrontational one in Reader Picks. The appeasing tone in NYT Picks is a reminder that the newspaper’s moderation policies favor civility. It conveys that a valuable comment section, according to The Times, serves as expressive outlet but also promotes agreement. For example, some comments seemed to indicate an intention to compromise (‘Kavanaugh is qualified to be a Supreme Court Justice. As a liberal I would find it hard to vote against him based on his vita, conservative, or not’ – Wordsworth from Wadsworth, NYT Picks, posted before Dr Blasey Ford came forward).

Besides anger and enthusiasm, NYT Picks were also marked by sadness, pessimism, and resignation, as reactions to powerlessness. Fear was also evoked in both lists. Some commenters provided instances in which all these emotions were in play, feeling disheartened about Kavanaugh’s nomination (‘I have never felt so depressed, so demoralized and dejected as an American. [. . .] We are failing morally, ethically, and politically as a country and I fear the worst is yet to come’ – HK, NYT and Reader Picks).

Reader Picks used stronger language to convey negative emotions and accusations in a more direct way. Although their very publication indicates that the moderation team at The Times did not find them uncivil, Reader Picks used a more aggressive tone. This conveys that a relevant comment section, according to readers, is one that serves as expressive outlet, a space to vent about events that take an emotional toll on them. Some commenters seemed to mean harm for Republicans, which goes against social norms and therefore, shows that they felt the comment section was a safe space (‘I hope the Republucans [sic] pay dearly for this, in November and forever after’ – Patriot, Reader Picks).

Closer to the date of Dr Blasey Ford’s accusation, comments turned private and emotionally charged, explaining past trauma and moving personal anecdotes. This allowed for The Times to reward those commenters, enhancing the notion of the comment section as a space for free expression. Also, readers gave and received support through voting. For example, a personal account of rape was the most voted comment (2151 votes) by readers in one of the stories (‘I was 8 years old when I was first raped by a man that was 60 years older than me. [. . .] I didn’t tell anyone because no one would believe me’ – V, NYT and Reader Picks).

In summation, both The Times and its readers valued comment sections as a safe space for the expression of emotions. The Times highlighted more conciliatory comments, striving to promote agreement between commenters. Conversely, Reader Picks displayed more confrontational comments (some of them against social norms). This suggests that commenters used the comment section to vent and express their feelings and emotions.

Formality

Comments were examined for eloquence, coherence, length, and complexity. Most comments were elaborate and articulate, with few capital letters and exclamation points. A formal style was more obvious in NYT Picks, indicating a preference for comments consistent with the newspaper’s editorial policy. Although some Reader Picks met these criteria, in general they were short and sharp. In the first case, commenters potentially had the desire to be eligible for NYT Picks in mind, while in the second, they could be writing for other readers or for themselves. For example, a few comments were brief and straight to the point (‘Failing to remember events is a blackout. Failing to remember a lifestyle is dishonesty’ – Tansu Otunbayeva, NYT and Reader Picks). These comments were rewarded by readers, with the comment above being the most voted in its story (2185 votes).

Anecdotal comments were valued in both tabs, although they were among the most voted for Reader Picks. This reinforces the commenters’ appreciation of the comment section as a space to share private stories. Comments in the Reader Picks tab used a more aggressive, less formal, and more confrontational tone. Some comments employed capital letters, several question marks, strong language, and made serious accusations. These comments used a less articulate style, placing a focus on expressing opinions and feelings instead of building arguments: (‘It doesn’t MATTER. This man, this mass of seething rage and sneering entitlement, WILL be confirmed. The deck is stacked, the dice are fixed. WE are at War, and the GOP is winning. WHY ??? [. . .]’ – Phyliss Dalmatian, Reader Picks).

Similarly, comments in Reader Picks were conversational. Commenters quoted news excerpts to argue or support them. While comments in opinion pieces included allusions to the article’s author, these were non-existent in political articles. In these cases, mentions to the author were mostly indirect in both tabs. However, direct mentions were pervasive in comments in a few opinion articles. For example, one story contained six direct allusions to the author versus one indirect in NYT Picks and five direct versus one indirect in Reader Picks. Similar ratios were found in three more stories. This shows that for both The Times and its readers, comment sections were perceived as a space for dialogue, making fruitful discussion more likely.

Direct mentions were used to express criticism toward the articles’ authors. Then, the tone was also more confrontational in Reader Picks. This connects with the newspaper’s moderation rules, which discourage personal attacks. Intense confrontational tone against The Times staff could have prevented selection of such comments as NYT Picks. Direct criticism reflected the readers’ perceived expertise of the question. For example, some commenters thought the author of the piece did not capture a central aspect of the topic (‘Brett: Your piece betrays a fundamental misunderstanding of trauma and memory’ – Eleanor, Reader Picks).

Finally, subtexts were developed in comments that affected their style, such as irony and intellectualism. Both the use of irony and of intellectual references evidence a high cultural level among the commenters. Additionally, both figures of speech can be employed to convey negative emotions without losing formality. Some commenters used irony to show their outrage, resulting from the welcoming of Kavanaugh by students of Georgetown Prep (‘Gee, what a shock! Kavanaugh welcomed by white, privileged, and entitled classmates. They like beer!’ – Peggy Hansen, Reader Picks) and the confirmation of Kavanaugh (‘Looking forward to hundreds of millions of more unregulated guns in the town square, hundreds of millions of more heavily regulated uteruses, unlimited corporate supremacy [. . .]’ – Socrates, Reader Picks). Irony was present in both lists, although ironic comments ranked high in the Reader Picks tab.

Cultural references were also present in both lists. Some referenced pop culture content, like Star Wars, Lord of the Rings or Game of Thrones. Others responded to less popular authors, like Milton Friedman, Alexander Hamilton, or George Orwell. Like irony, cultural references were used to convey emotions and arguably to demonstrate sophistication. While Reader Picks were marked by irony, the tone of NYT Picks was more literary, with some comments including more than one cultural reference:

‘“If men were angels, no government would be necessary” is possibly the quote for which Alexander Hamilton is best known. However, his next statement is rarely quoted: “If angels were to govern men, neither external nor internal controls on government would be necessary.” [. . .] like Senator Collins’s, contort too comfortably to join in as her party is led by a man who would confound even George Orwell’s skills to pen “doublethink.”’ – GuiG, NYT Picks.

In summation, both The New York Times and its readers rewarded comments using a formal style. Formality was reflected in NYT Picks by selecting comments that were articulate, long, less anecdotal, and included cultural references. Conversely, readers rewarded comments that were short and to the point. Irony was rewarded by readers as a strategy to convey negative emotions while keeping a formal style.

Diversity of viewpoints

This category offers a reflection on comment alignment, pervasiveness of diverse viewpoints, and inclusion of criticism. This is the dimension where the two tabs differed the most. The most voted comments by readers were aligned. Meanwhile, the NYT Picks tab was used as a display for unpopular opinions, absent in the Reader Picks. This was also reflected in the fact that – except when they overlapped – readers’ top picks always received far more votes than NYT Picks. Thus, the Reader Picks depicted a group identity based on a shared sense of belonging and liberal values. The NYT Picks reflected a community of respectful debate, whereas the Reader Picks resembled a homogeneous community, with less debate.

By highlighting outnumbered viewpoints, The Times conveyed that comment sections are a space where difference of opinions can spur debate. Comparing the NYT Picks with the Reader Picks, however, it is obvious that this debate is artificial and created by the newspaper, not the readers. Thus, The Times indicates that comment sections should be a space where different opinions are encountered and considered. The newspaper is using its authority to control and regulate this space, revealing its gatekeeper position. The Times is acknowledging that comment sections are structured by its readers but ultimately, the newspaper is claiming comment sections as journalistic spaces.

Unpopular viewpoints reflected criticism toward Democrats, Dr Blasey Ford and liberal values, challenging The Times’s liberal leaning. Some comments showed their disagreement with all Democrats by qualifying their actions as absurd (‘The democrats’ claim that these documents are being kept from the public because Kavanaugh is hiding something is absurd when you consider the documents are the property of a presidential library.’ – AACNY, NYT Picks). Others showed disagreement with typical liberal values such as feminism (‘. . .prosecutors across this nation should start to prosecute women and jail them for routinely making false accusations, because there is presently almost no punishment for this crime of misandry, a license to kill that has been handed out to women. . .’ – winthrop staples, NYT Picks).

Instances of indirect criticism toward the newspaper were reflected in praise for the Republican party, Trump, and Kavanaugh. Some posts showcased exclusionary judgments. For example, Trump supporters were depicted as good, decent Americans (‘It is a GREAT day for America. The true and real American middle class and the Silent Majority of good and decent Americans has finally been reaffirmed and restored by the Senate and our President, Donald Trump!’ – tcmitssr, NYT Picks). Displaying criticism is a good opportunity for the newspaper to prove it is welcome. It reflects journalists’ responsibility to expose their readers to diverse viewpoints. Thus, The Times emerges as an entity that wants to be perceived as being above partisanship. Acting as a debate referee and highlighting criticism contributes to legitimize the newspaper in fulfilling a social role that aids democracy.

This role also applies to explicit criticism toward The New York Times staff and readers. However, criticism of the newspaper was balanced by NYT Picks praising articles’ authors. For example, some NYT Picks thanked columnist Ross Douthat for capturing their viewpoint (‘Thanks for a level-headed discussion, Ross. You use the word “if” in reference to whether Kavanaugh is guilty, and I don’t see that word used much in the current “debate,” most people already having decided who is guilty.’ – Charlie Reidy, NYT Picks). The reader selection included praise to the authors too, but to a lesser extent. Conversely, although author criticism was also present in NYT Picks, it was more pervasive among Reader Picks. This is because when author criticism was present in Reader Picks, readers also tended to be aligned. In one story, except one comment, all the selected Reader Picks showed disagreement with the author of the piece.

In summation, a valuable comment section based on the NYT Picks was one that included diverse viewpoints and spurred civil debate. A comment section where everyone could feel comfortable to post an opinion even when it did not fit in the dominant paradigm. The presentation of a balanced account of diverse opinions depicted the newspaper as embracing the professional value of objectivity. In taking the role of guarantor of a space for freedom of expression while safeguarding objectivity, The Times legitimized its function as ally of democracy. Conversely, the most voted comments in the Reader Picks tab tended to be aligned under an umbrella of shared liberal values. In that sense, readers of The Times could be aware of the dominant line of thought among fellow readers and find support in a community of like-minded individuals.

Conclusion

The present study focuses on comment sections because they represent the tension between the readers agency and journalists’ control (Diakopoulos, 2015; Lewis, 2012; Robinson, 2015; Santana, 2016) over gatekeeping. Ultimately, journalists can decide whether they award readers some participation power, its extent, and what the result should look like. Newspapers can decide to close comment sections in controversial stories or deny commenter’s anonymity to promote civility (Santana, 2014, 2016). However, journalists’ decision-making might be based on their perception of what comment sections ought to be without considering the commenters’ viewpoint, which, as this study shows, differs from their own. In fact, this study revealed a wide comment gap, suggesting that both journalists and readers exerted a distinct gatekeeping influence over the content in The Times comment sections. Given the power imbalance in journalistic spaces, journalists might successfully re-assert their gatekeeping role by exerting control over comment sections (Santana, 2016) but might potentially hinder audience participation along the way. Moreover, that audiences have the power to affect gatekeeping has been established in the literature (Bruns, 2003; Ferrucci, 2020; Tandoc, 2014). However, this study shows how audiences use that power to further their interests, express themselves and claim control of the space and narrative.

While acknowledging journalists’ assertion of gatekeeping, this study offers a rich picture of how affective publics (Papacharissi, 2015) can use comment sections to challenge journalistic control over a narrative through structures of feeling (emotionality), textures of expression (formality), and structures of storytelling (diversity of viewpoints). Commenters are affective publics to the extent to which they abandon their passive role and decide to actively participate in the journalistic space and engage in a negotiation with journalists to claim their power over the discourse. Moreover, it is precisely during this negotiation that boundaries of journalism can be drawn, since both actors in the journalistic process can use their resources to claim what belongs in, and what is left out of, the journalistic space (Carlson, 2015; Lewis, 2012; Robinson, 2015; Wolfgang, 2018).

Carlson (2015) highlights the idea of boundary objects (Star and Griesemer, 1989) to encourage research to move toward the concept of boundaries as shared spaces that hold different meanings for different groups while allowing their cooperation and communication across boundaries. Following Carlson’s (2015) call, this study suggests that the tabs feature in The Times’ brings comment sections closer to becoming boundary objects. Readers and journalists coexist and negotiate their roles in comment sections (Diakopoulos, 2015; Lewis, 2012; Robinson, 2015). The Times’ strategy to navigate this tension has been to include a tab for each perspective. While this does not affect the journalists’ gatekeeping role over the news, the Reader Picks tab gives readers a space where they can actively challenge journalists. The newspaper allows readers to take agency in rewarding other readers’ comments while crafting their own curated list responding to their journalistic authority. These lists are not entirely separate systems. The newspaper’s interface shows them juxtaposed and facilitates their comparison. Moreover, it appeared that some readers posted with The Times’ editors in mind (reflected in their writing style) while The Times selected their picks thinking of the readers (to expose them to diverse viewpoints). Taken together, this reinforces comment sections as boundary objects.

Regarding structures of feeling or emotionality, both tabs reflected comment sections as a safe space for passionate comments, supporting the importance of social, and expressive motivations for commenting (Diakopoulos and Naaman, 2011; Springer et al., 2015). Readers appreciated comment sections as a safe space for their emotions, even when these were confrontational or not normatively accepted. Meanwhile, journalists rewarded comments that strived for a more conciliatory environment, in which agreement between commenters was likely. Also, emotionality was reflected in textures of expression, or level of formality. Readers rewarded comments that were more direct and ironical, at the cost of eloquence. Conversely, journalists cherished comments that were articulate, intellectual, and highly formal.

Additionally, comment sections allowed readers to tell their story. Readers appreciated comments that were aligned in opinions and values, suggesting that commenters could be seeking support. Conversely, a valuable comment section based on the NYT Picks spurred civil debate, diversity, and respect. Although the NYT Picks created an appearance of debate, the readers still wrote and upvoted the comments that best reflected their positions. Mostly aligned, these reflected homophily, also a driver for affective publics’ engagement on Twitter (Papacharissi, 2016).

The comment gap provides evidence against a possible feedback loop between tabs. While NYT Picks are likely to get high exposure, this does not seem to affect the reader’s votes on the Reader Picks tab, as shown in Table 1. Future studies could focus on the impact that a comment has on reader’s perceptions and actions after being highlighted as a NYT Pick. Such a study would need access to The Times’ internal data to consider the time component. Since I was interested in what journalists and readers reward through their NYT Picks or their upvotes, I did not strive for such access. Unfortunately, this also means that the feedback loop between tabs is out of the scope of the present study.

These findings might only apply to the coverage of the Kavanaugh hearings by The New York Times, especially considering the particularity of American journalism, and the relevance of cultural contexts for the quality of democratic debates in comment sections (Díaz Noci, 2012; Ruiz et al., 2011). Additionally, my findings regarding formality reinforce the image of The Times’ readers as remarkably sophisticated. The New York Times differs from most other newspapers in its readership and its resources, for example allowing the newspaper to invest in comment moderation. Thus, it is important to acknowledge that, given its uniqueness, what works for The Times might very well not work elsewhere. Although not representing journalism in general, The Times was selected for this study because its comment feedback system allows to examine comment sections uses by readers and journalists. Unfortunately, the uniqueness of such a system makes it complicated to diversify the sample by including other news organizations. Another limitation involves controlling for the effect of strong emotions (i.e. triggered by the issue or the comments) on readers’ engagement (Berger and Milkman, 2012). While including other issues in the sample could help distinguish these effects, the central role of affect in reader’s motives to comment (Diakopoulos and Naaman, 2011; Springer et al., 2015) will still be confounding. This is clear for The Times, which only includes both Reader and NYT Picks when there are many comments, indicating audience engagement.

The analyzed comments reflected engagement in the news, an intention to be read and a willingness for debate, raising questions about the responsibility to determine the functions and norms of comment sections. A tentative answer to this question has been provided by this study in the case of The New York Times and is reflected in the journalists’ effort to retain their professional authority through comments’ moderation and selection. Ultimately, journalists are the space guarantors. Gatekeepers that can enhance the readers’ agency in the degree they desire, showing an unbalanced power relationship. However, newspapers need an audience and my findings reveal that comment sections fulfill readers’ emotional and social needs. This study matters because it provides evidence that yielding readers some control over a journalistic space (comment sections) can provide a venue for news audiences to engage with the news visibly, potentially enticing others. They can serve as a qualitative data point to clarify the concept of audience engagement (Nelson, 2018), and potentially counterbalance other measures of audience interest (Boczkowski and Mitchelstein, 2013). Moreover, these findings suggest that when given space, affective publics (Papacharissi, 2015) will use it to express themselves and claim their agency in storytelling. Taken together, these practices can be a stepping-stone to foster audiences’ participation and restore trust in journalism.

This study builds on the potential of comment sections to provide a public sphere and their challenges in practice. While incivility is pervasive (Coe et al., 2014; Muddiman and Stroud, 2017; Santana, 2014), comment sections can fulfill commenters’ needs for emotional expression. I appropriated the concepts of affective publics (Papacharissi, 2015), in terms of their agency to affect journalistic discourse, and the news gap (Boczkowski and Mitchelstein, 2013), regarding different perceptions between journalists and readers. I expanded these concepts by applying them to the context of online news comments to explain the difference between what journalists and readers of The Times value in comment sections. This study expands gatekeeping theory by providing a new case in which audiences become a main influence in journalistic gatekeeping and offers a qualitative account of audience engagement, complementing web analytics, news sharing, and social media (Tandoc, 2018). Additionally, this study offers a further conceptualization and operationalization of Papacharissi’s (2015) affective publics that can be used in future research on platforms other than comment sections or Twitter.

By providing a picture of the comment gap, a conversation can begin to take place about the connection and disconnection between journalists and readers, as applied to comment sections. This study offers a starting point for news organizations to reflect on audience participation and inform their decisions regarding comment sections. These conversations are central to (re)define journalism’s boundaries in online newspapers to an extent that matters for both readers and journalists. Conclusively, this study shows that, in navigating the struggle for control of comment sections by journalists and readers, democratic hopes and fears are two sides of the same coin, one that flips depending on the side taken.

Footnotes

Acknowledgements

Thanks to Prof. Matt Carlson for inspiring this article’s idea, helping me refine it, and encouraging me to share it with a broader audience. I presented an early version of this work at the Inaugural Symposium on Media, Professions and Society in Volda, Norway. I wish to thank the chairs of my session, Prof. Valerie Belair-Gagnon and Prof. Oscar Westlund, and the other attendees, who kindly provided me with invaluable feedback that shaped this work and instilled me with confidence. I also thank the anonymous reviewers, whose suggestions greatly improved this study. Finally, I thank Prof. Giovanna Dell’Orto for her continued guidance, advice, and support.

Author’s note

The author agrees to this submission. This article is not currently being considered for publication by any other print or electronic journal.

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Clara Juarez Miro

Author biography

Clara Juarez Miro is a PhD candidate at the Hubbard School of Journalism and Mass Communication, University of Minnesota.

References

Berger

Milkman

(2012) What makes online content viral? Journal of Marketing Research 49(2): 192–205.

Boczkowski

Mitchelstein

(2013) The News Gap: When the Information Preferences of the Media and the Public Diverge. Cambridge, MA: The MIT Press.

Bruns

(2003) Gatewatching, not gatekeeping: Collaborative online news. Media International Australia Incorporating Culture and Policy 107(1): 31–44.

Carlson

(2015) Introduction. In: Carlson

Lewis

(eds) Boundaries of Journalism: Professionalism, Practices and Participation. New York: Routledge, pp. 1–18.

Chen

Pain

(2017) Normalizing online comments. Journalism Practice 11(7): 876–892.

Coe

, et al. (2014) Online and uncivil? Patterns and determinants of incivility in newspaper website comments. Journal of Communication 64(4): 658–679.

Diakopoulos

(2015) Picking the NYT picks: Editorial criteria and automation in the curation of online news comments. ISOJ Journal 6(1): 147–166.

Diakopoulos

Naaman

(2011) Towards quality discourse in online news comments. In: ACM conference on computer supported cooperative work, Hangzhou, China, 19–23 March 2011, pp. 133–142. Hangzhou: Association for Computing Machinery.

Díaz Noci

(2012) Mubarak resigns: Assessing the quality of readers’ comments in online quality media. Trípodos 1(30): 83–106.

10.

Etim

(2017a) Why no comments? It’s a matter of resources. The New York Times, 27 September. Available at: https://www.nytimes.com/2017/09/27/reader-center/comments-moderation.html (accessed 28 May 2019).

11.

Etim

(2017b) The Times sharply increases articles open for comments, using Google’s technology. The New York Times, 13 June. Available at: https://www.nytimes.com/2017/06/13/insider/have-a-comment-leave-a-comment.html (accessed 28 May 2019).

12.

Ferrucci

(2020) It’s in the numbers: How market orientation impacts journalists’ use of news metrics. Journalism 21(2): 244–261.

13.

Fletcher

(2018) New York Times subscription revenue topped $1 billion in 2017. Forbes, 11 February. Available at: https://www.forbes.com/sites/paulfletcher/2018/02/11/new-york-times-subscription-revenue-tops-1-billion-in-2017/#87e665d581b9 (accessed 28 May 2019).

14.

Glasser

Strauss

(1967) The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine de Gruyter.

15.

Habermas

(1989) The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. Cambridge, MA: MIT Press.

16.

Habermas

(2001) The public sphere: An encyclopedia article. In: Durham

Kellner

(eds) Media and Cultural Studies: Keyworks. Malden, MA: Blackwell, pp. 102–107.

17.

Hall

(2006) Encoding, decoding. In: Durham

Kellner

(eds) Media and Cultural Studies: Keyworks. Malden, MA: Blackwell, pp. 166–176.

18.

Jankowski

Van Selm

(2000) Traditional news media online: An examination of added values. Communications 25(1): 85–101.

19.

Jenkins

(2006) Convergence Culture: Where Old and New Media Collide. New York: New York University Press.

20.

Jomini Stroud

Scacco

Muddiman

, et al. (2015) Changing deliberative norms on news organizations’ Facebook sites. Journal of Computer-Mediated Communication 20(2): 188–203.

21.

Kolhatkar

Taboada

(2017) Using New York Times Picks to identify constructive comments. In: NLP meets journalism workshop, Copenhagen, Denmark, 7 September 2017, pp. 100–105. Copenhagen: Association for Computational Linguistics.

22.

Ksiazek

(2015) Civil interactivity: How news organizations’ commenting policies explain civility and hostility in user comments. Journal of Broadcasting & Electronic Media 59(4): 556–573.

23.

Ksiazek

(2018) Commenting on the news. Journalism Studies 19(5): 650–673.

24.

Lewin

(1947) Frontiers in group dynamics: II. Channels of group life; social planning and action research. Human Relations 1(2): 143–153.

25.

Lewis

(2012) The tension between professional control and open participation. Information, Communication & Society 15(6): 836–866.

26.

Muddiman

Stroud

(2017) News values, cognitive biases, and partisan incivility in comment sections. Journal of Communication 67(4): 586–609.

27.

Nelson

(2018) The elusive engagement metric. Digital Journalism 6(4): 528–544.

28.

Papacharissi

(2002) The virtual sphere: The internet as a public sphere. New Media & Society 4(1): 9–27.

29.

Papacharissi

(2004) Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society 6(2): 259–283.

30.

Papacharissi

(2015) Affective Publics: Sentiment, Technology, and Politics. New York: Oxford University Press.

31.

Papacharissi

(2016) Affective publics and structures of storytelling: Sentiment, events and mediality. Information, Communication & Society 19(3): 307–324.

32.

Park

Sachar

Diakopoulos

, et al. (2016) Supporting comment moderators in identifying high quality online news comments. In: CHI conference on human factors in computing systems, San Jose, CA, 7–12 May 2016, pp. 1114–1125. San Jose, CA: Association for Computing Machinery.

33.

Rheingold

(2000) The Virtual Community: Homesteading on the Electronic Frontier. Cambridge, MA: MIT Press.

34.

Robinson

(2015) Redrawing borders from within: Commenting on news stories as boundary work. In: Carlson

Lewis

(eds) Boundaries of Journalism: Professionalism, Practices and Participation. New York: Routledge, pp. 152–168.

35.

Ruiz

Domingo

Micó

, et al. (2011) Public sphere 2.0? The democratic qualities of citizen debates in online newspapers. The International Journal of Press/Politics 16(4): 463–487.

36.

Saldaña

(2013) The Coding Manual for Qualitative Researchers, 2nd edn. Thousand Oaks, CA: Sage Publications.

37.

Santana

(2011) Online readers’ comments represent new opinion pipeline. Newspaper Research Journal 32(3): 66–81.

38.

Santana

(2014) Virtuous or vitriolic: The effect of anonymity on civility in online newspaper reader comment boards. Journalism Practice 8(1): 18–33.

39.

Santana

(2016) Controlling the conversation: The availability of commenting forums in online newspapers. Journalism Studies 17(2): 141–158.

40.

Shirky

(2008) Here Comes Everybody: The Power of Organizing without Organizations. New York: Penguin Press.

41.

Shoemaker

Vos

(2009) Gatekeeping Theory. New York: Routledge.

42.

Springer

, et al. (2015) User comments: Motives and inhibitors to write and read. Information, Communication & Society 18(7): 1–18.

43.

Star

Griesemer

(1989) Institutional ecology, “translations’ and boundary objects: Amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907–39. Social Studies of Science 19(3): 387–420.

44.

Sukumaran

Vezich

McHugh

, et al. (2011) Normative influences on thoughtful online participation. In: CHI conference on human factors in computing systems, Vancouver, Canada, 7–12 May 2011, pp. 3401–3410. Vancouver: Association for Computing Machinery.

45.

Sunstein

(2007) Republic.com 2.0. Princeton, NJ: Princeton University Press.

46.

Surowiecki

(2004) The Wisdom of Crowds: Why the Many are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations. New York: Doubleday.

47.

Tandoc

Jr (2014) Journalism is twerking? How web analytics is changing the process of gatekeeping. New Media & Society 16(4): 559–575.

48.

Tandoc

Jr (2018) Journalism as gatekeeping. In: Vos

(ed.) Journalism. Boston, MA: De Gruyter Mouton, pp. 235–254.

49.

The New York Times (2019) Comments. Available at: https://help.nytimes.com/hc/en-us/articles/115014792387-Comments (accessed 28 May 2019).

50.

Wahl-Jorgensen

(2019) Emotions, Media and Politics. Medford, MA: Polity.

51.

White

(1950) The ‘gate-keeper’: A case study in the selection of news. Journalism Quarterly 27(4): 383–390.

52.

Williams

(2017) Measuring the journalism crisis: Developing new approaches that help the public connect to the issue. International Journal of Communication 11: 4731–4743.

53.

Wolfgang

(2018) Taming the ‘trolls’: How journalists negotiate the boundaries of journalism and online comments. Journalism. Epub ahead of print 9 March 2018.