Deliberation and Identity Rules: The Effect of Anonymity,Pseudonyms and Real-Name Requirements on the Cognitive Complexity of Online News Comments

Abstract

How do identity rules influence online deliberation? We address this question by drawing on a data set of 45 million comments on news articles on the Huffington Post from January 2013 to May 2015. At the beginning of this period, the site allowed commenting under what we call non-durable pseudonyms. In December 2013, Huffington Post moved to regulate its forum by requiring users to authenticate their accounts. And in June 2014, Huffington Post outsourced commenting to Facebook altogether, approximating a ‘real-name’ environment. We find a significant increase in the cognitive complexity of comments (a proxy for one aspect of deliberative quality) during the middle phase, followed by a decrease following the shift to real-name commenting through Facebook. Our findings challenge the terms of the apparently simple trade-off between the goods and bads of anonymous and real-name environments and point to the potential value of durable pseudonymity in the context of online discussion.

Keywords

deliberation anonymity cognitive complexity online commenting social media

Introduction

Does anonymity – the ability to act publicly while concealing one’s identity – promote or undermine public deliberation online? There are two general and opposed responses to this question. One is that anonymity is dangerous because it enables the evasion of accountability. Discussing a case of misogynistic abuse on an anonymous chat forum, Martha Nussbaum observes that

the ability to create a new world in which [the abusers] exercise power and the women are humiliated depends on their ability to insulate their Internet selves from responsibility in the real world, while ensuring real-world consequences for the woman (Nussbaum, 2010: 85).

In a similar vein, Levmore notes that online anonymity allows ‘communication without retribution’ (Levmore, 1996: 2192–2193). If the value of public deliberation is rooted in its insulation from sources of coercive power (Habermas, 1996), then on this view anonymity would seem to undermine it by introducing an important power asymmetry into contexts of public communication. On the other hand, by insulating citizens from soft social pressures or hard sanctions and punishments, anonymity can enable people to speak in public with greater sincerity. Timur Kuran thus suggests that anonymity can limit the dangers of ‘preference falsification’, that is, of misrepresenting one’s preferences in order to conform to perceived public opinion. Preference falsification, on Kuran’s view, can lead to the suppression of minority opinion, with the consequence that minority views either weaken and disappear, or break out in sudden and radical bursts (Kuran, 1997: 261). This claim is echoed by Danielle Allen in her discussion of Creon’s regime in Sophocles’ Antigone, in which it was only the anonymous chorus who were able to speak truth to Creon’s power (though he did not listen), and from which she concludes that regimes of enforced public silence may look stable, but are prone to ‘rapid, radical change’ (Allen, 2010: 117). So anonymity can, on one hand, introduce power asymmetries and the strategic use of speech in the public sphere and, at the same time, offer release from demands for social conformity that can themselves reflect power asymmetries (Asenbaum, 2018). A reasonable, if not especially helpful, general answer to the question of the effect of anonymity on public deliberation would thus seem to be: it depends.¹ Assessing the question of the relationship between identity rules and the deliberative quality of the public sphere would thus seem to be something to be done on a case-by-case basis (and we present such an empirical analysis in this article).

Before we go on, however, we should make an important conceptual point. It is common to frame the question of the deliberative value of anonymity in terms of a simple dichotomy, a trade-off between the goods and bads of identifiability and anonymity. This is how it is framed by the theorists mentioned above. This is also how it has appeared in debates on policy with respect to online discussion environments. For instance, Facebook’s (then) marketing director, Randi Zuckerberg, said during a panel discussion on social media in 2011 that ‘anonymity on the internet has to go away . . .. People behave a lot better when they have their real names down’, she continued. ‘I think people hide behind anonymity and they feel like they can say whatever they want behind closed doors’ (quoted in Chun, 2015: 105). Indeed, it was this same framing of the problem that led the Huffington Post (HuffPo) a couple of years later to stop accepting anonymous comments in order to reduce trolling and verbal abuse, one of the changes whose effects are discussed in our study. This is also how it has been framed by a number of empirical studies looking at the effect of anonymity on online commenting (Manosevitch et al., 2014: 1180; Janssen and Kies, 2005; Towne and Herbsleb, 2012). Yet a number of theorists have recently highlighted the conceptual complexity of anonymity and its productive and communicative dimensions (Asenbaum, 2018; Moore, 2018; Véliz, 2019). The case we examine in this article gives us an opportunity to empirically address these claims.

In this article, we aim to take a step towards answering the broad question of the effects of identity rules on online deliberation by focusing more narrowly on the effects of changing modes of user identification on the cognitive complexity (CC) of comments on online news articles. A change in the rules of the HuffPo comment forum presented a natural opportunity to explore this question. Its comment forum was characterized by what we call non-durable pseudonymity (with a lot of trolling and large numbers of multiple or spoof accounts), until December 2013, when HuffPo changed the rules for its comment forums to require participants to authenticate their accounts through Facebook. This created an environment in which users did not have to use their real name with their comments, but nor could they so easily set up new accounts. In a second change, in June 2014, HuffPo adopted Facebook as the commenting system. This meant that HuffPo usernames were replaced with user’s Facebook names and avatars, approximating a real-name environment. This enabled a large-scale ‘before and after’ analysis. We collected over 45 million comments on more than 50,000 articles featured on the HuffPo front page between January 2013 and May 2015 and analysed them for CC, which, we suggest below, is a good proxy for levels of argumentation and justification, which in turn are core elements of deliberative quality.²

We begin by reviewing the existing literature on the analysis of online news commenting and establishing the predominant expectations that would be associated with a shift from anonymous to real-name environments. We then describe the case of commenting on the HuffPo and the gathering of data, paying particular attention to the concept of anonymity and differentiating between anonymous, pseudonymous and real-name environments. We then introduce our distinctive method for analysing one particular dimension of deliberative quality at scale through CC. We then present our results, which both confirm and confound widespread expectations that real-name environments tend to better quality deliberation. We conclude with some reflections on the implications of these findings for online institutional design.

Analysing Comments Below the Line

Over the last decade, commenting on online news publications has become a prominent part of the networked public sphere. Research into online commenting has ranged widely (see Reagle, 2015; Stroud et al., 2017), although according to one recent survey, research has tended to focus on aspects of online institutional design and their effect on communicative behaviour within the forum (Friess and Eilders, 2015). Within this general orientation to design choices and their relation to communication, identity rules have been a particular focus of attention. Many of these studies are framed in terms of the trade-off mentioned above, between the goods and bads of anonymity, and the goods and bads of real-name environments. Thus, a common finding has been that anonymity increases participation (Davies and Chandler, 2011; 23; Leshed, 2009) but lowers the quality of discussion (Rowe, 2014; Towne and Herbsleb, 2012). Conversely, online discussion spaces requiring users to authenticate their identities have lower rates of participation, but are associated with improvements in civility, rationality and sincerity (Coleman and Moss, 2012: 8; Janssen and Kies, 2005: 231; Towne and Herbsleb, 2012).

The most commonly investigated relation is between anonymity and civility, typically framed by the expectation that

[p]eople who are able to post anonymously (or pseudonymously) are far more likely to say awful things . . . Speaking from behind a blank wall that shields a person from responsibility encourages recklessness – it’s far easier to simply hit the ‘send’ button without a second thought under those circumstances (Foxman and Wolf, 2013: 114).

Regarding civility, it has been argued on the basis of both psychological experiments and qualitative analysis of commenting forums that moving towards real-name environments is likely to improve civility. The dominant claim in the social psychology literature is that anonymity online promotes ‘deindividuation’ (see Reicher et al., 1995). The idea is that in certain social situations, individuals lose connection to personal and social constraints, the so-called the ‘online disinhibition effect’ (Suler, 2004). The loss of inhibition experienced with anonymity can be a good or bad thing, depending on the inhibition in question. Anonymity can release people from inhibitions linked to social and economic status (see Bowker and Tuffin, 2003; Kim, 2006). Anonymity can also enable people to test out arguments and change their positions (Black, 2011: 14). Yet others have emphasized the negative effects of such release from constraints. Sia et al. (2002) claim on the basis of a laboratory study that anonymity decreases ‘social presence’, which in turn increases polarization of discourse. Indeed, the predominant view is that anonymous environments are associated with uncivil, abusive or ‘anti-normative’ communication (Lea et al., 2001; Polat and Pratchett, 2009).

Studies focusing more broadly on deliberative quality lead to similar expectations. By closely examining a sample of comments from two regional US newspapers, Manosevitch and Walker (2009) were able to code comments and assess them for respectful engagement, exchange of arguments, accuracy of information and general deliberative quality. Santana compared samples of comments from American newspapers that did and did not allow anonymous commenting and found that ‘[a]nonymous commenters were significantly more likely to register their opinion with an uncivil comment than non-anonymous commenters’ (Santana, 2014: 27). Rowe’s (2014) comparison of comments on the Washington Post’s website (which allows anonymity) and discussions of the same topic on the Washington Post’s Facebook discussion board (a real-name environment) concludes that the website discussion is ‘significantly more likely to be uncivil’ (Rowe, 2014: 121). Undertaking a similar comparison between comments on a news website and comments on the same story on a Facebook page, Hille and Bakker (2014) also find that by ‘discouraging anonymous responses, the quality of comments improved but above all the quantity of comments decreased after outsourcing comments to Facebook’ (Hille and Bakker 2014: 563). And based on an online discussion among Korean voters in the run up to an election, Rhee and Kim (2009) found that when users could post anonymously they were more engaged, in terms of quantity, but that when there were identity cues (which does not necessarily mean real name, which can in some contexts reveal a lot less than descriptions like ‘38-year-old mother of two’), outside observers evaluated the quality of the discussion more highly.

Our study makes two distinctive contributions to this discussion. First, we explicitly thematize pseudonymous communication, in which commenters have stable identities that are not their real names and are not easily linked to other social contexts. Much of the existing work on identity rules and online commenting reproduces a simple dichotomy between anonymity and real-name identity (Hille and Bakker, 2014; Janssen and Kies, 2005; Rowe, 2015; Santana, 2014; Towne and Herbsleb, 2015) and pays relatively little critical attention to those terms. Thus, for instance, Towne and Herbsleb use terms like ‘anonymous’ and ‘pseudonymous’ interchangeably (Towne and Herbsleb, 2012: 108), as do Foxman and Wolf (2013: 114). Yet the fact that these researchers also invoke terms like ‘complete anonymity’ (Santana, 2014: 28) and ‘fully anonymous’ (Towne and Herbsleb, 2012: 108) indicates that there is recognition of the need for a further distinction. We will thus develop below an explicit theoretical account of identity rules, drawing on recent theoretical work on the conceptual complexity associated with anonymity (Asenbaum, 2018: 459), and within this the distinctive discursive value of pseudonymity (Moore, 2018; Véliz, 2019). Our second contribution is our focus on the dimension of argumentation and justification, which we analyse through a measurement of the CC of comments. We do not claim to address the overall deliberative quality of comments, since deliberative quality is typically taken to encompass dimensions ranging from rationality, interaction or exchange of arguments, equality, civility, constructiveness and reference to a common good (Friess and Eilders, 2015; Stromer-Galley, 2007). However, mutual justification is at the core of a wide range of approaches to the study of deliberation, whether ‘type I’ or ‘type II’ deliberation (Bächtiger et al. 2010). We will discuss this further, after we have set out our approach to identity rules.

Disaggregating Anonymity

Our study sets out not simply to compare commenting behaviour under anonymous and real-name conditions but also, as we noted above, to thematize pseudonymity. Before describing how we gathered the data, a few words are in order about how we define these terms. Following Fredheim et al. (2015) and Moore (2018), we disaggregate online identity into three main factors: traceability, durability and connectedness. Traceability means that observers can link online statements and behaviour to real persons. In our context, it is important to note that traceability is distinct from the disclosure of identity to fellow commenters. You can make comments anonymously or under a pseudonym and yet it may be possible (with some effort) for advertisers or the security services to trace those comments to the person that made them. Many scholars are concerned about online anonymity in the dimension of traceability (e.g. Zarsky, 2004) and seek mechanisms by which online users can remain ‘unreachable’ (Nissenbaum, 1999) or ‘untraceable’ (Froomkin, 1995) by advertisers or public authorities. Traceability is undoubtedly important to deliberation. It has the potential to chill communication, in so far as it creates a risk of exposure and retaliation from actors who object to certain forms of speech. Yet it also creates the possibility of accountability for abusive or otherwise unlawful behaviour. Depending on whether we have in mind examples of misogynist abusers (Nussbaum, 2010) or political dissidents (Zarsky, 2004), we might emphasize the value or the dangers of being untraceable. But in either case, traceability is to one side of our concerns in this article, which turn on the identifiability of commenters with respect to other commenters in the forum. Traceability does not help us grasp the relation between disclosure of identity and discussion quality, so in our analysis, we draw primarily on the distinction between durability and connectedness.

Durability refers to the ease or difficulty with which identities can be acquired and changed. A durable identity need not be a real name, but it must be stable over time within a particular context. This dimension has come to the fore in the context of online communication. The concept of durability opens up a distinction between easy anonymity, in which actors are able to easily create new and multiple identities, and stable or durable pseudonyms. It is this stability that grounds the possibility of a limited ‘internal’ communicative accountability (Moore, 2018). As Resnick and Friedman (2001: 173) put it, cheap pseudonyms create ‘opportunities to misbehave without paying reputational consequences’. Where hurdles such as registration and verification are introduced, it remains possible to create new identities, but it becomes a little harder and more time-consuming. Users are more likely to stick with a particular name, exposing them to the reputational consequences of their behaviour. Where new pseudonyms are easy to create, online identities are disposable; if you acquire a reputation for abusive or untrustworthy behaviour, you can just create a new pseudonym and start again. There is some empirical support for this suggestion in findings that pre-registration of users can improve the quality of comments (Santana, 2014: 28).

Connectedness refers to bridging and linking communication across different social contexts. This dimension can be illustrated by rules or norms of non-disclosure about who said what in a particular deliberative context, designed to block the connection of persons to statements, of which the Chatham House rules are a well-known example. In the online context, the use of real names opens the possibility of connectedness in the sense that your statements can potentially become known to your family, colleagues, friends and other social groups with whom you are associated. Connectedness enables statements to be attributed to particular individuals and thereby travel with them into different social contexts. Connectedness works in two directions. It involves the connection of your statements across different social contexts (your comments on a sports team appearing on your professional network). And it involves the exposure of more information about you to interlocutors in a particular context (if commenters on a news site can see your face, name, and perhaps also background, professional position and so on, then you are more connected in our terms). Connectedness, finally, admits of degrees. The greater the potential for linking your statements across social contexts and the greater the revelation of personal details to your immediate audience, the more connected you are.

This gives us three modes of identification:

Neither durable nor connected;

Durable but not connected;

Durable and connected.

The first two categories are usually described as ‘anonymous’ both in everyday language and much of the literature reviewed above and they do indeed share the quality of enabling users to act publicly while concealing aspects of their identity. However, we prefer to describe these modes in terms of durability and connectedness, not least because neither of these modes necessarily involves a complete absence of identifiers.³ There are two further points to note. First, both connectedness and durability admit of degrees. What we are describing below as a real-name environment means that some aspects or cues to your real identity available to interlocutors in the forum, and your statements in the forum are attributable to you in other social contexts, that is, that they display higher degrees of connectedness. Second, we recognize that these types are not perfectly recreated in any particular online environment. However, the changes to the commenting architecture made by the HuffPo over 2 years from January 2013 enabled us to take a look at something that approximates quite well to these three modes of identification.

Case Study: Changing Identity Rules on the HuffPo

In our period, HuffPo twice changed the identity rules governing its commenting space. The first change took place in December 2013. Prior to December, the platform allowed users to comment under any chosen name, and they could easily comment under different names if they wished. The platform experienced aggressive ‘trolling’ by users operating multiple or spoof accounts. HuffPo found it was investing too much in policing this increasingly unruly space and decided to regulate its forum by requiring new users to authenticate their accounts through Facebook.⁴ On the face of it, little changed: in the new, durable pseudonymous environment, usernames were still not required to be the user’s real name (though some took the option of appearing under their Facebook name and avatar), but behind the scenes, Facebook’s database helped weed out fake accounts. In June 2014, HuffPo changed to commenting through Facebook, meaning that HuffPo user profiles were replaced by Facebook profiles in a ‘real-name’ environment. In this phase, comments appear below the line of the news article under the user’s Facebook name and avatar, as well as – depending on a user’s privacy settings – appearing simultaneously on their Facebook page.

The changes made by the HuffPo allow a series of before and after analyses. We used a script to systematically collect all the conversations below each article. We unravelled each comment thread, recursively checked for, downloaded and processed any responses to each comment. Having inspected the data, we found that content from the ‘featured blog posts’ sidebar tended to attract very low commenting levels and therefore introduce a source of noise. For this reason, we excluded texts from the ‘featured blog posts’ sidebar. In this way, we collected more than 45 million comments on more than 50,000 articles featured on the HuffPo front page in the period January 2013 to May 2015.

Method: Measuring CC

The measurement of deliberative quality has tended to involve qualitative analysis of text or observation of communicative exchanges in light of a range of normative criteria, which typically include practices of justification and argumentation, reciprocity and mutual respect, sincerity, constructive politics and free participation (Gerhards, 1997; Steiner et al., 2004; Stromer-Galley, 2007). In our study, however, we focus primarily on the dimension of reason-giving or argumentation. One reason for our restriction is that justification, or asking for and giving reasons, is a common element in an otherwise diverse set of conditions for what makes for good deliberation. We assume that the argumentative dimension is at least necessary, albeit not sufficient, for the deliberative quality of large-scale public discourse (Gastil, 2008).

In order to operationalize (argumentative) deliberative quality, we make use of CC – a psycho-linguistic indicator that measures language complexity (Abe, 2011, 2012; Gruenfeld, 1995; Schroder et al., 1967; Suedfeld, 1992, 2010; Tetlock, 1983, 1984); While it does not directly measure deliberative quality, it serves as a capable proxy. Drawing on empirical comparisons of this method with qualitative analysis of discursive exchanges using tools such as the Discourse Quality Index, CC correlates strongly with the justificatory dimension of deliberation (Beste and Wyss, 2014; Jennstål, 2019). We can, therefore, draw on conceptual and empirical validation to warrant our operationalization.

While this approach does not allow us to show the degree to which communicative exchanges exhibited respect or reciprocity – we would not extrapolate from CC to, say, respectful conduct, unless there is a unitary or monolithic conception of deliberation tying together the internal components (e.g. justification and respect) of the discourse ethics – it does permit us to analyse data at large scale. Furthermore, the CC approach is particularly well suited to capture the argumentative dimension of deliberation. We acknowledge that argumentation, defined as the exchange of reasons for or against something (Manin, 2005), is perhaps not sufficient for good deliberation; however, it is by all accounts a necessary condition. As a result, recent analyses of political deliberation have started to utilize CC for the purpose of tackling deliberative quality (Brundidge et al., 2014; Wyss et al., 2015).

According to the psychological literature, CC refers to the degree of perception of the multidimensionality of a given problem (Schroder et al., 1967; Suedfeld et al., 1992; Van Hiel and Mervielde, 2003). It essentially measures how individuals’ thought processes are constituted rather than about what actors actually say about a problem, for example, how logical an argument is. CC consists of two dimensions: (1) differentiation and (2) integration (Owens and Wedeking, 2011). The former refers to the quantity of information, as well as the corresponding problem dimensions, facets and layers that are considered by a speaker or writer to relate to a given topic, capturing the degree of sophistication with which an individual perceives and conveys an issue. Thus, CC provides cues concerning the breadth of an individual’s perspective. Integration, on the other hand, captures whether perceptions of connections, dependencies and other relationships are recognized by a person with respect to a given topic and its substantive sub-dimensions. It refers to the depth of understanding of a speaker or writer, for example, whether interdependencies that are encapsulated in the problem under discussion (such as externalities, side effects and conflicting values) are taken into account. Hence, both differentiation and integration tackle disparate dimensions of sophistication: breadth and depth (Schroder et al., 1967).

When measuring CC, however, differentiation and integration are conceived as an aggregate (Owens and Wedeking, 2011). Originating in the works of 1960s psychologists, the target of the holistic concept (i.e. the aggregate of differentiation and integration) is to capture an individual’s capacity for opinion ‘flexibility, high levels of information search, and tolerance for ambiguity, uncertainty, and [accepting] lack of closure’ (Suedfeld, 2010: 1670). High levels of CC are, thus, indicative of an individual’s cognitions being embedded, organized and categorized within a dense intellectual system. At the other end of the scale, the CC score diminishes to the extent that the cognitions are narrow, superficial and fragmented. For example, CC has been found to negatively correlate with ideological rigidity, as individuals with a fixed set of beliefs and corresponding preferences are less able to incorporate conflicting new information into their argumentative repertoire (Tetlock, 1983, 1984). Thus, emancipatory ideals such as anti-authoritarianism and opinion moderation – both key components of critical theory and deliberative politics – are integral parts of high CC scores.

Usually, CC is assessed manually through the labour-intensive reading of documents by trained coders. However, fully automated analyses have become a viable approach to consider digital data in particular (Abe, 2011, 2012; Brundidge et al., 2014; Wyss et al., 2015). This is especially true when large quantities of data are analysed and manual coding becomes virtually impossible. As for the means of measuring CC in an automated fashion, the Language Inquiry and Word Count (LIWC) dictionary has been shown to yield valid results in a variety of formal and informal settings (Tausczik and Pennebaker, 2010). Since recent studies have also used the LIWC dictionary to gauge CC in a formal context such as prepared speeches and parliamentary debates, we will use it to analyse political online discussions. After careful validation of manifold operationalizations (Beste and Wyss, 2014), we apply the formula suggested by Owens and Wedeking (2011). They use a combination of complex words (words equal to or above six letters in length), causation words (e.g. ‘because’), as well as expressions indicating personal thought processes (e.g. ‘believe’), discrepancies between the is and ought status (e.g. ‘should’), inhibition (e.g. ‘obstacle’), tentative conclusions (e.g. ‘perhaps’), inclusivity (‘and’), exclusivity (e.g. ‘except’), negations and certainty (e.g. ‘absolutely’). We then calculate the Z-standardized values for each unit of analysis by inserting the above-mentioned LIWC indicators into the following formula in order to obtain the CC score

\begin{matrix} C C = Z (s i x l e t t e r s) + Z (c a u s a t i o n) + Z (i n s i g h t) + Z (d i s c r e p a n c y) + Z (i n h i b i t i o n) \\ + Z (t e n t a t i v e) + Z (i n c l u s i v e) - Z (e x c l u s i v e) - Z (n e g a t i o n s) - Z (certainty) \end{matrix}

The unit of analysis are individual user comments, which can be theorized as ‘speech acts’ in the online environment. By applying an automated dictionary-based approach, CC applies a similar ‘bag-of-words’ (BOW) approach: speech is not parsed in its entire functional differentiation (say, the grammatical structure), but speech acts are rather processed as agglomerations of words. Those words, in turn, have psycho-linguistic properties, which are indicative of the speeches’ complexity. For instance, the word ‘because’ is considered to signal a justification in a speech act, regardless of its embeddedness in the sentence structure.

BOW methods come with specific disadvantages. Since language merely is interpreted as a sum of expressions, there are no contextual considerations. For instance, automatic BOW analyses fail to detect topical or cultural specifics when it comes to, say justification levels. Normally, therefore, it is imperative to adapt the dictionaries to the domain they are applied to. In the case of the LIWC and CC, however, the linguistic indicators in the formula are such that the in- and extension of the concept CC can be maintained and validly measured in a rather universal fashion (Owens and Wedeking, 2011). This is because, first, the psycho-linguistic character of the LIWC dictionary in conjunction with the CC concept has already been successfully applied in a host of different political and non-political contexts (Abe, 2011, 2012; Tausczik and Pennebaker, 2010). Second, the LIWC is less vulnerable for error terms and systematic distortions when comparing cases within the same domain. Third, we performed random sample checks of the codings to ascertain the sensibility of the automated approach. Hence, we are confident that we can validly capture CC for the specific case of HuffPo user comments.

Results

For the verification of our hypotheses, we collected 45 million comments posted on the HuffPo website between January 2013 and May 2015. In order to analyse the evolution of CC over time, we aggregated the comments of the actively commenting HuffPo users to monthly averages. After aggregation, we end up with 2.7 million user-month observations from in total 336,574 users. That is, to say that we only considered the observations that ground on at least three user comments. This filter was applied to improve the robustness of the CC score.⁵ By this procedure, we end up with 2.7 million user-month observations from in total 336,574 users, and in total 43,554,480 comments.

For a first overview over the data, we plot the courses of our CC variable and the number of active users over each observed month (see Figure 1). Two observations are particularly striking. During the observed time, we recognize a significant decrease in the number of users posting comments (grey curve). In the years 2014 and 2015, the comments posted on the HuffPo website were only a fraction of the numbers in 2013. Certainly, this decrease can be seen as a success indicator: The first policy reform in December 2013 helped to remove fake accounts and made it harder for blocked users to reappear under a new account.

Figure 1.

User Averages of Cognitive Complexity Over Time.

The second striking observation is that the CC level (black curve) was not constant over time and seems to be influenced by the policy reforms. CC was highest in the pseudonymity regime in the first half of the year 2014. Under this regime, users required a verified account to post comments on the HuffPo website, while their public profile remained untouched. This first analysis suggests that the relatively soft reform had a positive effect on the CC index, which increased about 0.6 points. The index decreased again after the second policy change. Hence, under pseudonymity, the average comments found on the HuffPo website were written in a more complex manner than the comments found in the surrounding phases.

For us, an intriguing question is, whether this CC pattern can be attributed to shifts in individual behaviour. Did users indeed write comments in a higher quality? This question cannot be answered by the above analysis, because the reported increase in the CC level could also be triggered by a participation bias. That is, a numerical reduction of bots and abusive users could also have resulted in an increase in CC. To sort out this alternative effect, we are going to reduce the data set to users, who posted comments in more than one phase. In the next section, we only consider users who posted under pure anonymity as well as under pseudonymity, which allows us to get a more detailed picture on the effects of the first policy reform. In the subsequent section, we are going to gauge behaviour change triggered by the second reform. There, we limit the analyses to users who posted comments in the real-name identify period as well as at least in one of the former periods.

First Policy Reform: From Non-Durable to Durable Pseudonymity

To gauge whether comment quality changes during the first policy reform, we only consider users who posted comments both before and after the first reform. Following this procedure, we end up with a data set made up of 47,657 unique users.

Instead of just plotting the raw data, we estimate a statistical model that describes the course of measured CC level. It turns out that a simple but adequate model is the following linear regression model (Formula I)

Formula I : CC = β_{0} + β_{1} R + β_{2} M + β_{3} RM + e

In the above model, R stands for reform and represents a dummy variable which is coded as ‘0’ for all observations before and ‘1’ for all the observations after the first reform. M is a normalized variable measuring the time distance from the data point to the first reform. Finally, the model also features an interaction term between the reform dummy and the distance variable M. This enables to model a potential long-term effect of the reform.

By estimating this model (N: 362,591; F: 243; p < 0.001), we again find that the comment quality measured by CC increased just after the first policy reform (the coefficient of the reform dummy equals 0.24; p < 0.001). Hence, just after the reform, the posted comments are in average on a higher complexity level than the comments before the reform. The result suggests that the durability of user profiles indeed has a positive effect on comment quality. Because the measured effect size of 0.24 is about the same strength as it has already been suggested in Figure 1, we think that there is not much room for a potential alternative mechanism (i.e. the participation bias) to increase the CC level. Apparently, regarding the CC level, people who posted in the first period do not differ that much from the people who posted comments in the second phase. Last but not least, the reform seems to have also a small but significant long-term impact. The interaction term between reform dummy and time distance equals 0.05; p < 0.001. The model is visualized in Figure 2, which makes apparent that the operators of the HuffPo could achieve by this policy reform a turn-around regarding comment quality.

Figure 2.

Modelling CC Level during the First Reform.

How robust is this model? We tested the robustness of our results using different model specifications and a bandwidth test. The latter is a straightforward procedure to gauge model robustness. In the above model, we considered the largest possible time range before and after the reform (from January 2013 to May 2014, standing for a bandwidth of 18 months). It is helpful to learn, whether the estimated coefficients change, when we reduce month by month the bandwidth. According to Figure 3, both coefficients remain at a similar level when we shrink the bandwidth. This result supports that the estimated model is stable even when we significantly change the range of observation.

Figure 3.

Bandwidth Tests: Changes of Coefficients When Number of Months Included Varies.

Second Policy Reform: From Durable Pseudonyms to Real-Name Identity

We now turn to the second reform and gauge whether the outsourcing of the comment section to Facebook provoked a change in individual behaviour. The analysis of this second reform comes with a technical difficulty: because user profiles were transferred to Facebook, we cannot directly connect the active users of this third phase with the user profiles in the earlier phases. We could establish such a connection only for a subset of users who used already before the second reform the Facebook avatar for their HuffPo profile. By making use of this link, a total of 8805 users could be identified as users who posted before and after the second reform.

The formula of the model used to analyse this second reform looks slightly different from the one before (Formula II). It features two dummy variables and two interaction terms. Similar as before, R_1/2 stands for the first or the second reform and represent dummy variables coded as ‘0’ for all observations before, and ‘1’ for all the observations after the specific reform. M is a normalized variable measuring the time distance of the data point to the second reform

Formula II : CC = β_{0} + β_{1} R_{1} + β_{2} M + β_{3} R_{1} M + β_{4} R_{2} + β_{5} R_{2} M + e

After estimating this second model (N: 104,514; F: 38.8; p < 0.001), we again find support for the individual effect of the first reform (the coefficient of the first reform dummy equals 0.21; p < 0.001). Most importantly, the coefficient of the second reform is negative. Hence, it seems that the outsourcing to Facebook resulted in a diminishing level of CC. Furthermore, both interaction terms are significantly positive, suggesting that in both cases, a positive long-term effect occurred. This second model is visualized in the Figure 4, which makes apparent that the quality of comments of the HuffPo reached a peak during phase 2.

Figure 4.

Modelling CC Level during the Reforms.

Again, we test model robustness by using different model specifications, distinct filter criteria and bandwidth tests. While the above model is estimated over all available months (bandwidth of 30), we are interested to learn how coefficients change when we include only a smaller number of months. For estimating the models with bandwidth lower than 16 months (January 2014 until May 2015), we removed from the model specification the first interaction term and the first dummy variable. According to Figure 5, the coefficient of the dummy variable remains significantly negative over a large part of the bandwidth range. This is not the case for the interaction term, which becomes insignificant when bandwidth is less than 19 months. According to this test, it is everything but clear that the second reform triggered a long-term effect.

Figure 5.

Bandwidth Tests: Changes of Coefficients When Number of Months Included Varies.

What can we say about the effect size? Is a CC change of 0.24 points a small or a large effect? To better assess the effect size, it helps to look for comparable effects found in previous empirical literature. Wyss et al. (2015) found for the context of Swiss Parliament debates a difference in CC of 0.32 depending on whether the speaker is a representative of a Government party or an opposition party. In a study of 2014, researchers found that chronic stress was related to linguistic complexity with β of −0.25 (Saslow et al., 2014). Considerable higher effects on CC are found in case of the opinion complexity of US Supreme Court justices. The CC drops by more than 2 index points when justices state dissenting opinions. These studies set out a range of what effect strength we can expect. Regarding a relatively soft institutional change, the effect size we identify here meets expectations.

Discussion

The expectation that poor discursive behaviour in anonymous environments would be improved if users had their ‘real names down’ seems not to be clearly or straightforwardly borne out by this case. The most striking finding from our data set is that the CC of comments shows a marked improvement in the shift from non-durable to durable pseudonymity. But CC reduces again in the ‘real-name’ phase, when comments are made under the users’ Facebook names. We find this pattern repeated when we restrict the analysis to those present through all three phases under study. Here, we raise some potential explanations of this pattern and discuss its significance.

The idea that people behave better with their real names down is primarily underpinned by assumptions about communicative accountability. One potential explanation for the difference between the latter two phases can still draw on the mechanism of communicative accountability. However, following Moore (2018), we can distinguish two sorts of communicative accountability: accountability to the audience within the forum itself, on one hand, and a broader accountability for one’s speech and actions that is not limited to a particular discursive context, on the other. Moore’s discussion of the deliberative potentials of pseudonymity suggests that continuity of identity within a particular discursive context is a necessary condition for a minimal form of communicative accountability, which involves the possibility of making and meeting demands for justification within the forum.

What is introduced during the ‘real-name’ phase, however, is not only durability but also connectedness. Adding connectedness brings a shift from an audience of only other commenters and readers on the platform to an audience that might also include your other friends, colleagues and relatives. With the introduction of commenting through Facebook, users not only get cues about the identity of their interlocutors, they also find – depending on their settings – that their comments appear on their Facebook timeline as well as on the HuffPo comment page. Root comments (comments directly on a particular article) can be cross-posted if users choose. Comments on those comments are confined to the respective comment space. For instance, if a friend comments on a HuffPo article and cross-posts to Facebook, and I reply on Facebook, this will not appear below the line on HuffPo. And if I comment below the line on HuffPo, it will not show up as a comment on my friend’s Facebook timeline. The process is somewhat confusing, and it means that in the Facebook phase, the primary audience of any given commenter becomes uncertain. In the anonymous and pseudonymous phases, we can be confident that comments are oriented to the audience of other readers in the comment space. In the real-name space, users may orient more to the audience in the comment space, or to their own friendship group on Facebook, or both. This opens up the possibility that the comments people choose to make may be conditioned by the expected response of their friends.

What does this shift in audience entail? One possibility is that a network composed of family, friends and colleagues is one in which people are less likely to either (1) encounter difference (as one’s networks may reflect a degree of prior homogeneity) or (2) engage actively with those holding different positions, or both. Regarding exposure to diverse opinions, Bakshy et al. (2015), conducting research for Facebook, claim that Facebook users are in fact exposed to a wide range of diverse opinions on the platform. However, there is evidence that Facebook users are less likely to discursively engage and argue with those with different opinions on the platform. Recent research by Rossini (2018), for instance, finds that justified opinion expression is ‘80% less likely to occur on Facebook than on news sites’ (Rossini 2018, 20). She recognizes that discourse on Facebook may be more civil, but she – rightly, in our view – separates the question of civility from that of argumentation. With this in mind, we could say that Facebook is perhaps not where you go to engage the other side, but where you go to talk with your friends about the other side. While this is necessarily speculative, we think that the shift in audience from a space in which users engage with diverse others under terms of structured impersonality to a space in which users engage primarily with self-selected networks of friends is a plausible explanation for the observed reduction in argumentative and justificatory modes of discourse.

Limitations

One limitation of our study is that it is not a true experiment, as there is no overlap between the commenting regimes; it is possible that other factors explain the variation in user behaviour. One such factor might be the electoral calendar, though we do not consider this explanation likely. Second, we know that HuffPo used both manual and algorithmic moderation in all three phases, but we do not know how the policies changed under the different identificatory regimes. It is possible that some of the observed skew towards higher deliberative quality is due to intensive moderation of the politics section during the pseudonymous phase. Future projects may consider working with moderators, and even running experiments on different modes of moderation.

Third, these platforms are not static. Online news platforms and social media experienced rapid changes in their user base and the ways users engage with them during the 2-year period of this study. However, this problem may be limited in so far as our study considers a relatively short period of time, and within that period, we find clear changes in activity coinciding with policy changes. We must also keep in mind the likely cultural specificity of the ways in which both Facebook and online news commenting are used. Just as the character of these platforms changes over time, so too it is different in different parts of the world. Our study is limited to the American context.

A final limitation to keep in mind is that in constructing online discussion spaces, there are many important design features that are likely to influence commenting behaviour, including, but not limited to, whether comments are ordered chronologically, or curated by editors, or upvoted by other users, or some other selection method; whether commenting is real time or asynchronous; and how moderation is performed. We have focused in this study on identity disclosure, but we are aware that it is not the only or even perhaps the most important factor. Moderation practices in particular are hugely important (Grimmelmann, 2015). However, given the low apparent cost of changes in identifiability and their professed popularity among social media platforms and news providers seeking easy ways to sanitize their comment spaces, it is important to explore the dimensions and effects of anonymity, pseudonymity and real-name environments in more detail.

Conclusion

Our findings complicate the idea that there is a relatively straightforward trade-off between the benefits and drawbacks of anonymous commenting, on one hand, and the benefits and drawbacks of real-name environments, on the other. When it is said that commenters ought not to be anonymous, we argue that this does not necessarily mean they must use their real names or connect more fully to their real-world identities; rather, it means that commenting spaces should be structured so as to support the durability of identities, which can at least open the possibility of greater commitment to the commenting space and enhance the potential for learning effects. By emphasizing the distinct qualities of durable pseudonymity, our results suggest an important consideration in the design of online discussion spaces.

We conclude with a brief comment on an aspect of the relation of identity rules and deliberation that strikes us as extremely important even though our particular study has not been able to address it. We did not give ‘traceability’ a central place in our analysis because it seemed that the capacity of commercial entities or governments to trace particular users could apply to users in all of the phases of our study, and especially in the ‘durable pseudonym’ and ‘real-name’ phases, in which the identity of users is guarded by platforms. However, we think that this feature of the infrastructure of the public sphere – that platforms increasingly hold the effective power over the concealment and revelation of identity – is of great importance. Thus, while we hope to have contributed to a better understanding of the relation between identity rules and deliberative quality in a particular online forum, there remain a set of larger unanswered questions: What relations of power are enabled and constrained by practices of anonymity? Who has effective control over the conditions of concealment, revelation or construction of identity? These are, we think, crucial further questions if we are to better understand the many dimensions of the relationship between anonymity and public deliberation.

Footnotes

Acknowledgements

Particular thanks are due to John Naughton, who initiated the project on anonymity and online commenting from which this paper has grown, and to the Conspiracy and Democracy research project at CRASSH, University of Cambridge.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This research was supported by Leverhulme (grant no. RP2012-CO17).

ORCID iD

Alfred Moore

Notes

Author Biographies

Alfred Moore is a lecturer in the Department of Politics at the University of York. He works on contemporary political theory, with a particular interest in democratic theory. He has published in the Journal of Political Philosophy, Political Studies, Critical Review, Episteme, Economy and Society and Social Studies of Science, among others, and he is the author of Critical Elitism: Deliberation, Democracy, and the Politics of Expertise (2017, Cambridge University Press).

Rolf Fredheim is the principal scientist at the Technical and Scientific Development Branch of the NATO Strategic Communications Centre of Excellence in Riga, Latvia. Rolf leads the Centre’s data science research into malicious activity by bots and trolls on social media. Previously, he worked as a Research Associate at the University of Cambridge. He has published in Journal of Computational Social Science, Europe-Asia Studies, Politics, and Journal of Communication.

Dominik Wyss is a quantitative researcher in the Social Science Department at the University of Stuttgart. Much of his work has been in the field of political communication and deliberation with a special focus on development and assessment of digital democratic innovations.

Simon Beste is a researcher in the Methods of Political and Social Science Department at the Goethe University Frankfurt, specializing in political communication, discourse analysis and deliberation. His research particularly centres around empirical research on deliberative systems.

References

Abe

JAA

(2011) Changes in Alan Greenspan’s Language Use across the Economic Cycle: A Text Analysis of His Testimonies and Speeches. Journal of Language and Social Psychology 3 (2): 212–223.

Abe

JAA

(2012) Cognitive–Affective Styles Associated with Position on War. Journal of Language and Social Psychology 31 (2): 212–222.

Allen

(2010) Anonymous: On Silence and the Public Sphere. In: Sarat

(ed.) Speech and Silence in American Law. Cambridge: Cambridge University Press, pp.106–133.

Asenbaum

(2018) Anonymity and Democracy: Absence as Presence in the Public Sphere. American Political Science Review 112 (3): 1–14.

Bächtiger

Niemeyer

Neblo

, et al. (2010) Disentangling Diversity in Deliberative Democracy: Competing Theories, Their Blind Spots and Complementarities. Journal of Political Philosophy 18 (1): 32–63.

Bakshy

Messing

Adamic

(2015) Exposure to Ideologically Diverse News and Opinion on Facebook. Science 348 (6239): 1130–1132.

Beste

Wyss

(2014) A Psychological Proxy for Deliberation? A Theoretical and an Empirical Exploration of the Conceptual Boundaries of Cognitive Complexity and Deliberative Quality. In: Proceedings of the ECPR general conference, Glasgow, 3–5 September.

Black

(2011) The Promise and Problems of Online Deliberation. Dayton, OH: Charles F. Kettering Foundation

Bowker

Tuffin

(2003) Dicing with Deception: People with Disabilities’ Strategies for Managing Safety and Identity Online. Journal of Computer-Mediated Communication 8 (2): 00209.

10.

Brundidge

Scott

Choi

, et al. (2014) The ‘Deliberative Digital Divide’: Opinion Leadership and Integrative Complexity in the U.S. Political Blogosphere. Political Psychology 36 (6): 741–755.

11.

Chun

(2015) The Dangers of Transparent Friends: Crossing the Public and Intimate Spheres. In: Allen

Light

(eds) From Voice to Influence: Understanding Citizenship in a Digital Age. Chicago, IL: The University of Chicago Press, pp.105–128.

12.

Coleman

Moss

(2012) Under Construction: The Field of Online Deliberation Research. Journal of Information Technology & Politics 9 (1): 1–15.

13.

Davies

Chandler

(2011) Online Deliberation Design: Choices, Criteria, and Evidence. In: Nabatchi

Gastil

Weiksner

Leighninger

(eds) Democracy in Motion: Evaluating the Practice and Impact of Deliberative Civic Engagement. Oxford: Oxford University Press, pp.103–131.

14.

Foxman

Wolf

(2013) Viral Hate: Containing Its Spread on the Internet. New York: Palgrave Macmillan.

15.

Fredheim

Moore

Naughton

(2015) Anonymity and Online Commenting: the Broken Windows Effect and the End of Drive-By Commenting. In: Proceedings of the ACM web science conference, Oxford, 28 June–1 July.

16.

Friess

Eilders

(2015) A Systematic Review of Online Deliberation Research. Policy and Internet 7 (3): 319–339.

17.

Froomkin

(1995) Anonymity and Its Enmities. Journal of Online Law 4: 5621.

18.

Gastil

(2008) Political Communication and Deliberation. Thousand Oaks, CA: SAGE Publishing.

19.

Gerhards

(1997) Diskursive versus liberale Öffentlichkeit. Eine empirische Auseinandersetzung mit Jürgen Habermas. Kölner Zeitschrift für Soziologie und Sozialpsychologie 49 (1): 1–34.

20.

Grimmelmann

(2015) The Virtues of Moderation. Yale Journal of Law and Technology 17: 42–109.

21.

Gruenfeld

(1995) Status, Ideology, and Integrative Complexity on the U.S. Supreme Court: Rethinking the Politics of Political Decision Making. Journal of Personality and Social Psychology 68 (1): 5–20.

22.

Habermas

(1996) Between Facts and Norms: A Discourse Theory of Law and Democracy. Cambridge: Polity Press.

23.

Hille

Bakker

(2014) Engaging the Social News User. Journalism Practice 8 (5): 563–572.

24.

Janssen

Kies

(2005) Online Forums and Deliberative Democracy. Acta Politica 40: 317–335.

25.

Jennstål

(2019) Deliberation and Complexity of Thinking. Using the Integrative Complexity Scale to Assess the Deliberative Quality of Minipublics. Swiss Political Science Review 25: 64–83.

26.

Kim

(2006) The Impact of Internet Use Patterns on Political Engagement: A Focus on Online Deliberation and Virtual Social Capital. Information Polity 11 (1): 35–49.

27.

Kuran

(1997) Private Truths, Public Lies: The Social Consequences of Preference Falsification. Cambridge, MA: Harvard University Press.

28.

Lea

Spears

de Groot

(2001) Knowing Me, Knowing You: Anonymity Effects on Social Identity Processes within Groups. Personality and Social Psychology Bulletin 27 (5): 526–537.

29.

Leshed

(2009) Silencing the Clatter: Removing Anonymity from a Corporate Online Community. In: Davies

Gangadharan

(eds) Online Deliberation: Design, Research, and Practice. Stanford, CA: CSLI Publications, pp.223–232.

30.

Levmore

(1996) The Anonymity Tool. University of Pennsylvania Law Review 144: 2191–2236.

31.

Manin

(2005) Democratic Deliberation: Why We Should Promote Debate Rather Than Discussion. Paper Delivered at Program in Ethics and Public Affairs Seminar, Princeton University, Princeton, NJ, 13 October.

32.

Manosevitch

Walker

(2009) Reader Comments to Online Opinion Journalism: A Space of Public Deliberation. In: Proceedings of the 10th international symposium on online journalism, Austin, TX, 17–18 April.

33.

Manosevitch

Steinfeld

Lev-On

(2014) Promoting Online Deliberation Quality: Cognitive Cues Matter. Information, Communication, and Society 17 (10): 1177–1195.

34.

Moore

(2018) Anonymity, Pseudonymity, and Deliberation: Why Not Everything Should Be Connected. Journal of Political Philosophy 26 (2): 169–192.

35.

Nissenbaum

(1999) The Meaning of Anonymity in an Information Age. The Information Society 15 (2): 141–144.

36.

Nussbaum

(2010) Objectification and Internet Misogyny. In: Levmore

Nussbaum

(eds) The Offensive Internet: Privacy, Speech, and Reputation. Cambridge, MA: Harvard University, pp.68–87.

37.

Owens

Wedeking

(2011) Justices and Legal Clarity: Analyzing the Complexity of U.S. Supreme Court Opinions. Law & Society Review 45: 1027–1061.

38.

Polat

Pratchett

(2009) E-Citizenship: Reconstructing the Public Online. In: Durose

Greasley

Richardson

(eds) Changing Local Governance, Changing Citizens. Bristol: Policy Press, pp.193–210.

39.

Reagle

(2015) Reading the Comments: Likers, Haters, and Manipulators at the Bottom of the Web. Cambridge, MA: MIT Press.

40.

Reicher

Spears

Postmes

(1995) A Social Identity Model of Deindividuation Phenomena. European Review of Social Psychology 6 (1): 161–198.

41.

Resnick

Friedman

(2001) The Social Cost of Cheap Pseudonyms. Journal of Economics & Management Strategy 10 (2): 173–199.

42.

Rhee

Kim

(2009) Deliberation on the Net: Lessons from a Field Experiment. In: Davies

Gangadharan

(eds) Online Deliberation: Design, Research, and Practice. Stanford, CA: CSLI Publications, pp.223–232.

43.

Rossini

(2018) Does It Matter If It’s Uncivil? Conceptualizing Uncivil and Intolerant Discourse in Online Political Talk. In: Paper presented at the 68th annual conference of the international communication association (ICA), Prague, 24–28 May.

44.

Rowe

(2014) Civility 2.0: A Comparative Analysis of Incivility in Online Political Discussion. Information, Communication & Society 18 (2): 121–138.

45.

Santana

(2014) Virtuous or Vitriolic? The Effect of Anonymity on Civility in Online Newspaper Reader Comment Boards. Journalism Practice 8 (1): 18–33.

46.

Saslow

McCoy

van der Löwe

, et al. (2014) Speaking under Pressure: Low Linguistic Complexity Is Linked to High Physiological and Emotional Stress Reactivity. Psychophysiology 51: 257–266.

47.

Schroder

Driver

Streufert

(1967) Human Information Processing. Individuals and Groups Functioning in Complex Social Situations. New York: Holt, Rinehart and Winston.

48.

Sia

Tan

BCY

Wei

(2002) Group Polarization and Computer-Mediated Communications: Effects of Communication Cues, Social Presence, and Anonymity. Information Systems Research 13 (1): 70–90.

49.

Steiner

Bächtiger

Spörndli

, et al. (2004) Deliberative Politics in Action: Analyzing Parliamentary Discourse. Cambridge: Cambridge University Press.

50.

Stromer-Galley

(2007) Measuring Deliberation’s Content: A Coding Scheme. Journal of Public Deliberation 3 (1): 12.

51.

Stroud

Van Duyn

Alizor

, et al. (2017) Comment Section Survey Across 20 News Sites. Report published by the Center for Media Engagement, University of Texas at Austin. Available at: https://mediaengagement.org/wp-content/uploads/2017/01/Comment-Section-Survey-Across-20-News-Sites.pdf (accessed 1 December 2019).

52.

Suedfeld

(2010) The Cognitive Processing of Politics and Politicians: Archival Studies of Conceptual and Integrative Complexity. Journal of Personality 78 (6): 1669–1702.

53.

Suedfeld

Tetlock

Streufert

(1992) Conceptual/integrative Complexity. In: Smith

(ed.) Motivation and Personality: Handbook of Thematic Content Analysis. Cambridge: Cambridge University Press, pp.393–400.

54.

Suler

(2004) The Online Disinhibition Effect. CyberPsychology & Behavior 7 (3): 321–326.

55.

Tausczik

Pennebaker

(2010) The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. Journal of Language and Social Psychology 29 (1): 24–54.

56.

Tetlock

(1983) Cognitive Style and Political Ideology. Journal of Personality and Social Psychology 45: 118–126.

57.

Tetlock

(1984) Cognitive Style and Political Belief Systems in the British House of Commons. Journal of Personality and Social Psychology 46 (2): 365–375.

58.

Towne

Herbsleb

(2012) Design Consideration of Online Deliberation Systems. Journal of Information Technology & Politics 9 (1): 97–115.

59.

Van Hiel

Mervielde

(2003) The Measurement of Cognitive Complexity and Its Relationship With Political Extremism. Political Psychology 24 (4): 781–801.

60.

Véliz

(2019) Online Masquerade: Redesigning the Internet for Free Speech Through the Use of Pseudonyms. Journal of Applied Philosophy 36: 643–658.

61.

Wyss

Beste

Bächtiger

(2015) A Decline in the Quality of Debate? The Evolution of Cognitive Complexity in Swiss Parliamentary Debates on Immigration (1968-2014). Swiss Political Science Review 21: 636–653.

62.

Zarsky

(2004) Thinking Outside the Box: Considering Transparency, Anonymity, and Pseudonymity as Overall Solutions to the Problems of Information Privacy in the Internet Society. University of Miami Law Review 58: 991–1044.