Abstract
Although scholars have learned much about the online information environment in recent years, there is scant evidence on the relationship between pre-existing political network diversity and trust in political content encountered on social media. In this study, I use nationally representative survey data to show that reported political network diversity predicts trust in political information on Facebook and Twitter. I proceed to conduct an original survey experiment that reveals such trust is only partly warranted; social media users in reportedly diverse political networks indiscriminately trust both true and false content more than those in less diverse networks. Given this apparent trade-off, I conduct a second survey experiment to identify the presence of heterogeneous treatment effects of accuracy nudging interventions by network diversity level, but I find none. Collectively, this analysis establishes that those who report more political diversity in their social networks believe more of the information they encounter on social media regardless of its accuracy and suggests that accuracy nudging interventions fail to overcome this difference.
Introduction
The emergence of social media in the early 21st century has changed the way many Americans access political information (Mitchell et al. 2016) at the same time that trust in American journalism is in decline (Brenan 2022). Content on social media is less regulated than content disseminated through traditional media and is often awash in false, unverified, and hyper-polarized information (Barberá 2018; Vosoughi et al. 2018; Wang et al. 2019) that threatens to exacerbate the decline in news media trust. A host of researchers have sought to characterize the effects of this content on political attitudes and behaviors (Allcott and Gentzkow 2017; Arguedas et al. 2022; Müller and Schwarz 2021) and have articulated how social media platforms influence the mix of people and information that users are exposed to (Barberá et al. 2015; Guess 2021; Guess et al. 2020a). However, there has been insufficient research into whether pre-existing social network diversity predicts the amount of trust people place in the political information they encounter once they get there.
In recent years, several studies have explored individual and aggregate predictors of trust in information conveyed through media. Although differences exist across countries, scholars find that news trust tends to reflect trust in a broader array of social features and civic institutions (Hanitzsch et al. 2018; Kalogeropoulos et al. 2019; Lee 2010; Tsfati and Ariely 2014). This research suggests that when trust in pillars of social life and public institutions falter, so does trust in media. This phenomenon is most pronounced in the United States, where the decline in news trust has been especially strong (Hanitzsch et al. 2018), and social trust (Mewes et al. 2021; Putnam 1995) and trust in government (“Public Trust in Government: 1958-2022” 2022) have precipitously declined over the past half century or more.
Despite downward aggregate trajectories, Americans differ in the amount of trust they place in their civic and public institutions and, in turn, information conveyed through media. In this paper, I propose that the political diversity of Americans’ social networks is an important predictor of the amount of trust they place in the political information they encounter online. According to Lupton and Thornton (2017), network diversity is “the extent to which multiple viewpoints are expressed in [an] individual’s discussion network” (588). 1 By extension, political network diversity is the degree to which individuals of both partisan affiliations or holders of different political opinions are present in an individual’s social or online network. For example, individuals whose networks contain a near 50–50% mix of Democrats and Republicans possess high network diversity, whereas those whose networks contain only Democrats have low diversity.
Although scholars are only beginning to learn about political network diversity as a distinct type of political network heterogeneity (Platzman 2023), the political heterogeneity of Americans’ social networks has been shown to predict a wide array of important political outcomes, including attitudes towards partisans (Sumaktoyo 2021), beliefs about others’ policy preferences (Butters and Hare 2022), political knowledge (Hopp et al. 2020), and vote choices (Paulis and Ognibene 2022).
To date, no study has explored the relationship between individuals’ political network diversity and their trust in online political content, but there are good reasons to suspect that a relationship exists. First, scholars have long asserted that there are civic benefits associated with possessing a diverse social network. For example, Putnam et al. (1994) argued that societies rich with social ties forged through formal associations experience better civic outcomes, which engender trust in interconnected social systems and reinforce their subsistence. Granovetter (1973) showed that an individual’s interaction with “weak ties” in diverse networks facilitates pragmatically advantageous information transmission that is unavailable from strong ties alone. More recently, Mutz (2002) showed that network diversity engenders outgroup tolerance and enhances abilities to articulate opposing viewpoints. Similarly, Na (2006) argued that network diversity fosters social trust by virtue of the shared experiences, reciprocity, cooperation, and socialization with dissimilar others it encourages. In sum, there are a variety of sociological and psychological reasons to believe that Americans in diverse political networks are more likely to possess social and institutional trust, which may, in turn, promote trust in content they encounter online.
Alternatively, political network diversity may predict online information trust for reasons that do not depend on trust in other domains. One possibility is that social media users who have diverse networks may be exposed to substantively different political information than those within homogeneous networks. Social media algorithms may present users in diverse networks with content that is more moderate, fact-based, and therefore, trustworthy (Brown et al. 2022). A recent field experiment showed that when Facebook users are exposed to news feeds containing more moderate or mixed political content, they perceive more trust in the information they find on social media (Guess et al. 2023a).
A third possibility is that individuals in more diverse networks are exposed to a larger share of online political information from people and sources they trust, echoing the “two-step flow” theory of persuasive communication (Katz and Lazarsfeld 1955). Conversely, individuals in less diverse networks may be likelier to engage in partisan selection exposure and seek rationales to discredit information from uncongenial sources (Taber and Lodge 2006). This phenomenon may be especially pronounced in the United States, where online news audiences are particularly polarized (Fletcher et al. 2020).
Fourth, Americans in politically diverse networks may encounter a different volume of political news online. Diverse network users may get more incidental exposure to news on social media platforms (Scheffauer et al. 2021), which may influence the proportion of content that is trustworthy. At a minimum, studies of Facebook content exposure confirm that users vary in the amount of political content they encounter (Guess et al. 2023b).
Ultimately, there are several reasons why Americans in diverse political networks may be more likely to place trust in (i.e., believe in the accuracy of) political content they encounter online. One hypothesis proposes a mediation model among political network diversity, trust in social and civic institutions, and trust in online political information. 2 Other hypotheses assert that differences in network composition affect the content of an online user’s experience and the corresponding degree to which they trust information they encounter online. Although this paper is mechanism agnostic (both for scoping purposes and because of challenges in acquiring data to test each mechanism), each hypothesis leverages the idea that an individual’s exposure to a wide set of ideas, values, and belief systems from people they encounter erodes the possibility that they engage in reflexive dismissiveness of content containing unfamiliar, unrelatable, or discordant viewpoints. Well-established intergroup contact theories (Pettigrew et al. 2011; Zajonc 2001) open the door for individuals to find merit—and perhaps truth—in previously unfamiliar views.
In this paper, I evaluate the hypothesis that reported political network diversity—that is, self-professed personal exposure to an array of partisans or political opinions—is associated with trust in political information encountered on social media platforms. First, I use the 2020 American National Election Studies (ANES) Social Media Study to show that diversity predicts trust in online content in a nationally representative panel data sample. Second, I ask if the trust that those in politically diverse networks place in political information is warranted; are they more capable of discerning factual from fictional content or are they indiscriminately more trusting of both true and false news? I conduct and report the results of an original survey experiment that suggests the answer is the latter. Third, I turn to the literature on accuracy nudging interventions designed to reduce the belief in and spread of false news and conduct a second survey experiment designed to estimate the heterogeneous effects of these interventions by network diversity level. In all, my analysis establishes that those who report political diversity in their social networks are more trusting of information they encounter online and that accuracy nudging interventions do not moderate the different baseline levels of trust expressed by those with varying degrees of political diversity.
Network Diversity and Trust: ANES 2020 Social Media Study Analysis
I start my analysis with the ANES 2020 Social Media Study, a two-wave panel survey that was administered shortly before and after the 2020 presidential election.
3
In both waves, the survey asked respondents to report the shares of their personal and “Facebook friend” networks composed of Republicans and Democrats, respectively.
4
For each response choice, respondents were presented with a five-point ordinal scale ranging from “None or almost none” to “All or nearly all.” To compute network diversity, I convert each response to a value between zero and one in quarter-point increments and insert it into the network diversity formula preferred by Lupton and Thornton (2017):
The distributions of values were nearly identical across the four instances of measurement (a personal network and a Facebook network measure in each of two survey waves). In each instance, approximately one-third of respondents reported having equal shares of Republicans and Democrats in their network (the maximum value), and an additional one-third reported having “a lot” of one party and “a few” members of the other (a moderate value). Less than 10% of respondents reported having no diversity at all.
The survey also asked respondents how much they trust political information they encounter on Facebook and Twitter via five-point ordinal scales ranging from “Not at all” to “A great deal.” I convert these responses to a zero-to-one scale in quarter-point increments. In both survey waves, trust in Facebook and Twitter was exceedingly scarce: approximately 60% of respondents reported not at all trusting political information on these platforms. Additionally, less than 2% of respondents in any of the four measurements reported having “a great deal” or “a lot” of trust. These responses show that although Americans increasingly access political information on social media, they do not claim to believe what they encounter on these platforms. This is consistent with findings in other surveys (Mitchell et al. 2016).
Analysis
To explore the relationship between political network diversity and expressed trust in political information on Facebook and Twitter, I formulate eight multivariate ordinary least squares regression equations. The eight models are defined by the set of unique combinations among the two social media trust measures (for Facebook and Twitter) and two network diversity measures (for personal and Facebook networks) across the two survey waves. Each model contains an identical set of control variables for network, demographic, and political attributes. These include reported network size, gender, partisan identification, political ideology, and other attributes that may correlate to network structure or to information trust. Outputs from all eight models are reported in Appendix Tables C.1 and C.2.
Each of the eight coefficient estimates of political network diversity’s relationship to trust, in addition to two “pooled” precision-weighted estimates computed via fixed-effects meta-analysis, appears in the top panel of Figure 1. All eight unpooled estimates are positive, though only two are statistically significant. However, the pooled estimates for both outcomes leverage the increased sample sizes to render more precise, and statistically significant, positive estimates. On average, those who reported the most network diversity—that is, those whose networks are reportedly comprised of half Democrats and half Republicans—expressed 1.4 and 1.3 percentage points more trust in Facebook and Twitter, respectively, as a source of political information than those who reported the least network diversity. ANES 2020 pooled and unpooled OLS coefficient estimates for diversity on Facebook trust and Twitter trust within cross-sectional data (top) and change in diversity on change in Facebook trust and change in Twitter trust among first-differenced panel data (bottom). 95% confidence intervals for pooled estimates are displayed.
More compelling evidence of the relationship between reported network diversity and expressed trust is revealed by leveraging the longitudinal nature of the ANES data. Appendix Tables C.3 and C.4 report the results of four first-differenced OLS regression models that estimate the relationship between change in diversity and change in trust within subjects over time, rather than differences between subjects at single points in time. 5 The bottom panel of Figure 1 displays these estimates and a pooled estimate for each outcome variable. Across the four estimates, respondents who reported more network diversity in the second survey wave than the first also reported more trust in Facebook or Twitter in the second wave. Both pooled estimates and three of the four unpooled estimates are statistically significant.
In all, the estimates in Figure 1 indicate a small but positive association between reported network diversity and expressed trust in political information on Facebook and Twitter across a variety of measurement opportunities. Not only does the relationship present at single points in time, but within-subjects changes in reported diversity over time predict within-subjects changes in reported trust.
The Duality of Trust: Results from an Original Survey Experiment
The 2020 ANES data portrays a relationship between Americans’ reported network diversity and the trust they place in political content on social media. In this section, I broach two follow-up questions. First, it is one thing to claim to trust an information source in the abstract and another to display trust in particular instances. Do respondents in politically diverse networks indeed express more trust in political information when presented with specific examples, or do they respond with as much skepticism as those who inhabit less diverse networks, despite what the ANES findings suggest? Second, if they do express more trust in political content on social media platforms, is this trust warranted? Given recent concerns about the proliferation of misinformation on social media platforms, might individuals in diverse networks be more prone to mistaking untruthful content for fact?
To explore these questions, I conducted a pre-registered survey experiment in which I presented respondents with examples of political content in the form of contrived Facebook posts containing news article headlines and images. After the presentation of each headline, I asked respondents to rate how accurate they thought each headline was and how likely they would be to share it on social media. I found that the more respondents reported diversity in their social networks, the more likely they were to perceive accuracy in and express an intention to share headlines, and that this relationship appeared to pertain regardless of whether the headline was true or false.
Research Design and Measurement
On April 20–22, 2023, I fielded an original “pre-test” survey experiment to U.S. adults via the Lucid Theorem platform and collected 550 responses. Lucid uses a quota sampling procedure to match survey samples to U.S. population benchmarks according to age, gender, race, and region, which Coppock and McClellan (2019) found to produce suitably representative samples. 6 Respondents were first asked to report their partisanship in the standard two-question sequence that produces a seven-point scale ranging from “Strong Democrat” to “Strong Republican” before being asked a binary question about their political discussions: “During the past six months, did you talk with anyone face-to-face, on the phone, online, or in any other way about government, politics, or elections, or did you not do this with anyone during the past six months?” 7 Respondents who reported not having had a political discussion during the past six months were coded as not possessing a political discussion network.
A random subset of those who reported having had a political discussion proceeded to an original question sequence that measured respondents’ perceptions of their political discussion network. The first question in the sequence asked respondents to report the number of people they discussed “government, politics, or elections” with in the past six months. The second question asked respondents to report the amount of disagreement they perceive they have with members of their network: “Think about the people you discussed government, politics, or elections with during the past six months. In general, how different are their opinions about government, politics, or elections from your own?” The third question asked respondents to report the amount of diversity they perceive among the political opinions in their network: “Think about the people you discussed government, politics, or elections with during the past six months. In general, how different are their opinions about government, politics, or elections from one another’s?” Respondents were presented with five answer choices to each of the second and third questions, ranging from “Extremely different” to “Not different at all.” Responses to these questions were normalized such that they ranged from zero to one.
Following network measurements, the main treatment sequence consisted of a series of four mock Facebook posts drawn from a pool of 12 that each contained a political news headline, an accompanying image, and stylistic surroundings designed to give the respondent the impression of a Facebook news feed setting (see an example post in Figure 2). The pool of 12 posts was equally divided between those that contained true and false headlines and those that were favorable to the Democratic and Republican parties. Thus, the pool contained three true, pro-GOP headlines; three true, pro-Democratic headlines; three false, pro-GOP headlines; and three false, pro-Democratic headlines. Each respondent received a randomly selected example from each category in a random order. Headlines were adapted from claims that appeared on Snopes.com, a political fact-checking website, and the truth value attributed to each headline corresponded to the claim’s rating on Snopes.com. The images that accompanied the headlines were sometimes found on the Snopes.com webpage associated with the claim and were sometimes found elsewhere on the web.
8
This contrived Facebook post contains one of the three examples of a true, pro-GOP headline that was presented to respondents in the April 2023 “pre-test” survey experiment.
Beneath each post, respondents were asked how accurate they perceive the claim in the headline to be and how likely they would be to share the article on social media. 9 The order of these questions was randomly determined for each respondent but consistent across each of the four posts. This design convention, which asks about both accuracy perception and sharing intentions, follows the example set by recent papers (Pennycook et al. 2020a; Pennycook et al. 2020b; Roozenbeek et al. 2021) and gains leverage over reported news trust and a behavioral indicator of news trust. Although self-reported behavioral intentions are a step removed from direct behavioral measurements, Mosleh et al. (2020) found that expressed sharing intention in survey responses is a useful predictor of actual sharing behavior on Twitter.
Although this design does not attempt to experimentally manipulate respondents’ political network diversity values, and the analysis of the relationship between network diversity and social media trust that results from this study must be interpreted as observational, this design provides a different test of the relationship between diversity and trust than the one afforded by the 2020 ANES analysis. This study’s measure of network diversity concerns political opinion differences rather than partisan affiliation differences and construes values directly rather than indirectly. The outcome measures are also more revealing. Rather than generic expressions of trust in Facebook or Twitter as political information sources, respondents assess the veracity of specific political news headlines and report how likely they would be to share them. 10
Analysis
Approximately two-thirds of respondents reported having talked about politics with another person during the past six months (the one-third who did not are unable to be assigned a network diversity value and are excluded from analysis). Like the network diversity measures derived from 2020 ANES responses, the modal diversity value is the middle option in the response scale; one-third of respondents reported that the people they had discussed politics with hold “moderately different” views from one another. Unlike the 2020 ANES responses, only about one-eighth of respondents reported observing “extremely different” political opinions (the maximum value) in their network.
Regarding accuracy perception and sharing intentions of Facebook post headlines, respondents were more likely to perceive true news to be accurate than false news (mean values were 0.55 and 0.43, respectively, on a one-point scale). This difference is compared via a two-sample t-test and is statistically significant (p < 0.001). Respondents were also more likely to express an intention to share true news than false news (0.38 versus 0.34), but this difference is depressed by the 22% of respondents who reported being “extremely unlikely” to share all four articles they were presented with. Accuracy perception and sharing intentions are also strongly and positively correlated with each other (r = 0.60). Together, these findings are encouraging indications of respondent attentiveness to survey treatments.
I evaluate the relationships between reported network diversity and headline accuracy perception and sharing intentions via multivariate OLS regressions. Following the design employed by Pennycook et al. (2020a), Clayton et al. (2020), and others, I use the response, rather than the respondent, as the unit of analysis and cluster the standard errors by respondent (see Judd et al. 2012 for an articulation of the bias inherent in ignoring the randomizations of both respondent treatment assignment and treatment stimulus selection). I fit six total regression equations: one each for responses to true, false, and all headlines for each of the two outcome variables (accuracy perception and sharing intentions). 11 In each equation, I include a battery of control variables, including network size, network disagreement, and the demographic and political attributes I have access to that might correlate to diversity or either outcome measure. I also include control variables for the randomized order of the accuracy perception and sharing intention questions and the truth value of each headline in regressions that contain responses to both true and false headlines. All variables are normalized such that they range from zero to one.
Figure 3 plots the network diversity coefficients for each of the six fitted models. Across all political headlines, reported network diversity is positively associated with both increased accuracy perception and sharing intentions. Specifically, those who report “extremely different” political opinions in their discussion networks perceive, on average, one-quarter of the length of the accuracy perception scale more accuracy than those who report no opinion differences within their networks. They also report an average of two responses higher on the five-point sharing intention scale than those who report the least diversity. Both estimates are considerably larger than those of the 2020 ANES analysis. Pre-test diversity estimates on accuracy perception and sharing intentions for true, false, and all political headlines. Standard errors are clustered by respondent. 95% confidence intervals for total headline estimates are displayed.
Differences across diversity levels appear to exist irrespective of the truth value of headlines. When analysis is restricted to headlines that contain truthful content, those who report the most network diversity perceive significantly more accuracy and are significantly more likely to share them than those who report the least diversity. Among false content, those who report the most diversity are significantly more likely to share headlines and perceive more accuracy, though not significantly more accuracy, than those who report the least diversity. 12
The pre-test findings are remarkable for two reasons. First, they corroborate the central finding of the 2020 ANES analysis by demonstrating a positive relationship between reported network diversity and trust in political content on social media under markedly different survey conditions. Not only do survey respondents who have more diverse compositions of partisans in their networks report more generic expressions of trust in political information on Facebook and Twitter, but those who report more opinion diversity in their networks perceive more accuracy in specific political headlines in mock Facebook posts. Second, they suggest that the heightened trust exhibited by those in reportedly more diverse networks may not reflect enhanced responsiveness to the underlying truth value of political content. Instead, those who report more diversity are more likely to trust and share political content that is either true or false.
Although scholars tend to frame the decline in news trust in the United States strictly in negative terms, these results portray enhanced trust as a double-edged sword. Trust may lead to greater receptivity to factual content, but it may also lead to the acceptance and redistribution of false content that would be rejected by an informed public. Given Americans’ increasing exposure to online political content, this trade-off should be acknowledged. Scholars may be wise to identify how to retain the benefits of increased trust without incurring as much of its cost. I direct my attention to this challenge in the next section.
Network Diversity and Fake News Reduction Interventions: The Search for Heterogeneous Treatment Effects
After concerns about the proliferation of online fake news began to materialize, scholars began to explore the efficacy of interventions deployed by fact-checking organizations and social media companies to reduce the belief in and sharing of false content (Clayton et al. 2020; Guess et al. 2020b; Pennycook et al. 2020b; Roozenbeek et al. 2021). This research has shown that accuracy nudges, in the form of fact-checking labels applied to specific news stories or generalized inducements to reflect on the veracity of content, reduce the sharing of false information online.
However, these interventions also produce unintended consequences. Clayton et al. (2020) found that generalized warning messages about the existence of false information online reduced trust in factual content. Pennycook et al. (2020a) identified the presence of the “implied truth effect,” which occurs when some false stories are labeled false, other false stories that are unlabeled are more likely to be perceived to be true because people assume that unlabeled stories have already been checked and deemed accurate. Given that many more online political stories are unlabeled than labeled, this phenomenon presents a serious threat to the overall effectiveness of fact-checking campaigns.
Understanding these trade-offs, entities that deploy accuracy nudging interventions may seek to optimize their delivery to users who are likeliest to reduce belief in and sharing of false information without generating much unwarranted trust in equally dubious but unlabeled content. That requires knowledge of who is likeliest to respond to nudging interventions and who is most susceptible to the implied truth effect. Given the apparent relationship between political network diversity and trust in political information on social media, Americans in high diversity networks may be more likely to respond to accuracy nudges but also more likely to fall prey to the implied truth effect. The net effect of exposing high diversity individuals to accuracy nudges, then, may be especially counterproductive.
In this section, I set out to achieve three aims. First, I test the robustness of the pre-test results by repeating the survey experiment on a much larger sample size with additional pre-treatment control variables. Second, I explore the possibility that Americans in more diverse networks are more responsive to accuracy nudging interventions than those in less diverse networks due to their higher baseline levels of social media trust. Third, and for the same reason, I explore the possibility that those in more diverse networks are more likely to fall prey to the implied truth effect.
Research Design and Measurement
To achieve these objectives, I administered a more complex version of the “pre-test” survey experiment to 2,019 respondents between June 3–6, 2023. As before, the survey was distributed to members of the Lucid Theorem panel. In addition to screening for U.S. adults, this pre-registered “main test” required respondents to have reported using social media.
In the pre-treatment phase, respondents were asked to complete a battery of questions about their political attributes. In addition to partisanship, which had been solicited in the pre-test, questions about political ideology, placement knowledge, and political participation that were identical to those that had appeared in the 2020 ANES were introduced because all three had been significantly predictive of social media trust in the 2020 ANES analysis.
Respondents proceeded to a series of questions about the composition of their political discussion networks. As in the pre-test, respondents began with a binary question about whether they had had a political discussion with another person during the past six months. Those who responded affirmatively proceeded to either of two sequences according to random assignment. The first contained measures similar to those that appeared in the 2020 ANES. Specifically, respondents were asked to report the partisan compositions of their “friends and family” network and their “social media connections” network. The other sequence was nearly identical to the original network measurement sequence I introduced in the pre-test, and the specific question about network diversity was unchanged. See Appendix A for more details.
Main Test Experimental Conditions and Sample Sizes.
Frequency table reporting the number of main test respondents randomly assigned to each of six experimental conditions that varied along two treatment dimensions. Across the columns, conditions were defined by the absence or presence of a generalized warning message preceding the presentation of eight mock Facebook posts. Down the rows, conditions were defined by the number and type of fact-checking labels applied to the content within posts.
The six conditions reflected the full set of treatment combinations across two dimensions. Along the first treatment dimension, respondents either received or did not receive a generalized warning message immediately prior to the eight-post series that some of the following headlines may be false and that all deserved a skeptical reading (“Treatment A”). The text of this message was adapted from a version that appeared in Clayton et al. (2020), where this treatment was shown to significantly reduce belief in false headlines. See Appendix Figure E.13 for a portrayal of Treatment A.
The second dimension concerned the placement of fact-checking labels on individual headlines. In a control condition, none of the eight headlines were accompanied by a fact-checking label. In one treatment condition (“Treatment 1”), half of the false news headlines (i.e., two of the eight headlines in the series) contained a fact-checking label that appeared across the top of the image in the post. In another treatment condition (“Treatment 2”), half of the false and half of the true headlines (i.e., four of the eight) were accompanied by fact-checking labels across the top of their images. Labels applied to true headlines read “True Information: Verified by independent fact-checkers” and labels applied to false headlines read “False Information: Checked by independent fact-checkers.” See Figure 4 for examples of posts with fact-checking labels and Appendix E for the full set. See Appendix B for more label details. These contrived Facebook posts presented to respondents in the June 2023 “main test” survey experiment contain fact-checking labels applied to the top of their images. The example on the left contains a fact-checking label that verifies the accuracy of the headline, and the example on the right contains a fact-checking label that disputes the veracity of the headline.
Because respondents in the Treatment 1 condition were only exposed to “false” labels, the truth status of unlabeled posts was ambiguous—were they unlabeled because they had been deemed true or had they simply not been checked? This ambiguity created the condition for the implied truth effect to materialize. By virtue of exposure to both true and false labeled headlines, respondents in the Treatment 2 condition were presented with enough information to infer that unlabeled posts had not been fact checked—if some true headlines had been labeled true, then unlabeled headlines could not reasonably be inferred to have been rated true by default. See Pennycook et al. (2020a) for a demonstration of this configuration’s ability to alleviate the implied truth effect.
This design had many advantages. First, it established continuity with the design utilized in the pre-test. Second, it provided a stronger opportunity to estimate the relationship between reported political network diversity and trust in specific political headlines using a much larger sample than the pre-test with more response opportunities per respondent. Third, by introducing three additional pre-treatment political attributes to use as control variables in regression equations, it reduced the possibility that an observed relationship between diversity and trust would be due to omitted variables bias. Fourth, it provided an opportunity to estimate heterogeneous treatment effects of accuracy nudging interventions by network diversity level. 14
Analysis
For the purposes of maximizing statistical power and streamlining the forthcoming analyses, I create a concatenated diversity measure that combines responses to the ANES-style and whole network diversity measures across randomized treatment conditions. The combined measure takes either the average of the normalized partisan composition scores (see equation (1)) for respondents’ personal and social media connections networks or the value associated with respondents’ networks’ political opinion differences. Like its component scores, the combined measure ranges from zero to one.
The distributions of uncombined diversity measures look like they did in the 2020 ANES and pre-test data. About one-third of responses to the main test ANES-style measures appear at the middle of the diversity spectrum, and the whole network diversity measure contains approximately 60% of its density at the second and third response choices on its five-point scale (these reflect “slight” and “moderate” opinion differences, respectively). The combined measure, by definition, blends these two patterns of responses. Its modal value is located at the middle of the scale, and relatively few responses appear at either extreme (about 10% reported no diversity at all, and about 10% reported a maximal value).
As was true in the pre-test, accuracy perception and sharing intentions are highly correlated (r = 0.67), and both outcomes are responsive to the underlying truth value of news content. The difference between mean accuracy perception of true and false content is statistically significant (0.53 versus 0.38, p < 0.001), as is the difference between true and false content sharing intentions (0.33 versus 0.29, p < 0.001).
I conduct the main analyses via a series of multivariate OLS regressions. As in the pre-test, regression models use the response as the unit of analysis and cluster standard errors by respondent. Model equations contain either accuracy perception or sharing intentions as outcome variables regressed on reported network diversity, binary indicators for each of the three treatment variables, and a set of control variables. In more expansive model versions, I include interaction terms between diversity and each of the three treatment indicators to estimate heterogeneous treatment effects. Thus, the simpler model form (as specified by equation (2)) is nested within the more expansive form (as specified by equation (3)):
The list of control variables consists of network disagreement and size, demographic information contained within the Lucid panelist background data, four pre-treatment political attributes, the randomized order of the accuracy perception and sharing intention questions, the truth value of a headline in cases where both true and false headlines are present, and an indicator of whether a headline is labeled in cases where both labeled and unlabeled headlines are present.
Figure 5 displays the diversity coefficient estimates for three versions of the model specified in equation (2)—one for all responses, one for responses to true headlines, and one for responses to false headlines—for each of the two outcome variables. Consistent with the findings in the 2020 ANES and pre-test analyses, Figure 5 shows that reported network diversity is positively and significantly associated with political headline accuracy perception and sharing intentions. Regarding accuracy perception, the relative magnitudes of true and false news estimates are inverted compared to those in the pre-test. In the main test, the estimated relationship between diversity and false headline accuracy perception is greater than that of diversity and true headline accuracy perception, and only the estimate regarding false headlines exceeds the threshold of statistical significance. Regarding sharing intentions, the estimates associated with true and false headlines are similar. All six diversity coefficient estimates are of much lower magnitudes than those of the pre-test, suggesting that the three additional control variables introduced into the main test equations are influential. Nevertheless, diversity is a significant predictor of both outcomes.
15
Main test diversity estimates on accuracy perception and sharing intentions for true, false, and all political headlines. Standard errors are clustered by respondent. 95% confidence intervals for total headline estimates are displayed.
Finally, I turn to the evaluation of potential heterogeneous treatment effects by network diversity level. In this study and in others, accuracy nudging interventions altered social media users’ perceptions of the political content they encounter online (see Appendix D for a presentation and discussion of this study’s experimental treatment effect estimates). Might they produce varying degrees of responsiveness to those who vary along the network diversity continuum, a dimension that is significantly predictive of social media news trust? The main test pre-analysis plan articulates an exhaustive series of two-sided hypotheses for the interaction effects between reported network diversity and each of the three treatment interventions for various types of headlines. Hypotheses are two-sided because, on the one hand, those who report more diversity perceive higher average baseline levels of accuracy and may have more trust to lose if notified that content is false. On the other hand, their higher baseline levels may suggest that they display a reduced propensity to abandon trust even when given reason to, consistent with theories of motivated reasoning (Flynn et al. 2017; Van Bavel and Pereira 2018). Analogously, those with more network diversity may be especially responsive to verification that a headline is true. On the other hand, they may not be as moved by a verification label because of their higher baseline levels, particularly if they encounter ceiling effects.
F-Test Results Comparing Nested Models to Models with Interaction Effects (Main Test).
Statistical Significance Indicators: ∗p < 0.05; ∗∗p < 0.01; ∗∗∗p < 0.001
F-test results for each of 14 model comparisons using main test data. No test produced a statistically significant F-statistic.
To illustrate the lack of heterogeneity in more detail, Figure 6 presents marginal effects plots of reported diversity on accuracy perception across a variety of treatment conditions in which experimental treatment effects were most apparent (see Appendix Figure D.1). Each plot shows the predicted accuracy perception (on a zero-to-one scale) for those who reported no political network diversity (solid line) and those who reported maximal network diversity (dashed line) when other covariates are held at their median or modal values. In any given plot, the dashed line tends to appear above the solid line, reflecting the propensity of respondents who report maximal diversity to perceive more headline accuracy than respondents who report no diversity, and the slopes of the two lines tend to tilt in the same direction, reflecting the effects of the experimental treatment. For example, headlines labeled false in the Treatment 2 condition were perceived as less accurate by both high diversity and low diversity respondents than when they were unlabeled in the control (no labels) condition, but high diversity respondents were more likely to perceive headline accuracy in both experimental conditions. Main test marginal effect estimates of reported network diversity on accuracy perception by labeled treatment condition. Estimates are derived from models containing interaction effects (see equation (3)) in which other covariates are held at their median or modal values. 95% confidence intervals are displayed.
If heterogeneity were to be observed, it would be found in the difference between the two slopes of lines within a given plot. For example, unlabeled false headlines in the Treatment 1 condition were more likely to be perceived as accurate by respondents with no diversity than when they appeared in the control (no labels) condition, reflecting the familiar pattern of the implied truth effect. However, respondents with high diversity were no more likely to perceive unlabeled false headlines as accurate in the treatment condition than the control condition. Nevertheless, the difference in these slopes, like the differences in others panels, fails to rise to the level of statistical significance.
These findings suggest that although diversity and fact-checking labels are both associated with accuracy perception, those who reported the most network diversity responded to fact-checking labels by the same absolute magnitude and in the same direction as those who reported the least diversity. They may have started at higher baseline trust levels, but they generally moved in parallel to those who started at lower levels. This was true whether headlines were true or false and whether they were labeled or unlabeled. As a result, treatment conditions 1 and 2 produced the implied truth effect, but the effect was no more or less pronounced among those who reported different levels of network diversity.
In one sense, these results are reassuring. Although those who report high network diversity are more prone to mistaking false content for fact, they are just as malleable and responsive to declarations about matters of fact as those who perceive false information more accurately. In another sense, the lack of treatment effect heterogeneity is discouraging. For fact-checking organizations looking to optimize the trade-offs inherent to the deployment of accuracy nudging interventions, these results do not illuminate a path forward. If optimization within the paradigm of deploying warning messages and fact-checking labels is achievable, it is with respect to another dimension of variation. 16
Discussion
In this paper, I have explored the possibility that an underappreciated aspect of Americans’ political identities—the perceived diversity of partisan affiliations and political opinions in their social networks—predicts their likelihood of believing online political content. The evidence suggests that it does. I have found that reported political network diversity is positively associated with perceived accuracy of political information on social media in three different survey settings using different types of diversity and outcome measures and a variety of modeling configurations.
A skeptic may argue that the importance of this relationship would be negligible if the share of Americans who possess diverse political networks were small. Contrary to popular perception, this paper has shown that partisan and opinion diversity are commonplace in American social networks and only a tiny fraction of Americans reports no diversity in their networks at all. The most common network composition possesses a moderate degree of diversity, in which one political party or opinion is more frequently represented than another, but a minority contingent is present.
Having demonstrated the prevalence of diversity and the existence of a relationship between diversity and online news trust, I asked whether the increased trust displayed by those who reported high diversity is warranted. Across two survey experiments, I found disconcerting evidence that respondents who reported more network diversity may be likelier to perceive either true or false content to be accurate. This discouragingly suggests that occupants of diverse networks are no better equipped than others to wade through political content at a time when concerns about the quality of online information abound.
Finally, I evaluated the possibility that fake news reduction interventions have disproportionate effects on Americans in politically diverse networks. Using an original survey experiment that collected nearly 20,000 reactions to political news headlines, I found that interventions designed to reduce belief in and sharing of false news did not moderate the relationships between reported network diversity and these outcomes. This suggests that Americans in high diversity networks are no more or less prone to adhering to nudging interventions despite possessing higher baseline levels of trust.
Heterogeneity aside, it is worth noting that the estimated effect sizes of warning messages and fact-checking labels on accuracy perception and sharing intentions are less than the estimated coefficients associated with network diversity. This suggests that an individual background characteristic that is not intuitively related to news perception is more strongly associated with the perceptions of particular headlines than the presence of labels that authoritatively declare headlines to be true or false. That is a remarkable testament to the relatively strong influence that a historically unmeasured feature of social identity has over a problem that has attracted considerable scholarly and public attention and to the relatively marginal role that accuracy nudging interventions play in addressing it.
These results should not discourage the search for an optimal deployment of accuracy nudging interventions; other subsets of users may respond more favorably to nudging interventions net of anticipated second-order effects. But the nature of the interventions may need to be expanded. Because network diversity is demonstrably related to trust in both true and false content, an intervention designed to prime users’ awareness of their network’s diversity before delivering accurate content, but not before warning of inaccurate content, may be worthwhile. Another option is to leverage the known benefits of source credibility (Pornpitakpan 2004) by attributing warning messages or labels to trusted sources or showcasing a trusted third party who relies on interventions in their own assessments of online content.
As these strategies imply, enhancing news trust indiscriminately should not be the preferred approach to addressing sagging news trust. Instead, democratic polities seek to enhance public trust in credible information and skepticism of misinformation. Possessing network diversity is neither inherently helpful nor harmful in realizing this objective; diversity appears to come with a trade-off between heightened trust in verified content and exaggerated trust in false content. Harnessing the benefits of network diversity requires careful management.
Despite a recent emphasis on the interplay between network features and political attributes, there is still much to learn about the different aspects of network heterogeneity, and measures of diversity need refinement. Throughout this paper, I have taken Americans’ reported levels of network diversity at face value, but whether and to what extent their reported perceptions of network features are representative of their networks’ full characters needs to be further explored. 17 So do the people and political opinions they call to mind when evaluating network attributes. Although nationally representative surveys have increasingly included questions about respondents’ social networks in recent years, this level of insight needs to come from tailored data collection efforts designed specifically for these purposes.
To conclude, scholars are concerned that the public’s migration to online news environments hampers its ability to make discerning choices in public affairs (Snow and Vaccarezza 2021). This paper finds that the scope of this threat may be as dependent on the political compositions of Americans’ social networks as it is on the nature of the content that Americans are exposed to in this new digital domain.
Supplemental Material
Supplemental Material - Political Network Diversity and Trust in Online Political Content
Supplemental Material for Political Network Diversity and Trust in Online Political Content by Paul B. Platzman in Political Research Quarterly
Footnotes
Acknowledgments
I thank the reviewers who provided comments on earlier drafts of this paper at Columbia University, the Southern Political Science Association 2024 Annual Conference, and Political Research Quarterly.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
I thank Columbia University’s political science department for supporting this research with a dissertation development grant.
Ethical Statement
Data Availability Statement
ANES Social Media Study panel data is available publicly on the ANES website. “Pre-test” and “main test” original survey data are available at https://osf.io/v4hwg and
, respectively.
Supplemental Material
Supplemental material for this article is available online at the Political Research Quarterly website.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
