Abstract
This study examines audience relationships to fact-checking sites in the United States. Focus is placed on predictors of audience awareness of, attitudes toward, and visits to such sites within a stage model framework drawn from the persuasive message literature. Analysis of survey data from a U.S. sample shows that liberals and liberal/mainstream news consumers are more aware of, positive toward, and likely to report using fact-checking sites. Conservatives are less positive and conservative news consumers see such sites as less useful to them. Findings indicate that while specific combinations of predictors of awareness, attitudes, and behavior vary, fact-checking sites have a particular appeal to liberals and liberal/mainstream news consumers. Results point to U.S. fact-checking sites being absorbed into wider ideological discourses and patterns of ideological news consumption.
Introduction
“People are always going to get Pinocchios for this stuff,” Mitt Romney’s pollster Neil Newhouse said during the 2012 presidential campaign. “Fact-checkers come to this with their own sets of thoughts and beliefs and you know what? We’re not going let our campaign be dictated by fact-checkers” (Simmons 2012). Although checking facts has always been a core journalistic practice, this quote from the Romney camp in 2012 reveals the tension that emerged when dedicated U.S. fact-checking organizations started gaining momentum in the 2010s. After all, the type of fact-checking conducted by sites like FactCheck.org and PolitiFact was different from traditional journalism. In this brand of reporting, fact-checkers adjudicate factual disputes following a procedure that resembles the scientific method (Coddington et al. 2014), decide the veracity of political claims (Graves 2018), and publicize corrections through forums such as social media (Graves 2016).
While the practice of fact-checking has since spread worldwide, following the U.S. example (Graves 2018), scholars and audiences have critiqued fact-checkers and their ability to determine “truth” (Brandtzaeg et al. 2018; Lim 2018; Uscinski and Butler 2013). Nevertheless, fact-checking has become a key feature of U.S. presidential campaigns and sites have found their place in the media ecosystem (Graves 2016; Lowrey 2017).
This study investigates audience relationships with fact-checking through a survey fielded after the election of President Trump in 2016. We build on literature from journalism studies and persuasion to investigate predictors of three different outcomes: awareness of fact-checking organizations, attitudes toward them, and frequency of visiting these sites.
Our main guide to theorizing is the stage model approach to how persuasive messages work (e.g., Lavidge and Steiner 1961; McGuire 1968); relevant because fact-checking is, at its core, about persuasive messages intended to correct the beliefs of readers (Amazeen 2019). Stage models suggest that people first become aware of fact-checking sites. Once aware, they develop attitudes toward them. After becoming aware and developing attitudes, they may visit the sites (behavior). While these stages have been conceptualized as a hierarchy, there is disagreement as to whether they occur in a fixed order (Barry and Howard 1990), with attitudes potentially preceding use, or vice versa. Significant to the present approach, each stage tends to be predicted by different sets of antecedents (Barry and Howard 1990).
Fact-Checking and Its Mission
Although the practice of checking facts has long been present in journalism, the establishment of dedicated fact-checking organizations is only a recent phenomenon driven forward by prominent outlets such as PolitiFact and FactCheck.org (Graves 2016, 2018). Since the mid-2000s, the number of political fact-checkers in the United States and elsewhere has risen (Amazeen 2017; Graves 2018; Graves and Glaisyer 2012), with dedicated sites becoming a “new journalistic institution” seeking to “revitalize the ‘truth seeking’ tradition in journalism” (Graves 2016: 6). Following the U.S. example, this form of journalism has spread worldwide, with operations advised by PolitiFact and FactCheck.org being established in South America, Africa, and Europe (Graves 2018). Amazeen (2017: 10) argues that some fact-checking outlets have emerged partly as “democracy-building tools”—correctives to institutional failings—in some nations. They diverge from traditional journalism (Coddington et al. 2014), seeking to educate the public and improve both political behavior and journalism itself (Amazeen 2019; Graves and Glaisyer 2012).
However, the fact-checking movement is not without its detractors, with criticism coming from both the academy (Uscinski and Butler 2013) and the public (Brandtzaeg et al. 2018). Attitudes of the public are important as fact-checkers seek to reach as wide an array of people as possible while also targeting opinion leaders such as politicians and mainstream news outlets, alongside younger people who use social media (Graves 2016; Graves and Glaisyer 2012). Audience perceptions are also crucial given the central role that organic spread of content has for these outlets. While fact-checks have been spread through syndicated partnerships (Amazeen 2013; Graves 2016), emphasis has been placed on social media as a medium through which messages can reach a wider audience (Graves 2016). This is an orientation toward an audience-centered model of information diffusion in a context where the influence of fact-checking on the news media is not large (Vargo et al. 2018) and where news outlets do not frequently incorporate fact-checking into their political coverage (Amazeen 2015).
But who actually pays attention to and visits fact-checking sites? Despite the effort of fact-checkers to reach a wider audience, evidence indicates that people are not that familiar with their work (Nyhan and Reifler 2016), that fact-checks constitute a very small portion of websites visited (Guess et al. 2018), that a limited number of people share fact-checks (Amazeen et al. 2019), and that they are shared selectively on social media for political reasons (Shin and Thorson 2017). In the latter case, fact-checks become part of a pattern of ideological news consumption and sharing, with Democrats sharing fact-checks favorable to them and filtering out those which are not (and vice versa for Republicans). The consequence of this selective use of “identity-relevant media” may be to “further polarize audiences as well as undercut trust in the process of fact-checking” insofar as selectively chosen fact-checks are used as partisan weapons to undercut the “other side” (Shin and Thorson 2017: 250). Finally, some audience members, particularly conservatives, have very negative attitudes toward fact-checkers (Brandtzaeg et al. 2018; Robertson 2019).
Such findings make questions of who is aware of, uses, and has positive or negative attitudes toward fact-checking sites important to address in a systematic fashion, as without a wide audience paying attention to fact-checks, the mission of these outlets to inform the public and correct false beliefs is undermined.
Theoretical Approach
Years of research on how advertising messages wield their impact has emphasized the importance of three types of responses: awareness, attitudes, and behavior. Although the earliest instance of models that examined these go back more than 100 years (E. St. Elmo Lewis 1898; as referenced in Preston and Thorson 1984), such models experienced a rebirth in the early 1960s (e.g., McGuire 1968).
Lavidge and Steiner (1961) suggest that when people are first exposed to an advertisement (a persuasive message; in this context, a fact-check), they are likely unaware of the product or service advertised (e.g., the fact-checking brand). However, given enough opportunities to be exposed to the product/service they become aware of it and, with repeated exposures, they develop attitudes toward the object (fact-checking). Depending on those attitudes, they may come to purchase or use the product or service; that is, to exhibit a behavior toward it (here, deliberately visiting fact-checking sites). At each stage, different factors come into play, from the cognitive (awareness), to the affective (attitudes), to the conative (behavior; Lavidge and Steiner 1961). The fundamental stages continue to be foundational for theories of advertising effects (e.g., Percy and Rossiter 1992). Furthermore, awareness, attitudes, and behaviors are also used to talk about how innovative ideas become diffused into populations; that is, they transition from awareness to attitudes to adoption (e.g., Rogers 2003).
Although the three stages were initially conceptualized as a “hierarchy” of effects, in the sense that they were thought to occur in a fixed order, there was ample demonstration that they could appear in other orders, and that their chronology was not the important aspect (Barry and Howard 1990). The significant aspect for the present study is that awareness, attitudes, and behavior are distinct and driven by different antecedents (although the stages are interlinked).
Applying this to the fact-checking context, research indicates that while around half of the U.S. public is not very familiar with the major fact-checkers, a majority have favorable views of the form (Nyhan and Reifler 2016). This disconnect suggests that attitudes can be expressed by individuals unfamiliar with fact-checking sites and that predictors of awareness and attitudes may be different. In this case, for instance, an individual’s initial positive attitude toward a fact-checker may be influenced by what they heard on the television news, without that individual necessarily being completely familiar with the fact-checker in question.
That these stages do not necessarily occur in a fixed order also speaks to the potential impact of changes in the contemporary media system. With regard to fact-checks, greater possibilities for incidental exposure and exposure through intermediary actors (such as friends on social media; Thorson and Wells 2016), as well as the spread of partisan discourses via social media (Brandtzaeg et al. 2018; Shin and Thorson 2017), mean greater possibilities for disjuncture between awareness, attitudes, and behavior. Positive or negative framing of fact-checking on social and partisan media may have attitudinal consequences, regardless of how familiar people are with what fact-checking actually is. At the same time, use may occur spontaneously without any prior awareness or attitudes (Lavidge and Steiner 1961) if one is incidentally exposed. Again, these possibilities point to the need to separately analyze the antecedents of fact-checking awareness, attitudes, and behavior, treating them as distinct (though interlinked) stages.
Awareness of, Attitudes Toward, and Behavioral Responses to Fact-Checking Sites
Awareness
Insofar as use of fact-checking sites indicates awareness, evidence suggests that those more likely to be aware are those with higher education, who are more liberal, more interested in and engaged with politics, and who consume more news (Amazeen et al. 2019; Gottfried et al. 2013; Graves and Glaisyer 2012; Shin and Thorson 2017). There are mixed results when it comes to the role that age plays, with visitors to fact-checking sites likely to be younger (Gottfried et al. 2013), but sharers of fact-checks likely to be older (Amazeen et al. 2019; Shin and Thorson 2017). In this study, we focus on visits to fact-checking sites (final behavior). As a result, we posit:
Moreover, given that fact-checking sites have been cited more frequently by mainstream outlets such as CNN and MSNBC (Graves 2016: 208), that these outlets are viewed as biased against conservatives (Knight Foundation 2018), that liberals have more favorable views (Guess et al. 2018; Nyhan and Reifler 2016), and that liberals are more likely to share fact-checks on social media (Amazeen et al. 2019), we ask whether patterns of ideological news consumption will be associated with awareness of fact-checking sites. These factors, combined with tendencies toward the consumption of “identity-relevant media” (Shin and Thorson 2017: 250), may see liberals and consumers of liberal/mainstream news being more aware of fact-checkers than consumers of conservative news.
Attitudes
Accusations of bias leveled at fact-checking sites are common (Amazeen 2013; Graves and Glaisyer 2012) and a consistent claim is that fact-checkers are biased against Republicans (Graves 2016). Studies of social media users have found that people are often skeptical of fact-checkers, questioning their objectivity and viewing them as politically biased toward the left (Brandtzaeg et al. 2018; though those on the left do also complain about fact-checks; Graves 2016). Shin and Thorson (2017) also found that those on the right tend to be more critical of fact-checkers. The authors found that the proportion of negative posts made by Republicans on Twitter about fact-checking sites was significantly larger than the proportion of negative posts made by Democrats (though both did make complaints). Other research has also found conservatives to have less favorable views of fact-checkers (Guess et al. 2018; Nyhan and Reifler 2016). Liberals are also more likely to share fact-checks (Amazeen et al. 2019; Shin and Thorson 2017) and visit fact-checking sites (Guess et al. 2018).
In light of this research, there is a pertinent question about why conservatives might have more negative views of fact-checkers—particularly when both liberals and conservatives are susceptible to biased processing of corrective information (Lewandowsky et al. 2012). First, it is possible that conservative attitudes may be shaped in response to actual fact-checks, given the tendency for fact-checkers to check Republican claims more often and to rate more Republican claims as false (Card et al. 2018; Farnsworth and Lichter 2016). These results suggest either bias in fact-checking, or, alternatively, raise a question about whether Republicans have a greater tendency to lie (Graves 2016). To conservative audience members, the former explanation would be favored.
Relatedly, another possibility is that fact-checking sites are seen as being part of the wider “liberal media apparatus” (Robertson 2019). Conservatives have long claimed that the mainstream media are biased against them and a large number of outlets have been accused of unfairness (Jamieson and Cappella 2010: 145). In 2017, Gallup found that seventy-seven percent of Republicans viewed the media as biased, compared to forty-four percent of Democrats (Swift 2017). Among Republicans who viewed the media as biased, eighty-eight percent believed it favored liberals. If conservatives are consistently exposed to a “liberal bias” narrative from conservative media and politicians (Watts et al. 1999), they may come to see fact-checking organizations as part and parcel of a wider “liberal media apparatus” that does not share their values (Robertson 2019; Slater 2007). Indeed, a well-publicized “media bias” chart placed the main fact-checking organizations, PolitiFact, Snopes, and FactCheck.org , on the left of the spectrum (Attkisson 2018), suggesting conservatives perceive such outlets as part of the left-leaning media.
This literature suggests that political ideology will be an important predictor of attitudes. Thus, we hypothesize the following for both general attitudes and perceived usefulness:
Moreover, if fact-checking sites are indeed seen as being part of a wider “liberal media apparatus” (Robertson 2019)—associated with news outlets viewed as liberally biased—we ask whether patterns of liberal and conservative news consumption are associated with attitudes toward fact-checking sites. Attendance to identity-relevant media sources may create a reinforcing spiral whereby such use bolsters partisan attitudes and negative perceptions of out-group media (Shin and Thorson 2017; Slater 2007).
Behavior
Although attitudes about fact-checking can be influenced by political ideology or by hearing discussions about these organizations, visiting fact-checking sites requires additional effort. That is, a person may have an attitude toward an object, but not engage in behavior toward that object. In this study, we focus on direct visits to fact-checking sites. We recognize that some exposure to and use of fact-checking sites may be incidental and spontaneous (e.g., clicking a fact-check shared by a friend on social media). We also note that studies have looked at sharing of fact-checks as an outcome (e.g., Amazeen et al. 2019). However, we focus on self-reported visits to sites as a direct and deliberate form of media behavior which theoretically precedes sharing.
We expect frequency of visiting fact-checking sites first to be predicted by demographics and news media use (Gottfried et al. 2013), given the sites’ focus on news and politics (Graves 2016). We also expect political antecedents (motivational factors) associated with news consumption (e.g., Mourão et al. 2018) such as interest in politics, political efficacy, political discussion, and political ideology (liberal) to be positive predictors of frequency of visiting (see also Nyhan and Reifler 2016).
Finally, again given that fact-checking sites have been cited more frequently by mainstream outlets such as CNN and MSNBC (Graves 2016: 208), that these outlets are viewed as biased against conservatives (Knight Foundation 2018), that liberals more likely to share fact-checks on social media (Amazeen et al. 2019), and following our discussion of the possible perceptual link between fact-checking sites and the mainstream media (Robertson 2019), we also ask:
Method
Data for this study were collected using a survey conducted on Qualtrics in early 2017. The company provided a sample of adults that was reflective of the United States, based on age, gender, and education (N = 1,805). Census data comparisons and full question wordings are provided in our supplementary materials. Respondents who did not fail attention checks and who had complete data on dependent variables were included in the analyses (N = 1,033). Given our theoretical argument that even those unfamiliar with fact-checking may express attitudes and report visitation, these respondents are included in all models.
Dependent Variables
Fact-checking familiarity
We operationalized awareness of fact-checking sites by assessing self-reported familiarity in a range from very unfamiliar (0) to very familiar (6) (M = 2.38, SD = 2.06).
Fact-checking attitudes
Respondents were asked “in general, how do you feel about fact-checking sites like PolitiFact, Snopes, and others?” (0 = very unfavorable to 6 = very favorable). For all individuals in the sample, including those “very unfamiliar” with fact-checking, the average response was 3.4 (SD = 1.70). Around sixty-four percent of those very unfamiliar with fact-checking still expressed positive or negative attitudes toward such sites despite their stated lack of familiarity (rather than giving a neutral response), while ninety-two percent of those very familiar expressed attitudes.
Fact-checking perceived usefulness
Respondents were asked (0 = strongly agree to 6 = strongly agree; midpoint = neither agree nor disagree) whether they thought that “news stories that include reports on fact-checking are helpful to me” (M = 3.82, SD = 1.41, range). Around forty-nine percent of those very unfamiliar with fact-checking still expressed an opinion about the usefulness of fact-checking sites despite their stated lack of familiarity (rather than giving a neutral “neither agree nor disagree” response), while ninety-two percent of those very familiar expressed an opinion.
Fact-checking site visitation
Respondents were asked: “how often did you go to any fact-checking websites during the presidential election to find out whether a statement about any of the candidates was accurate or not?” (0 = not at all to 3 = went often). About fifty-three percent of respondents answered that they did not go to a fact-checking site at all, 12.4% went once, 25.3% went several times, and 9.3% went often (M = 0.91, SD = 1.07). Among those very unfamiliar with the fact-checking movement, only fifteen percent reported visiting a fact-checking site once or more. Fully eighty percent of those very familiar reported going several times or often.
Independent Variables
Demographics
Respondents ranged in age from eighteen to seventy-six years (M = 47.39, Mdn = 49, SD = 16.13), with 54.7% identifying as female. The sample was 78.2% white. Income was measured using five categories ranging from “less than $25,000” to “$100,000 or more” (M = 2.51, SD = 1.29, modal category = “$25,000 to $49,999”). For education, respondents were asked to locate themselves across six categories, ranging from “less than high school” to “postgraduate degree” (M = 3.17, SD = 1.21, modal category = “some college”).
Political interest
Respondents were asked “In general, how interested are you in politics and national government?” (0 = not at all to 4 = extremely; M = 2.60, SD = 1.18).
Political efficacy
Following Saldaña et al. (2015), political efficacy was measured with five items asking how much respondents agreed with statements such as “People like me can influence government” and “I consider myself well qualified to participate in politics” (Cronbach’s α = .70, M = 4.41, SD = 1.90, 0 = strongly disagree to 9 = strongly agree).
Political ideology
To create the political ideology variable, three questions were combined, probing respondents’ party affiliation and ideological position on economic and social issues. Responses were averaged to create an overall measure of political ideology, Cronbach’s α = .89, M = 5.19, SD = 2.72, ranges 0 = strong Republican to 10 = strong Democrat (5 = Independent), 0 = strong conservative to 10 = strong liberal.
Political discussion
Frequency of political discussion was measured using six questions asking how often (0 = never to 9 = all the time) respondents discussed political issues with different types of people (Cronbach’s α = .91, M = 3.25, SD = 2.34).
News media use
Following the approach of Mourão et al. (2018), respondents were asked to indicate how often in the last week they had consumed content from a range of news sources (thirteen in total; range 1 = never to 5 = very often). A factor analysis identified three patterns of news media use: mainstream/liberal (e.g., the New York Times, Washington Post, BBC, Guardian, CNN, liberal news websites, MSNBC, NPR; Cronbach’s α =.88, M = 2.01, SD = 1.05), local/some broadcast television (e.g., nightly news on CBS, ABC, and NBC, The Today Show, Good Morning America, CBS This Morning, local TV, and newspapers in one’s area; Cronbach’s α = .77, M = 2.91, SD = 1.08), and conservative (e.g., Fox News, conservative news websites, conservative talk radio; Cronbach’s α = .82, M = 1.97, SD = 1.11). The supplementary materials show the survey questions and rotated component matrix. The three categories identified correspond with surveys of audience perceptions of and profiles for each outlet. Mitchell et al. (2017) found that the New York Times, Washington Post, MSNBC, and NPR had a liberal/Democrat audience profile; Fox News and Rush Limbaugh a conservative/Republican audience profile; and ABC, CBS, and NBC a mixed audience profile. Moreover, audience perceptions of the outlets diverge, with Republicans viewing CNN, MSNBC, the New York Times, NPR, and Washington Post as biased against them, and Democrats viewing Fox News as biased against them (Knight Foundation 2018). Thus, we label the first category liberal/mainstream because it contains a collection of outlets viewed as liberal and attended to by liberals, the second category local/broadcast because it contains local news consumers and broadcast outlets which appeal to mixed audiences, and the third category conservative because it contains outlets which are viewed as conservative and attended to by conservatives.
Analysis
Research questions were answered and hypotheses tested using hierarchical linear regression models. The first block of the regressions includes demographics, the second block political antecedents, and the third block news consumption variables. Independent variables are held constant across models to aid with comparisons.
Results
Familiarity with Fact-Checking
Results presented in Table 1 show that demographics explain about 16.4% of the variance observed, with those who are more educated (β = .11, p < .001) and younger (β = −.10, p < .01) being more likely to be familiar with fact-checking sites (H1a supported). As predicted (H1b-e supported), political interest predicts familiarity (β = .15, p < .001), as does efficacy (β = .14, p < .001) and political discussion (β = .19, p < .001). Identifying as liberal also predicts greater familiarity (β = .07, p < .05). Political antecedents explain 19.2% of the variance. The media use block predicts an additional 3.1% of the variance, with more mainstream/liberal (β = .28, p < .001) and less local news (β = −.14, p < .001) consumption being associated with awareness of fact-checking sites. This provides partial support for H1f and addresses RQ1: Mainstream/liberal news consumption is associated with greater familiarity with fact-checking sites, while conservative news consumption is not significantly related and local/broadcast news consumption is negatively related. The full model explains 38.7% of the variance.
Hierarchical Regression Predicting Familiarity with Fact-Checking Sites (Standardized Coefficients).
p < .05. **p < .01. ***p < .001.
Results indicate that fact-checking sites are most likely to enter the purview of (educated, younger, male) individuals who are already more engaged with politics and the news (via liberal/mainstream outlets). All four political antecedents (along with age, gender, and education) are predictive of familiarity. Demographics and political antecedents account for most of the variance explained. This is indicative of the fact that individuals can be made aware of fact-checking in a variety of ways (and consistent with a stage model perspective where individuals can be made cognitively aware of something in numerous ways; Lavidge and Steiner 1961): the more politically interested, politically conversive, or liberal an individual is, the more likely they are to hear about fact-checking in one way or another.
General Attitudes toward Fact-Checking
Demographics account for only 6.5% of the variance in attitudes toward fact-checking, with those younger (β = −.09, p < .01) and identifying as white (β = .12, p < .001) being the only significant predictors of attitudes here (Table 2). Political antecedents, meanwhile, explain 13.1% of the variance, with political interest (β = .17, p < .001) and efficacy (β = .14, p <.001) predicting more positive attitudes. We find that identifying as liberal is a significant predictor of more positive attitudes (β = .08, p < .05), which supports H2. Media use variables add another 2.2% to the variance accounted for. Addressing RQ2, liberal/mainstream news use is positively related to attitudes (β = .25, p < .001) and conservative news use is negatively related (β = −.15, p < .001). Liberal/mainstream news use is the largest single predictor of (positive) attitudes toward fact-checking sites, while conservative news use is a strong negative predictor. The full model accounts for 21.9% of the variance.
Hierarchical Regression Analysis Predicting General Positive Attitudes toward Fact-Checking Sites (Standardized Coefficients).
p < .05. **p < .01. ***p < .001.
We note that the role of demographic variables is diminished when it comes to accounting for attitudes (compared to awareness). Political antecedents account for most of the variance, indicating that attitudes are less a function of who individuals are and more about their political involvement. Moreover, news consumption plays a key role, with liberal/mainstream news consumers having more positive views and conservative news consumers having more negative views. These findings are consistent with a stage models framework, where attitudes are predicted by affect/emotion (e.g., ideological commitment) and message framing (Lavidge and Steiner 1961).
Perceived Usefulness of Fact-Checking
Demographics account for only three percent of the variance in perceived usefulness, with no demographic variable remaining significant in the Step 3 model (Table 3). Political antecedents explain 11.3% of the variance, with political interest (β = .21, p < .001) and efficacy (β = .08, p < .05) being significant predictors of perceived usefulness. Regarding the association between political ideology and perceived usefulness, findings show that this is not a statistically significant relationship once media use variables are added to the regression model (H3 not supported).
Hierarchical Regression Analysis Predicting Perceived Usefulness of Fact-Checking Sites (Standardized Coefficients).
p < .05. **p < .01. ***p < .001.
Addressing RQ2, mainstream/liberal news use is positively related to perceived usefulness (β = .24, p < .001), while conservative news use is a negative predictor (β = −.20, p < .05). Again, liberal/mainstream news use is the strongest predictor of (positive) attitudes—in this case, perceived usefulness—and conservative news use is a strong negative predictor. Media use variables add 2.8% to the variance accounted for, with the full model explaining 17.2% of the variance.
The findings with regard to perceived usefulness are similar to those with regard to general attitudes: political antecedents account for most of the variance and there are different relationships between liberal/mainstream and conservative news consumption and fact-checking attitudes.
Visiting Fact-Checking Sites
Table 4 shows that demographics explain eleven percent of the variance observed, with those who are younger (β = −.07, p < .05) being likely to report visiting fact-checking sites more often. Education is not related to visiting in the Step 3 model (H4a partially supported). Meanwhile, as predicted, political interest (β = .15, p < .001), political discussion (β = .20, p < .001), and identifying as liberal (β = .07, p < .05) are related to more frequent visits (H4b, c, and e supported). However, political efficacy is not (H4d not supported). Political antecedents explain 19.6% of the total variance.
Hierarchical Regression Analysis of Predicting Frequency of Visiting Fact-Checking Sites (Standardized Coefficients).
p < .05. **p < .01. ***p < .001.
With regard to media use (H4f), only liberal/mainstream news consumption is positively related to self-reported visiting (β = .31, p < .001). Local/broadcast news consumption is negatively related to visiting (β = −.11, p < .01). Thus, H4f is partially supported and an answer to RQ3 is provided: liberal/mainstream news consumption is predictive of more self-reported visiting of fact-checking sites, while conservative news consumption is not significantly related, and local/broadcast news consumption is negative. News consumption variables add four percent to the variance explained, with the full model accounting for 34.6% of the variance.
Findings here paint a picture of a form of journalism which has an appeal to liberals and those who consume news from a repertoire of mainstream/liberal outlets. Political antecedents also account for a large portion of the variance, indicating that political interest, ideology, and discussion are factors which may push individuals to try out fact-checking sites. This is consistent with a stage models approach, whereby behavior is predicted by motivational factors (Lavidge and Steiner 1961).
Discussion
This study sought to identify predictors of awareness of fact-checking sites, people’s attitudes toward them, and self-reported visits to the sites. Drawing from persuasive message literature, we argue that audience relationships with fact-checking sites are dependent on three distinct stages—awareness, attitudes, and visiting behavior—and that these stages are predicted by different sets of antecedents. While we cannot make any causal statements with respect to the relationships between our independent and dependent variables (given the use of a cross-sectional survey), we note a number of important associations.
In general, we find that each stage is predicted by different combinations of antecedents, though there are common through lines. Familiarity with fact-checking sites is predicted by a range of variables, consistent with a stage models framework (Lavidge and Steiner 1961): age, gender (male), education, political interest, political efficacy, ideology (liberal), political discussion, and news consumption. Attitudes (both general and in terms of perceived usefulness) are predicted by ideological news consumption patterns, as well as political interest and efficacy. Finally, self-reported visiting of fact-checking sites is predicted by ideology (liberal), liberal/mainstream news consumption, as well as political discussion and interest. Local/broadcast news consumption is a negative predictor. Across all models, interest in politics and mainstream/liberal news consumption are large and significant predictors of fact-checking awareness, attitudes, and behavior. Thus, across models, there is some overlap in predictors, though each model sees different combinations of variables. This suggests interlinking between the three stages, with common factors being interest in politics, liberal/mainstream news consumption, and liberal ideology.
Focus in this study was placed on the United States, given that it is the center of the global fact-checking movement and outlets in other nations have followed the U.S. model (Graves 2018). However, our findings and stages of persuasion theoretical approach may provide guidance in studies of other national contexts where fact-checking has developed. A relevant question for future research is whether similar or different sets of antecedents predict the stages in other national contexts; variations can elucidate the role of country-specific factors. Next, we turn to a more detailed discussion of each stage and its predictors in the U.S. context.
Awareness
Our findings indicate that fact-checking sites are more familiar to liberals and those most engaged with news from outlets such as CNN and MSNBC—outlets which have been noted to cite fact-checks more frequently (Graves 2016: 208) and which have been labeled/perceived as liberal (Jamieson and Cappella 2010; Knight Foundation 2018). Vargo et al. (2018) found the influence of fact-checkers in the media landscape to be small and declining and here we do find local/broadcast news consumption to be negatively related to fact-checking familiarity, which accords with Vargo et al.’s (2018) findings. But we also find that mainstream/liberal news consumption positively predicts awareness. Thus, our findings raise questions of whether individuals are exposed via news outlets or whether exposure is being facilitated through other means, for instance, via more frequent shares of fact-checks by liberals on social media (Amazeen et al. 2019; Shin and Thorson 2017; Vargo et al. 2018).
Alternatively, our results may also be a product of context: Vargo et al. (2018) only analyzed data up to 2016 and it is likely that liberal/mainstream news outlets have since increased their level or use of fact-checking in response to the rhetoric of President Trump. This could see more exposure to fact-checking among liberal/mainstream news consumers. Future research is needed to explore these questions. Moreover, there is a need to explore predictors of awareness in other national contexts: How do the specificities of other countries’ political systems and media routines relate to familiarity with fact-checking sites? For example, Verificado’s decentralized, social media–focused, and collaborative approach to fact-checking may mean social media use is a strong predictor of fact-checking awareness in Mexico (Martínez-Carrillo and Tamul 2019).
Attitudes
When it comes to attitudes, we find that, in the U.S. context, liberals have more positive views of fact-checkers, as do more frequent consumers of liberal/mainstream news. Conservatives and more frequent conservative news consumers, on the other hand, have more negative views. News consumption plays a key role: while ideology is a predictor of attitudes toward fact-checking sites (in line with Nyhan and Reifler (2016) and Guess et al. (2018)), when ideology is held constant and media use variables are added to models, liberal/mainstream news consumption becomes a strong positive predictor (while conservative news consumption becomes a strong negative predictor). This suggests that it is not ideology alone that is related to attitudes; partisan media consumption may amplify individual predispositions.
Findings are consistent with a stage model perspective, where attitudes are predicted by affect/emotion (e.g., strength of ideology) and message framing (Lavidge and Steiner 1961). It may be that liberal/mainstream news outlets speak positively of fact-checkers, while conservative outlets speak negatively, factors which could account for differences in attitudes. However, in light of Vargo et al.’s (2018) finding that the influence of fact-checking in the media landscape is minimal, an alternative explanation is that attitudes toward fact-checking sites could be influenced by discourse from other sources (such as social media users and politicians themselves; Brandtzaeg et al. 2018; Shin and Thorson 2017).
To the extent that attitudes are related to ideological news consumption, we query whether fact-checking sites are viewed as part of a liberal/mainstream media apparatus (Robertson 2019), a factor which might influence the attitudes of liberals and conservatives differentially. In the United States, fact-checkers have forged relationships with the mainstream media (Graves 2016), which may aid in explaining the association. Future research should assess specifically whether fact-checking sites are part of a liberal/mainstream news consumption repertoire, how fact-checking sites are framed by different sources, and whether attitudes toward the mainstream media are predictive of attitudes toward fact-checking sites.
In addition, future research should also focus on other nations where fact-checking has taken hold and where different types of practices are employed. We believe our theoretical approach can be helpful in comparative research on global fact-checking. In other contexts, different (perceived) relationships between fact-checkers and the media may see lower faith in the mainstream media being associated with more faith in fact-checking, given that it is a format which seeks to fix problems with mainstream journalism (Amazeen 2019). See, for example, Martínez-Carrillo and Tamul (2019) with respect to Verificado keeping the press accountable in Mexico and Amazeen’s (2017) discussion of fact-checking as tool for institutional repair.
Behavior
Findings with respect to visits to fact-checking sites again paint a picture of a form of journalism which has an appeal to liberals and those who consume news from a repertoire of mainstream/liberal outlets. We also note that our results may be influenced by the fact our survey was fielded after President Trump’s election and a relevant question is whether predictors of visitation would be different if Democrats controlled Congress and the White House; contextual factors which might see fact-checkers investigate more claims from Democrats (Amazeen 2013). It is likely that fact-checking sites, at present, have greater perceived utility for liberals given the frequency with which President Trump’s claims are challenged by fact-checkers (Card et al. 2018).
In this context, liberal attitudes can be seen as a motivating factor for people to seek out fact-checks, alongside political interest and propensity for political discussion. Thus, our findings parallel the results from Amazeen et al. (2019) regarding the greater information-seeking motivation and need for orientation among sharers of fact-checks on social media. They are also reflective of the motivational factors which underpin behavior, consistent with a stage models approach (Lavidge and Steiner 1961). Comparative studies in other nations can assess whether visits to fact-checking sites might be influenced by who is in power there or by other country-specific contextual factors. It may be that conservatives are more motivated to seek out fact-checks in contexts where a left-leaning government is in power.
A limitation of this study is that we employed a self-reported measure of behavior (visiting sites), which may be subject to error. Future studies should look at using trace data to assess visits to fact-checking sites, given the possibility that individuals supportive of fact-checking may over-report and those opposed may under-report. Moreover, we did not measure familiarity with, attitudes toward, and visits to specific fact-checking sites separately. Thus, we can only speak to U.S. fact-checkers generally without differentiating between outlets when it comes to these outcomes. Observations are limited to the fact-checkers mentioned in question prompts (PolitiFact, Snopes, and FactCheck.org) and the U.S. audience.
Finally, we did not assess other ways in which people might come into contact with and thereby read fact-checks, focusing only on those who report directly visiting such sites. Future studies should also consider who is most likely to be (incidentally) exposed to and read fact-checks and how they come across them: for example, via social media platforms or apps. Other behavioral outcomes to focus on include sharing, liking, and commenting on fact-checks on social media platforms. In a media landscape increasingly shaped by the online diffusion of information by various actors (Thorson and Wells 2016), these are necessary considerations—and especially important given that fact-checkers often rely on social media audiences, as well as other actors, to spread their content (Graves 2016). In Mexico, for instance, Verificado directly sent fact-checks to their followers via WhatsApp (Martínez-Carrillo and Tamul 2019). In this study, we may have found liberals reporting more frequent visits to fact-checking sites because of other liberals sharing fact-checking on social media more frequently (Amazeen et al. 2019; Shin and Thorson 2017).
Conclusion
The results of this study with respect to awareness of, attitudes toward, and visits to fact-checking sites in the U.S. point to such sites being absorbed into wider ideological discourses (Brandtzaeg et al. 2018; Shin and Thorson 2017) and patterns of ideological news consumption. While fact-checking sites in the U.S. position themselves as neutral and objective actors (Graves 2016), findings suggest that they are viewed politically by audiences. If their brand names serve as heuristic cues (e.g., “PolitiFact is liberal”), then this may shape how fact-checks are processed by audiences.
More broadly, important questions are raised if the influence of fact-checking on media agendas is limited (Vargo et al. 2018), if visits to sites are infrequent, if liberals are more likely than conservatives share their content (Amazeen et al. 2019) and visit such sites directly, if audiences view them as biased (Brandtzaeg et al. 2018), and if fact-checks are shared in a biased way (Shin and Thorson 2017). These limitations severely undercut the mission of fact-checkers to inform the wider public. Patterns of identity-relevant, ideological consumption undermine the visibility, reach, and effectiveness of fact-checks (Shin and Thorson 2017). At its core, fact-checking is an activity focused on investigating the veracity of claims, employing a form of “scientific objectivity” that attempts to remove bias (Coddington et al. 2014). In the eyes of audiences, however, fact-checkers are perceived as partisan actors in a divided media system.
Supplemental Material
Supplementary_Materials – Supplemental material for Who Uses Fact-Checking Sites? The Impact of Demographics, Political Antecedents, and Media Use on Fact-Checking Site Awareness, Attitudes, and Behavior
Supplemental material, Supplementary_Materials for Who Uses Fact-Checking Sites? The Impact of Demographics, Political Antecedents, and Media Use on Fact-Checking Site Awareness, Attitudes, and Behavior by Craig T. Robertson, Rachel R. Mourão and Esther Thorson in The International Journal of Press/Politics
Footnotes
Acknowledgements
The authors would like to thank the Reynolds Journalism Institute at the University of Missouri for supporting this research.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
