Signaling News Outlet Credibility in a Google Search

Abstract

This study extends the literature on how transparency influences news credibility perceptions by examining trust signals at the news outlet level, rather than at the story level, as earlier research has done. Experiments in the United States (n = 1,037) and Germany (n = 1,000) found that exposure to trust signals in a Google search about a known news brand, rather than an unknown brand, and the German cultural context increased news credibility perceptions. Participants were more likely to click on trust signals that gave background about the news brand or offered ways to engage with a news outlet.

Keywords

online news transparency credibility trust

U.S. newsreaders have low news trust (Mitchell et al., 2018; Newman et al., 2020), with more than a majority seeing it as inaccurate or biased (Knight Foundation, 2020). News trust is higher in other countries, such as Germany, but lagging trust remains a concern, at least in Anglo-Saxon countries (e.g., Hanitzsch et al., 2018; Newman et al., 2020). This has far-reaching ramifications because low trust inhibits the news media’s goal to keep the electorate informed. Efforts to cue credibility perceptions through transparency—disclosing information about the news outlet or how a story was done—alongside or within a story have shown limited promise (e.g., Curry & Stroud, 2021; Masullo et al., 2022; Peifer & Meisinger, 2021). Yet, these efforts have largely failed (e.g., Karlsson & Clerwall, 2018; Karlsson et al., 2014; Peacock et al., 2022; Tandoc & Thomas, 2017) or been inconsistent (Masullo et al., 2022; Peifer & Meisinger, 2021). This study explores news distrust in a new way by examining whether news credibility perceptions can be cued through transparency at the domain level—the level of the overall news brand—not of individual stories.

We examined transparency at the domain level by manipulating what information was shown about a news outlet in a functional replica of the sidebar that appears when people do a Google search for the outlet. This sidebar, which Google calls the Knowledge Panel, normally provides information about the news outlet, such as when it was founded and who owns it. We wanted to see if individual indicators in this sidebar could be manipulated to effectively signal credibility to newsreaders. We conducted experiments in the United States (n = 1,037) and Germany (n = 1,000) because both are affluent and prominent global democracies, but news trust is higher in Germany than in the United States (Newman et al., 2020), and their media systems vary (Knobloch-Westerwick et al., 2015). We also compared real news brands to unknown news brands to examine whether signaling news credibility perceptions in the Knowledge Panel was more or less effective when people recognized a brand.

Our study offers important practical and theoretical contributions, although our findings showed that no configuration of transparency signals in a Knowledge Panel cued people to perceive news outlets as more credible. From a practical standpoint, we found that people clicked more on information that provided background about the news brand or offered ways to engage with journalists, suggesting that newsrooms make this information easily available to the news audience. From a theoretical standpoint, our findings showed that what actually increased credibility perceptions was exposure to a known news brand, versus an unknown brand, highlighting how the brand may operate as a heuristic cue of credibility. Also, we provide new knowledge about news credibility by showing that people may process credibility differently in countries like the United States where media trust is quite low, compared with countries like Germany, where news trust is higher. These findings suggest that domain-level transparency signals may not operate as heuristic cues of credibility, as we proposed, or, if they do, they may need to be designed differently. Yet, these transparency signals still may be beneficial for newsrooms by eliciting attention from the news audience.

Literature Review

Conceptualizing Credibility and Trust

A rich canon of literature has explored news credibility and the related but distinct concept of news trust (e.g., Austin & Dong, 1994; Gaziano & McGrath, 1986; Hovland et al., 1959), making it a seminal topic for journalism research. Research has examined organizational credibility (Gass & Seiter, 1999), website credibility (Q. Chen & Wells, 1999), and how credibility is connected to visual social media cues, such as Twitter’s “verified” badge (Edgerly & Vraga, 2019). Studies have identified factors that shape credibility perceptions, such as quality of content (G. M. Chen et al., 2017; Cummins & Chambers, 2011); mistakes (Karlsson et al., 2017); and whether the content is consumed on a blog, website, etc. (Johnson & Kaye, 2014). Notably, our study contributes to this literature by examining perceptions of credibility at the domain level of the news outlet, rather than at the story level, as much research has done.

Some unpacking of the concepts of credibility and trust is worthwhile. Although some scholars treat them interchangeably because making a clear distinction is challenging, we do not. We ascribe to the idea put forth by Tseng and Fogg (1999), who posit that trust is “a positive belief about the perceived dependability of, or confidence in a person, object, or process” (p. 41) whereas credibility is defined as a perceived quality or believability in a particular object. Similarly, Rieh and Danielson (2007) view credibility as “a perceived quality of a source” that “may or may not result in associated trusting behaviors” (p. 314). This is because the concept of trust, or trusting behavior, includes a trustor’s expectation toward a trustee, thereby, encompassing risk and uncertainty (Coleman, 1990).

Following this logic, media trust is about people’s perceptions of how well the news media will meet certain expectations whereas media credibility is perceived believability of media sources or content (Metzger & Flanagin, 2013). For example, even if a person A trusts (or has expectations) that news outlet B will produce reliable news stories, A could evaluate a particular story published by B as not credible. Therefore, A takes risks and uncertainties when trusting B. Thus, we focus on the news credibility perceptions. News credibility encompasses the believability of a media source, message, and channel (Metzger & Flanagin, 2008). In the early days of research on media credibility, the focus was mostly on source credibility and its two dimensions, trustworthiness and expertise of whoever created the message (Hovland et al., 1959). These two dimensions provided a foundation for subsequent studies of the credibility of particular messages (message credibility) and of particular news outlets (news credibility; Metzger et al., 2003). However, Hovland et al.’s (1959) work remains important today, as it theorizes that the audience for particular mediated messages makes credibility assessments about message creators based on perceptions of the expertise and trustworthiness of those creators. In our study, we adopt this conceptualization of credibility, positing that newsreaders make assessments about the credibility of a news outlet based on their perceptions of the expertise and trustworthiness of that brand in their own minds.

Credibility Heuristics in an Online News Environment

The heuristic-systematic model (HSM) of information processing (Chaiken, 1980) is helpful for understanding how news media audiences make these credibility assessments based on their perceptions of the expertise and trustworthiness of a news brand. Under this model, people rely on cues or mental shortcuts called heuristics to make snap judgments about media, such as credibility perceptions. This heuristic processing occurs when people tap into pre-existing notions or beliefs and process messages quickly, rather than take a more effortful or systematic approach (Chaiken, 1980). Heuristic processing is particularly likely in today’s frantic media environment, where people are confronted with an overload of information and have to make split-second choices about what to pay attention to by skimming information (V. Y. Chen & Chen, 2020; Holton & Chyi, 2011). In days past, people could evaluate news media credibility more slowly and easily because they had fewer choices (Callister, 2000), but today making credibility perceptions is more challenging amid increasing uncertainty about what can be believed (Metzger et al., 2003; Rieh & Danielson, 2007). Hence, people are more likely to rely on heuristics to make credibility assessments because this allows them to preserve their limited cognitive energy (Metzger & Flanagin, 2013).

Metzger and Flanagin (2013) classified six heuristics for credibility evaluations online: reputation, endorsement, consistency, self-confirmation, expectancy violation, and persuasive intent. We focus on the reputation heuristic, which is when people rely on the prestige of a media source they recognize to make a credibility assessment. The reputation heuristic allows people to transfer their credibility assessments about a traditional print newspaper outlet, for example, to that outlet’s website (Metzger et al., 2003). The reputation of a media company can be a powerful indicator of how people perceive that news outlet, and credibility is a core aspect of that assessment (Victoria-Mas et al., 2018). People perceive sources they are more familiar with as more credible, relative to those they have not heard of (Hilligoss & Rieh, 2008). For example, research has found that disclosing information can increase credibility perceptions for a known news brand (Masullo et al., 2022) or a certain news topic (Peifer & Meisinger, 2021). Based on the literature just discussed, we predicted that the reputation of a known news outlet would be a heuristic powerful enough to lead people toward evaluating that outlet as more credible than an unknown news brand. Therefore, we hypothesized:

H1: Credibility perceptions will be higher for known news brands, compared with unknown news brands.

Transparency and Credibility

Transparency, which is explaining to the public information how journalism operates, is one way scholars and practitioners have attempted to increase media credibility (e.g., Craft & Heim, 2009; Karlsson, 2010; Karlsson & Clerwall, 2018). Transparency in this sense can be defined in two ways: Disclosure transparency explains how news is made, while participatory transparency invites readers into the news production process (Karlsson, 2010). The argument that transparency would cue credibility is based on interpersonal research that has found people trust (Wheeless & Grotz, 1977) and appreciate others (Collins & Miller, 1994) who disclose information about themselves. In journalism, transparency has developed as a new norm that espouses that if news organizations open up to the public, the public will see them as more trustworthy (Van Der Wurff & Schönbach, 2011). Thus, transparency is separated from the more established journalistic norms of objectivity (Tandoc & Thomas, 2017), responsibility, and accountability (Karlsson et al., 2017) yet is related to them.

Journalists enact transparency by providing hypertext to original reporting documents (Mor & Reich, 2018), correcting errors (Karlsson, 2010), or sharing information about themselves (Curry & Stroud, 2021). Scholars have tested whether transparency at the news article level increases credibility or trust, and success has been limited. Curry and Stroud (2021) found that disclosing five transparency items in a news story, such as explaining how a story was done or including information about journalists, collectively boosted credibility perceptions, although few people actually clicked on these elements in their experiment. Similarly, disclosing details about the reporting process in a box alongside a story increased credibility perceptions for a known news brand (Masullo et al., 2022) and for one story but not another (Peifer & Meisinger, 2021). Labeling stories as “news” or “opinion” in a form of disclosure transparency produced no effect on credibility perceptions across two experiments (Peacock et al., 2022). Karlsson and colleagues (2014) experimentally manipulated both disclosure transparency, such as corrections to a story, and participatory transparency, such as comments on a story, and found almost no effects on either source or message credibility. Similarly, findings from an experiment, survey, and focus group offered little evidence that transparency at the story level is effective, although focus group participants did appreciate some indicators, such as efforts to explain a story’s angle (Karlsson & Clerwall, 2018). In fact, Tandoc and Thomas (2017) found that adding personal information about the journalist to a news story actually dampened credibility perceptions, rather than heightened them.

Domain-Level Approach

It is notable that all these studies examined transparency at the individual story level. In other words, these efforts attempted to cue newsreaders’ credibility perceptions about the news outlet by disclosing information either within or alongside a particular story. These story-level transparency efforts may have worked inconsistently because newsreaders may not have necessarily linked these cues on a news story to the news outlet itself. Essentially, even if these story-level transparency efforts could work as heuristics of credibility, people may not necessarily think about the news outlet when reading a particular story. Indeed, in an early study, Austin and Dong (1994) found that people assess news stories without thinking about the source. Therefore, adding transparency cues intended to signal credibility to a story might influence other factors, such as whether people perceive the story as well written, interesting, and fair, as Karlsson and colleagues (2014) found, yet have no influence on credibility perceptions of the news outlet itself.

Given this argument and the inconsistent success of the news story-level approach, we tackled the idea of transparency in a different way by focusing on the domain level. The domain-level approach is different from the story-level approach because it situates transparency cues beside information about the news outlet itself, rather than within or alongside a particular story. Google’s Knowledge Panel offers an enviable opportunity to test the theoretical relevance of this domain-level approach to transparency. According to Google, Knowledge Panels are

automatically generated . . . information boxes that appear on Google when you search for entities (people, places, organizations, things). . . . They are meant to help you get a quick snapshot of information on a topic based on Google’s understanding of available content on the web.¹

We varied what transparency signals appeared in a functional replica of a Knowledge Panel about a news outlet to see whether disclosure transparency at the domain level could cue credibility perceptions and, if so, which specific signals were most effective. Because details of our experimental design are essential to understanding our upcoming research questions, we provide some methodological details here, although we elaborate on these in the “Method” section.

We drew theoretical support from the HSM, positing that if domain-level transparency indicators work as heuristics, they could be used to cue credibility perceptions. We varied transparency at the domain level along with three major signals: background about the brand, which included information about ownership and newsroom policies; audience engagement, which showed journalists’ bios and provided contact information for them, so the public could engage with the journalists; and external evaluations, which included awards and assessments from independent groups. We examined audience engagement because conceptual research on reciprocal journalism emphasizes that interactions between journalists and their communities result in “better community, and, indeed, better journalism” (Lewis et al., 2014, p. 236). We considered each category alone and in combination with the other categories and compared each to a control condition that only provided the news outlet’s name. We developed these three signals based on a review of research on credibility indicators (e.g., Curry & Stroud, 2021; Karlsson, 2010; Karlsson et al., 2014) and based on what news trust organizations, such as Trusting News and the Trust Project,² advocate (Table 1).

Table 1.

Knowledge Panel Transparency Signal Configurations, Means, Standard Errors, and Significant Differences.

Condition	Transparency signal	Content	Credibility perceptions	Clicks
Condition	Transparency signal	Content	M (SE)	M (SE)
1	Brand background	• Date founded • Corporate headquarters • Ownership • Mission statement • Ethics policy • Revenue source • Verification policy	3.54 (0.06)	0.15^a (0.02)
2	Audience engagement	• Editorial team contact information • Bios of journalists • Membership in journalism collaborative that indicates trustworthiness	3.47 (0.06)	0.18^{b, c} (0.02)
3	External evaluations	• Ideological leaning from independent source • Traffic rating • Awards • Participation in trust initiative called ProtectNews	3.52 (0.06)	0.06^{a, b} (0.02)
4	Brand background and audience engagement		3.45(0.06)	0.16(0.02)
5	Brand background and external evaluations		3.49(0.06)	0.11(0.02)
6	Audience engagement and external evaluations		3.44(0.06)	0.08^c (0.02)
7	Brand background, audience engagement, and external evaluations		3.46(0.06)	0.12(0.02)
8	Control	• Name of organization and URL	3.38(0.06)	0.00(0.02)

Note. No significant differences were found for credibility perceptions. For clicks, significant differences are indicated when means have the same letter subscript. Significant differences with the control are not noted for clicks because the control condition had nothing clickable.

To increase realism and face validity, these Knowledge Panels were designed to be interactive, so people could click parts of them and gain more information about different domain-level signals (e.g., clicking on the words “ethics policy” would produce a pop-up window to an ethics policy). As a result, we considered whether there would be significant differences in which configuration of signals elicited the most clicks. We conceptualized clicking on a signal as an expression of interest in that signal, in accordance with research on clicks (e.g., Tenenboim & Cohen, 2015). Because no research has examined domain-level transparency signals and because story-level transparency signals have had mixed success, we posed the following:

RQ1: Can domain-level transparency signals in the Google search Knowledge Panel serve as heuristic cues of credibility perceptions?

RQ2: Which specific configurations of the domain-level transparency signals—if any—in the Google search Knowledge Panel are most effective at (a) increasing credibility perceptions or (b) eliciting clicks on those signals?

Comparing German and U.S. Media Systems

Because news distrust is a global issue and Google’s Knowledge Panel is employed worldwide, we considered how domain-level transparency signals might operate as heuristic cues in two countries. We focused on the United States and Germany because both face similar challenges, such as escalating right-wing populism, backlashes against the surveillance capacities of social media platforms, and the political weaponization of disparaging remarks toward mainstream media outlets. In the United States, former President Donald J. Trump has demonized the mainstream media as fake news (Meeks, 2020), while in Germany, politicians have weaponized “Lügenpresse”—the “lying press”—to attack mainstream news (Henke et al., 2020). The United States and Germany are also relevant to compare because their media systems differ (Knobloch-Westerwick et al., 2015). The U.S. media system has a weak state role (Brüggemann et al., 2014) with revenue drawn mainly from advertising (Revers, 2017) while the German media system has strong state intervention with public press subsidies and revenue from paper sales (Revers, 2017).

Most notably for this study, one stark way that the United States and Germany differ is in overall news trust. The most recent Reuters Institute for the Study of Journalism’s annual Digital News Report found that overall trust in news is at 45% in Germany, compared with 29% in the United States (Newman et al., 2020). A Pew Research Center report found that 72% of Germans think the media are “doing very/somewhat well at reporting political issues fairly,” compared with 47% of Americans (Mitchell et al., 2018). The same study found that 75% of Germans felt the media were doing “very/somewhat well at reporting news accurately,” compared with 56% of Americans.

However, the gap between the United States and Germany narrows for trust in search results (24% in Germany, 22% in the United States) and trust in news on social media (14% for both countries; Newman et al., 2020). Yet, overall, Germans’ higher media trust is relatively stable and has been linked to higher interpersonal and institutional trust more generally (Granow et al., 2020) and the presence of a strong public service broadcasting system in Germany. Thus, these two countries offered an enviable opportunity to extend our theoretical understanding of how transparency signals may operate as heuristics cues at the domain level in contexts that share important similarities but also differ in notable ways. Based on this literature, we hypothesize:

H2: Credibility perceptions will be higher in Germany, compared with the United States.

Finally, we examined whether either the country (United States or Germany) or the brand type (known versus unknown) would heighten the effects of the Knowledge Panel transparency signals on credibility perceptions. Our rationale for this research question is that if known brands would be expected to lead to higher credibility perceptions overall (Hilligoss & Rieh, 2008; Masullo et al., 2022), as we predicted earlier, then it seems plausible that the potential effect of transparency signals in the Knowledge Panel would be amplified for known brands. Similarly, if, as we predicted, credibility perceptions would be higher overall among Germans based on their higher news trust (e.g., Newman et al., 2020), the possible influence of transparency signals might be amplified in an interaction effect.

RQ3: Will (a) country or (b) brand type moderate the influence of transparency signals in the Google search Knowledge Panel on credibility perceptions?

Method

Design and Procedures

We conducted two experiments, both with 2 (brand: known vs. unknown) by 8 (eight different Knowledge Panel configurations) between-subjects multifactorial designs. Our goal was to examine which of three transparency signals—background about the brand, audience engagement, or external evaluations—were most influential in increasing credibility perceptions. We examined each of these three signals alone and in combination with other signals and compared them to a control condition that just listed the name of the news site. This resulted in eight conditions (Table 1).

One experiment involved U.S. participants (n = 1,037)³ and was conducted in English, and one involved German participants (n = 1,000)⁴ and was conducted in German. The German and U.S. experiments were conducted separately, but the same conditions and survey questions were used, and data were ultimately merged. All researchers speak English, and one also speaks German. All survey materials and stimuli were professionally translated from English into German, and the German-speaking researcher reviewed the translation.

In both countries, participants were randomly assigned to exposure to one of eight configurations of a functional replica of the Knowledge Panel that provides information about a news site when people do a Google search for that site. They were also randomly assigned to be exposed to Knowledge Panel information about a known news site or about a mock news site that was created to resemble a real news site.⁵ In the U.S. experiment, the known news site was USA TODAY, chosen because it is among news sites trusted across the ideological spectrum (Jurkowitz et al., 2020). The mock site in the United States was The Gazette-Star, named and created by the researchers. For the German experiment, the known news site was Süddeutsche Zeitung⁶ because it is perceived as a quality publication that is ideologically central (Engesser et al., 2014). The mock German site was Allgemeines Tagblatt, which translates to “general daily newspaper,” named and created by the researchers.

Participants joined the survey on their own computers through a link. After consenting, they were exposed to the Knowledge Panel example they were assigned to with the prompt:

Imagine that after reading a news article online you wanted to know more about the news outlet that published the article. On the following page, you will see an example of what you might see if you searched for more information about the news outlet. Please take a look at the information carefully. You will be asked to answer questions about it.

Then they answered dependent measures explained below.

Stimuli Construction

Computer programmers hired by the researchers coded the stimuli to resemble the Knowledge Panel. In seven experimental conditions, information in the Knowledge Panel was clickable, so people could access more information, but in the eighth condition (the control) nothing was clickable because it only showed the news outlet name. For example, when a news outlet’s mission statement was mentioned in the Knowledge Panel, people could click on the words “mission statement” and an actual mission statement would show in a popup window (Figure 1). In most cases, the information for the signals in the Knowledge Panel was drawn from USA TODAY’s real news site or a composite of other sites, and it was the same in all the conditions (including for the known and unknown brands) to prevent differences between conditions from influencing results (Table 1). For the external evaluations category, we made up an evaluator called ProtectNews rather than using existing evaluators (e.g., the Trust Project) to guard against the influence of participants’ pre-existing attitudes toward real evaluators.

Figure 1.

Sample of stimuli.

Recruitment of Participants

Institutional Review Board approval was granted. Participants in both countries were recruited through Dynata, an online survey panel company that recruits potential survey participants through direct mail, online marketing, and targeted websites for marketing and academic research and rewards them with gift cards or other incentives. We employed a quota-sampling strategy to match demographic percentages for each country’s population. In the United States, we used percentages for gender, race/ethnicity, and age for the quotas, but in Germany, we used only gender and age because race and ethnicity are not collected or reported in Germany for historical reasons. Although our sample demographics (Table 2) do not perfectly match the population demographics, this is unproblematic because a representative sample is not required for experiments where successful random assignment is employed, although using quotas to attempt to mirror the population demographics enhances the representativeness of findings (Mutz, 2011). Furthermore, in both countries, we conducted a series of analyses to test that random assignment was successful. All analyses showed that participants were randomly distributed across all our experimental conditions, showing random assignment was successful. This underscores the fact that our lack of a perfectly representative sample is not problematic. Data collection for both experiments took place simultaneously in July and August 2020.

Table 2.

Sample Demographics.

	United States		Germany
Variable	n = 1,037	Internet population	n = 1,000	Population
Gender
Female	52.4%	49.0%	51.1%	51.0%
Male	47.0	51.0	48.4	48.0
Other	0.6	0.0	0.0	0.0
Race/ethnicity
White	74.7	70.0	—	—
Black	12.8	13.0	—	—
Mixed race/Other	5.3	12.0	—	—
Asian	7.2	5.0	—	—
Hispanic/Latino/Latina
Yes	15.1	15.0	—	—
No	84.9	85.0	—	—
Age
18–29	18.4	24.0	19.6	19.0
30–49	32.7	36.0	33.1	38.0
50–64	24.2	25.0	25.8	30.0
65 and above	24.7	15.0	21.5	13.0

Note. Race and ethnicity are not collected or reported in Germany for historical reasons.

Dependent Measures

News credibility

This measure was adapted from the literature (Strömbäck et al., 2020). On a 1 (strongly disagree) to 5 (strongly agree) scale, subjects rated their agreement or disagreement with the following: “[News site] seems like a trustworthy news source,” “I would rely on information from [news site],” “[News site] cares about its audience,” “[News site] is a legitimate news source,” “[News site] values feedback from its readers,” “[News site] is a respected news source,” “[News site] is fair when covering the news,” “[News site] is unbiased when covering the news,” “[News site] tells the whole story when covering the news,” “[News site] is accurate when covering the news,” and “[News site] separates facts from opinions when covering the news.” On a 1 (very unlikely) to 5 (very likely) scale, participants also rated how likely or unlikely they would be to “Seek out news from [News site],” “Read a story from [News site],” “Share an article from [News site],” “Click on an article from [News site] that appeared in a Google Search result,” or “Recommend news from [News site] to others” (G. M. Chen et al., 2017). In each case, the words “news site” were replaced with the name of the actual news outlet to which the participant was exposed. A principal component analysis⁷ with promax rotation suggested these items loaded on one factor in both countries. Items were averaged into indices for each country, both with high reliability, M_us= 3.33, SD_us = 0.91, Cronbach’s α_us = 0.96; M_Ger= 3.46, SD_Ger = 0.77, Cronbach’s α_Ger = 0.94.

Clicks

The stimuli were designed so that they captured every click stroke to reach additional information (such as the actual ethics policy) that participants made on the stimuli. In the United States, 130 (12.5%) participants clicked on some part of the Knowledge Panel they were shown, while in Germany, 95 (9.5%) did.

Results

To test the hypotheses and research questions, a multifactorial ANOVA was conducted with the brand type (known vs. unknown), country (United States vs. Germany), and Knowledge Panel configuration (eight types) as between-subjects factors. Interactions were tested between country and Knowledge Panel configuration and between brand type and Knowledge Panel configurations. Across both countries, results showed support for H1, which predicted that credibility perceptions would be higher for known brands versus unknown brands, F (1, 2010) = 37.49 p < .001, η² = 0.02. Credibility perceptions of known brands were significantly higher (M = 3.58, SE = 0.03) than for unknown brands (M = 3.36, SE = 0.03).

RQ1 asked whether domain-level transparency signals in the Google search Knowledge Panel serve as heuristic cues of credibility perceptions. RQ2a probed which configuration (if any) of domain-level transparency signals was most effective in increasing credibility perceptions. The same ANOVA used to test H1 was used to answer RQ1 and RQ2a, and it showed no significant main effect, F (7,2010) = 0.79 p = .59, η² = 0.002,⁸ of the transparency signals on credibility perceptions (Table 1). This suggests the domain-level transparency signals do not operate as heuristic cues, answering RQ1, so no configuration was more or less effective, answering RQ2a.

A separate ANOVA was used to answer RQ2b, which asked whether any Knowledge Panel configuration would increase clicks. Across both countries, the ANOVA showed a main effect, F (7, 2013) = 9.12 p < .001, η² = 0.03, and some significant differences emerged when pairwise comparisons with a Sidak post hoc correction were examined. As shown in Table 1, people clicked on the configuration showing audience engagement items the most (M = 0.18, SE = 0.02), and that was significantly more than people who clicked on the external evaluations configuration (M = 0.06, SE = 0.02, p < .001) or the configuration that combined audience engagement and external evaluations (M = 0.08, SE = 0.02, p = .01), answering RQ2b. Participants also clicked significantly more on the configuration showing background about the brand (M = 0.15, SE = 0.02) than the configuration with external evaluations (p = .03), also answering RQ2b. It is worth noting that participants clicked significantly more on all configurations relative to the control condition, but that finding is not instructive because the control condition had nothing in it that was clickable.

In support of H2, credibility perceptions were significantly higher in Germany than in the United States, F (1, 2010) = 30.92 p < .001, η² = 0.01. Germans rated the news sites on average at 3.59 (SE = 0.03) compared with the Americans’ average rating of 3.35 (SE = 0.03). A significant interaction between country and Knowledge Panel configuration was significant, F (7, 2010) = 2.09 p = .04, η² = 0.01, offering some evidence that country heightened the influence of which Knowledge Panel configuration was most effective at increasing credibility perceptions, answering RQ3a. However, pairwise comparisons with a Sidak post hoc correction showed no significant differences between any groups, so this finding should be interpreted with extreme caution. An interaction between brand type and Knowledge Panel configuration was not significant, F (7, 2010) = 1.83 p = .08, η² = 0.01, showing brand type did not heighten the influence of Knowledge Panel configurations on credibility perceptions, answering RQ3b.

Overall, our results offer some instructive findings. In both countries, credibility perceptions were higher for known news brands, over unknown, and credibility perceptions were higher in Germany than in the United States. None of the domain-level transparency signals we tested increased credibility perceptions, suggesting they do not operate as heuristic cues, at least the way we tested them. However, based on patterns of what people clicked on in the Knowledge Panels, people seemed to favor the audience engagement and background about the brand transparency signals.

Discussion

This study manipulated what transparency signals appeared in a functional replica of a Google Knowledge Panel that comes up when people search for a news outlet to test a new way to use disclosure transparency to heighten news credibility perceptions. We examined whether being transparent by disclosing information about a news outlet at the domain level—alongside information about the outlet itself in the Knowledge Panel—might be effective for increasing news credibility perceptions. Our main takeaway is that cuing news credibility perceptions at the domain level did not work, at least not in the way we employed it. We found people clicked more on Knowledge Panel configurations that offered background about the brand or ways to engage with the news outlet, suggesting some interest in that content, although overall clicks were infrequent. Yet, we found little evidence that transparency signals placed in the Knowledge Panel worked as heuristic cues that indicate to the public that they should perceive a news outlet as credible.

There are theoretical, conceptual, and methodological reasons why the transparency signals we tested may not have had the hypothesized effect. Theoretically, it may be that the whole premise of transparency signals cuing credibility perceptions is flawed. People may take a more holistic approach to assess credibility that is not depending on something as discrete as our signals. Some support for this notion comes from earlier studies (e.g., Karlsson & Clerwall, 2018; Karlsson et al., 2014; Peacock et al., 2022; Tandoc & Thomas, 2017) that showed transparency did not consistently cue news credibility perceptions despite some limited success (Curry & Stroud, 2021; Masullo et al., 2022; Peifer & Meisinger, 2021). It is also possible that transparency signals in a Knowledge Panel did not affect credibility perceptions because participants lacked news literacy—the knowledge needed to make sense of what news media they consume (Maksl et al., 2015)—so they could not effectively notice or process the cues embedded in the Knowledge Panel. This is particularly possible because media literacy interventions have been shown to increase news credibility perceptions (Vraga et al., 2009). Furthermore, as Strömbäck et al. (2020) point out, what ultimately matters is whether people trust the information that the media provides, so Google Knowledge Panel transparency signals can only do so much if public trust in the media is broken. Future research should take a broader focus on mending the fractured trust between the public and the media through news literacy education and facilitating engagement between journalists and news consumers.

Second, our conceptual definition of transparency may be problematic. The signals may not have had the intended effect because the public does not connect signals in a Knowledge Panel with the concept of a news outlet being transparent. Future research should pair the Knowledge Panel with a news story from that site to see if that would enhance people’s connection between the news outlet and the transparency signals and, as a result, cue credibility perceptions. It also may be that the specific transparency signals we tested—background about the brand, audience engagement, and external evaluations—are not meaningful to how the news audience shapes credibility perceptions. Other potential signals that deserve testing are recommendations or endorsements, like those on Netflix (Limov, 2020) or Amazon.com. This type of signal might read “This news site was recommended for you based on your previous news searches,” or “People who frequent this news site also frequent the following other sites.” These two signals should be tested in future research, and qualitative research, such as interviews and focus groups, would be useful in surfacing additional signals to test.

Finally, several methodological changes should be considered. The artificial nature of experiments in general and our use of fictitious stimuli (Gross et al., 2019) may have contributed to our null findings. Although we designed the Knowledge Panels to be interactive and to resemble real ones, people may have responded differently to transparency signals in a Knowledge Panel that was indeed real. Future research would benefit from a field experiment conducted in a real-world setting (e.g., Gerber, 2011). Furthermore, we designed the Knowledge Panel to resemble the real ones on Google, but the panel itself may need to be more prominent—using bigger type or bright colors—to get people to notice it enough for it to have an effect. Some support for this rationale comes from research that showed only prominent story labels were noticed (Peacock et al., 2022).

Our participants viewed the Knowledge Panel briefly once, so future experiments should show the panel multiple times to increase potential effects. Also, our three transparency signals each contained multiple indicators grouped within them. The brand background signal, for example, contained seven indicators, such as a mission statement, a revenue source, and an ethics policy. It may be that one or more of those indicators could have influenced credibility perceptions, but our design did not allow for parsing the effect of any particular indicator because it tested the effect of each signal collectively. We recommend a conjoint experimental design (Hainmueller et al., 2014) for future research that would allow testing of each indicator individually with an estimate of its relative importance compared with every other indicator, rather than grouping indicators into three signals. Finally, there is great merit as well in considering how political beliefs shape credibility perceptions in regard to transparency signals, as conservatism is linked to lower news trust in general (Mourão et al., 2018).

However, our finding still offers clear guidance for both scholars and practitioners. Clearly, transparency is not the solution for news distrust, despite its popularity among practitioners (e.g., Craft & Heim, 2009). Both practitioners and scholars should consider other avenues for boosting news credibility perceptions, such as news literacy (Vraga et al., 2009), encouraging journalists to engage more with the public (Lewis et al., 2014), or ombudsman columns where journalists speak directly to their audience in a conversational manner (Craft & Heim, 2009).

It is also worth noting that even if transparency does not move the needle on credibility perceptions, it may have other benefits. Our findings showed that people were more likely to pay attention to the signals that offered background about the brand or information about engaging with journalists, so that suggests they find something valuable about these signals, even if they do not cue credibility. Other benefits of transparency unrelated to credibility that scholars have found are that the public generally views transparency items positively and appreciates them (Karlsson & Clerwall, 2018). Also, transparency may lead to perceptions that a story is well written, interesting, and fair (Karlsson et al., 2014). Future research should test these possible benefits of transparency, as well as consider others, such as making the news more engaging, enhancing the clarity of storytelling, and increasing learning and retention from news.

We also made notable contributions to journalism research by offering evidence of two factors that can contribute to increased news credibility perceptions. Specifically, we found that known news brands were seen as more credible in both the United States and Germany, suggesting that the name of a news brand operates as a heuristic cue, adding to our understanding of how people process online news. Against this backdrop, our finding is important as it suggests that people pay attention to the news brand, at least in some cases, so making the name highly visible is advisable. In addition, we found that the way people process credibility perceptions might differ across cultures, media systems, or in countries with higher baselines of news trust. Notably, we found that Germans had higher credibility perceptions, relative to Americans, experimentally confirming what has been shown in survey research (Newman et al., 2020). This suggests that methods to cue news credibility may need to differ in varying cultural contexts.

Our study is limited because, like all experiments, it provides a snapshot in time. Longitudinal research is needed. In addition, relatively few participants clicked on interactive portions of the Knowledge Panel, suggesting more research is needed to understand why. Did they not click because they were not paying enough attention or because they were not interested in those signals? Finally, we considered two countries, the United States and Germany, with differing levels of media trust and varying media and political cultures, but future research should test the potential of domain-level transparency signals in other countries.

Our findings suggest that disclosure transparency at the domain level, at least the way we employed it, shows less merit for increasing news credibility perceptions than transparency at the story level. This suggests transparency alone will not solve news distrust. Notably, we did find that known news brands may operate as heuristic cues of news credibility perceptions and that news trust varies across cultural contexts.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Google News Project funded this study through a contribution to the Center for Media Engagement (CME) in the Moody College of Communication at The University of Texas at Austin. The authors thank CME Director Natalie J. (Talia) Stroud for her feedback on the project design and the CME team members for their suggestions on an earlier draft of this paper.

ORCID iDs

Gina M. Masullo

Taeyoung Lee

Martin J. Riedl

Notes

Author Biographies

Gina M. Masullo (PhD, Syracuse University, USA) is Associate Director of the Center for Media Engagement and an Associate Professor in the School of Journalism and Media, both at The University of Texas at Austin, USA. Her research focuses on how the digital space both connects and divides people and how that influences society, individuals, and journalism. She is the author of Online Incivility and Public Debate: Nasty Talk and The New Town Hall: Why We Engage Personally with Politicians and co-editor of Scandal in a Digital Age. She spent 20 years as a newspaper journalist before becoming a professor.

Taeyoung Lee (MA, Indiana University, USA) is a doctoral candidate in the School of Journalism and Media, at the University of Texas at Austin, USA. Her research focuses on how political misinformation, political satire, and journalism contribute to fostering, reinforcing, or undermining democratic values.

Martin J. Riedl (MA, Hanover University of Music, Drama and Media, Germany; M.A., Humboldt University of Berlin, Germany) is a doctoral candidate in the School of Journalism and Media, as well as a research associate for the Center for Media Engagement and the Technology and Information Policy Institute, all at the University of Texas at Austin, USA. He is a post-doctoral fellow at CME’s propaganda research lab. His research interests include social media, digital journalism, internet governance, disinformation, and content moderation.

References

Austin

E. W.

Dong

(1994). Source v. content effects on judgments of news believability. Journalism Quarterly, 71(4), 973–983.

Brüggemann

Engesser

Büchel

Humprecht

Castro

(2014). Hallin and Mancini revisited: Four empirical types of western media systems. Journal of Communication, 64, 1037–1065. https://doi.org/10.1111/jcom.12127

Callister

T. A.

Jr. (2000). Media literacy: On-ramp to the literacy of the 21st century or cul-de-sac on the information superhighway. Advances in Reading/Language Research, 7, 403–420.

Chaiken

(1980). Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39(5), 752–766.

Chen

G. M.

Chen

P. S.

Chang

Abedin

(2017). News video quality affects online sites’ credibility. Newspaper Research Journal, 38(1), 19–31.

Chen

Wells

W. D.

(1999). Attitude toward the site. Journal of Advertising Research, 39(5), 27–37.

Chen

V. Y.

Chen

G. M.

(2020). Shut down or turn off? The interplay between news overload and consumption. Atlantic Journal of Communication, 28(2), 125–137. https://doi.org/10.1080/15456870.2019.1616738

Coleman

J. S.

(1990). Foundations of social theory. Harvard University Press.

Collins

N. L.

Miller

L. C.

(1994). Self-disclosure and liking: A meta-analytic review. Psychological Bulletin, 116(3), 457–475.

10.

Craft

Heim

(2009). Transparency in journalism: Meanings, merits, and risks. In Wilkins

Christians

C. G.

(Eds.), The handbook of mass media ethics (pp. 217–228). Routledge.

11.

Cummins

R. G.

Chambers

(2011). How production value impacts perceived technical quality, credibility, and economic value of video news. Journalism & Mass Communication Quarterly, 88(4), 737–752.

12.

Curry

A. L.

Stroud

N. J.

(2021). The effects of journalistic transparency on credibility assessments and engagement intentions. Journalism, 22(4), 901–918. https://doi.org/10.1177/1464884919850387

13.

Edgerly

Vraga

E. K.

(2019). The blue check of credibility: Does account verification matter when evaluating news on Twitter? Cyberpsychology, Behavior, and Social Networking, 22(4), 283–287.

14.

Engesser

Esser

Reinemann

Scherr

Matthes

Wonneberger

(2014). Negativität in der Politikberichterstattung [Negativity in political reporting]. Medien & Kommunikationswissenschaft, 62(4), 588–605. https://doi.org/10.5771/1615-634x-2014-4-588

15.

Gass

R. H.

Seiter

J. S.

(1999). Persuasion, social influence, and compliance gaining. Allyn & Bacon.

16.

Gaziano

McGrath

(1986). Measuring the concept of credibility. Journalism Quarterly, 63, 451–462.

17.

Gerber

A. S.

(2011). Field experiments in political science. In Druckman

Green

D. P.

Kuklinski

J. H.

Lupia

(Eds.), Handbook of experimental political science (pp. 115–140). Cambridge University Press.

18.

Granow

Jackob

Ziegele

Quiring

Schemer

Schultz

(2020). Interpersonales Vertrauen als Prädiktor für Medienvertrauen: Befunde der Mainzer Langzeitstudie [Interpersonal trust as a predictor for media trust: Evidence from the Mainz longitudinal study]. In Blome

Eberwein

Averbeck-Lietz

(Eds.), Medienvertrauen. Historische und aktuelle Perspektiven [Media trust. Historical and current perspectives] (pp. 95–116). De Gruyter. https://doi.org/10.1515/9783110590470-006

19.

Gross

Porter

Wood

T. J.

(2019). Identifying media effects through low-cost, multiwave field experiments. Political Communication, 36(2), 272–287.

20.

Hainmueller

Hopkins

D. J.

Yamamoto

(2014). Causal inference in conjoint analysis: Understanding multidimensional choices via stated preference experiments. Political Analysis, 22(1), 1–30. https://doi.org/10.1093/pan/mpt024

21.

Hanitzsch

Van Dalen

Steindl

(2018). Caught in the nexus: A comparative and longitudinal analysis of public trust in the press. The International Journal of Press/Politics, 23(1), 3–23. https://doi.org/10.1177/1940161217740695

22.

Henke

Leissner

Möhring

(2020). How can journalists promote news credibility? Effects of evidences on trust and credibility. Journalism Practice, 14(3), 299–318. https://doi.org/10.1080/17512786.2019.1605839

23.

Hilligoss

Rieh

S. Y.

(2008). Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context. Information Processing & Management, 44(4), 1467–1484.

24.

Holton

A. A.

Chyi

H. I.

(2011). News overload and the overloaded consumer. Cyberpsychology, Behavior and Social Networking, 15(11), 619–624.

25.

Hovland

C. I.

Janis

I. L.

Kelley

(1959). Communication and persuasion. Psychological studies of opinion change. Yale University Press.

26.

Johnson

T. J.

Kaye

B. K.

(2014). Credibility of social network sites for political information among politically interested internet users. Journal of Computer-Mediated Communication, 19(4), 957–974.

27.

Jurkowitz

Mitchell

Shearer

Walker

(2020, January 24). U.S. Media polarization and the 2020 election: A nation divided. Pew Research Center. https://www.journalism.org/2020/01/24/u-s-media-polarization-and-the-2020-election-a-nation-divided/

28.

Karlsson

(2010). Rituals of transparency. Journalism Studies, 11(4), 535–545. https://doi.org/10.1080/14616701003638400

29.

Karlsson

Clerwall

(2018). Transparency to the rescue? Journalism Studies, 19(13), 1923–1933. https://doi.org/10.1080/1461670X.2018.1492882

30.

Karlsson

Clerwall

Nord

(2014). You ain’t seen nothing yet: Transparency’s (lack of) effect on source and message credibility. Journalism Studies, 15, 668–678. https://doi.org/10.1080/1461670X.2014.886837

31.

Karlsson

Clerwall

Nord

(2017). Do not stand corrected: Transparency and users’ attitudes to inaccurate news and corrections in online journalism. Journalism & Mass Communication Quarterly, 94(1), 148–167.

32.

Knight Foundation. (2020, November 9). American views 2020: Trust, media, and democracy. https://knightfoundation.org/wp-content/uploads/2020/08/American-Views-2020-Trust-Media-and-Democracy.pdf

33.

Knobloch-Westerwick

Mothes

Johnson

B. K.

Westerwick

Donsbach

(2015). Political online information searching in Germany and the United States: Confirmation bias, source credibility, and attitude impacts. Journal of Communication, 65(3), 489–511. https://doi.org/10.1111/jcom.12154

34.

Lewis

S. C.

Holton

A. E.

Coddington

(2014). Reciprocal journalism: A concept of mutual exchange between journalists and audiences. Journalism Practice, 8(2), 229–241.

35.

Limov

(2020). Click it, binge it, get hooked: Netflix and the growing U.S. audience for foreign content. International Journal of Communication, 14, 6304–6323.

36.

Maksl

Ashley

Craft

(2015). Measuring news media literacy. Journal of Media Literacy Education, 6(3), 29–45.

37.

Masullo

G. M.

Curry

A. L.

Whipple

K. N.

Murray

(2022). The story behind the story: Examining transparency about the journalistic process and news outlet credibility. Journalism Practice, 16(7), 1287–1305. https://doi.org/10.1080/17512786.2020.1870529

38.

Meeks

(2020). Defining the enemy: How Donald Trump frames the news media. Journalism & Mass Communication Quarterly, 97(1), 211–234. https://doi.org/10.1177/1077699019857676

39.

Metzger

M. J.

Flanagin

A. J.

(2008). Introduction. In Metzger

Flanagin

(Eds.), Digital media, youth, and credibility (pp. 1–4). MIT Press.

40.

Metzger

M. J.

Flanagin

A. J.

(2013). Credibility and trust of information in online environments: The use of cognitive heuristics. Journal of Pragmatics, 59, 210–220.

41.

Metzger

M. J.

Flanagin

A. J.

Eyal

Lemus

D. R.

McCann

R. M.

(2003). Credibility for the 21st century: Integrating perspectives on source, message, and media credibility in the contemporary media environment. Annals of the International Communication Association, 27(1), 293–335.

42.

Mitchell

Simmons

Matsa

K. E.

Silver

(2018, January). Publics globally want unbiased news coverage, but are divided on whether their news media deliver. Pew Research Center. https://www.pewresearch.org/global/2018/01/11/publics-globally-want-unbiased-news-coverage-but-are-divided-on-whether-their-news-media-deliver/

43.

Mor

Reich

(2018). From “trust me” to “show me” journalism. Journalism Practice, 12(9), 1091–1108.

44.

Mourão

R. R.

Thorson

Chen

Tham

S. M.

(2018). Media repertoires and news trust during the early Trump administration. Journalism Studies, 19(13), 1945–1956. https://doi.org/10.1080/1461670X.2018.1500492

45.

Mutz

D. C.

(2011). Population-based survey experiments. Princeton University Press.

46.

Mutz

D. C.

Pemantle

(2015). Standards for experimental research: Encouraging a better understanding of experimental methods. Journal of Experimental Political Science, 2(2), 192–215.

47.

Newman

Fletcher

Schulz

Andı

Nielsen

R. K.

(2020). Digital news report 2020. Reuters Institute. https://www.digitalnewsreport.org/

48.

Peacock

Masullo

G. M.

Stroud

N. J.

(2022). The effect of news labels on perceived credibility. Journalism, 23(2), 301–319. https://doi.org/10.1177/1464884920971522

49.

Peifer

J. T.

Meisinger

(2021). The value of explaining the process: How journalistic transparency and perceptions of news media importance can (sometimes) foster message credibility and engagement intentions. Journalism & Mass Communication Quarterly, 98, 828–853. https://doi.org/10.1177/10776990211012953

50.

Revers

(2017). Contemporary journalism in the U.S. and Germany: Agents of accountability. Palgrave Macmillan.

51.

Rieh

S. Y.

Danielson

D. R.

(2007). Credibility: A multidisciplinary framework. In Cronin

(Ed.), Annual review of information science and technology (Vol. 41, pp. 307–364). Information Today.

52.

Strömbäck

Tsfati

Boomgaarden

Damstra

Lindgren

Vliegenthart

Lindholm

(2020). News media trust and its impact on media use: Toward a framework for future research. Annals of the International Communication Association, 44(2), 139–156. https://doi.org/10.1080/23808985.2020.1755338

53.

Tandoc

E. C.

Thomas

R. J.

(2017). Readers value objectivity over transparency. Newspaper Research Journal, 38(1), 32–45. https://doi.org/10.1177/0739532917698446

54.

Tenenboim

Cohen

A. A.

(2015). What prompts users to click and comment: A longitudinal study of online news. Journalism, 16(2), 198–217. https://doi.org/10.1177/1464884913513996

55.

Tseng

Fogg

B. J.

(1999). Credibility and computing technology. Communications of the ACM, 42(5), 39–44.

56.

Van Der Wurff

Schönbach

(2011). Between profession and audience: Codes of conduct and transparency as quality instruments for off-and online journalism. Journalism Studies, 12(4), 407–422.

57.

Victoria-Mas

Lacasa-Mas

Marimon

(2018). Assessing the consumer-based brand equity of news media firms: A new validated scale. Journal of Media Business Studies, 15(3), 214–235. https://doi.org/10.1080/16522354.2018.1522199

58.

Vraga

Tully

Rojas

(2009). Reducing hostile media perceptions: Can news media literacy reduce perceptions of bias? Newspaper Research Journal, 30(4), 68–81.

59.

Wheeless

L. R.

Grotz

(1977). The measurement of trust and its relationship to self-disclosure. Human Communication Research, 3(3), 250–257.