Who Fact-Checks and Does It Matter? Examining the Antecedents and Consequences of Audience Fact-Checking Behaviour in Hong Kong

Abstract

This study investigated the ways in which people engaged in fact-checking in a highly divided context—the Anti-Extradition Bill Movement (AEBM) in Hong Kong. A telephone survey representative of the Hong Kong population was conducted in 2020 (N = 1,004). The findings showed that males with greater news consumption and issue involvement were more likely to engage in fact-checking behavior. Nevertheless, the effects of fact-checking appeared mixed. We first found that fact-checking behavior reduced belief in disagreeable misinformation only for supporters of the AEBM. More robust evidence showed that frequent fact-checking behavior reinforced, rather than reduced, partisans’ belief in misinformation regarding the opponent group. A warning of the backfire effects of fact-checking on exacerbating opinion polarization and social division is issued.

Keywords

anti-extradition bill movement backfire effects fact checking Hong Kong opinion polarization

Fake news and misinformation have proliferated rapidly in today's media environment and have prompted global concern (for a review, see Haciyakupoglu et al. 2018). Although news organizations and dedicated fact checkers have started to use fact-checking as a combating strategy, the effectiveness of these professionals’ efforts to reduce media consumers’ beliefs in misinformation essentially relies on media consumers’ own fact-checking behavior. Only when media consumers take the initiatives to seek verification or facts regarding received information can fact-checks conducted by professionals or other sources reach and influence them (Walter et al. 2021). There is a need to investigate who actively engages in fact-checking, where they look for facts, and whether they are better informed than others. Answers to these questions are particularly important in a highly divided society, where misinformation and disinformation precipitate misunderstandings and even antagonism between different factions. Whether fact-checking can effectively reduce misperceptions and promote social reconciliation in such a context needs to be elucidated.

Extant studies concerning media consumers’ fact-checking behavior and its consequences have predominantly focused on the use of dedicated fact-checking websites and exposure to the information they release. Only very few studies have investigated verification-seeking behavior or the search for factual information from other sources (e.g., Edgerly et al. 2020; Lee and Ramazan 2021; Yu and Shen 2021). Although this exclusive research focus echoes the recently growing interest in dedicated fact-checking websites, it has several limitations. First, only internet users are included in the examination. How non-users or light users of internet verify received news information remains unknown. Second, although online fact-checking organizations have received substantial attention, only a small number of people are familiar with their work or have visited their websites (Gottfried et al. 2013; Nyhan and Reifler 2015). Some people even accuse these organizations of being biased (Shin and Thorson 2017). It is necessary to know what additional means or channels people use to verify or fact-check information. Lastly, people have reported that they seek facts or verify information using personal networks or various media platforms (Bode and Vraga 2015; Godes and Mayzlin 2009). Investigating how these other information sources are used for fact-checking allows us to develop a more comprehensive understanding of people's behavioral responses to media content in the so-called post-truth era.

Including a broad range of fact-checking behavior in investigations is also critical for evaluating the effects of fact-checking on reducing people's misperceptions and cultivating well-informed citizens. To date, most studies that investigated the effects of fact-checks utilized experimental designs (e.g., Fridkin et al. 2015; Nyhan and Reifler 2015; Wood and Porter 2019) and exposed participants to messages from fact-checking sites that they seldom used (Shin and Thorson 2017). They did not consider the spontaneous fact-checking or verifying behavior that people might engage in using interpersonal associations or multiple media channels. The external validity of the findings was compromised. In particular, people typically receive more homogeneous views through interpersonal communication than from media channels or dedicated fact-checking websites (Mutz and Martin 2001). The information obtained through various types of fact-checking behaviors might be complementary or contradictory.

Therefore, this study utilizes a survey with a representative sample to investigate the various fact-checking behaviors of individuals and their potential effects on a highly divided society, where media information is likely to be manipulated, misrepresented, or even fabricated by opposing factions. Media consumers who are aware of this possibility may engage in fact-checking behavior. To fully grasp the antecedents and consequences of such behavior, we propose and evaluate a theoretical framework built on an in-depth explication of media consumers’ fact-checking behavior and the theory of motivated political reasoning (Taber and Lodge 2006). Our objectives are to theorize media consumers’ fact-checking behavior and to identify the attributes of fact seekers. To uncover the effects of fact-checking, we also examine whether people who engage in fact-checking behavior more frequently report less belief in misinformation. Our departure point is the growing literature regarding individuals’ reception of and reactions to fact-checking information, which has rapidly accumulated in the last decade (see Walter et al. 2020).

Anti-Extradition Bill Movement in Hong Kong

Unlike previous studies confined to the United States, this study extends the examination to Hong Kong, which was returned to China as a special administrative region (HKSAR) in 1997 after being a British colony for about 150 years (Tsang 2004). This historical background has led to divided opinions on issues related to politics and national identity in this harbor city (Lee 2016). There is the pro-Beijing camp that pledges patriotism to China and supports the notion that the HKSAR government should be designated by Beijing. Conversely, the pro-democracy camp advocates Hong Kong's identity and autonomy (Lo 2017). The political divide manifested in 2019, when the HKSAR government proposed amending its extradition law to empower itself to transfer suspected criminals among Hong Kong residents or visitors to mainland China, thus subjecting them to China's authority and legal system (Ku 2020). The pro-democracy camp interpreted the proposal as a severe violation of Hong Kong's autonomy and held a series of street protests, dubbed the Anti-Extradition Bill Movement (AEBM). The protests continued for months and escalated into violent and brutal confrontations between police and protesters.¹ The city was awash with rumors and misinformation, including reports that protesters were engaging in unethical behavior (e.g., attending protests for monetary reward) or that police were committing irregularities (e.g., using excessive and unnecessary violence against protesters). Worrying that unsubstantiated claims might further divide the city, social scientists and observers urged citizens to check facts and verify the information.² Independent websites and organizations dedicated to fact-checking also started to gain traction (e.g., https://www.facebook.com/kauyim/ or https://www.factchecklab.org/, see Feng et al. 2021). We seized this opportunity to investigate the fact-checking behaviors exercised by citizens in Hong Kong and examine whether the behaviors lessened their belief in baseless rumors and unfounded claims. This study's significance lies in its attempt to identify the attributes of people who actively perform fact-checking and provide insights into whether media consumers’ fact-checking behavior can minimize misperceptions and facilitate reconciliation in a highly divisive political and media context outside the US.

Media Consumers’ Fact-Checking Behavior

Similar to professional fact-checking, media consumers’ fact-checking behavior is based on normative ideals about seeking the truth and preventing the spread of mis- or dis-information. However, their approach differs from professional practice in three respects. First, professionals choose to fact-check news stories or political claims that are suspected to be false, whereas media consumers tend to verify media information that they consider true or congruent with their ideological leanings (Edgerly et al. 2020; Graves 2016). Walter et al. (2021) attributed this tendency to directionally motivated reasoning; that is, people tend to assume that their beliefs are accurate and that they are motivated to avoid incongruent information. They are reluctant to fact-check information if they think the results may contradict their positions. They are motivated or even eager to seek verification only when they believe that their beliefs will be validated.

Second, professional fact-checkers usually go through a rigorous process to corroborate information, starting by contacting the person who makes the seemingly erroneous claims and moving on to searching for original reports or official data for validation. They also interview experts or other sources to triangulate the data (Graves 2016). By comparison, media consumers have limited time and resources. They normally seek verification from commonly available sources, such as other media consumers in their social networks. Previous research revealed that when verifying information quality, people pay attention not only to the author of the information but also to others who react to the information (Kim et al. 2014). They may share rumors with others in an attempt to verify the rumors’ truthfulness (Oh and Lee 2019). They may also verify facts by comparing information across different media channels, including television, newspapers, and social media (Lee and Ramazan 2021). In fact, media literacy programs have been advocating these fact-checking behaviors since fake news have become a popular concern (e.g., ‘How to Spot Fake News’, International Federation of Library Associations and Institutions, 2020). Lastly, people visit the websites of dedicated fact-checking organizations, which play a key role in debunking erroneous or misleading information in today's media (Robertson et al. 2020). Although people at times also consider additional reference materials, such as scientific reports (Lee and Ramazan 2021), these efforts are less prevalent for verifying political information given that reference materials in political arena (e.g., government reports or meeting minutes) are rarely available to media consumers. Therefore, examinations of media consumers’ fact-checking shall investigate the behavior of seeking verification from personal networks, multiple media channels, and dedicated fact-checking websites (Edgerly et al. 2020; Yu and Shen 2021).

The final difference between professionals and media consumers lies in their efforts to remain neutral and balanced. Professional fact-checkers strive to gather and scrutinize all evidence in an even-handed and bias-free manner because they are aware that they will be held accountable (Graves 2016). By contrast, media consumers are motivated to maintain their existing beliefs. This directional motivation leads them to expose themselves consciously or unconsciously to congruent information and to avoid incongruent information. This selective exposure can result in confirmation bias (Stroud 2008; Taber and Lodge 2006). Hameleers and van der Meer (2020) found that media consumers tend to attitudinally select congruent fact checkers when verifying counter-attitudinal information, and attitudinally avoid incongruent fact checkers after exposure to pro-attitudinal information. Shin and Thorson (2017) also found that people tended to share fact-checks that affirmed, rather than challenged, their partisan stances.

In summary, media consumers’ fact-checking is far from ideal and is subject to motivated reasoning, limited resources, selective exposure, and confirmation bias. Media consumers usually seek information verification from personal networks, multiple media channels, and dedicated fact-checking websites. These three sources are prone to varying degrees of bias. The use of dedicated fact-checking websites exposes media consumers to professional fact-checking, whereas evaluating media coverage across various media platforms may result in confirmation bias (Taber and Lodge 2006). Seeking verification from personal networks probably requires less effort and resources, but it can be detrimental if family and friends are misinformed. Therefore, the three fact-checking behaviors should be separately evaluated.

Who Engages in Fact-checking?

Despite possible errors and biases, the three fact-checking behaviors demonstrate individuals’ concerns about misinformation and their intentions to seek facts. Such concerns and intentions can be affected by demographics, although the findings are not consistent. For example, women were found to be less skeptical and to share more misinformation than men (Chen et al. 2015), but they reported more concerns about the harmful effects of misinformation than men (Almenar et al. 2021). Age was found to be positively correlated with the frequency of information verification activities for users of Facebook and Twitter (Nee 2019), but age and education were found to be negatively associated with the use of dedicated fact-checking sites (Gottfried et al. 2013). Moreover, once political interests and news consumption were accounted for, only age remained a significant predictor (Robertson et al. 2020). In light of these mixed results, we propose to test how demographics relate to each of the three fact-checking behaviors in Hong Kong.

Research Questions 1–3: How are demographic factors (i.e., age, gender, and education) each related to the frequency with which individuals in Hong Kong seek information verification from (1) personal networks, (2) multiple media outlets, and (3) dedicated fact-checking websites?

Individuals’ news consumption and issue involvement are also believed to prompt fact-checking behavior. Research has suggested that these two factors are related to the frequency with which people engage in political discussions with others (Scheufele 1999) or seek information from the media (Kim et al. 1999; Min 2010). These behaviors often involve information exchange and verification. In particular, exposure to information verification in the media, including dedicated fact-checking sites, is often ‘incidental and spontaneous’ (Robertson et al. 2020: 223). People who expose themselves to news information more frequently naturally have a greater chance of encountering fact-checking reports. Similarly, issue involvement refers to ‘a motivational state induced by an association between an activated attitude and the self-concept’ (Johnson and Eagly 1989: 290). People who are more involved in an issue may feel more emotional about the issue and be more motivated to act on their attitudes, such as talking with others or seeking facts in the media, including dedicated fact-checking websites (Gottfried et al. 2013; Robertson et al. 2020).

Political ideologies also predict fact-checking activities. Jang et al. (2014) found that college students with more liberal ideologies were more likely to discuss political issues with friends on Facebook. Through interpersonal discussions, the students sought facts and verification. Research in the US has also suggested that people who identify as Liberal or Democrat seek facts from various media channels and visit fact-checking websites more frequently than people who identify as Conservative or Republican (Gottfried et al. 2013; Robertson et al. 2020). One explanation is that the media channels and fact-checking sites in the US typically challenge more claims made by conservative Republicans (including former President Donald Trump) than by liberal Democrats, at least during the past four years, when the Republican party was in power. Citizens who ideologically lean toward Republicans develop greater distrust and hostile perceptions of the media (including fact-checking websites) than Democrats or people with liberal ideologies (Shin and Thorson 2017). Robertson et al. (2020) suggested that researchers should consider ideological orientation when examining citizens’ fact-checking behavior. Given that AEBM in Hong Kong was initiated by the pro-democracy camp to challenge the in-power pro-Beijing camp, we expect that citizens who identify with pro-democracy parties would be more likely to seek facts or news verification than people who identify with other political parties.

Hypotheses 1a–c: People's (a) consumption of AEBM-related news, (b) personal involvement in the issue of AEBM, and (c) identification with the pro-democracy camp are each positively associated with their fact-checking behavior.

Effectiveness of Fact-checking

The expectation that fact-checking behavior would reduce misconceptions is developed based on a relatively straightforward assumption that people form their beliefs based on information received from the environment. In normal circumstances, people engage in fact-checking behavior because they desire accurate information. Their beliefs in a claim are expected to diminish when they are told that the claim is unconfirmed or untrue. This logic, although somewhat simplistic, has been supported by encouraging evidence. A meta-analysis of 30 individual studies on the effectiveness of fact-checking showed that exposure to fact-checks significantly reduced misperceptions (Walter et al. 2020). Gottfried et al. (2013) also found that voters who used fact-checking websites during the 2012 presidential election in the US were more knowledgeable about election campaigns than non-users.

However, media consumers are typically unable to avoid the impact of their preconceptions, even when they are motivated to obtain accurate information. Taber and Lodge (2006) proposed the theory of motivated political reasoning to describe how media consumers seek confirmatory arguments from like-minded groups rather than opposing arguments from out-groups. Even when exposed to arguments from both sides, people tend to evaluate congruent evidence favorably and denigrate incongruent evidence. Such selective exposure and confirmation bias impact the consequences of media consumers’ fact-checking behaviors. Research has demonstrated that partisans tend to accept pro-attitudinal media information and reject counter-attitudinal information regardless of whether the information is verified or debunked by fact-checking reports (e.g., Jarman 2014). In some cases, exposure to fact-checks even increases beliefs in pro-attitudinal misinformation. Nyhan and Reifler (2010) called this consequence the ‘backfire effect’ of fact-checking. In particular, people tend to seek fact-checks that debunk counter-attitudinal misinformation and avoid fact-checks that correct pro-attitudinal misinformation (Hameleers and van der Meer 2020). Even when they are receptive to fact-checks, the effect of fact-checks in reducing media consumers’ beliefs about misinformation appears to be significantly weaker and less lasting when the misinformation is pro-attitudinal than when it is counter-attitudinal (Nyhan 2021; Walter et al. 2020). Consequently, media consumers’ fact-checking behavior might decrease their belief in counter-attitudinal misinformation but increase their belief in pro-attitudinal misinformation. We propose to test this hypothesis in the highly divided context of AEBM in Hong Kong.

Hypothesis 2a–b: The frequency of individuals’ fact-checking behavior is (a) negatively associated with their belief in counter-attitudinal misinformation but (b) positively associated with their belief in pro-attitudinal misinformation.

Across the three types of fact-checking behaviors, verifying information with personal networks is the least likely to introduce counter-attitudinal views because like-minded people tend to interact with one another. People who hold opposing views typically avoid discussing politics to evade conflict. Evidence of such human tendencies can be found in research work regarding the spiral of silence theory (Noelle-Neumann 1995) or echo chambers (Sunstein 2009). By comparison, the media content that people receive is more heterogeneous. Even when people habitually use only one or two media channels/platforms, there are variances across programs within the channels/platforms. Mutz and Martin (2001) examined the media content available to citizens and surveyed individuals’ political discussants. They found more diversity in the political views presented in the media than in the views presented in interpersonal associations. Fact-checks through personal networks are necessarily more confirmatory than fact-checks obtained from media, including dedicated fact-checking websites. We, therefore, expect that fact-checking using personal networks would more likely minimize belief in disconfirming, counter-attitudinal misinformation than fact-checking with multiple media channels or dedicated fact-checking websites. We also expect that fact-checking with personal networks would more likely increase belief in pro-attitudinal misinformation than fact-checking using multiple media channels or fact-checking websites.

Hypothesis 3a Fact-checking using personal networks leads to weaker belief in counter-attitudinal misinformation than fact-checking using multiple media channels or dedicated fact-checking websites.

Hypothesis 3b: Fact-checking using personal networks leads to stronger belief in pro-attitudinal misinformation than fact-checking using multiple media sources or dedicated fact-checking sites.

Methods

A computer-assisted dual-frame (landline and mobile) telephone interview survey with a random sample was conducted from late July to early August 2020 in Hong Kong. Except for ineligible and unqualified numbers (e.g., disconnected or fax), each telephone number was called at least three times. The next-birthday method was used to select one adult from each sampled household. Partial completes and mid-terminates were excluded. A total of 1,004 Hong Kong adults were successfully interviewed. The response rate was 33%, according to the American Association for Public Opinion Research (AAPOR) Response Rate 6 (AAPOR 2016).

Consistent with the population statistics in Hong Kong, 52.8% (n = 530) of the respondents were female, about half (n = 505) were 49 years old or younger, and 35% (n = 351) had received a college education or higher (Census and Statistics Department 2022a). However, only 10% of the respondents, as opposed to 18% of the Hong Kong population (Census and Statistics Department 2021), reported elementary school as their highest education. Further, the median family income of our respondents fell in the range of HK$30,000–39,999 per month, which was higher than that for the overall Hong Kong population (i.e., HK$27,700, Census and Statistics Department 2022b). The final sample was weighted based on the gender, age, and education of the Hong Kong population. Respondents were asked to report the information detailed below.

Consumption of AEBM-Related News

Using a five-point scale where one meant ‘never’ and 5 meant ‘very frequently’, respondents reported consuming AEBM-related news in professional news media (i.e., TV, newspapers, radio, or online news portals, M = 4.29, SD = 0.98) more frequently than on user-generated social media (including social media and online forums, M = 3.43, SD = 1.46). They appeared to have a greater reservation for news circulated on social media than stories reported by professional media (t(1,003) = 17.77, p < .001). These two items had a low correlation (r = 0.25, p < .001) and were analyzed separately.

Involvement with AEBM

Respondents reported their psychological involvement by indicating how ‘stirred up’ they felt when they thought about AEBM. Responses ranged from 1 = ‘not stirred up at all’ to 5 = ‘very stirred up’ (M = 3.56, SD = 1.16). They also reported how frequently they participated in AEBM-related street protests using a five-point scale ranging from ‘never’ to ‘very frequently’ to describe their behavioral involvement (M = 1.89, SD = 1.14). The two measures were lowly correlated and analyzed separately (r = 0.21, p < .001).

Political Identification

In Hong Kong, political parties are characterized according to their positions on the divide between pro-democracy and pro-Beijing groups (Wong 2020). Localist and the Pan-Democracy camps, which advocate for democracy in Hong Kong and oppose Beijing's interference, are viewed as pro-democracy groups, though the Pan-Democracy camp expressed more favorable attitudes toward immigrants from mainland China than the Localist camp (Lee 2020; So 2017). The pro-Beijing group is called the pro-establishment faction, which includes pro-China parties and organizations. Lastly, the Centrist camp includes parties that maintain a neutral position between the pro-democracy and pro-establishment camps (Lee 2016). In this study, we asked respondents to indicate which camp their political attitudes leaned toward. A dichotomous variable was created to code respondents who leaned toward the Localist (n = 186) or the Pan-Democracy camp (n = 269) as 1 and others who leaned toward the pro-establishment faction (n = 98), the Centrist camp (n = 260), or no specific camp (n = 171) as 0. Twenty respondents refused to indicate their political orientation.

Attitudes Toward AEBM

Using a scale that ranged from 1 = ‘strongly oppose’ to 5 = ‘strongly support’, respondents reported their support for the ‘five demands’ that were proposed by protesters during the AEBM: (1) The HKSAR government should formally withdraw the amended Extradition Bill (M = 3.77, SD = 1.30); (2) the HKSAR government should withdraw criminal charges against all AEBM protestors (M = 3.33, SD = 1.34); (3) the HKSAR government should retract the proclamation that AEBM was a riot (M = 3.62, SD = 1.35); (4) an independent investigation committee should be established to investigate the police's abuse of power during AEBM (M = 4.18, SD = 1.19); (5) universal suffrage should be immediately implemented for the Chief Executive and Legislative Council elections³ (M = 4.06, SD = 1.14). The five items were averaged to form a composite measure of attitudes toward AEBM (α = 0.93, M = 3.84, SD = 1.09). Higher scores indicated more favorable attitudes.

Fact-Checking Behavior

Measures were adopted from previous studies (Edgerly et al. 2020; Yu and Shen 2021). Respondents indicated the frequency with which they engaged in the following three behaviors using options ranging from ‘1 = never’ to ‘5 = very frequently’: (a) verify AEBM-related information with family or friends, (b) make efforts to verify the AEBM-related information using multiple media channels, and (c) visit dedicated fact-checking websites to verify AEBM-related information. Only 7% of the respondents reported that they never engaged in any type of fact-checking behavior to verify AEBM-related information. Although the three items were inter-correlated (r_ab = .46, p < .001; r_bc = .68, p < .001; r_ac = .47, p < .001), we analyzed the three items separately because of their different nature.

Misperceptions of AEBM

Respondents were asked to report their beliefs in six widely circulated AEBM-related claims that were revealed to be unfounded or erroneous.⁴,⁵,⁶,⁷ Responses ranged from 1 = ‘absolutely untrue’ to 5 = ‘absolutely true’. Three of the claims cast protesters in a negative light and were combined to form beliefs regarding anti-protester misinformation (α = 0.80, M = 2.27, SD = 1.09): (1) The mob attack against protesters at the Yuen Long subway station on July 21, 2019, which caused severe injuries among protesters, was actually provoked by a pro-Democracy (pro-protester) legislator (M = 2.15, SD = 1.33); (2) protesters were offered free sex with young and innocent girls (M = 2.01, SD = 1.10); (3) protesters were paid to attend the AEBM (M = 2.68, SD = 1.42). The other three items that scandalized the HKSAR government or police formed a composite measure of beliefs in anti-police misinformation (α = 0.86, M = 3.15, SD = 1.22): (1) protesters were beaten to death by the police at the Prince Edward subway station on the night of 31 August 2019 (M = 2.80, SD = 1.35); (2) the police infiltrated the protesters and engaged in activities such as throwing petrol bombs at people or vandalizing subways to make the protesters appear violent and destructive (M = 3.57, SD = 1.43); (3) several suicide cases reported across Hong Kong were actually murder cases in which protesters were killed by the police (M = 3.04, SD = 1.38).

Results

RQ1–3 and H1a–c aimed to find predictors of fact-checking behavior. Three sets of hierarchical linear regression analyses were performed. For each set of analyses, demographics, news consumption, political identification, and involvement with AEBM were entered as the independent variables. The results in Table 1 suggest that males engaged in fact-checking using multiple media channels and fact-checking websites more frequently than females. Age was negatively associated with all three fact-checking behaviors, but the negative associations became non-significant when media consumption, political orientation, and issue involvement were accounted for. Similarly, education level and family income were positively associated with one or more fact-checking behaviors, but only family income remained positively associated with the use of dedicated fact-checking websites after media consumption, political orientation, and issue involvement were entered into the analyses (see Table 1).

Table 1.

Standardized Linear Regression Coefficients (with Standard Errors in Parentheses) Predict Three Types of Fact-checking Behaviors.

	Fact-checking with personal networks		Fact-checking with multiple media channels		Fact-checking with fact-checking websites
Gender (Male = 1, Female = 2)	.02 (.08)	.04 (.07)	−.12*** (.08)	−.11***(.07)	−.11*** (.08)	−.09** (.07)
Age	−.10* (.03)	.05 (.03)	−.14*** (.03)	.02 (.03)	−.22*** (.03)	−.02 (.03)
Education	.07 (.04)	.01 (.04)	.14** (.04)	.07^# (.04)	.06 (.04)	−.03 (.04)
Family income	.12** (.02)	.07^# (.02)	.10* (.02)	.06 (.02)	.12** (.02)	.07* (.02)
Adjusted R²	.05	.05	.11	.11	.12	.12
F	12.06***	12.06***	30.45***	30.45***	31.68***	31.68***
Professional media consumption	—	.07* (.04)	—	.09** (.04)	—	.04 (.04)
Social media consumption	—	.25*** (.03)	—	.29*** (.03)	—	.39*** (.03)
Pro-democracy (Yes = 1, Other = 0)	—	.07* (.09)	—	.02 (.09)	—	.04 (.09)
Psychological involvement	—	.14*** (.03)	—	.13*** (.03)	—	.09** (.03)
Behavioral involvement	—	.11** (.04)	—	.17*** (.04)	—	.14*** (.04)
Adjusted R²		.19		.29		.32
F		24.26***		43.46***		48.85***

Notes. ^#p < .10; *p < .05; **p < .01; ***p < .001.

Respondents with greater exposure to AEBM-related news in the professional media reported verifying received information through personal networks or multiple media channels more frequently (see Table 1). By comparison, news exposure on social media was positively associated with the frequency of all three fact-checking behaviors (see Table 1). The respondents’ psychological involvement and behavioral involvement in AEBM were positively related to every type of fact-checking behavior. Respondents who identified with the pro-democracy camp reported conducting fact-checking through personal networks more frequently than others. However, political orientation was not significantly associated with fact-checking using multiple media channels or dedicated fact-checking websites (see Table 1). H1a and H1b received robust support, but H1c was only partially supported.

H2a proposed that the frequency of fact-checking behavior would be negatively associated with belief in counter-attitudinal misinformation, whereas H2b predicted that fact-checking behavior would be positively associated with belief in pro-attitudinal misinformation. To create the dependent variables in the two hypotheses, we coded AEBM supporters’ (n = 615) belief in anti-protester misinformation and AEBM opponents’ (n = 150) belief in anti-police misinformation as beliefs in counter-attitudinal misinformation. We also coded supporters’ belief in anti-police misinformation and opponents’ belief in anti-protester misinformation as beliefs in pro-attitudinal misinformation. Respondents who held neutral attitudes (n = 63) or who refused to reveal their stances (n = 176) were excluded as there was no way to indicate whether a piece of misinformation was counter-attitude or pro-attitude for them. Two sets of hierarchical linear regression analyses were then performed to predict respondents’ belief in counter-attitudinal misinformation and their belief in pro-attitudinal misinformation, respectively. For both regression analyses, respondents’ demographic information, attitudes toward AEBM, news consumption, and issue involvement were entered as control variables. The three fact-checking behaviors were first regressed on one another, and the standardized residuals were entered into the regression analyses simultaneously (see Table 2). The results showed that, as expected, the three fact-checking behaviors were each negatively associated with belief in counter-attitudinal misinformation. Fact-checking with multiple media channels and the use of dedicated fact-checking websites were each positively associated with belief in pro-attitudinal misinformation (see Table 2).

Table 2.

Standardized Linear Regression Coefficients (with Standard Errors in Parentheses) Predict Belief in Misinformation.

	Supporters and opponents		Supporters		Opponents
	Counter-attitudinalmisinformation	Pro-attitudinalmisinformation	Anti-protestermisinformation	Anti-policemisinformation	Anti-protestermisinformation	Anti-policemisinformation
Gender	.12*** (.05)	.04 (.06)	.13** (.06)	.03 (.07)	.01 (.14)	.14 (.11)
Age	−.14** (.02)	−.00 (.02)	−.18*** (.02)	−.01 (.03)	.07 (.06)	.05 (.04)
Education	−.08^# (.03)	−.04 (.03)	−.13** (.03)	−.05 (.03)	.02 (.07)	.14 (.05)
Family income	−.05 (.02)	.03 (.02)	−.01 (.02)	−.07 (.20)	.36*** (.03)	−.31** (.03)
Attitudes	.28*** (.07)	−.09* (.09)	—	—	—	—
PMC	−.03 (.03)	.09* (.03)	−.03 (.03)	.04 (.04)	.37*** (.07)	−.15^# (.06)
SMC	−.08^# (.02)	.15*** (.03)	−.14** (.03)	.21*** (.03)	−.12 (.05)	.18^# (.04)
PI	−.11** (.03)	.19*** (.03)	−.11** (.03)	.14*** (.04)	.25** (.05)	−.13 (.04)
BI	−.29*** (.03)	.27*** (.03)	−.27*** (.03)	.27*** (.03)	.17* (.13)	.01 (.10)
Adjusted R²	.16	.19	.17	.20	.36	.07
PN	−.11* (.04)	.04 (.04)	−.15** (.04)	.04 (.05)	.00 (.08)	.13 (.06)
MC	−.14* (.05)	.23*** (.06)	−.18* (.06)	.20** (.07)	.24* (.10)	.01 (.08)
FW	−.18** (.05)	.29*** (.06)	−.21** (.06)	.25** (.07)	.24** (.11)	−.10 (.09)
Adjusted R²	.17	.22	.18	.21	.40	.09
F	13.91***	18.38***	13.40***	16.03***	9.45***	2.30*

Notes. For gender, Male = 1, Female = 2; For attitudes, Supporters of AEBM = 1, Opponents of AEBM = 0; PMC = professional media consumption; SMC = social media consumption; PI = psychological involvement; BI = behavioral involvement; FB = fact-checking behavior; PN = fact-checking with personal networks; MC = fact-checking with multiple media channels; FW = fact-checking with dedicated fact-checking websites. #p < .10; *p < .05; **p < .01; ***p < .001.

As there were significantly more supporters of AEBM than opponents, the regression results might be skewed and biased by supporters’ attitudes and behavior. In particular, there was a significant association between attitude and beliefs in misinformation (see Table 2). We thus conducted an exploratory test to examine whether the consequences of fact-checking behavior were consistent for supporters and opponents of AEBM. We repeated the regression analyses for the two camps separately. All three fact-checking behaviors were found to be negatively associated with belief in anti-protester misinformation for supporters of AEBM. However, none of the fact-checking behaviors were significantly associated with opponents’ belief in anti-police misinformation (see Table 2). H2a was supported only by the supporters but not by the opponents. We also found that the frequency with which supporters performed fact-checking using multiple media channels or fact-checking websites was positively associated with their belief in anti-police misinformation. The frequency with which opponents verified information using multiple media channels or fact-checking websites was also positively associated with their beliefs in anti-protester misinformation (see Table 2). H2b was supported for both supporters and opponents. Thus, the opinions of the two opposing camps became more polarized as the frequency of fact-checking increased. We plotted the mean scores of beliefs in misinformation according to the frequency of fact-checking behavior to show such polarization in Figure 1.

Figure 1.

Associations between fact-checking behavior and belief in misinformation. (a) Using personal networks, (b) Using multiple media channels, and (c) Using fact-checking websites.

H3a–b proposed to examine whether fact-checking using personal networks was more effective in reducing beliefs in counter-attitudinal misinformation and increasing beliefs in pro-attitudinal misinformation than fact-checking using multiple media channels or dedicated fact-checking websites. To test the hypotheses, we conducted z-tests to compare the regression coefficients for the three fact-checking behaviors in the first two columns of Table 2. As the correlation coefficients under comparisons were measured on the same sample (i.e., dependent correlation coefficients) with one variable in common, we first convert each correlation coefficient into a z-score using Fisher's r-to-z transformation (Steiger 1980). We then used asymptotic z-tests to compute the asymptotic covariance of the estimates (Lee and Preacher 2013). To examine H3a, we compared the regression coefficient for fact-checking behavior using personal networks (β = −.11, p < .05) to the regression coefficient for fact-checking behavior using multiple media channels (β = −.14, p < .05) and to the regression coefficient for using dedicated fact-checking websites (β = −.18, p < .01) when the dependent variable was belief in counter-attitudinal misinformation (see Table 2). The results showed that the extent to which fact-checking using personal networks decreased belief in counter-attitudinal misinformation was not significantly different from the extent to which fact-checking using multiple media channels (z = .54, n.s.) or fact-checking using dedicated fact-checking websites (z = 1.24, n.s.) decreased belief in counter-attitudinal misinformation. H3a was not supported. We also examined H3b by comparing the regression coefficient for fact-checking using personal networks (β = .04, p > .05) to the regression coefficient for fact-checking using multiple media channels (β = .23, p < .001) and to the regression coefficient for fact-checking using dedicated fact-checking websites (β = .29, p < .001) when the dependent variable was belief in pro-attitudinal misinformation (see Table 2). Contrary to our expectation, we found that fact-checking through personal networks was less, not more, likely to increase belief in pro-attitudinal misinformation than fact-checking using multiple media channels (z = −3.45, p < .001) or dedicated fact-checking websites (z = −4.50, p < .001). H3b was not supported either.

Discussion

This study shows that the majority of citizens in Hong Kong sought verification when encountering contentious media information regarding AEBM. However, such fact-checking behavior only reduced the belief in disagreeable misinformation for supporters of the AEBM. More robust evidence suggests that media consumers’ fact-checking behavior enhanced their belief in agreeable misinformation. The findings correspond to the phenomenon of ‘factual backfire’ (Nyhan and Reifler 2010).

Our study first sketched out the characteristics of active fact seekers in AEBM in Hong Kong. These characteristics mainly lie in individual factors, including issue involvement and media consumption. Socio-demographic factors seemed to be relevant only when the fact-checking behavior involved media use. One possible explanation is that seeking facts or verifying information using media channels requires time and resources. Social groups that are more skeptical of media information or that have more time and resources are more motivated to do so. Therefore, males, who in general are more skeptical of media information than females (Chen et al. 2015), used multiple media channels, including dedicated fact-checking websites, for fact-checking more frequently than the females in our study. Similarly, people from more affluent families generally have more time and resources to understand political issues. In our study, they used media channels and dedicated fact-checking websites for information verification more frequently. By comparison, seeking information verification with personal networks does not require as much time and resources. We found no significant gender differences or income disparity in this type of fact-checking behavior. Further, individuals’ political orientation was associated with fact-checking through personal networks but not with fact-checking using multiple media channels or dedicated fact-checking websites. It is possible that supporters of the pro-democracy camp in Hong Kong were more skeptical about the media than others, so they relied more on personal networks for information verification than others. In summary, our findings suggest that more hardcore partisans are more motivated to perform fact-checks, but the sources they use may vary for different social or political groups.

We then investigated the effectiveness of fact-checking behavior in reducing people's belief in misinformation. The results showed that fact-checking behavior seemed to reduce counter-attitudinal misinformation only when the misinformation challenged people's self-concepts (Flynn et al. 2017); as such, for AEBM supporters, the anti-protester misinformation investigated in this study severely threatened their reputation. Fact-checking activities were found to reduce their belief in such misinformation. By contrast, the anti-police misinformation investigated in this study only challenged the image of the police but not AEBM opponents’ self-concept. Thus, fact-checking behavior was not found to reduce their belief in such misinformation.

We also found that media consumers’ fact-checking enhanced their belief in pro-attitudinal misinformation that excoriated the opponent group. The findings indicate that media consumers’ fact-checking is affected by selective exposure and motivated reasoning. A number of media and fact-checking websites in Hong Kong were found to take distinct partisan stances in their coverage of AEBM (Feng et al. 2021). It is possible that the supporting and opposing respondents in our study selected congenial media sources for fact-checking and interpreted the fact-checking information in such a manner as to denigrate the opposing side. Instead of combatting misinformation, media consumers’ fact-checking activities amplified the impact of misinformation.

Unlike our expectations, fact-checking through personal networks was not more effective in reducing counter-attitudinal misperceptions than fact-checking using media sources. It also did not enhance pro-attitudinal misperceptions, as fact-checking using multiple media channels or fact-checking websites would do. One explanation is that our respondents considered media sources to be more credible than members of their social groups; thus, they were more receptive to verification obtained from media sources than from family and friends. This finding also brings one to argue that the use of personal networks is not a valid component of audience fact-checking. However, several facts argue for including it in this study: one, it has high correlations with other established fact-checking measures, and two, although it is not what one would describe as a professional fact-checking activity, it is a behavior that the media audience exercises. Since previous studies of audience fact-checking have included this measure, we felt it would be important to test its role in this contentious political context. Although our data indicate that this factor has less influence on misperceptions, we call for further discussion on this measure.

This study has some limitations. First, the cross-sectional nature of the survey data limited our confidence in the causal conclusions. The small sample size for opponents of AEBM also adversely influenced the power of our statistical results, especially for the split-sample exploratory analyses for H2. Further, respondents might have been unable to avoid social desirability bias when self-reporting their attitudes and behavior. Second, our study did not consider the partisan stances of the sources that our respondents used to check facts or the exact fact-checking information that they received. Such examinations could further validate the confirmation bias that we found. Lastly, due to the time limit of the telephone interviews, not all AEBM-related misinformation was included in the examination. We did not investigate if fact-checking behavior was associated with beliefs in misinformation for people who held a neutral stance. The findings presented in this study are confined to selected misinformation and people who reported their partisan stances.

Despite these caveats, our study serves as an early endeavor to theoretically conceptualize and empirically test media consumers’ fact-checking behaviors. Rather than dispelling misperceptions, frequently engaging in spontaneous fact-checking appears to strengthen ideologically grounded factual beliefs among citizens and deepen the political divide in Hong Kong society. Besides highlighting the importance of fact-checking to media consumers in the post-truth era, efforts—such as educating people about the motivated reasoning or selective exposure they may succumb to when seeking or receiving fact-checking reports—must be made before fact-checking mechanisms can successfully foster accurately informed citizens and reconcile opposing camps.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is funded by the Special Public Policy Research Funding Scheme (SR2020.A1.002) from the Policy Innovation and Co-ordination Office of the Government of the Hong Kong Special Administrative Region as well as a grant awarded by City University of Hong Kong (SRG#9678207).

ORCID iD

Stella C. Chia

Notes

Author Biographies

Stella C. Chia is a Professor in the Department of Communication Studies, Concordia University, Montreal, Canada. Her research interests include public opinion, media effects, and health communication.

Fangcao Lu is a Postdoctoral Fellow in the Department of Family Medicine and Primary Care at the University of Hong Kong. Her research examines personal perception and health communication.

Albert C. Gunther is a Professor Emeritus in the School of Journalism and Mass Communication, the University of Wisconsin-Madison. His research interests focus on the psychology of media audience, particularly partisans and special interest groups.

References

Almenar

Aran-Ramspott

Suau

Masip

. 2021. “Gender Differences in Tackling Fake News: Different Degrees of Concern, But Same Problems.” Media and Communication 9 (1): 229–38. doi:10.17645/mac.v9i1.3523.

American Association for Public Opinion Research. 2016. “ Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys .” https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf.

Bode

Vraga

E. K.

. 2015. “In Related News, that was Wrong: The Correction of Misinformation through Related Stories Functionality in Social Media.” Journal of Communication 65 (4): 619–38. doi:10.1111/jcom.12166.

Census and Statistics Department. 2022a. “Hong Kong Monthly Digest of Statistics.” https://www.censtatd.gov.hk/en/data/stat_report/product/B1010002/att/B10100022022MM01B0100.pdf.

Census and Statistics Department. 2022b. “Statistics on Domestic Households.” https://www.censtatd.gov.hk/en/web_table.html?id=5.

Census and Statistics Department. 2021. “Hong Kong in Figures.” https://www.censtatd.gov.hk/en/data/stat_report/product/B1010006/att/B10100062021AN21B0100.pdf.

Chen

Sin

S. C. J.

Theng

Y. L.

Lee

C. S.

. 2015. “Why Students Share Misinformation on Social Media: Motivation, Gender, and Study-level Differences.” The Journal of Academic Librarianship 41 (5): 583–92. doi:10.1016/j.acalib.2015.07.003.

Edgerly

Mourão

R. R.

Thorson

Tham

S. M.

. 2020. “When do Audiences Verify? How Perceptions about Message and Source Influence Audience Verification of News Headlines.” Journalism & Mass Communication Quarterly 97 (1): 52–71. doi:10.1177/1077699019864680.

Feng

Tsang

N. L.

Lee

F. L.

. 2021. “Fact-Checking as Mobilization and Counter-Mobilization: The Case of the Anti-Extradition Bill Movement in Hong Kong.” Journalism Studies 22 (10): 1358–75. doi:10.1080/1461670X.2021.1952474.

10.

Flynn

Nyhan

Reifler

. 2017. “The Nature and Origins of Misperceptions: Understanding False and Unsupported Beliefs about Politics.” Advances in Political Psychology 38: 127–50. doi:10.1111/pops.12394.

11.

Fridkin

Kenney

P. J.

Wintersieck

. 2015. “Liar, Liar, Pants on Fire: How Fact-Checking Influences Citizens’ Reactions to Negative Advertising.” Political Communication 32 (1): 127–51. doi:10.1080/10584609.2014.914613.

12.

Godes

Mayzlin

. 2009. “Firm-Created Word-of-Mouth Communication: Evidence from a Field Test.” Marketing Science 28 (4): 721–39. doi:10.1287/mksc.1080.0444.

13.

Gottfried

J. A.

Hardy

B. W.

Winneg

K. M.

Jamieson

K. H.

. 2013. “Did Fact-checking Matter in the 2012 Presidential Campaign?” American Behavioral Scientist 57 (11): 1558–67. doi:10.1177/0002764213489.

14.

Graves

2016. “Thinking Ahead: Prospects for Public-Service Journalism in the New News World.” New Media and Society 18 (3): 521–27. doi:10.1177/1461444815616115.

15.

Haciyakupoglu

Hui

J. Y.

Suguna

V. S.

Leong

Bin Abdul Rahman

M. F.

. 2018. “Countering Fake News: A Survey of Recent Global Initiatives.” Policy Report, Nanyang Technological University, Singapore, March.

16.

Hameleers

van der Meer

T. G. L. A.

. 2020. “Misinformation and Polarization in a High-Choice Media Environment: How Effective Are Political Fact-Checkers?” Communication Research 47 (2): 227–50. doi:10.1177/0093650218819671.

17.

Jang

S. M.

Lee

Park

Y. J.

. 2014. “The More Friends, the Less Political Talk? Predictors of Facebook Discussions among College Students.” Cyberpsychology, Behavior, and Social Networking 17 (5): 271–75.

18.

Jarman

J. W.

2014. “The Failure Of Fact-Checking.” Proceedings in 2014 International Society for the Study of Argumentation . https://rozenbergquarterly.com/?s=The + Failure + Of + Fact-Checking.

19.

Johnson

B. T.

Eagly

A. H.

. 1989. “Effects of Involvement on Persuasion: A Meta-Analysis.” Psychological Bulletin 106 (2): 290. doi:10.1037/0033-2909.106.2.290.

20.

Kim

K.-S.

Sin

S.-C. J.

Yoo-Lee

E. Y.

. 2014. “‘Undergraduates’ Use of Social Media as Information Sources.” College and Research Libraries 75 (4): 16. doi:10.5860/crl.75.4.442.

21.

Kim

Wyatt

R. O.

Katz

. 1999. “News, Talk, Opinion, Participation: The Part Played by Conversation in Deliberative Democracy.” Political Communication 16 (4): 361–85. doi:10.1080/105846099198541.

22.

A. S.

2020. “New Forms of Youth Activism – Hong Kong’s Anti-Extradition Bill Movement in the Local-National-Global Nexus.” Space and Polity 24 (1): 111–7. doi:10.1080/13562576.2020.1732201.

23.

Lee

F. L. F.

2016. “Impact of Social Media on Opinion Polarization in Varying Times.” Communication and the Public 1 (1): 56–71. doi:10.1177/2057047315617763.

24.

Lee

F. L. F.

2020. “Solidarity in the Anti-extradition Bill Movement in Hong Kong.” Critical Asian Studies 52 (1): 18–32. doi:10.1080/14672715.2020.1700629.

25.

Lee

I. A.

Preacher

K. J.

2013. “Calculation for the Test of the Difference between Two Dependent Correlations with One Variable in Common [Computer software].” http://quantpsy.org.

26.

Lee

D. K. L.

Ramazan

. 2021. “Fact-Checking of Health Information: The Effect of Media Literacy, Metacognition and Health Information Exposure.” Journal of Health Communication 26 (7): 491–500. doi:10.1080/10810730.2021.1955312.

27.

S. S.-H.

2017. “Factionalism and Chinese-style Democracy: The 2017 Hong Kong Chief Executive Election.” Asia Pacific Journal of Public Administration 39 (2): 100–19. doi:10.1080/23276665.2017.1325620.

28.

Min

S. J.

2010. “From the Digital Divide to the Democratic Divide: Internet Skills, Political Interest, and the Second-Level Digital Divide in Political Internet Use.” Journal of Information Technology and Politics 7 (1): 22–35. doi:10.1080/19331680903109402.

29.

Mutz

D. C.

Martin

P. S.

. 2001. “Facilitating Communication Across Lines of Political Difference: The Role of Mass Media.” The American Political Science Review 95 (1): 97–114. doi:10.1017/S0003055401000223.

30.

Nee

R. C.

2019. “Youthquakes in a Post-Truth Era: Exploring Social Media News Use and Information Verification Actions Among Global Teens and Young Adults.” Journalism and Mass Communication Educator 74 (2): 171–84. doi:10.1177/107769581882521.

31.

Noelle-Neumann

1995. “Public Opinion and Rationality.” In Public Opinion and the Communication of Consent, ed. Glasser

T. L.

Salmon

C. T.

, 33–54. New York: The Guiford Press.

32.

Nyhan

2021. “Why the Backfire Effect Does not Explain the Durability of Political Misperceptions.” Proceedings of the National Academy of Sciences 118 (15): e1912440117.

33.

Nyhan

Reifler

. 2010. “When Corrections Fail: The Persistence of Political Misperceptions.” Political Behavior 32 (2): 303–30. doi:10.1007/s11109-010-9112-2.

34.

Nyhan

Reifler

. 2015. Estimating Fact-Checking’s Effects. Arlington, VA: American Press Institute.

35.

H. J.

Lee

. 2019. “When Do People Verify and Share Health Rumors on Social Media? The Effects of Message Importance, Health Anxiety, and Health Literacy.” Journal of Health Communication 24 (11): 837–47. doi:10.1080/10810730.2019.1677824.

36.

Robertson

C. T.

Mourão

R. R.

Thorson

. 2020. “Who Uses Fact-Checking Sites? The Impact of Demographics, Political Antecedents, and Media Use on Fact-Checking Site Awareness, Attitudes, and Behavior.” The International Journal of Press/Politics 25 (2): 217–37. doi:10.1177/194016121989805.

37.

Scheufele

D. A.

1999. “Framing as a Theory of Media Effects.” Journal of Communication 49 (1): 103–22. doi:10.1111/j.1460-2466.1999.tb02784.x.

38.

Shin

Thorson

. 2017. “Partisan Selective Sharing: The Biased Diffusion of Fact-Checking Messages on Social Media.” Journal of Communication 67 (2): 233–55. doi:10.1111/jcom.12284.

39.

Alvin Y.

2017. “A New Wave of Anti-mainland Protests Since 2012: Characteristics, Socio-political Origins, and Political Implications.” Asian Education and Development Studies 6 (4): 333–42. doi:10.1108/AEDS-10-2015-0054.

40.

Steiger

J. H.

1980. “Tests for Comparing Elements of a Correlation Matrix.” Psychological Bulletin 87 (2): 245. doi:10.1037/0033-2909.87.2.245.

41.

Stroud

N. J.

2008. “Media Use and Political Predispositions: Revisiting the Concept of Selective Exposure.” Political Behavior 30 (3): 341–66. doi:10.1007/s11109-007-9050-9.

42.

Sunstein

2009. Republic. Com 2.0. New York, NY: Princeton UP.

43.

Taber

C. S.

Lodge

. 2006. “Motivated Skepticism in the Evaluation of Political Beliefs.” American Journal of Political Science 50 (3): 755–69. doi:10.1111/j.1540-5907.2006.00214.x.

44.

Tsang

2004. A Modern History of Hong Kong: 1941–1997. NY: I.B. Tauris & Co Ltd.

45.

Walter

Cohen

Holbert

R. L.

Morag

. 2020. “Fact-Checking: A Meta-Analysis of What Works and for Whom.” Political Communication 37 (3): 350–75. doi:10.1080/10584609.2019.1668894.

46.

Walter

Edgerly

Saucier

C. J.

. 2021. “‘Trust, Then Verify’: When and Why People Fact-Check Partisan Information.” International Journal of Communication 15 (2021): 21.

47.

Wong

M. Y. H.

2020. “Party Competition and Ideology in Hong Kong: A new Manifesto Coding Dataset.” Journal of East Asian Studies 20 (2): 207–30. doi:10.1017/jea.2020.3.

48.

Wood

Porter

. 2019. “‘The Elusive Backfire Effect: Mass Attitudes’ Steadfast Factual Adherence.” Political Behavior 41 (1): 135–63. doi:10.1007/s11109-018-9443-y.

49.

Shen

. 2021. “Does Fact-checking Habit Promote COVID-19 Knowledge during the Pandemic? Evidence from China.” Public Health 196: 85–90. doi:10.1016/j.puhe.2021.05.005.