Abstract
Given growing interest in the potential importance of news literacy around the world, a theoretically grounded and empirically validated measure of news literacy is essential. Building on existing theory, we developed and validated a 15-item true/false measure of news literacy knowledge. This measure comprehensively operationalizes the five C’s of news literacy—context, creation, content, circulation, and consumption—in a concise, adaptable, knowledge-based format. Using item response theory and differential item functioning analysis, we followed a three-survey process with representative U.S. samples, developing and assessing 80 true/false items in Study 1 (N = 1,502) to reduce to 43 items in Study 2 (N = 1,273). The final reduced set of 15 items was evaluated and validated in Study 3 (N = 681) along with related measures of civics and current events knowledge, which were positively predicted by the news literacy knowledge measure. While this measure is designed and tested in the U.S. context, our process of operationalizing these complicated concepts and the novel true/false format facilitates its applicability to those interested in studying news literacy around the globe.
Introduction
Equipping people with the knowledge and skills necessary to navigate complex media environments is essential in light of continuing media fragmentation and the spread of misinformation. News literacy education and interventions are among the responses proposed to enable people to critically and competently engage with news and information and thus make more informed personal, social, and political decisions (Schulz et al., 2024; Tamboer et al., 2022). However, research and practice in this area has been hampered by theoretical murkiness regarding how to define news literacy, producing a lack of measurement clarity that, in turn, hinders efforts to evaluate the effectiveness of educational interventions and to advance our understanding of news literacy more generally (Jones-Jang et al., 2021; Tully et al., 2022; Vraga et al., 2021).
To address the challenge of competing measurement approaches and facilitate theory building in this space, we have developed, tested, and validated a 15-item news literacy knowledge measure that aligns with a conceptualization of news literacy as comprising knowledge and skills across five domains: context, creation, content, circulation, and consumption (Tully et al., 2022; Vraga et al., 2021). This “five C’s” approach is part of our broader theoretical reframing of news literacy that draws on the Theory of Planned Behavior (Ajzen, 1985) to distinguish the knowledge and skills required to be news literate from their application to news literacy behaviors (Vraga et al., 2021). In making that distinction, this new theoretical framework offers a corrective to previous research that sometimes incorporated attitudes or self-efficacy regarding news literacy into the definition of news literacy itself (e.g., Ashley et al., 2013), making it difficult to determine the individual contributions of knowledge, skills, attitudes, social norms, and perceived behavioral control to the enacting of news literacy behaviors, such as identification of misinformation or verification of information.
Previous research in news literacy (and the broader umbrella domain of media literacy) is marked by a strong emphasis on knowledge (e.g., Amazeen & Bucy, 2019; Tully et al., 2022) and the assumption among researchers “that knowledge forms the starting point for any kind of media literacy skills” (Rosenbaum et al., 2008, p. 340). Therefore, we developed a measure of news literacy knowledge informed by the “five C’s” conceptualization of news literacy. Multiple choice questions that feature correct and incorrect choices have been used to measure news literacy in a variety of survey- and experiment-based studies (e.g., Amazeen & Bucy, 2019; Maksl et al., 2015; Schulz et al., 2024; Vraga & Tully, 2021). The news literacy knowledge scale reported here improves cross-national adaptability and concision by moving away from multiple-choice items to true/false questions that are more parsimonious for survey administration and require less translation and rewriting when adapted to diverse news environments. Creating a knowledge measure that captures the complexity of news literacy while also being adaptable to diverse contexts and broad definitions of “news” expands opportunities for research to further refine and test conceptualizations and theorizing around news literacy (Schulz et al., 2024; Tully et al., 2022).
Defining News Literacy
Defining and measuring complex concepts like news literacy is notoriously difficult (Malik et al., 2013). Prior work has drawn on conceptualizations from media literacy research, as well as related and sub-domains of media literacy including information literacy and digital literacy (Lopes et al., 2018; Tully et al., 2022). Educators have long promoted the value of media literacy, typically defined as “the ability to understand, analyze, evaluate and create media messages in a wide variety of forms” (Aufderheide & Firestone, 1993), a definition later adopted by the National Association for Media Literacy Education (NAMLE) and other similar organizations. The emphasis in media literacy education on qualitative research methods has contributed to the diversity of definitions and approaches (Anderson, 1980; Hobbs & Frost, 2003; Singer et al., 1980), but these methods are difficult to scale and offer limited generalizability. One of the benefits of this broad definition—its amenability to a variety of pedagogical approaches and interpretations—is also one of its drawbacks.
Why Study News Literacy?
The ability to evaluate the credibility of news is vital to navigating civic life, especially in the face of increasing threats to democratic institutions and growing authoritarianism around the world (Freedom House, 2021). Propaganda flows through the same channels and across the same platforms as independent information; advertisements look like news stories; and manipulated images spread virally across social media. Understanding that messages are constructed—by people with particular motives and purposes in mind, employing a variety of techniques and tools—is the basic building block of media literacy, helping to empower audiences to make informed choices about content and, related, to protect them from potential harm of manipulation (Hobbs, 2021; Potter, 2022). Research based on this definition has frequently centered on the role of media literacy knowledge and awareness and demonstrated the ways effective media literacy education can lead to positive behavioral change (Geers et al., 2020; Jeong et al., 2012; Kahne & Bowyer, 2019; Kurz et al., 2022; Lu et al., 2024).
News messages are distinct from other media messages not only in terms of their content but also the organizational and institutional structures and the legal, cultural-historical and economic contexts in which they are produced. News literacy 1 captures knowledge of the contexts and structures that influence the production and consumption of news messages as opposed to other forms of media content such as advertising and entertainment (e.g., Amazeen & Bucy, 2019; Chan et al., 2021; Tamboer et al., 2022). As sources of news (broadly defined) proliferate and news avoidance increases, the need for news literacy likewise continues to grow (Newman et al., 2023). News literacy researchers and educators assert that knowledge about factors shaping news production as well as knowledge about the increasingly fragmented news media environment in which news circulates improves the ability to competently engage with news (Ashley, 2020). Furthermore, research demonstrates that literate engagement with news is linked to positive outcomes including a lower likelihood of endorsing conspiracy theories (Craft et al., 2017), reduced beliefs in COVID-19 misinformation (Ashley et al, 2023; Hameleers, 2022), and increased political knowledge (Ashley et al., 2017).
But these studies are hampered by inconsistent measurement of news literacy, making it difficult to understand sometimes contradictory findings or evaluate the success of news literacy interventions. For example, a case study of news literacy curricula in Myanmar, Hong Kong, and Vietnam noted the need both for sensitivity to local conditions in designing such curricula but also the lack of “rigorous assessment tools to gauge the impact of pedagogical methods” that inhibits the ability of news literacy education to progress (Kajimoto, 2016). Accurate measurement of news literacy is particularly important given recent research that finds overconfidence in one’s ability to identify false information is linked to false news consumption, engagement, and belief (Lyons et al., 2021).
Measuring News Literacy
News literacy researchers have typically focused on adapting quantitative media literacy measures, such as the smoking media literacy (SML) scale developed by Primack et al. (2006), to the news literacy context. Building on this work, Ashley et al. (2013) created the first measure of news media literacy, applying the same attitude-based media literacy domains used by Primack et al. (2006). Because respondents scored consistently highly on their scale, Ashley et al. (2013) concluded that “the attitude framework alone may not be appropriate for assessing literacy in adults” (p. 17) and found greater discriminatory value in a related knowledge-based measure.
Responding to the need for knowledge-based measures, Maksl et al. (2015) relied on Potter’s (2004) theoretical model of media literacy to create and validate a set of 15 multiple-choice knowledge questions about the U.S. news media system, such as the commercial nature of news media, trends in news coverage, and news media effects. Other studies have used and adapted the Ashley et al. (2013) and Maksl et al. (2015) measures in other contexts; the piecemeal use of these measures and occasional confounding of different scales and their authorship (e.g., Ashley et al., 2013 vs. Maksl et al., 2015) demonstrates the need for additional refinement and better conceptualization (e.g., Chan et al., 2021; Geers et al, 2020; Kleemans & Eggink, 2016; Schulz et al., 2024; Vraga et al., 2015). Furthermore, a variety of news literacy stakeholders such as teachers and policymakers have shown a need for greater consistency and an easy-to-use assessment tool, particularly as news literacy is increasingly seen as a response to media misinformation (Cunliffe-Jones et al., 2021).
We are aware of just a few other attempts to develop theoretically grounded measures of literacy related to media and information (e.g., Lopes et al., 2018; Purington et al., 2023). Amazeen and Bucy (2019) addressed news literacy specifically in their measure of “procedural news knowledge,” and Johnson et al. (2021) created new measures related to knowledge about news headlines and hard news standards. Dogruel et al. (2022) designed an algorithm literacy scale focused on algorithm awareness and knowledge.
Developing a New Measure of News Literacy
As we have noted, a hodgepodge of definitions and measures has emerged in news literacy research, frustrating efforts to build theory. Even so, prior research does converge in demonstrating the essential role that knowledge plays in news literacy (Jeong et al., 2012; Rosenbaum et al., 2008). News literacy knowledge, more than motivation, attitudes or other measures, has yielded consistent and robust results (e.g., Craft et al., 2017; Vraga et al., 2015). In addition, our recent theoretical reframing of news literacy (Vraga et al., 2021), building on the work of Potter (2004) and Maksl et al. (2015), conceives of news literacy knowledge and skills as essential precursors to the performance of news literacy behaviors. We consider the development of a news literacy knowledge measure that aligns with this model as aiding theory building in two ways. First, operationalizing a critical component of the model (knowledge) allows researchers to refine the model based on empirical findings and to deepen our understanding of news literacy. Second, creating a measure that is adaptable to diverse national and cultural contexts helps extend our understanding beyond the Global North where most news literacy research has focused.
We again define news literacy as “knowledge of the personal and social processes by which news is produced, distributed, and consumed, and skills that allow users some control over these processes” (Vraga et al., 2021, p. 5), across five domains: context, creation, content, circulation, and consumption. The context domain captures knowledge of macro-level factors, including the social, legal, and economic environment of news; creation involves knowledge of the process by which news is conceived, reported, and created by journalists and other newsmakers; content knowledge refers to the characteristics that distinguish news from other types of media content; circulation is knowledge of the process through which news is distributed and spread; and consumption involves knowledge of personal or individual factors that contribute to news exposure, attention, and evaluation (see Tully et al., 2022). The measures that we proposed and tested were derived from these five domains and designed to be conceptually consistent but empirically adaptable across divergent media systems and other socio-political contexts, making them both theoretically robust and suitable for much-needed comparative work on news literacy (Schulz et al., 2024; Tully et al., 2022; Vraga et al., 2021).
We note that these five domains are “connected and mutually constitutive” (Tully et al., 2022, p. 1593), such that “news literacy” might be understood as a sort of emergent property of the domains. Therefore, empirical overlap is possible. For example, one’s knowledge about how news spreads (circulation) may be linked to one’s evaluation of that news (consumption). However, while the items operationalizing each domain might differ according to the national context in which a study takes place, the domains those items operationalize would not change. (The specific items operationalizing the 5C’s in this study are detailed in the methods section.)
We also note that the measure reported here operationalizes news literacy knowledge only and does not address related skills as described in Tully et al. (2022) and elsewhere. The reasons for focusing exclusively on knowledge are both practical and theoretical. First, knowledge measures are considerably more scalable in quantitative research; for this reason, many more studies incorporate measures of knowledge as either a predictor or outcome variable than measures of skills (e.g., Craft et al., 2017; Hobbs, 2017; Maksl et al., 2015; Vraga & Tully, 2021). Skills assessment may require more labor-intensive or technologically intensive techniques, such as the live internet search exercises employed by the Stanford History Education Group (McGrew et al., 2018) or qualitative interviews, which are difficult to scale (Swart, 2023; Tamboer et al., 2020). Moreover, even studies that purport to measure skills often use proxies like self-reported behaviors (e.g., Valenzuela et al., 2024) or efficacy to perform behaviors (e.g., Tamboer et al., 2023). Therefore, building a robust measure of knowledge will contribute to a broader range of literature. Second, knowledge is typically a precursor to skills (Rosenbaum et al., 2008), just as literacy is a precursor to its application (Vraga et al., 2021). Therefore, our approach is to establish what constitutes news literacy knowledge in this paper, which will enable researchers to later develop skills assessments that represent that knowledge in action.
To summarize, we propose a 15-item battery of true/false questions to measure news literacy knowledge. This concise approach—especially compared to previously used multiple-choice questions—provides opportunity for scholars to more easily include measures of other concepts on questionnaires. Additionally, the true/false approach in contrast to multiple-choice does not require scholars adapting a measure to other contexts to develop additional distractor answer options, which could otherwise create confounding results if the conceptual distance between the correct answer and incorrect distractor answers varied across adaptations. Despite its concise and relatively novel approach, the proposed 15-item true/false measure of news literacy knowledge was rigorously developed and performs well psychometrically, as our studies here establish.
Method
Scale Development Procedure
Our intent was to develop (1) a brief assessment of news literacy knowledge (2) with true/false questions that could (3) produce sum scores that (4) correlate well with the results of a previously developed multiple choice measure of news literacy knowledge, and (5) did not exhibit differential item functioning across several characteristics of respondents (see Appendix A for the final 15 items). We followed an iterative process of scale development, including using Item Response Theory (IRT) to assess items and evaluating validity against scores on other measures of news literacy knowledge and its correlates. First, 80 true/false items were constructed, with multiple questions associated with each of the five C’s. They were developed by both adapting items used in previous studies (e.g., Amazeen & Bucy, 2019; Maksl et al., 2015; Vraga et al., 2015) and through a process that inferred questions based on the underlying theory and definitions of the five C’s. We then used a three-step, three-survey process to reduce, refine, and validate the items. The protocols for all three studies were approved by the Institutional Review Board at the University of Iowa. In Study 1, we evaluated responses (N = 1,502) to subsets of the 80 true/false items, assessing difficulty parameters and face validity among items within each domain, reducing the number of items down to 43. In Study 2, we analyzed responses (N = 1,273) to all 43 items, looking at both fit and IRT difficulty parameters. The items in Study 2 were further evaluated and the potential for differential item functioning (DIF) assessed over demographic and political characteristics. This led to further item refinement and reduction, reducing the total number of items to 15 spread across the five C’s (three items per domain). The final set of items was evaluated in Study 3 (N = 681) and assessed for DIF. DIF was evaluated across categories of political party affiliation (Republican, Independent, Democrat), race/ethnicity (White vs. Non-White), education (HS or less, Some or two-year college, four-year college, college plus), gender (Male, Female), age (18–35, 36–54, 55+), political interest level (low, medium, high), and ideology (conservative, moderate, liberal). Further, convergent validity was evaluated. Measures used included a previously developed multiple-choice news literacy knowledge scale, knowledge of current events, and knowledge of civics.
Measurement Theory
We used IRT to develop the scale (Embretson & Reise, 2000). Using IRT, a non-linear functional relationship between an individual’s ability on a latent trait and the probability of response is estimated. When the response is dichotomous (i.e., correct or incorrect), the functional form is non-linear and typically modeled using a logistic function. There are several possible parameters of the non-linear logistic function that may be estimated, including the point of inflection, the slope at the point of inflection, and the lower asymptote (Boedeker, 2021). In IRT parlance, these parameters correspond, respectively, to difficulty, discrimination, and guessing parameters. When only the difficulty parameter is estimated for each item (and all other parameters are fixed), the one-parameter logistic (1PL) or Rasch model is used. When both the difficulty and discrimination are estimated, the 2PL model is fit. The 3PL model is used when the difficulty, discrimination, and guessing parameters are all three estimated. The result for each item, regardless of the number of parameters estimated, is an item characteristic curve (ICC) that maps the relationship between ability and the probability of answering the item correctly.
A useful and differentiating feature of the Rasch model is the statistical sufficiency of the sum score in estimating the ability of each respondent. With the Rasch model, the sum score contains all the necessary information for estimating an individual’s ability on a continuous scale; the 2PL and 3PL models do not have the same characteristic. Instead, for the 2PL and 3PL models, the pattern of responses across items (instead of simply the sum of correct responses) is sufficient for estimating an individual’s ability on a continuous scale. Said differently, in Rasch modeling all individuals with the same sum score are estimated to have the same ability score; however, with the 2PL and 3PL models individuals with the same sum score may not be estimated to have the same ability score. This makes the Rasch model particularly suited for developing a measure that allows for a simple sum score of news literacy knowledge.
With our definition of news literacy as encompassing five C’s, it could be argued that the measure created should be multidimensional. However, the five C’s represent a careful definition of news literacy that captures the full domain of the construct rather than five separate, independently discernable constructs. Given the utility of and need for a news literacy knowledge measure yielding a single, sufficient sum score, we do not attempt to create a multidimensional measure of news literacy knowledge but instead refine a set of items for measuring news literacy knowledge that encompasses equal representation of each C. The unidimensionality of the measure is assessed with PCA on the residuals after fitting the Rasch model. Doing so, a first eigenvalue less than 2 provides rationale for a unidimensional scale (Boone & Staver, 2020). Additionally, a modified procedure of parallel analysis proposed by Drasgow and Lissak (1983) is used to assess the hypothesis that the second eigenvalue in the observed data is equivalent to the second eigenvalue in simulated data in which there is no factor. A non-significant result of this test supports unidimensionality.
Using IRT, we focus on the assessment of each item. Infit and outfit values are commonly used to evaluate the fit of an item to the Rasch specification. Infit and outfit values are based on the standardized differences in the observed and expected responses of individuals. These values are squared and either averaged or weighted to produce estimates of infit or outfit, respectively. Averaging (weighting) across individuals within items gives an assessment of item fit and averaging (weighting) across items within individuals gives an assessment of person fit. Linacre (2002) suggests that values of infit and outfit between 0.5 and 1.5 are acceptable.
DIF occurs when individuals with the same ability have different probabilities of answering a question correctly. DIF would indicate that scores produced when using the measure are biased in favor of participants with certain characteristics. We intend for our measure to be utilized broadly and therefore assessing and eliminating DIF is a critical step in measure development. We assess DIF in two stages. First, Lord’s chi-square test (Lord, 1980) is used in Study 2 to evaluate pairwise DIF. In Study 3, items identified from Study 2 as not having DIF are used as anchor items in conjunction with a likelihood ratio test to assess for DIF in the revised set of items (Thissen et al., 1986).
Study 1: Developing and Evaluating Item Pool
The 80 potential true/false items were developed to each align to one of the five C’s—including multiple potential conceptual sub-parts of each domain (see Appendix C online for a full list of the 80 items originally developed for Study 1). Some items were used or adapted from those used in other studies (e.g., Amazeen & Bucy, 2019; Maksl et al., 2015; Vraga et al., 2015), but all items were either newly developed operationalizations of the five C’s or were adaptations of previously used multiple choice items transformed into true/false items. For example, a question from Amazeen and Bucy (2019) and Maksl et al. (2015) asking, “Which of the following U.S. news outlets does NOT depend primarily on advertising for financial support?” (with the correct answer PBS listed among several for-profit media companies) became “Public media such as PBS and NPR depend primarily on advertising for financial support” (the correct answer being false). Other items were adapted and new items created to clearly align with the conceptual definitions of each of the five C’s and to ensure that all five C’s were represented across the set of items.
Individual members of the research team took the lead on a C, adapting existing items and writing new ones for their assigned C, with the remainder of the team iteratively reviewing and revising other C’s to arrive at the final item pool. In total, we developed 80 items, with each C having between 13 and 23 individual items.
All items were closely tied to conceptual definitions of knowledge of each C we have outlined in prior work (Tully et al., 2022). Because the measure was tested with U.S. respondents, items reflected the U.S. news context. Items were also constructed to have a single correct answer based on fact-based, empirical reality or research about the related construct. For example, context knowledge included questions about the economic, legal, and socio-political contexts in which news is created, such as knowing that in the U.S. journalists are not required to be licensed. Creation knowledge included knowing how news is made, including journalistic routines and newsgathering techniques. One such item included knowing that local journalists prioritize news values such as proximity when choosing what stories to cover (e.g., Jenkins & Nielsen, 2020). Content knowledge included identifying common structures and frames in which news content is often presented, such as knowing that news aggregators like Google News include content that may not employ consistent expectations for fact verification. Circulation knowledge included the impact of both human and algorithmic factors in news and information distribution, such as the fact that many communities in the U.S. have lost newspapers over the last few decades (e.g., Joiner et al., 2021). And consumption knowledge included identifying common empirically supported individual and social-level media effects. One such example is knowledge that some people are more influenced by media than others, known as the cultivation effect and supported by decades of empirical research (e.g., Hermann et al., 2021).
Each item presented a statement to the respondents, who were asked to indicate if they believed the statement to be true or false, or if they did not know. There were a mix of true and false items. Correct answers were coded as 1. Though an incorrect answer and a “don’t know” response are conceptually different, they both represent absence of the defined areas of knowledge we were intending to measure; additionally, including “don’t know” helps guard against guessing (Jessee, 2015). Therefore, both incorrect and “don’t know” answers were coded as 0.
Participants and Procedure
To assess the potential value of each of the 80 true/false items used in Study 1, we conducted an online survey in October 2020. Respondents were sampled using Lucid to be representative of the U.S. population. Of the 1,948 participants who started the survey, 446 were removed for failing an attention check question. Of the remaining 1,502, their mean age was 44.8 years (SD = 16.7), 69.4% were white, and 51.1% identified as women. The demographic makeup of the sample skewed slightly older than the population average age of 38.4 years. Additionally, our sample skewed more educated than the U.S. population, with 21.2% having a bachelor’s degree (compared to 21% of the population) and 26.7% having a graduate or professional degree (compared to just 12% of the population).
Because of the large number of true/false items developed across each of the C’s, we randomly assigned respondents to one of two conditions, where each answered a subset of the five C’s. One group (N = 738) was presented with items reflecting Context and Creation (with 23 and 13 items, respectively). A second group (N = 765) was presented with items reflecting Content, Circulation, and Consumption (with 18, 13, and 13 items, respectively). There were no significant demographic differences between the groups.
For each C, we evaluated the difficulty of each item by analyzing both the percent of respondents who answered the question correctly as well as the relative difficulty parameter produced by IRT analysis using the Rasch model. In working toward item reduction, we grouped items across a range of difficulty levels and used the Wald test to check the degree to which items fit our model, choosing those items with the strongest fit within each difficulty group. In addition to evaluating items based on statistical criteria, we again assessed conceptual validity of each item during this process.
Results
Items were organized within each C into between three and five groups based on difficulty. Within each difficulty group, we chose between one and three items based largely on the fit statistics, also evaluating item language to ensure a range of conceptual sub-components of each were retained in the reduced set of items.
For items measuring knowledge of context, we reduced 23 items down to 12, with difficulty parameters ranging from −1.287 to 1.167. We eliminated items that were overly specific, such as knowing the corporate owner of specific media brands, and we retained items that could be easily adapted to non-U.S. contexts (e.g., knowing whether journalists need to be licensed).
For items measuring knowledge of the process of news creation, we reduced 13 items down to six, with difficulty parameters ranging from −1.313 to 1.192. We also eliminated items that were overly specific in referencing the demographic makeup of media creators or comparing common characteristics of media creators to the general population (e.g., knowing how the average age of journalists in a country compares to the population).
With items measuring knowledge of the content category, we reduced 18 items down to 10, with difficulty parameters ranging from −0.875 to 1.1312. Among the items eliminated included those that were vague or could be misinterpreted (e.g., knowing that news content is considered “objective” if it omits a reporters’ opinions), and among those retained were items that emphasized common features of news content (e.g., knowing that breaking news stories are more likely to have fewer sources than non-breaking stories).
For items measuring knowledge of news circulation, we reduced 13 items down to seven, with difficulty parameters ranging from −1.113 to 1.055. We eliminated questions that referenced specific media or technology (e.g., “All users who type in the same search terms in Google will get the same results”) and retained those that could be more easily adapted to broader uses (e.g., “Users’ previous behavior on the Internet is one factor that determines the news they see online”).
And finally, with knowledge about common consumption practices and effects, we reduced 13 items down to eight, with difficulty parameters ranging from −0.832 to 0.609. We eliminated items that were vaguely worded or were extremely difficult (e.g., Only 15.9% answered correctly that “People usually say they prefer news and information that matches their beliefs”) and retained items that emphasized common media effects or public opinion research findings (e.g., that the media does not have the same effect on all people or that most people don’t feel comfortable holding two beliefs that are in conflict).
After this initial round of item analysis and reduction, we reduced 80 items down to a more manageable 43 that could be used in a subsequent study in which all items could be administered to a single sample.
Study 2: Refining Item Pool and Testing for DIF
Participants and Procedure
To further refine the 43 items to a more manageable set, we conducted another online survey in July 2021. Respondents were sampled using Lucid to be representative of the U.S. population. Of the 1,803 participants who started the survey, 473 were removed for failing an attention check question; an additional 57 were removed for having either skipped all the true-false questions or for answering all of them as “don’t know.” Of the remaining 1,273, their mean age was 46.9 (SD = 17.7), 74.3% were white, and 51.3% identified as women. Again, our respondents skewed slightly older than the population. However, while Study 2’s sample still skewed more educated than the population, it was more representative than Study 1, with 30% of our sample having high school or less (compared to 39% of the U.S. population), 19.7% having some college (compared to 18% of the population), 12.2% having a two-year college degree (compared to 10% of the population), 23% having a four-year degree (compared to 21% of the population), and 15.1% having a graduate or professional degree (compared to 12% of the population).
The 43 items were narrowed to 15 based on several factors. First, within each C we wanted items that spanned a range of difficulties. This would allow each C to contribute to the full range of sum scores. Notably, we do not have reason to believe that one C is inherently more challenging (or less challenging) than any other C, which was supported by analyses that showed the average percent correct of all items within each C to be similar to the percent correct of items in the other C’s, in both Study 1 and Study 2. Therefore, representation across the range of possible difficulty parameters provides better coverage of the news literacy knowledge construct than having, for example, one C with only positive difficulty parameters and another C with only negative difficulty parameters. Second, the final measure needed to have equal representation of each C so that each C contributes equally to the sum score. If, for example, one C had five items and another had only two, then the sum score would be more impacted by performance on the former C and less by the latter. No single C is considered more important in the determination of news literacy than any other, as each plays a “connected and mutually constitutive” role in the comprehensive conceptualization of news literacy (Tully et al, 2022). Therefore, equal numbers of items from each C is appropriate given our desire to closely operationalize the theory upon which this work is based. Third, the measure needs to be comprehensive but as short as possible. Given that a 15-item multiple choice measure exists, we decided that 15 items would be the maximum number for the true/false measure. With these constraints in mind, we then compared the estimated difficulty parameter, infit, and outfit across items.
The 15 items were evaluated for potential DIF and revised. Differential item functioning between each pair of categories for each item was assessed using Lord’s chi-square test (p = .05). We then compared the item characteristic curves for pairs of categories that tested positive for DIF. Given that we were still in measure development, we reviewed all item wording with particular attention to those that exhibited potential for DIF. Item revision was based on two factors: (1) the detection of DIF, and (2) to allow for near equal distribution of true and false items.
Results
The set of 43 items were reduced to 15 items based on the previously described criteria. After re-estimation of item parameters using the Rasch model with the 15 items only, difficulties ranged from −1.15 to 1.67, infit values ranged from 0.81 to 1.08, and outfit values ranged from 0.70 to 1.07. Each item was reviewed and tested for DIF.
DIF testing flagged 10 of the 15 items as potentially problematic. The review of these items resulted in changing item wording. For instance, an item that stated “Employees at Facebook verify all news before it shows up on Facebook” was changed to be more platform agnostic (changed “Facebook” to “social media”). We adjusted three questions to change the correct answer from false to true (e.g., “It is common for journalists to be given deadline extensions if they can’t reach the best possible sources,” which is false, became “Journalists are rarely given deadline extensions if they can’t reach the best possible sources,” which is true). This resulted in 15 items with seven being true. Given that items were altered, an additional study for validation purposes was required.
Study 3: Validating the News Literacy Knowledge Scale
Participants and Procedures
The edited 15 true/false items were administered to 681 participants using Prolific, an online research platform in August 2022 (see Palan & Schitter, 2018; Peer et al., 2018). Participants were sampled to be representative of the population of the U.S. Of the 680 respondents in our sample, their mean age was 45.7 (SD = 16.3), 73.8% were white, and 50.9% identified as women. Again, the respondents were slightly older than the population. Like the first two studies, Study 3 also skewed more educated, though this time with substantially fewer participants with high school or less (15.4% compared to 39% of the U.S. population) and substantially more with bachelor’s degrees (36.9% compared to 21% of the population). Other education categories were within a few percentage points of the population.
Measures
News Literacy Knowledge True/False Items
In this third study we tested the reduced pool of 15 items consisting of three items per C.
News Literacy Knowledge Multiple Choice
Fifteen items were used as a previously employed measure of news literacy knowledge (e.g., Ashley et al., 2023). Like the new proposed measure, these 15 multiple-choice items (each with four or five response options, including a “don’t know” option) were designed to tap into the five C’s of news literacy we have previously proposed (Tully et al., 2022). Each item was recoded to indicate whether the answer was correct or incorrect (don’t know included with incorrect responses), before a summative scale was created (M = 9.84, SD = 2.88, ω = .68 [using ML procedure]). We used McDonald’s omega (ω) rather than the more familiar Cronbach’s alpha (α) as the former offers a more robust measurement of consistency that does not violate assumptions about inter-item relationships (for a discussion, see Hayes & Coutts, 2020).
Political Knowledge
Two separate indices were created to tap into political knowledge. The first scale used six multiple-choice items to measure a participants’ civic knowledge, including their knowledge of American political structures and institutions. The second scale included four multiple choice items measuring participants’ knowledge of current events at the time of fielding the survey. For both civics knowledge (M = 4.98, SD = 1.29, max = 6, ω = .69) and current events knowledge (M = 2.47, SD = 1.01, max = 4, ω = .49) a summative scale was used.
Demographics
In addition to our measures to describe content and criterion validity, our survey included a number of demographic variables to assess whether our true/false measure of news literacy knowledge differed by groups. For gender, we recoded a measure asking people to report their gender into female (N = 346, 50.9%) versus male (N = 325, 47.8%) to exclude the nine participants who reported they were non-binary. For education, we used a single item asking people to report their highest level of education using eight categories, ranging from did not complete high school to doctoral degree; we recoded this into four categories to account for low response categories: high school degree or less (15.4%), some or two-year college degree (32.9%), a four-year college degree (36.9%), or an advanced degree (14.7%). For race, participants were asked to check-all-that apply with regards to race categories, as well as asked a separate question about their ethnicity. Given the distribution of responses, we collapsed this variable into white (73.8%) versus minority (26.2%). Additionally, we used a single item asking people to report their age.
Political Orientations
As a subcategory of demographics, we also asked people about their political orientations. Specifically, participants reported what best described their party affiliation (M = 4.76, SD = 1.80) and their political ideology (M = 4.72, SD = 1.84) on 7 point scales, with a higher score indicating a more liberal or Democratic affiliation. We then recoded each of these measures to compute partisan strength (M = 1.66, SD = 1.03) and ideological extremity (M = 1.69, SD = 1.04), so that a higher value means a more extreme answer (i.e., a 7 or 1 for party affiliation becomes a “3” in partisan strength). Given the high correlation between these measures of party affiliation and ideology (r = .85, p < .001), we consider them separately in all analyses. Finally, we had three items which asked participants to rate their interest in national, state, and local politics on a 5-point scale, from not at all to extremely interested. These items were averaged to create a scale (ω = .88, M = 3.00, SD = 1.01).
Results
Construct Validity
Assessing dimensionality confirmed that the 15 true/false items represent a single factor. A principal components analysis was fit using the Rasch model residuals. The first eigenvalue for the principal components analysis of the residuals was 1.43, indicating that unidimensionality is plausible (<2; Boone & Staver, 2020). The argument for unidimensionality is further strengthened by the non-significant result of the modified parallel analysis test (p = .16; Drasgow & Lissak, 1983). Additionally, a CFA with the 15 items loading on a single factor yielded acceptable fit indices (i.e., Hu & Bentler, 1999), RMSEA = 0.026, 90% CI [0.016, 0.036], SRMR = 0.049, CFI = 0.960, TLI = 0.954.
The 15 true/false items ranged in IRT difficulty from −3.78 to 0.74, corresponding to a range of proportion of correct responses from 0.96 to 0.35. Infit ranged from 0.81 to 1.06 and outfit ranged from 0.55 to 1.04 (within the limits of Linacre’s, 2002, recommended range of 0.5 to 1.5). Regarding person fit, approximately 2.5% of respondents had standardized infit values in absolute magnitude greater than 1.96. Similarly, 2.6% of respondents had standardized outfit values in absolute magnitude greater than 1.96. Factor scores ranged from −3.35 to 1.49 (M = 0, SD = 0.81). When summing correct responses to produce a sum score of news literacy knowledge, sum scores ranged from 0 to 15 (M = 10.98, SD = 2.51). Figure 1 shows the item characteristic curves and Figure 2 the distribution of news literacy knowledge scores. More difficult items could be developed and added to the scale in the future.
Item characteristic curves of the 15 retained items. The item characteristic curve maps the probability of an individual answering a question correctly given their position on the underlying trait that is being measured. Using the Rasch model, all items have the same slope (fixed to 1) at the location of the difficulty parameter.
Wright Map displaying the location of each item’s difficulty parameter and the associated distribution of estimated latent trait scores in the sample. The distribution of scores is skewed negatively, such that outlying scores exist at the lower end of the latent trait range. Items are most useful for differentiating low and moderate trait levels, as seen in the distribution of item difficulties extending into the negative tail.
Reliability
Reliability in IRT can be assessed as a ratio of the variability in estimated factor scores to the sum of the variability in the factor scores and the square of their standard errors. Reliability is then an estimate of the proportion of variability that can be explained by the “true” latent ability. Reliability using this empirical approach was 0.70. While this value of reliability is generally considered acceptable during scale development (Cho & Kim, 2015; Nunnally, 1978), reliability could be improved by adding more difficult items.
Differential Item Functioning
We assessed for DIF across race (White, non-White), education (high school or less, some or two-year college, four-year college degree, college degree plus graduate education), self-reported political party affiliation (Republican, independent, Democrat), ideological leaning (conservative, moderate, liberal), gender (male, female), age category (18–35, 36–54, 55+), and self-reported political interest (low, medium, high). In our previous iteration of DIF testing (Study 2) we identified five items (one from each C) that did not exhibit DIF. These items were not changed and were used as anchor items in the assessment of DIF in Study 3. A likelihood ratio test was used in a multiple group analysis to evaluate if equality constraints on model parameters (assuming a lack of DIF) yielded statistically significantly worse model fit than if these parameters were freely estimated (Thissen et al., 1986). Given the large number of tests, a Benjamini-Hochberg correction was used to adjust p-values (Benjamini & Hochberg, 1995; Kim & Oshima, 2013). None of the items exhibited DIF across the assessed dimensions.
Convergent and Criterion Validity
As in any measurement development study, it is critical that we develop a reasonable argument that the true/false measure is measuring news literacy knowledge as opposed to another construct. Doing so can be done by establishing a nomological network, or a series of correlations between scores on measures to establish validity (Messick, 1989). The true/false measure developed here is an adaptation of a multiple-choice measure; therefore, a necessary first step is evaluating the correlation between the scores on the true/false measure and the multiple-choice measure. The second step is then evaluating if the two measures correlate similarly with other variables. Establishing that the true/false measure correlates with other variables similarly to the correlations found between the multiple-choice scores and those same variables contributes to the development of a validity argument: that the true/false measure is indeed assessing the same construct as the multiple-choice measure. An established relationship exists between the multiple-choice measure of news literacy knowledge and measures of political knowledge. Therefore, we correlate the true/false measure sum scores, true/false measure Rasch theta scores, scores on the multiple-choice measure of news literacy knowledge, and two measures of political knowledge. Doing so, we evaluate whether our true/false measure correlates with the multiple-choice measure and correlates similarly as the multiple-choice measure to scores on the political knowledge measures.
Table 1 contains the correlation matrix of our true/false measure and all other measures used to assess validity (see Appendix B for a correlation matrix of all analyzed variables). The sum scores and factor scores are very highly correlated; the correlation between sum scores and the multiple-choice news literacy knowledge measure were strong and positive (r = .647); the correlations between our true/false measure and the remaining measures used for validation were similar in magnitude and direction to the correlations between the multiple-choice news literacy knowledge measure and political knowledge. In this process, we establish that sum scores from the true/false measure are well correlated with scores on the multiple-choice measure of news literacy knowledge and that the true/false scores also correlate in similar ways as the multiple-choice measure of news literacy knowledge with measures of political knowledge, demonstrating convergent validity. This result contributes to the validity argument that the construct measured by the multiple-choice scale and the construct measured by the true/false scale is the same.
Descriptive Statistics and Correlation Matrix for the Factor and Sum Scores of the 15-Item True/False Measure of News Literacy Knowledge, and All Other Measures Used to Assess Convergent Validity.
Note. NL MC = news literacy knowledge multiple choice measure; Civics = knowledge of political structures and institutions; Current = knowledge of current events.
p < .001.
In addition to the relationship between news literacy knowledge and political knowledge, we know that news literacy knowledge systematically differs across groups in society; therefore, we next evaluate the relationship between the true/false measure of news literacy knowledge with the multiple-choice measure while controlling for other meaningful demographic variables. These results (see Table 2) reinforce the zero order correlations reported above. Even controlling for age, gender, education, race, and political orientations, our true/false measure is significantly associated with the multiple-choice measure of news literacy knowledge, explaining an additional 28.4% of the variance in the multiple-choice measure, above and beyond those demographic differences.
Regression Analysis Predicting the Multiple-Choice News Literacy Knowledge Scale, Civics Knowledge, and Current Events Knowledge.
Note. Standardized beta coefficients are reported to facilitate comparison. NL MC = news literacy knowledge multiple choice measure; Civics = knowledge of political structures and institutions; Current = knowledge of current events; NL TF = news literacy knowledge true/false measure.
p < .01. ***p < .001.
Turning to our measures of criterion validity, we examine whether our true/false measure of news literacy knowledge is associated with civics knowledge and current events knowledge. We again control for the demographic factors that are associated with knowledge, given its variation in the population. These results provide evidence of the criterion validity of our true/false measure of news literacy knowledge: the measure is significantly correlated with both civics and current events knowledge, accounting for an additional 6.1% of variance in civics knowledge and 3.2% of variance in current events knowledge. This is consistent with Maksl et al. (2015), whose measure of news literacy knowledge structures was significantly correlated with current events knowledge. Notably, the size of the effects of our true/false news literacy knowledge measure rivals those of more traditional demographic predictors of knowledge like age, education, and political interest.
Given these results, we explore differences between our participants in terms of their news literacy knowledge using the true/false scale proposed. Our regression suggests that meaningful differences exist in who is more or less news literate (see Table 3). Specifically, older adults, white participants, men, more educated adults, those who have a more liberal political ideology, and those with more extreme ideological beliefs all tended to have higher levels of news literacy knowledge, results consistent with research using similar knowledge-based measures of news literacy (e.g., Maksl et al., 2015; Vraga et al., 2015).
Regression Analysis Predicting the True/False News Literacy Knowledge Measure.
Note. Standardized beta coefficients are reported to facilitate comparison
Creating four categorical measures of education, with high school degree as the excluded category, does not fundamentally change the results presented here. Having some college (B = 0.18, p < .001), a four year degree (B = 0.24, p < .001), or an advanced degree (B = 0.27, p < .001) are all associated with higher news literacy knowledge as compared to a high school education; it also does not change the significance of any of the other demographic predictors.
Breaking race into three categories—black, white, or other minority—and using white as the reference category produces similar effects. Specifically black participants (B = −0.18, p < .001) and other minority participants (B = −0.14, p < .001) had lower news literacy knowledge as compared to white participants. Unfortunately, small sample size prohibits us from breaking out other racial groups such as Latino or Asian from our more general “other” category.
Using party affiliation, rather than political ideology, produces substantively similar results. Being more strongly affiliated with the Democratic party is associated with higher news literacy knowledge (B = 0.10, p < .01), as is extreme partisan strength (B = 0.08, p < .05).
p < .05. **p < .01. ***p < .001.
Discussion
The rise of media fragmentation and misinformation require attention for democratic participation and civic life to flourish. News literacy is an educational effort that can help individuals learn to better navigate the complex media landscape and find their way to more accurate representations of reality. While much work has been done to advance a conceptual understanding of news literacy (e.g., Amazeen & Bucy, 2019; Ashley, 2020; Johnson et al., 2021; Tully et al., 2022; Vraga et al., 2021), the work of concept explication and theory building requires a clear operationalization of these new concepts (McLeod & Pan, 2005; Shoemaker et al., 2004). Although existing work shows the promise of news literacy for making predictions about a range of significant outcome variables such as political knowledge and engagement, the mix of tools and techniques for defining and measuring news literacy in different ways hinders our confidence in the meaning of this work. This study enables this theory-building by advancing a measurement of news literacy knowledge that can be easily deployed and adapted in a variety of contexts. This measurement tool is based on rigorous conceptualization and has been thoroughly psychometrically evaluated. The 15-item true/false news literacy knowledge measure also shows significant relationships with important outcome variables that capture prosocial behaviors that can help support stable, self-governing societies.
The value of this new measure rests on two primary features. First, it provides a comprehensive operationalization of news literacy as we have previously proposed (Vraga et al., 2021) and further conceptualized (Tully et al., 2022). Defined in part as the “knowledge of the personal and social processes by which news is produced, distributed, and consumed” (Tully et al., 2022, p. 5), the concept is organized around five domains (context, creation, content, circulation, and consumption). Our 15-item measure uses these five C’s as the underlying framework for item development and refinement. Scholars have found similar knowledge-based measures valuable in assessing news, media, and related literacies (e.g., Amazeen & Bucy, 2019; Chan et al., 2021; Dogruel et al., 2022; Geers et al., 2020; Johnson et al., 2021; Maksl et al., 2015; Vraga et al., 2015). These domains are comprehensive in how they emanate from significant scholarship in fields such as media sociology and psychology about how news is produced, distributed, and consumed.
Though the measure is comprehensive in its foundations, its actual construction is concise, which is its second valuable feature. News literacy as a concept is typically not measured in isolation. Some scholars and especially educators have been interested in its antecedents, including designing and assessing educational interventions intended to build greater news literacy. Others have explored how news literacy may be a factor in predicting prosocial outcomes, including political engagement, news use, and voting. For example, we have posited a comprehensive model to predict what we call “news literacy behaviors” (such as news consumption, fact-checking, or identifying misinformation; Vraga et al., 2021). Based on the Theory of Planned Behavior, our model of news literacy behaviors includes other concepts such as perceptions of attitudes, norms, and perceived control over one’s ability to engage in such behaviors. To fully test this and other complex and comprehensive theories where news literacy may play only a small part requires a set of items concise enough to allow for multiple other measures to be simultaneously deployed without making a questionnaire too onerous for respondents. Other approaches, such as multiple-choice items, had led to a piecemeal approach where only a subset of items are usually presented, which lacks full conceptual validity.
In addition to its conciseness value, constructing true/false items instead of multiple-choice questions increases adaptability, especially across differing cultural and legal boundaries. The adaptability of the measure is critically important as more comparative news literacy research is needed (Schulz et al., 2024). Research tends to focus on Western democracies, which limits the scope and applicability of the findings. Understanding news literacy in diverse news and political environments should remain a priority for news literacy research. True/false items can be more easily adapted in part because they eliminate the need to come up with distractor options, which could create additional error and noise in validity of items in their measurement of intended concepts. Though some concern remains about the reliability of “true/false” items because the lack of options could increase the likelihood that a respondent guesses the correct answer, experts in measurement design suggest that true/false items constructed well—such as through using more false than true statements or wording items so that superficial ability suggests the wrong answer—mitigate these concerns (e.g., Ebel & Frisbie, 1991; Haladyna & Rodriguez, 2013). Additionally, our inclusion of the “don’t know” option further reduces the effects of guessing by allowing respondents to indicate their lack of knowledge.
Not only is this new scale based on a clear conceptual definition of news literacy and informed by literature that shows a clear need for a comprehensive, concise, and adaptable measure, it also has been developed using a comprehensive scale-building process with data collected across three studies. We used IRT to assess model fit and difficulty of items, using the Rasch model to create a final set of items for which sum scores on the number of items correctly answered would be an adequate measure of a respondent’s underlying news literacy knowledge. Further, we use differential item functioning analysis to assess how respondents with the same underlying ability but different demographic characteristics answered each question, adjusting item wording to address potential discrepancies. And finally, we established convergent and criterion validity by analyzing the relationship between the 15-item true/false scale and related measures. By establishing the similarity between the true/false measure and the multiple-choice measure as well as the similarity of relationships between these two and measures of political knowledge, we have provided evidence that the true/false measure is indeed measuring news literacy knowledge. More specifically, we found that the scale positively predicted civics and current event knowledge, controlling for various demographic and political variables, findings similar to other measures of news literacy. There was also a very strong, positive relationship between the 15-item true/false scale and a similar multiple-choice news literacy knowledge scale. Finally, the 15-item true/false measure exhibited similar relationships to demographic variables as other news literacy measures.
One might question the need for a new measure that shows such equivalency to existing measures. However, that equivalence is important, because the usefulness and value of this measure stands not only on psychometric properties but on its closer conceptual alignment with news literacy theory and most importantly its adaptability and speed. In clinical scholarship such as medicine, an intervention that shows equivalent or non-inferior outcomes may still be preferred if, for example, it shows additional value in other ways, such as ease of administration (Schumi & Wittes, 2011). In other words, even though other measures have operationalized news literacy, including those that we’ve used to establish concurrent validity of the measure developed here, our proposed 15-item true/false measure is more concise, can be more easily integrated into surveys that measure other constructs, and can be more easily adapted across cultural contexts, qualities that make it a significant contribution to news literacy scholarship. Additionally, these updated items better reflect the current realities of the complicated media system. While these items will undoubtedly need to be adapted and updated as researchers apply these measures in different media contexts and as the worldwide information ecosystem continues to change, this article also provides a roadmap for researchers to make these adaptations.
These studies are not without some methodological limitations. For example, though we worked to choose items across a range of difficulties, our respondents still scored highly on the 15-items news literacy knowledge measure (Mstudy 3 = 10.98, SDstudy 3 = 2.51). In adapting the items for further use, scholars may work to create more difficult items, which may also improve reliability of the measure. Our samples used in these studies also skewed older and more educated than the U.S. population, so more representative samples could address these concerns. Also, while news literacy knowledge-focused assessments have been used to study news literacy among school-aged participants (e.g., Craft et al, 2016; International Research & Exchanges Board, n.d.), scholars should specifically explore the usefulness of this true/false measure with youth and young adults, especially given that news literacy is often taught in schools and colleges. Additionally, we did not independently assess the discriminant validity of the 15-item true/false measure, though we are confident that it is measuring the intended construct, especially given its strong relationship to the 15-item multiple choice measure used to establish convergent validity. Finally, the measure was developed through a rigorous process that narrowed a set of 80 items addressing the conceptual definitions of all five C’s down to a 15-item true/false measure. While any such reduction may result in some loss of definitional nuance, that loss may be outweighed by the ease of use and adaptability of a more succinct measure. Our analyses show the measure to be empirically valid, though scholars ought to take special note when adapting the items to ensure alignment between the most important and salient aspects of a given C in a particular context and the true/false items used to measure it.
One of the chief values of the measure we’ve proposed is its conciseness, useful for inclusion in longer questionnaires used to test more comprehensive theoretical models, especially those that posit news literacy plays a role in various prosocial behaviors. Future research should use the measure in this way, not just exploring the relationship between news literacy and such behaviors but also the role of attitudes, norms, and perceived control and intention to engage in behaviors like identifying misinformation, verification, and seeking diverse sources of information. Such work could more concretely isolate and explain the individual and combined effects of these variables on such behaviors. Additionally, the measure herein proposed is chiefly a measure of news literacy knowledge. Future scholarship should also operationalize skills, in that news literacy consists of both knowledge and skills. Nonetheless, these studies present strong evidence of both the conceptual value of this measure and our comprehensive development process establishing its validity. We are confident that this scale will provide value in extending the study of news literacy, providing scholars with a theoretically grounded and empirically valid approach useful at both evaluating the efficacy of news literacy interventions and in exploring the role news literacy plays in predicting a range of behaviors.
Supplemental Material
sj-docx-1-crx-10.1177_00936502241284406 – Supplemental material for Developing and Validating a 15-Item True/False Measure of News Literacy Knowledge
Supplemental material, sj-docx-1-crx-10.1177_00936502241284406 for Developing and Validating a 15-Item True/False Measure of News Literacy Knowledge by Adam Maksl, Peter J. Boedeker, Emily K. Vraga, Stephanie Craft, Melissa Tully and Seth Ashley in Communication Research
Footnotes
Appendix A
Appendix B
Correlation Matrix for Study 3.
| Measure | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1. NL TF | 1 | ||||||||||
| 2. NL MC | .665*** | 1 | |||||||||
| 3. Current | .326*** | .331*** | 1 | ||||||||
| 4. Civics | .472*** | .449*** | .466*** | 1 | |||||||
| 5. Age | .261*** | 0.063 | .113** | .383*** | 1 | ||||||
| 6. Gender (female) | −.167*** | −.182*** | −.275*** | −.193*** | 0.039 | 1 | |||||
| 7. Education | .268*** | .263*** | .241*** | .299*** | .191*** | 0.021 | 1 | ||||
| 8. Minority | −.224*** | −.150*** | −0.024 | −.125** | −.140*** | 0.005 | .081* | 1 | |||
| 9. Ideology (liberal) | .146*** | .272*** | .120** | 0.060 | −.182*** | 0.076 | .138*** | 0.039 | 1 | ||
| 10. Ideological extremity | .163*** | .112** | 0.042 | .126** | −0.041 | 0.065 | 0.052 | −0.073 | .283*** | 1 | |
| 11. Political interest | .245*** | .195** | .220*** | .370*** | .194*** | −0.030 | .198*** | −0.049 | .123** | .214*** | 1 |
Note. Small differences between the correlations in this matrix and the correlations in Table 1 are due to additional cases being eliminated for a complete case estimation of the correlations. NL TF = news literacy knowledge true/false measure; NL MC = news literacy knowledge multiple choice measure; Current = knowledge of current events; Civics = knowledge of political structures and institutions.
p < .05. **p < .01. ***p < .001.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors received grant support from the Mass Communication and Society Division of the Association for Education in Journalism and Mass Communication to conduct this research.
Supplemental Material
Supplemental material for this article is available online.
Notes
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
