Abstract
Objective:
Women aged 45 or younger with breast cancer, or those who are at high risk of breast cancer due to a family history of the disease or genetic test results indicating risk, have distinct health risks and needs. Facing Our Risk of Cancer Empowered (FORCE), a non-profit advocacy organisation, developed an online resource designed to address these needs.
Design:
In-person surveys were administered to compare baseline and post-test knowledge of two cohorts of 55 volunteer participants. Both groups read a printed media news article related to breast cancer research. The intervention group also read a printed brief review that provided a summary of research findings, and discussion and ratings of research evidence and reporting quality.
Setting:
Participants were recruited at two FORCE conferences. Data were collected at the conference sites.
Method:
Pilot and follow-up sample results were pooled and analysed using t-test comparisons.
Results:
The intervention resource use was associated with a significantly greater increase in knowledge than only reading the media news article.
Conclusion:
An online tool developed to respond to audience needs, offering ratings of evidence quality and relevance, can help readers to better understand research reporting.
Keywords
Introduction
Women aged 45 years or younger with breast cancer, or those who are at high risk of breast cancer due to a family history of the disease or genetic test results indicating risk, have distinct health risks and needs from their older counterparts. For example, breast cancer risk, aetiology, treatment, outcomes, and survivorship and quality-of-life concerns often vary between younger and older women (Freedman and Partridge, 2013; National Cancer Institute, 2014). Diagnosis at a young age is associated with higher risk of recurrence, second malignancy, and mortality, morbidity and quality-of-life issues (Freedman and Partridge, 2013).
These distinct issues facing young women with or at high risk of breast cancer correspond to distinct needs for information about the latest research and recommendations regarding the benefits and risks of prevention and treatment options as well as genetic risk and testing. Young women frequently seek health information through the Internet and mainstream media, but often find it does not address their particular concerns, that it is difficult to evaluate or interpret, or even that it is misleading (Rowlands et al., 2015).
Often the media miss or omit the nuances of a research publication (West and Bergstrom, 2021). Constrained by deadlines, space or relevant context, media articles about health can appear to contradict one another, causing confusion in those they intend to inform (Dobransky and Hargittai, 2012; Freedman, 2013). Interpreting media articles on research findings, including determining the implications for younger women, is often challenging for readers (Fergie et al., 2013). Information available through the Internet and mainstream media often has significant gaps in addressing the needs of young women diagnosed with breast cancer (Rowlands et al., 2015). Information about the risk is often unclear, which may lead readers to be less likely to take preventive measures (Fergie et al., 2013). Research findings indicate that young women with breast cancer or at high risk of breast cancer need accurate, clearly presented information based on sound scientific evidence to help them make informed decisions about their specific health needs.
Vulnerable audiences seeking health information may be challenged to identify credible sources that can serve the important function of sharing relevant, accurate and understandable information that is pertinent to health decisions. This need emphasises the importance of critical media literacy, a core component of which is ability to analyse and evaluate a broad range of media messages (Kellner and Share, 2019). The ability to assess health information critically is a core component of health literacy. With national data indicating that only 12% of adults in the USA have good health literacy, improved health literacy has become a public health priority. Benefits of improved health literacy include understanding causes of breast cancer and links between lifestyle factors and disease outcomes (Sheng and Schultz, 2022).
To help women better assess the accuracy, relevance and health implications of media articles about new research, we developed an online resource that provides brief plain language reviews that discuss the quality and implications of recent research relevant to people with or at elevated risk of breast cancer as well as quality of media articles about the research. The programme was funded by a cooperative agreement from the US Centers for Disease Control and Prevention and implemented by Facing Our Risk of Cancer Empowered (FORCE), a non-profit advocacy organisation serving people and families affected by hereditary cancer. The resource was designed specifically to serve young women with or at high risk of breast cancer. The programme microsite allows users to browse or search for reviews based on publication date, cancer type and subtype, topics, such as treatment, side effects, prevention, menopause, and tumour testing, and keywords. Search results yield links to programme reviews of media news articles about research related to breast cancer. Reviews rate the quality and relevance of new research, and the quality of media reporting about the research; suggest question prompts that may be useful for readers to address with their health care providers; and provide information on any related clinical trials. An About the Programme page describes the process of rating information, rating criteria, reviewers and reviewer qualifications.
Two senior researchers who work for the advocacy organisation select research papers for review from those suggested by users, research currently receiving a lot of public attention, findings with the potential to generate confusion in priority audiences and recent findings with potential to result in major changes in clinical practice or health outcomes. All articles are available on the Internet. Nearly all articles are reported through news outlets. Some are peer-reviewed scientific journal publications. Most are free of charge. In some cases, only the abstract is available without charge. The senior research staff scientists use the following process to develop the research reviews: select and prioritise media news articles and peer-reviewed journal articles for review; read and critically analyse the relevant research publication; rate scientific evidence quality, rate relevance to clinical decision-making, summarise key findings in plain language and develop questions users can ask clinicians in order to support shared decision making informed by recent research; evaluate the quality of the media coverage of the science; and provide information about any available relevant national guidelines and other resources as well as related clinical trials enrolling patients. When findings are reported by multiple new outlets, reviews rate and comment on coverage by each source.
The scientific accuracy of each review is determined by members of FORCE’s external scientific advisory board, which comprises active researchers and clinicians.
One critical aim of the resource is to summarise the information about recent research to make it clearer for lay audiences. A pilot assessment of 19 participants aged 18–45 years demonstrated that those who had read an article review offered through the online resource in addition to a media news article improved their knowledge about the information reported significantly more than participants who had read only the media news article (Yi et al., 2019b).
Participants in the pilot assessment also participated in focus groups. During these focus groups, all participants reported that the online educational resource filled a critical need for information among young breast cancer survivors and previvors (survivors of a predisposition for cancer), that it helped users determine which recent media articles are relevant and accurate, and that it would facilitate shared decision making with healthcare providers. Focus group participants indicated that they found the organisation and the review process to be trustworthy and credible. Participants also agreed that they trusted the processes used to develop the educational research reviews and that they found these reviews to be useful for determining which media articles on health research were most accurate and relevant to the needs of young women with or at risk of breast cancer.
The current study was conducted to assess whether earlier results would be replicated and to assess the effects of the educational intervention with a pooled sample analysis of pilot and follow-up data. Follow-up data results were presented at the 2019 San Antonio Breast Cancer Symposium (Yi et al., 2019a).
Methods
For the pilot and follow-up assessment, the Evaluation Lead collected pre- and post-test data from volunteer participants at two conferences for people at risk of hereditary cancer, hosted by the organisation that developed the resource. All data collection and analyses were conducted independently of the organisation offering the educational resource.
A member of the project’s independent steering committee and the scientific lead (L.R.) reviewed and approved data collection procedures to ensure they complied with ethical standards regarding participant risk and burden. Participants were notified that participation was voluntary and would not affect access to any conference activities or resources and that responses were confidential. Personally identifying information was collected only to ensure participants received incentive gift cards and was not linked to responses. Individual-level health status data were not collected.
Conference materials described assessment activities and compensation, and invited attendees who met the age criteria and who either had a history of breast cancer or genetic risk for breast cancer to register to participate in assessing a new online educational resource. Logistical resources allowed for as many as three groups in meeting rooms at the first conference, where pilot data were collected, and four groups in meeting rooms at the second conference, where follow-up data were collected, with a capacity of as many as 12 participants in each group. A total of 59 people enrolled to participate and 55 provided survey data. Assessments were conducted with small groups in meeting rooms in order to facilitate discussions about the online resource.
Instruments
A senior researcher from the non-profit advocacy organisation selected a single media news article and associated review for each assessment session. News articles were selected from those presented via the online resource. Articles selected were considered relevant to broad audiences. Topics included recent research finding related to survival rates of women with a BRCA mutation, treatment of metastasis in mice, and effects of a meatless diet, immunotherapy, acupressure, chemotherapy and exercise on breast cancer outcomes. Biases and limitations discussed in reviews included a headline that was misleading regarding the degree of risk associated with BRCA mutations, inappropriate generalisations based on a single case study and inappropriate assumptions of implications of animal research for human patients. Materials were provided in print format to allow participants to write notes and comments. The Evaluation Lead (R.H.P.Y) developed a set of five questions about factual information presented for each of seven educational summaries about the content of the media news article reviewed.
Participants
Participants were 54 women and 1 transgender man between ages of 18 and 55 years, who attended either conference and volunteered to participate in one of seven groups. Participants were asked to report all applicable race/ethnicities. A total of 51 participants were White, 2 were African American, 1 was Latine, 2 were Asian and 2 did not provide information about the race or ethnicity.
Procedures
At the beginning of each session, the Evaluation Lead (R.H.P.Y.) explained the purpose of the study, randomly assigned each participant to the intervention or comparison group, and assigned each participant a random unique identifier. Participants were asked not to record their names or other personally identifying information. Using paper and pen, participants then answered the five questions corresponding to the news article selected for their group prior to reading the news article. Participants were instructed to answer independently and not to discuss answers with each other.
Both groups of participants then read a media news article. Participants in the intervention group also read the corresponding educational review in print format. After reading the material, participants answered the same questions, following the same instructions. There was no time limit on reading or test completion. All groups took approximately 30 minutes to complete pre-test surveys, assigned reading, and post-test assessment. Because they had less material to read, comparison group participants typically completed assessment shortly before intervention group participants.
Following completion of the knowledge assessment, participants participated in a 30- to 45-minute discussion about the usefulness, relevance, clarity and appropriateness of the educational resource materials. Results of focus groups were published separately after each conference (Yi et al., 2018).
Results
Analysts pooled data collected at each of the two conferences to compare groups’ changes in knowledge. A comparison of the intervention and comparison groups’ scores at pre-test indicated they were not significantly different, t (53) = −1.43, p > .05. Table 1 presents within-group comparisons of pre- and post-test changes.
Results of within-group comparisons.
ns: not significant.
A between-group comparison of average pre- and post-test change indicated that the groups’ pre- to post-test changes differed significantly, t (53) = 3.57, p < .001. Figure 1 illustrates these changes in user knowledge.
Pre- to post-test changes in user knowledge.
Pooled analysis results show that the comparison group’s knowledge did not change significantly between pre- and post-test, that the intervention group’s knowledge increased significantly between pre- and post-test and that the differences in change scores between the two groups were statistically significant.
Discussion
Future directions
An online tool developed to respond to lay audience needs, offering ratings of evidence quality and relevance, helped study participants to understand media articles on breast cancer, suggesting that this resource is a promising approach for offering lay audiences credible information that is relevant to their health decisions.
A transparent description of the review process and reviewer qualification on the About page of the tool assured users that credible gatekeepers had assessed information. Future research should assess the utility of the resource for diverse audiences. Evaluations should also assess the effectiveness of the educational reviews in accomplishing other aims, such as teaching audiences about reporting limitations and facilitating shared decision making with healthcare providers.
FORCE continually solicits users’ input regarding resource utility, format, content and accessibility. Future evaluations should assess whether changes to the programme made in response to users’ requests are associated with changes in the resource’s impact on users’ understanding of media articles on breast cancer–related research. It would be useful to develop and evaluate similar resources to support lay audiences in assessing other types of media articles on scientific research.
Limitations
Limitations of this study include the fact that participants were attendees of conferences on hereditary breast cancer, hosted by the organisation that developed the resource, suggesting that they are highly motivated to seek information about cancer and may have had relatively high levels of health literacy. Most participants were White.
These findings may not be generalisable to people who do not already find the host organisation to be trustworthy, who are not highly motivated to seek health information or who identify as racial or ethnic minorities.
Participants at the first FORCE conference (2017), during which data were collected, were not asked to provide socioeconomic data. The organisation has updated its data collection protocol to collect education and annual household income data and will continue to collect more detailed demographic data as part of ongoing evaluation. This will provide information about the generalisability of findings and the effectiveness of the organisation’s efforts to reach diverse audiences.
Conclusion
Results indicate that an online tool can help laypeople to understand media articles on breast cancer research. The intervention group’s correct answers increased by an average of about 25%, whereas the percentage of the comparison group’s accurate responses did not change significantly. While not statistically significant, the comparison group’s accuracy decreased. It is possible that this may be partially due to the process for selecting media reports for review: FORCE prioritises reviewing reports that are likely to be confusing or misleading. It may be useful for the purpose of determining future priorities and the planning of future evaluations to assess whether reading media articles with poor quality reporting is associated with decreased understanding of the research and results presented.
Footnotes
Author Note
The contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the US Department of Health and Human Services.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: this project was funded the US Centers for Disease Control and Prevention under Grant/Cooperative Agreement No. DP005404.
