Abstract
Health literacy is a driver of community empowerment and a key determinant of health and equity. The World Health Organisation has established a strong global mandate for addressing health literacy through public policy action, by positioning it as one of three key pillars for achieving sustainable development and health equity in the Shanghai Declaration on Health Promotion. Policy document analysis is a useful way of determining the strengths and limitations of past and current policies, as well as the likely success of their implementation and impact on health outcomes. In this study, we developed a framework for analysing policy documents to assess the extent to which they prioritise and operationalise an emergent public health issue, in this case health literacy. Specifically, the framework enables a systematic exploration of (i) the stated policy goals and strategic objectives; (ii) the stated policy actions; and (iii) whether there is an intention to resource and monitor the stated policy activities. We tested the utility of the framework by applying it to a set of public policies in the Australian context. The framework presented in this study may be a useful tool for systematically analysing policy documents to reveal their strengths and limitations, and the extent to which they prioritise and operationalise health literacy.
Background
Health literacy is a multidimensional concept that is broadly defined as the cognitive and social skills that determine the motivation and ability of individuals to access, understand and use information in ways that promote and maintain good health (1). It is now widely acknowledged that health literacy is context specific, and that a person’s ability to access, use and understand information is greatly influenced by the environment in which they are required to apply the information to make health-related decisions (2,3). Health literacy is a key determinant of health, and poor health outcomes resulting from limited health literacy signal structural injustices within health and social systems. Improving health outcomes and reducing health inequities that arise from limited health literacy requires that governments develop and implement effective policies to address it.
Population studies conducted over the last decade indicate that limited health literacy is a public health challenge across the world, including in high-income countries. Nearly half of the adult population in the United States (U.S.A) has limited literacy skills, which has significant implications for health-related literacy in the U.S.A (4). In both Canada (5) and Australia (6), 60% of the adult population lack the functional health literacy required to meet the demands of everyday life. In Europe there is variation in health literacy levels between countries. Results from the European Health Literacy Survey found that 20–62% of respondents rate themselves as having difficulty accessing, understanding and using health information and services (7).
Low functional health literacy across populations, coupled with the growing body of evidence linking low health literacy with poor health outcomes (8) has led to the emergence of health literacy as a key public policy issue. In many contexts, this has been driven by a quality and safety agenda, in which the primary objective is to improve patient safety and reduce clinical risks and adverse events. In others, it has been motivated by a desire to make health services and systems more consumer centred, to support self-management, and to increase the role of consumers in decision-making about their own health. Health literacy appears in health policies in Australia (9–11), the U.S.A (12) and Europe (13), and has been incorporated in several high profile policy statements (14–16). Increasingly, researchers and public health organisations are calling for effective health literacy policy development and implementation (17). The World Health Organisation (WHO) has been instrumental in influencing and shaping the public policy agenda at a global level. This has included the creation of a global coordinating mechanism on health literacy (18), and more recently the positioning of health literacy as one of three key pillars for achieving sustainable development and health equity, as set out in the Shanghai Declaration on Health Promotion (19). In doing so, the WHO has established a strong mandate for countries around the world to prioritise health literacy through public policy development and implementation.
Effective policy development is critical to ensuring that health literacy is adequately prioritised, resourced, operationalised and monitored (20,21). Policy analysis is a useful way of understanding the strengths and limitations of past and current policies. More specifically, policy document analysis is a useful method for determining the extent to which policies include the dimensions necessary for successful implementation, and therefore for strengthening future policy development (22). It is also useful for assessing the uptake of research evidence in policy (23). However, frameworks for guiding policy document analysis are currently lacking.
In this study, we developed a framework for systematically analysing policy documents to assess the extent to which they prioritise and operationalise an emergent public health concept. We then tested the utility of the framework by applying it to a set of public policies relevant to the health and social services sectors in Victoria, Australia. In doing so, we sought to (i) develop a framework that allows identification of the strengths and weaknesses of disparate policy documents and for comparisons to be made between them, and (ii) determine what the application of the framework reveals about the health literacy policy environment in Victoria, Australia.
Methods
This study was undertaken in two parts. Firstly, we developed a framework to enable the systematic analysis of public policy documents for their prioritisation, operationalisation and resourcing of health literacy. Secondly, we compiled and analysed a set of policies relevant to health and social services within Victoria, and analysed them using the framework.
Development of a policy analysis framework
We developed the framework (Table 1) to facilitate judgements about the quality and potential of a policy to address health literacy and improve health literacy-related outcomes. It was developed on the assumption that intended outcomes of public policies are more likely to be achieved if the policy issue is well-defined, specific goals and actions are stated, sufficient resources are allocated, and policymakers are accountable through monitoring mechanisms. These assumptions are based on recommendations by the Commission on the Social Determinants of Health for developing public policy (20).
Framework for analysing health literacy in public policy documents.
Related concepts applied in this study: health education, patient activation, cultural competence, patient empowerment, treatment adherence, literacy, cognitive capacity, self-management, consumer participation, service access, health promotion, peer education.
The framework was derived by combining elements of two existing policy analysis frameworks, the EquiFrame Framework (24) and the ADEPT (Analysis of Determinants of Policy Impact) Framework (25). The EquiFrame was designed to analyse policy documents for the extent to which they address equity and human rights. It is comprised of four dimensions: (i) the concept of interest is only mentioned, (ii) the concept of interest is mentioned and explained, (iii) specific policy actions to address the concept are stated, and (iv) there is an intention to monitor the concept of interest. The ADEPT was designed to explain and influence the development and implementation of health promotion policies, and assess their impact (25). It has four dimensions: (i) goals, (ii) obligations, (iii) resources, and (iv) opportunities.
Both frameworks have useful elements for undertaking policy document analysis. However, in order to comprehensively analyse the policy dimensions relevant to our aims, an adapted framework was required. Thus, we incorporated the first two dimensions of EquiFrame (the concept of interest is only mentioned; the concept of interest is mentioned and explained) into Category 1 of our framework, which examines whether health literacy or related concepts are mentioned or defined.
We incorporated the goals criterion of the ADEPT framework, which examines whether goals are well defined and concrete enough to achieve a desired outcome, into Category 2 of our framework, which examines the extent to which health literacy is prioritised within public policy. To derive Category 3 of our framework, we combined the third (specific actions identified) and fourth (intention to monitor) dimensions of EquiFrame, with the resources dimension of ADEPT (which examines whether sufficient financial and human resources are allocated to a specific issue in a policy). Category 3 examines the extent to which health literacy is operationalised within public policies through specific health literacy actions, and whether these actions are supported through the allocation of resources and the establishment of monitoring mechanisms.
Procedure for testing the framework
To illustrate the way in which the framework operates, we undertook a review and analysis of a set of policies relevant to health and social services within Victoria. An overview of the review process is provided in Figure 1.

Overview of the policy review process.
Document selection procedure
All health and social wellbeing-related policy documents published by Australian and Victorian government departments/bodies, and available on their websites were included in the review. We included all departments that have a role in developing health and wellbeing related policy, recognising that non-health specific departments often share this responsibility. Only documents deemed ‘current’ at the time of the search (July–September 2014), and which intended to achieve at least one of the following were included: (i) define a vision for population health and wellbeing, (ii) establish priorities, targets and performance indicators for achieving health outcomes, (iii) outline the requirements/expectations of funded organisations in Victoria. Hence, the following policy types were included; framework, agreement, implementation plan, strategic plan, operational plan, action plans, strategy, roadmap, strategic priorities document, policy, program guidelines, and program plan.
One member of the research team carried out an audit of government websites and downloaded the documents that met the inclusion criteria. Two researchers then analysed the text of the documents using the following procedure.
Document analysis procedure
Firstly, documents were assessed against Category 1 of the framework and assigned a rating between 0 and 3, according to whether health literacy was mentioned, defined or explained. Only documents that explicitly mentioned health literacy against Category 1 were analysed against Categories 2 and 3.
The retained documents were then analysed against Category 2 of the framework. Documents were assigned a rating of 1 if they mentioned health literacy; a 2 if they explained the relationship between health literacy and health outcomes, but did not clearly identify it as being a strategic priority; and a 3 if it was evident that health literacy was a strategic priority. A document was judged as including health literacy as a strategic priority if it specifically stated it as such, or it positioned health literacy as a strategic direction, or a policy goal, aim or objective.
Finally, the remaining documents were analysed against Category 3 of the framework. Documents were assigned a rating of 0 if they did not include specific health literacy actions, and a rating of 1 if they included specific health literacy actions. If in addition to actions, documents specified the monitoring of health literacy outcomes or they allocated resources to support policy implementation, they were given a rating of 2. Documents were only given a rating of 3 if specific actions were documented, monitoring of outcomes was specified and resources were allocated to support policy implementation.
The two researchers then compared their results to confirm their ratings matched for each category. In instances where there was a discrepancy in a rating, the researchers re-analysed the policy together to reach a consensus on the final rating.
Results
An audit of government websites yielded 94 relevant documents, of which 10 documents did not meet the inclusion criteria and were excluded from further review. The remaining 84 documents were analysed against the framework.
Prominence of health literacy within public policies
Of the 84 documents analysed, 29 explicitly mentioned health literacy and only 11 of these provided a definition or explanation of health literacy. Seven documents provided a definition incorporating the access, understand and use health information components of health literacy, whilst six referred to the ability of people to make health-related decisions. Two documents expanded the basic definition of health literacy to describe the ‘health literacy environment’, which referred to the infrastructure, policies, processes, materials and relationships that influence health literacy.
Prioritisation of health literacy within public policies
Ratings of the 29 documents against the framework are shown in Table 2. Nine of these documents positioned health literacy as a strategic priority, which was expressed in various ways. For example, the Australian Commission on Safety and Quality in Health Care afforded health literacy the highest level of strategic prioritisation by having dedicated health literacy policies (11,26), and by including it as a key action area under the ‘consumer centred’ strategic priority in the Australian Safety Quality Framework for Health Care (27). Two Victorian Government policies described health literacy as either a strategic direction or strategic priority (28,29).
Rating of policy documents against the analysis framework.
HL: Health literacy.
In the National Women’s Health Policy, health literacy is a key component under the objective ‘promoting the empowerment and participation of women in decision making about their own health care’ (30), while in the National Male Health Policy, health literacy is expressed as an overall focus of the policy, and described as a key action area under two priorities (31). Two other national policy documents described health literacy as a priority action area (32,33).
Health literacy actions described within public policies
A total of 16 documents specified health literacy actions. The most commonly noted actions related to health information provision and resource development, which appeared in 11 of the 16 documents. Five documents referred to school-based education programs, while seven documents referred to community-based education programs or activities. Two documents referred to social marketing campaigns, and one document specified the use of online technology and social media to engage the community and promote mental health. Other notable actions specified in the documents included; strategies to reduce access barriers, and engagement with workplaces and communities to improve mental health literacy and enhance resilience. One document provided a list of recommended actions organisations and policy makers could implement to address health literacy, including interpersonal communication strategies, training for the health workforce, and the development of legislation, policies and standards.
Intentions to resource health literacy activities and monitor policy outcomes
Of the 29 documents reviews, only the Veteran Mental Health Strategy (34) outlined an intention to resource the implementation of policy activities, including those that could be considered health literacy specific activities. The Roadmap for National Mental Health Reform (35) and the Fourth National Mental Health Plan (36) were the only two documents to specify an intention to monitor policy outcomes, which included a broad set of key performance indicators to monitor the progress of all governments, and to monitor mental health awareness and attitudes through a mental health literacy survey.
Discussion
We developed a framework to enable the systematic analysis of health literacy in public policy documents, and tested the utility of the framework by applying it to a set of policies relevant to the health and social services sectors in Victoria, Australia. We found the framework to be a useful tool for systematically sorting through detailed, complex and ‘messy’ policy information to reveal the strengths and weaknesses of policy documents and to make judgements about their potential to impact on health literacy outcomes by revealing the extent to which health literacy is prioritised, resourced, operationalised and monitored. We also demonstrated its utility in making comparisons between disparate documents, and rating them according to their key strengths and weaknesses.
We addressed our second objective by applying the framework to a defined set of policy documents to examine the health literacy policy environment in Victoria, Australia. This revealed a number of important insights. Firstly, we found that a surprisingly high number of documents mentioned health literacy, which may indicate there is an increased awareness of the concept and its potential to improve health outcomes. While some policies gave health literacy a high degree of prominence by including it as a strategic priority, very few of them clearly specified concrete actions to address health literacy. Further, the actions that were specified to address health literacy were limited in scope, focusing largely on health information and/or resource development (9,10,31). Some documents specified the use of education programs, information technology and social marketing campaigns to improve health literacy, but these were relatively few. The failure of these policies to articulate a broad set of actions to address health literacy is potentially indicative of the lack of relevant evidence available to guide the policy making process. The field of health literacy research is adept at defining and measuring the problem, which is evidenced by the large volume of literature on conceptual models (37–40) the development of measurement instruments (41) and the application of these instruments to assess the health literacy abilities of specific populations (42–46). In contrast, a lack of intervention research has been undertaken to date (47); therefore, there is a lack of intervention evidence to guide policy makers in their decisions about effective policy actions.
Of particular concern for the potential of current policies to improve health literacy is their failure to outline a commitment to funding health literacy activities and to monitoring and evaluating policy impact and outcomes. For policies to be effective in improving health literacy outcomes, sufficient resources will need to be allocated and should be specified in enough detail to ensure that governments are accountable for their investments.
Likewise, a clear commitment to, and plan for, monitoring and evaluating policy implementation and outcomes is needed. This is not only critical for ensuring government and stakeholder accountability, but also for measuring their impact on the population and contributing to the evidence base on effective interventions. The two policies that met the monitoring criteria in this review provide promising examples for ensuring accountability. The Fourth National Mental Health Plan, for example, incorporates accountability (measuring and reporting progress) as a key strategic priority of the policy, and specifies four key actions: (i) the establishment of comprehensive national reporting mechanisms on mental health reforms; (ii) monitoring the performance of the service delivery system through quality indicators; (iii) develop national mental health data collections; and (iv) conduct a rigorous evaluation of the plan. In addition, each priority area within the plan is accompanied by a set of specific indicators for monitoring change, including indicators and data collection instruments for measuring changes in mental health literacy. Together, these actions represent a comprehensive approach to monitoring the progress of policy implementation, measuring the performance of stakeholders, measuring the impact of policies on population health outcomes, and evaluating the outcomes of policies.
With the WHO having established a strong global mandate for governments to address health literacy through effective public policy (48), there is likely to be a proliferation in the number of health literacy policies in coming years. The framework we developed in this study, and the evidence presented here may be useful for informing the development and evaluation of current and future policies at local, regional and national levels.
Conclusion
We developed a framework that is likely to be useful for identifying the strengths and weaknesses of disparate policy documents and drawing comparisons between them, as well as making judgments about their potential to improve health literacy outcomes. By applying the framework in the Victorian context, we revealed that policy responses do not reflect best practice in policy development and implementation. That is, a relatively small number of policies describe specific actions to address health literacy, only one specifies funding to support the implementation of activities, and two seek to monitor policy outcomes. For future policies to be effective in addressing health literacy, they will need to be strengthened in these areas.
Footnotes
Conflict of interests
The authors declare that there is no conflict of interest.
Ethics approval
Ethics approval not required.
Funding
The Ophelia project is funded by a peer reviewed nationally competitive Australian Research Council (ARC) Linkage Project grant, with partnership funding from the Victorian Department of Health, Deakin University and Monash University. Anita Trezona’s PhD Scholarship was funded through the Ophelia Project. Richard Osborne was funded through a National Health and Medical Research Council (NHMRC) Senior Research Fellowship #APP1059122.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
