Abstract
Purpose
To validate the content of the “Health Literacy Behaviour” nursing outcome (NO).
Methods
A content validation study was conducted during 2022. Each indicator was included in the NO, and its response levels were operationally defined. The initial version of each indicator and its response levels were refined and validated through the Delphi method. A panel of health literacy (HL) and nursing taxonomies experts evaluated the content through two content validity indicators. The content validity of the NO and each index was determined by calculating the content validity index (CVI). A qualitative analysis of the recommendations provided by the experts was carried out to improve the understanding of the indicators and their levels.
Findings
A total of 108 experts participated in this study. Mostly females with more than 10 years of professional experience. The results demonstrated a high CVI of the indicators and the NO “Health Literacy Behaviour.” All indicators achieved excellent (CVI ≥ 0.80) relevance and clarity. The CVI universal average method (CVI-p) of the NO achieved an excellent result of 0.90.
Conclusions
The indicators included in the NO “Health Literacy Behaviour” have content validity.
Implications for nursing practice
These findings provide evidence-based indicators to measure the patient's actions to obtain, process, and understand information about health and disease, interact with the health system, and make informed health decisions. The validation of this NO would identify populations with low HL, allowing the health of this community to be promoted. Health literacy should be a priority objective of health management and policies.
Objetivo
Validar el contenido del resultado de enfermería “Comportamiento de alfabetización en salud”.
Métodos
Se realizó un estudio de validación de contenido durante el año 2022. Se definió operativamente cada indicador incluido en el resultado de enfermería y sus niveles de respuesta. La versión inicial de cada indicador y sus niveles de respuesta fueron refinados y validados mediante el método Delphi. Un panel de expertos en alfabetización para la salud y taxonomías de enfermería evaluó el contenido a través de dos indicadores de validez de contenido. La validez de contenido del resultado de enfermería y de cada indicador se determinó mediante el cálculo del Índice de Validez de Contenido. Se realizó un análisis cualitativo de las recomendaciones brindadas por los expertos para mejorar la comprensión de los indicadores y sus niveles.
Resultados
En este estudio participaron un total de 108 expertos. En su mayoría mujeres con más de diez años de experiencia profesional. Los resultados demostraron un CVI alto de los indicadores y del resultado de enfermería “Comportamiento de alfabetización en salud”. Todos los indicadores lograron una relevancia y claridad excelentes (CVI ≥ 0.80). El Índice de Validez de Contenido Método Promedio Universal CVI-p del resultado de enfermería alcanzó un resultado excelente de 0.90.
Conclusión
Los indicadores incluidos en el resultado de enfermería “Comportamiento de alfabetización en salud” tienen validez de contenido.
Implicaciones para la práctica de enfermería
Estos hallazgos proporcionan indicadores basados en evidencia para medir las acciones del paciente para obtener, procesar y comprender información sobre la salud y la enfermedad, interactuar con el sistema de salud y tomar decisiones de salud informadas. La validación de este resultado de enfermería identificaría poblaciones con baja alfabetización en salud, permitiendo promover la salud de esta comunidad. La alfabetización para la salud debería ser un objetivo prioritario de la gestión y las políticas sanitarias.
Keywords
INTRODUCTION
Health literacy (HL) has been defined as “the knowledge, motivation, and competencies to access, understand, appraise, and apply information, to make judgments and decisions in terms of healthcare, disease prevention, and healthy behaviours, to maintain and promote quality of life throughout the life course” (Sørensen et al., 2012). Therefore, HL is a social determinant of health, which depends on individual abilities and skills, the cultural context, and the health system, and it is a step before patient empowerment (Nutbeam & Lloyd, 2021; Sørensen et al., 2021).
Numerous studies have shown how low HL levels are associated with low participation in health promotion and disease detection activities and inadequate management of chronic diseases (Cabellos-Garcia et al., 2018; Figueroa Saavedra et al., 2020), a misunderstanding of complex treatments and errors in taking medication (Cabellos-Garcia et al., 2018; McDonald & Shenkman, 2018), a higher rate of hospitalization, re-hospitalization, and misuse of emergency services (McDonald & Shenkman, 2018), scarce use of preventive services and, ultimately, an increase in morbidity and mortality (Bas-Sarmiento et al., 2015; Fernández-Gutiérrez et al., 2018; Figueroa Saavedra et al., 2020). Similarly, low HL represents an obstacle for patients with chronic diseases to learn details of their disease and treatment, making it difficult to control their signs and symptoms (Bas-Sarmiento et al., 2015). Therefore, it should be noted that HL is a crucial factor to consider in patient care because it affects the ability to make decisions and supports the self-management of health (Poza-Méndez et al., 2023).
Numerous studies show low levels of HL in the world (Duong et al., 2017; Prince et al., 2018), which have become even more evident during the pandemic. The lack of HL in the population in the face of COVID-19 was considered a public health problem worldwide, where a majority of the adult population confirmed having problems and not having enough skills to care for their health and that of others (Instituto de Salud Carlos III, 2022; Zarocostas, 2020).
BACKGROUND
One of the roles of nursing is to create a health-literate society that takes an active role in research, education, and health promotion (Cobo Sánchez, 2019; Correro-Bermejo et al., 2023). Strategies to educate the population should be incorporated into individual patient care plans to become part of the routine clinical practice of nurses (Cobo Sánchez, 2019).
In 2008, the intervention 5515 Health Literacy Enhancement was incorporated into the fifth edition of the Manual Nursing Interventions Classification (NIC), which proposes, among other activities, to determine the HL status of the patient through an evaluation (Bulechek et al., 2013). Years later, in 2016, the North American Nursing Diagnosis Association (NANDA) approved the diagnosis “00262 Readiness for Enhanced Health Literacy” (Herdman & Kamitsuru, 2018). Subsequently, in 2018, the nursing outcome (NO) Health Literacy Behavior (2015) (Moorhead et al., 2018) was incorporated into the Nursing Outcomes Classification (NOC). This NO is associated with 21 indicators that allow the evaluation and promotion of the patient's autonomy to make decisions about their health, involving them in activities that promote health and disease prevention, training them in self-care, and achieving therapeutic adherence (Correro-Bermejo et al., 2023). Its inclusion in the taxonomy and nursing care plans allows nurses to provide more individualized and efficient care. Nurses must have tools that facilitate the systematic, objective, and reliable evaluation of the HL level of the population. In the same way, the need to determine the importance of the scores of the indicators of an NO has been detected (Paloma-Castro et al., 2017). However, publications on NO remain scarce (Dias Emidio et al., 2020), especially on HL. Consequently, it is required to provide clinical evidence for effective nursing practice validation of the NO Health Literacy Behaviour and its indicators.
Content validity is defined as “logical judgment about the correspondence between the trait or the learning characteristic of the evaluated person and what is included in a test.” It aims to determine if the proposed items or questions reflect the content domain (knowledge, abilities, or skills) to be measured (Urrutia Egaña et al., 2014). In order to carry out more precise content validation, some authors agree that it is necessary to have an intentional sample of experts, know and analyze the characteristics of said sample, and be clear about the dimensions that are to be evaluated in the said instrument (Lynn, 1986; Urrutia Egaña et al., 2014). Additionally, studies suggest that the most appropriate way to collect data to support content validity is by conducting qualitative research (MacDermid, 2021).
This study aims to validate the content of the NO Health Literacy Behaviour (2015) for use in individualized care plans.
METHODS
Design
An exploratory and cross-sectional study was carried out in 2022 to validate the content of the NO “Health Literacy Behaviour (2015)” (Moorhead et al., 2018), belonging to the NOC taxonomy.
This study is based on a recently published protocol (Correro-Bermejo et al., 2023).
Sample/participants
A non-probabilistic convenience sampling was conducted among HL and nursing taxonomy experts to assess the face and content validity of the indicators (Polit & Tatano, 2008). The sampling strategy aimed to ensure that participants met the following inclusion criteria: (a) minimum clinical experience of at least 2 years; (b) a minimum of 5 years of experience in development and use in the area of nursing taxonomy; (c) to participate or have participated in research activities and have scientific academic productions in areas related to taxonomies, instrument validation, or HL; (d) academic experience in the field of at least 2 years. To participate as an expert, the expert must meet criterion (c), as well as one more (a, b, or d) (Seganfredo & Almeida, 2010).
There is still no consensus in the literature on the number of experts recommended to carry out the content validation of an NO (Oh & Moorhead, 2019). The optimal size for a Delphi group is estimated between a minimum of 6 and a maximum of 30 participants (Bellido-Vallejo et al., 2016; Landeta Rodríguez, 1999). In the present work, expert panels of more than 20 experts were formed.
The experts were recruited by email through professional nursing colleges, nursing associations and societies, and Spanish universities. They completed an anonymous survey through Google Forms.
Measurement and variables
The following grouped variables were considered: (a) sociodemographic variables—sex, age, position in the institution where they work, level of study, academic degree, whether they have a doctoral degree, and years of professional experience; (b) indicators of NO Health Literacy Behaviour (2015) (Moorhead et al., 2018); (c) an open-ended question—“Write a proposal for new indicators or changes/suggestions to indicators already proposed.” A 5-point Likert-type measurement scale was used to evaluate each indicator. Responses ranged from 1 (“no knowledge”) to 5 (“extensive knowledge”).
Indicators of NO Health Literacy Behaviour (2015) and each level of their 5-point Likert-type measuring scale were operationally defined.
Data collection
Data collection was developed from January to December 2022. A web-based platform to facilitate the participation of experts was deemed appropriate as it is cost-effective and efficient (Gill et al., 2013). The online survey consisted of three sections: The first section contained a description of the study, the informed consent, and the survey instructions, including a clear definition for each choice category of the ranking scale; the second section collected sociodemographic data; and the third section contained the indicators (operationally defined) included in the NOC outcome to assess the relevance and clarity of each indicator. Finally, within each indicator was a free-text section for experts to provide feedback, opinions, and comments.
Three phases of consensus were conducted using the Delphi technique (Bellido-Vallejo & Pancorbo-Hidalgo, 2017; Bellido-Vallejo et al., 2016). In the first and second rounds of consensus, the relevance of each indicator and its definition were measured. An indicator will be relevant if “it is essential or important, that is if it should be included.”
Following Escobar-Pérez and Cuervo-Martínez (2008), the relevance and clarity of each indicator and its definition were measured in the third round of consensus. An indicator will be clear if it is “easily understood, that is, its syntactic and semantics are adequate.” Both clarity and relevance were measured with a 4-point Likert scale, where 1 means “not clear/irrelevant”; 2 = unclear/low relevant; 3 = “fairly clear/fairly relevant”; 4 = totally clear/totally relevant.
Data analysis
A univariate descriptive analysis was performed to determine the sample distribution for each variable studied. The characterization variables were summarized using descriptive statistics, expressing qualitative variables in frequency and percentages and quantitative variables in mean and standard deviation (SD).
For the content validity of the indicators included in the NOC outcome “Health Literacy Behaviour (2015),” the approach advocated by Lynn (1986) was used. The content validity index (CVI) was calculated (Almanasreh et al., 2019) both at the individual indicator level (I-CVI) and the average of the content validation index of all the indicators.
I-CVI was computed as the number of experts rating 3 or 4 “relevance” for each indicator divided by the total number of experts. The CVI of all the indicators was calculated in two methods; one was the CVI universal agreement method (CVI-au), and the second was the CVI universal average method (CVI-p). CVI-au was calculated by adding all indicators that achieve a relevance rating of 3 or 4 by the experts divided by the total number of indicators, whereas CVI-p was calculated by taking the sum of the I-CVIs divided by the total number of indicators (Almanasreh et al., 2019). The clarity index was computed as the number of experts rating 3 or 4 “clarity” for each indicator divided by the total number of experts.
Taking into account the size of the expert panel and according to the bibliography consulted, the relevance/clarity of the indicators was considered good if the CVI were more significant than or equal to 0.80 (Almanasreh et al., 2019; Lynn, 1986). Those indicators that did not reach these scores were reviewed and reformulated based on the feedback collected until a final version was agreed upon.
Qualitative analysis used thematic analysis following the methods of Braun and Clarke (2006). The themes identified were grouped by indicator. Each recommendation was coded according to the expert number. Two nurses and a psychologist (all PhD) were responsible for the manual analysis using office software. Each made a detailed division of the content of the interviews to group them according to the indicators under study. For the semantic analysis of the content of the recommendations given by experts, the software Sphinx iQ3 v.8.2.2 was used.
A data matrix was created, and data were processed statistically using SPSS, version 22 (IBM).
Ethical consideration
This research was carried out following the principles established in the current revised version of the Declaration of Helsinki and following the applicable legal requirements for biomedical research, Law 14/2007 on Biomedical Research. The Cadiz Ethics Committee approved this research with register number 56.23 (code: PEIBA 0686-N-23). Likewise, informed consent was included in the first section of the online survey. In legal terms, Spanish Organic Law 5/2018 on Personal Data Protection guaranteed the anonymity of the participants and the database, with no personal data that can be used to identify them.
RESULTS
Panel of experts
Table 1 shows the sociodemographic characteristics of the sample for each consensus round.
Sample sociodemographic profile.
In the first round, the sample consisted of 51 experts, with a mean age of 37 years (SD = 12.065). The total number of women was 80.4%, mainly with undergraduate and graduate studies. Of the sample, 37.3% had a teaching/research profile, and 51% worked in the care setting. A large majority (82.4%) had experience in the last 5 years of the development/use of nursing taxonomies.
In the second consensus round, a sample of 27 experts with a mean age of 39 (SD = 12.32) was obtained. Overall, 74.1% were women, 51.9% had doctoral studies, and most were nurses. In this case, 48.2% had a teaching and research profile. Of the sample, 33.3% had professional experience of more than 20 years.
Finally, in the third round of consensus, the sample consisted of 30 experts with a mean age of 46 (SD = 12.258). Overall, 76.7% were women, the majority being nurses, and 43.3% with doctoral studies. It should be noted that 33.3% had a teaching and research profile, and 60% had professional experience of more than 20 years.
Content validity result
Initially, each indicator and its response levels were defined.
Table 2 shows the I-CVI for each indicator. The expert panel evaluated the 21 indicators that showed an acceptable I-CVI regarding relevance and clarity. I-CVI values calculated for the 21 indicators ranged from 0.74 to 0.96.
Agreement rate of expert penalties and content validity index.
Content validity index (CVI) = (the number of experts rating an item ≥3)/(the total number of experts), CVI ≥ 0.80 is acceptable.
CVI < 0.80 is not acceptable.
In the first round, the indicator number 11 obtained a CVI of 0.76; in the second round, the indicator number 10 obtained a CVI of 0.74. In both cases, as the ICV result was close to 0.8 and no expert requested the removal of these indicators, it was decided to reformulate based on the experts’ recommendations.
Finally, the results of the CVIs for relevance and clarity indicated that 100% of the respondents agreed with the content improvements made in the third round.
Table 3 shows the CVI-au and the CVI-p. The CVI-au provides the degree of agreement among the experts to classify the indicators as “fairly and fully appropriate.” The CVI-p indicates the average content validation index of all indicators.
Agreement rate of expert penalties and content validity index.
Note: Content validity index universal agreement method (CVI-au); content validity index universal average method (CVI-p).
Qualitative analysis
In the first round, a qualitative analysis of the recommendations suggested by the experts to improve the definition of the indicators and their levels was carried out. Supplementary Material 1 shows the changes made in rounds 1 and 2.
Following the qualitative analysis of the recommendations made by the experts, the definitions of indicators 4, 6, 8–10, 15, 17, and 18, and the levels of indicators 1, 3–18, and 20 were modified. For example, the definition and levels of indicator 13 were modified based on the following recommendations: Expert-1: I would put health care providers instead of service providers (sounds more like material supplies). Expert-2: There is an indicator that says analyse, I would put the verb tense the same as the others. Instead of health care providers, I would put health care professionals directly. Expert-3: Regarding the indicator “Identifies health service providers,” it is difficult to think, even in the most marginal communities, that they do not know how to identify health professionals, so levels 1 and 2 could be put together in a single level 1, displacing the rest of the levels and adding a 5th level aimed at knowing if they can distinguish between specialists within a health profession.
For indicator 11, the definition and levels were modified due to the low I-CVI obtained, and taking into consideration the comment of one of the experts: “In 5th level, include that it is capable of raising doubts or clarifying questions (it is indicative that we are not just signing for the sake of signing)”.
A second round was passed to the same experts invited in the first round to verify that the changes introduced were appropriate. Following this second consultation, minor modifications were made to the definitions of indicators 9, 10, and 19. The definition of indicators 9 and 10 included the link to the laws listing patients’ rights and duties. In the case of indicator 19, an example was added.
In addition, at the request of several experts, minor changes were made to the levels of indicators 4–6, 8, 12, 13, 15–19, and 21 to clarify and distinguish the different levels.
DISCUSSION
Tools must be available to facilitate the systematic, objective, and reliable assessment of this NO by professionals, to carry out the much-needed individualized nursing interventions to achieve the NO, and subsequently allow the results to be evaluated. In this scenario, the NO, Health Literacy Behaviour, can guide nurses and researchers on which aspects of HL management are essential to assess. This content validation process allows the improvement of the nursing language and the execution of the planning and implementation of interventions in a guided way to identify which vulnerable populations need special care. In this way, it ensures the execution of appropriate actions for the individual. Therefore, this study aimed to validate the content of the Health Literacy Behaviour NO for use in individualized care plans.
Several studies identify the lack of a nursing gold standard to help us determine the effectiveness of nursing interventions (Morilla-Herrera et al., 2011; Silva et al., 2015). In the case of HL, numerous validated tools are used to assess the HL level of the population, such as the Health Literacy Survey European Questionnaire 16 (HLS-EU-Q16) (Pelikan et al., 2014). Since 2018, the NOC Health Literacy Behaviour (2015) has opened the possibility for nursing to assess the HL of their patients and incorporate it into care plans to measure the effectiveness of their interventions. However, validation of its content is essential for it to be effective (da Costa Ferreira et al., 2021). In the present work, as in other studies (Oh et al., 2023; Silva et al., 2015), the validation has given rise to a NO with some indicators more valid and reliable.
Several studies note that one of the essential characteristics of content validation is that the indicators are defined to avoid controversy when interpreting them (da Costa-Ferreira et al., 2021; Mokkink et al., 2016). There is research showing how having operationalized indicators increases agreement among nurses (Chantal Magalhães da Silva et al., 2017). Furthermore, Martín-Dorta et al. (2021) suggested that operational definitions are essential for studies that address nursing taxonomies, as these provide us with basic information that describes what is to be measured and how it can be done. For this reason, the definitions of the NO Health Literacy Behaviour (2015) indicators are elaborated based on scientific evidence in the present work. Subsequently, modifications are made based on experts’ recommendations, as in the case of Bellido-Vallejo and Pancorbo-Hidalgo (2017).
For content validation studies to be reliable, the group of experts must be specialized in the topic to be dealt with, in this case, experts in nursing taxonomies and HL. Following the evidence recommendations (Bellido-Vallejo et al., 2016), we tended to maximize the sample of experts. In the first round, we obtained a sample of 51 experts; in the second, 27 experts; and in the third, 30 experts.
Although there are various methods to perform content validation, such as the Ferhing method (da Costa-Ferreira et al., 2021; Oh & Moorhead, 2019) or the CVI (Martín-Dorta et al., 2021), one of the advantages of using the latter is that it is easy to calculate. In addition, it allows for determining the relevance (Devriendt et al., 2012) and clarity of each of the measured NO indicators, thus determining whether the indicator needs to be eliminated or reformulated. However, as indicated by MacDermid (2021), it should be noted that the quantitative information obtained in content validation must be completed with the contribution of qualitative analysis. This technique allows an in-depth exploration of the evaluated indicators (Morgan, 1996). Thus, in this study, we start with the opinions and feedback of the experts to carry out the indicators’ reformulations, definitions, and levels.
Limitations
The NO and their indicators have been validated in Spanish. The results and recommendations of this study cannot be generalized to the NOC outcome in other languages and other settings, although the NOC has been translated into 11 different languages at this time. In order to obtain credible evidence of internationally validated NOC outcomes, further validation studies based on specific cultures are recommended. The next step is to use the validated NO in clinical settings and then evaluate the effects of using the NOC outcome.
Implications for nursing practice
Validation of this instrument will provide evidence-based indicators to measure how patients obtain, process, and understand information about health and illness, how they interact with the health system, and how they make informed decisions about their health. This will allow a deeper understanding of patients’ needs and facilitate identifying populations with low HL. By identifying these communities, nurses can direct specific efforts to promote health more effectively, thus making HL a priority objective in health management and policies.
Once validated, this instrument can be implemented by nurses in various areas of health care, including chronic patient consultations and other health units. Its application will facilitate the early identification of patients with a low level of HL, which will allow the implementation of appropriate measures to improve it. This will improve patients’ quality of life and self-care and guide necessary interventions to address their needs more effectively.
The integration of this instrument into clinical practice will directly influence nursing interventions. By providing a more complete understanding of patients’ health needs and capabilities, nurses will be able to offer a more personalized and effective approach to their care, thereby optimizing health outcomes. This means that the use of this instrument will not only improve the effectiveness of nursing interventions but will also contribute to more patient-centered care and, ultimately, to an improvement in the overall quality of healthcare. In fact, this study has contributed to filling a gap in the nursing care plan (a specific nursing intervention—NIC—requires results nursing—NOC—to determine the patient's AS status through an evaluation).
CONCLUSIONS
This study provides evidence of the content validity of 21 indicators of the Spanish version of the NO, Health Literacy Behaviour (2015), included in the sixth NOC. The 21 indicators and each level of their 5-point Likert-type measuring scale were operationally defined. Two indicators were reformulated throughout the validation process based on expert opinions.
The validation of this NOC outcome would facilitate nurses in establishing individualized and efficient care interventions and identifying populations with low HL.
The results support the need for studies assessing the accuracy of NO indicators to guide nurses’ decisions in selecting indicators sensitive to nursing interventions.
AUTHOR CONTRIBUTIONS
CONFLICT OF INTEREST STATEMENT
The authors declare no conflicts of interest.
FUNDING INFORMATION
The authors declare no funding sources.
ETHICS STATEMENT
The Cadiz Ethics Committee approved this research with registre number 56.23 (code:PEIBA 0686-N-23).
Footnotes
ACKNOWLEDGMENTS
The authors would like to thank all the professionals who participated in the expert panel. They would also like to thank the University Research Institute for Sustainable Social Development (INDESS) for supporting the translation of the manuscript.
