Abstract
Background: Heart failure (HF) knowledge as well as compliance are considered to be underlying mechanisms of the effects of HF management programs. However, there are no valid and reliable measurement instruments available which measures knowledge of HF patients.
Aim: To develop a reliable and valid instrument, which measures the knowledge, patients have on their disease and the HF-related health care regimen.
Methods: The HF knowledge scale was developed in 3 phases; (1) concept analysis and first construction, (2) revision of items and (3) testing for validity and reliability.
Results: The Dutch HF knowledge scale is a 15-item, self-administered questionnaire that covers items concerning HF knowledge in general, knowledge on HF treatment (including diet and fluid restriction) and HF symptoms and symptom recognition.
Face validity as well as content and construct validity was tested in HF patients in 19 hospitals in the Netherlands. The scale was able to differentiate between HF patients with high and low level of HF knowledge. Cronbach's α of the knowledge scale in this population (n = 902) was .62.
Conclusion: The instrument is a valid and reliable scale that can be used in research to gain insight in the effect of education and counselling of HF patients. After additional testing, the instrument seems to be a valid and reliable scale to be used in clinical practice to measure HF knowledge.
Introduction
Heart failure (HF) is a serious chronic condition, which is a substantial burden to patients and their families that contributes to the enormous costs associated with the care of HF patients. Although there are several studies that have demonstrated a positive effect of education and counselling on readmission and mortality of HF patients [1–4], little is known about the underlying mechanisms that explain the improvement of these outcomes [5]. Improvement of knowledge about HF self-care and compliance with the HF regimen is considered to be part of these underlying mechanisms [1,6–8]. To evaluate the effect of education and counselling, it is therefore important to assess the level of knowledge of HF patients. No valid and reliable instrument that measures knowledge of HF patients is available. Therefore, we developed an instrument to measure knowledge of patients about HF in general as well as specific knowledge about diet, fluid restriction and symptom recognition.
Background
The importance of knowledge
In order to maintain an optimal condition with a minimum of symptoms, it is important that HF patients comply with their therapeutic regimen. Non-compliance contributes to worsening HF symptoms as well as to hospitalisation [9–12]. A major factor related to compliance is knowledge of HF and the HF-related regimen [13]. Although knowledge alone does not guarantee compliance with the therapeutic regimen, patients can comply better when they have knowledge about their disease and the HF-related health care regimen. Therefore, knowledge can be seen as one of the underlying mechanisms of the effects of HF management programs on morbidity and mortality.
From recent HF guidelines, it can be concluded that HF patients need knowledge on HF in general, pharmacological and non-pharmacological treatment of HF, as well as symptom recognition and knowledge on what to do in case of worsening HF symptoms [14,15]. Written and verbal information has become an important component of HF management programs. The use of a computer-based program in the education of HF patients is rather new. In Sweden, a CD-ROM that is user-friendly was developed for elderly patients and is used in many HF clinics to give patients information on HF [16]. In a recent study of Stromberg, an increase of knowledge was reported after this computer-based education [17].
Studies that measured knowledge
At the time of the start of our study on the effect of education and counselling in HF patients (COACH) [5], there were no reliable and valid instruments available that measured knowledge of HF and the HF regimen.
Linne et al. [18] developed a knowledge questionnaire for HF patients, which consists of 23 items on HF medication in general with an emphasis on diuretics. The questions were based on an interactive CD-Portfolio program and correspond to information that was given to HF patients at the ward during hospitalisation. No information was reported on validity and reliability of the scale. Cline et al. [19] measured knowledge of HF patients by interviewing patients after education and counselling on medication. Individual knowledge scores were not quantified in this study. This method however, is very time-consuming and not suitable to use in quantitative studies. Because we had a wider interest in HF knowledge than knowledge on medication alone, the Dutch Heart Failure Knowledge Scale was developed. After the development of the scale, it was tested in HF patients from 19 hospitals in the Netherlands.
Methods
Development of the scale/testing of the scale
To investigate knowledge that is important for HF patients, the Dutch guidelines [14] on the management of HF were consulted. In those guidelines, it was stated that it is important for HF patients to have at least some knowledge on HF in general, the HF treatment and knowledge on symptoms and symptom recognition.
The knowledge scale that we developed consists of 15 multiple-choice items concerning HF in general (4 items), HF treatment (6 items on diet, fluid restriction and activity) and symptoms and symptom recognition (5 items). Questions in this scale were based on an existing knowledge scale [18], on the content of the CD-ROM program that is used in HF clinics in Sweden [16] and on a knowledge test of the Netherlands Heart Foundation [20]. The items were also derived from important self-care concepts covered by the European Heart Failure Self-care Behaviour Scale [21]. The Dutch Heart Failure Knowledge Scale is a self-administered questionnaire. For each item, patients can choose from three options, with one of the options being the correct answer. The scale has a minimum score of 0 (no knowledge) and a maximum score of 15 points (optimal knowledge). The translation of the scale is presented in Fig. 1.
Translated Dutch Heart Failure Knowledge Scale.
Reliability
Internal consistency of an instrument estimates the extent to which different parts of the instruments are equivalent in measuring the concept of interest. Internal consistency was assessed in a population of 902 HF patients from 17 Dutch hospitals (mean age 71) who were hospitalised for HF and all participated in the COACH study [5].
Cronbach's α in this population was .62. The internal consistency of the instrument could not be improved by deletion of any of the 15 items.
Spearman's correlation coefficient between each item and the overall score was calculated. The strongest association of an item and the total score was .51 both in item 14 (‘what to do in case of increase of weight’) and item 15 (‘what to do in case of thirst’). Spearman's correlation between 15 separate items showed that item 2 (‘why is weighing important’) and item 10 (‘why is a sodium restricted diet important’) showed the highest correlation (r = .24).
Validity
Content validity and face validity
After the first construction of the scale, a panel of 10 experienced HF nurses from the working group on HF of the Dutch Society of Cardiovascular Nursing and 2 cardiologists who were involved in the care of HF patients assessed content validity of the scale. They only made some remarks on the formulations and rated the scale as complete. They did not add or delete any item.
After that procedure, 6 patients who received comprehensive education and counselling on the HF clinic in the University Medical Center in Amsterdam assessed face validity. They all judged the items relevant, clear and easy to understand. They did not add any relevant items to the scale.
Construct validity
One approach to construct validation is the known-groups technique, which refers to the ability of an instrument to distinguish between 2 groups of patients who theoretically should have a different score on an instrument. In this study, we compared knowledge scores of 18 patients (mean age 74) referred to the HF nurse who were newly diagnosed with HF before they received education on HF with knowledge scores of 19 HF patients (mean age 67) after comprehensive education. All patients were recruited from the HF clinic in a general hospital in Leiderdorp (the Netherlands). The HF nurse selected the moment in which he judged patients as well educated. It can be expected that these two patient groups differ in HF knowledge. Newly diagnosed patients had a mean score of 6.8 on the scale; patients who received comprehensive education had a mean score of 13.4. This difference was statistically significant (t = − 7,14; p = .0001). The findings suggest that the scale discriminates between patients with and without comprehensive education on HF and the HF-related regimen.
Another approach to construct validation employs a statistical procedure known as factor analysis. Factor analysis is a method for identifying clusters of related factors. To evaluate the construct validity of the Dutch HF Knowledge Scale, factor analysis was conducted on the knowledge scales that were completed by 902 patients (mean age 71) who were hospitalised for HF in 17 Dutch hospitals.
The factor analysis was completed using principal components extraction with oblimin rotation. Theoretically, it was assumed that the scale consists of 3 dimensions (HF in general, life style recommendations and symptom recognition). Although not all of the items of the scale loaded on the factors that theoretically were assumed, there were indications that the instrument may consist of 3 different subscales. However, the factor loadings were not strong enough to justify a clustering of 3 subscales (Table 1). Therefore, the scale was used as a total scale and reliability analysis was performed only for the total scale.
Factor loadings of the Dutch HF knowledge scale (n = 902)
Factor loadings of the Dutch HF knowledge scale (n = 902)
Highest factor loading.
The sensitivity of an instrument affects how small a variation in an attribute (knowledge) can be detected and measured. The first step was to investigate if there was a difference in knowledge between newly diagnosed HF patients before and after they received comprehensive education.
In a small population of 5 patients (mean age 72), there was a significant difference between knowledge of newly diagnosed HF patients before they received comprehensive education on HF (8.2) and knowledge of the same patients after comprehensive education (13.0).
The second step was to assess sensitivity of the HF Knowledge Scale in a larger, more heterogeneous population. We therefore tested the scale in 168 consecutive patients who were hospitalised for symptomatic HF as determined by a cardiologist, suffered from an underlying heart disease and gave informed consent (mean age 69.5 year, 41% female). The study population consisted of both newly diagnosed and patients who suffered from HF for a longer period. All patients completed the HF Knowledge Scale before and after they received comprehensive education on HF and HF related subjects. The knowledge score at baseline (i.e. before education) was significantly lower (10.9 ± 2.3) than the score after education (12.4 ± 1.8) (Fig. 2).
Knowledge score before and after comprehensive education (N = 168).
Non-compliance with the therapeutic regimen is one of the underlying mechanisms that may explain the improvement of outcomes of HF management programs. Another important factor that is closely related to compliance and is also considered to be an underlying mechanism of success of a HF management intervention is HF knowledge. However, in order to accurately measure knowledge to shed light on it as an important factor, a valid and reliable instrument that measures knowledge is needed.
After the first testing reported here, the Dutch Heart Failure Knowledge Scale was found to be a valid and reliable instrument to measure knowledge of HF patients. The instrument has the ability to differentiate between patients with and without education and counselling. In a new instrument, however, it is always difficult to determine what difference is really needed to distinguish between clinical and statistical significance. Although the difference in knowledge score between patients with and without education was statistically significant in this study, we do not know at this moment whether patients with a higher score on knowledge, have better outcomes, for example higher compliance with the HF regimen. Prospective studies in which this knowledge scale is used are needed to find out if a statistical significant change in knowledge is related to better patient outcomes.
After additional testing, it is possible that the scale can be used in clinical practice to assess the level of knowledge HF patients. Together with the patient, a knowledge deficit can be assessed and discussed. Knowledge can be improved by education and counselling that is directed to those items on which a knowledge deficit exists.
Although the Cronbach's α of the scale was sufficient for a newly developed scale (.62), it is possible that adding more questions to the scale can improve the reliability. Another possibility to improve the reliability is to add 1 or 2 answers to every question.
The first testing however suggests that the Dutch Heart Failure Knowledge Scale can be successfully used in research settings to gain insight in the effect of education and counselling of HF patients.
Footnotes
Acknowledgements
We thank G. Cleuren and T. Veelenturf for providing their data for analysis. This study was supported by the Netherlands Heart Foundation (Grant 2000Z003).