Abstract
Background: A RCT, conducted to examine the effectiveness of a structured collaboration in general practice to provide intensified preventive care in patients at high cardiovascular risk yielded no effect in the total group but differences across healthcare centres and ethnic groups become apparent. We conducted a process evaluation to explain these differences.
Methods: We assessed the reach of the target group and whether key intervention components (individual educational sessions, structured team meetings, and risk assessments) were performed as planned (maximum score for protocol completion is 11).
Results: The reach was initially 91%, but only a minority of patients completed the intervention activities as planned. The average score of the number of intervention components was low (5.66 out of 11 (sd 2.8)) and varied between centres (4.84 to 7.40) and ethnic groups (4.89 to 7.38), with team meetings as the least implemented activity conform plan.
Conclusion: This study indicates that adding a practice nurse and a peer health educator to the general practice did not seem to result in the desired collaboration between the healthcare personnel. Further research is needed to investigate the reasons behind the low participation rate of the patients in the intervention.
Keywords
Introduction
Preventive activities aimed at reducing cardiovascular disease (CVD) in general practice may be hampered by factors at the organisation level, such as lack of supporting staff, low guideline compliance and bad team climate [1–4], as well as at the patient level by, for example, low educational attainment [5].
In the Netherlands, supporting staff (such as practice nurses) have been introduced in general practice to improve the quality of care and reduce workload [6] and peer health educators to help bridge language and cultural differences in general practices with a multi-ethnic patient population [7].
As far as we know, no information is available regarding the effect of the structured collaboration of these supporting staff with general practitioners (GPs) on preventive activities [6]. We conducted an innovative intervention to reduce cardiovascular risk in three large primary healthcare centres located in deprived neighbourhoods of Rotterdam and The Hague involving a multi-ethnic patient population at high risk of developing CVD. This randomised controlled trial (Quattro-study) showed no effect of intensified preventive care provided by a structured collaboration of the practice nurse, peer health educator, GP and GP assistant on the reduction of absolute risk, in addition to usual GP care. Nevertheless, some indications were found for differences in the rate of risk reduction achieved after one-year follow-up between the participating healthcare centres and between ethnic groups [8].
To explain the lack of effect of the intervention and to provide more insight in the differences between centres and ethnic groups, we conducted a process evaluation. A process evaluation is considered essential to make programme effectiveness results more interpretable and to avoid type III error, i.e. evaluating an intervention programme that has not been adequately implemented and thus drawing incorrect conclusions about the effectiveness of a given intervention [9,10].
The following questions were posed:
Was the target group reached by the intervention activities? Was the intervention carried out as planned (i.e. fidelity to intervention programme)?
Methods
Participants
The Quattro-study was carried out in three primary healthcare centres representing 5 general practices (18 GPs) located in deprived neighbourhoods of Rotterdam and The Hague [12]. Inclusion criteria were: a fully computerised information system, capacity to appoint a peer health educator and to be able to fulfill national criteria to get a practice nurse financed: a collaboration of at least 3 general practices with a total patient population of over 7050 patients [6].
The study population consisted of patients at high risk of developing CVD from different ethnic origins, aged 30–70 years, and living in a deprived neighbourhood. A total of 275 high-risk individuals, defined as having a modifiable part of the 10-year absolute risk ≥ 5% (due to smoking and elevated levels of blood pressure and cholesterol) [11], were randomly assigned to either the intervention or the control group. The main outcome measure to determine the effectiveness of the intervention was the change in the 10-year absolute risk to develop CVD. We performed selection, enrolment and randomisation of patients. Details of the study design, inclusion and randomisation of patients are reported elsewhere [8]. For the present process evaluation we use data from the patients assigned to the intervention group only (n = 137).
Intervention
The intervention as planned consisted of adding a practice nurse and a peer health educator to the general practices to implement prevention activities in collaboration with the GP and the GP assistant. In each general practice a team (the so-called Quattro-team) is formed that assigned different tasks to the aforementioned four healthcare workers based on the needs of the individual patient (tailor made).
In order to re-set treatment goals as required, each patient is evaluated every 3 months during a team meeting. The team consisted of the GP, a practice nurse, a GP assistant and a peer health educator, in which the practice nurse had the coordinating role. All team members including the GP and GP assistant were invited to join a one-day course before the start of the intervention. The course gave information on how to deal with the intervention protocol, and the team members were trained in performing the structured team meetings.
The intervention protocol was based on the current Dutch College of General Practice Guidelines for hypertension, hypercholesterolemia, diabetes and smoking and described the procedures for the GP (first responsible and treatment decisions), practice nurse (risk assessment, health education, coordination and informative task), GP assistant (logistical task including measurements, patient recall and follow-up) and peer health educator (ethnic-specific health education).
Structural intervention components were individual face-to-face sessions (including an intake session) with patients performed at the general practice, team meetings (including an intake team meeting) by the Quattro team, and risk assessments every three months (consisting of physical measurements conducted at the general practice and clinical measurements at the laboratory) (see box 1). An average of three risk assessments followed by team meetings and individual educational sessions were considered sufficient during the 9-month follow-up period [13]. The team meetings took also place in the general practice, and to efficiently organise them the different patients were discussed during one session.
The intervention as planned consisted of the following components
The intervention as planned consisted of the following components
Active enrolment of patients: the practice nurse or peer health educator (as soon as they received the names of the patients allocated to intervention) telephoned or mailed the patients to visit the general practice in order to start follow-up. An intake session carried out by the practice nurse and/or health educator to evaluate the patient's health problems, specifically the cardiovascular risk profile, and the likelihood to comply with health education and treatment. Organisation of a structured intake team meeting attended by all team members to discuss prevention strategies according to the information collected during the intake session, and to agree about which team members would be involved in implementing the preventive tasks. This meeting resulted in a “treatment plan” tailored to the patient's risk factors and was re-evaluated every three months. Three risk assessments at 3, 6 and 9 months follow-up consisting of blood pressure and weight/height measurement, and measurement of lipids, glucose and HbAc1. Three follow-up structured team meetings of the Quattro members (led by the practice nurse) to discuss and evaluate the achieved results and bottlenecks encountered with regard to patient education, treatment and compliance. Three individual follow-up educational sessions, based on the “treatment plan” and conducted by the practice nurse and/or health educator followed each of the structured team meetings. The results of the risk assessments were discussed with the patient and feedback was given.
In addition to the structural components of the intervention, the intervention protocol allowed adaptation in order to tailor the intervention to the individual practice needs and organisation; for example, whether the GP assistant or the practice nurse should perform the logistical tasks concerning the intervention patients, and whether the practice nurse, peer health educator or both should perform the intake and educational sessions with ethnic minority patients. Further, the intervention team could be extended to allow experts from other disciplines (e.g. a dietician or a diabetes nurse) to participate in the structured team meetings. This tailoring of the individual practices was allowed in order to increase success, bearing in mind the diversity between the general practices.
After randomisation of the included patients by the research team, the patients' names were mailed to the practice nurses. Similarly, for each patient a document containing relevant characteristics, cardiovascular risk and risk factors based on baseline measurements was sent to the practice nurse and served as a registration tool for the intervention activities. Subsequently, the practice nurse invited the patients to the general practice to start intervention activities.
In two out of three healthcare centres, almost all practice nurses involved in the intervention were former GP assistants from the same general practices. In the third centre, a practice nurse (with an extensive experience as a nurse) was provided by the research team. The health educators (1 to 2 per healthcare centre) were trained medical educators fluent in the Turkish and Dutch language or Moroccan and Dutch language respectively, and regarded to be representative of the majority of the target population (i.e. ethnic minorities). These migrant health educators received relevant information about cardiovascular risk and risk factors during a two-day course before the start of the intervention.
During the intervention, 4 meetings were organised by the intervention supervisor (PU) for the practice nurses and the peer health educators to discuss the implementation of the intervention. Twice a year, all staff members of the participating general practices received a newsletter about the progress of the study and the intervention.
Reach of the target group
The reach is the degree to which the intervention contacted or was received by the targeted group participating in the intervention programme [10].
To determine which patients were reached, we computed the proportion of participants who received an intake session and, of these, the proportion of participants discussed during the intake team meeting. We calculated the means (and standard deviations; sd) of follow-up intake sessions, follow-up team meetings, and risk assessments received. To determine any differences between patient subgroups, we subdivided various proportions and means to gender, ethnicity, age, educational level, and baseline modifiable risk in relation to reach.
We also registered the dropout from the intervention programme defined as the number (proportion) of patients initially allocated to the intervention but who did not complete the programme (i.e. did not attend the final individual session at 9-months follow-up).
Programme fidelity
We assessed the fidelity, defined as the extent to which the intervention programme was delivered according to key components of the intervention [10], by using the following indicators (see box 1).
Enrolment period of patients to the healthcare centres: this was defined as the mean time (in weeks) between patient's allocation to the intervention group by the researchers until the date when the intake session took place. An average enrolment period of four weeks was considered as appropriate. Numbers and proportions of different activities from the intervention protocol carried out: individual intake session, intake team meeting, follow-up individual sessions, follow-up team meetings, and the three-monthly risk assessments. Involvement of the Quattro team members with the intervention activities. Participation of the peer health educator in the structured meetings was not “imposed” by the intervention protocol because they were part-timers and employed by different general practices. However, to bridge cultural and language differences when carrying out the individual sessions with Turkish and Moroccan participants, involvement of a peer health educator was essential because a large majority of these patients were first-generation immigrants whose knowledge of the Dutch language was very limited.
In addition, we created an index of protocol completion to summarise whether participants completed the intervention activities (intake session, intake team meeting, 3 individual follow-up sessions, 3 team meetings, and 3 risk assessments) as planned. For each activity, we scored a “1” if completed and a “0” if not completed. The scores for each participant were summed to determine the index of protocol completion ranging from 0 to 11. A score of 11 means that a patient participated in all activities and a lower score means that one or more activities were missed [14].
To determine the number of individual sessions and team meetings, as well as team members who participated in these activities, we used data registered continuously in the patient documents by the practice nurse and/or the peer health educator. The organisation of the three-monthly risk assessments and data collection of the results was carried out by the researchers throughout the study.
Data analysis
We computed descriptive statistics to assess the reach of the target group and the programme fidelity. To determine whether reach differs according to gender, ethnicity, age, educational level or modifiable risk, Chi-square tests were performed for nominal and categorical variables and analyses of variance for continuous variables. Differences in protocol completion between centres and ethnic groups were assessed using analyses of variance. Data for Turkish and Moroccan patients were grouped together because both obtained educational sessions from a peer health educator.
Results
The majority of the patients that entered the study was male (61%); 47% was Dutch, 23% Turkish; 10% Surinamese; 7% Moroccan and the remaining 13% was from different ethnic backgrounds. The mean age of the study populations was 55.8 (sd 11.6) years. A majority of individuals had no or low educational level (59%) and 34% was employed. Only 20% of the Turkish and Moroccan participants reported to speak and understand the Dutch language well.
Reach of the target group
Twelve patients failed to attend any intake sessions (not reached after repeated efforts; refused follow-up due to their dissatisfaction with the organisation of risk assessments and GP care) and hence did not start the program. Of the 125 patients who started the programme, 63 participants did not attend the final (third) individual session; the corresponding dropout rate is 55%. Finally, only 16 participants completed the intervention as planned: i.e. attended the intake session, all three individual sessions and three risk assessments.
Table 1 shows some general trends in participation in the different intervention components of patient groups. Men participated less in intervention components than women.
Reach of the intervention programme in relation to patient's characteristics (values are mean (sd) unless stated otherwise)
Reach of the intervention programme in relation to patient's characteristics (values are mean (sd) unless stated otherwise)
Patients who did not attend the individual intake session (n = 12) were excluded from these analyses.
Surinamese had the most favourable scores for most intervention components. Patients younger than 45 years less often participated in the individual intake activities. Patients with the highest level of modifiable risk (≥ 15%) participated more often and were more frequently discussed during the intake team meeting than patients with a lower modifiable risk.
Table 2 shows to what extent major features of the intervention were carried out in the healthcare centres.
Fidelity to major components of the intervention programme (in total and by healthcare centre) (values are n (%) unless stated otherwise)
Fidelity to major components of the intervention programme (in total and by healthcare centre) (values are n (%) unless stated otherwise)
Based on all patients who attended the intake session (n = 125).
The enrolment period of patients between randomisation and first visit to the healthcare centre was on average 12.8 (sd 11.3) weeks, with the longest period achieved in the first centre (18.2 (sd 10.6) weeks). Analyses according to ethnicity did not show any important differences in enrolment periods between the ethnic groups (data not shown).
Two third of the patients with an intake (66%) were discussed during the first structured team meeting, 46% completed 3 individual follow-up sessions and 26% completed all 3 monthly risk assessments. Only 11% were discussed during all the three follow-up team meetings indicating that organizing the team meetings was the most difficult activity to implement in the healthcare centres.
About half of the intake sessions (54%) were conducted by the practice nurse and 14% by both the practice nurse and health educator, but the proportions differed by healthcare centre: in one centre the practice nurse and the health educator carried out more than half of the intakes (53%) while this proportion was equal to or less than 5% in the other two centres. A large proportion of the intake sessions with Turkish and Moroccan patients were conducted by the health educator alone and incidentally in the presence of the practice nurse (data not shown). Almost all intake team meetings (93%) that took place were performed by the GP and practice nurse with no important differences between the centres. Although the peer health educator was involved in 38% of the intake team meetings, the extent of participation differed greatly per centre (14% to 82%). The GP assistant did not attend any structured team meeting.
Table 3 shows that the average scores of the number of intervention components were relatively low (5.66 out of 11(sd 2.8)) but most favourable in the third healthcare centre (7.40 out of 11 (sd 2.8)). With regard to ethnicity, Turkish and Moroccan participants had the lowest (4.68 out of 11 (sd 3.4)) and Surinamese the highest score regarding protocol completion (7.38 out of 11 (sd 2.2)). In the first centre, Turkish and Moroccan individuals completed only 2.4 (sd 1.7) of 11 intervention components.
Index for protocol completion of key components from intervention by healthcare centre and ethnic group (values are means (sd)) (maximum score is 11)
No standard deviation could be calculated because n = 1.
The present study shows that the target group was reached in the first instance (91% had an intake session) but subsequently only a minority completed the intervention programme. Almost none of the patients completed all key components of the intervention protocol as planned, implying that the fidelity to the intervention programme was low.
Although our study initially succeeded in organising a hard-to-reach group (i.e. ethnic minorities and individuals with a low socio-economic status) to start the intervention programme, during the follow-up period the reach of the various subgroups of patients was not uniform. We have no robust explanation for this low participation rate. Based on previous research this limited participation might be attributed to both organisational aspects of the intervention as well as to patient characteristics. Nevertheless, the role of patient characteristics should be emphasised in this since the limited reach was in line with previous intervention studies performed in low-income populations and some ethnic groups. In a study by Havas et al. [15] among low-income women, participants faced many obstacles in order to attend educational sessions (such as work, transportation, and stress of inner-city life) and in a study by Uitewaal et al. [7] going abroad for a longer period was the main reason for non-attendance in an intervention programme tailored to Turkish individuals. The characteristics that affected participation in the intervention activities in our study (i.e. male sex and younger age) are consistent with another study on a cancer prevention programme involving a multi-ethnic patient population which showed lower participation rates in younger age, men, non-whites and patients with a lower acculturation and a low income status [14].
The low participation observed in the present study is particularly worrying, because the prevalence of modifiable cardiovascular risk factors among young (male) people living in deprived neighbourhoods is high and should be targeted by intervention strategies. Further research is needed to investigate the reasons behind this low participation rate and whether tailoring interventions to this dropout group is beneficial.
Our findings showed that the fidelity to key components of the intervention was low, particularly the structured team meetings. These results are in line with previous studies reporting on the complexity of putting interventions into practice [16]. According to Grol and Grimshaw [16], change is rarely as easy if an innovation requires complex changes in clinical practice, or better collaboration between disciplines, or changes in the organisation of care. The low implementation of structured team meetings is not surprising because this key intervention component was by far the most complex one, requiring the collaboration of different healthcare professionals, clear task division and defined roles [17]. In our study, practice nurses reported they encountered many bottlenecks when organising these meetings, and GPs considered these meetings as time-consuming activities. Similarly, the role of the health educator was marginal and the participation of GP assistants was (almost) completely absent, which may be an indication of the lack of clear responsibilities within the intervention teams. The latter could be due to the overlap between the tasks of the practice nurse and those of the GP assistant, since most practice nurses involved in this study were former GP assistants working within the same general practices. Consequently the role of the GP assistants was perhaps too small and easy to brush aside, whereas the workload of the practice nurses probably increased due to their attempts to take over the tasks of the GP assistant. This could explain why the intervention (logistically) was not carried out in a satisfactory way given, for example, the long enrolment period of the patients. Our findings suggest that the desired collaboration with regard to the management of CVD did not take place in daily practice, but rather unclear division of some roles.
The variation in programme fidelity between the healthcare centres is not straightforward. We hypothesize that the organizational level of the general practices, the culture with regard to teamwork, and the practice nurse's skills and expertise are mainly responsible for the large differences observed between the centres [17–19].
With regard to ethnic differences in the programme fidelity our results show that Turkish and Moroccan individuals had the lowest scores for protocol completion. The most logical explanation is the marginal role of the health educator in this intervention programme compared to other intervention studies [20]. The limited command of the Dutch language of this subgroup, the absence of language support, and the practice nurse's lack of knowledge concerning these migrants' cultural backgrounds may have negatively influenced participation in the individual educational sessions [21].
A major limitation of the present study is that the results were mainly based on data collected using a patient document, which could be biased by differences in registration between the practices nurses. It is possible that some intervention activities (such as the individual sessions or team meetings) were not always filled in this document, or perhaps only in the GP's electronic records. However, when considering only the performance of the three months risk assessments (which were continuously registered by the research team), the same trends emerge as for the individual sessions and team meetings, suggesting that differences in the extent of implementation of intervention activities were not likely to be influenced by differences in data registration.
Although the numbers of the different ethnic groups per centre were limited, the trends in all three centres were similar (e.g. for Dutch and Turkish/Moroccan individuals). This subgroup analysis may help other investigators when planning intervention studies in targeted populations.
Conclusion and practice implications
The data reported in this study provide healthcare professionals, policy makers and investigators with a description of how an intervention to reduce cardiovascular risk in a multi-ethnic patient population (based on a pragmatic trial in general practices located in deprived neighbourhoods) is implemented in daily practice. The healthcare centres adapted the intervention to their own conditions by adopting some intervention activities (e.g. use of structured risk assessments and individual sessions) and leaving out other ones (e.g. team meetings). Consequently, an explanation for the differences in results between healthcare centres is difficult because the centres did not carry out the intervention activities in exactly the same way. The present study underlines the contribution of a process evaluation to our understanding of the lack of effectiveness of the intervention programme.
Our results imply that general practices in deprived neighbourhoods in the Netherlands are a suitable setting for initially reaching high-risk patients from different ethnic origins. Nevertheless, there is much to gain in terms of reducing the dropout. We suggest that practice nurses and peer health educators should be involved in active enrolment and the close follow-up of high-risk patients to ensure early detection of adverse effects and problems with compliance.
Because different disciplines have been added to the general practice, more emphasis should be put on collaboration between these professionals to improve the quality of care. To achieve structured collaboration among the different team members to treat patients at high cardiovascular risk, adequate composition of the team, clear division of tasks and well-defined roles are a prerequisite. Appropriate training regarding teamwork for the management of cardiovascular diseases in high-risk patients should be part of the curriculum for different healthcare professionals, including general practitioners.
Footnotes
Acknowledgements
The authors thank all the patients and staff of the healthcare centres (general practitioners, practices nurses, peer health educators and GP assistants) who took part in the study. This study was financed by a grant from the Netherlands Organisation for Health Research and Development (ZON-MW).