Intervention Fidelity to VITAL Start (Video Intervention to Inspire Treatment Adherence for Life) in a Randomized Controlled Trial Among Women Living With HIV in Malawi

Abstract

Background.

Intervention effectiveness in a randomized controlled trial is attributed to intervention fidelity. Measuring fidelity has increasing significance to intervention research and validity. The purpose of this article is to describe a systematic assessment of intervention fidelity for VITAL Start (Video intervention to Inspire Treatment Adherence for Life)—a 27-minute video-based intervention designed to improve antiretroviral therapy adherence among pregnant and breastfeeding women.

Method.

Research Assistants (RAs) delivered VITAL Start to participants after enrolment. The VITAL Start intervention had three components: a pre-video orientation, video viewing, and post-video counseling. Fidelity assessments using checklists comprised self (RA assessment) and observer (Research Officers, also known as ROs) assessment. Four fidelity domains (adherence, dose, quality of delivery, and participant responsiveness) were evaluated. Score scale ranges were 0 to 29 adherence, 0 to 3 dose, 0 to 48 quality of delivery and 0 to 8 participant responsiveness. Fidelity scores were calculated. Descriptive statistics summarizing the scores were performed.

Results.

In total, eight RAs delivered 379 VITAL Start sessions to 379 participants. Four ROs observed and assessed 43 (11%) intervention sessions. The mean scores were 28 (SD = 1.3) for adherence, 3 (SD = 0) for dose, 40 (SD = 8.6) for quality of delivery, and 10.4 (SD = 1.3) for participant responsiveness.

Conclusion.

Overall, the RAs successfully delivered the VITAL Start intervention with high fidelity. Intervention fidelity monitoring should be an important element of randomized control trial design of specific interventions to ensure having reliable study results.

Keywords

intervention fidelity randomized controlled trial HIV adherence VITAL start quality of delivery

Background

Randomized controlled trials (RCTs) of health care interventions are increasingly employed as the preferred gold standard for evaluating the effect of an intervention (Hariton & Locascio, 2018). Using an RCT design provides highly controlled conditions to evaluate an intervention’s efficacy; however, the context of delivering the intervention and the influence exerted by implementers on participants can lead to biased intervention effects and inaccurate study findings (Altman et al., 2001; Bonell et al., 2012). For efficacy to be correctly attributed to the intervention being tested, it is important that intervention fidelity is maintained (Feely et al., 2018; French et al., 2015; Sweeney-Magee et al., 2016). Intervention fidelity is essential to support the validity of the study design, implementation, interpretation of the results, and translation of the intervention into everyday practice settings.

The importance of fidelity in RCTs has long been recognized. Intervention fidelity is central for accurately interpreting findings of both efficacy and effectiveness trials by understanding and measuring how an intervention has been implemented (Chang et al., 2019; Toomey et al., 2020). Fidelity assessment allows researchers to demonstrate with greater confidence that study results were due to the intervention and not to other confounding factors. Furthermore, measuring the impact of intervention fidelity on study outcomes helps intervention designers, implementers, and professional development facilitators understand threatening and nonthreatening adaption of an intervention, thus helping them to make informed decisions and achieve desirable outcomes (Stains & Vickrey, 2017; Wolgemuth et al., 2014).

An intervention meets fidelity requirements if each of its components is delivered in a similar way to all participants and if it is implemented according to the theory and objectives underlying the intervention (Dumas et al., 2001; Dusenbury et al., 2003; Gaitán-Rossi et al., 2019). Various components of intervention fidelity and strategies for assessment such as treatment manuals, real-time observation, or videotapes or audiotapes of treatment sessions have been reported (An et al., 2020; Noble et al., 2019). It is acknowledged that using multiple methods as part of a fidelity assessment can enhance the validity and reliability of specific interventions.

While there is growing recognition of the importance of and challenges to maintaining fidelity in RCTs, frequently, no empirical evidence is available regarding what an intervention entailed and the extent to which the participants were exposed to it, which makes it difficult to compare the effectiveness of different health care interventions. In the absence of fidelity data, significant results may reflect either the effectiveness of the intervention or the influence of unknown variables (An et al., 2020; Dyas et al., 2014). Procedures must be established in the early phases of a research study to account for how the intervention will be implemented and in what way it will be assessed or monitored during its delivery to study participants (Mars et al., 2013; Tang et al., 2020; Toomey et al., 2020).

In Malawi, antiretroviral therapy (ART) uptake among pregnant and breastfeeding women living with HIV increased, resulting in a notable reduction of vertical HIV transmission (Haas et al., 2016; Tippett Barr et al., 2016). Despite significant gains in the uptake of ART, suboptimal ART adherence and retention in care remain (Chagomerana et al., 2018; Landes et al., 2019; Tenthani et al., 2014). There is an urgent need for innovative, effective, and scalable interventions to improve adherence and retention that can be feasibly implemented in overburdened health care systems. In recognizing the critical need for evidence-based, cost-effective, and scalable interventions to improve maternal adherence and retention, we developed a brief video-based intervention that provides pregnant women living with HIV initiating ART a VITAL Start (Video intervention to Inspire Treatment Adherence for Life) at the critical moment before committing to lifelong treatment. The pilot demonstrated that VITAL Start was very feasible to implement, highly acceptable by both pregnant women and health care workers (HCWs), and resulted in excellent knowledge acquisition and better short-term self-reported adherence as compared with the control group (Kim et al., 2019). These preliminary findings led to an RCT to assess the efficacy of the intervention before broader scale implementation (Kim et al., 2020).

Despite the known importance of fidelity in RCTs, there is a scarcity of published literature about intervention fidelity in studies of ART adherence interventions in primary health care settings. Therefore, the purpose of this article is to describe a systematic, evidence-based intervention fidelity approach specific to an RCT intervention (Feely et al., 2018; Song et al., 2010; Wickersham et al., 2011) designed to improve adherence to ART among pregnant and breastfeeding women. This study focused on measuring fidelity through an implementer and individual client interactions framework. This interaction was measured through four dimensions of fidelity: adherence, dose, quality of delivery, and participant responsiveness (Dusenbury et al., 2003). We describe the fidelity assessment conducted for the VITAL Start intervention and present the findings in the context of intervention implementation. The aim was to evaluate to what extent VITAL Start was delivered according to protocol, and what refinements, if any, are needed to the training and intervention delivery prior to broader scale dissemination.

Method

This study was derived from a larger RCT to evaluate the impact, implementation, and cost-effectiveness of a novel video-based intervention-VITAL Start—delivered to pregnant women living with HIV at an antenatal clinic in Malawi (Kim et al., 2019, 2020). At present, no published clinical trial has reported the use of video-based interventions to improve HIV treatment-related patient outcomes in developing countries. The intervention (VITAL Start) was administered by a research assistant (RA) after the baseline survey.

Setting and Population

The RCT was conducted in three health facilities in Malawi. Two were peri-urban health centers in the Central Region, namely, Kawale Health Centre and Area 25 Health Centre, and the third one was a rural tertiary hospital, Mangochi District Hospital in the Southeastern Malawi. The three health facilities were operated by the government and provide free antenatal care (ANC) as well as HIV testing and treatment services according to Malawi Ministry of Health (MOH) guidelines. Approximately 95% of pregnant women attended ANC clinic at least once (“Malawi Demographic and Health Survey 2015–16,” 2017). The HIV prevalence among ANC women at these health facilities ranged from 11% to 13% (Kim et al., 2020). Recruited pregnant women living with HIV were randomized to receive the VITAL Start or the standard of care (pre-ART counseling provided by a HCW using the Malawi MOH National ART counseling flipchart) during October 2018 to July 2021 at baseline (Kim et al., 2020).

VITAL Start Intervention

The VITAL Start intervention development process has been described elsewhere (Kim et al., 2019). The VITAL Start intervention comprised a 27-minute video followed by 10 minutes of question and answer with an RA to reinforce key messages and provide an opening for individual counseling if needed. VITAL Start was administered to the participants a single time prior to the start of treatment. The intervention provided information about HIV treatment and prevention, modeled positive attitudes about lifelong ART, utilized gain-framed messaging for partner disclosure and engagement, and provided education regarding infant HIV testing and treatment. The VITAL Start intervention had three components: a pre-video orientation, video viewing, and post-video counseling (Figure 1). The intervention was delivered after consenting and randomization processes on a portable electronic Samsung Galaxy tablet in a counseling room at the health facility dedicated for study activities.

Figure 1

VITAL Start Intervention Core Elements

Intervention Training

Prior to intervention delivery, all RAs completed a 1-day VITAL Start intervention training to ensure competency to deliver the intervention to participants as planned. Competency assessed the extent to which the RA delivered VITAL Start as intended, their ability to address participant’s questions or to make referrals when necessary, and demonstrate delivery of the post-video script with confidence and ease. The training was conducted at a central venue and was facilitated by the clinic system’s Research Officer (RO), who supervised the RAs. Eight RA with Malawi School Leaving Certificate Exam (MSCE) equivalent to O-level (or high school diploma) were trained to administer VITAL Start. The training comprised a PowerPoint presentation of intervention delivery, followed by watching the video and practicing the pre- and post-video scripts through role-plays. The RO assessed the intervention competencies of the RA by piloting the completion of the RA Self-administered Intervention Fidelity Checklist with the interventionist (RA) during a VITAL Start counseling session and comparing the RA’s completed form with the RO’s Observer Intervention Fidelity checklist until the forms were filled identically.

Intervention Delivery

The RAs were responsible for delivering VITAL Start (Figure 1). During the pre-video orientation session, the RA was required to read the pre-video script to the participant, demonstrate how to stop and play the video, complete the pre-video checklist to ensure that tablet was set up appropriately, set the timer for 25 minutes, place a “do not disturb” sign on the door, and leave the client in the counseling room to watch the video alone. The RA returned to the counseling room after 25 minutes and waited for the client to finish watching the remaining 2 minutes of the video. The RA then went through the post-video counseling script with the participant, which comprised a flipchart with key messages from the video. At the end of the session, the RA completed one Self-administered Fidelity Checklist per session which had a pre- and post-video section to document the entire session delivery. The RO completed an Observer Intervention Fidelity checklist to evaluate the delivery of the intervention to discern if it was delivered as intended (e.g., strategies delivered, session length) and if the interventionists (RA) were adherent to the delivery plan during observed VITAL Start sessions during the training, and intermittently throughout the intervention delivery period. The RO completed one checklist per observed session that had a pre- and post-video section. The two fidelity checklists were used to monitor, measure, and ensure fidelity to the VITAL Start intervention.

Intervention Fidelity

An overall fidelity score for the individual interventionist was determined to assess the level of fidelity maintained throughout the implementation period based upon four key measurement criteria as outlined in Dusenbury et al. (2003): adherence, dose, quality of delivery, and participant responsiveness. Each component was measured individually to give a breakdown of fidelity on every level. As noted above, fidelity assessments were completed via the RA Self-administered Intervention Fidelity Checklist and the Research Observer Intervention Fidelity Checklist. All VITAL Start sessions the RO had not observed directly were audio recorded. The RO reviewed the paper and audio-recorded fidelity assessments. The data were transferred to an observation checklist and were then included in the analyses. Item definitions, measurement methods/tools, sources of data, and scoring/analysis are outlined in Table 1. To ensure maximally objective measurements of subjective items (e.g., quality of delivery), specific definitions of each choice option were developed.

Table 1

Summary of Individual-Level Fidelity Measures

Domain	Definition	Development/adaptations	Source of data	Scoring & analysis
Adherence	Extent to which implementation of a particular activity and methods are consistent with how the intervention is written. Specifically assessing if all key activities were done.	Self-developed based on VITAL Start (VS) intervention script 29 items Measures key intervention activities, split into two sections: (1) pre-video and (2) post-video	Observer (RO) Intervention Fidelity Checklist—Section 1: Adherence Self-Administered (RA) Intervention Fidelity Checklist—Section 1: Adherence	Yes/No responses RO: Total score out of 29. RA: Total score out of 2. A higher score indicates higher adherence fidelity.
Dose	Amount of program content received by the participant.	Self-developed based on VS intervention Three items measured by completion of the following sections of the intervention: (1) introduction/pre-video session, (2) video, and (3) post-video session	Observer (RO) Intervention Fidelity Checklist—Section 2: Dose Self-Administered (RA) Intervention Fidelity Checklist—Section 2: Dose	Yes/No responses RO & RA: Total score out of 3. A higher score indicates higher dose fidelity.
Quality of delivery	Ratings of provider effectiveness. Assesses the extent to which a provider approaches a theoretical ideal in terms of delivering program content. Quality of delivery is defined as “the manner in which a teacher, volunteer, or staff member delivers a program” (Carroll et al., 2007; Mihalic, 2004)	12 items total 11 items from Behavior Change Counseling Index (BECCI) (University of Wales College of Medicine, 2002) measuring quality of behavior change counseling (BCC). One item from Webster et al. (2011) assessing quality of explanation. Likert scale adapted to be the same as BECCI for consistency.	Observer (RO) Intervention Fidelity Checklist—Section 3: Quality of Delivery	Likert scale ranging from 0 to 4 (not at all to a great extent) RO: Scores range from 0 to 48. A higher score indicates higher quality of delivery.
Participant responsiveness	Ratings of the extent to which participants are engaged by and involved in the activities and the content of the program	Self-developed (Mishima et al., 2000) Four to six items Two items assess engagement (2 & 3); two items assess involvement in activities (1 & 4) Likert scale adapted from BECCI (University of Wales College of Medicine, 2002) for consistency.	Observer (RO) Intervention Fidelity Checklist—Section 4: Participant Responsiveness Self-Administered (RA) Intervention Fidelity Checklist—Section 4: Participant Responsiveness	Likert scale ranging from 0 to 2 (not at all to a great extent) Item 3 must be reverse scored. RA: Scores range from 0 to 8. RO: Scores range from 0 to 12. A higher score indicates higher participant responsiveness.

Source. Adapted from Dusenbury et al. (2003).

Note. VS = VITAL start; RO = research officer; RA = research assistant.

Scores were calculated based on the completion of the checklists. Adherence was defined as “according to protocol” that meant the information given had to be accurate and complete. In cases where the term was broad and may include multiple points, the RA had to cover all points on the script to be “Yes.” For Dose, the participant completed the whole of each session part to be considered “Yes.” Each “Yes” carried a score of “1” and no “0.” For quality of delivery, each item carried a score of 0 to 4, and was scored on a 5-point Likert-type scale, where 0 represented not at all, 1 minimally, 2 to some extent, 3 a good deal, and 4 to a great extent. For participant responsiveness, each item carried a score of 0 to 2, where 0 represented not at all, 1 to some extent, and 2 a great extent. Mean total scores were calculated as the mean across all items. The overall score was the mean of all items.

Research Assistant Self-Administered Intervention Fidelity Checklist

The RA completed the Self-Administered Intervention Checklist immediately after completion of the post-video counseling session. The checklist had a pre- and post-video section on one form. The checklist comprised four sections: adherence, dose, participant responsiveness, and general comments. Adherence was defined as the extent to which implementation of a particular activity and methods were consistent with the VITAL Start intervention delivery protocol, specifically assessing if all key activities were completed. This was measured by completing two items that assessed whether the pre-video and post-video scripts were delivered according to the specified protocol. Dose was defined as the amount of VITAL Start content received by the participant and was measured with three items consisting of assessing whether the participant went through the entire pre-video, video, and post-video sessions. Participant’s responsiveness was defined as ratings of the extent to which participants were engaged by and involved in the activities and the content of VITAL Start and was measured through four items looking at participants’ active listening cues and asking questions for clarification. In addition, the RA documented the duration of the entire session by indicating start time before the pre-video script to end time after the post-video session. This fidelity assessment was recorded on paper or via a form programmed in a SurveyCTO application on a Samsung Galaxy tablet for each participant as they received the intervention.

Research Officer Observer Intervention Fidelity Checklist

The RO completed the Observer Intervention Checklist during the VITAL Start pre- and post-video counseling session. The checklist had a pre- and post-video section on one form. The checklist comprised six sections: adherence, dose, quality of delivery, participant responsiveness, general comments, and review with the RA. Adherence was measured by completing 29 items, which assessed whether the RA delivered the pre-video and post-video sessions according to the specified protocol. Dose was measured with three items assessing whether the participant went through the entire pre-video, video, and post-video sessions. Quality of delivery was defined as the manner in which the RA delivered VITAL Start and was measured through 12 items looking at whether the RA was responsive to the participant’s well-being and engaged the participant in a client-centered manner. Participant’s responsiveness was measured through six items looking at participants’ active listening cues and asking questions for clarification. In addition, the RO documented the duration of the entire session by indicating the start time before the pre-video script and end time after the post-video counseling session. The RO completed one direct observation per site per month. In cases where a direct observation could not be conducted, a randomly selected audio recorded session from the month was used. RA retraining was conducted for any low-performing fidelity standards defined as a score ≤55 points out of 120 points. Two low fidelity scores were observed at the beginning of the study and re-training occurred soon after the observation was completed.

Intervention Receipt

Participants demonstrate intervention receipt when they engage with and participate in the intervention components and its related skills (Bellg et al., 2004; Dusenbury et al., 2003; McGuire et al., 2019). To measure receipt of VITAL Start, participants were asked questions and allowed to ask questions during the post-video session. As another measure of intervention receipt, the RO reviewed the completed live and audio-recorded fidelity assessments and offered further instruction to the RA on areas to improve or reinforcement, as appropriate. The RA and RO observed cues to assess participants’ engagement during VITAL Start sessions.

Data Analysis

Fidelity data were collected on paper and entered into SurveyCTO by research staff. Descriptive statistics were calculated for participants reached, and total and mean scores for adherence, dose, quality of delivery, and participant responsiveness. Cronbach’s alpha was used to evaluate the reliability of the fidelity scores of the RAs and ROs. The data were exported into STATA version 13.0 for cleaning and analysis.

Ethical Approval

The study was reviewed and approved by the National Health Sciences Research Committee, the local ethics committee in Malawi, and the Baylor College of Medicine Institutional Review Board. Informed consent was obtained from all study participants.

Results

Reach

Of the 1,012 pregnant women who attended ANC from October 2018 to July 2021 and who were eligible (Kim et al., 2020), 800 (79%) consented to the study, and of those, 379 (47%) received VITAL Start. Eight RAs delivered 379 VITAL Start counseling sessions. Of the 379 sessions, 43 (11%) were observed by the RO. The mean age of the participants was 28 (6.6).

Adherence

RA self-assessment mean score was 2 (SD = 0). RO observers mean score was 28 (SD = 1.3; Table 2). Overall adherence to intervention components was very high (95%). The coefficient alpha was .78.

Table 2

Mean Fidelity Scores From RA and RO Ratings per Domain

Domain	RA, M (SD)	RO, M (SD)
Adherence	2 (0)	28 (1.3)
Dose	3 (0)	3 (0)
Quality of delivery^a	–	40 (8.6)
Participant responsiveness	6.1 (1.3)	10.4 (1.3)
Overall fidelity score	11.1 (1.3)	77.8 (8.6)

Note. Adherence and dose score range from 0 (no) to 1 (yes). Adherence total score range was 0 to 2 and 0 to 29 for RA and RO, respectively. Dose total score range was 0 to 3 for both RA and RO. Quality of delivery score range was from 0 (not at all) to 4 (a great extent) and total score range was 0 to 48. Participant responsiveness score range was from 0 (not at all) to 2 (a great extent) and total score range was 0 to 8 and 0 to 12 for RA and RO, respectively. RA = research assistant; RO = research officer.

Quality of delivery was only assessed by the RO.

Dose

Of the 379 participants randomized to the intervention group, 100% received all three counseling sections (intervention components). The planned intervention components were implemented during the counseling session. RAs delivered all counseling components. RA self-assessment mean score was 3 (SD = 0). RO observers mean score was 3 (SD = 0).

Quality of Delivery

In reviewed counseling sessions, the RAs sufficiently delivered the intervention. RO observers mean score was 40 (SD = 8.6). The mean scores for each quality of intervention delivery checklist item are reported in Table 3. The coefficient alpha was .93. Quality of intervention delivery was high indicating that RA’s capability to deliver VITAL Start was high.

Table 3

Quality of Delivery Descriptive Statistics

No.	Item	M (SD)
1	Did the RA invite the participant to talk about behavior change?	3.3 (0.9)
2	Did the RA demonstrate sensitivity to discuss other issues?	3.1 (1.0)
3	Did the RA encourage participant to talk about how they feel about their HIV diagnosis/lifelong ART?	3.4 (0.9)
4	Did the RA encourage participant to talk about how their HIV diagnosis/lifelong ART may impact their behavior?	3.3 (1.0)
5	Did RA ask question(s) about how the participant thinks/feels about making behavior changes for lifelong ART?	3.4 (1.0)
6	Did the RA use empathic listening statements when the participant talked about her HIV diagnosis, and behavior changes needed to take lifelong ART?	3.3 (0.9)
7	Did the RA use summaries to bring together what the participant said about these behavior changes?	3.0 (1.2)
8	Did the RA acknowledge possible challenges about behavior change that the participant brought up?	3.3 (1.0)
9	When the RA provided information, was it thoughtful/sensitive to patient concerns and understanding?	3.3 (1.2)
10	Did the RA actively convey respect regarding the behavior changes (related to lifelong ART) the participant said they wanted to make?	3.6 (0.8)
11	Did the RA and the participant exchange ideas about what behavior changes the patient could make for lifelong ART?	3.4 (0.9)
12	Throughout the session, did the RA explain things in an understandable way?	3.8 (0.4)
13	Overall score	40.3 (8.6)

Note. Quality of delivery score range from 0 (not at all) to 4 (a great extent) and total score range was 0 to 48. Quality of delivery was only assessed by the research officer. RA = research assistant; HIV = human immunodeficiency virus; ART = antiretroviral therapy.

Participant Responsiveness

RA self-assessment mean score was 6.1 (SD = 1.3). RO observers mean score was 10.4 (SD = 1.3). The coefficient alpha was .62. The high participant responsiveness scores demonstrate that the participants were engaged during the VITAL Start session.

Discussion

Intervention fidelity is a key element to ensure that an intervention has been implemented as intended and accurately tested, which results in making studies more reliable, internally valid, and externally generalizable. The aim of this intervention fidelity study was to evaluate to what extent the VITAL Start intervention was delivered according to protocol, specifically assessing the following fidelity domains: adherence, dose, quality of delivery, and participant responsiveness. The results demonstrate that the intervention contents were delivered as specified, the dose of the intervention was delivered as specified, the quality with which the intervention was delivered, and participant responsiveness was high. Furthermore, the study highlights that the intervention was delivered with high fidelity to counseling protocols.

Systematically monitoring adherence to intervention delivery needs to be built into the design of the randomized controlled trials. High-quality implementation is more likely when core intervention components are defined in advance and then systematically monitored (An et al., 2020; Stockdale et al., 2020). Our results indicate that the RAs were competent and able to achieve high adherence to VITAL Start protocol. RA and RO adherence scores were very high. The results indicated over 95% adherence to delivery of the intervention. Moreover, every single item was found to have been fully delivered to the participants. These findings support that the estimates of the intervention’s effects that will be produced by the RCT in due course can be interpreted as accurate impressions of its benefits or otherwise.

Interventions are less effective when study participants do not receive the intended dosage. In our case, the complete amount of the intervention was delivered to the participants. Participants completed all parts of the counseling session: pre-video, video, and post-video. Different studies have shown that programs that implemented all intervention components led to greater improvements than those with modifications (Mihalic, 2004; Noble et al., 2019). Therefore, it is important for participants to receive the intended number of sessions. Interventionists may compromise the dosage because of lack of time and commitment, undermining any effects that the intervention may have (Broekhuizen et al., 2012; Sprange et al., 2021). In addition, a common theme uttered by interventionists is that some exposure to the intervention elements is better than none. In our case, full intervention exposure was supported by (1) emphasis during the training on the importance of delivering the whole session and (2) conducting direct observations and audio-recording those sessions not directly observed.

Fidelity captures the way in which the intervention is delivered and involves interactions between implementers and participants. Observers (ROs) noted how most of the time the RAs were able to correctly deliver the intervention using overall strategies as prescribed and answer the participants’ questions. Furthermore, this demonstrated that the training RAs received was adequate to conduct the counseling as intended. As a measure of the quality of intervention delivery, VITAL Start implementation was rated based on the RA’s ability to respect the participants’ feelings and views, explain things clearly, and provide opportunities for discussions with the participants, and to focus on the participants’ behavior change.

This study shows that the participants who were randomized to VITAL Start did receive the full intervention—their responsiveness to the intervention was favorable. The participants were comfortable with the intervention and were able to follow what the RA was explaining. The participants were engaged as illustrated by the RAs’ and ROs’ high participant responsiveness scores and this may also explain why the participants attended the whole intervention session. Intervention receipt was high similar to other RCT (McGuire et al., 2019).

Our study offers insights in measuring intervention fidelity in an RCT. To capture a comprehensive picture of intervention fidelity and consequently produce an accurate examination of the effectiveness of an intervention, we were able to fully measure four dimensions of fidelity (Dusenbury et al., 2003; Mihalic, 2004; Proctor et al., 2011). We were able to comprehensively evaluate fidelity through measuring four domains, namely, adherence, exposure, quality of intervention delivery, and participant responsiveness, which is an important strength of the overall study.

Our study has some limitations. First, we used direct observation of intervention delivery to evaluate fidelity. Future studies should consider video-recording actual VITAL Start counseling session to avoid the potential Hawthorne effect on the assessment of interventionist performance (Davies & Shackleton, 1975). However, this was also a strength as an opportunity for additional feedback to the RAs may have been missed. Second, to enhance intervention fidelity, some RAs needed to be re-trained; however, the fidelity assessments helped us to more easily identify those in need of additional training. Finally, RAs were used to administer the intervention in our study. Assuming the intervention is demonstrated to be successful, it will likely be implemented by routine HCWs typically with a higher education level and training background than the RAs. Therefore, measuring and maintaining fidelity would be important.

Implications for Practice and Research

Our findings will inform both practice and future research. We have demonstrated that intervention fidelity assessment can be achieved with simple tools for implementers and observers. Knowing that fidelity enhances intervention effectiveness, practitioners and researchers should include measures to evaluate whether implementers are adhering to procedures/protocols when delivering new interventions. This evaluation illustrates how providers or researchers could measure intervention fidelity as part of implementation to improve the quality of service delivery and adherence to standard protocols. These results have implications for establishing the effectiveness of new interventions and could avoid negating important innovation that may demonstrate minor impact due to implementation errors. Furthermore, awareness that easy-to-use tools to measure fidelity exist may increase uptake of fidelity assessments, thereby enhancing implementation of and robust evaluations of interventions. In addition, a simple training is adequate for implementers to follow intervention delivery protocols and perform consistently; therefore, practitioners should provide the training before and at intermediate time points during the implementation phase. This will result in maximum benefit not only for intervention efficacy but also to the individual clients who receive the service in real-world settings. Researchers should recommend fidelity assessments for different interventions as part of the study design. In summary, assessing intervention fidelity in practice and research is feasible and will ensure that individual clients are treated consistently and receive the full dose of the intervention with high quality for maximum health benefits in research or real-world settings.

Conclusion

Intervention fidelity enhances confidence in scientific findings, increases power to detect effects, and facilitates theory testing. Given the importance of fidelity in documenting the effectiveness of an intervention, developing a reliable and valid measure of fidelity should receive the same level of attention as other aspects of the study. Through this study, we showed that with only a modest training, implementers and observers can conduct reliable, comprehensive fidelity assessments. The intervention contents and dose were delivered as specified, and the quality with which the intervention was delivered, and participant responsiveness were high. These findings are promising and timely as researchers conducting RCTs of specific types of interventions ought to consider fidelity measures for more domains to boost internal validity and reliability of their study results.

Supplemental Material

sj-docx-1-hpp-10.1177_15248399231177303 – Supplemental material for Intervention Fidelity to VITAL Start (Video Intervention to Inspire Treatment Adherence for Life) in a Randomized Controlled Trial Among Women Living With HIV in Malawi

Supplemental material, sj-docx-1-hpp-10.1177_15248399231177303 for Intervention Fidelity to VITAL Start (Video Intervention to Inspire Treatment Adherence for Life) in a Randomized Controlled Trial Among Women Living With HIV in Malawi by Tapiwa A. Tembo, Christine M. Markham, Steven P. Masiano, Rachael Sabelli, Elizabeth Wetzel, Saeed Ahmed, Mtisunge Mphande, Angella M. Mkandawire, Mike J. Chitani, Innocent Khama, Rose Nyirenda, Alick Mazenga, Elaine J. Abrams and Maria H. Kim in Health Promotion Practice

Supplemental Material

sj-docx-2-hpp-10.1177_15248399231177303 – Supplemental material for Intervention Fidelity to VITAL Start (Video Intervention to Inspire Treatment Adherence for Life) in a Randomized Controlled Trial Among Women Living With HIV in Malawi

Supplemental material, sj-docx-2-hpp-10.1177_15248399231177303 for Intervention Fidelity to VITAL Start (Video Intervention to Inspire Treatment Adherence for Life) in a Randomized Controlled Trial Among Women Living With HIV in Malawi by Tapiwa A. Tembo, Christine M. Markham, Steven P. Masiano, Rachael Sabelli, Elizabeth Wetzel, Saeed Ahmed, Mtisunge Mphande, Angella M. Mkandawire, Mike J. Chitani, Innocent Khama, Rose Nyirenda, Alick Mazenga, Elaine J. Abrams and Maria H. Kim in Health Promotion Practice

Footnotes

Authors’ Note:

We acknowledge and thank all study participants who took their valuable time to participate in this study. We also gratefully acknowledge the Malawi Ministry of Health, Tingathe Outreach Program Team, Baylor College of Medicine Children’s Foundation Malawi, Baylor International Pediatric AIDS Initiative, the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR), and the United States Agency for International Development (USAID). Research reported in this publication was supported by the National Institute of Mental Health (NIMH). T.A.T. was supported by the Fogarty International Center of the National Institutes of Health under Award Number D43 TW010060. M.H.K. was supported by the Fogarty International Center of the National Institutes of Health under Award number K01 TW009644 and R01 MH115793-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

ORCID iD

Tapiwa A. Tembo

Suplemental Material

Supplemental material for this article is available online at .

References

Altman

D. G.

Schulz

K. F.

Moher

Egger

Davidoff

Elbourne

Lang

, & CONSORT GROUP (Consolidated Standards of Reporting Trials). (2001). The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Annals of Internal Medicine, 134(8), 663–694. https://doi.org/10.7326/0003-4819-134-8-200104170-00012

Dusing

S. C.

Harbourne

R. T.

Sheridan

S. M.

(2020). What really works in intervention? Using fidelity measures to support optimal outcomes. Physical Therapy, 100(5), 757–765. https://doi.org/10.1093/ptj/pzaa006

Bellg

A. J.

Borrelli

Resnick

Hecht

Minicucci

D. S.

Ory

Ogedegbe

Denise, Orwig

Ernst

Czajkowski

, & Treatment Fidelity Workgroup of the NIH Behavior Change Consortium (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23(5), 443–451. https://doi.org/10.1037/0278-6133.23.5.443

Bonell

Fletcher

Morton

Lorenc

Moore

(2012). Realist randomised controlled trials: A new approach to evaluating complex public health interventions. Social Science & Medicine, 75(12), 2299–2306. https://doi.org/10.1016/j.socscimed.2012.08.032

Broekhuizen

Althuizen

van Poppel

M. N.

Donker

van Mechelen

(2012). From theory to practice: Intervention fidelity in a randomized controlled trial aiming to optimize weight development during pregnancy. Health Promotion Practice, 13(6), 816–825. https://doi.org/10.1177/1524839912447190

Carroll

Patterson

Wood

Booth

Rick

Balain

(2007). A conceptual framework for implementation fidelity. Implementation Science, 2, Article 40. https://doi.org/10.1186/1748-5908-2-40

Chagomerana

M. B.

Miller

W. C.

Tang

J. H.

Hoffman

I. F.

Mthiko

B. C.

Phulusa

John

Jumbe

Hosseinipour

M. C.

(2018). Optimizing prevention of HIV mother to child transmission: Duration of antiretroviral therapy and viral suppression at delivery among pregnant Malawian women. PLOS ONE, 13(4), Article e0195033. https://doi.org/10.1371/journal.pone.0195033

Chang

P. S.

Chao

A. M.

Jang

Y. Y. F.

(2019). Intervention fidelity in Qigong randomized controlled trials: A method review. Geriatric Nursing, 40(1), 84–90. https://doi.org/10.1016/j.gerinurse.2018.07.001

Davies

D. R.

Shackleton

V. J.

(1975). Psychology and work. London: Methuen.

10.

Dumas

J. E.

Lynch

A. M.

Laughlin

J. E.

Phillips Smith

Prinz

R. J.

(2001). Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the EARLY ALLIANCE prevention trial. American Journal of Preventive Medicine, 20(1 Suppl.), 38–47. https://doi.org/10.1016/s0749-3797(00)00272-5

11.

Dusenbury

L. B. R.

Falco

Hansen

W. B.

(2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18(2), 237–256. https://academic.oup.com/her/article/18/2/237/820533

12.

Dyas

J. V.

Togher

Siriwardena

A. N.

(2014). Intervention fidelity in primary care complex intervention trials: Qualitative study using telephone interviews of patients and practitioners. Quality in Primary Care, 22(1), 25–34.

13.

Feely

Seay

K. D.

Lanier

Auslander

Kohl

P. L.

(2018). Measuring fidelity in research studies: A field guide to developing a comprehensive fidelity measurement system. Child and Adolescent Social Work Journal, 35(2), 139–152. https://doi.org/10.1007/s10560-017-0512-6

14.

French

C. T.

Diekemper

R. L.

Irwin

R. S.

, & CHEST Expert Cough Panel. (2015). Assessment of intervention fidelity and recommendations for researchers conducting studies on the diagnosis and treatment of chronic cough in the adult: CHEST guideline and expert panel report. Chest, 148(1), 32–54. https://doi.org/10.1378/chest.15-0164

15.

Gaitán-Rossi

De la Cerda Lobato

Pérez Navarro

A. C.

Aguilar Esteva

Vargas García

M. R.

Vilar-Compte

(2019). Fidelity of implementation of Prospera digital: Evaluation of a multi-site mHealth intervention aimed at improving maternal health outcomes in Mexico. Current Developments in Nutrition, 3(10), Article nzz107. https://doi.org/10.1093/cdn/nzz107

16.

Haas

A. D.

Tenthani

Msukwa

M. T.

Tal

Jahn

Gadabu

O. J.

Spoerri

Chimbwandira

van Oosterhout

J. J.

Keiser

(2016). Retention in care during the first 3 years of antiretroviral therapy for women in Malawi’s option B+ programme: An observational cohort study. The Lancet HIV, 3(4), e175–e182. https://doi.org/10.1016/S2352-3018(16)00008-4

17.

Hariton

Locascio

J. J.

(2018). Randomised controlled trials—The gold standard for effectiveness research: Study design: Randomised controlled trials. BJOG: An International Journal of Obstetrics & Gynaecology, 125(13), Article 1716. https://doi.org/10.1111/1471-0528.15199

18.

Kim

M. H.

Ahmed

Tembo

Sabelli

Flick

Mazenga

Le Blond

Simon

Hartig

Wetzel

Nyirenda

Kazembe

P. N.

Mphande

Mkandawire

Chitani

M. J.

Abrams

E. J.

(2019). VITAL start: Video-based intervention to inspire treatment adherence for life-pilot of a novel video-based approach to HIV counseling for pregnant women living with HIV. AIDS and Behavior, 23(11), 3140–3151. https://doi.org/10.1007/s10461-019-02634-1

19.

Kim

M. H.

Tembo

T. A.

Mazenga

Myer

Sabelli

Flick

Hartig

Wetzel

Simon

Ahmed

Nyirenda

Kazembe

P. N.

Mphande

Mkandawire

Chitani

M. J.

Markham

Ciaranello

Abrams

E. J.

(2020). The video intervention to inspire treatment adherence for life (VITAL Start): Protocol for a multisite randomized controlled trial of a brief video-based intervention to improve antiretroviral adherence and retention among HIV-infected pregnant women in Malawi. Trials, 21(1), Article 207. https://doi.org/10.1186/s13063-020-4131-8

20.

Landes

van Lettow

Nkhoma

Tippett Barr

Truwah

Shouten

Jahn

Auld

Kalua

van Oosterhout

J. J.

(2019). Low detectable postpartum viral load is associated with HIV transmission in Malawi’s prevention of mother-to-child transmission programme. Journal of the International AIDS Society, 22(6), Article e25290.

21.

Malawi Demographic and Health Survey 2015–16. (2017, February). https://dhsprogram.com/pubs/pdf/FR319/FR319.pdf

22.

Mars

Ellard

Carnes

Homer

Underwood

Taylor

S. J. C.

(2013). Fidelity in complex behaviour change interventions: A standardised approach to evaluate intervention integrity. British Medical Journal Open, 3(11), Article e003555. https://doi.org/10.1136/bmjopen-2013-003555

23.

McGuire

Duncan

Pozehl

(2019). Incorporating intervention fidelity components into randomized controlled trials promoting exercise adherence in heart failure patients. Research in Nursing & Health, 42(4), 306–316. https://doi.org/10.1002/nur.21949

24.

Mihalic

(2004). The importance of implementation fidelity. Emotional and Behavioral Disorders in Youth, 4(4), 83–105.

25.

Mishima

Kubota

Nagata

(2000). The development of a questionnaire to assess the attitude of active listening. Journal of Occupational Health, 42(3), 111–118. https://doi.org/10.1539/joh.42.111

26.

Noble

A. J.

Snape

Ridsdale

Morgan

Nevitt

S. J.

Goodacre

Marson

(2019). Assessing treatment fidelity within an epilepsy randomized controlled trial: Seizure first aid training for people with epilepsy who visit emergency departments. Behavioural Neurology, 2019, Article 5048794. https://doi.org/10.1155/2019/5048794

27.

Proctor

Silmere

Raghavan

Hovmand

Aarons

Bunger

Griffey

Hensley

(2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7

28.

Song

M. K.

Happ

M. B.

Sandelowski

(2010). Development of a tool to assess fidelity to a psycho-educational intervention. Journal of Advanced Nursing, 66(3), 673–682.

29.

Sprange

Mountain

Craig

(2021). Evaluation of intervention fidelity of a complex psychosocial intervention lifestyle matters: A randomised controlled trial. British Medical Journal Open, 11(4), Article e043478. https://doi.org/10.1136/bmjopen-2020-043478

30.

Stains

Vickrey

(2017). Fidelity of implementation: An overlooked yet critical construct to establish effectiveness of evidence-based instructional practices. CBE: Life Sciences Education, 16(1), Article 0113. https://doi.org/10.1187/cbe.16-03-0113

31.

Stockdale

S. E.

Hamilton

A. B.

Bergman

A. A.

Rose

D. E.

Giannitrapani

K. F.

Dresselhaus

T. R.

Yano

E. M.

Rubenstein

L. V.

(2020). Assessing fidelity to evidence-based quality improvement as an implementation strategy for patient-centered medical home transformation in the veterans health administration. Implementation Science, 15(1), Article 18. https://doi.org/10.1186/s13012-020-0979-y

32.

Sweeney-Magee

Kale

Galton

Hamill

Gilbert

(2016). Assessing the fidelity of delivery of an intervention to increase attendance at the English stop smoking services. Implementation Science, 11(1), Article 166. https://doi.org/10.1186/s13012-016-0498-z

33.

Tang

Tong

Irby

B. J.

Lara-Alecio

Guerrero

(2020). Fidelity of implementation in a randomized controlled trial study: The effect of virtual professional development on bilingual teachers. Bilingual Research Journal, 43(1), 111–124. https://doi.org/10.1080/15235882.2019.1711268

34.

Tenthani

Haas

A. D.

Tweya

Jahn

van Oosterhout

J. J.

Chimbwandira

Chirwa

Ng’ambi

Bakali

Phiri

Myer

Valeri

Zwahlen

Wandeler

Keiser

(2014). Retention in care under universal antiretroviral therapy for HIV-infected pregnant and breastfeeding women (“Option B^+”) in Malawi. AIDS, 28(4), 589–598. https://doi.org/10.1097/qad.0000000000000143

35.

Tippett Barr

Schouten

Oosterhout

Gupta

Phiri

Chimbwandira

Thindwa

Blair

Jahn

Lettow

. (2016). National HIV transmission in 4–12 week olds in Malawi’s PMTCT option B+ program. In Proceedings of the Conference of Retroviruses and Opportunistic Infections.

36.

Toomey

Hardeman

Hankonen

Byrne

McSharry

Matvienko-Sikar

Lorencatto

(2020). Focusing on fidelity: Narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions. Health Psychology and Behavioral Medicine, 8(1), 132–151. https://doi.org/10.1080/21642850.2020.1738935

37.

University of Wales College of Medicine. (2002). The Behaviour Change Counselling Index (BECCI): Manual for coding behaviour change counselling.

38.

Webster

T. R.

Mantopoulos

Jackson

Cole-Lewis

Kidane

Kebede

Abebe

Lawson

Bradley

E. H.

(2011). A brief questionnaire for assessing patient healthcare experiences in low-income settings. International Journal for Quality in Health Care, 23(3), 258–268. https://doi.org/10.1093/intqhc/mzr019

39.

Wickersham

Colbert

Caruthers

Tamres

Martino

Erlen

(2011). Assessing fidelity to an intervention in a randomized controlled trial to improve medication adherence. Nursing Research, 60, 264–269. https://doi.org/10.1097/NNR.0b013e318221b6e6

40.

Wolgemuth

J. R.

Abrami

P. C.

Helmer

Savage

Harper

Lea

(2014). Examining the impact of ABRACADABRA on early literacy in Northern Australia: An implementation fidelity analysis. The Journal of Educational Research, 107(4), 299–311.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB

0.03 MB