The YGetIt? Program: A Mobile Application,PEEP,and Digital Comic Intervention to Improve HIV Care Outcomes for Young Adults

Abstract

Introduction. In New York State (NYS), young adults account for the largest number of new human immunodeficiency virus (HIV) infections and struggle to seek and remain in HIV care. Digital interventions and access to peer support have demonstrated positive influences on the HIV care continuum and health outcomes. The New York State Department of Health (NYS DOH) developed YGetIt? (YGI) that combines a mobile application, GET!, peer navigation (PEEPs), and a compelling digital comic series, “Tested,” to facilitate the timely entry of young people into HIV care, to prevent vulnerable youth from dropping out of care, and to achieve sustained viral load suppression among those in care. This article describes the development and early implementation of the YGI digital intervention. Intervention Design. GET! provided a high level of confidentiality and security, ease of access, and Wi-Fi accessibility. YGI enrolled 113 HIV-positive participants from a clinical setting who were individually randomized at a 1:1 ratio to receive access to GET! plus PEEPs (n = 53) or the app alone (n = 60). Lessons learned. For recruitment, staff and organization buy-in was essential to the success of the intervention, and building relationships was critical. GET! development was an iterative process. Peer Engagement Educator Professionals (PEEPs) who were tech savvy, representative of the priority population, and had shared life experience with participants were most impactful. Interest in apps declines over time and participants in the APP alone arm were less engaged. Conclusion. GET! is a communication and engagement tool that supports HIV care and may serve as a model for like digital interventions.

Keywords

HIV/AIDS technology health promotion minority health patient education program planning and evaluation sexual health social determinants of health social marketing/health communication health equity

Incidence of HIV infections (2015–2019) was highest among young adults aged 13 to 34 years in the United States (Centers for Disease Control, 2019). In NYS, young adults accounted for the largest number of new infections in 2018 and struggled to seek and remain in HIV care (NYS DOH AIDS Institute (AI), 2019). Young adults frequently utilize social media and digital technology (Schaeffer, 2019). Digital social networks and mobile health (mHealth) technologies offer ideal health intervention tools. The use of digital tools was essential during the COVID-19 health crisis when in-person contact was limited, and nonurgent medical appointments were restricted. Technologies and innovative approaches are essential to maintaining patient–provider contact and offer a convenient method of communication. Health organizations have been utilizing mHealth platforms to reach and serve their respective communities (Pasipanodya et al., 2020). A review of HIV-focused social media interventions revealed increased access to information and enhanced ability to communicate (Cao et al., 2017). These interventions also provided anonymity, fostered virtual community, and expanded geographical reach (Lemley et al., 2020).

Digital interventions are increasingly used to address complex health issues, such as HIV (Brooks, 2020). Studies have shown that visually vivid and culturally resonant engagement tools, such as comic book formats shared over social media, can effectively educate and reach HIV-positive young adults and other vulnerable populations (Shimazaki et al., 2018). However, human engagement through health educators, social workers, patient navigators, and particularly peer navigators continue to play a significant role in service delivery. Research has shown peer navigators that share disease experience, culture, and socioeconomic backgrounds with the communities they serve can be pivotal to navigating the health care system (Roland et al., 2020). Care navigators reported guiding clients in the HIV care continuum by addressing stigma, connecting patients to physical and behavioral health services and education, providing wraparound services like linkage to housing, and assisting patients with self-management of care (Sarango et al., 2017). Increased access to peer support through a social media platform may facilitate further uptake of healthy behaviors among HIV-positive youth. Innovative interventions are needed to engage and retain young people in HIV care and improve health outcomes.

NYS DOH, AI, Digital Health Initiative (DHI) (2021) developed the YGetIt? Program to address the state’s Ending the Epidemic Initiative (ETE) 2020 goals. Evidence supporting use of social media and peer-led intervention models to improve health outcomes among HIV-positive youth informed the YGetIt? Program. YGetIt? combines a mobile application with a peer navigator, and a compelling educational comic. YGetIt? is designed to (a) facilitate the timely entry of young people into HIV care, (b) prevent vulnerable youth from dropping out of care, and (c) achieve sustained viral load suppression among participants. This article describes the conceptual framework, development, and early implementation of the multifaceted YGetIt? intervention highlighting the program’s elements, challenges, and lessons learned.

Conceptual framework

Development and implementation of YGetIt? aligned with the Integrate, DEsign, Assess, and Share (IDEAS), theoretical framework (Mummah et al., 2016). IDEAS combines behavior theory with design thinking by integrating design, development, and evaluation of mobile apps as interventions to change health behavior in a stepwise process (Mummah et al., 2016). YGetIt? uses health messages and various entertainment mediums that elicit emotional experiences that contribute to behavioral change (Shimazaki et al., 2018).

YGetIt? Background

DHI created the YGetIt? intervention that uses nontraditional digital approaches to meet youth and young adults where they are. DHI, a division of the NYS DOH AI, develops and implements digital strategies to address public health needs. YGetIt? was primarily funded by the U.S. Department of Health and Human Resources and Services Administration (HRSA) Special Projects of National Significance (SPNS) initiative, which explored the impact of innovative digital interventions among HIV-positive youth and young adults to address disparities in HIV incidence and engagement among this population. As a culturally diverse team that is reflective of the communities they serve and collaborates with interdisciplinary teams, DHI recognized the shift in digital media use among youth and young adults and was well positioned to design and implement this intervention. DHI coordinated with the Mt. Sinai AppLab (AppLab), Northwell Health’s Center for AIDS Research and Treatment (CART), and Long Island Crisis Center’s Pride for Youth (PFY) to begin developing YGetIt? in September 2015, with client enrollment beginning in early May 2017.

The goal of the SPNS social media initiative was to implement, evaluate, and disseminate findings from innovative methods for linking and retaining HIV-positive underserved young adults in HIV care and supportive services through social media approaches. YGetIt? utilized a HIV-focused health management mobile application (GET!) paired with PEEPs who provide support and care coordination in-person and through secure text messaging. As part of the program, a graphic serial comic called “Tested” was developed featuring compelling stories and imagery inspired by the lived experiences of young people affected by HIV. The intervention was tested in a randomized design that compared outcomes among those randomized to the App alone arm and those randomized to the App + PEEP arm.

Development of ygetit?

The YGetIt? Program followed the IDEAS theoretical framework.

Figure 1: IDEAS Framework (Mummah et al., 2016) and Table 1.

Figure 1

IDEAS Framework (Mummah et al., 2016)

Table 1

IDEAS Framework

IDEAS (Integrate, Design, Assess, and Share) phases for developing digital health behavior change interventions. (Mummah, et al., 2016)

• Phase 1: Empathize with priority populations by gathering qualitative insights from users from in-depth interviews and focus groups and understand the needs and motivations of the priority group.
• Phase 2: Specify target behavior by translating broad behavioral goals into highly specific target behavior that is aligned with research findings.
• Phase 3: Ground the intervention in behavioral theory by identifying behavioral strategies best suited to the priority users.
• Phase 4: Ideate implementation strategies by brainstorming creative strategies for translating theory and insights into app features and after generating many ideas, quickly winnowing them down.
• Phase 5: Quickly develop prototypes and winnow down features to the most promising ones.
• Phase 6: Gather user feedback through interviews and questionnaires.
• Phase 7: Build a minimum viable product with only the most essential features and incorporate analytics to collect data on app usage patterns. Phase 7: Build a minimum viable product with only the most essential features and incorporate analytics to collect data on app usage patterns.
• Phase 8: Conduct a pilot test of the intervention for efficacy and usability using a small-scale evaluation, analysis of usage behavior, interviews, and questionnaires.
• Phase 9: Evaluate the tool’s efficacy in a randomized controlled trial, while studying the app’s analytics and assessing its broader effects on behaviors and risk factors.
• Phase 10: Share the results by publishing the results and working with industry partners to also disseminate the results and meanwhile continue to refine the product for potency and usability.

Note. IDEAS = Integrate, DEsign, Assess, and Share.

Phase 1: Empathize

YGetIt? identified young adults, 18 to 34 years of age, who are living with HIV as the priority population. In the empathize phase, we conducted focus groups and interviews with various stakeholder representatives including physicians, community advisory board members including people living with HIV/AIDS (PLWHA), providers, and AI program staff to better understand the priority population’s needs, attitudes, and motivations associated with HIV care. Identified issues to address included pervasive stigma, gaps in access to health information and care, and support from and connections to health systems and providers.

Phase 2: Specify

The YGetIt? team focused on target behaviors to achieve SPNS initiative goals of (a) facilitating timely entry into care, (b) linking and retaining clients in HIV care, and (c) increasing and sustaining viral load suppression. Behavioral intervention goals include utilizing a new digital tool to accessing health information, adhering to medication, engaging with the care team through the app, maintaining health and wellness, and building patient–provider relationships.

Phase 3: Ground

Behavioral change theories have been employed to inform each of these goals. Social cognitive theory (SCT) concepts such as observational learning, self-regulation, and self-efficacy align with the YGetIt? program offerings (Glanz et al., 2008). Observational learning expands the opportunity for acquiring and modifying behaviors (Alicke et al., 2013). Applications of observational learning in media have shown an ability to influence health behaviors via a modeling effect (Bandura, 2001; Barak & Fisher, 2003) and were made available through the GET! mobile application. GET! provided users with behavioral change tools rooted in the self-regulation tenet, such as medication and appointment adherence (self-monitoring) and engagement with peer navigators (enlistment of social support and feedback). GET! supports self-efficacy ideas through users’ ability to independently access health information and communicate with the care team on-demand.

Phase 4: Ideate

The predesign phase of initial intervention concepts, proposed by the YGetIt? Team in alignment with theory and input from stakeholders, led to identification of the following potential features for a mobile app: (a) a newsfeed featuring short informational articles and video content, (b) a digital comic, (c) a secure one-to-one messaging feature, (d) group messaging forum, (e) gamification, (f) video chat, (g) image manipulation, (h) medication and appointment reminders and tracking, (i) medication library, (j) surveys, (k) virtual medical appointments, and (l) a human component in the form of a peer navigator.

Phase 5: Prototypes

Once desired features were identified, app development was an iterative process. The YGetIt? Team quickly realized the importance of selecting an app developer with experience in health applications, health care systems, and an in-depth understanding of the Health Insurance Portability and Accountability Act (HIPAA) compliant technology and privacy security components. YGetIt? identified an app development partner embedded in a prominent NYC hospital network who was well versed in developing mobile applications focusing on chronic illness. Once features were established, the YGetIt? team worked with stakeholders, an artist, and the Sinai App Lab to develop the look and feel of the user interface (UI). Over the course of 12 months, the YGetIt? team developed a beta version of the multifaceted GET! mobile application. The GET! mobile application (Figure 2) was a call to action: Get access to quality health care, Get connected to accurate health information, and Get control of your overall health.

Figure 2

Get Mobile Application

Phase 6: Gather

As the project progressed, the team consulted numerous outside stakeholders including AI’s Young Adults Consumer Advisory Committee (YACAC), as well as NYS-based community and health care organizations to garner feedback via interviews and focus groups. Participants were presented with mock-ups of a mobile app and asked for their input about various app features. Stakeholders were critical to refining aspects of the intervention such as features within GET! and character and story development for “Tested.”

Phase 7: Build – Intervention Design

GET! mobile application

GET! provided a high level of confidentiality and security, ease of access, and discretion (Figure 2). It consisted of an inconspicuous logo design, opt in and out selection, Wi-Fi accessibility, and was transferable from one device to another. In developing the desired functionality of the GET! app, AppLab drew on their previous experience creating similar health management apps for various health conditions (i.e., HealthPROMISE app) (Atreja et al., 2015). To create GET!, AppLab used a combination of custom-developed technologies and the best available software services and toolkits. Through a hybrid development process, AppLab was able to ensure both iOS (Apple) and Google Play (Android) versions of GET! AppLab technologies are compliant with HIPAA and HITECH Act standards. Security measures in the app included but were not limited to industry-leading encryption (of data at rest, in transit, and in use), device security measures, and use of secure protocols like HTTPS. Data were stored in Microsoft’s HIPAA-compliant, secure database cloud servers, governed by Business Associate Contract (as outlined in HIPAA rules) with Mount Sinai Health System.

After careful consideration, the YGetIt? team deemed several components of the GET! app intervention as integral and essential. Those features include peer and administrative dashboards, resources libraries, a messenger, medication and appointment reminders, laboratories results tracking, provider contact information, gamification, and the “Tested” comic.

The app included the following features (Table 2):

Table 2

Demographics of YGetIt? Participants at Enrollment by Intervention Arm (N = 112)

Characteristic	App + PEEP(n = 53), n (%)	App alone (n = 59),n (%)
Age (years)
Median (IQR)	28 (26–30)	29 (26–32)
Gender
Cisgender man	38 (71.7)	43 (72.9)
Cisgender woman	12 (22.6)	15 (25.4)
Other	3 (5.7)	1(1.7)
Race^a
Black/African American	26 (49.1)	29 (49.2)
White	14 (26.4)	20 (33.9)
Ethnicity
Hispanic	15 (28.3)	18 (30.5)
Non-Hispanic	38 (71.7)	41 (69.5)
Education level
Some high school	2 (3.8)	5 (8.5)
Completed high school, or received GED	15 (28.3)	16 (27.1)
Some college, vocational, or trade school	11 (20.7)	14 (23.7)
Associates degree or trade certificate	4 (7.6)	6 (10.2)
Bachelor’s degree	17 (32.1)	13 (22.0)
Higher than Bachelor’s degree	4 (7.5)	5 (8.5)
Born with HIV
No	44 (83.0)	52 (88.1)
Yes	9 (17.0)	7 (11.9)
Results of last viral load test
Viral load ≧200 copies/mL	1 (1.9)	10 (16.9)
Viral load <200 copies/mL or undetectable	32 (60.4)	25 (42.4)
Haven’t had a viral load test	1 (1.9)	0 (0.0)
Don’t know/Not sure	19 (35.8)	24 (40.7)

Note. Table contains self-reported data from participants through a baseline survey. Total enrolled was 113; however, a survey was missing for one participant. Data for 112 participants are displayed. PEEP = Peer Engagement Educator Professional; IQR = interquartile range; mL = milliliter; GED = Graduate Equivalency Degree or General Edcation Diploma.

Race: Percentages do not add up to 100 as participants could choose to identify as multiple races.

• PEEP Messenger: GET! users received standardized messages from a PEEP and were able to text a PEEP anytime support was needed;

• Medication and appointment tracking: Participants could log their medication adherence daily, as well as appointment attendance;

• Access to laboratories: User CD4 counts and viral loads from Northwell Health Center were logged into GET! by PEEPs. Participants could also enter their respective laboratory values;

• Library of resources: GET! offered a library of resources to address common barriers to care observed in YGetIt?’s users. Resources included links to supportive services such as housing, insurance (ADAP), drug user health, mental health, support groups, sexually transmitted infection (STI) education and testing, partner services and more. GET! also offered health, exercise, and education videos that addressed viral load suppression, medication adherence, and general health.

• My Team: Participants could enter their health care team’s contact information into the app;

• “Tested” comic: Participants could view the comic and engage in open forum discussions of “Tested” episodes.

PEEPS

Inclusion of a human element was a key to YGetIt? intervention. A health educator and peer navigator served as PEEPs for the YGetIt? program. PEEPs were paid part-time staff who were embedded in the CART care team. PEEPs were responsible for recruitment, building rapport with enrolled participants and encouraging medication adherence and retention in care through continued communication via the GET! app and in-person interactions. Within GET!, PEEPs utilized the PEEP messenger to provide participants with basic education on HIV and sexual health, emotional support, and aid in facilitation of care logistics. PEEPs were required to send participants enrolled in the App + PEEP arm (see “evaluation design” section) routine weekly text messages through GET! and respond to any requests for support from participants within 24 hours. PEEPs were also responsible for entering participant laboratory values (CD4 and viral load counts) into GET! as soon as they were reported, so they were readily available to participants.

YGetIt? PEEPs were primarily selected based on their ability to communicate with the priority audience, and their HIV and social media knowledge and experience. PEEPs were reflective of the intervention’s priority population. PEEPs were trained in motivational interviewing, protected health information privacy laws, and provided refresher trainings on STIs, HIV, gender identity, and promotion of positive prevention. They received training on app utilization, social media, and data management. Other PEEP training materials focused on cultural sensitivity, knowledge of the HIV care continuum, proficiency with GET! mobile application, and seamless integration into existing program goals at CART.

Text messaging strategy managed by PEEPs

A standardized schedule of text messages was sent by PEEPs to all participants in the App + PEEP arm two to three times a week for an 8-week period following enrollment. Messages were derived from the U Care 4 Life validated library (HRSA HIV/AIDS Bureau TARGET Center, 2016), engagement hubs feedback, and the YGetIt? study team. Messages focused on building rapport with participants, health education, and facilitation of care (Table 3: Examples of messages). As conversations progressed, messages became tailored to the individual participants. PEEPs were asked to return messages to participants within 24 hours of receipt. Participants had the ability to opt out of receiving messages at any time.

Table 3

PEEP Messenger Examples

Themes	(A) PEEP message to participant	(B) Participant response	(C) Unsolicited participant message to PEEP
Rapport building	“Okay, so it’s Monday but who said Mondays have to suck? Monday can be a fresh start! Is there something your looking to start this week??”	“I started doing the 22 push-up challenge this morning. I’m going to go with that. It’s 22 push-ups a day for 22 days!”	“Hey PEEP, Just checking in with you letting you know all is well. See you Wednesday!”
Emotional support	“Strong. Supportive. Trusting. These are important characteristics of any good relationship and I hope you feel you have that in us! Have a great weekend ”	“I do feel I have that in my PEEP as well as my Northwell team! Thanks ”	“Hi [PEEP]!!! Im alittle nervous over yesterdays visit i expressed to my doctor that iv been bleeding when i go and he recommend i send in a specimen to the labs so see if im cancer free from my colon. Im nervous and i need some support my moms in the hospital and i cant get a hold of her”
Health education	“Do you know about PrEP? It may be something you talk to your partners about.”	“What is PrEP?”	“Can I ask you a question? Is I’m still undeductible”
Care facilitation	“Hey there. How’s your day going so far? Is there anything I can help you with?”	“Hi [PEEP], I hope you had a great holiday weekend. Would you be able to help with booking my next appointment with dr [name]?”	“hey [PEEP] its [client name]. im living at my dads now the adress here [address withheld]this is my permanent adress so any cabs i will be utilizing for transportation can come here to pick me up and drop me off. thank you”
Invitation to event	“I wanted to let you know this Friday we’re having our annual World AIDS Day event and really hope you can join us! I hope to see you Friday, Dec. 1 at 1pm in {address]”	“If i didn’t have work I would’ve! Hope all is well, [PEEP] ”	N/A
Promotion of “Tested” serial comic	“Hey. Have you ‘Tested’ yet? There’s a new season out and the story is amazing. Now it’s easier to scroll through, so go to ‘Tested’ in the menu and get caught up. . .”	“I did get started on the new season of tested! I love how all i have to do now is touch the arrow. Very convenient! Also will the app update show up for me? I remember I had to get the link sent to me to begin last year?. . . . . . It updates on its own I just checked! ”	“I will be sure to take photos! Now to catch up on TESTED!

Note. PEEP = Peer Engagement Educator Professional.

“Tested” comic and social media engagement

Initially created to help keep participants engaged with GET! app, “Tested” is a comic that leads viewers into the world of a diverse group of characters who are affected by HIV, Hepatitis-C, substance use disorders (SUD), and other related social issues. It features bold images, short narratives, and draws from cultural influences to appeal to our priority population (Figure 2). The story is kept contemporary by incorporating current events, fashion trends, and colloquial phrasing. The comic was made available within GET!, on public facing social media platforms Facebook, Instagram, Twitter, the online comic forum Webtoons.com, and our website (YGetIt.org). “Tested” was designed to be an iterative and ongoing serial comic released over time.

An experienced writer and illustrator were tapped to develop the story and images for the digital comic. “Tested” draws inspiration from members of the AI’s Young Adult HIV Consumer Advisory Committee (YACAC), AI staff, focus groups conducted at HIV service organizations throughout NYS, and popular culture. Feedback is also gathered from various events and comments posted on YGetIt? social media sites. The stories are relatable to a wide youth and young adult audience and motivate repeat engagement and viewership.

Phase 8: Pilot – Intervention Implementation

The City University of New York Institute of Implementation Science in Population Health (CUNY ISPH) served as the local evaluators for YGetIt?.

YGetIt? intervention engagement hubs included Northwell Health—CART (Northwell Health), and community-based setting, PFY—Long Island Crisis Center (PFY). Northwell provides comprehensive HIV services to more than 2,400 persons living with HIV (PLWH) in the Long Island region. Northwell Health serves an ethnically and racially diverse urban and suburban population including a sizable number of immigrant and migrant clients. The majority of Northwell Health clients are Ryan White eligible (Federally funded HIV/AIDS treatment and related services provided to people who are uninsured and underinsured), 41% are African American, 14% are Hispanic, and 40% are women. In 2016, approximately 10% of Northwell Health patients aged 18 to 34 years had a detectable viral load. PFY provides supportive services to approximately 1,000 Lesbian Gay Bisexual Transgender Queer (LGBTQ) persons and hosts HIV support groups for PLWH.

Recruitment for YGetIt? program clients began in May 2017. Clients were recruited during scheduled appointments, support groups, and/or other Northwell Health and PFY events. As PFY is not a primary service provider, all recruited PFY clients were referred to Northwell Health. To be eligible to participate in YGetIt?, PLWH had to be between the ages of 18 to 34, read and speak English, own a personal functioning smart phone, live in NYS with no plans to leave for >6 months within the next year, and receive or willing to receive care at Northwell Health. This study was approved by Northwell Health’s Feinstein Institute and University of California Los Angeles Institute Institutional Review Board (IRB), and received an exemption by the City University of New York Integrated IRB.

Upon enrollment, participants completed a comprehensive baseline survey, received a US$50 gift card, and an additional US$50 gift card every 6 months after completing follow-up surveys during an 18-month period. The YGetIt? team maintained consistent communication with the engagement hubs staff through emails, phone calls, and weekly and annual in-person meetings. In addition, engagement hub staff received intervention trainings through in-person presentations with small groups, webinars, and print materials. Table 2 provides a baseline demographic profile of YGetIt? participants who agreed to a 2-year commitment.

Phase 9: Evaluate – Evaluation Design

To evaluate the efficacy and impact of GET! and the PEEP component at improving participant health outcomes, a randomized design was developed. Enrolled participants were individually randomized at a 1:1 ratio to either receive access to GET! plus access to a PEEP (App + PEEP, Arm 1), or to receive access to the app alone (App alone, Arm 2). After 1 year, those randomized to receive the app alone were linked to a PEEP. A total of 113 participants were enrolled in the intervention (53 in the App + PEEP arm and 60 in app alone arm). Primary outcomes examined include engagement in care: For short-term analyses, patients with an HIV care visit in two consecutive 182-day intervals in the first 12 months of follow-up post intervention, with the first and last visit being >91 days apart, were classified as being engaged in HIV care; virologic suppression: For short-term analyses, patients whose most recent VL test during the second half of the first 12 months of follow-up is <200 copies/mL were classified as being virally suppressed. Secondary outcomes include linkage to care: A person newly diagnosed with HIV with evidence of HIV care within 91 days of the date of diagnosis (we also examined 30- and 180-day intervals) was considered linked to care. Outcomes were assessed and compared for those participants in intervention Arm 1 compared with intervention Arm 2. Evaluation is ongoing, and final results will be reported in a future publication.

Data on CD4 values and viral load were collected from CART electronic medical records (EMR). PEEPs also maintained an Excel file tracking contact with participants in-person or through the PEEP messenger in GET!. Demographic data, risk factor assessments (i.e., HIV-related stigma, substance use, mental health, and food security), and engagement in care were collected at baseline and every 6 months thereafter (for up to 18 months) by survey conducted through Audio Computer Assisted Self-Interview Software (ACASI). Finally, backend data from GET! were provided by AppLab containing click counts on YGetIt? content (articles, resources), medications and appointments logged, laboratories entered, and text messaging content from both PEEPs and participants. Interviews were conducted to gather YGetIt? user’s experience and perception of the intervention.

Phase 10: Share – Lessoned Learned

Several development and implementation lessons were learned (Table 4). The following challenges and successes were identified through weekly program staff calls, informal conversations, PEEPs’ and their supervisor’s reports, beta tests and meetings with the AppLab team, and quarterly evaluation data presentations. These findings may be beneficial to other institutions or organizations developing similar interventions.

Table 4

Lessons Learned From Implementation of the YGetIt?

Program implementation key considerations

• When designing a mobile clinical health app, it is important to work with a partner who has experience with HIPAA compliance and privacy security technology.
• Despite extensive beta testing of the new GET! app, glitches will still be found by participants when the app is officially rolled out.
• Staff and organization buy-in can be critical to participant recruitment success, and building relationships with partners is essential.
• Given that interest in apps declines over time, program designers should be aware that interactivity with the intervention may decrease and plans to move participants to the full App + PEEP intervention may be challenging.
• PEEPs/peer navigators selected for an App intervention need to be able to build trust and translate the ease and warmth of an in-person interaction to a digital format.
• Discuss and continuously adapt workload, expectations, and responsibilities of the PEEPS.
• To reach a wider audience for the “Tested” comic, an online comic forum, and not other traditional social media platforms (i.e., Instagram), was found to be the most far reaching avenue for comic dissemination.

Note. HIPAA = Health Insurance Portability and Accountability Act; PEEP = Peer Engagement Educator Professionals.

Recruitment

Recruitment of the users at the community-based organization engagement sites was challenging. The community-based organization clients stated they attended the center to relax and socialize and not to think about health issues; therefore, they were uninterested in enrolling in YGetIt?. Consequently, community-based organization responsibilities were adjusted from recruitment to engagement and promotion of the YGetIt? program and the “Tested” comic. Recruitment at the clinical facility was also initially challenging due to difficulties with staff time management and adherence to the protocol and procedures. PEEPs voiced concerns about the time burden of messaging participants, and tedious data entry which reduced time to recruit and enroll participants. Staff and organization buy-in was found to be essential to the success of the intervention, and spending time building relationships was critical.

Mobile App Development

The development of realistic time frames was required to select desired features, define the overall look and feel of the app, and to beta test. Software glitches slowed YGetIt?’s recruitment and enrollment of participants. App issues were resolved via communication and continuous troubleshooting with the developer and stakeholders. The app’s functionality made it accessible to participants even during connectivity issues (loss of cellular service), as they were able to remain connected via Wi-Fi. GET!’s messenger feature allowed a particular user to re-download the app across three different devices when the previous devices were lost. The GET! messenger allowed a two-way communication between the user and the PEEP to build rapport and facilitate care logistics.

Evaluation Design and App Use Over Time

The evaluation for YGetIt? randomized half of participants to receive only the GET! app (no PEEP) for a year, after which they would receive access to a PEEP. Re-engagement and connection of participants to PEEPs in this arm after the year were very difficult. Some participants, who may not have opened the app in a while or uninstalled the app to make more room on their phones, may have disengaged from the program and were unaware of the PEEP option newly available to them.

Previous studies have suggested that engagement with health-related applications can decline over time (Betthauser et al., 2020; Laurence et al., 2019; Vaghefi & Tulu, 2019) and that was validated by participant usage for GET! during the study period.

Finally, the baseline ACASI survey at enrollment took approximately 45 minutes to complete. Some participants expressed frustrations with the length of the survey which posed a barrier to enrollment and follow-up. The enrollment process and survey completion prolonged the time they had to spend at the health care center. The subsequent 6-, 12-, and 18-month surveys were designed to be shorter.

PEEPs

It became evident that PEEPs who were tech savvy, representative of the priority population, and had shared life experience with participants were most impactful. For the YGetIt? intervention, we selected a peer navigator and a health educator. However, other health care staff such as case managers, and nurses may be equally as effective. It is crucial to find someone who is able to build trust and translate the ease and warmth of an in-person interaction to a digital format.

The YGetIt? program also initially experienced high PEEP turnover. PEEPs were challenged by YGetIt?’s strict data collection, and recruitment and messaging protocol. In response, adjustments to the messaging schedule were made, and data collection refresher trainings were conducted.

Dissemination of “Tested”

“Tested” was designed as an engagement tool to keep all participants interested in the GET! app, but was not well utilized by YGetIt? participants. “Tested” was expanded to a wider audience through various digital platforms (i.e., Webtoons, Instagram, Facebook, Twitter, www.ygetit.org).

Conclusion

This article summarizes the development of a mHealth, peer navigation-based intervention, baseline participant characteristics, the evaluation design, and the challenges experienced. The intervention and lessons learned may be helpful to organizations interested in developing and implementing a similar digital approach. GET! is a communication and engagement tool that supports HIV care. YGetIt?’s, “Tested,” and GET! mobile application features could be adapted to various disease processes (Hepatitis-C, SUD, STIs, diabetes, etc.). More novel mHealth and social media-based interventions may improve timely entry, engagement, and retention in care for priority populations. The YGI platform offers a structured digital approach with components that can be tailored to address a broad range of health issues and customized for the desired population. Working tools like YGI into standard practice may give providers another viable option to maintain connections to patients and improve health outcomes. As a matter of policy, health care must move toward delivery of services in digital spaces, as an increasing number of people are accustomed to and now expect this transformation.

Footnotes

Authors’ Note:

This publication was made possible by award number H97HA28890 from the U. S. Department of Health and Human Services, Health Resources and Services Administration (HRSA), HIV/AIDS Bureau’s Special Projects of National Significance Program (SPNS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the government.

ORCID iD

Beatrice Aladin

References

Alicke

M. D.

Zell

Guenther

C. L.

(2013). Chapter Four - Social Self-Analysis: Constructing, Protecting, and Enhancing the Self. In Olson

J. M.

Zanna

M. P.

(Eds.), Advances in Experimental Social Psychology (Vol. 48, pp. 173–234). Academic Press. 387. https://doi.org/10.1016/B978-0-12-407188-9.00004-1

Atreja

Khan

Rogers

J. D.

Otobo

Patel

N. P.

Ullman

. . .Sands

B. E.

(2015). Impact of the mobile HealthPROMISE platform on the quality of care and quality of life in patients with inflammatory bowel disease: Study protocol of a pragmatic randomized controlled trial. JMIR Research Protocols, 4(1), Article e23. https://doi.org/10.2196/resprot.4042

Bandura

. (2001). Social cognitive theory: an agentic perspective. Annu Rev Psychol, 52,1–26. https://doi.org/10.1146/annurev.psych.52.1.1.PMID: 11148297.

Barak

Fisher

W. A.

(2003). Experience with an Internet-based, theoretically grounded educational resource for the promotion of sexual and reproductive health. Sexual and Relationship Therapy, 18(3), 293–308. https://doi.org/10.1080/1468199031000153928

Betthauser

L. M.

Stearns-Yoder

K. A.

McGarity

Smith

Place

Brenner

L. A.

(2020). Mobile app for mental health monitoring and clinical outreach in veterans: Mixed methods feasibility and acceptability study. Journal of Medical Internet Research, 22(8), Article e15506. https://doi.org/10.2196/15506

Brooks

R. A.

(2020). Qualitative evaluation of social media and mobile technology interventions designed to improve HIV health outcomes for youth and young adults living with HIV: A HRSA SPNS initiative. Health Promotion Practice, 21(5), 693–704. https://doi.org/10.1177/1524839920938704

Cao

Gupta

Wang

Hightow-Weidman

L. B.

Muessig

K. E.

Tang

Pan

Pendse

Tucker

J. D.

(2017). Social Media Interventions to Promote HIV Testing, Linkage, Adherence, and Retention: Systematic Review and Meta-Analysis. Journal of Medical Internet Research, 19(11), e7997. https://doi.org/10.2196/jmir.7997

Centers for Disease Control. (2019). Estimated HIV incidence and prevalence in the United States, 2010–2016. http://www.cdc.gov/hiv/library/reports/hiv-surveillance.html

Glanz

Rimer

B. K.

Viswanath

(2008). The scope of health behavior and health education. In Glanz

Rimer

B. K.

Viswanath

(Eds.), Health behavior and health education: Theory, research, and practice (pp. 3–22). Jossey-Bass.

10.

Laurence

Wispelwey

Flickinger

T. E.

Grabowski

Waldman

A. L.

Plews-Ogan

Debolt

Reynolds

Cohn

Ingersoll

Dillingham

(2019). Development of PositiveLinks: A mobile phone app to promote linkage and retention in care for people with HIV. JMIR Formative Research, 3(1), Article e11578. https://doi.org/10.2196/11578

11.

Lemley

S. M.

Klausner

J. D.

Young

S. D.

Stafylis

Mulatya

Oden

Xie

Revoredo

Shmueli-Blumberg

Hichborn

McKelle

Moran

Jacobs

Marsch

L. A.

(2020). Comparing Web-Based Platforms for Promoting HIV Self- Testing and Pre-Exposure Prophylaxis Uptake in High-Risk Men Who Have Sex With Men: Protocol for a Longitudinal Cohort Study. JMIR Research Protocols, 9(10), 445 e20417. https://doi.org/10.2196/20417

12.

Mummah

S. A.

Robinson

T. N.

King

A. C.

Gardner

C. D.

Sutton

(2016). IDEAS (Integrate, Design, Assess, and Share): A Framework and Toolkit of Strategies for the Development of More Effective Digital Interventions to Change Health Behavior. Journal of Medical Internet Research, 18(12), e317. https://doi.org/10.2196/jmir.5927

13.

New York State Department of Health AIDS Institute. (2019). New York State HIV AIDS annual surveillance report: For person diagnosed through December 2018. https://www.health.ny.gov/diseases/aids/general/statistics/annual/2018/2018_annual_surveillance_report.pdf

14.

New York State Department of Health, AIDS Institute, & Digital Health Initiative. (2021). Tested (Vol. 2021). https://www.webtoons.com/en/challenge/tested/list?title_no=231173&page=1

15.

Pasipanodya

E. C.

Montoya

J. L.

Watson

C. W.-M.

Marquine

M. J.

Hoenigl

Garcia

Kua

Gant

Trambley

Moore

D. J.

(2020). Tailoring a mobile health text- messaging intervention to promote antiretroviral therapy adherence among African Americans: A qualitative study. PLoS ONE, 15(6), e0233217.

16.

Roland

K. B.

Higa

D. H.

Leighton

C. A.

Mizuno

DeLuca

J. B.

Koenig

L. J.

(2020). HIV Patient Navigation in the United States: A Qualitative Meta-Synthesis of Navigators’ Experiences. Health Promotion Practice, 1524839920982603. 469. https://doi.org/10.1177/1524839920982603

17.

Sarango

de Groot

Hirschi

Umeh

C. A.

Rajabiun

(2017). The Role of Patient Navigators in Building a Medical Home for Multiply Diagnosed HIV-Positive Homeless Populations. Journal of Public Health Management and Practice, 23(3), 276–282.

18.

Schaeffer

(2019). Most U.S. teens who use cellphones do it to pass time, connect with others, learn new things. Pew Research Center. https://www.pewresearch.org/fact-tank/2019/08/23/most-u-s-teens-who-use-cellphones-do-it-to-pass-time-connect-with-others-learn-new-things/

19.

Shimazaki

Matsushita

Iio

Takenaka

(2018). Use of health promotion manga to encourage physical activity and healthy eating in Japanese patients with metabolic syndrome: a case study. Archives of Public Health, 76(1). https://doi.org/10.1186/s13690-018-0273-5

20.

Vaghefi

Tulu

(2019). The continued use of mobile health apps: Insights from a longitudinal study. JMIR mHealth and uHealth, 7(8), Article e12983. https://doi.org/10.2196/12983