Abstract
Previous studies have recognized the potential of mobile technology to improve health outcomes among underserved populations, but the challenges in conducting research into the use of mobile technology to improve health (mHealth) are not well understood. This manuscript identifies some of the most important challenges in conducting mHealth research with a sample of urban American Indian and Alaska Native mothers. We examined these challenges through an existing partnership with a community health agency. We conducted community consultations and a process monitoring phase for a pilot trial aimed at measuring the effect of a brief counselling session on participants’ adherence to use of a mobile app. We identify generalizable challenges in administrative, technological, and logistical domains that will be useful foreknowledge to other investigators planning to conduct mHealth research with underserved populations.
Keywords
Introduction
The use of mobile technology to improve health (mHealth) holds great promise, particularly for populations with limited access to healthcare but widespread ownership of mobile phones. 1 Recent research has recognized the potential of mHealth to reduce health disparities and improve health outcomes in underserved populations. 2 However, the difficulties inherent in implementing mHealth interventions with these populations are poorly understood. Which devices and features are best suited to achieve specific goals? What knowledge and experience with technology can researchers assume in study participants? Extant literature shows that mHealth research faces challenges related to the gap between research needs and the requirements of commercial technology. 3 This manuscript aims to offer preliminary lessons learned during the opening phases of an ongoing randomized clinical trial (RCT) that can be generalized to other studies interested in conducting mHealth research with underserved populations.
Our trial uses a popular smartphone application to promote weight reduction in an urban sample of American Indian and Alaska Native (AI/AN) women. Obesity reduces life expectancy and is a known risk factor for diabetes, atherosclerosis, hypertension, heart disease, and stroke. 4 Current data show that AI/AN communities have higher levels of overweight and obesity than the all-races U.S. population, 5 with obesity prevalence as high as 87% in some tribes. 6 According to 2011 data from the Behavioral Risk Factor Surveillance System,7,8 51.3% of AI/AN women are either obese or overweight, compared to 41.9% of non-Hispanic White women. We chose to focus on AI/AN women because they are traditionally responsible for purchasing and preparing food for their families. Hence, targeting obesity in AI/AN mothers is expected to help improve health outcomes at the household level. 9
Previous work has shown that mHealth interventions for weight loss can reduce dietary energy intake 10 and increase physical activity levels. 11 The success of such interventions depends critically on ensuring that participants adhere to the use of a specific weight management application.11,12 These findings encouraged us to design an intervention to increase adherence to the use of Myfitnesspal, which recent research has shown to be the most popular smartphone application and companion website for daily tracking of caloric input and energy expenditure. 13 Our study design includes a small RCT to examine whether a brief session of motivational interviewing can improve adherence to Myfitnesspal in a sample of overweight or obese AI/AN women in Washington State. Motivational interviewing is a patient-centred approach that focuses on the patient’s own reasons for committing to behaviour change. For underserved populations, such as AI/ANs, it has been found to enhance adherence to behavioural interventions for weight management,14,15 and to support lifestyle and behaviour changes that reduce cardiovascular disease risk, tobacco use, and alcohol consumption.16–18
The goal of this article is to share some of the challenges and lessons learned from the community consultation and process monitoring phases of our ongoing RCT. We offer several considerations for researchers who contemplate mHealth studies with underserved urban populations, along with approaches to overcoming the challenges we encountered. This research was possible thanks to the longstanding collaboration between a research centre specializing in AI/AN health at the University of Washington, and the Seattle Indian Health Board, the largest Urban Indian Health Organization in the country. Both organizations aim to improve health outcomes in AI/ANs and reduce health disparities between AI/ANs and other racial and ethnic groups.
mHealth for AI/ANs
Little information is available on the use of new technologies by AI/ANs. Extant literature refers only to mobile phones, not smartphones. We define smartphones as mobile phones with advanced computing capabilities, including the ability to access the Internet and run applications. In 2008, the Corporation for Public Broadcasting surveyed 196 AI/ANs from 120 tribes across 28 states to assess their use of new media, including the Internet and mobile phones. No data were collected on smartphones, since their market share at the time was tiny. Survey results were compared to those from similar surveys of the U.S. general population conducted by the Pew Foundation during the same period. Surprisingly, more AI/AN respondents owned a mobile phone than did respondents from the general population (93.4% versus 78%), and more AI/ANs regularly used mobile phones to send and receive text messages (73.5% versus 41%). 19 However, the authors cautioned that their results were not representative of the overall Native population, as the AI/AN survey was limited to ‘selected participants’. Although the report did not provide a rural/urban breakdown of the participant sample, its content was limited to rural issues.
Recent data from the Pew survey indicated that mobile phone ownership among the general population has increased (90% in 2014 versus 78% in 2008), and included data on smartphone ownership (64% in 2015), but provided no information about AI/ANs specifically. 20 A 2011 study that recruited AI/AN youth from both urban and tribal communities in the Pacific Northwest also found that these youth were equally or more likely than mainstream youth to use the Internet to find health information. 21 However, this study did not examine the use of smartphones.
Since 2008, access to Web-based technology in the U.S. has increased rapidly, 22 with rates of Internet traffic continuing to double every year since 1997. 23 We expected that smartphone ownership would be common in an urban sample of AI/AN mothers, given studies showing that urban residents have relatively wide access to all types of digital technology, regardless of race or ethnicity. 24 We also assumed that urban AI/ANs in 2014 would be at least as well-connected as the reservation residents surveyed in 2008. 19 Since we recruited participants in Seattle, home of many leading technology companies, 25 we believed that the availability of smartphones and Internet access was unlikely to present barriers to using an application like Myfitnesspal.
Study phases
The opening phases of our study included community consultations and process monitoring. Both phases were conducted between November 2013 and March 2014 at the Seattle Indian Health Board. Members of the Seattle Indian Health Board informed this research in three meaningful ways. First, they assisted in the cultural tailoring of the intervention. Second, they facilitated the recruitment of study participants and confirmed their eligibility. Third, they contributed to the data collection and data analysis during the focus groups and the process monitoring phase. All study procedures were approved by the University of Washington Institutional Review Board and the Seattle Indian Health Board Privacy Committee.
Community consultation phase
Our approach to the community consultations was consistent with the philosophical orientation of grounded theory.26,27 We had no underlying theoretical structure other than the pragmatic position of seeking to determine the clarity of our measurement tools, and acceptability of the proposed design and procedures through community consultations. The only methodological assumption we had was that participants with the same characteristics as those to be later recruited for the RCT would be best suited to provide information about the clarity and acceptability of our study design and procedures.
Over a two-month period, we conducted six community consultation meetings with a total of 17 AI/AN women. Each community consultation included a complete review of the design and purpose of the RCT, as well as all devices and data collection methods that we planned to use. 28 The number of participants ranged from two to four per meeting. We explained the rationale and structure of the intervention and previewed all questionnaire items, which include the weight related symptom measure, 29 overweight and weight loss quality of life, 29 and the general nutrition knowledge questionnaire. 30 We also presented a tutorial on Myfitnesspal and demonstrated a brief session of motivational interviewing. We made smartphones available during each meeting to test the usability of the application. We also described our randomization procedures, explained our outcome measures, and offered details on participant compensation. We were especially interested in participants’ opinions of the tutorial, which was designed to introduce them to Myfitnesspal, and their feedback on our plan to distribute smartphones to participants who did not have one. We also wanted to see how willing they were to participate in a randomized trial, and how comprehensible they found our proposed outcome measures. These include adherence to Myfitnesspal, changes in BMI, and responses to questionnaire items.29,30 Our community consultations showed that our plan to use mHealth for weight management was adequate. Participants expressed interest in the study, agreed with our randomization procedure, proposed rephrasing one item of the questionnaire, and requested that a female researcher conducted all motivational interviewing sessions.
Process monitoring phase
After completing the community consultations, we enrolled 10 participants in our process monitoring phase. Our goal was to test and troubleshoot all data collection and intervention methods planned for the RCT. Testing included collection of baseline data from our pilot sample for all proposed outcome measures by using Assessment Center. 31 As in the full-scale RCT, we randomized our 10 pilot participants to receive either a basic tutorial on Myfitnesspal or a basic tutorial plus a brief session of motivational interviewing. We provided smartphones to participants who did not have a mobile phone capable of supporting Myfitnesspal. Since the primary outcome measure of the RCT will be adherence to Myfitnesspal for daily logging of food intake and physical activity, we had to ensure that all participants had compatible devices.
RCT design and execution
These preparatory activities informed the conduct of a small (N = 50) RCT which was conducted between October 2013 and October 2014. We were able to complete follow-up data collection with 87% of the participants. It is also of note that approximately 80% of the phones which were loaned to participants in the study were returned. The data from this RCT are undergoing analysis and preparation for publication at the time of this writing. The primary aim of these analyses will be to determine whether using brief motivational interviewing as an intervention can affect adherence to the use of Myfitnesspal for daily logging of food intake and physical activity. These findings will be reported in a future publication.
Lessons learned
Participants might not have their own smartphones
Nearly all the participants in our process monitoring phase owned a mobile phone, but only one already had her own smartphone. Extant literature had encouraged the erroneous assumption that smartphone uptake would parallel mobile phone uptake among AI/ANs. 19 Since our preliminary budget anticipated lending phones to no more than half of the participants, we redistributed funds so that we could provide a smartphone to every participant who might need one, and ensure consistent access to Myfitnesspal.
We could not simply give our participants a smartphone
While planning our RCT, we expected to simply give participants a smartphone if they didn’t have one. However, the University of Washington Institutional Review Board did not allow us to make a gift of study smartphones, for three highly pragmatic reasons. First, the cost of maintaining service on each line after the intervention ended would fall to the participants themselves, placing an unacceptable burden on them as a consequence of study participation. Second, if participants wanted to keep the smartphones for their own use after the intervention finished, we would have to perform a legal transfer of ownership. Unfortunately, the terms of smartphone service obtained by the University through negotiation with vendors were not transferable to individual study participants. Third, smartphones can be used for illegal activities, and the Institutional Review Board raised questions about potential liability in such cases. For these reasons, we decided only to lend smartphones to participants for two months during active data collection, with the agreement that the phones would be returned to us at follow-up.
Lost or stolen smartphones raise sticky issues
Even without an outright gift of smartphones to study participants, our lending strategy required difficult decisions. What if a participant’s phone was lost or stolen? How could she continue in the study? Who might be using the missing phone, and for what purpose? In response, we devised a ‘lost phone’ protocol to prevent unexpected expenses and avoid liability for potential criminal activity. First, if a phone is reported as lost or stolen, we would replace the phone only once per participant, and we would repeat the consent process on issuing the second phone. Second, immediately after learning about a lost phone, we would tell the vendor to place it in ‘suspend’ mode for 30 days, preventing further use. If the phone was not reactivated during that period, it would be terminated from the parent service contract.
If possible, purchase all smartphones at the same time, with the same specifications, and the same service plan
Purchasing smartphones for different users at different times presents manifold challenges. Service plans on offer by vendors can change from month to month, so investigators who are constrained to stagger their purchases are likely to end up with a range of device types, plans, and costs among study participants. This situation can make expenses difficult to project over the lifetime of a study. Nevertheless, we understand that the structure of many federal funding mechanisms would leave insufficient funds to make all necessary purchases during any single project year.
Research participants do not enrol all at once
If study devices are purchased in bulk, the project faces the additional expense of maintaining several accounts that are not in active use, but are still charged a monthly service fee. This conundrum stems from the fact that vendors are reluctant to sell smartphones that have not been activated. Their rationale is that they often sell devices at a loss, trusting that they will recoup their expense through the associated service contracts. The solution we found for our study was to place devices in ‘suspended mode’ on receipt to avoid service fees. This approach is only feasible if investigators can activate smartphones in real time, when participants are actually recruited. If service providers’ customer care centres cannot provide this level of responsiveness, our workaround would fail.
Smartphone applications change much faster than traditional research instruments
Smartphone application developers are in the business of constant product improvement. 32 For this reason, all tutorials and other informational materials need to be updated regularly to reflect the current version of any application used in a research study. New iterations are likely to change not only the interface design and the application features, but also the data available for collection. This prospect is especially concerning for researchers who plan to use a passive method of data collection, such as observing participants’ newsfeeds, or in our case, noting how often participants log their dietary intake and physical activity levels or how often they use Myfitnesspal. When application features are reconfigured or eliminated, data collection can become difficult, unpredictable, or even impossible. A given application might not function in the same way at the end of a study as it did at the beginning. Therefore, investigators should work closely with their ethical review boards to ensure that changes in study procedures and materials can be incorporated into consent forms in a timely fashion. Whenever possible, they should also work with application developers to anticipate potential changes throughout the study.
Conclusion
We have presented a few of the challenges that we met in the early stages of our ongoing study, which uses mHealth to encourage weight reduction in overweight and obese AI/AN mothers. Many of the same challenges are likely to arise during smartphone interventions with underserved populations, since they result from the mismatch between standard research procedures and the business practices of smartphone service providers. Our goal in the present work is to help other researchers anticipate the logistical challenges associated with managing devices and their associated accounts. Notably, we found that the solution to one problem often leads to another problem. Therefore, we offer these lessons learned in the hope that other investigators can benefit from our experience and successfully respond to the challenges of using mHealth with underserved populations. We also hope that mHealth applications continue to be rigorously developed and evaluated to improve health outcomes in the years to come.
Footnotes
Acknowledgements
We thank Raymond Harris, PhD, for assisting with manuscript preparation and Catherine Stockdale, for assisting with participant recruitment and study logistics.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support for this research was provided by a grant from the Agency for Healthcare Research and Quality (AHRQ 1K12HS021686) to the Department of Pharmacy in the School of Public Health at the University of Washington, and by a grant from the National Institute of Mental Health (T32 MH082709) to the Department of Psychiatry and Behavioral Sciences at the University of Washington.