Mobile phone–based clinical guidance for rural health providers in India

Abstract

There are few tried and tested mobile technology applications to enhance and standardize the quality of health care by frontline rural health providers in low-resource settings. We developed a media-rich, mobile phone–based clinical guidance system for management of fevers, diarrhoeas and respiratory problems by rural health providers. Using a randomized control design, we field tested this application with 16 rural health providers and 128 patients at two rural/tribal sites in Tamil Nadu, Southern India. Protocol compliance for both groups, phone usability, acceptability and patient feedback for the experimental group were evaluated. Linear mixed-model analyses showed statistically significant improvements in protocol compliance in the experimental group. Usability and acceptability among patients and rural health providers were very high. Our results indicate that mobile phone–based, media-rich procedural guidance applications have significant potential for achieving consistently standardized quality of care by diverse frontline rural health providers, with patient acceptance.

Keywords

Clinical decision-making decision-support systems health-care service innovation and information technology mobile health primary care

Introduction

In developing countries, several types of primary health workers with varying levels of literacy and training provide health care at first contact to the vast majority of populations,¹ but they may lack essential health information on how to diagnose and manage common diseases according to existing protocols and guidelines.^2,3 Health-care providers’ lack of adherence to standardized evidence-based clinical guidelines is widely regarded as a barrier to improve health outcomes.⁴ The World Health Organization’s⁵ (WHO) Integrated Management of Childhood Illnesses (IMCI) is one example of standardized guidelines available for primary health-care workers. However, even with standard guidelines in place, primary health-care workers may lack the training and motivation necessary to successfully apply these guidelines.⁶

The need for innovation is crucial, especially in lower and middle-income countries, in order to reduce disparities in health-care access, quality and resource distribution.⁷ Mobile health (mHealth) has already shown the potential to overcome many of these obstacles;⁸ it is now a widely accepted approach to transform health-care delivery in resource-poor countries.^9,10 Smartphones are one of the most powerful tools that can be used to enhance access to quality health care in rural settings.¹¹ However, most smartphones depend on Internet connectivity to be ‘smart’, and a system that is independent of Internet connectivity would be preferable for rural settings.¹² A clinical practice guideline when presented in a structured algorithmic fashion improves standardization of care.¹³ Hence, it appears that smartphones in conjunction with clinical guidelines like the IMCI presented as media-rich algorithms can be potent tools for frontline primary health-care workers. Here, ‘media-rich’ refers to presenting information with audio (speech), images and videos in order to overcome literacy and training deficits among both health workers and the patients they treat. For brevity, we shall refer to such instantiations of clinical guidelines as mobile media-rich interactive guidelines (mMRIGs) in the remainder of this article.

One such mHealth project studied the use of clinical practice guidelines presented in an interactive, structured, step-by-step guidance form, accompanied by instructions in audio, text and images/video on mobile phones to community health workers (CHWs) in a laboratory setting in Colombia.¹⁴ Use of mMRIGs improved the performance of the CHWs by decreasing errors in the performance of diagnostic/treatment medical tasks, and increasing protocol adherence, leading to high acceptance and intention to use. This system was shown to enable incorporation of principles of persuasive technology.^15–17 Use of systems incorporating persuasive technologies, especially in the context of rural health, can greatly enhance health workers’ self-efficacy and the confidence needed to provide high-quality client counselling on essential health-related knowledge, attitudes and practices.^18,19

This study was set up to field test such a system in a real-time field setting in two rural locations in Southern India. The study objectives were to (a) use a randomized control design to measure changes in protocol compliance (PC) by health workers in their everyday work settings, (b) assess the usability and acceptability of the mobile application with health workers in the experimental group and (c) obtain patient feedback on health workers’ use of the mobile system during treatment.

Methods

Study settings and samples

A total of 16 willing and consenting rural health providers (RHPs) were drawn into the study from two field sites in two neighbouring districts, Sivagangai and Dharmapuri, in rural Tamil Nadu, a South Indian state. At one of these sites – Thirupathur in district Sivagangai – the RHPs consisted of eight independent informal health-care providers based in villages (all male). These providers had been part of a previous technology-enabled distance learning project developed by one of the co-authors. The other group of RHPs consisted of eight female community health auxiliaries linked with a non-governmental hospital (Tribal Health Initiative, or THI, hospital) in a tribal area called Sittilingi in the district of Dharmapuri. While the informal providers in Thirupathur had been practising for more than 10 years on average, THI’s community health auxiliaries had been recruited less than a year prior to this study and all were undergoing training by the hospital. These two different groups of RHPs – one, a group of independent practitioners, and another, a group that was directly linked to a formal hospital set up – were selected because we assumed that they would provide us with valuable additional lessons on the contribution of local contexts in the way our application was finally used and its sustainability and success.

In each location, four RHPs were randomly assigned to the experimental group and four to the control group; thus, there were eight RHPs in each group. All 16 RHPs underwent a 2-day training programme in guideline-based care and in use of the mHealth system by physicians who spoke the local language. We assessed all RHPs on their understanding of guideline-based care using a simple assessment instrument developed by the research team, on which every RHP received a pre- and post-training test score (PTTS). This PTTS initially was to be used as a covariate, but was not, because mixed-model and regression analyses showed that the PTTS was not significantly (α = .05) correlated with PC. At the end of the training, we randomly assigned half the RHPs to the control group and other half to an experimental group. We repeated the explanation about the study design and further explained that the ones in the control group would be given their data cards with the application after the field testing was over. Thus, after the training, the RHPs in the experimental group went back to their field sites with the applications on their mobile phones, while for the ones in the control group, the data card with the application was taken out and they were given only the phone plus a set of paper guidelines to use in the field. (The application was installed on their phones after the field testing was over.)

Technology and phone

A system called guideVue® was used for developing mMRIGs and deploying them on cell phones. Details of the system can be found elsewhere.^14,15 This system enables development and deployment of mMRIGs without programming. Figure 1 shows a sample screen from an mMRIG developed for our study.

Figure 1.

Rich-media guidelines (IMCI) in Tamil language on Windows Mobile 6.5 cell phone.

Development of the mMRIGs for field testing

Our team of three physicians and a public health specialist adapted two validated clinical guidelines for implementation as mMRIGs, deployed on Windows Mobile 6.5 cell phones. These were the WHO’s well-known IMCI and Integrated Management of Adult Illnesses (IMAI). With a focus on three conditions – fever, diarrhoea and respiratory problems – the available WHO’s guidelines were customized to the local Indian settings by our team of senior and experienced physicians and then converted into interactive, structured, rich-media clinical algorithms in Tamil language using the guideVue system (see Figure 1).

Field test and data collection

Two months after the first training, all RHPs were evaluated in their field settings using several methods and instruments. All study instruments were administered by four trained study fieldworkers, recruited with the help of a local partner organisation. All fieldworkers had a social work background and prior experience of working on community health–related projects. The field research team was supervised by two field managers, qualified in social work and nursing, respectively, and with prior experience of qualitative field research methods in health services research, including patient–provider observations. The principal investigator (first author (M.G.)), along with the supervisors, first trained the field investigators in a 2-day classroom training, and thereafter, the supervisors provided further supervised practice in the field, before the research observations were formally initiated.

Data relating to PC, acceptability and usability of the phone and patient feedback were collected as described in the following. Usability and acceptability of the mHealth system by the RHPs was assessed by means of a structured questionnaire, with some open-ended questions (only with the experimental group of RHPs). PC was evaluated by direct observations of the RHPs’ performance as they were treating the patients. Trained field investigators observed each RHP managing eight patients comprising an ideal mix of four paediatric and four adult patients (a total of 128 across the 16 RHPs). Patients were recruited as they presented spontaneously at the RHP clinics and consented to participate in the study. Patients’ reactions to their RHP’s use of the mobile system was assessed by means of an exit interview questionnaire.

Analysis

All data collected were entered into Microsoft Excel spreadsheets. Basic data analysis including simple statistics and histograms were developed using Microsoft Excel. The SPSS 19 MIXED procedure was used to perform multilevel linear model and repeated measures analyses.

PC was measured by the following simple formula that measures the percentage of guideline steps that were actually performed, to the number of steps that are specified in the guideline and are relevant for that patient

Protocol compliance = \frac{Number of guideline steps actually performed}{Number of steps specified in the guideline for the specific patient and medical condition}

The study design was reviewed by a local ethical committee comprising two medical doctors from the All India Institute of Medical Sciences and a social scientist in Delhi. Ethical approval was obtained on 16 March 2011 from this committee.

Results

This study included 16 RHPs divided equally between 8 male practitioners (in Thirupathur, Sivagangai) and 8 female health auxiliaries (in Sittilingi, Dharmapuri). The male practitioners were older, with a mean age of 48.75 years. They had been working as independent health-care providers for more than 10 years out of solo clinics in different villages. All had studied up to class 10 or more. Two held diplomas in laboratory technology and one in pharmacology; the rest had learned mainly as doctors’ apprentices. The women auxiliaries attached to the tribal hospital were younger, with a mean age of 24.75 years. Five had studied up to class 10 or more, and three had completed secondary school. Except for two senior RHPs who had been working with the hospital for several years, the remaining six had been recruited about a year ago and had been undergoing training by the hospital staff over the last 9 months prior to the study.

Pre–post training scores

At the end of the 2-day training on use of the mobile phone to understand protocol-based health-care delivery, all RHPs showed an increase in knowledge as measured through the pre–post training questionnaire. However, while there was no difference in the mean pre-training scores of the two groups, the control group scored significantly higher than the experimental group (mean scores: control group, 13.68; experimental group, 9.51; p < .05) in the post-training evaluation (see Table 1).

Table 1.

Pre- and post-training scores obtained by the control and experimental groups.

Scores	Control group	Experimental group	t-test p values
	Score (SD)	Score (SD)	t-test p values
Mean pre-training score (maximum score = 19)	8.58 (2.03)	7.01 (1.85)	0.193
Mean post-training score (maximum score = 19)	13.68 (2.17)	9.51 (2.48)	<0.05

SD: standard deviation.

Usability of the mobile system

The first three histograms below relate to RHPs’ reactions to use of the system (Figures 2 to 4). More than 85 per cent RHPs said they were comfortable with the system and found it easy to use; 75 per cent said it helped them follow standardized guidelines better and more consistently than before. Table 5 summarizes RHPs’ responses to other usability-related questions. We note that all of the RHPs would like to have more conditions presented in the system and also that without the system, they would not be able to remember and perform all steps. This last result indicates that without this kind of job-aid, RHPs felt they could not be fully protocol compliant. The majority (87.5%) of RHPs also agreed that they would recommend the system to friends and colleagues, continue using it, and that the combination of rich media was indeed useful. Figures 5 and 6 depict the patients’ reactions to the use of the mobile phone–based system by their providers. The majority of patients said it increased their confidence and respect in their RHPs. This is significant for future adoption of mMRIG systems in the rural health context. If, on the contrary, patients had reported decreased respect and confidence, RHPs would not continue use of mHealth tools.

Figure 2.

RHPs’ level of comfort using the mHealth system.

Figure 3.

RHPs’ ease of use with the mHealth system.

Figure 4.

Use of mHealth system improves RHPs’ ability to deliver guideline-based care.

Figure 5.

Patients confidence in their RHP enhanced by RHPs’ use of mHealth system.

Figure 6.

Patients reported increased respect of their RHP due to RHPs’ use of mHealth System.

Protocol compliance

To study PC, trained field investigators observed all 16 RHPs as they treated 126 adult and paediatric patients. Although the target was 128 patients, finally 126 patients were recruited for reasons beyond control, and it was not always possible to get a perfect mix of four adult and four paediatric patients per RHP. Thus, out of 126 patients seen by all 16 RHPs, there were 57 paediatric patients (45.2%) and 69 (54.8%) adult patients. A paediatric patient was defined as someone whose age was less than or equal to 10 years. With respect to patients’ gender, 45 (35.7%) were male and 81 (64.3%) were female.

Tables 2 and 3 show the further breakdown of patients categorized by age and sex in the experimental and control groups. While patients were more or less equally distributed across the two groups with respect to age, with respect to gender, there were more male patients in the experimental group (60%) than in the control group (40%). Female patients were distributed more equally.

Table 2.

Patients’ gender categories across the experimental and control groups.

Patient gender	Experimental group	Control group	Total
Males	27 (60%)	18 (40%)	45 (35.7%)
Females	38 (47%)	43 (53%)	81 (64.3%)
Total	65 (51.6%)	61 (48.4%)	126 (100%)

Table 3.

Patients’ age categories across the rural health provider groups.

Patient age category	Experimental group	Control group	Total
Paediatric	30 (52.6%)	27 (47.4%)	7 (45.2%)
Adult	35 (50.7%)	34 (49.3%)	69 (54.8%)
Total	65 (51.6%)	61 (48.4%)	126 (100%)

Mean PC was calculated for the experimental and control groups across a variety of stratifications of interest, including patients’ gender and age categories (paediatric vs adult) and study locations

Figures 7 and 8 and Table 4 show that experimental RHPs’ mean PC initially exceeded control RHPs’ by 26.80 percentage points for males (p = .003) and 11.95 for females (14.86 less, p = .12). The treatment by patient number interaction displayed shows that as additional patients were served, experimental RHPs’ initial superiority decreased, as experimental RHPs’ mean PC increased at a rate of 2.65 (p = .042) less per patient than the 4.26 (p < .001) rate for control RHPs. The treatment by patient sex interaction (p = .027), graphed in Figure 9, shows control RHPs’ 12.75 (p = .01) patient sex difference (female minus male) in PC means was 14.86 more than experimental RHPs’, and experimental RHPs’ PC mean superiority over controls was 14.86 greater for male than female patients. Put another way, experimental RHPs’ mean PC was (significantly for male patients) superior to, and significantly more consistent across genders than, control RHPs’. Finally, RHPs’ mean PC for adults significantly exceeded children’s (by 28.14, p < .001). Location PC means did not differ significantly (p = .611).

Figure 7.

Estimated protocol compliance means for the significant treatment by patient number interaction for males.

Figure 8.

Estimated protocol compliance means for the significant treatment by patient number interaction for females.

Table 4.

Estimates of provider protocol compliance fixed predictor effect.

Parameter	Estimate	Standard error	p
Intercept	24.78	7.21	0.001
Patient number	4.26	0.91	0.000
Patient age (years)	28.14	3.12	0.000
Patient sex	12.75	4.88	0.010
Location (Thirupathur vs Sittilingi)	−2.96	5.70	0.611
Intervention	26.80	8.71	0.003
Intervention × patient number	−2.65	1.29	0.042
Intervention × patient sex	−14.26	6.62	0.027

Figure 9.

Estimated protocol compliance means for the significant treatment by sex interaction.

From RHPs’ comments on usability of the system, we also identified a few problematic areas of use. These included difficulties in navigating through the largely English phone menu (before entering into the Tamil mMRIG application), difficulties with some letters of the available Tamil font that was used in the system, the time required for navigating through some of the screens which had heavier visual files and the time required for entering patient data on the first screen. These are problem areas that will need to be refined in the next version of the system.

Discussion

This article describes a randomized control study to evaluate the practical application of an mHealth system by village-based RHPs at two sites in rural Southern India. PC was calculated across a mix of adult and paediatric cases – a total of eight per RHP, for both the experimental and control groups. Even though the control group scored significantly higher immediately after the 2-day training on the mHealth system and content, 2 months later, the experimental groups at both sites exhibited higher PC, significantly so for initial male patients.

The overall emerging picture was that, statistically controlling for all other predictors, experimental group RHPs’ PC exceeded that of control RHPs, significantly so (by 26.8 percentage points) for initial male patients. The experimental group RHPs’ PC exhibited significantly greater consistency and at a higher level across patient order and across sexes than control RHPs’. After training, experimental RHPs started at high PC levels and exhibited consistent improvements. Their initially sizable PC superiority for both male and female patients diminished, as RHPs continued to serve patients. Finally, RHP PC for adult patients substantially exceeded that for children.

The increase in PC across patient order exhibited by control RHPs could be the result of having opportunities to intermingle with and observe the ways of the experimental group, as was the case in Sittilingi. As it was a real-time field setting, both groups were at work in the hospital’s outreach clinic, and it was difficult to isolate one group from the other. In Thirupathur, however, this could not have been the case as the practitioners were all independent clinic-based providers, quite far from each other. Here, the reason may be attributed to an observation bias that led to the control group providers becoming more and more conscious of the observers and attempting to perform better and better perhaps by consulting the paper guidelines more frequently. Florez-Arango et al.¹⁴ have also reported a similar effect of patient order on control providers’ PC. Even though their study was based on human patient simulators, their RHPs did not have a chance to mingle or receive feedback during observations, similar to our Thirupathur setting.

Another similarity with the Florez-Arango study was the ease of use reported by a majority of providers. It took us by surprise to see how well the women from Sittilingi adapted to the phone during the 2-day training and even more so in the field. In fact, during our in-depth analysis, we found that one of the women auxiliaries with low literacy skills and a very high no-response rate in the pre–post training evaluation had scored the highest PC scores in the field test. We also saw that contrary to our earlier apprehensions, providers were able to easily accommodate the ear phones to hear the instructions (usually in one ear) and carry on with their treatment procedures. They also found (Table 5) the audio instructions to be one of the most useful features of the phone. Another study similarly found that use of audio instructions significantly improved effectiveness of counselling sessions.²⁰

Table 5.

Health workers’ responses to usability questions.

Question	Yes (%)	No (%)
Would you like more conditions to be included in this system in the future?	100	0
Will you continue using the system in your work?	87.5	12.5
Will you recommend the system to your friends and colleagues?	87.5	12.5
Do you find it useful to have the combination of audio, video and picture clips in the system?	87.5	12.5
If you did not have the system, would you still be able to remember and use all the necessary steps?	0	100

There were a few practical difficulties such as navigating through the English phone menu and the time delays caused by the heavier visuals on some screens. We considered that this may come in the way of regular use of the system in providers’ daily work flows, so we sought providers’ views on removing the visuals, but they were unanimously against this step. They said that they frequently used the system as a memory aid also, as well as a training tool, and would like to retain the pictures and videos. One option may be then for us to develop two sets of applications in future – one with just audio (and some line drawings perhaps) and text for use in regular patient encounters, and another visually embellished for refreshing providers’ knowledge. Pictures and videos are essential for demonstrating certain conditions (e.g. lethargy in a dehydrated child) and procedures (e.g. skin pinch test for dehydration), so it would not be possible to do away with them.

The small sample size of our study was its main limitation. Despite this, valuable lessons were learned that can be applied and generalized to other larger studies.

This study supports the implication that mMRIGs comprise a feasible and effective solution for standardizing and enhancing quality of care delivered by millions of primary health workers with varying levels of training and literacy.^21,22 The use of mobile technology has the potential to transform the way health service is delivered across the globe.⁷ The practice of medicine is an information-intensive field, and the principles of informatics have been an integral part of the medical practice.²³ Facilitating the storage and delivery of vast amount of clinical guidelines to practising health workers in an organized media-rich format is essential for success of any health program.^18,24 The benefits of any new technology or self-help tools should be applied in a bottom-up strategy to reach even those in the lowest strata of society, only then can we make true progress in terms of empowerment and independence.^25,26 The mMRIG technology is an important tool in the success of this system because it allows for easy customization of a variety of clinical guidelines, by local interdisciplinary teams even without any programming expertise. Other benefits can include the ability to document patients seen and the care they received in electronic form on the mobile device. These data can subsequently be uploaded to an electronic health record. Such mHealth-enabled capabilities can greatly improve documentation of care as well as reduce data quality errors.²⁷ Future work will be related to scaling up this application and including more patient-level data in study designs, such as treatment adherence and health outcomes. As Internet access and, in particular, participation in social networks become more widespread even in developing countries, an interesting aspect would be to integrate mMRIG systems with social media at least for the benefit of RHPs. As shown in Miron-Shatz et al.,²⁸ social media have the potential to greatly improve and enhance medical knowledge and practice. In particular, in the context of RHPs, these social media could help them develop mMRIGs for patient education and care collaboratively.

Footnotes

Acknowledgements

The authors would like to thank Drs Regi and Lalitha George of the Tribal Health Initiative, Sittilingi, Tamil Nadu, India, for their support and cooperation.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The second author (M.S.I.) is part owner of a company incorporated in the United States for commercialization of the GV technology.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was carried out with financial support provided by a small grant from the Information Society Innovation Fund (ISIF) Asia allocated in a competitive process to Garhwal Community Development and Welfare Society (GCDWS) in 2010. ISIF Asia operates through a partnership between the International Development Research Centre of Canada, the Swedish International Development Agency – SIDA and the Asia Pacific Network Information Centre (APNIC), with sponsorship from the Internet Society and the Dot Asia Organization.

References

Gautham

Binnendijk

Koren

. ‘First we go to the small doctor’: first contact for curative healthcare sought by rural communities in Andhra Pradesh and Orissa, India. Indian J Med Res 2011; 134: 627–638.

Pakenham-Walsh

Bukachi

. Information needs of health care workers in developing countries: a literature review with a focus on Africa. Hum Resour Health 2009; 7: 30.

Rowe

de Savigny

Lanata

. How can we achieve and maintain high-quality performance of health workers in low-resource settings? Lancet 2005; 366(9490): 1026–1035.

Cabana

Rand

Powe

. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282(15): 1458–1465.

World Health Organization (WHO). Integrated management of childhood illness: caring for newborns and children in the community. Geneva: WHO, 2011.

Ramachandran

Goswami

Canny

. Research and reality: using mobile messages to promote maternal health in rural India. In: Proceedings of the 4th ACM/IEEE international conference on information and communication technologies and development, London, 13–16 December 2010, Article no. 35. New York: ACM.

Kay

. mHealth: new horizons for health through mobile technologies. Geneva: WHO, 2011.

Schweitzer

Synowiec

. The economics of eHealth and mHealth. J Health Commun 2012; 17(Suppl. 1): 73–81.

Akter

Ray

. mHealth – an ultimate platform to serve the unserved. Yearb Med Inform 2010; 94–100.

10.

Derenzi

Borriello

Jackson

. Mobile phone tools for field-based health care workers in low-income countries. Mt Sinai J Med 2011; 78(3): 406–418.

11.

Kulkarni

Agrawal

. Smartphone driven healthcare system for rural communities in developing countries. In: Proceedings of the 2nd international workshop on systems and networking support for health care and assisted living environments, Breckenridge, CO, 17 June 2008, Article no. 8. New York: ACM, pp. 75–81.

12.

Fraser

Blaya

. Implementing medical information systems in developing countries, what works and what doesn’t. AMIA Annu Symp Proc 2010; 2010: 232–236.

13.

Patel

Arocha

Diermeier

. Methods of cognitive analysis to support the design and evaluation of biomedical systems: the case of clinical practice guidelines. J Biomed Inform 2001; 34: 52–66.

14.

Florez-Arango

Iyengar

Dunn

. Performance factors of mobile rich-media job aids for community health workers. J Am Med Inform Assoc 2011; 18(2): 131–137.

15.

Iyengar

Florez-Arango

Garcia

. GuideView: a system for developing structured, multimodal, multi-platform persuasive applications. In: Proceedings of the 4th international conference on persuasive technology, Claremont, CA, 26–29 April 2009, Article no. 31. New York: ACM.

16.

Pinzon

Iyengar

. Persuasive technology and mobile health: a systematic review. In: 7th international conference persuasive technology, Linköping, 6–8 June 2012, pp. 45–48. Berlin: Springer.

17.

Fogg

. Persuasive technology: using computers to change what we think and do. Ubiquity, vol. 2002, December 2002, Article no. 5.

18.

Ramachandran

Adviser-Canny

. Mobile persuasive technologies for rural health. PhD Thesis, University of California, Berkeley, Berkeley, CA, 2010.

19.

Tamrat

Kachnowski

. Special delivery: an analysis of mHealth in maternal and newborn health programs and their outcomes around the world. Matern Child Health J 2012; 16(5): 1092–1101.

20.

Ramachandran

Canny

Das

. Mobile-izing health workers in rural India. In: Proceedings of the 28th international conference on human factors in computing systems, Atlanta, GA, 10–15 April 2010, pp. 1889–1898. New York: ACM.

21.

Chang

Kagaayi

Arem

. Impact of a mHealth intervention for peer health workers on AIDS care in rural Uganda: a mixed methods evaluation of a cluster-randomized trial. AIDS Behav 2011; 15(8): 1776–1784.

22.

Mechael

Nemser

Cosmaciuc

. Capitalizing on the characteristics of mHealth to evaluate its impact. J Health Commun 2012; 17(Suppl. 1): 62–66.

23.

Perera

. The evolution of e-health – mobile technology and mHealth. J Mob Technol Med 2012; 1(1): 1–2.

24.

Iyengar

Sarkar

Bacal

. GuideView: structured multi-modal delivery of clinical guidelines. AMIA Annu Symp Proc 2005; 2005: 992.

25.

Hammond

. Digitally empowered development. Foreign Aff 2001; 80: 96–106.

26.

Norris

Stockdale

Sharma

. A strategic approach to m-health. Health Informatics J 2009; 15(3): 244–253.

27.

Orfanidis

Bamidis

Eaglestone

. Data quality issues in electronic health records: an adaptation framework for the Greek health system. Health Informatics J 2004; 10: 23–36.

28.

Miron-Shatz Hansen

Grajales

II . Social media for the promotion of holistic self-participatory care: an evidence based approach. Contribution of the IMIA Social Media Working Group. Yearb Med Inform 2013; 8(1): 162–168.