Systematic review of health data quality management and best practices at community and district levels in LMIC

Abstract

Research findings have reported lack of reliable health data and poor management for district health information systems in low and middle-income countries (LMIC). This paper aims to review the literature on problems with health data quality management and health information evidences and evidences of best practices and use at community and district levels in LMIC, with a view to making recommendations for future research. Research citations, conference proceedings and diseases surveillance reports from 2000–2011 were accessed in PubMed, Medline, LISTA (EBSCO), CINAHL, Cochrane, and Google. Relevant studies were selected, the methodologies critiqued and synthesized. The researchers accessed 1383, and 38 were reviewed by three reviewers. Poor quality health data, low level of health information use, and poor management of health information systems were found. These findings hinder evidence-based decisions based and planning at community and district levels in LMIC. Though poor practices were found, improved health care services delivery with improved health data efficiency was found to be possible.

Keywords

data quality management health information use community district health information systems systematic review

It is feasible to design, implement and utilize health information systems that fit into local contexts.

Introduction

Reliable health data are the foundation of credible evidence of the health status of patients, and can assist policy makers in making decisions and plans to improve health care services delivery for better health (Haux, 2006). This review focuses on the available evidence of strategies that could facilitate data quality management and the utilization of health information in low and middle-income countries (LMIC) (Lippeveld et al., 2000; Kerr and Norris, 2010).

Data quality has to be ensured through all phases of data collection and utilization (World Health Organization, 2003). The utilization of health information includes making health information available to intended users through feedback and data sharing; and using the data for evidence-based decisions, policy-making, and planning interventions. To facilitate data management and utilization, health information management systems (HIMS) are essential. These systems refer to the components and procedures to organize data and include two entities: (1) health information data collection and utilization processes, and (2) health information system management structures.

The process of data quality management includes ensuring the reliability of data collected from data input, which includes data acquisition and verification; data management or processing, which comprises data storage, aggregation, classification, data update and data computation; and data output, that includes data retrieval and data presentation (Tan, 2001 p.9). Health professionals and policy makers rely on available data as reported from healthcare facilities to make decisions and the decisions they make depend directly on the quality of the data they receive (AbouZahr and Boerma, 2005).

Globally, research findings have reported poor quality of health data (Calle et al., 2000; Goeree et al., 2009) and underutilization of health information for planning and decision-making (Corrao et al., 2009). These problems have hindered decision-making, planning processes and the performance evaluation of health care programs and diseases outcomes (Corrao et al., 2009). It is also of particularly major concern in LMIC, where health information systems are newly initiated within resources constrained contexts (World Health Organization, 2006; Bosch-Capblanch et al., 2009).

In most LMIC, the health information systems are also paper-based and often decentralized at district level (LaFond and Siddiqi, 2003). The district level information systems (DHIMS) are used to compile monthly reports on geographical, clinical and administrative from routine health services data, as well as disease notification from epidemiological surveillance data (Lippeveld et al., 2000). Most LMIC use paper-based data collection processes at primary health care level and paper and computer-based health information systems at district level (Haux et al., 2007; Heywood and Rhode, 2001). However, paper-based information systems are often found to generate data with poor quality and are underutilized within the health information management (Lium et al., 2008).

It is important to strengthen health systems in LMIC through good data management and data availability for better decision-making (Lippeveld et al., 2000, Haux, 2006). As there are concerns that quality health information is lacking and evidence-based planning and decision-making might be affected, the reported progress towards Millennium Development Goals by LMIC may be questionable (AbouZahr and Boerma, 2005; Kerr et al., 2007). It is in the light of these concerns that we reviewed and summarized the literature on data quality management and health information use at community and district levels in LMIC.

Purpose of the review

The objectives of the review were to:

Identify and review studies that evaluated health data quality management processes, health information use, and health information system management issues in LMIC.

Critique research methods used in those studies.

Describe problems with health data quality management and health information use in LMIC and identify reported best practices for implementation.

Methods

MeSH terms in the PubMed and their synonyms were identified to extensively search and retrieve relevant literature on quality management, health information use, and health information management structures in LMIC from 2000 to early 2011.

The following search terms were used: (Health OR Medical OR Patient) AND (Data OR information OR records OR Electronic Medical Records (EMRS) AND (Quality OR management OR reporting OR use OR systems OR data collection OR data processing OR reporting OR data analysis OR information use OR information literacy OR Feedback OR district OR primary care) AND (Evaluation studies OR randomized controlled trials OR experimental studies OR quasi-experimental studies OR descriptive studies OR qualitative studies OR Systematic Review) AND (information use OR decision-making OR policy-making OR planning). Databases searched included: PubMed, Medline, LISTA (EBSCO), CINAHL, Cochrane Collaboration, Joanna Briggs, Google and Google Scholar.

Relevant published papers, reports and conference proceedings were retrieved. No manual searching was performed. Only studies reporting on data quality management and health information in or relevant to LMIC in the English language were included. Two reviewers independently screened the titles and abstracts of all retrieved studies and disagreements were referred to the third reviewer to resolve. Once the studies were selected, the full articles were retrieved, reviewed for quality of study design type and relevant information extracted.

Results

The reviewers identified 1383 studies from the initial search. Of these, 253 (18.3 percent) were selected after the exclusion of irrelevant studies. A further 70 (5.1 percent) studies remained after studies not applicable to LMIC were also excluded. Based on the quality of the study designs, 38 studies (2.7 percent) were selected for full review (See Figure 1), though the 38 studies included three low quality design studies due to the nature of the information provided by these studies.

FIGURE 1.

Flow diagram for study selection.

Characteristics of reviewed literature

Twenty-five of the 38 studies (65.8 percent) were conducted in Africa, 16 (42.1 percent) were evaluation studies, 13 (34.2 percent) were descriptive studies and seven (18.4 percent) were systematic reviews (Table 1).

Table 1.

Summary of the 38 studies for review.

Studies	No (%)
Data from Country
South Africa	8 (21.1%)
Kenya	4 (10.5%)
Other African Countries	13 (34.2%)
Other relevant LMIC	13 (34.2%)
Year
2000–2005	11 (29.0%)
2006–2011	27 (71.1%)
Focus of Study
Data quality evaluation (4 of them reported also on feedback)	7 (18.4%)
Health information sharing and feedback	2 (5.3%)
Health information use for decision-making and planning	6 (15.8%)
Health information use for research	3 (7.9%)
Health Information System Management structure	14 (36.8%)
EMRS	7 (18.4%)
Study Design
Systematic reviews	7 (18.4%)
RCTs	0 (0.0%)
Quasi-experimental studies with and without control groups	2 (5.3%)
Evaluation studies	16 (42.1%)
Descriptive studies and other designs	13 (34.2%)

Data quality management studies

In reviewing the studies that reported on the quality of data found in LMIC, the following criteria identified by the reviewers were used: accuracy or correctness, completeness, timeliness, comprehensiveness, consistency, accessibility, reliability and acceptability of data. Out of the 38 studies, seven (18.4 percent) studies (Odhiambo-Otieno, 2005; Ronveaux et al., 2005; Garrib et al., 2008; Bosch-Capblanch et al., 2009; Mate et al., 2009; Harper et al., 2011; Heunis et al., 2011) specifically evaluated the quality of health data (see Table 2). Of the seven studies, four (57.1 percent) were descriptive case studies evaluating the quality of the data collected in health facilities (see Table 2).

Table 2.

Summary of studies evaluating health data quality.

Article	Country	Study Design and Focus	Main Findings	Quality of Study
Ronveaux et al. (2005)	Global	Retrospective evaluation of the quality of immunization data from 27 LMIC using standardized data quality audits Accuracy=verification	Lower than 85% of verified DTP-3 doses recorded in 16 countries All countries reported weaknesses in their monitoring systems for data quality Overall poor management of immunization programs	No interventions Big sample size High validity with other HMIS evaluation findings
Odhiambo-Otieno (2005)	Kenya	Cross-sectional study to evaluate DHIMS in three districts	Accuracy 30% Completeness 19% Timeliness 26% Relevance 72%	Moderate quality (cross-sectional) Large sample size, High external validity in Kenya
Garrib et al. (2008)	South Africa	Retrospective evaluation of the quality of reported health data by nurses from 10 clinics of one District Accuracy= data missing and in expected ranges	Poor quality of health data 2.5% missing data values 25% outside expected ranges Rare data quality checking Good understanding of data collection	No intervention Big sample size High external validity of findings with other DHIMS evaluation reports
Mate et al. (2009)	South Africa	Retrospective evaluation of the quality of PMTCT routine data management from 316 clinics Accuracy= within 10% of constructed values	Completeness: 50.3% Overall data accuracy: 12.8 Best: Record of HIV testing 19.8% Worst: Record of HIV PCR testing 5.3%	No intervention Big sample size High validity with other findings
Bosch-Capblanch et al. (2009)	Global	Retrospective data quality audit of the quality of immunization data from 41 LMIC Accuracy=verification factor (VF) of 80% & quality score (QS)	Accuracy: 50% countries obtained 80% VF 9 countries (22%) high VF & QS Poor availability of guidelines for data management Incorrect denominators to calculate coverage Data duplications	No intervention Big sample size Standard methodology High validity with other HIMS findings
Harper et al. (2011)	Aboriginal Population in Canada	Retrospective evaluation of patients' data captured in a health registry system in aboriginal communities: a case study Accuracy=verification	Data completeness high: 99% Data inconsistency 18.3% Data accuracy of diagnosis 6.2% incorrect Data analysis was done at annually basis	No intervention Small sample size Low external validity with other HIMS findings
Heunis et al. (2011)	South Africa	Retrospective evaluation of the quality of TB-HIV routine data and nurses' views on data management in 5 districts Accuracy=verification	Overall inconsistencies: 32% Worst TB treatment start 44% & end: 41% Best TB patient file number 4% Rare data quality checking	No intervention Big sample size High external validity with other HIMS evaluation findings

None of the studies reported on all the criteria of data quality. Overall, the seven studies reported poor health data quality management in three domains. These include: (1) Incompleteness of data: Though 99 percent completeness was reported in a health registry for an aboriginal community in Canada (Harper et al., 2011), data completeness was poor in LMIC with missing data ranging from 19 percent (Odhiambo-Otieno, 2005) to 50.3 percent (Mate et al., 2009). (2) Inconsistencies in data collection and processing:Levels of inconsistency were reported for country immunization programs with only nine (22.0 percent) out of 41 countries reporting consistent data (Bosch-Capblanch et al., 2009). There were also inconsistencies in general for TB data (32.0 percent) (Heunis et al., 2011) and diagnosis data in health registries (18.3 percent) (Harper et al., 2011). (3) Inaccuracy or incorrectness: Though inconsistent definitions of accuracy were used, the study using a verification factor to assess accuracy showed that only 50 percent of the countries obtained 80 percent accuracy (Bosch-Capblanch et al., 2009). Likewise, in a study on HIV data accuracy, only 12.8 percent of PMTCT (Preventing Mother-to-Child Transmission) data, 19.8 percent of HIV Counselling and Testing (HCT) data and 5.5 percent of HIV PCR testing data of babies born to HIV mothers were reported as accurate (Mate et al., 2009). Other issues identified were poor usage of standard data collection tools, data duplications, multiple registers (Garrib et al., 2008; Bosch-Capblanch et al., 2009), poor and infrequent data quality checking procedures (e.g. tally sheets) and inconsistencies in the use of denominators to estimate coverage (Bosch-Capblanch et al., 2009). Though a lack of guidelines for data quality management was reported (Bosch-Capblanch et al., 2009), one study reported good understanding of the data collection and data collation processes (Garrib et al., 2008).

Studies on utilization of health information

Fifteen studies were found that reported on the utilization of health information, six on evidence of feedback initiatives and nine on the utilization of health information (see Table 3).

Table 3.

Summary of studies evaluating health information use.

Article	Country	Study Design and Focus	Main Findings	Quality of the Study
Feedback initiatives
Garrib et al. (2008)	South Africa	Retrospective evaluation of the quality of reported health data by nurses from 10 clinics of one District	Good understanding of data collection Rare feedback to health facilities High perceived burden of health data management	No intervention Big sample size
Mate et al. (2009)	South Africa	Retrospective evaluation of the quality of PMTCT routine data management from 316 clinics	Rare evidence of feedback	No intervention Big sample size
Harper et al. (2011)	Aboriginal Population in Canada	Retrospective evaluation of patients' data captured in a health registry system in aboriginal communities: a case study	Data analysis was done annually Low accessibility of data to the users	No intervention Small sample size
Taylor and Shimp (2010)	Nigeria and India	Description of health information use for polio eradication in India and Nigeria	Communities became active agents in polio eradication in Nigeria and India through feedback and sharing health information Increased immunization rate	No intervention
van der Veer et al. (2010)	Global	Systematic review of the effect of a feedback on quality care (53 studies)	Feedback improved quality care processes and quality of data, Motivation of health personnel	No intervention High external validity Large sample size
Heunis et al. (2011)	South Africa	Retrospective evaluation of the quality of TB-HIV routine data and nurses' views on data management in 5 districts	Rare evidence of feedback	No intervention Big sample size
Health information use for decisions and planning
Booman et al. (2000)	South Africa	Quasi-experimental study with description of the use of geographical health data for malaria control in Mpumalanga Province	Calculation of Risk Targeted interventions Malaria control at all levels, particularly malaria data management at community level of in	Intervention Historical control study design Large sample size
Merrell et al. (2004)	Kenya	Description of the use of ambulant community EMRS in 2700 patients in four villages	Data accessibility and interoperability improved Clinical decisions support reasonable cost Few live telemedicine consults were needed	Intervention No control group No Pre and Post data Large sample size of patients, but few villages Low external validity
Siika et al. (2005)	Kenya	Description of the use of EMRS for more than 4000 ambulatory HIV clients	Improved monitoring and outcome evaluation of ambulatory HIV clients from remote areas at reasonable cost was found; Facilitated research.	Intervention Large sample size No control group No Pre and Post data High internal validity.
Rahimi and Vimarlund (2007)	Global	Systematic review of 61 studies on IT-based systems in health care settings	User satisfaction, Financial benefits Improved organizational work Found no standard framework for evaluation effects and outputs of implementation and use of IT	Systematic review Large sample size
Guy et al. (2009)	Global	Systematic review of 20 studies on the usability of HIV routine data (VCT) to assess the impact of public health interventions	Routinely VCT data was found useful to evaluate the impact of public health interventions	Good systematic review Large sample size High external validity with other findings
Farias et al. (2010)	Argentina	Disease surveillance report and health data sharing with communities: a case study	Positive effect of integrated information tools Effective influenza A (H1N1) pandemic management was found, Funds reallocation and efficiency were found.	Intervention No control group No Pre and Post data Small sample size limiting generalizability
Were et al. (2010)	Uganda	Time and Motion evaluation of EMR-based paper clinical summaries to support HIV-care of 15,000 clients	Improved quality of data Save time Reduced mistakes Facilitated research with efficiency	Implementation Evaluation Historical control Pre and Post data Large sample size High external validity with other findings
Health Information use for Research
Martin et al. (2002)	South Africa	Description of the use of a GIS for malaria control program	Appropriate malaria control, Improved the quality of data with efficiency Saved time Facilitated research	Description of development and use Large sample size
Diero et al. (2006)	Kenya	Prospective study of computer-based EMRS to follow up 381 patients with tract infections	Follow up of patients in remote areas was feasible Facilitated prospective clinical research and evaluation of disease outcomes with limited resources.	Prospective study Large sample size No control

Health information sharing and feedback.

Of the six studies reporting on feedback initiatives (Garrib et al., 2008; Mate et al., 2009; Harper et al., 2011; Taylor and Shimp, 2010; van der Veer et al., 2010; Heunis et al., 2011) (see Table 3), two studies specifically reported infrequent evidence of feedback (Garrib et al., 2008; Heunis et al., 2011). Garrib (2008) reported incorrect usage of data collation tools, little analysis, interpretation or utilization of data and infrequent feedback to clinics (Garrib et al., 2008). Similarly, the study by Heunis et al. (2011) found lack of supervision to address data quality issues resulted in clinic personnel being unaware of their performance.

Overall, factors associated with effective feedback in the reviewed studies were: improved quality of data, motivation of the health care personnel and their expectations, and the availability of necessary equipment that enforce quality improvement.

The improvement of data quality as a result of feedback was reported from three studies. A systematic review on how medical registries provide feedback to health care providers found that feedback initiatives could positively influence the process and quality of care (van der Veer et al., 2010). This was also illustrated in a study in India and Nigeria which found that by sharing health information with the community, there was a significant increase in immunization (Taylor and Shimp, 2010). Similarly, in clinics where health information was shared among health care providers and the clinic received feedback, TB data were used for planning, monitoring and evaluation of the performance and outcome of the TB programs (Heunis et al., 2011).

Utilization of health information for decision-making and planning

Two systematic reviews (Rahimi and Vimarlund, 2007; Guy et al., 2009) and two studies (Booman et al., 2000; Farias et al., 2010) reported on practices in using health information for public health interventions planning (see Table 3). These included: A systematic review of moderate quality on the application of IT-based systems in health which showed positive results in user satisfaction, financial benefits and improved organizational work (Rahimi and Vimarlund, 2007); A good systematic review on the usability of HCT data in assessing the impact of public health interventions which concluded that data from routinely collected data was useful for public health interventions in resource limited settings (Guy et al., 2009); Geographical data used for targeted risk identification and malaria control in South Africa (Booman et al., 2000); and Influenza (H1N1) pandemic control which used an integrated IT-based HIMS to evaluate program performances and disease outcomes in Argentina (Farias et al., 2010).

Three studies of high to moderate quality reported on evaluations of utilization of electronic medical records systems (EMRS) and diagnosis supporting tools to improve patients’ management by using patient-specific clinical summaries in Uganda and Kenya (Merrell et al., 2004; Siika et al., 2005; Were et al., 2010). In Kenya health care providers in remote areas reported getting clinical decision support through health data sharing with medical experts (Merrell et al., 2004); and in Uganda and Kenya, the use of EMRS were shown to be efficient in managing HIV/AIDS patients, monitoring and evaluation of diseases outcome, tracking missing patients and infants enrolled in PMTCT from remote areas, saving time spent on care and facilitating research (Siika et al., 2005; Were et al., 2010).

Health information use for research

Four studies (Martin et al., 2002; Siika et al., 2005; Diero et al., 2006; Were et al., 2010) reported on the utilization of health information for research (see Table 3). Reports of prospective clinical studies using EMRS which could be undertaken at larger scale were useful to evaluate disease outcomes in Kenya (Siika et al., 2005; Diero et al., 2006) and Uganda (Were et al., 2010). Lastly, the study by Martin et al. (2002) described the efficiency in undertaking a research-based, Geographical Information System (GIS) for disease surveillance to influence health programs planning for Malaria Control Programs in South Africa (Martin et al., 2002).

Studies on practices for health information management system structure

Ten studies were found on Health Information Management System (HIMS) structure and processes. Seven studies evaluated structural factors and processes in DHIMS in South Africa, Kenya, Tanzania, Zambia, Mozambique and Nigeria (see Table 4), while three studies reported on interventions to improve data quality and feedback (see Table 5).

Table 4.

Summary of studies on health information system management structures and processes.

Article	Country	Study Design and Focus	Main Findings	Quality of Study
Akandeand Monehin (2004)	Nigeria	Descriptive survey of HIMS in 37 private clinics	Structures Lack of necessary equipment and data management tools and form 29.7% supplied forms 16.2% had forms 36.4% made returns Processes Lack of training Lack of knowledge (10.8% knowledge of forms) Lack of awareness of HIMS (67.6%).	No intervention Moderate sample size Poor study design (descriptive)
Odhiambo-Otieno (2005)	Kenya	Cross-sectional study to evaluate DHIMS in three districts	Structures 0% DHIMS computerized Adequacy of structures poor: Staff 47%, Working space 40%, Storage space 34% and Stationary 20% Processes DHIMS not supportive of managers IT and health collaboration Confidentiality 32% Use of information 22% Training 73%	Moderate quality (cross-sectional) Large sample size, High external validity with other HIMS findings in Kenya
Mutemwa (2006)	Zambia	Qualitative comparative case-study of two DHIMS implementation in two districts	Processes Using information for strategic decision making: organizational processes human resources organizational structure	No intervention Small sample size Low external validity
Smith et al. (2008)	Tanzania	Description of integration issues associated with DHIMS: understanding issues of integration	Structure Parallel systems creates problems Processes Health data need to be integrated Workflow	No intervention Low external validity
Lima et al. (2009)	Brazil	Systematic review of 78 evaluation studies of Brazilian HIMS	Processes Need for policy for HIMS, Paucity of continuous evaluations	Systematic review No interventions Large sample External validity
Hanmer et al. (2010)	South Africa	Survey of factors associated with computerized health information system success in two provinces	Processes Perceptions of usefulness of the CHIS Capacity building of the personnel Management commitment.	No intervention Case studies Low external validity
Hotchkiss et al. (2010)	Uganda	Evaluation of the Performance of Routine Information System Management (PRISM) framework in 12 districts	Structures PRISM effective conceptual framework to evaluate HIMS strengthening interventions Processes Promote culture of information Decision making Use of information Motivation for use	No intervention Large sample size Good external validity with other DHIMS findings

Table 5.

Summary of studies on health information system management interventions.

Article	Country	Study Design and Focus	Intervention	Main Findings	Study quality
Jacucci et al. (2006)	South Africa	Case study description of standardization of health information systems and challenges for local sustainability	Integration of health information systems	Networking with DHIMS Layering of systems e.g. Health and budget information Importance of workflow design Relevance of information for use Enforcing information culture Formative supervision, Monitoring and evaluation, Guaranteed feedback	Single case study Intervention No control No Pre and Post Limited generalizability
Wong and Bradley (2009)	Ethiopia	A pre and post-test time motion study to measure accessibility and completeness of medical records	Hospital wide registration Medical records re-engineering One simple computer database Standardized forms	Improved data quality Retrieval of data for returning patient from 14% to 87% (p < .01) Completeness 6.5% to 45.7% (p<.01) Saves time spent on record location 31.2 sec to 15.7 sec (p < .01) Physician satisfaction (p = .02)	Single case study Pre and Post No control Low sample Limited generalizability
Osa-Eloka et al. (2009)	Nigeria	Evaluation of the outcome of a training on data management among 107 clinics: before and after intervention design	Training scheme on data collection, recording, reporting and utilization practices among primary health care workers	Training increases greatly data management skills among HIMS users in intervention group (p<.001) Reporting and local uses of data improved in intervention group (p<.001)	Intervention Design Control group Large sample Pre and Post High external validity with other DHIMS findings

District Health Information System management structures and processes

Issues and problems with DHIMS described in the studies included: (1) Structural problems, namely: lack of clear policy for health information systems (Lima et al., 2009), lack of resources and facilities, lack of management commitment, pressure for good results, overloading forms to complete for different programs and parallel reports (Mutemwa, 2006), lack of feedback and poor collaboration between information technology experts, managers and users, especially in the conception phases (Odhiambo-Otieno, 2005), and parallel health information systems and lack of integration (Smith et al., 2008); (2) Behavioural issues, namely: lack of information culture and few evaluation studies which hindered comprehensive data quality assessments (Akande and Monehin, 2004; Hotchkiss et al., 2010); and (3) Technical issues, namely: use of inappropriate and complex technology, lack of training, lack of skills, lack of pre-operational and post-implementation evaluation (Odhiambo-Otieno, 2005).

Interventions to improve data quality and feedback

Specific interventions to improve data quality were reported in studies in South Africa (Jacucci et al., 2006), Ethiopia (Wong and Bradley, 2009) and Nigeria (Osa-Eloka et al., 2009). One district in South Africa overcame problems associated with poor quality data management through formative supervision, monitoring and evaluation, guaranteed feedback to increase the quality of health data and the integration of data management and networking actions at all district levels (Jacucci et al., 2006). The following outcomes from their interventions were reported: improved quality of reported data from health facilities, increased accessibility of data, enhanced skills in the use of data collection tools and data analysis software, increased utilization health information use, and tangible management commitment to enforce good practice culture (Jacucci et al., 2006). In addition, daily quality data checks, utilization of health information to plan health interventions, and the use for financial negotiations with the district were observed (Jacucci et al., 2006). A study conducted in Ethiopia found that using hospital wide registration, one computer database, medical records re-engineering and standardized forms improved the retrieval of records, time spent on retrieval and improved records completeness (Wong and Bradley, 2009). And lastly, a study in Nigeria showed that training greatly increases data management skills and reporting and local use of data among HIMS users (Osa-Eloka et al., 2009).

Studies on practices for Electronic Medical Information System Management

Internationally there has been a dramatic increase in the use and popularity of technology to develop electronic medical record systems (EMRS) for health data management. Though the context of LMIC makes the application of these systems difficult, 10 studies were found on the use of EMRS in LMIC. Three studies focused on factors affecting EMRS implementation in LMIC (see Table 6) and seven studies evaluated the effectiveness of the EMRS interventions in LMIC (see Table 7).

Table 6.

Summary of studies on electronic medical records systems implementation.

Article	Country	Study Design and Focus	Main Findings	Quality of Study
Kimaro and Twaakyondo (2005)	Tanzania	Analyzing the hindrance to the use of information and technology for improving efficiency of health care delivery system: Action research in 5 districts	Lack of skills for data interpretation and utilization, Lack of clear policy guidelines on information and human capacity building, Lack of flexible system hinder proper HIMS management	Case study Large sample size
Kimaro and Nhampossa (2005)	Mozambique and Tanzania	Analyzing the problem of unsustainable DHIMS less-developed economies: Case studies Mozambique and Tanzania	Integration of a HIMS Local shaping of new cultures Cultivation approach to systems development	Action Research Study
Ludwick and Doucette (2009)	Global	Systematic review of 86 studies on the adoption of EMRS in primary care	Quality of implementation process Health system usability Computer skills Organizational culture.	Systematic review High external validity with other findings

Table 7.

Summary of studies on effectiveness of EMRS in data management.

Article	Country	Study Design and Focus	Intervention	Main Findings	Quality of Study
Fraser et al. (2004)	Haiti	Description of the implementation of an EMRS system for HIV data (7000 HIV clients)	Web-based EMRS system for HIV data	Outcomes Improved patients follow up Improved data quality Data accessibility Interoperability Issues Lack of connectivity Literacy and patient IDs	Case study description Larger sample No control No Pre and Post
Chaudhry et al. (2006)	Global	Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care (257 studies)	Health information Technology	Outcomes Increased adherence to guideline care Increases data quality Decreases medical errors, Improve diseases surveillance Improve cost effectiveness	Good Systematic Review Not LMIC focused
Lane et al. (2006)	Global	Systematic Review of 9 RCTs of effectiveness of hand held computers with papers methods for data management	Handheld computers	Outcomes Improved speed and data processing Timely reports Adherence to data handling instructions, Preferred by the users.	Good Systematic Review Not LMIC focused
Amoroso et al. (2010)	Rwanda	Description of the implementation of EMRS system for HIV data: a case report for one district	EMR for HIV	Outcomes Improved patients follow up, Improved data quality Improved data accessibility: critical CD4 data that were not reaching clinicians by 34.2% (p=.002), Decreased systematic errors 92% On-time reports delivery Issues Structural issues Lack of skilled personnel	Evaluation study Small sample No control No Pre and Post
Kim et al. (2004)	Global	Conference proceeding on Personal Health Information Management System (PHIMS)	Web-based PHIMS in a surgical referral practice.	Outcomes Improved completeness of data Efficiency of patients care Users’ satisfaction : Usability 85% Online referral 94%	Evaluation study Small sample No control No Pre and Post
Kotze and McDonald (2010)	South Africa	Description of the implementation of an information system for antiretroviral treatment program	Palm computer project followed by HIMS	Outcomes Data quality improvement Timely reports delivery Patients follow up Issues Lack of integration of all HIV information systems	Action Research Large sample size
Krishnan et al. (2010)	India	Retrospective evaluation of HIMS for primary health care in rural area: a case study	Computerized HIMS	Outcomes Improved data accuracy (>95%) Health workers useful Health Managers improved services, monitor and supervision Annual cost saving	Action Research Low sample No control No Pre and Post

Electronic Health Information System implementation processes

The studies reviewed identified a number of factors which may encumber an EMRS implementation (Table 6). A case study in Tanzania found that poor information literacy, unclear policies and inflexible systems hindered the implementation of an EMRS (Kimaro and Nhampossa, 2005). A systematic review conducted on the adoption of EMRS implementation in primary care showed that the quality of the implementation process is as important as the quality of the EMRS (Ludwick and Doucette, 2009). Other important factors identified were personnel satisfaction, usability and interoperability of EMRS for patient care and data sharing for clinical support and referrals (Ludwick and Doucette, 2009), and integration of HIMS systems (Kimaro and Nhampossa, 2005). Ludwick and Doucette (2009) also found that strong leadership, sound management, well established standards, staff training and due consideration of the technical and social context are fundamental to the success of any HIMS implementation.

Evaluation of effectiveness of EMRS in data management

Studies evaluating the effectiveness of technological interventions to improve data collection focused on EMRS (6 studies and a systematic review) and hand held devices for data collection (1 systematic review) (see Table 7).

A study conducted in India found that using a computerized HIMS assisted program managers in Primary Health Centres in the monitoring and supervision of their daily activities, funds allocation and resulted in time efficiencies, increased data quality, and improved health services delivery (Krishnan et al., 2010). A systematic review on the impact of health information technology on quality, efficiency, and cost of medical care (Chaudhry et al., 2006) concluded that the health information technology improves the quality care and efficiency through adherence to guidelines and protocols for decision support.

Three studies conducted on HIV/AIDS Management systems in South Africa (Kotze and McDonald, 2010), Rwanda (Amoroso et al., 2010) and Haiti (Fraser et al., 2004) showed improved monitoring and evaluation of antiretroviral data and HIV/AIDS treatment across the three countries (Fraser et al., 2004; Merrell et al., 2004, Amoroso et al., 2010). In Rwanda, the use of detailed HIV/AIDS and TB computerized data facilitated patient follow up, improved data accessibility and quality (92 percent reduction in systematic errors) and improved patient care (Amoroso et al., 2010). In terms of capturing data using hand held computers, the systematic review showed that hand held computers were better than paper use for data collection in terms of speed of data collection, ease of data processing, timely delivery of reports, adherence to data handling instructions, and were the most preferred by the users (Lane et al., 2006).

Structural problems in EMRS identified in Haiti and Rwanda included the lack of necessary infrastructure and equipment, inadequate program officers at different sites, and communication issues (Fraser et al., 2004; Amoroso et al., 2010).

Discussion

International evidence has shown that health information systems are fundamental to the delivery of quality health care (Chaudhry et al., 2006). In LMIC the burden of diseases such HIV/AIDS and the need for health information systems to coordinate care and treatment, is accompanied by concerns that paper-based records are no longer adequate and should be replaced by more efficient EMRS (Kalogriopoulos et al., 2009). Countries in Europe and the USA are increasingly using EMRS to improve healthcare service delivery quality (Williams and Boren, 2008; Ali et al., 2011). Unfortunately LMIC face many challenges in implementing and utilizing health information systems, including concerns about quality of data and inadequate communications technology, human expertise and financial resources (Williams and Boren, 2008).

For health decision makers in LMIC, these challenges are compounded by a lack of high quality evaluations of health information systems and effective data quality management strategies (Chaudhry et al., 2006). Similarly, in this review of health information system evaluation studies, study quality ranged from high quality systematic reviews to low quality descriptions of single case studies. Despite these limitations, the reviewed studies showed that the use of technology could improve both the quality and the use of health data in LMIC and could lead to good outcomes such as: improved patients’ follow up from remote areas, increased users’ satisfaction, adherences to data quality management standards, efficient management of health clinics, hospitals and public health programs and improved guideline adherence (Oluoch et al., 2012). In addition, the review showed that health information systems structures were found very helpful in overcoming problems associated with paper-based health information systems and facilitating clinical and health systems researches within limited resources contexts.

While the benefits of health information systems are clear, adapting new health information systems has proven difficult in the past and rates of use have been limited (Chaudhry et al., 2006). In LMIC, the overall aim of a health information system, which is to facilitate care, may not be reached, with the severe resource constraints hindering implementation, data quality management and utilization of health information. This review also supported this finding with studies identifying two major problems hindering data quality management: Firstly, technical infrastructure issues such as unreliable electric power and erratic Internet connectivity and clinicians’ limited computer skills (Oluoch et al., 2012); and secondly, data quality management issues such as the lack of policies and guidelines, training for health personnel, required equipment, good communication and networking actions among all stakeholders of HIMS, supervision, monitoring and continuous evaluations, and information culture at different levels of district health information systems.

There are also evidence that in LMIC data are incomplete, inaccurate, unreliable and not timely (Simba, 2004) and that poor data quality management have a major impact on the utilization of health data and the potential use of an EMRS. Against a 75th percentile standard for data quality, the reviewed studies reported poor quality of data from LMIC health facilities and district health information systems. One of the main problems was data incompleteness with no LMIC country reporting data completeness over 50 percent. Though accuracy was also measured using different verification methods, data inconsistencies and inaccuracies were common and often the reviewed studies did not report the standards against which they appraised the level of data quality. Inaccurate and incomplete data may result in data not being seen as reliable enough to inform on programs performance and disease outcomes or to be used for decision making and planning and that EMRS might expedite the dissemination of wrong or poor data (Simba, 2004)

However, with accurate data, feedback to users, information sharing, decision-making and program planning based on available health information could be good standards of quality usage of health information. Our review showed that good practices in data quality management and utilization of health information exist with a few studies reported using data for public health interventions and research. However, the review also showed that problems outweighed the good practices with health data being neither accurate (correct) nor reliable, poor evidence of feedback practices, district health information systems being poorly managed, a lack of information culture in LMIC, problems implementing HIMS and a failure to deliver reliable health information for better decisions and planning. To improve utilization of health information, it is believed that the establishment of strong leadership, using project management techniques, establishing standards and training their staff is essential for implementation success (Ludwick and Doucette, 2009) and more work needs to be done to overcome the barriers to implementation such as technical infrastructure and care providers’ computer illiteracy (Oluoch et al., 2012). Similarly, this review identified a number of key strategies to improve the implementation and utilization of health information systems. Specifically relevant for LMIC were the development of integrated health information systems, reengineering of work flow and medical records, management commitment to create an information culture and training of data collectors and users. In addition, the development of EMRS with open standards and sharable components using open source software should also be developed tailored to the population, location, and availability of resources (Kalogriopoulos et al., 2009).

Conclusion

This review reinforces the need for well-designed studies evaluating the role and feasibility of health information systems which are customized to LMIC localities (Ali et al., 2011). In these studies, factors specific to implementation of health information systems and EMRS in LMIC settings should be addressed before such countries can demonstrate their full benefits. Similarly, it is important that existing health information systems take significant steps to improve their data management processes so that quality health data is available timely to all users to improve health services delivery. However, despite the challenges faced by LMIC, it is believed that it is feasible to design, implement and utilize health information systems that fit into local contexts and can contribute to the delivery and monitoring of quality health care (Williams and Boren, 2008).

Footnotes

Conflict of interest

No conflict of interest associated with this work.

Contribution of authors

We declare that this work was done by the authors named in this article and all liabilities pertaining to claims relating to the content of this article will be borne by the authors. The authors conceived and designed the study together, and the first author collected and analysed the data.

References

AbouZahr

Boerma

(2005) Health information systems: the foundations of public health. Bulletin of the World Health Organisation 83 :578-583.

Akande

Monehin

(2004) Health management information system in private clinics in Ilorin, Nigeria. Nigerian Medical Practitioner Journal 46(5): 102–107.

Ali

Shah

Tandon

(2011) Review of electronic decision-support tools for diabetes care: a viable option for low- and middle-income countries? Journal of Diabetes Science and Technology 5(3): 553–570.

Amoroso

Akimana

Wise

Fraser

(2010) Using electronic medical records for HIV care in rural Rwanda. Studies in Health Technology and Informatics 160(Pt 1): 337–341.

Booman

Durrheim

La Grange

Martin

Mabuza

Zitha

Mbokazi

Fraser

Sharp

(2000) Using a geographical information system to plan a malaria control programme in South Africa. Bulletin of the World Health Organization 78(12): 1438–1444.

Bosch-Capblanch

Ronveaux

Doyle

Remedios

Bchir

(2009) Accuracy and quality of immunization information systems in forty-one low income countries. Tropical Medicine and International Health 14 (1 ) : 2–10.

Calle

Saturno

Parra

Rodenas

Perez

Eustaquio

Aguinaga

(2000) Quality of the information contained in the minimum basic data set: results from an evaluation in eight hospitals. European Journal of Epidemioly 16(11): 1073–1080.

Chaudhry

Wang

Maglione

Mojica

Roth

Morton

Shekelle

(2006). Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care. Annals of Internal Medicine 144(10): 12–18.

Corrao

Arcoraci

Arnone

Calvo

Scaglione

Di Bernardo

Lagalla

Caputi

Licata

(2009). Evidence-based knowledge management: an approach to effectively promote good health-care decision-making in the Information Era. Internal and Emergency Medicine 4(2): 99–106.

10.

Diero

Rotich

Bii

Mamlin

Einterz

Kalamai

Tierney

(2006) A computer-based medical record system and personal digital assistants to assess and follow patients with respiratory tract infections visiting a rural Kenyan health centre. BMC Medical Informatics and Decision Making 6: 21.

11.

Farias

Raffo

Bacigalupo

Cremaschi

Vence

Ramos

Salguero

Claudio

Meites

Cubito

(2010) Data for decision making: strategic information tools for hospital management during a pandemic. Disaster Medicine and Public Health Preparedness 4(3): 207–212.

12.

Fraser

HSF

Jazayeri

Nevil

Karacaoglu

Farmer

Lyon

Fawzi

MKCS

Leandre

Choi

Mukherjee

(2004). An information system and medical record to support HIV treatment in rural Haiti. BioMedical Journal 329 : 1142–1146.

13.

Garrib

Stoops

Mckenzie

Dlamini

Govender

Rohde

Herbst

(2008) An evaluation of the District Health Information System in rural South Africa. South African Medical Journal 98(7): 549–552.

14.

Goeree

Levin

Chandra

Bowen

Blackhouse

Tarride

Burke

Bischof

Xie

O'Reilly

(2009) Health technology assessment and primary data collection for reducing uncertainty in decision making. Journal of the American College of Radiology 6(5): 332–342.

15.

Guy

Prybylski

Fairley

Hellard

Kaldor

(2009) Can data from HIV voluntary counselling and testing be used to assess the impact of public health interventions? A literature review. International Journal of STD & AIDS 20(6): 378–383.

16.

Hanmer

Isaacs

Roode

(2010) Factors associated with health information system success: results of a survey of hospitals in South Africa. Studies in Health Technology and Informatics 160(Pt 1): 347–351.

17.

Harper

Edge

Schuster-Wallace

Ar-Rushdi

McEwen

(2011) Improving Aboriginal health data capture: evidence from a health registry evaluation. Epidemiolog & Infection 39(11): 1774–1783.

18.

Haux

(2006) Health information systems past, present, future. International Journal of Medical Informatics 75(3-4): 268–281.

19.

Haux

Knaup

Leiner

(2007) On educating about medical data management - the other side of the electronic health record. Methods of Information in Medicine 46(1): 74–79.

20.

Heunis

Wouters

Kigozi

Engelbrecht

Tsibolane

van der Merwe

Motlhanke

(2011) Accuracy of tuberculosis routine data and nurses' views of the TB-HIV information system in the Free State, South Africa. The Journal of the Association of Nurses in AIDS Care 22(1): 67–73.

21.

Heywood

Rhode

(2001) Using Information for Action: A manual for health workers at facility level. Arcadia, Pretoria: The Equity Project.

22.

Hotchkiss

Aqil

Lippeveld

Mukooyo

(2010). Evaluation of the Performance of Routine Information System Management (PRISM) framework: evidence from Uganda. BMC Health Services Research 10(188). Available at: http://www.biomedcentral.com/1472-6963/10/188.

23.

Jacucci

Shaw

Braa

(2006) Standardization of health information systems in South Africa: The challenge of local sustainability. Information Technology for Development 12(3): 225–239.

24.

Kalogriopoulos

Baran

Nimunkar

Webster

(2009). Electronic medical record systems for developing countries: review. In: Engineering in Medicine and Biology Society. Annual International Conference of the IEEE , Mineapolis, USA, 3-6 September 2009,1730-1733. Available at: http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=5307844 .

25.

Kerr

Norris

(2010) Improving data quality in health care. In: Rodrigues

JJPC

(ed) Health information systems: concepts, methodologies, tools, and applications, Vancouver, Canada, Igi Global, 218–225.

26.

Kerr

Norris

Stockdale

(2007) Data Quality Information and Decision Making: A Healthcare Case Study. In: Proceedings of the18th Australasian Conference on Information Systems: Data Quality in Healthcare, Toowoomba, Australia, 5-7 December 2007: 1017-1026. Available at: http://researchbank.swinburne.edu.au/vital/access/manager/Repository/swin:28320.

27.

Kim

Wang

Lau

Kim

(2004) Application and evaluation of personal health information management system. In: Engineering in Medicine and Biology Society. Annual International Conference of the IEEE , 1-5 September 2004: 3159-3162. Available at: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=1403891.

28.

Kimaro

Nhampossa

(2005). Analyzing the problem of unsustainable health information systems in less-developed economies: Case studies from Tanzania and Mozambique. Information Technology for Development, 11(3): 273–298.

29.

Kimaro

Twaakyondo

(2005) Analysing the hindrance to the use of information and technology for improving efficiency of health care delivery system in Tanzania. Tanzania Health Research Bulletin 7(3): 189–197.

30.

Kotze

McDonald

(2010) An information system to manage the rollout of the antiretroviral treatment programme in the Free State. Curationis 33(2): 60–68.

31.

Krishnan

Nongkynrih

Yadav

Singh

Gupta

(2010) Evaluation of computerized health management information system for primary health care in rural India. BMC Health Services Research 10: 310. Available at: http://www.biomedcentral.com/1472-6963/10/310.

32.

LaFond

Siddiqi

(eds) (2003) Second International RHINO Workshop on: Enhancing the data quality and use of routine health information at district level. September 29-October 4 2003. Eastern Cape, South Africa. Available at: https://www.globalhivmeinfo.org/DigitalLibrary/Digital%20Library/MEASURE-Enhancing%20the%20Quality%20and%20Use%20of%20Routine.pdf.

33.

Lane

Heddle

Arnold

Walker

(2006) A review of randomized controlled trials comparing the effectiveness of hand held computers with paper methods for data collection. BMC Medical Informatics and Decision Making 6 : 23. Available at: http://www.biomedcentral.com/1472-6947/6/23.

34.

Lima

Schramm

Coeli

Da Silva

(2009) Review of data quality dimensions and applied methods in the evaluation of health information systems. Cadernos de Saúde Pública 25(10): 2095–2109.

35.

Lippeveld

Sauerborn

Bodart

(2000) Design and implementation of health information systems. Geneva: World Health Organization.

36.

Lium

Tjora

Faxvaag

(2008) No paper, but the same routines: a qualitative exploration of experiences in two Norwegian hospitals deprived of the paper based medical record. BMC Medical Informatics and Decision Making 8:2. Available at: http://www.biomedcentral.com/1472-6947/8/2.

37.

Ludwick

Doucette

(2009) Adopting electronic medical records in primary care: Lessons learned from health information systems implementation experience in seven countries. International Journal of Medical Informatics 78(1): 22–31.

38.

Martin

Curtis

Fraser

Sharp

(2002) The use of a GIS-based malaria information system for malaria research and control in South Africa. Health & Place 8(4): 227–236.

39.

Mate

Bennett

Mphatswe

Barker

Rollins

(2009) Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS One, 4 (5), e5483. Available at: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0005483.

40.

Merrell

Merriam

Doarn

(2004) Information support for the ambulant health worker. Telemedicine and e-Health 10(4): 432–436.

41.

Mutemwa

(2006) HIMS and decision-making in Zambia: re-thinking information solutions for district health management in decentralized health systems. Health Policy and Planning 21(1): 40–52.

42.

Odhiambo-Otieno

(2005) Evaluation of existing district health management information systems a case study of the district health systems in Kenya. International Journal of Medical Informatics 74(9): 733–744.

43.

Oluoch

Santas

Kwaro

Were

Biondich

Bailey

Abu-Hanna

De Keizer

(2012). The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: A systematic review. International Journal of Medical Informatics 81(10): e83–e92.

44.

Osa-Eloka

Nwakoby

Onajole

Chika

Nwobi

(2009) Enhancing data management skills of primary health care workers in Enugu state, Nigeria. The Nigerian Postgraduate Medical Journal 16(1): 14–20.

45.

Rahimi

Vimarlund

(2007) Methods to evaluate health information systems in healthcare settings: a literature review. Journal of Medical Systems 31(5): 397–432.

46.

Ronveaux

Rickert

Hadler

Groom

Lloyd

Bchir

Birmingham

(2005) The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bulletin of the World Health Organization 83(7): 503–510.

47.

Siika

Rotich

Simiyu

Kigotho

Smith

Sidle

Wools-Kaloustian

Kimaiyo

Nyandiko

Hannan

Tierney

(2005) An electronic medical record system for ambulatory care of HIV-infected patients in Kenya. International Journal of Medical Informatics 74(5): 345–355.

48.

Simba

(2004) Application of ICT in strengthening health information systems in developing countries in the wake of globalization. African Health Sciences 4(3): 195–199.

49.

Smith

Madon

Anifalaje

Malecela

Michael

(2008) Integrated health system in Tanzania: experiences and challenges. Electronic Journal of Information Systems in Developing Countries 33 (1). Available at: http://www.ejisdc.org/ojs2/index.php/ejisdc/article/view/395.

50.

Tan

JKH

(2001) Health management information systems: methods and practical applications, Maryland: Aspen Publishers.

51.

Taylor

Shimp

(2010) Using data to guide action in polio health communications: experience from the Polio Eradication Initiative (PEI). Journal of Health Communication 15(Suppl 1): 48–65.

52.

Van der Veer

de Keizer

Ravelli

Tenkink

Jager

(2010) Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. International Journal of Medical Informatics 79(5): 305–323.

53.

Were

Shen

Bwana

Emenyonu

Musinguzi

Nkuyahaga

Kembabazi

Tierney

(2010) Creation and evaluation of EMR-based paper clinical summaries to support HIV-care in Uganda, Africa. International Journal of Medical Informatics 79(2): 90–96.

54.

World Health Organization Regional Office for the Western Pacific (2003). Improving data quality: A Guide for developing countries. Philippines, WHO.

55.

World Health Organization Regional Office for South-East Asia (2006) 10-Point regional strategy for strengthening health information systems. India WHO.

56.

Williams

Boren

(2008) The role of the electronic medical record (EMR) in care delivery development in developing countries: a systematic review. Informatics in Primary Care 16(2): 139–145.

57.

Wong

Bradley

(2009) Developing patient registration and medical records management system in Ethiopia. International Journal for Quality in Health Care 21(4): 253–258.