Accuracy of smartphone apps for heart rate measurement

Abstract

Background

Smartphone manufacturers offer mobile health monitoring technology to their customers, including apps using the built-in camera for heart rate assessment. This study aimed to test the diagnostic accuracy of such heart rate measuring apps in clinical practice.

Methods

The feasibility and accuracy of measuring heart rate was tested on four commercially available apps using both iPhone 4 and iPhone 5. ‘Instant Heart Rate’ (IHR) and ‘Heart Fitness’ (HF) work with contact photoplethysmography (contact of fingertip to built-in camera), while ‘Whats My Heart Rate’ (WMH) and ‘Cardiio Version’ (CAR) work with non-contact photoplethysmography. The measurements were compared to electrocardiogram and pulse oximetry-derived heart rate.

Results

Heart rate measurement using app-based photoplethysmography was performed on 108 randomly selected patients. The electrocardiogram-derived heart rate correlated well with pulse oximetry (r = 0.92), IHR (r = 0.83) and HF (r = 0.96), but somewhat less with WMH (r = 0.62) and CAR (r = 0.60). The accuracy of app-measured heart rate as compared to electrocardiogram, reported as mean absolute error (in bpm ± standard error) was 2 ± 0.35 (pulse oximetry), 4.5 ± 1.1 (IHR), 2 ± 0.5 (HF), 7.1 ± 1.4 (WMH) and 8.1 ± 1.4 (CAR).

Conclusions

We found substantial performance differences between the four studied heart rate measuring apps. The two contact photoplethysmography-based apps had higher feasibility and better accuracy for heart rate measurement than the two non-contact photoplethysmography-based apps.

Keywords

mHealth heart rate measurement smartphone applications

Introduction

Mobile electronic devices such as smartphones or tablets are gaining increasing popularity worldwide. Recent estimates suggest 7.7 billion mobile broadband subscriptions in 2020, while the number of smartphone subscriptions is expected to equal about 70% of the world’s population.¹ The ubiquitous use of smartphones coupled with expanding mobile broadband connectivity could change the way healthcare is accessed, monitored and delivered (‘mobile health technologies’, mHealth). mHealth is defined by the practice of medicine supported by portable diagnostic devices.² For healthcare systems, the importance of mHealth strategies has been demonstrated as they may play an important role in the control of epidemic disease such as cholera³ or Ebola.⁴ On an individual patient level, mHealth technology can be used, for example, to increase medication adherence^5,6 or control of blood glucose levels and insulin doses in patients with diabetes.⁷ For cardiac patients, a wide array of smartphone-connected cardiac monitoring devices and health apps for the diagnosis and prevention of cardiovascular disease is available.⁸

Overall, it is estimated that the number of health-related wearable sensors will reach 80 million in 2017.⁹ To reach the transformative potential of mHealth, validation of the technical capabilities and accuracy as well as the clinical impact of these technologies is needed.¹⁰ As heart rate monitoring is an essential component of almost all clinical situations, in the present study we evaluated the usability and accuracy of four different heart rate measuring apps. The accuracy of the apps was compared to the electrocardiogram (ECG) and pulse oximeter-derived heart rate using medically approved professional devices.

Methods

Basic principles

Photoplethysmography (PPG) is based on the principle that blood absorbs more light than the surrounding tissue. In addition, variations in blood volume (i.e. in systole and diastole) affect the transmission or reflectance of light.¹¹ These two principles can be used to detect blood flow. The PPG technique is commercially used in pulse oximeters for the determination of arterial blood oxygen saturation (SaO₂), achieved by computing the differences of light absorption in the red and infrared range by oxygenated and deoxygenated haemoglobin.¹² In contrast, heart rate detection (but not SaO₂ measurement) based on PPG is feasible using only light in the visible range.¹¹

Two different concepts of measuring heart rate by PPG are known: contact and non-contact PPG. In contact PPG, the subject places a finger on the built-in camera of the phone. The camera is placed directly on the skin and the built-in flash provides the necessary light source in the visible range for reflection by blood cells (Figure 1(a)). In non-contact PPG, the camera is used in the classic way by holding the camera in front of the patient’s face (up to 1.5 m away) without the need for direct skin contact (Figure 1(b)). There is no need for a dedicated light source, ambient light is sufficient.

Figure 1.

Two different concepts of measuring heart rate by PPG are know: In contact PPG, the subject places a finger on the built-in camera of the phone directly on the skin and the built in flash provides the necessary light source in the visible range for reflection by blood cells (Figure 1a). In non-contact PPG, the camera is used in the classical way by holding the camera in front of the patients face without the need for direct skin contact (Figure 1b). (a) Contact photoplethysmography; (b) non-contact photoplethysmography.

Technology

Applications (apps)

Overall, four commercially available apps have been tested (downloaded from iTunes store). For contact PPG, we used ‘Instant Heart Rate’ (IHR) (version 3.0.1; Azumio Inc., USA) and ‘Heart Fitness’ (HF) (version 2.0.3; Senscare SAS, France). For non-contact PPG, we tested ‘Whats My Heart Rate’ (WMH) (version 2.0; Vitrox Technologies, Malaysia) and ‘Cardiio’ (CAR) (version 2.0; Cardiio Inc., USA).

Mobile devices

We tested the four different apps on two different portable phones (iPhone 4, iPhone 5). The exact camera specifications of the phones are shown in Table 1. The primary camera was used for the contact PPG-based measurements, while the secondary camera was used for all non-contact measurements.

Table 1.

Camera specifications.

Property	iPhone 4 primary camera	iPhone 4 secondary camera	iPhone 5 primary camera	iPhone 5 secondary camera
CMOS sensor	OV5650	Unknown	IMX145-derivative	Omnivision
Sensor format	1/3.2” (4.54 × 3.42 mm)	Unknown	1/3.2” (4.54 × 3.42 mm)	1/6” (∼2.6 × 1.6 mm)
Pixel size	1.75 µm	Unknown	1.4 µm	1.75 µm
Image capture size	2592 × 1936 (5 MP)	0.3 MP	3264 × 2448 (8 MP)	1280 × 960 (1.2 MP)
LED flash	Yes	No	Yes	Yes
Focal length	3.85 mm	Unknown	4.10 mm	2.2 mm
Aperture	F/2.8	Unknown	F/2.4	F/2.4

Source: AnandTech.¹⁸

Study population

From April to December 2013 we randomly included adult patients requiring heart rate monitoring on the chest pain unit or the emergency room of the University Hospital Zurich, Switzerland. All patients willing to participate in the study were eligible; however, patients in critical medical conditions were excluded from this study. Patients participating in this study gave written informed consent. The study was approved by the ethics committee of the Canton of Zurich, Switzerland (KEK-ZH-NR: 2012-0524).

After recruitment, initial heart rate and rhythm was measured using a 12-lead ECG (Schiller AT-104 PC/SDS 101 and Schiller AT 102 Plus; Schiller AG, Baar, Switzerland). Heart rate was then measured simultaneously by pulse oximetry (Draeger Infinity Delta XL; Draegerwerk AG & Co. KG, Luebeck, Germany), an ECG-based monitor (Philips Intellivue X2; Koninklijke Philips NV, Amsterdam, the Netherlands) and by each app. The patient history was taken from medical records, and additional clinical characteristics were collected (age, body temperature, blood pressure and blood oxygen saturation). The graphic conceived by Jubran¹³ was used for grading of the pulse oximetry curve quality.

Statistics

Continuous data are expressed as medians and interquartile ranges (IQRs) or as mean ± standard deviation (SD) as appropriate, and categorical data as number and percentage (%). Categorical data were analysed using the chi-square test. To compare the values provided by the four apps to the ECG and oximeter-derived heart rate we used Pearson’s correlation and Bland–Altman analysis. To assess the level of agreement between methods we computed the mean absolute error.¹⁴ Linear and multiple regression analyses were performed to test the influence of the defined factors on the mean absolute error of the four different apps. A P value of less than 0.05 was considered statistically significant. Statistical analyses were performed using Stata SE, version 13.

Results

Clinical characteristics

We included a total of 108 patients. In total, 432 datasets (each consisting of an app-based measurement, a simultaneous ECG-based measurement and simultaneous pulseoxymetry-based measurement for each patient) were obtained. Nineteen datasets were excluded due to missing values (due to either the app-based measurement or ECG being unable to give a value). Median patient age was 65 years (range 19–99 years, IQR 52–76 years); 31% of patients were women. During the measurements, median systolic blood pressure was 126 mmHg (range 76–189 mmHg, IQR 112–142 mmHg), body temperature ranged from 35.0℃ to 37.5℃ (median 36.0℃). Median arterial oxygen saturation measured by pulse oximetry was 96% (range 62–100%, IQR 94–98%). Sinus rhythm was present in 85 patients (79%), atrial fibrillation in 11 patients (10%), ventricular pacemaker stimulation in five patients (5%) and atrial tachycardia/third-degree AV-block/junctional escape rhythm each in one patient (1%). The baseline characteristics of the study population are given in Table 2. In a total of 413 measurements by ECG we recorded a wide heart rate range from 31 to 124 bpm (median 75 bpm, IQR 62–90 bpm). The quality of the oximetry pulse curve was judged to be satisfactory in 86% (IHH measurements), 96% (HF measurements), 80% (WMH measurements) and 69% (CAR measurements) of the cases.

Table 2.

Baseline characteristics of patients (n = 108).

Age	65 (52–76)
Heart rate, beats per minute^a	75 (62–90)
Systolic blood pressure (mmHg)^a	126 (112–142)
Body temperature (℃)^a	36 (35.0–37.5)
Arterial oxygen saturation (%)^a	96 (94–98)
Female gender, n (%)	35 (31)
Heart rhythm
Sinus rhythm, n (%)	85 (79)
Atrial fibrillation, n (%)	11 (10)
Pacemaker rhythm, n (%)	5 (5)
Other, n (%)	7 (6)

Data given as median (IQR 25–75).

ECG measurements compared to pulse oximetry measurements with approved medical devices

ECG-based measurements compared to heart rate measurement using the pulse oximetry device showed a very high level of agreement (Figure 2, Table 3). Pearson correlation r between the ECG determined heart rate and the oximetric heart rate was 0.92 (P < 0.001), showing a mean absolute error of 2 ± 0.35 bpm.

Figure 2.

Heart rate measurement by electrocardiogram compared to pulse oximetry device.

Table 3.

Comparison of heart rate assessed by the four different apps and medically approved pulse oximetry compared to electrocardiogram.

Measurement method	Mean absolute error ( ± SD)	Pearson’s r
Instant Heart Rate (IHR)	4.52 (1.12)*	0.83
Heart Fitness (HF)	1.96 (0.48)*	0.96
Whats My Heart Rate (WMH)	7.08 (1.39)*	0.62
Cardiio (CAR)	8.11 (1.38)*	0.61
Pulse oximetry	2.0 (0.35)*	0.92

P < 0.001.

ECG measurements compared to app-based measurements

The accuracy of heart rate measured by apps as compared to ECG, reported as mean absolute error (in bpm ± standard error) was 4.5 ± 1.1 (IHR), 2.0 ± 0.5 (HF), 7.1 ± 1.4 (WMH) and 8.1 ± 1.4 (CAR). The ECG-derived heart rate correlated well with IHR (r = 0.83) and HF (r = 0.96), but less with WMH (r = 0.62) and CAR (r = 0.60) (see Table 3). The Bland–Altman plots (Figure 3) show considerable differences between the tested apps. Non-contact PPG measurements performed significantly worse compared to fingertip-based (contact) measurements. Both non-contact PPG-based apps performed significantly worse at higher heart rates. They also have a tendency to underestimate higher heart rates.

Figure 3.

The Bland-Altman plots show considerable differences between the tested apps. Non-contact PPG measurements performed significantly worse compared to contact measurements. Both non-contact PPG based apps performed significantly worse at higher heart rates. They also have a tendency to underestimate higher heart rates. (a) Contact photoplethysmography; (b) non-contact photoplethysmography.

Influencing factors in app-based measurements

Using multiple regression analyses, we were unable to demonstrate any relationship between influencing factors and mean absolute error for the contact PPG apps IHR and HF (Table 4). However, the two non-contact PPG apps performed worse, with declining body temperature and increasing heart rate. Furthermore, camera technology tended to influence the performance of the non-contact PPG apps.

Table 4.

Influencing factors in App-based measurements.

	Simple regression		Multiple regression
	Value	P value	Value	P value
Instant Heart Rate (IHR)
Age	0.0580*	(0.0390)	0.0482	(0.145)
Temperature	−0.0584*	(0.0206)	0.00956	(0.896)
iPhone 5	−2.778	(0.153)	−3.497	(0.132)
Heart rate	0.000704	(0.989)	−0.0298	(0.651)
Atrial fibrillation	0.334	(0.860)	1.592	(0.576)
Male gender	3.161	(0.105)	3.909	(0.0909)
Systolic blood pressure	−0.0914	(0.123)	−0.101	(0.141)
Heart Fitness (HF)
Age	0.00354	(0.770)	−0.00573	(0.606)
Temperature	−0.0108	(0.204)	−0.15	(0.0852)
iPhone 5	−0.441	(0.638)	−0.215	(0.764)
Heart rate	0.0320	(0.319)	0.0223	(0.521)
Atrial fibrillation	5.312	(0.129)	7.322	(0.0680)
Male gender	−0.701	(0.561)	−0.624	(0.580)
Systolic blood pressure	−0.0181	(0.469)	−0.000145	(0.991)
Whats My Heart Rate (WMH)
Age	−0.0473	(0.611)	0.00385	(0.953)
Temperature	−0.0774	(0.0669)	−0.359*	(0.00153)
iPhone 5	−6.892*	(0.00249)	−4.198*	(0.0329)
Heart rate	0.406*	(0.000135)	0.390*	(0.000168)
Atrial fibrillation	2.223	(0.719)	−2.041	(0.645)
Male gender	−0.939	(0.768)	−0.950	(0.711)
Systolic blood pressure	−0.105	(0.123)	−0.0614	(0.282)
Cardiio (CAR)
Age	−0.0671	(0.382)	−0.0447	(0.510)
Temperature	−0.119*	(0.000167)	−0.374*	(0.00104)
iPhone 5	−6.044*	(0.0162)	−4.394	(0.0642)
Heart rate	0.336*	(0.00205)	0.294*	(0.0147)
Atrial fibrillation	0.989	(0.781)	2.081	(0.643)
Male gender	−2.074	(0.505)	−0.352	(0.895)
Systolic blood pressure	−0.168*	(0.0226)	−0.0937	(0.115)

P < 0.05.

Discussion

The widespread use of smartphones, expanding mobile broadband connectivity and the emergence of new mHealth technologies will result in an increase of collection (and transmission) of individualised health-related data. The potential to increase patient engagement, to reduce healthcare costs and to improve outcomes is remarkable. Nevertheless, establishing reliable and clinically meaningful digital health data is problematic: in the present study, the performance of different apps for heart rate measurements by smartphone (without any external connectable sensor device) was very heterogeneous. While one app (HF) measured heart rate with comparable accuracy to pulse oximetry, other apps did not perform as well. In some apps, absolute differences of over 20 bpm (or even more) occurred in more than 20% of all measurements (see Table 5). The precise reason for WMH and CAR performing slightly worse is unclear. Theoretically they should be performing similarly. We could not reliably identify an influencing factor. The two non-contact PPG apps performed significantly worse compared to fingertip-based (contact PPG) apps. This may be to due to inferior camera technology (in both apps the front camera with less advanced specifications was used) as well as generally a more demanding measurement environment. Uncontrolled ambient light may make it more difficult to discriminate between two distinct peaks in the pulse wave. Furthermore, the accuracy of non-contact PPG apps was significantly worse at higher heart rates, which is in line with previously published data.^15,16 In a different more recent study comparing the accuracy of wrist-worn heart rate monitors using contact PPG technology, no such effect was observed.¹⁶ One possible explanation for the worse accuracy of non-contact PPG could be the inferior resolution of the secondary camera. The trend to more reliable performance in next generation devices (iPhone5 vs. iPhone4) observed in the present study could prognosticate better performance in the future using more advanced camera technology.

Table 5.

App-performance grouped by differences (bpm) to ECG.

	Instant Heart Rate (IHR)	Heart Fitness (HF)	Whats My Heart Rate (WMH)	Cardiio (CAR)
Difference to ECG (bpm)	Percentage of measurements
>10	13	6	19	20
>20	7	4	14	15
>40	2	3	9	5

While some apps are developed using evidence-based guidelines and are continuously evaluated,¹⁷ most remain unevaluated. The sheer number of unregulated apps can lead to consumer confusion, reluctant promotion by healthcare providers and unnecessary consumption of healthcare resources. It is largely unclear how we should develop the resources necessary for administrating digital health services and the requirement for healthcare personnel to monitor the wave of incoming patient-generated data. Besides these logistic aspects, several other issues are problematic and need to be addressed (medico-legal liability, personal data safety, health insurance collaborations, reimbursement, incorporation in governmental healthcare systems/programmes, etc.).

Nevertheless, mHealth technologies potentially offer amazing opportunities for patients, healthcare providers, researchers and healthcare delivery systems. To identify the most effective and robust technologies for clinical use it is mandatory to create an evidence base that validates generated measurements and assesses the impact of specific mHealth products and concepts on healthcare quality, cost and outcomes.

Conclusion

We found substantial performance differences between the four studied heart rate measuring apps. The two contact PPG-based apps had higher feasibility and better accuracy for heart rate measurement than the two non-contact PPG-based apps. Careful analysis of app accuracy is warranted before using these apps in clinical practice.

Footnotes

Author contribution

TC, CT, CAW, AB, PB, JH and TFL contributed to the conception of the work. TC, CT, CAW, AB, AAT, FS and SM contributed to the acquisition, analysis, or interpretation of data for the work. TC, CAW and PB drafted the manuscript. All authors critically revised the manuscript, gave final approval and agree to be accountable for all aspects of work ensuring integrity and accuracy.

Acknowledgements

The use of the phones in the study was provided by Swisscom for free. At the end of the study, all data were deleted on the phones, and all phones were returned to Swisscom. Measurements were performed by the medical and nursing staff of the cardiology clinic at the University Hospital of Zurich.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

The Broadband Commission for Digital Development. The State of Broadband 2015. 2015; http://www.broadbandcommission.org/doccuments/reports/bb-annualreport2015.pdf.

Bhavnani

Narula

Sengupta

. Mobile technology and the digitization of healthcare. Eur Heart J 2016; 37: 1428–1438–1428–1438.

Bengtsson

Thorson

. Improved response to disasters and outbreaks by tracking population movements with mobile phone network data: a post-earthquake geospatial study in Haiti. PLoS Med 2011; 8: e1001083–e1001083.

O’Donovan

Bersin

. Controlling Ebola through mHealth strategies. Lancet Global Health 2015; 3: e22–e22.

Anglada-Martinez

Riu-Viladoms

Martin-Conde

. Does mHealth increase adherence to medication? Results of a systematic review. Int J Clin Pract 2015; 69: 9–32.

Dale

Dobson

Whittaker

. The effectiveness of mobile-health behaviour change interventions for cardiovascular disease self-management: a systematic review. Eur J Prev Cardiol 2016; 23: 801–817.

Årsand

Muzny

Bradway

. Performance of the first combined smartwatch and smartphone diabetes diary application study. J Diabet Sci Technol 2015; 9: 556–563.

Nguyen

Silva

. Use of smartphone technology in cardiology. Trends Cardiovasc Med 2016; 26: 376–386.

Swan

. Sensor mania! the internet of things, wearable computing, objective metrics, and the quantified self 2.0. J Sensor Actuator Netw 2012; 1: 217–253.

10.

Bruining

Caiani

Chronaki

. Acquisition and analysis of cardiovascular signals on smartphones: potential, pitfalls and perspectives. Eur J Prev Cardiol 2014; 21(2 Suppl): 4–13.

11.

Verkruysse

Svaasand

Nelson

. Remote plethysmographic imaging using ambient light. Optics Express 2008; 16: 21434–21445.

12.

Mendelson

. Pulse oximetry: theory and applications for noninvasive monitoring. Clin Chem 1992; 38: 1601–1607.

13.

Jubran

. Pulse oximetry. Crit Care 1999; 3: R11–R17.

14.

Willmott

Matsuura

. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Res 2005; 30: 79–79.

15.

Wackel

Beerman

West

. Tachycardia detection using smartphone applications in pediatric patients. J Pediatr 2014; 164: 1133–1135.

16.

Wang R, Blackburn G, Desai M, et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol 2017; 2: 104–106.

17.

McManus

Chong

Soni

. PULSE-SMART: Pulse-based arrhythmia discrimination using a novel smartphone application. J Cardiovasc Electrophysiol 2016; 27: 51–57.

18.

Lal Shimpi A, Klug B and Gowri V. The iPhone 5 Review. AnandTech; 2012. http://www.anandtech.com/show/6630/the-iphone-5-review (accessed 20 March 2017).