This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
As the sensing capabilities of wearable devices improve, there is increasing interest in their application in medical settings. Capabilities such as heart rate monitoring may be useful in hospitalized patients as a means of enhancing routine monitoring or as part of an early warning system to detect clinical deterioration.
To evaluate the accuracy of heart rate monitoring by a personal fitness tracker (PFT) among hospital inpatients.
We conducted a prospective observational study of 50 stable patients in the intensive care unit who each completed 24 hours of heart rate monitoring using a wrist-worn PFT. Accuracy of heart rate recordings was compared with gold standard measurements derived from continuous electrocardiographic (cECG) monitoring. The accuracy of heart rates measured by pulse oximetry (Spo2.R) was also measured as a positive control.
On a per-patient basis, PFT-derived heart rate values were slightly lower than those derived from cECG monitoring (average bias of −1.14 beats per minute [bpm], with limits of agreement of 24 bpm). By comparison, Spo2.R recordings produced more accurate values (average bias of +0.15 bpm, limits of agreement of 13 bpm,
Personal fitness tracker–derived heart rates were slightly lower than those derived from cECG monitoring in real-world testing and not as accurate as Spo2.R-derived heart rates. Performance was worse among patients who were not in sinus rhythm. Further clinical evaluation is indicated to see if PFTs can augment early warning systems in hospitals.
ClinicalTrials.gov NCT02527408; https://clinicaltrials.gov/ct2/show/NCT02527408 (Archived by WebCite at http://www.webcitation.org/6kOFez3on)
Over the last 5 years, consumer interest in self-monitoring and personal health tracking has grown considerably [
There has been growing enthusiasm for the potential use of wearable devices to improve health care delivery [
Current clinical uses for wearable devices are mostly limited to outpatient and ambulatory settings, with a focus on the management of chronic diseases [
There is ample opportunity to leverage the sensing capabilities of wearable devices in the inpatient setting as well. Many newer wearable devices use photoplethysmography (PPG) to record heart rate by measuring differential reflection of light from the skin, based on the pulsatility of superficial blood vessels [
The ability of wearable PPG sensors to reliably measure heart rate in the outpatient population has been demonstrated in at least one study [
In order to address these and other questions, we examined the accuracy of heart rate measurements derived from PFTs. We focused on patients in the ICU as this cohort is closely monitored using continuous ECG (cECG) monitoring, which provides a gold standard measurement of heart rate. Because the degree of agreement that would be sufficient for clinical applications is not well defined, we also examined the agreement between cECG-derived heart rate measurements and a more widely accepted method of heart rate measurement, namely, pulse oximetry (Spo2) monitoring.
We used the Fitbit Charge HR (Fitbit, San Francisco, CA) PFT to monitor heart rate in 50 patients admitted to the ICU at Kingston General Hospital (KGH), a tertiary academic medical center in Ontario. The 33-bed ICU at KGH is a mixed medical, surgical, trauma, and neurosciences unit. The PFT device studied is a wrist-worn device resembling a watch, which uses PPG to detect periodic changes in blood flow beneath the sensor, thereby deriving heart rate measurements. Heart rate values are recorded every 5 minutes. The Fitbit Charge HR is a commercially available PFT and is not currently regulated by the US Food and Drug Administration.
In order to study a cohort of patients that would best resemble hospital ward patients, we included only stable patients who were not receiving mechanical ventilation, continuous analgesia, or sedation. To reduce the risk of transmitting nosocomial infections, we excluded patients under contact precautions for methicillin-resistant
The study used 6 separate PFTs (3 size large, 3 size extra-large), each of which was assigned a unique email address and log-in credentials for the Fitbit website. An automated R script was used to download and process PFT data from the Fitbit website. Heart rate data are recorded by the PFT every 5 minutes. To provide a gold standard measurement of heart rate, we recovered data from the ICU bedside monitors using specialized software (BedMasterEX, Excel Medical, Jupiter, FL). Data included heart rate values, as well as heart rate data derived from continuous Spo2 monitoring (Spo2.R), both recorded every minute. These data were acquired as XML files and processed using an automated Python script to derive minute-level heart rate data. We synchronized bedside monitor data and PFT data using a correction factor that accounted for the difference between each device’s internal clock.
We analyzed heart rate data in aggregate across all patients, as well as on a per-patient basis. We determined the difference between cECG and PFT readings measured simultaneously, the median of differences over a 24-hour period, the interquartile range (IQR) of differences, and the Pearson correlation coefficient between cECG and PFT measurements. We used a Wilcoxon signed rank test to determine if the distribution of cECG-derived heart rates differed from that of the PFT-derived heart rates. Finally, we used Bland-Altman analysis to measure the agreement between the PFT and cECG methods of heart rate monitoring, as well as the bias of the PFT relative to cECG.
We calculated all the above-mentioned metrics for cECG-Spo2.R pairs in order to compare the accuracy of PFT measurements with that of a well-established and widely used alternative for heart rate measurement. On the basis of the mechanism of sensing used by the PFTs, we hypothesized that accuracy would differ in patients not in sinus rhythm and conducted a subgroup analysis to test this effect. Rhythm status was based on examination of cECG recording both at the time of device application and at the time of device removal, with patients designated as being in sinus rhythm only if this was present at both time points. To assess the potential for degradation in PFT performance over time, we compared the accuracy in the first 20 patients with that of the last 20 patients.
The study was approved by the Health Sciences Research Ethics Board of Queen’s University (DMED-1818-15) and is registered with ClinicalTrials.gov (NCT02527408). Patients or their substitute decision makers provided informed consent. All study data were deidentified. The study did not receive funding from the device manufacturer or from any other source. All statistical analyses were done using R (v 3.2.2).
Between August 2015 and January 2016, we enrolled a convenience sample of 50 patients meeting our enrollment criteria. Patients were admitted with a variety of medical and surgical conditions and were of low clinical acuity at the time of monitoring (
Characteristics of patients included in the study (N=50)
Characteristics | Values | |
Mean heart rate, beats per minute | 88.3 | |
Mean age, years | 64 | |
Male | 26 (52) | |
Female | 24 (48) | |
Respiratory | 12 (24) | |
Sepsis | 7 (14) | |
Surgical | 7 (14) | |
Neurologic | 11 (22) | |
Trauma | 3 (6) | |
Cardiovascular | 6 (12) | |
Medical | 4 (8) | |
At start of monitoring | 43 (86) | |
At end of monitoring | 42 (84) | |
Large | 23 (46) | |
Extra large | 27 (54) |
The PFT device was removed prematurely in 2 cases; in one case a patient was discharged from the ICU early, and in another case a patient developed a diffuse drug rash. In 4 cases cardiorespiratory monitoring was discontinued early, resulting in incomplete comparison data. Personal fitness tracker devices in these cases continued to collect data for the full 24-hour period. Excluding the 2 patients whose devices were removed early, PFTs showed a high degree of data capture (mean 98% of eligible data).
We analyzed a total of 12,358 cECG-PFT heart rate pairs and 56,385 cECG-Spo2.R heart rate pairs. Most of the 24-hour heart rate recordings conformed to a skewed or bimodal distribution (
Scatterplots for individual patients are provided in
Results of the pooled analysis comparing continuous electrocardiogram (cECG)-derived heart rates and personal fitness tracker (PFT)-derived heart rates (in red), as well as cECG-derived heart rates and pulse oximetry heart rates (SpO2.R, in blue). A and B, Scatterplots showing simultaneous heart rate measurements from cECG (x-axis) compared with alternative methods (y-axis). C and D, Bland-Altman plots for heart rate measured by PFT and SpO2.R compared with cECG. Mean heart rate is shown on the x-axis, with the difference between heart rates shown on the y-axis. The solid horizontal line represents a difference between measurements of 0, while the dashed lines represent the observed mean difference (bias) and limits of agreement. HR: heart rate; bpm: beats per minute.
Results of the per-patient analysis of heart rate recording accuracy.
Variable | PFTa | Spo2.Rb | ||
Pairs | 281 | 1328 | <.001 | |
Zeros (median) | 0 | 0 | <.001 | |
Median difference (bpmc) | 0 | 0 | .003 | |
Interquartile range (bpm) | 4 | 1 | <.001 | |
Correlation coefficient | .57 | .89 | <.001 | |
Wilcoxon | 1.52E−06 | 5.06E−11 | .57d | |
Bias (bpm) | −1.14 | 0.15 | <.001 | |
Limits of agreement (bpm) | 23.88 | 13.00 | <.001 |
aPFT: personal fitness tracker.
bSpo2.R: pulse oximetry heart rate.
cbpm: beats per minute.
dcomparing the number of recordings with Wilcoxon
Our subgroup analyses compared 8 patients who were not in sinus rhythm with 40 patients in sinus rhythm. Median heart rate difference, IQR, Pearson correlation coefficient, bias, and limits of agreement were all significantly worse in patients with rhythms other than sinus (
Heart rate measurement accuracy in patients in sinus rhythm compared with those not in sinus rhythm.
Measurement | Sinus rhythm (n=40) | Nonsinus rhythm (n=8) | |
Median difference (bpma) | 0 | 3.5 | .04 |
Interquartile range (bpm) | 4 | 8.6 | .01 |
Correlation coefficient | .58 | .23 | <.001 |
Bias (bpm) | −0.99 | −5.02 | .02 |
Limits of agreement (bpm) | 22.9 | 46.4 | .049 |
abpm: beats per minute.
Summary of the per-patient analysis of the differences between personal fitness tracker (PFT, shown in red) and pulse oximetry heart rate values (SpO2.R, shown in blue) as compared with continuous electrocardiogram (cECG)–derived heart rate. A, Number of heart rate pairs analyzed. B, Number of zero measurements recorded. C, Median difference between the device-derived heart rate and cECG-derived heart rate. D, Interquartile range (IQR) of the differences. E, Pearson correlation coefficient (PFT vs cECG and SpO2.R vs cECG). F, P values indicating the likelihood that the distribution of heart rate values derived from the other devices differed from that derived from cECG (Wilcoxon signed rank test). G, Mean difference between device-derived heart rate and cECG-derived heart rate from Bland-Altman analysis. H, Limits of agreement between device-derived heart rate and cECG-derived heart rate from Bland-Altman analysis. For each boxplot, individual patients are represented by an individual point. All comparisons showed statistically significant differences by Wilcoxon rank sum test (
Scatterplots showing simultaneous heart rate (HR) measurements derived from continuous electrocardiogram (cECG; x-axis) compared with heart rate from personal fitness tracker (PFT, left) and pulse oximetry (SpO2.R, right) in a patient with atrial fibrillation. In the absence of normal sinus rhythm, the PFT consistently underestimated heart rate.
Patient recordings arranged according to median difference between personal fitness tracker–derived heart rate and heart rate from continuous electrocardiogram. The least accurate recordings were from patients who were not in sinus rhythm. HR: heart rate.
Although the use of wearables in health care has garnered considerable attention in recent years, few objective studies exist resulting in a substantial dearth of clinical evidence regarding their use. A recent PubMed search of the term “wearable technology” revealed nearly 1000 articles published in the last 5 years, only 3% of which were clinical trials [
We found that, overall, Fitbit PFT-derived heart rate measurements were less accurate and consistent than heart rate values recorded by continuous pulse oximetry. There was, however, considerable between-patient heterogeneity, with PFT heart rate values proving highly accurate in some cases and less so in others. With heart rate values analyzed on a per-patient basis, the differences between the PFT and pulse oximetry methods were less pronounced. The accuracy of PFT-based heart rate monitoring was poor among patients not in sinus rhythm.
Our results show that, on average, the PFT devices tested tended to underestimate heart rate values slightly, particularly with heart rate values in the range of 75 to 120 bpm. The clinical implications of this degree of bias are uncertain and likely depend on the intended purpose of the monitoring. A difference of the magnitude observed might be acceptable for detecting acute clinical deterioration, which is often accompanied by marked changes in heart rate, but may not be adequate for identifying more subtle physiological derangements.
Wrist-worn heart rate sensing devices have the potential to enhance inpatient safety by identifying episodes of clinical deterioration faster than current nurse-driven vital signs monitoring practices allow. With only a small minority of hospitalized patients receiving cECG monitoring in intensive care settings, most have heart rate measurements taken only 2 to 3 times in a 24-hour period. Early warning systems (EWSs) have been shown to accurately predict cardiac arrest and hospital mortality, with some studies suggesting a reduction in these events following EWS implementation [
There are a number of potential advantages to augmenting hospital monitoring practices using wrist-worn PPG-based heart rate sensors such as the one we studied. A cost advantage may be achievable given that the device we tested retails for approximately US $170 and that we were able to reuse devices on average 9 times without seeing a decrement in performance. By comparison, conventional heart rate monitoring on inpatient wards (ie, telemetry) has been estimated to incur expenses of just under US $40 per patient per day in direct costs and as much as an additional US $170 in opportunity costs [
The use of wrist-worn devices for heart rate monitoring in hospitals also has potential disadvantages. Consumer-grade PFTs do not provide information regarding respiratory rate or blood pressure, both of which have been shown to add value in EWS [
Our study has a number of strengths. We examined the use of PFTs in a sizeable cohort of hospitalized patients under real-world conditions. Devices were adjusted only once at the time of application and were not reassessed for the duration of the 24-hour recording period by either study personnel or clinical staff. We used high-frequency data captured from continuous bedside monitoring to provide an accurate gold standard assessment of heart rate and analyzed PFT performance on both a pooled and per-patient level.
One of the potential limitations of our study arises from the fact that the PFT-derived and cECG-derived heart rate values were obtained from different devices, with separate internal clocks. Although correction factors were used to synchronize the time stamps from the 2 heart rate sources, it is possible that in some cases the heart rate values that were treated as simultaneous were in fact separated by a short time interval. As the PFT device only recorded heart rate measurements every 5 minutes—an interval longer than the maximal device time discrepancy observed—the impact of any potential asynchrony was likely minimal.
Our study was conducted in the ICU, where cECG monitoring provides a gold standard comparator for heart rate. The extent to which our results can be generalized to hospitalized patients on the wards is therefore not certain; however, all patients enrolled were stable and were receiving ward-level care at the time of monitoring. Finally, our subgroup analysis included a relatively small number of patients not in sinus rhythm, thereby limiting the statistical power of the results.
Our study used one particular type of PFT, namely, the Fitbit Charge HR. Although many consumer-grade PFTs have similar intended functionality and use similar heart rate sensing technology, our results cannot necessarily be generalized to other wearable devices. Given that the different performance characteristics of various PFTs are not known, a study in which a mix of devices is used would be vulnerable to unwarranted mixing of effects or would require an increase in sample size proportional to the number of different devices tested.
Our study is the first to report on the accuracy of heart rate recordings from wearable devices among hospital inpatients. Previous work has focused on the technical and engineering aspects of PPG-based wearable heart rate sensors, as well as discussion of their potential uses in health care settings [
The health care sector is expected to drive a large proportion of sales of wearable devices in the coming years [
Frequency distributions for continuous electrocardiogram–derived heart rate (black), personal fitness tracker heart rate (red), and pulse oximetry heart rate (green) for each patient.
Table showing the results of the pooled analysis of heart rate comparisons.
Histogram showing the distribution of the obtained heart rate differences (x-axis) of both personal fitness tracker and pulse oximetry (SpO2) when compared with continuous electrocardiogram, in beats per minute.
Scatterplots (continuous electrocardiogram–derived heart rate vs device-derived heart rate) for each patient.
beats per minute
electrocardiogram
early warning system
continuous electrocardiogram
intensive care unit
interquartile range
Kingston General Hospital
personal fitness tracker
photoplethysmography
pulse oximetry
pulse oximetry heart rate
DMM and JGB are supported by funding from the Southeastern Ontario Academic Medical Organization (SEAMO).
RRK collected data for the study, interpreted the results, and helped draft the manuscript. DMM and JGB developed the study concept, developed the trial design, and interpreted the results. DMM collected data for the study, performed the data analysis, and helped draft the manuscript. All the authors contributed to revisions of the manuscript and approved the final version.
None declared.