Failure to recognize and act on physiological indicators of deterioration in hospital wards is a well-recognized problem [1-3]. Current practice involves manual measurement of vital signs and the use of early warning scores. However, the acquisition of these measurements can be time consuming for health professionals in a busy ward [4]; therefore, the desired monitoring frequency of observations is often not achieved [5]. This is identified as a limitation of periodic monitoring systems, as they are dependent on the frequency of vital sign observations [6]; of more concern is that even when the ideal frequency is achieved, patients might deteriorate between observation sets [7]. Continuous vital sign monitoring using wearable devices may overcome these limitations and improve timely detection of deterioration [8-10]. However, recent observational studies and pilot randomized controlled trials of wearable monitoring devices were inconclusive, and large randomized clinical trials of ambulatory monitoring devices are required to demonstrate improved patient outcomes [1,8,11,12]. This may be because of limitations in currently available ambulatory monitoring devices; for instance, it has been demonstrated that the detection and reduction of noise and artifacts during movement can be challenging in this type of monitoring [13]. Therefore, there is a need for diagnostic accuracy studies addressing motion artifacts to support the development of reliable wearable devices [3,9,14,15].

The VitalPatch [16] is a wireless, battery-operated chest-worn wearable biosensor that records heart rate (HR), electrocardiography (ECG), HR variability, R-R interval (the time elapsed between two successive R-waves of the QRS signal on the electrocardiogram), respiratory rate (RR), body temperature, skin temperature, fall detection, activity (including step count), and posture (body position relative to gravity including fall detection) [17,18]. This device (Figure 1) was well tolerated by users because of its wearability and low impact on activities [19,20]. However, validation studies regarding the VitalPatch showed mixed results [3]; for HR, the mean bias was within acceptable ranges for all validation studies [14,18,21-23] and within acceptable limits of agreement (LoA) in 3 studies [18,21,22]. For RR, the mean bias was also acceptable; however, all studies reported LoA outside of the clinically acceptable range [14,18,21-23].

This work is a substudy of our virtual high dependency unit (vHDU) hypoxia study [24]. In the main protocol, our goal was to determine the specificity and sensitivity of currently available ambulatory pulse oximeters for the detection of hypoxemia. In this substudy, the aim is to assess the VitalPatch estimation accuracy during several simulated movements and in controlled hypoxic conditions against gold standard measurements, ECG for HR [25] and capnography for RR [26].

The primary outcome is accuracy, defined as the mean absolute error (MAE) of the VitalPatch HR and RR estimation versus gold standard recording of HR (3-lead ECG) and RR (capnography) in healthy volunteers during movement and during induced hypoxia (down to peripheral oxygen saturation [SpO₂] of 80%). In line with the American National Standards Institute/Association for the Advancement of Medical Instrumentation (ANSI/AAMI) EC13:2002 standard [27], we defined the MAE of the patch estimates to be clinically acceptable if the rates are within 5 beats per minute (bpm) and 3 respirations per minute (rpm) [3,21,22,27].

This was a prospective, observational, cross-sectional cohort study that validated the HR and RR of a single-lead ECG chest patch. The VitalPatch HR is computed via the detection of Q wave, R wave, and S wave complexes from a single-lead ECG waveform. The RR is derived from combined information from three sources, that is, an embedded algorithm uses a weighted average of two characteristics of the ECG signal: (1) Q-, R-, and S-wave amplitude modulation and (2) respiratory sinus arrhythmia (as both ECG-derived signals change during inspiration and expiration), and (3) changes in the accelerometer signal induced by chest movement during respiration [23,29,30]. We compared the HR and RR estimates with the gold standard HR derived from 3-lead ECG and RR determined from capnography measurements, respectively. The protocol for the full study has been published [24].

The screening process and inclusion and exclusion criteria are described in the study protocol [24]. Participants were recruited consecutively from the Oxford area, United Kingdom, between June 18 and August 8, 2019. The study sessions took place at the Cardiovascular Clinical Research Facility, John Radcliffe Hospital, Oxford, United Kingdom.

After the participants provided written informed consent, an arterial line was inserted into the participant’s nondominant radial artery. The participants were then fitted with a VitalPatch in a modified lead-II configuration on the left midclavicular line over the intercostal space (as recommended by the vendor), a nonambulatory Philips MX 450 pulse oximeter (dominant arm), a 3-lead ECG (standard configuration) for acquisition of HR, and a nasal cannula for acquisition of RR, both connected to the Philips monitor. The participants were continuously monitored with these devices throughout the study session by trained health care professionals with appropriate resuscitation facilities.

For each participant, the following information was recorded in the case report form: age, sex, height, weight, skin type (Fitzpatrick scale [31]), baseline HR (Philips MX 450 3-lead ECG), and arterial oxygen saturation. Data from all participants were identified using a study number.

All data collection devices (wearables, tablets, laptops, and physiological monitors), and respective software, were synchronized to the same time server at the start of each study session day. Continuous HR and RR data (sampled at 0.25 Hz) from the VitalPatch were sent via Bluetooth Low Energy to an Android tablet–based data capture system, developed in-house. Once the VitalPatch is connected to the tablet, it updates its internal clock and timestamps each sample internally. The Phillips MX 450 3-lead ECG data were collected in real time on a laptop using the ixTrend version 2.1 software (ixcellence GmbH) [32]. For each step of the movement phase and prespecified oxygen saturation level of the hypoxia phase, the corresponding start and stop time stamps were recorded both in the in-house data capture software and on the respective case report forms [24].

The flow diagram of the data set collected in this study is shown in Figure 2. A total of 42 participants were recruited, of which 29 (69%) were included in our analysis. The demographic descriptors are listed in Table 1. Exemplar HR and RR traces collected in the movement and hypoxia phases for one participant are shown below.

The movement phase lasted approximately 13 minutes for each participant. The exemplar HR and RR traces shown in Figure 3 confirm higher variability during the STS movement. The scatter and Bland-Altman plots for HR and RR in Figure 4 also show a higher mean bias and LoA for the dynamic activity (STS) versus one of the sitting movements (using a tablet). Scatter and Bland-Altman plots for all movement tests can be found in Multimedia Appendix 1. Table 2 lists the performance metrics for the complete movement phase. The VitalPatch HR showed marginal overestimation throughout the sitting activities, with a mean MAE ranging from 0.55-0.84 bpm, RMSE from 0.89-1.32 bpm, bias from 0.06-0.32 bpm, and with a correlation of approximately 1 (P<.001) throughout. In the dynamic activity (STS), there was an increase in the MAE, 2.69 (95% CI 1.58-3.63) bpm, and RMSE 5.75 (95% CI 3.83-7.94) bpm, mean bias 1.92 (95% LoA −8.75 to 12.59) bpm, with a correlation of approximately 0.95 (P<.001).

In contrast, a consistent underestimation was observed for RR throughout the sitting activities. MAE ranged from 2.14-3.05 rpm, RMSE from 2.86-4.13, mean bias from −1.68 to −2.43 rpm, and the correlation coefficient from 0.53-0.74 (P<.001). During the dynamic activity, the MAE was 3.45 (95% CI 2.71-4.11) rpm, RMSE was 4.89 (95% CI 3.91-5.92) rpm, mean bias was −1.59 (95% LoA −10.7 to 7.53) rpm, and the correlation was 0.56 (P<.001). An overview of the MAE, RMSE, and mean bias (LoA) for all movements is shown in Figure 5.

The VitalPatch HR estimation was within clinically acceptable accuracy, achieving a MAE 0.72 (95% CI 0.66-0.78) bpm and a RMSE 1.16 (95% CI 1.02-1.29) bpm in the hypoxia phase. Figure 6 shows an exemplar participant during the hypoxia phase, and Figure 7 shows the scatter and Bland-Altman plots for HR and RR. The mean bias was 0.49 (95% LoA −1.58 to 2.56), and the correlation was approximately 1 (P<.001). Although more variable, the RR was also within the acceptable accuracy achieving a MAE 1.89 (95% CI 1.75-2.03) rpm and a RMSE 2.92 (95% CI 2.64-3.2) rpm. A minimal mean bias of −0.08 (95% LoA −5.8 to 5.65) and a correlation of approximately 0.72 (P<.001) were observed.

There were also no significant differences in the error metrics between the SpO₂ subgroups (normoxia, SpO₂ 100% to 90%; mild hypoxia, SpO₂ 85%-89.9%; and severe hypoxia, SpO₂<85% [24]), as shown in Multimedia Appendix 2. Finally, Figure 8 shows the MAE, RMSE, and mean bias (LoA) computed for each oxygen saturation level from the hypoxia phase.

The key findings from this study were as follows: (1) The VitalPatch HR estimation was well within acceptable accuracy limits when compared with the gold standard device (Philips 3-lead ECG), (2) The VitalPatch RR estimation was also in agreement with the gold standard device (Philips capnography) for most of the movements and throughout the full hypoxia phase, and (3) there was no impact on the HR and RR estimation performance when participants were exposed to induced hypoxia. These findings support the clinical use of VitalPatch in patients with reduced baseline saturation or at risk of hypoxia.

Accuracy testing during motion is important before clinical use, as we aim to implement an ambulatory monitoring system that promotes patient mobility and independence without compromising safety. Our results suggest that the VitalPatch can measure HR with high accuracy and minimal bias during simple movements (mimicking in-hospital patient movements). Reliability was reduced during STS activity but was still within acceptable limits. These results agree with the results of previous clinical validation studies when compared with a reference standard device in healthy volunteers [18,23] and surgical patients [21,22]. When compared with nurse manual measurements in surgical patients [14], VitalPatch meets the accuracy criteria for HR estimation.

The MAE of the VitalPatch RR estimation was comparatively higher, especially during the movement phase. However, it was also mostly within the prespecified limits (except for the STS and turning a page movements), in agreement with previous studies [14,18,21-23]. High variation during STS was expected as it is a dynamic activity, potentially creating several artifacts that could affect VitalPatch accuracy during this test [21,22]; however, a high MAE for RR was not expected for the turning pages activity (even if close to the 3-rpm threshold). We postulate that the latter may have created a low-frequency periodic artifact that influenced both the ECG and accelerometer waveforms (both used in the computation of the RR estimate by the VitalPatch).

The most recent study showed that the VitalPatch tended to overestimate RR with a bias of 4.4 rpm [22], but previous studies showed a predominant RR underestimation [14,18,21]. Our study seems to agree with the latter, as VitalPatch mostly underestimated RR throughout the movement phase and varied in the hypoxia phase. However, in contrast to these previous studies, we collected vital signs from healthy participants.

This study also assessed the VitalPatch reliability during a controlled hypoxia phase, and our results suggest that there is no impact on this patch’s reliability for HR and RR measurements during periods of reduced SpO₂ levels. The VitalPatch estimates followed the gold standard HR and RR variations caused by the controlled desaturation of the participants during the hypoxia phase, reliably, supporting its use in clinical populations with reduced baseline saturation or at risk of hypoxia.

This study had some limitations. First, the participants were healthy volunteers in a controlled environment that is not representative of hospitalized patients. In addition, the main study was powered to assess the accuracy of SpO₂ and not that of HR and RR. Although the range required to validate RR was achieved (10 to 30 rpm is required [18] and 4 to 31 rpm was collected), the HR range achieved in this substudy was 40 to 150 bpm, a subset of the range required to validate its accuracy, 30 to 200 bpm [27].

The ANSI/AAMI EC13:2002 standard [27] only requires that a measure of systematic error (eg, mean bias or MAE) be below the clinically accepted limits. In more recent studies, MAE has been used because it is less susceptible to outliers than the mean bias [18,23,30]. Analyzing the RMSE results, which combines both the systematic and random errors, we note that they follow the trend of those of the MAE metric as the RMSE was also higher during the STS activity for both variables and for the turning pages movement for RR. In our study, the LoA (also a measure of random error) seemed to be within the acceptable range when compared with the results of other studies [14,18,21-23] (except for the aforementioned cases). Previous studies reported higher LoA (defined as out of the acceptable range by the authors) for RR when compared with reference standard devices both in healthy volunteers (−10.1 to 6.5) [18] and surgical patients (−15.8 to 11.2) [21] and (−4.4 to 13.3) [22], and when compared with nurse manual measurements in surgical patients (10.32 to 9.04) [14].

In this study, the VitalPatch was subjected to robust testing, during movement and under induced hypoxia, against a gold standard in a controlled environment. Our results suggest that this device can be safely tested in clinical environments (within the collected input range for each vital sign). Our results will inform future phases of our vHDU research project, aiming to develop an accurate wearable system for monitoring ambulatory patients in hospitals.

In conclusion, the VitalPatch was highly accurate throughout movement tests except for the measurement of RR during the STS activity and the turning pages movement. This device was also reliable throughout the hypoxia phase, with no significant MAE or bias differences in hypoxic levels. This study demonstrated that VitalPatch can be safely tested in clinical environments to support earlier detection of clinical deterioration.