Heart failure (HF) is estimated to affect more than 1 million Canadians [1] and 6.5 million adults in the United States [2], many of whom experience chronic symptoms of fatigue and shortness of breath, punctuated by sporadic episodes of decompensation [3]. The unpredictability of these episodes leads to more HF hospitalizations compared with other conditions, representing a significant burden on health systems [4]. For patients with HF, hospitalizations and daily symptoms have a negative impact on daily functioning and ultimately their quality of life (QoL) [5].

Existing medical interventions, including pharmaceutical treatments, have been successful in prolonging the lives of patients with HF. However, with the exception of heart transplantation, full recovery is unlikely. Similar to many other chronic conditions, guideline-directed medical therapy also calls for patients with HF to play an active role through self-management of their diet, fluid restriction, and adherence to the medication schedules [6]. Although many patients with HF receive education for HF self-management during face-to-face clinic visits with care providers [6], mechanisms to support self-care between planned visits are needed to support patients once they go back to living their daily lives.

Telemonitoring (TM), which uses noninvasive electronic devices to collect and transmit physiological and disease-related data collected in patients’ homes to a care provider, can provide this self-care support [5,7], particularly when the TM system includes an algorithm that can provide targeted personalized feedback [8]. When combined with timely data transmission to clinicians, which can enable the early detection and remote clinical intervention of symptom exacerbations [9], TM has the potential to optimize HF management. This is supported by several meta-analyses of randomized controlled trials (RCTs), which have concluded that TM reduces the risk of mortality and the number of hospitalizations when compared with the standard of care [10-13]. Most recently, results from the large Telemedical Interventional Management in Heart Failure II study found that HF TM significantly reduced the percentage of days lost because of unplanned cardiovascular hospital admissions and all-cause death [14]. However, this evidence generally comes from efficacy trials, which are designed to measure an intervention’s impact under ideal conditions [15]. Such conditions are attained through the use of restrictive inclusion criteria and additional resources (eg, implementation staff, training plans, and trial supervision) aimed at ensuring intervention uptake and appropriate use [16]. In fact, the results of high-profile neutral HF TM trials [17-19] have been largely attributed to problems in the intervention’s delivery and uptake [20]. As real-world interventions tend to have broader inclusion criteria and more barriers to appropriate use, it is not uncommon for them to demonstrate less benefit compared with the outcomes of similar interventions in more controlled trials [15].

The overarching trend toward positive evidence in efficacy trials is likely sufficient to encourage many health organizations to make HF TM available to their patient populations. However, questions remain about when, how, and under what conditions HF TM interventions should be delivered. Such questions are best answered by what has been termed practice-based evidence [16], which is the output of research that emphasizes an understanding of context through the use of pragmatic and mixed method study designs often seen in the quality improvement evaluations of real-world health services [21].

The objective of this study was to evaluate the 6-month impact of an HF TM program, called Medly, on health service utilization, clinical outcomes, QoL, and patient self-care. A published protocol outlined the mixed method quality improvement evaluation of the implementation and impact of the Medly program [22], which is implemented as part of the standard of care in an outpatient heart function clinic in Toronto, Canada.

This pragmatic quality improvement study employed a pretest-posttest design to compare impact outcome measures after 6 months of enrollment with those at baseline. The study period spanned from August 23, 2016 (the date when the first patient was enrolled), to June 31, 2019, and was conducted at an ambulatory heart function clinic at the Peter Munk Cardiac Centre (PMCC). The PMCC is a part of the University Health Network (UHN), which is a large university-affiliated organization composed of 5 hospitals and institutes located in Toronto. The UHN Research Ethics Board (16-5789) approved the study as a quality improvement project. Under this definition, data generated as part of the standard of care could be analyzed for quality improvement purposes. However, collecting additional research data through questionnaires required informed consent from patients. Therefore, although all patients entering the program were invited to consent to complete patient-reported outcome questionnaires, this consent was not required to analyze the health service utilization and laboratory data that were retrospectively collected as part of this study.

The Medly program features a clinically validated algorithm [23] to provide patients with personalized self-care messages and to alert members of their core HF care team when clinical intervention may be required. By outsourcing much of the self-care support to the algorithm, clinician resources are freed to manage more urgent cases within minutes of receiving patient data. This, according to the US Food and Drug Administration, is a form of active monitoring [24]. In contrast, passive TM is when patient data get transmitted but a clinician is not expected to take immediate clinical action [24], as is the case if a TM system cannot contextualize data based on urgency or if the telehealth clinician does not have rapid access to the patient’s most responsible physician (MRP) to make a necessary change to the patient’s care plan. The Medly program is hypothesized to improve patient self-care and enable early clinical intervention at the onset of symptom exacerbations. This, in turn, is expected to reduce avoidable health service utilization and improve HF clinical outcomes as well as patients’ QoL.

Participants in this study included all those who were enrolled in the Medly program between August 23, 2016, and January 31, 2019 (6 months before the end of the study period). To be eligible for the program, patients had to meet the following criteria: (1) aged 18 years or older, (2) diagnosed with HF and followed by a cardiologist at the heart function clinic, (3) can speak and read English (or have an informal caregiver who does), and (4) are able to comply with using Medly. In addition, clinicians use clinical judgment in determining whether they believe a patient will benefit. Considerations typically include disease severity (eg, New York Heart Association [NYHA] classification class 2 or 3), a need for self-care support, and a perception that patients will be engaged enough to take daily readings.

Outcomes to evaluate the pre-post impact of the Medly program over a 6-month period are classified into 4 categories: (1) health service utilization, (2) clinical outcomes, (3) QoL, and (4) self-care.

Demographic and clinical characteristics were collected to describe the study population, and a subset of these was used as control variables in the impact analyses to increase the likelihood that any observed changes in the outcomes could be attributed to the Medly program. Selected control variables included sex, age at enrollment, NYHA class, LVEF at enrollment (categorized as reduced ejection fraction [LVEF <40%] vs preserved ejection fraction [≥40%]), location of enrollment (inpatient ward vs outpatient clinic), and duration followed at the outpatient clinic (<6 months vs >6 months). The latter is based on results from a previous RCT evaluating the Medly system, which found that new patients (regardless of treatment arm) improved more than long-term patients because of the confounding effect of being enrolled at the outpatient clinic [35].

Data for the health service utilization outcomes, clinical outcomes, inputs for the SHFM, and available demographic and control variables were extracted from patients’ EMRs and the Medly program’s administrative records. Laboratory values at baseline and 6 months were taken from laboratory tests performed closest to the actual baseline or 6-month date within a 2-month window.

Questionnaires for the remaining demographic information and the QoL and self-care outcomes were administered to patients who consented. Baseline questionnaires were given to patients during the enrollment session, and although patients were encouraged to complete it before leaving, they were permitted to take it home and return the completed questionnaire using prepaid postage. The 6-month questionnaires were mailed to patients at the appropriate time with instructions to complete and return it using prepaid postage.

Although many patients are enrolled in the Medly program indefinitely, the intended primary analysis was to compare baseline outcome values with those at 6 months [22]. Paired-sample t tests and Wilcoxon signed-rank tests were originally planned [22]; however, we ultimately opted to perform multivariate regressions to allow for the controlling of possible confounders.

Linear regressions (ordinary least squares method) of the aforementioned control variables were performed to analyze differences in the QoL, self-care, and clinical outcomes. These regressions required the transformation of non-normal outcome data when applicable (ie, cubic transformation for EQ-5D-5L data and log transformation for the BNP and creatinine data). Finally, the Breusch-Pagan test was used to test the presence of heteroscedasticity [36]. If found, the linear regressions were reported with robust standard errors to correct for heteroscedasticity [37], as was the case for the BNP and sodium linear regressions.

Most of the health service utilization outcomes were regressed with negative binomial distribution to account for the presence of overdispersion [38]. An exception was the analysis for HF-related hospitalizations, which used Poisson regression because no overdispersion was detected. As, by definition, patients new to the outpatient clinic would have a lower number of visits at their baseline measure relative to the number of visits at 6 months, an interaction term between time and duration followed at the outpatient clinic was added to this regression. All outcome data generated by patients who entered the program during the study period were analyzed under the intention-to-treat principle. However, because health service utilization data could not be generated following a person’s death, patients who died before completing 6 months in the program were excluded from the health utilization analyses to avoid biasing the results in a positive direction.

Baseline and 6-month descriptive statistics for each outcome variable (before adjusting for the control variables) and descriptive statistics for the variables used to characterize the study population were obtained using SPSS version 24 (IBM Corporation). The data transformations and regressions were conducted in RStudio version 1.0.153 (RStudio Inc). All statistical tests results were 2-tailed, and a P value of less than .05 (P<.05) was used to indicate statistical significance.

A total of 315 patients were enrolled in the program during the study period, of which 255 consented to complete questionnaires (211 patients returned a baseline questionnaire and 156 returned a completed 6-month questionnaire).

Participants of the Medly program were predominantly men (245/315, 77.8%), with an average age of 58.3 years (SD 15.5). With regard to clinical characteristics, approximately half experienced relatively mild daily HF symptoms with 47.1% (143/304) of patients being classified as NYHA class 2 or less at the time of program enrollment, and the average LVEF of patients was 31.8% (SD 13.4). Three-fourth (235/315, 74.6%) of the participants were enrolled during regularly scheduled outpatient visits, whereas 25.5% (80/315) were enrolled from the inpatient ward before being sent home following a hospital stay. Slightly more than half (183/315, 58.1%) had been followed by the HF clinic for more than 6 months at the time of enrollment, with the remaining being considered new to the clinic. Additional patient characteristics are presented in Multimedia Appendix 1.

Of the 315 patients who entered the program during the analysis period, 30 patients were no longer enrolled after 6 months: 57% (17/30) were removed for clinical reasons (eg, received a heart transplant, recovered, and became palliative), 27% (8/30) left for personal reasons (eg, perception that the benefits were not worth the effort and life circumstances), and 17% (5/30) of these patients died. A comprehensive analysis of why patients were removed or chose to leave the Medly program was published elsewhere [27].

The descriptive statistics for the baseline and 6-month outcome values before adjusting for the control variables are presented in Table 1.

This paper presents the results from a pragmatic pretest-posttest study aimed at determining the 6-month impact of an HF TM program that has been implemented as part of the standard of care in an outpatient heart function clinic. We found a 50% reduction for the primary outcome of HF-related hospitalizations and a 24% reduction in the number of all-cause hospitalizations after controlling for the key demographic and clinical variables of age, sex, NYHA class, LVEF, location of enrollment, and newness to the outpatient clinic. No significant changes were found for the other health service utilization outcomes of length of stay, ED visits, and outpatient clinic visits. However, because the Medly program was intended to fill the gap between scheduled clinic visits rather than to replace existing elements of care, the number of outpatient clinic visits was not expected to decrease. Thus, the fact that closer remote monitoring did not contribute to an increase in outpatient visits can be interpreted as a positive finding, particularly because no increases in ED visits were observed.

This study also showed that enrollment in the Medly program was associated with a significant reduction in BNP levels after 6 months, which, when interpreted alongside the lack of significant changes in other HF biomarkers (eg, creatinine), signals an improvement in patients’ physical health status. Finally, this study found statistically and clinically significant improvements in overall, physical, and emotional HF-related QoL as well as in self-care maintenance and self-care management.

In 2012, an RCT evaluating an earlier version of the Medly system found overall improvements in QoL compared with a control group in addition to significant improvements in BNP and self-care in patients who have been followed in the outpatient heart function clinic for more than 6 months. However, that study was underpowered (50 patients in each control and intervention arm) to show an impact on health service utilization outcomes. Now, with a larger sample size and a sustained program more firmly established within a clinic’s existing services, our study has replicated the original positive findings of that RCT in addition to showing a significant reduction in the number of hospitalizations.

These results are also consistent with systematic reviews and meta-analyses of RCTs [5,39,40], with the latest (at the time of writing) by Zhu et al [13] concluding that HF TM significantly reduces the number of all-cause and cardiac hospitalizations. Importantly, meta-analyses also point to a decreased risk of mortality, an outcome that could not be evaluated, given the lack of a control group in our study.

Practice-based evidence provides insights into an intervention’s real-world effectiveness and can further our understanding of when, how, and under what conditions interventions should be delivered. Therefore, the results of this study are most useful for decision makers or TM program planners when interpreted alongside the contextual detail provided in this and previous publications about the Medly program [26,27] in addition to overarching recommendations about when and how to implement HF TM interventions. For instance, a recent consensus statement from the Heart Failure Society of America broadly concluded that HF TM has the most impact when (1) patients are most at risk (eg, recent hospitalization, prone to fluid overload, and struggles with medication adherence), (2) rates of TM system usage and adherence are high, and (3) clear actions can be taken in response to the TM data [20]. In many respects, the results from this study are consistent with this consensus statement considering the high rates of patient adherence to the Medly program [27] and the fact that actionable feedback is sent to patients and that clinicians are part of the patient’s immediate circle of care. In addition, the statistical significance of the clinical variables used as controls in our impact analysis (ie, NYHA class and whether patients were enrolled immediately following an inpatient stay) is coherent with the idea that HF TM is most beneficial for patients most at risk. In light of this, these results are particularly meaningful because they suggest that an outpatient HF TM program can demonstrate impact under real-world conditions even when the inclusion criteria are left broad and decisions about enrollment are made based on clinical judgment.

The pragmatic study design, including the fact that much of the study data were collected as part of the standard of care, has methodological limitations. First, there was no control group, meaning that even if the analyses controlled for key demographic and clinical variables, the outcomes may have been influenced by subject maturation. Second, health service utilization data were restricted to events occurring at the 5 urban hospitals and institutes that make up the UHN. Therefore, although it is unlikely that patients would voluntarily seek HF care outside of the UHN, any such visits could not be analyzed, representing a major limitation of this study. This also applies to clinical outcome data, which were restricted to laboratory tests conducted at the UHN. However, because the Medly program does not necessarily aim to reduce the number of scheduled outpatient visits (which typically occur every 6 months), the impact of this limitation on laboratory data is expected to be minimal. Third, because the administrative data analyzed were not collected for the purposes of this study, important context is missing that would enable the drawing of more definitive conclusions. For example, although we can conclude that enrollment in the Medly program did not increase the overall number of outpatient visits, we did not have data that would allow us to further analyze what this means in terms of changes in scheduled versus unscheduled outpatient visits. Fourth, not all patients enrolled in the program consented to complete the questionnaires, leaving the analysis of patient-reported outcomes subject to selection bias. Similarly, allowing clinicians to use their judgment in determining who might benefit from the program may also contribute to difficulties in generalizing the results. Fifth, although the skew toward the enrollment of men in the Medly program is consistent with lower proportions of women in heart function clinics and HF research [41,42], this sex- and possibly gender-based limitation is an important consideration for the design of future TM interventions and research. Finally, because there is no defined duration of follow-up in the Medly program, not all patients who were initially a part of the program were enrolled for the full 6 months. However, analyses followed the intention-to-treat principle to mitigate any potential bias of excluding patients who left the program early.

This study presented the results of a pretest-posttest study to evaluate the impact of an HF TM program by comparing the change in outcome measures at 6-month follow-up with those at baseline. After controlling for key demographic and clinical variables, regression analyses found that enrollment in the TM program led to a 50% reduction in the number of HF-related hospitalizations, a 24% reduction in all-cause hospitalizations, and a 59% reduction in BNP values. In addition, enrollment in the TM program was associated with statistically and clinically significant improvements in HF-related QoL and self-care maintenance and management. This study suggests that a real-world HF TM program, which provides patients with self-care support and active clinical monitoring by their existing care team, can reduce health service utilization and improve clinical, QoL, and patient self-care outcomes.