Background: Heart failure (HF) is a common clinical syndrome associated with substantial morbidity, a heavy economic burden, and high risk of readmission. eHealth self-management interventions may be an effective way to improve HF clinical outcomes.
Objective: The aim of this study was to systematically review the evidence for the effectiveness of eHealth self-management in patients with HF.
Methods: This study included only randomized controlled trials (RCTs) that compared the effects of eHealth interventions with usual care in adult patients with HF using searches of the EMBASE, PubMed, CENTRAL (Cochrane Central Register of Controlled Trials), and CINAHL databases from January 1, 2011, to July 12, 2022. The Cochrane Risk of Bias tool (RoB 2) was used to assess the risk of bias for each study. The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) criteria were used to rate the certainty of the evidence for each outcome of interest. Meta-analyses were performed using Review Manager (RevMan v.5.4) and R (v.4.1.0 x64) software.
Results: In total, 24 RCTs with 9634 participants met the inclusion criteria. Compared with the usual-care group, eHealth self-management interventions could significantly reduce all-cause mortality (odds ratio [OR] 0.83, 95% CI 0.71-0.98, P=.03; GRADE: low quality) and cardiovascular mortality (OR 0.74, 95% CI 0.59-0.92, P=.008; GRADE: moderate quality), as well as all-cause readmissions (OR 0.82, 95% CI 0.73-0.93, P=.002; GRADE: low quality) and HF-related readmissions (OR 0.77, 95% CI 0.66-0.90, P<.001; GRADE: moderate quality). The meta-analyses also showed that eHealth interventions could increase patients’ knowledge of HF and improve their quality of life, but there were no statistically significant effects. However, eHealth interventions could significantly increase medication adherence (OR 1.82, 95% CI 1.42-2.34, P<.001; GRADE: low quality) and improve self-care behaviors (standardized mean difference –1.34, 95% CI –2.46 to –0.22, P=.02; GRADE: very low quality). A subgroup analysis of primary outcomes regarding the enrolled population setting found that eHealth interventions were more effective in patients with HF after discharge compared with those in the ambulatory clinic setting.
Conclusions: eHealth self-management interventions could benefit the health of patients with HF in various ways. However, the clinical effects of eHealth interventions in patients with HF are affected by multiple aspects, and more high-quality studies are needed to demonstrate effectiveness.
Heart failure (HF), a major global public health concern, is a common clinical syndrome caused by cardiac structural or functional impairment . The global prevalence of HF was estimated at 64.34 million cases, and the global economic burden of HF was roughly calculated at US $346.17 billion. Global expenditure related to HF is expected to increase to approximately US $400 billion by 2030 [ ]. Since HF cannot be completely cured, it has a major impact on quality of life and requires long-term management [ - ]. The management of patients with HF is a complex issue. Even when the patient is clinically stable, the quality of life is reduced due to dyspnea, depression, fatigue, and cognitive impairment [ ]. The symptom burden of patients with HF prevents them from maintaining adequate social life and roles, participating in social activities, and maintaining relationships [ ].
Self-management is a dynamic, iterative process in which patients need to employ multidimensional strategies to meet their self-needs for coping with chronic illness in their daily lives [, ]. Self-management is an effective way to improve the outcome of chronic disease and an important part of the treatment of patients with chronic diseases [ ]. Successful self-management requires the active participation of individuals, families, and health care providers [ ]. Self-management has been demonstrated to improve clinical outcomes and health-related quality of life, while reducing health care utilization and costs [ - ].
eHealth, as broadly defined, refers to a variety of information and communication technologies used to deliver health care services [, ]. The use of eHealth interventions to deliver health care services can reduce or eliminate some of the barriers to face-to-face treatment, improve access to treatment, reduce waiting times, and be more cost-effective than face-to-face interventions [ - ]. The World Health Organization (WHO) states that to improve health and reduce health inequities, a rigorous evaluation of eHealth is necessary to generate evidence and facilitate the appropriate integration and use of technology [ ]. In addition, the WHO and the International Telecommunication Union launched the National eHealth Strategy Toolkit, which is a comprehensive and practical guide to help all governments, along with their ministries, departments, and agencies, to adapt to suit their own circumstances, goals, and vision in achieving eHealth [ ].
There is a large body of research demonstrating the effectiveness of eHealth self-management interventions for chronic diseases [, ]. eHealth self-management supports patients’ empowerment to better manage HF and improve quality of life [ ]. Furthermore, eHealth applications and systems could help to reduce the rehospitalization rate of patients with HF and lower the cost of treatment [ ].
However, previous research has not specifically focused on the impact of eHealth in self-management. Although eHealth self-management interventions have been used in patients with HF, the results of some systematic reviews have been inconsistent [, ]. Furthermore, there is a lack of understanding of the effectiveness and influencing factors of eHealth self-management approaches.
To fill these gaps, the aim of this study was to perform a systematic review and meta-analysis of the effectiveness of eHealth self-management interventions in patients with HF. This review will provide a reference for the clinical application and better understanding of the possible benefits of eHealth self-management interventions for patients with HF. These findings will further identify gaps and inform the development of future eHealth interventions.
This systematic review and meta-analysis was performed following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines . We searched the PubMed, EMBASE, CENTRAL (Cochrane Central Register of Controlled Trials), and CINAHL databases from January 1, 2011, to July 12, 2022, to identify randomized controlled trials (RCTs) that provide eHealth self-management interventions for HF patients. We used combined search terms such as (heart failure OR cardiac failure OR heart decompensation) AND (self-management OR self care OR self-administration OR self-medication) AND (Telemedicine OR mobile health OR eHealth OR m-health) AND (randomized controlled trial OR randomization OR randomly). Screening titles/abstracts and full-text evaluation of the articles were managed in a database created with Rayyan software [ ]. The full protocol of this systematic review and meta-analysis has been registered in PROSPERO (CRD42021246973).
Inclusion and exclusion criteria strictly followed the Participants-Intervention-Comparison-Outcome (PICO) framework : (1) participants were defined as adults (aged≥18 years) who had been diagnosed with HF (studies in children or adolescents were excluded); (2) interventions consisted of self-management tools including at least one eHealth component (eg, mobile apps), with traditional interventions that did not use any technical support excluded, such as face-to-face meetings; (3) the comparison was HF patients versus usual care; and the (4) outcomes included patient and process outcomes. Patient outcomes are measures directly related to the disease and were considered as the primary outcomes in our study, including all-cause readmission rate, all-cause mortality, HF readmission, and cardiovascular mortality. Process outcomes are measures related to patients’ behavior, which were considered as the secondary outcomes in our study, including quality of life, self-care behaviors, HF knowledge, and medication adherence. In addition, only RCTs or cluster RCTs reporting one or more selected outcomes were included. Reviews, editorials, protocols, and non-English papers were excluded.
The following characteristics were extracted from studies that met the inclusion criteria: name of the first author, publication year, country, type of eHealth technologies, target of eHealth intervention, duration of intervention, patient demographics, number of participants, recruitment setting, outcomes, descriptions of the control, and the intervention. All outcome data used for the meta-analysis were extracted by the same two independent reviewers (J Li and DW). The synthesis of the results and data charting were performed independently by two researchers (SL and J Li). Disagreements were discussed and consulted with a third reviewer (J Liu) to reach consensus.
Risk of Bias Assessment
We used the Cochrane Risk of Bias tool (RoB 2) to assess the risk of bias for each study . This tool assesses 5 domains to address different types of bias: randomization process, deviations from the intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result. Each component includes a low risk of bias, some concerns, or high risk of bias. Risk of bias assessment was performed independently by two reviewers with consultation of the third reviewer when necessary.
Certainty of Evidence
The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach was used to rate the certainty of evidence for each outcome of interest . This approach rates the risk of bias, inconsistency, indirectness, imprecision, and other considerations (eg, publication bias) as “high,” “moderate,” “low,” or “very low.” The GRADE assessment was completed with the GRADEpro Guideline Development Tool.
We performed meta-analyses using Review Manager  (RevMan v.5.4; Cochrane Training) and R (v.4.1.0 x64). The Mantel-Haenszel odds ratio (OR) was used as the effect size for dichotomous outcomes and the standardized mean difference (SMD) was used to calculate the intervention effect for continuous outcomes. Heterogeneity was then analyzed using the Cochran Q test and the I2 statistic. For the Q test, P<.10 was considered to indicate statistically significant heterogeneity [ ]. According to Cochrane Handbook guidelines [ ], the interpretation of the I2 statistic was as follows: 0% to 40% may represent not important heterogeneity, 30% to 60% may represent moderate heterogeneity, 50% to 90% may represent substantial heterogeneity, and 75% to 100% may represent considerable heterogeneity. The fixed-effects model was used for outcomes with low heterogeneity and the random-effects model was used when the heterogeneity was significantly high. To measure the publication bias, we constructed funnel plots and performed the Egger test for all primary outcomes to determine the significance of potential asymmetry [ ]. The threshold of a significant P value was .05. A sensitivity analysis was performed to check the robustness of the pooled results using the leave-one-out approach [ ].
A total of 1884 articles were retrieved in the literature search. After eliminating duplicates, 1203 titles and abstracts were screened in relation to the inclusion/exclusion criteria. Of these, 1144 articles were excluded and a total of 59 articles were subject to full-text review. Finally, 24 articles were included in the systematic review and meta-analysis ().
Risk of Bias and Quality Assessment
The results of risk of bias assessment are summarized in. The majority of studies showed a low risk of bias for the randomization process, and only 5 studies were judged to have some concerns due to lack of detailed information about randomization. The studies could not have been blinded to participants considering the nature of the intervention and 11 studies were judged to have some concerns regarding deviations from the intended interventions. One study was determined as “high risk” for the category of measurement of the outcome because all questionnaire scores were patient-reported, which may have possibly affected the outcome [ ]. In addition, one study did not report all of the outcomes according to the trial registry record, resulting in a judgment of “high risk” regarding selection of the reported result [ ]. For studies included in the meta-analysis, the overall bias was low in 6 studies (25%), with some concerns in 15 studies (63%) and high bias in 3 studies (13%).
A total of 9634 participants with HF were included in the 26 (22+2×2) RCTs, with 4820 patients allocated to the eHealth group and 4814 to the usual-care group. Since two studies performed a three-group parallel-randomized trial and reported their results separately [, ], we divided each of the three-group trials into two RCTs. These studies were performed in 20 different countries, with the largest number of studies performed in the United States (7 studies). Most (15 studies) were multicenter studies, while the other 11 RCTs recruited patients from a single center. The proportion of male HF patients ranged from 46.8% to 83% in the eHealth groups and from 30.2% to 93% in the usual-care groups. The New York Heart Association classification, which classifies patients into one of four categories based on their limitations during physical activity and has been used clinically to determine trial eligibility, was reported in 24 studies. The basic characteristics of the included studies are presented in , whereas the characteristics of participants and details of interventions are summarized in .
|Reference (year)||Country||Type of eHealth technologies||Target of eHealth intervention||Duration of intervention||Recruitment||Setting|
|Negarandeh et al  (2019)||Iran||Telephone||Education||2 months||Single center||After discharge|
|Seto et al  (2012)||Canada||Designed telemedical system||Monitoring||6 months||Single center||Ambulatory clinic|
|Hindricks et al  (2014)||Australia, Europea, and Israel||Designed telemedical system||Monitoring||12 months||Multicenter (36 sites)||No information|
|Galinier et al  (2020)||France||Telephone+designed telemedical system||Monitoring+education||18 months||Multicenter (38 sites)||After discharge (26.4%), hospitalized (73.6%)|
|Hale et al  (2016)||United States||Designed telemedical system||Reminders||3 months||Multicenter (2 sites)||No information|
|Yanicelli et al  (2021)||Argentina||Mobile or tablet app||Monitoring+education||3 months||Single center||Ambulatory clinic|
|Chen et al  (2019)b||China||Mobile text message||Education+reminders||6 months||Single center||After discharge|
|Chen et al  (2019)b||China||Telephone||Education+reminders||6 months||Single center||After discharge|
|Koehler et al  (2018)||Germany||Designed telemedical system||Monitoring+education||365-393 days||Multicenter (200 sites)||After discharge|
|Çavuşoğlu et al  (2016)||Turkey||Telephone||Education||6 months||Multicenter (10 sites)||After discharge|
|Dendale et al  (2011)||Belgium||Designed telemedical system||Monitoring||6 months||Multicenter (7 sites)||After discharge|
|Wagenaar et al  (2019)b||Netherlands||Internet website||Education||12 months||Multicenter (9 sites)||Ambulatory clinic|
|Wagenaar et al  (2019)b||Netherlands||Designed telemedical system||Monitoring||12 months||Multicenter (9 sites)||Ambulatory clinic|
|Dang et al  (2017)||United States||Designed telemedical system||Monitoring||3 months||Single center||Ambulatory clinic|
|Oliveira et al  (2017)||Brazil||Telephone||Education||4 months||Single center||Ambulatory clinic|
|Ritchie et al  (2016)||United States||Designed telemedical system||Mixed interventions||1 months||Single center||After discharge|
|Cichosz et al  (2019)||Denmark||Designed telemedical system||Monitoring||12 months||Multicenter (3 sites)||Ambulatory clinic|
|Bakitas et al  (2020)||United States||Telephone||Mixed interventions||4 months||Multicenter (2 sites)||Ambulatory clinic, hospitalized|
|Pedone et al  (2015)||Italy||Telephone+designed telemedical system||Monitoring||6 months||Single center||Ambulatory clinic after discharge|
|Jayaram et al  (2017)||United States||Designed telemedical system||Monitoring||6 months||Multicenter (33 sites)||After discharge|
|Melin et al  (2018)||Sweden||Mobile or tablet app||Monitoring+education||6 months||Multicenter (3 sites)||After discharge, hospitalized|
|Boyne et al  (2014)||Netherlands||Designed telemedical system||Monitoring+education||12 months||Multicenter (3 sites)||Ambulatory clinic|
|Pekmezaris et al  (2018)||United States||Designed telemedical system||Monitoring||3 months||Single center||After discharge|
|Koehler et al  (2011)||Germany||Designed telemedical system||Monitoring||Median 26 months (range 12-28 months)||Multicenter (165 sites)||Ambulatory clinic|
|Dorsch et al  (2021)||United States||Mobile or tablet app||Mixed interventions||3 months||Single center||After discharge, hospitalized|
|Sahlin et al  (2022)||Sweden||Designed telemedical system||Monitoring+education||8 months||Multicenter (7 sites)||Ambulatory clinic|
aEurope includes only Austria, Czech Republic, Denmark, Germany, and Latvia.
bTwo types of eHealth interventions employed in a three-group parallel randomized controlled trial design.
These studies used various types of eHealth technologies for the intervention (). Of these, 14 studies used a designed telemedical system; 5 studies used the telephone; 3 studies used a mobile phone or tablet app; 2 studies used an internet website and text messages, respectively; and 2 studies used a combination of telephone with a designed telemedical system intervention. The duration of the intervention ranged from 1 month to 28 months (17 studies had interventions lasting≥6 months). The target of the intervention was classified into three types: education, monitoring, and reminders. The mixed targets were defined as the presence of the three types in one study. Educational interventions were defined as interventions that aimed at improving the HF knowledge available through educational programs and instructions. Reminder interventions were defined as interventions that prompted patients to do something (eg, take medication on time) and could be delivered through a designed telemedical system, mobile text messages, or telephone calls. Monitoring interventions were defined as interventions that transmitted vital signs or symptoms (eg, weight, blood pressure, or heart rate) to an external telemedicine center, which may lead to lasting improvements in behaviors. The eHealth intervention varied regarding intensity and the extent of the human component, although most trials provided the eHealth intervention every day ( ).
Effects of eHealth Interventions on All-Cause Mortality
A total of 15 studies reported all-cause mortality data. The 15 studies included 6610 participants and showed moderate heterogeneity (P=.05; I2=40%). The analysis showed that all-cause mortality was significantly lower in the eHealth intervention group than in the control group (OR 0.83, 95% CI 0.71-0.98, P=.03; GRADE: low quality) ().
Effects of eHealth Interventions on Cardiovascular Mortality
A total of 8 studies reported cardiovascular mortality data. The 8 studies included 4787 participants and had low heterogeneity (P=.42; I2=1%). The analysis showed that cardiovascular mortality was significantly lower in the eHealth intervention group than in the control group (OR 0.74, 95% CI 0.59-0.92, P=.008; GRADE: moderate quality) ().
Effects of eHealth Interventions on All-Cause Readmission
A total of 16 studies reported all-cause readmission data. The 16 studies included 4310 participants and had moderate heterogeneity (P=.02; I2=48%). The analysis showed that all-cause readmission was significantly lower in the eHealth intervention group than in the control group (OR 0.82, 95% CI 0.73-0.93, P=.002; GRADE: low quality) ().
Effects of eHealth Interventions on HF-Related Readmission
A total of 11 studies reported HF-related readmission data. The 11 studies included 4268 participants and had no heterogeneity (P=.98; I2=0%). The analysis showed that HF-related readmission was significantly lower in the eHealth intervention group than in the control group (OR 0.77, 95% CI 0.66-0.90, P<.001; GRADE: moderate quality) ().
Effects of eHealth Interventions on HF Knowledge
A total of 4 studies reported HF knowledge data, 3 of which used the 15-item Dutch HF Knowledge Scale (DHFKS) . One study used a modified questionnaire containing 14 questions, which has proven to be an adequate tool to evaluate the knowledge of Brazilian HF patients [ ]. Higher scores across both scales indicate that the patients have more knowledge of HF. The analysis demonstrated an improvement in HF knowledge among patients in the eHealth intervention group compared with that in the control group. However, there was no significant difference between the eHealth and usual-care groups (SMD 0.35, 95% CI –0.10 to 0.81, P=.13; GRADE: very low quality), with significantly high heterogeneity between studies (P=.008; I2=75%) ( ).
Effects of eHealth Interventions on Quality of Life
A total of 9 studies reported general quality of life data. Three of these studies used the HF-specific 23-item Kansas City Cardiomyopathy Questionnaire (KCCQ), in which higher scores indicate better quality of life . Five of these studies used the Minnesota Living with Heart Failure Questionnaire (MLHFQ), which consists of 21 questions using a 6-point Likert scale [ ]. Since lower scores of the MLHFQ indicate a higher quality of life, we calculated the change using the baseline score minus the final score. The analysis demonstrated an improvement in quality of life among patients in the eHealth intervention group compared with that of the control group. However, there was no significant difference between the eHealth and usual-care groups (SMD 0.04, 95% CI –0.03 to 0.11, P=.26; GRADE: moderate quality), with low heterogeneity among the 9 studies (P=.22; I2=25%) ( ).
Effects of eHealth Interventions on Medication Adherence
A total of 3 studies reported medication adherence data. The analysis demonstrated a significant improvement in medication adherence among patients in the eHealth intervention group compared with the control group (OR 1.82, 95% CI 1.42-2.34, P<.001; GRADE: low quality), with no heterogeneity among 3 studies (P=.63; I2=0%) ().
Effects of eHealth Interventions on Self-care Behaviors
A total of 3 studies reported self-care behavior data. Four of these studies used the European Heart Failure Self-Care Behavior Scale (EHFSC) . Two of these studies used the EHFSC revised to a 9-item scale (EHFScB-9) [ ], and 2 used the self-care of heart failure index (SCHFI), which comprises three subscales [ ]. Three of the 8 studies had imperfect data and therefore were not included in this meta-analysis. The analysis demonstrated a significant improvement in self-care behaviors among patients in the eHealth intervention group compared with the control group (SMD –1.34, 95% CI –2.46 to –0.22, P=.02; GRADE: very low quality), with significantly high heterogeneity among 5 studies (P<.001; I2=95%) ( ).
Since the inclusion criteria of the trials were different, which contributed to the heterogeneity of studies, subgroup analyses of primary outcomes regarding the enrolled population setting (after discharge or ambulatory clinic) were performed and the results are summarized in. Compared with patients in an ambulatory clinic setting, the eHealth intervention showed a larger effect in patients with HF after discharge on the reduction of all-cause mortality (OR 0.73, 95% CI 0.58-0.93, P=.01), cardiovascular mortality (OR 0.71, 95% CI 0.53-0.95, P=.02), all-cause readmission (OR 0.70, 95% CI 0.56-0.87, P=.001), and HF-related readmission (OR 0.75, 95% CI 0.56-1.00, P=.05) than the usual-care group. The results showed that the effects of an eHealth intervention may vary in certain population settings.
Sensitivity Analysis and Publication Bias
Sensitivity analysis was performed for the primary outcomes of meta-analyses, including all-cause mortality, cardiovascular mortality, all-cause readmission rate, and HF-related hospitalizations, using the leave-one-out approach. The direction and magnitude of the combined estimates did not vary markedly with the exclusion of individual studies, indicating the reliability of the findings (). The results showed no publication bias in the primary outcomes. No publication bias was detected for any of the primary outcomes. The funnel plots and linear regression test results of funnel plot asymmetry are shown in .
There is a reasonable amount of original research exploring the effects of eHealth interventions in HF. However, among the 1884 articles we identified, only 24 met the inclusion criteria for this systematic review. The pooled results suggest that eHealth self-management interventions can improve primary and secondary outcomes in patients with HF.
Although eHealth self-management interventions have the potential to improve chronic disease management, successful implementation in routine clinical practice is rare [, ]. The main challenges in implementing and utilizing eHealth include user acceptance, standards and interoperability, regulations, ensuring cost-effectiveness and sustainability, and the organization and implementation environment, as discussed in turn below.
eHealth User Acceptance
End-user acceptance of eHealth products is the key to a successful eHealth program, which is influenced by many factors, although eHealth usability and user training seem to be more important. eHealth products need to improve the understanding of the design and development process from the user’s perspective, and focus on developing detailed representations of the user’s needs and wants [- ]. Furthermore, the process of eHealth interventions is complex, which requires active participation, cooperation, and familiarity with the associated processes by all involved [ ]. Through eHealth self-management, health care providers can connect more closely with patients, identify problems earlier, provide guidance, and improve patient compliance.
Standards and Interoperability
A successful eHealth self-management implementation requires unified integration with electronic health record (EHR) and clinical workflows to enable secure and fast access to patient data and information at various locations . This requires interoperability within EHRs, as well as interoperability between EHR and eHealth self-management systems. Ultimately, intelligent eHealth self-management could reduce phone calls and paperwork by automating data exchange. However, eHealth self-management and EHR systems often lack interoperability and standards for clinical data exchange [ , ].
There is also a lack of eHealth regulations, which is a major barrier to the development of eHealth . The most common regulations used in eHealth implementation are national data and privacy protection laws and regulations, as well as national EHR and health financing legislation [ ]. Health financing legislation is critical to ensure the continued sustainability and support of eHealth. National health service laws also provide guidelines for the use and implementation of eHealth [ ].
Cost-Effectiveness and Sustainability
The cost-effectiveness and sustainability of eHealth are key factors that directly affect the successful implementation of eHealth in practice . Many of the costs relate to the development of the eHealth product, deployment, training, product iterations, services, or ongoing maintenance costs of the technology. eHealth self-management programs should ensure cost-effectiveness and sustainability of the program.
Organization and Implementation Environment
The hospital’s management capacity, human resources, and implementation environment are key obstacles to the successful implementation of eHealth . In addition, eHealth technologies also need to fit the organization. The information literacy of patients and easy access to high-speed internet are conducive to the implementation of eHealth [ , ]. eHealth applications should be implemented on top of existing organization-centric and process-controlled systems [ ]. This will benefit the efficiency and effectiveness of eHealth solutions and health care services, as well as support the development process and change in management.
eHealth Self-management in HF
eHealth self-management is a rapidly growing area of HF management, and considerable research is still needed to promote the widespread use of eHealth self-management and enhance its clinical outcomes. Future research should investigate how to expand the content of eHealth self-management tools while adapting to different HF patients. There is a need for a cost-benefit analysis of eHealth self-management, and how to most effectively integrate eHealth self-management into workflows and deploy it in different settings. Further work is needed to understand how eHealth self-management can help with the transformation of care delivery models and how to combine eHealth self-management with a workflow-oriented quality improvement program.
Although this systematic review and associated meta-analyses showed that eHealth self-management interventions benefit patients with HF, this study has several limitations. First, the included studies varied in participant demographics, types of HF, sample size, assessment tools, period of interventions, and type of eHealth interventions, which can lead to heterogeneity and bias. Furthermore, reliance on self-reports and the use of questionnaires to score outcomes may lead to bias in self-reported data. Therefore, the results of this systematic review should be interpreted with caution. Second, participants were recruited at a single site or concentrated in a single population in some cases; thus, the resulting sample was not representative of the entire HF patient population, which would limit the generalizability of the findings. Finally, we only searched for literature in English, excluding other languages. This may limit the retrieval of non-English but related papers.
In this systematic review, we performed a literature search and provided a comprehensive overview of eHealth self-management in patients with HF. We observed that eHealth self-management can support the health of HF patients in many ways. However, the effectiveness of eHealth in the self-management of patients with HF is affected by multiple aspects and more clinical studies are needed to prove its effectiveness.
J Liu and SL conceived the study. J Li, SL, J Liu, DW, RL, ZQ, and YH performed the analysis, interpreted the results, and drafted the manuscript. All authors revised the manuscript. All authors read and approved the final manuscript.
Conflicts of Interest
Basic characteristics of the studies.XLSX File (Microsoft Excel File), 17 KB
Subgroup analyses for primary outcomes.DOCX File , 88 KB
Sensitivity analysis of the influence of each study on the pooled estimates.DOCX File , 1310 KB
Funnel plots of all-cause mortality.DOCX File , 48 KB
- Ponikowski P, Voors AA, Anker SD, Bueno H, Cleland JGF, Coats AJS, ESC Scientific Document Group. 2016 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: The Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure of the European Society of Cardiology (ESC) developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur Heart J 2016 Jul 14;37(27):2129-2200. [CrossRef] [Medline]
- Lippi G, Sanchis-Gomar F. Global epidemiology and future trends of heart failure. AME Med J 2020 Jun 25;5(15):1-6 [FREE Full text]
- Habal MV, Garan AR. Long-term management of end-stage heart failure. Best Pract Res Clin Anaesthesiol 2017 Jun;31(2):153-166 [FREE Full text] [CrossRef] [Medline]
- Morken IM, Storm M, Søreide JA, Urstad KH, Karlsen B, Husebø AML. Posthospitalization follow-up of patients with heart failure using eHealth solutions: restricted systematic review. J Med Internet Res 2022 Feb 15;24(2):e32946 [FREE Full text] [CrossRef] [Medline]
- Nordfonn OK, Morken IM, Bru LE, Larsen AI, Husebø AML. Burden of treatment in patients with chronic heart failure - a cross-sectional study. Heart Lung 2021;50(3):369-374 [FREE Full text] [CrossRef] [Medline]
- Pressler SJ, Gradus-Pizlo I, Chubinski SD, Smith G, Wheeler S, Sloan R, et al. Family caregivers of patients with heart failure: a longitudinal study. J Cardiovasc Nurs 2013;28(5):417-428. [CrossRef] [Medline]
- Olano-Lizarraga M, Wallström S, Martín-Martín J, Wolf A. Interventions on the social dimension of people with chronic heart failure: a systematic review of randomized controlled trials. Eur J Cardiovasc Nurs 2022 Jun 23:online ahead of print. [CrossRef] [Medline]
- Miller WR, Lasiter S, Bartlett Ellis R, Buelow JM. Chronic disease self-management: a hybrid concept analysis. Nurs Outlook 2015 Mar;63(2):154-161 [FREE Full text] [CrossRef] [Medline]
- Ryan P, Sawin KJ. The individual and family self-management theory: background and perspectives on context, process, and outcomes. Nurs Outlook 2009;57(4):217-225 [FREE Full text] [CrossRef] [Medline]
- Lin C, Hwang S. Patient-centered self-management in patients with chronic kidney disease: challenges and implications. Int J Environ Res Public Health 2020 Dec 16;17(24):9443 [FREE Full text] [CrossRef] [Medline]
- Durosier Mertilus DS, Lengacher CA, Rodriguez CS. A review and conceptual analysis of cancer pain self-management. Pain Manag Nurs 2022 Apr;23(2):168-173. [CrossRef] [Medline]
- Ko D, Bratzke LC, Roberts T. Self-management assessment in multiple chronic conditions: a narrative review of literature. Int J Nurs Stud 2018 Jul;83:83-90. [CrossRef] [Medline]
- van Beest W, Boon W, Andriessen D, Moors E, van der Veen G, Pol H. Successful implementation of self-management health innovations. J Public Health 2020 Jul 01;30(3):721-735. [CrossRef]
- van Riel PL, Zuidema RM, Vogel C, Rongen-van Dartel SA. Patient self-management and tracking: a European experience. Rheum Dis Clin North Am 2019 May;45(2):187-195 [FREE Full text] [CrossRef] [Medline]
- Borrelli B, Ritterband LM. Special issue on eHealth and mHealth: challenges and future directions for assessment, treatment, and dissemination. Health Psychol 2015 Dec;34S:1205-1208. [CrossRef] [Medline]
- Kelly M, Fullen B, Martin D, McMahon S, McVeigh JG. eHealth interventions to support self-management in people with musculoskeletal disorders: a scoping review protocol. JBI Evid Synth 2021 Mar;19(3):709-720. [CrossRef] [Medline]
- de Grood C, Raissi A, Kwon Y, Santana MJ. Adoption of e-health technology by physicians: a scoping review. J Multidiscip Healthc 2016;9:335-344. [CrossRef] [Medline]
- Jhamb M, Cavanaugh KL, Bian A, Chen G, Ikizler TA, Unruh ML, et al. Disparities in electronic health record patient portal use in nephrology clinics. Clin J Am Soc Nephrol 2015 Nov 06;10(11):2013-2022 [FREE Full text] [CrossRef] [Medline]
- Mauco KL, Scott RE, Mars M. Validation of an e-health readiness assessment framework for developing countries. BMC Health Serv Res 2020 Jun 23;20(1):575 [FREE Full text] [CrossRef] [Medline]
- World Health Organization. WHO guideline: recommendations on digital interventions for health system strengthening. Geneva, Switzerland: World Health Organization; 2019.
- World Health Organization, International Telecommunication Union. National eHealth strategy toolkit. World Health Organization. 2012. URL: http://handle.itu.int/11.1002/pub/8069793a-en [accessed 2022-01-12]
- Butler S, Sculley D, Santos D, Fellas A, Gironès X, Singh-Grewal D, et al. Effectiveness of eHealth and mHealth interventions supporting children and young people living with juvenile idiopathic arthritis: systematic review and meta-analysis. J Med Internet Res 2022 Feb 02;24(2):e30457 [FREE Full text] [CrossRef] [Medline]
- Villarreal V, Berbey-Alvarez A. Evaluation of mHealth applications related to cardiovascular diseases: a systematic review. Acta Inform Med 2020 Jun;28(2):130-137 [FREE Full text] [CrossRef] [Medline]
- Schmaderer M, Miller JN, Mollard E. Experiences of using a self-management mobile app among individuals with heart failure: qualitative study. JMIR Nurs 2021 Aug 09;4(3):e28139 [FREE Full text] [CrossRef] [Medline]
- de la Torre Díez I, Garcia-Zapirain B, Méndez-Zorrilla A, López-Coronado M. Monitoring and follow-up of chronic heart failure: a literature review of eHealth applications and systems. J Med Syst 2016 Jul;40(7):179. [CrossRef] [Medline]
- Akinosun AS, Polson R, Diaz-Skeete Y, De Kock JH, Carragher L, Leslie S, et al. Digital technology interventions for risk factor modification in patients with cardiovascular disease: systematic review and meta-analysis. JMIR Mhealth Uhealth 2021 Mar 03;9(3):e21061 [FREE Full text] [CrossRef] [Medline]
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021 Mar 29;372:n71 [FREE Full text] [CrossRef] [Medline]
- Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016 Dec 05;5(1):210 [FREE Full text] [CrossRef] [Medline]
- Kloda LA, Boruff JT, Cavalcante AS. A comparison of patient, intervention, comparison, outcome (PICO) to a new, alternative clinical question framework for search skills, search results, and self-efficacy: a randomized controlled trial. J Med Libr Assoc 2020 Apr;108(2):185-194 [FREE Full text] [CrossRef] [Medline]
- Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019 Aug 28;366:l4898. [CrossRef] [Medline]
- Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol 2011 Apr;64(4):380-382. [CrossRef] [Medline]
- Review Manager (RevMan). Cochrane Training.: Nordic Cochrane Centre TTC; 2014. URL: https://training.cochrane.org/online-learning/core-software/revman [accessed 2022-01-12]
- Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003 Sep 06;327(7414):557-560 [FREE Full text] [CrossRef] [Medline]
- Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M. Cochrane handbook for systematic reviews of interventions. Hoboken, NJ: John Wiley & Sons; 2019.
- Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997 Sep 13;315(7109):629-634 [FREE Full text] [CrossRef] [Medline]
- Liu M, Luong L, Lachaud J, Edalati H, Reeves A, Hwang SW. Adverse childhood experiences and related outcomes among adults experiencing homelessness: a systematic review and meta-analysis. Lancet Public Health 2021 Nov;6(11):e836-e847 [FREE Full text] [CrossRef] [Medline]
- Boyne JJJ, Vrijhoef HJM, Spreeuwenberg M, De Weerd G, Kragten J, Gorgels APM, TEHAF investigators. Effects of tailored telemonitoring on heart failure patients' knowledge, self-care, self-efficacy and adherence: a randomized controlled trial. Eur J Cardiovasc Nurs 2014 Jun;13(3):243-252. [CrossRef] [Medline]
- Wagenaar KP, Broekhuizen BDL, Jaarsma T, Kok I, Mosterd A, Willems FF, et al. Effectiveness of the European Society of Cardiology/Heart Failure Association website 'heartfailurematters.org' and an e-health adjusted care pathway in patients with stable heart failure: results of the 'e-Vita HF' randomized controlled trial. Eur J Heart Fail 2019 Feb;21(2):238-246. [CrossRef] [Medline]
- Chen C, Li X, Sun L, Cao S, Kang Y, Hong L, et al. Post-discharge short message service improves short-term clinical outcome and self-care behaviour in chronic heart failure. ESC Heart Fail 2019 Feb;6(1):164-173. [CrossRef] [Medline]
- Negarandeh R, Zolfaghari M, Bashi N, Kiarsi M. Evaluating the effect of monitoring through telephone (tele-monitoring) on self-care behaviors and readmission of patients with heart failure after discharge. Appl Clin Inform 2019 Mar;10(2):261-268 [FREE Full text] [CrossRef] [Medline]
- Seto E, Leonard KJ, Cafazzo JA, Barnsley J, Masino C, Ross HJ. Mobile phone-based telemonitoring for heart failure management: a randomized controlled trial. J Med Internet Res 2012 Feb 16;14(1):e31 [FREE Full text] [CrossRef] [Medline]
- Hindricks G, Taborsky M, Glikson M, Heinrich U, Schumacher B, Katz A, IN-TIME study group. Implant-based multiparameter telemonitoring of patients with heart failure (IN-TIME): a randomised controlled trial. Lancet 2014 Aug 16;384(9943):583-590. [CrossRef] [Medline]
- Galinier M, Roubille F, Berdague P, Brierre G, Cantie P, Dary P, OSICAT Investigators. Telemonitoring versus standard care in heart failure: a randomised multicentre trial. Eur J Heart Fail 2020 Jun;22(6):985-994. [CrossRef] [Medline]
- Hale TM, Jethwani K, Kandola MS, Saldana F, Kvedar JC. A remote medication monitoring system for chronic heart failure patients to reduce readmissions: a two-arm randomized pilot study. J Med Internet Res 2016 Apr 17;18(5):e91 [FREE Full text] [CrossRef] [Medline]
- Yanicelli LM, Goy CB, González VDC, Palacios GN, Martínez EC, Herrera MC. Non-invasive home telemonitoring system for heart failure patients: a randomized clinical trial. J Telemed Telecare 2020 Jan 23;27(9):553-561. [CrossRef]
- Koehler F, Koehler K, Deckwart O, Prescher S, Wegscheider K, Kirwan B, et al. Efficacy of telemedical interventional management in patients with heart failure (TIM-HF2): a randomised, controlled, parallel-group, unmasked trial. Lancet 2018 Sep 22;392(10152):1047-1057. [CrossRef] [Medline]
- Çavuşoğlu Y, Zoghi M, Eren M, Bozçalı E, Kozdağ G, Şentürk T, Hit-PoinT Investigators. Post-discharge heart failure monitoring program in Turkey: Hit-PoinT. Anatol J Cardiol 2017 Feb;17(2):107-112 [FREE Full text] [CrossRef] [Medline]
- Dendale P, De Keulenaer G, Troisfontaines P, Weytjens C, Mullens W, Elegeert I, et al. Effect of a telemonitoring-facilitated collaboration between general practitioner and heart failure clinic on mortality and rehospitalization rates in severe heart failure: the TEMA-HF 1 (TElemonitoring in the MAnagement of Heart Failure) study. Eur J Heart Fail 2012 Mar;14(3):333-340. [CrossRef] [Medline]
- Dang S, Karanam C, Gómez-Marín O. Outcomes of a mobile phone intervention for heart failure in a minority county hospital population. Telemed J E Health 2017 Jun;23(6):473-484. [CrossRef] [Medline]
- Oliveira JAD, Cordeiro RG, Rocha RG, Guimarães TCF, Albuquerque DCD. Impacto do monitoramento telefônico em pacientes com insuficiência cardíaca: ensaio clínico randomizado. Acta Paul Enferm 2017 Aug;30(4):333-342. [CrossRef]
- Ritchie CS, Houston TK, Richman JS, Sobko HJ, Berner ES, Taylor BB, et al. The E-Coach technology-assisted care transition system: a pragmatic randomized trial. Transl Behav Med 2016 Sep;6(3):428-437 [FREE Full text] [CrossRef] [Medline]
- Cichosz SL, Udsen FW, Hejlesen O. The impact of telehealth care on health-related quality of life of patients with heart failure: Results from the Danish TeleCare North heart failure trial. J Telemed Telecare 2020;26(7-8):452-461. [CrossRef] [Medline]
- Bakitas MA, Dionne-Odom JN, Ejem DB, Wells R, Azuero A, Stockdill ML, et al. Effect of an early palliative care telehealth intervention vs usual care on patients with heart failure: the ENABLE CHF-PC randomized clinical trial. JAMA Intern Med 2020 Sep 01;180(9):1203-1213 [FREE Full text] [CrossRef] [Medline]
- Pedone C, Rossi FF, Cecere A, Costanzo L, Antonelli Incalzi R. Efficacy of a physician-led multiparametric telemonitoring system in very old adults with heart failure. J Am Geriatr Soc 2015 Jun;63(6):1175-1180. [CrossRef] [Medline]
- Jayaram NM, Khariton Y, Krumholz HM, Chaudhry SI, Mattera J, Tang F, et al. Impact of telemonitoring on health status. Circ Cardiovasc Qual Outcomes 2017 Dec;10(12):e004148 [FREE Full text] [CrossRef] [Medline]
- Melin M, Hägglund E, Ullman B, Persson H, Hagerman I. Effects of a tablet computer on self-care, quality of life, and knowledge: a randomized clinical trial. J Cardiovasc Nurs 2018;33(4):336-343. [CrossRef] [Medline]
- Pekmezaris R, Nouryan CN, Schwartz R, Castillo S, Makaryus AN, Ahern D, et al. A randomized controlled trial comparing telehealth self-management to standard outpatient management in underserved Black and Hispanic patients living with heart failure. Telemed J E Health 2019 Oct 01;25(10):917-925 [FREE Full text] [CrossRef] [Medline]
- Koehler F, Winkler S, Schieber M, Sechtem U, Stangl K, Böhm M, Telemedical Interventional Monitoring in Heart Failure Investigators. Impact of remote telemedical management on mortality and hospitalizations in ambulatory patients with chronic heart failure: the telemedical interventional monitoring in heart failure study. Circulation 2011 May 03;123(17):1873-1880. [CrossRef] [Medline]
- Dorsch MP, Farris KB, Rowell BE, Hummel SL, Koelling TM. The effects of the ManageHF4Life mobile app on patients with chronic heart failure: randomized controlled trial. JMIR Mhealth Uhealth 2021 Dec 07;9(12):e26185 [FREE Full text] [CrossRef] [Medline]
- Sahlin D, Rezanezad B, Edvinsson M, Bachus E, Melander O, Gerward S. Self-care Management Intervention in Heart Failure (SMART-HF): a multicenter randomized controlled trial. J Card Fail 2022 Jan;28(1):3-12 [FREE Full text] [CrossRef] [Medline]
- van der Wal MHL, Jaarsma T, Moser DK, van Veldhuisen DJ. Development and testing of the Dutch Heart Failure Knowledge Scale. Eur J Cardiovasc Nurs 2005 Dec;4(4):273-277. [CrossRef] [Medline]
- Rabelo ER, Mantovani VM, Aliti GB, Domingues FB. Cross-cultural adaptation and validation of a disease knowledge and self-care questionnaire for a Brazilian sample of heart failure patients. Rev Lat Am Enfermagem 2011;19(2):277-284 [FREE Full text] [CrossRef] [Medline]
- Green CP, Porter CB, Bresnahan DR, Spertus JA. Development and evaluation of the Kansas City Cardiomyopathy Questionnaire: a new health status measure for heart failure. J Am Coll Cardiol 2000 Apr;35(5):1245-1255 [FREE Full text] [CrossRef] [Medline]
- Riegel B, Moser DK, Glaser D, Carlson B, Deaton C, Armola R, et al. The Minnesota Living With Heart Failure Questionnaire: sensitivity to differences and responsiveness to intervention intensity in a clinical population. Nurs Res 2002;51(4):209-218. [CrossRef] [Medline]
- Jaarsma T, Strömberg A, Mårtensson J, Dracup K. Development and testing of the European Heart Failure Self-Care Behaviour Scale. European Journal of Heart Failure 2003 Jun 09;5(3):363-370. [CrossRef]
- Jaarsma T, Arestedt KF, Mårtensson J, Dracup K, Strömberg A. The European Heart Failure Self-care Behaviour scale revised into a nine-item scale (EHFScB-9): a reliable and valid international instrument. Eur J Heart Fail 2009 Jan;11(1):99-105. [CrossRef] [Medline]
- Riegel B, Carlson B, Moser DK, Sebern M, Hicks FD, Roland V. Psychometric testing of the self-care of heart failure index. J Card Fail 2004 Aug;10(4):350-360. [CrossRef] [Medline]
- Lobban F, Appelbe D, Appleton V, Aref-Adib G, Barraclough J, Billsborough J. Health ServicesDelivery Research. An online supported self-management toolkit for relatives of people with psychosis or bipolar experiences: the IMPART multiple case study. Health Serv Deliv Res 2020 Sep;8(37):1-248. [CrossRef] [Medline]
- Versluis A, van Luenen S, Meijer E, Honkoop PJ, Pinnock H, Mohr DC, et al. SERIES: eHealth in primary care. Part 4: Addressing the challenges of implementation. Eur J Gen Pract 2020 Dec 07;26(1):140-145 [FREE Full text] [CrossRef] [Medline]
- Ahmed B, Dannhauser T, Philip N. A systematic review of reviews to identify key research opportunities within the field of eHealth implementation. J Telemed Telecare 2019 Jun;25(5):276-285. [CrossRef] [Medline]
- Petersen LS, Bertelsen P, Bjørnes C. Cooperation and communication challenges in small-scale eHealth development projects. Int J Med Inform 2013 Dec;82(12):e375-e385. [CrossRef] [Medline]
- Polillo A, Gran-Ruaz S, Sylvestre J, Kerman N. The use of eHealth interventions among persons experiencing homelessness: a systematic review. Digit Health 2021 Feb 02;7:2055207620987066 [FREE Full text] [CrossRef] [Medline]
- Liu J, Liu S, Zheng T, Bi Y. Physicians' perspectives of telemedicine during the COVID-19 pandemic in China: qualitative survey study. JMIR Med Inform 2021 Jun 01;9(6):e26463 [FREE Full text] [CrossRef] [Medline]
- Bernstam EV, Warner JL, Krauss JC, Ambinder E, Rubinstein WS, Komatsoulis G, et al. Quantitating and assessing interoperability between electronic health records. J Am Med Inform Assoc 2022 Apr 13;29(5):753-760. [CrossRef] [Medline]
- Fadrique LX, Rahman D, Vaillancourt H, Boissonneault P, Donovska T, Morita PP. Overview of policies, guidelines, and standards for active assisted living data exchange: thematic analysis. JMIR Mhealth Uhealth 2020 Jun 22;8(6):e15923 [FREE Full text] [CrossRef] [Medline]
- Marija A, Kristina M. Legal aspects of eHealth development in North Macedonia. Bull St. Petersburg Univ 2021;12(3):660-675. [CrossRef]
- Scheibner J, Sleigh J, Ienca M, Vayena E. Benefits, challenges, and contributors to success for national eHealth systems implementation: a scoping review. J Am Med Inform Assoc 2021 Aug 13;28(9):2039-2049 [FREE Full text] [CrossRef] [Medline]
- Metzger M, Durand T, Lallich S, Salamon R, Castets P. The use of regional platforms for managing electronic health records for the production of regional public health indicators in France. BMC Med Inform Decis Mak 2012 Apr 03;12:28 [FREE Full text] [CrossRef] [Medline]
- Kiberu VM, Mars M, Scott RE. Barriers and opportunities to implementation of sustainable e-Health programmes in Uganda: a literature review. Afr J Prim Health Care Fam Med 2017 May 29;9(1):e1-e10 [FREE Full text] [CrossRef] [Medline]
- Kujala S, Ammenwerth E, Kolanen H, Ervast M. Applying and extending the FITT framework to identify the challenges and opportunities of successful eHealth services for patient self-management: qualitative interview study. J Med Internet Res 2020 Aug 12;22(8):e17696 [FREE Full text] [CrossRef] [Medline]
- Thouvenot VI, Hardiker N. Facilitating eHealth for all through connecting nurses and the Women Observatory for eHealth. Stud Health Technol Inform 2016;225:777-779. [Medline]
- Sharifi M, Ayat M, Jahanbakhsh M, Tavakoli N, Mokhtari H, Wan Ismail WK. E-health implementation challenges in Iranian medical centers: a qualitative study in Iran. Telemed J E Health 2013 Feb;19(2):122-128. [CrossRef] [Medline]
|CENTRAL: Cochrane Central Register of Controlled Trials|
|DHFKS: Dutch Heart Failure Knowledge Scale|
|EHFSC: European Heart Failure Self-Care Behavior Scale|
|EHFScB-9: 9-item European Heart Failure Self-Care Behavior Scale|
|EHR: electronic health record|
|GRADE: Grading of Recommendations, Assessment, Development and Evaluation|
|HF: heart failure|
|KCCQ: Kansas City Cardiomyopathy Questionnaire|
|MLHFQ: Minnesota Living with Heart Failure Questionnaire|
|OR: odds ratio|
|PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses|
|RCT: randomized controlled trial|
|RoB 2: Cochrane Risk of Bias tool|
|SCHFI: self-care of heart failure index|
|SMD: standardized mean difference|
|WHO: World Health Organization|
Edited by R Kukafka; submitted 12.04.22; peer-reviewed by P Li, M Dorsch; comments to author 09.07.22; revised version received 02.08.22; accepted 06.09.22; published 26.09.22Copyright
©Siru Liu, Jili Li, Ding-yuan Wan, Runyi Li, Zhan Qu, Yundi Hu, Jialin Liu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 26.09.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.