Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 28.07.20 in Vol 22, No 7 (2020): July

Preprints (earlier versions) of this paper are available at, first published Dec 09, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Changing the Health Behavior of Patients With Cardiovascular Disease Through an Electronic Health Intervention in Three Different Countries: Cost-Effectiveness Study in the Do Cardiac Health: Advanced New Generation Ecosystem (Do CHANGE) 2 Randomized Controlled Trial

    1Open Evidence Research Group, Universitat Oberta de Catalunya, Barcelona, Spain

    2Department of R&D, Badalona Serveis Assistencials, Badalona, Spain

    3Smart Homes, Eindhoven, Netherlands

    4Department of Medical and Clinical Psychology, Tilburg University, Tilburg, Netherlands

    5Department of Cardiology, Elisabeth-TweeSteden Hospital, Tilburg, Netherlands

    6Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands

    7Department of Information and Communication Sciences, Universitat Oberta de Catalunya, Barcelona, Spain

    Corresponding Author:

    Jordi Piera-Jiménez, MSc

    Open Evidence Research Group

    Universitat Oberta de Catalunya

    Rambla del Poblenou, 156

    Barcelona, 08018


    Phone: 34 651041515



    Background: During the last few decades, preventing the development of cardiovascular disease has become a mainstay for reducing cardiovascular morbidity and mortality. It has been suggested that interventions should focus more on committed approaches of self-care, such as electronic health techniques.

    Objective: This study aimed to provide evidence to understand the financial consequences of implementing the “Do Cardiac Health: Advanced New Generation Ecosystem” (Do CHANGE 2) intervention, which was evaluated in a multisite randomized controlled trial to change the health behavior of patients with cardiovascular disease.

    Methods: The cost-effectiveness analysis of the Do CHANGE 2 intervention was performed with the Monitoring and Assessment Framework for the European Innovation Partnership on Active and Healthy Ageing tool, based on a Markov model of five health states. The following two types of costs were considered for both study groups: (1) health care costs (ie, costs associated with the time spent by health care professionals on service provision, including consultations, and associated unplanned hospitalizations, etc) and (2) societal costs (ie, costs attributed to the time spent by patients and informal caregivers on care activities).

    Results: The Do CHANGE 2 intervention was less costly in Spain (incremental cost was −€2514.90) and more costly in the Netherlands and Taiwan (incremental costs were €1373.59 and €1062.54, respectively). Compared with treatment as usual, the effectiveness of the Do CHANGE 2 program in terms of an increase in quality-adjusted life-year gains was slightly higher in the Netherlands and lower in Spain and Taiwan.

    Conclusions: In general, we found that the incremental cost-effectiveness ratio strongly varied depending on the country where the intervention was applied. The Do CHANGE 2 intervention showed a positive cost-effectiveness ratio only when implemented in Spain, indicating that it saved financial costs in relation to the effect of the intervention.

    Trial Registration: NCT03178305;

    J Med Internet Res 2020;22(7):e17351





    In the last few decades, prevention at both population and individual levels in patients with established cardiovascular disease (CVD) has become a mainstay for reducing cardiovascular morbidity and mortality [1]. However, CVD remains the leading cause of death globally [2].

    One of the cornerstones of CVD prevention is the promotion of lifestyle changes, including physical activity, a healthy diet, and avoidance of unhealthy behaviors such as smoking and drinking alcohol [1]. However, providing patients with relevant information regarding the importance of lifestyle habits seems to be insufficient to prompt these changes and maintain them over time [3]. Instead, it has been suggested that the preventive paradigm should shift from passive to more committed approaches of self-care based on the following three core elements: self-care maintenance, self-care monitoring, and self-care management [4-6].

    The emergence of solutions based on information and communication technologies (ICTs), such as telemedicine, has greatly contributed to filling some of the gaps of effective self-care. One of the most classical ICT solutions has been the use of self-monitoring devices in patients with high cardiovascular risk to facilitate successful blood pressure (BP) control [7]. The expansion of mobile apps and their peripheral devices has raised the number of ICT-based interventions aimed at improving not only self-monitoring but also behavior changes in various patient profiles, including older people with high cardiovascular risk [8-10]. The emergence of lifestyle data-driven apps illustrates the increasing interest in this approach in various health care areas [11].

    To date, evidence regarding the efficacy of these interventions is still evolving. Clinical guidelines for the prevention of CVD highlight that cost-effectiveness data from randomized controlled trials (RCTs) are scarce, and most data regarding the cost-effectiveness of cardiovascular prevention strategies combine clinical evidence with simulation approaches [1,12,13]. Simulation modeling is currently used to address important issues in clinical practice and health policy that have been very difficult to study within high-quality clinical trials but provide necessary insights for making health care decisions. Nonetheless, assumptions and personal choices are required to conduct simulation modeling, leading to potentially biased outcomes. Transparency in decision-making is therefore critical to adequately understand the observed outcome [14]. In this regard, there is a need for providing the various stakeholders, particularly policy makers, with evidence from nonsimulated research trials to understand the financial consequences of scaling up ICT solutions for health care systems [15]. In this study, we aimed to determine whether the Do Cardiac Health: Advanced New Generation Ecosystem (Do CHANGE) 2 preventive intervention is a cost-effective alternative for patients with CVD in Spain, the Netherlands, and Taiwan.

    The Do CHANGE Project

    The Do CHANGE program was developed as an ICT-based alternative for providing health education, which leads to behavioral changes in care recipients [16,17]. The Do CHANGE program consists of a 6-month intervention with a set of devices that include self-monitoring tools and the Do Something Different (DSD) behavior change program (only available during the first 3 months of the intervention), which has been shown to be effective in changing health behaviors in previous studies targeting different populations [18]. The Do CHANGE program included the following two phases: Do CHANGE 1 and Do CHANGE 2, which were assessed in two consecutive RCTs (Figure 1).

    Patients included in the Do CHANGE 1 study received the DSD behavior change program, which was provided via text messages on patients’ mobile phones. Behavioral flexibility is associated with a broad range of behavioral repertoires, making people more open to experience and adopt new behaviors [19]. This is achieved by disrupting patients’ daily behavioral routines for a short period (eg, a few seconds) with behavioral prompts (referred to as “do’s”) delivered through patients’ mobile phones. These messages challenge patients to do something different and get out of their comfort zone. Do’s have been developed by a multidisciplinary team including cardiologists and psychologists, ensuring that they apply to the target population and are thus related to daily behaviors or needs. Patients received a total of 32 do’s during the 3-month intervention period (2-3 do’s per week). The program was tailored to the cardiac population with slight differences in the program depending on patients’ primary diagnosis (coronary artery disease, heart failure, and hypertension), as the preferred health behaviors may vary depending on the diagnosis [20]. In order to obtain objective measures on patients’ physical functioning, all participants received a BP monitor, the Moves app (Facebook Inc; to register GPS location), and the Careportal (Docobo Ltd, home monitoring device measuring daily symptoms and an electrocardiogram).

    Figure 1. Do CHANGE 1 and 2 randomized controlled trial design including intervention details. Do CHANGE: Do Cardiac Health: Advanced New Generation Ecosystem; DSD: Do Something Different; HF: heart failure.
    View this figure

    The main additional features of the Do CHANGE 2 compared with the Do CHANGE 1 trial were the greater number of devices for self-monitoring and collecting behavioral information, and the capacity of the DSD program to tailor the behavioral prompts to the actual behavior of the care recipient, thus allowing for a personalized approach. Do CHANGE 2 integrates the principle of theory-driven behavioral change techniques, which can guide behavior change, within the offered interventions [21]. As a natural evolution from the Do CHANGE 1 trial, the second phase aimed to increase the ability of a person to express behavior in a more context-dependent way [22], thus being more open to experience and increasing the likelihood of adopting new behaviors [19].

    Care recipients perceived the Do CHANGE 1 program as helpful and easy to use; however, it failed to prompt relevant lifestyle changes (measured with the Health Promotion Lifestyle Profile-II questionnaire) compared with treatment as usual (TAU) [20]. The Do CHANGE 2, based on a more personalized approach, resulted in a relevant change in lifestyle behavior over time in the intervention group. In addition, the intervention was perceived as useful and feasible by patients and health care professionals [23]. In order to provide a broader perspective of the effects of this program, we present herein the results of the cost-effectiveness analysis of the Do CHANGE 2 compared with TAU.


    Trial Design and Patients

    This was a multisite RCT to assess the cost-effectiveness of an ICT-based program to change behavior in patients with CVD compared with TAU. Local clinical specialists and research assistants recruited adult patients treated in the following three hospitals in three different countries: Badalona Serveis Assistencials (Spain), Elisabeth TweeSteden Ziekenhuis (the Netherlands), and Buddhist Tzu-Chi Dalin General Hospital (Taiwan). The planned sample size based on the available project resources was 75 patients for Spain, 75 for the Netherlands, and 100 for Taiwan. Once accepted to participate in the study, patients at each study site were randomized to receive either the TAU or the Do CHANGE 2 intervention. The primary outcomes were lifestyle change and quality of life. Additionally, behavioral flexibility was considered a mediator variable in this relationship. As the project aimed to provide proof of concept and examine the feasibility of the intervention, no sample size calculation was performed a priori. Recruiting a comparable number of patients across the countries was considered relevant to provide proof of concept. The details regarding the study patients and trial design are described in the report by Habivovic et al [16].

    The most remarkable changes from the original study protocol (Do CHANGE 1) were the changes in the DSD program (moving from predefined messages according to the patient psychological profile to nudges tailored according to their behavior as gathered by the measurement devices), the addition of two new wearable devices (Beddit [Apple] and Fitbit [Fitbit Inc]), and the Vire app (Do CHANGE app). Considering the importance of weight in heart failure (HF), patients with this diagnosis also received a weight scale.

    Inclusion Criteria

    Participants were screened from adult patients (aged 18-75 years) who had been primarily diagnosed with either hypertension (ie, systolic BP [SBP]/diastolic BP [DBP] ≥140/90 mmHg in two consecutive measurements), coronary artery disease (ie, occurrence of myocardial infarction or angina pectoris, or previous percutaneous coronary intervention and/or coronary artery bypass graft surgery), or symptomatic HF (ie, New York Heart Association class I-IV). Patients also had to have two or more of the following risk factors: increased cholesterol, smoking, diabetes, sedentary lifestyle, and psychosocial risk factors. The presence or absence of each of the risk factors was assessed following the local guidelines in each participant country. For HF patients, additional inclusion criteria were a diagnosis of systolic or diastolic HF and the presence of HF symptoms (eg, exhaustion, shortness of breath, and chest pain). Other general inclusion criteria were an adequate level of the native language, access to the internet at home, having a smartphone compatible with the apps used in the study, and having the skills necessary to use a personal computer and a smartphone.

    Exclusion Criteria

    Patients with life expectancy less than 1 year, life-threatening comorbidities, a history of psychiatric diseases other than anxiety and depression, and relevant cognitive impairments and those on the waiting list for heart transplantation were excluded from the study.

    The reasoning for establishing the exclusion criteria was to prevent the inclusion of patients whose disease severity may critically increase during the intervention. These patients may perceive participation as an extra burden, are more likely to drop out due to illness-related complaints or early mortality, and may be less likely to benefit from a lifestyle intervention owing to severe comorbidities. Patients with mental illness were also excluded because the intervention might become stressful and trigger symptoms in these patients. The selected exclusion criteria are in line with the DO CHANGE 1 trial, safeguarding that the study is not perceived as a burden and meets patients’ needs as much as possible.


    The Do CHANGE 2 program implemented in this trial was similar to that described by Broers et al for Do CHANGE 1 [20]. Patients randomized to the intervention group received devices for measuring key clinical parameters needed for monitoring their CVD, such as a BP monitor, weight scale (in HF patients only), and the Careportal, which allowed monitoring of daily symptoms and an electrocardiogram. The patient’s location was monitored by the Moves app. In addition to the aforementioned ICT solutions (also used in the Do CHANGE 1 program), the Do CHANGE 2 program included the Vire app, a purpose-designed app to integrate the input from all the monitoring devices, so that the patient could interact with a unique easy-to-use source of information. The app integrates the information coming from the following apps: the Beddit app (provided with the device under the mattress cover sheet) aimed at monitoring sleep efficiency, the Fitbit app (with the wristband) aimed at measuring physical activity through step count, and the HORUS feature embedded in and aimed at collecting pictures of the different meals of the patients in order to provide diet recommendations. Study participants in the Do CHANGE 2 intervention group were also provided with leaflets (Multimedia Appendix 1) and multimedia resources explaining the use of the Do CHANGE environment.

    Like in the Do CHANGE 1 program, patients in the intervention group received a 3-month behavior change program. The program was based on providing care recipients with short messages aimed at disturbing daily routines. Messages were delivered through their mobile phones and suggested them to “do something different.” However, unlike the Do CHANGE 1 program, in the Do CHANGE 2 program, behavioral nudges were not only predefined according to the patient’s personality profile but also tailored to the patient’s behavior, as recorded by the monitoring devices. These behavior-driven messages called to-do’s were delivered to the patients based on their current functioning. Patients receive their do’s and to-do’s through the Careportal or the Vire app, or via SMS, depending on patients’ preferences [16].

    The GPS data from the Moves app and activity data from the Fitbit device were used to calculate higher abstraction scores called activity, variety, and social opportunities. The to-do’s were tailored based on the trends of these scores over time (eg, a patient with declining activity would receive a message that focuses on increasing activity). The granularity of this system is not restricted to one score; alternatively, new scores are calculated for each update of data, and the system determines whether the score is a “target” for a message. Multiple combinations of targets are possible; therefore, a to-do can tackle both variety and social opportunities if needed based on the scores. A detailed description of the construction process of to-do’s can be found in previously published work [23,24].

    Besides receiving personalized prompts (eg, based on activity levels), patients were contacted each week by the research assistant to check how everything was going and to provide dietary coaching. This might have greatly contributed to the high adherence rate during the first 3 months, as they received personalized feedback. After this period, patients were not contacted in person anymore; however, they were allowed to keep all the devices (eg, Fitbit, Beddit, etc) in order to monitor their behavior for the remaining 3 months.

    Cost-Effectiveness Analysis

    The cost-effectiveness analysis of the Do CHANGE 2 RCT was conducted using the Monitoring and Assessment Framework for the European Innovation Partnership on Active and Healthy Ageing (MAFEIP) tool [25]. The MAFEIP tool performs a cost-utility analysis through a web app that analyzes incremental costs and effects. The cost-effectiveness estimates are based on the principles of decision analytic modeling and Markov models that assess the impacts that health-related innovations have in terms of health outcomes and resource usage. For Do CHANGE 2, we parametrized the tool on a Markov model of five health states from the perspective of the three service providers (Figure 2) [26].

    Figure 2. Markov model of five health states applied for the Do CHANGE cost-effectiveness analysis. Do CHANGE: Do Cardiac Health: Advanced New Generation Ecosystem.
    View this figure

    The MAFEIP tool requires the user to provide the following three main types of inputs: (1) the health states and the corresponding transition probabilities between them, (2) the costs, and (3) the utility, for which the EuroQol five-dimension three-level (EQ-5D-3L) questionnaire was used as recommended by the National Institute for Health and Care Excellence [27].

    In order to estimate the incremental health gain from a particular intervention delivered, the defined model needs to be run twice. Once with parameter estimates for the respective intervention under assessment (ie, Do CHANGE 2), and once with parameters corresponding to the standard care scenario (ie, TAU). In the model, these two scenarios may differ in terms of transition probabilities (disease incidence, recovery, and mortality), as well as the utility weight and health care and societal costs related to the health states. When the model simulates a hypothetical cohort of patients moving between these health states over time, the differences in survival, utility, and cost accumulate until reaching an estimate of the incremental costs (ΔC) and health effects (ΔE) that can be expected from the intervention under evaluation. Therefore, the tool can be used to estimate the incremental cost-effectiveness ratio (ICER=ΔC/ΔE) or the incremental net monetary benefit (ΔE×λ−ΔC) of one intervention compared with another, where λ is defined as the willingness-to-pay (WTP) threshold for an additional unit of health gain.

    Besides the transition probabilities among health states, the utility, and the cost, the tool allows the user to include the relative risks for mortality, the discount rates, and the time horizon for the analysis (cycle length). The parameters included in this analysis and their assessments during the study are explained in detail below.

    Definition of Health States

    The first stage in the construction of a Markov model is defining the different states of the disease in relation to the important clinical and economical effects of the disease. Evidence suggests that high BP is the predominant risk factor for CVD [28]. Following the scientific evidence and for the purpose of the Do CHANGE assessment, the health states were established based on SBP and DBP, according to the classification of the American Heart Association [29] as follows: baseline disease stage (SBP <120 mmHg and DBP <80 mmHg), progressive disease stage 1 (SBP 120-129 mmHg and DBP <80 mmHg), progressive disease stage 2 (SBP 130-139 mmHg or DBP 80-89 mmHg), progressive disease stage 3 (SBP ≥140 mmHg or DBP ≥90 mmHg), and death. On a side note, the latest stage was not included, as it is considered to be a hypertensive crisis (SBP >180). The transition probabilities were calculated based on the changes between the initial health states (at baseline) and those at 3 months. These transitions can be of incidence (ie, the annual probability for an individual to move from baseline to each progressive stage of the disease) and recovery (ie, the annual probability of improving).

    Cost Estimate

    The following two types of costs were considered for both study groups: (1) health care costs (ie, costs associated with the time spent by health care professionals on service provision, including consultations, unplanned hospitalizations, etc) and (2) societal costs (ie, costs associated with the time spent by patients and informal caregivers on care activities).

    The data collected by the research team in each country were provided in local currency units (Euro for both Spain and the Netherlands and New Taiwan Dollars for Taiwan). Taiwan prices were converted into a common basis of 2018 Euros using simple exchange rate conversion factors, reflecting the average market exchange rate between New Taiwan Dollars and Euros during the year in question (NT $1=€0.02862). A currency exchange rate of €1=US $1.12 is applicable (average exchange rate for 2018).

    For computing the time spent by health care professionals, we considered an average duration of 15 minutes and 25 minutes for a visit to a general practitioner and specialist, respectively. The personnel cost was established based on the average cost for one full‐time employee, including employer contributions to social security. The average hourly costs were €29.23 (Spain), €59 (the Netherlands), and €16.88 (Taiwan) for a general practitioner; €20.79 (Spain), €40 (the Netherlands), and €6.33 (Taiwan) for a nurse; and €34.81 (Spain), €113.50 (the Netherlands), and €84.91 (Taiwan) for a specialist. The estimations of cost per bed‐day for hospitalizations were €733.56 (Spain) and €1853.57 (the Netherlands), which were obtained by dividing the expenditure for inpatient curative care in hospitals by hospital bed‐days for services of curative care (both publicly available) [30]. The corresponding costs for Taiwan were calculated by dividing the average expenses of hospitalization by the number of hospital days, which was €342.50 [31].

    Societal costs differed according to the study group. For patients allocated to the control group, we considered the extra travel time spent by patients and caregivers in usual care compared with Do CHANGE 2, whereas for those allocated to the intervention group, we considered the time spent by patients using the service.

    Additionally, for patients in the Do CHANGE group, the following costs were added: time spent by professionals in service development and training (4 hours per professional, divided by the number of randomized subjects), time spent by nurses in training patients (30 minutes per participant) and installing the Do CHANGE service ecosystem (45 minutes per patient), and the cost of the devices (including taxes). Data are provided in Euro (2018).

    Utility Calculation

    Utility was estimated using the EQ-5D-3L tool [32]. The EQ-5D is a standardized questionnaire-based measure of self-rated health-related quality of life developed by the EuroQol Group to provide a simple and generic measure widely used for both clinical and economic appraisals. In the case of the Do CHANGE 2 project, we used the EQ-5D-3L version, which was administered at baseline and at 3 and 6 months.

    The resulting scores of the questionnaire were weighted using the trade-off method previously described for Spain [33], the Netherlands [34], and Taiwan [35]. The EQ-5D health states, defined by the EQ-5D descriptive system, were subsequently converted into a single summary index by applying specific weights to each of the levels in each dimension of quality of life. The index was calculated by deducting the appropriate weights from 1, which was the value assigned to full health. In the case of the cost-effectiveness analysis of the Do CHANGE intervention, our interest was to measure the change over time, rather than the absolute values. Therefore, we calculated the changes in utility for each of the five health states and for each of the study conditions and added a common initial measure for the whole sample to each of them. The MAFEIP requires EQ-5D utility scores combined with time indicators to compute quality-adjusted life-years (QALYs) automatically.

    Relative Risks of Mortality, Discount Rates, and Time Horizon

    The MAFEIP tool allows mortality rates to be internally calculated by using the all-cause mortality rates (age- and sex-dependent) extracted from the Human Mortality Database. The relative risk of mortality is a measure that estimates the mortality in a specific population (eg, people who participated in the Do CHANGE 2 study) compared with (ie, divided by) the mortality in a reference population or condition (in this case, from the Human Mortality Database). The reference condition considers CVD mortality for the population of the specified country (ie, Spain, the Netherlands, and Taiwan).

    The discount factors for costs and effects are used to estimate outcomes while taking into account the future costs and health effects, that is, adjusting for differences in the timing of costs (expenditure) compared with health benefits (outcomes). Therefore, adequately applied discount factors express future costs or benefits at today’s equivalent value. In Do CHANGE 2, we followed the recommendations from the Health Technology Assessment authorities in each country [36-40]. The discount factors for costs and health outcomes applied in Do CHANGE 2 were 3% for both costs and health outcomes in Taiwan and Spain, and 4% for costs and 1.5% for health factors in the Netherlands.

    Finally, the MAFEIP framework allows specifying the number of cycles that the model will run, which represents the timeframe in which the impact of the intervention will be evaluated. Markov models are used to simulate both short-term and long-term processes (ie, CVDs) [41]. In the case of Do CHANGE 2, we wanted to see estimates of the incremental costs (and effects) of the intervention in a time horizon of 5 years. The cycle length we selected is not in line with CVD’s etiology (ie, a long disease development process) [41] but considers the nature of the intervention and the maximum time frame it can be sustained in light of the deprecation of the wearable and medical device technology that was used.

    WTP Threshold

    The balance between the economic benefit and clinical effectiveness varies and is entirely dependent on the relationship between the ICER and the threshold value the society is willing to pay at a specific point of time, which is known as the WTP threshold. The fact that WTP thresholds can be specified after the ICER is calculated raises concerns about researchers selecting WTP thresholds that suit their hypothesis, hence compensating for technology of relatively lower value [42].

    While there is an agreement about CVDs being preventable to a certain extent, there has also been a discussion as to whether prevention interventions offer good value for money. Previous research has shown a positive relation between lower lifetime risk for CVD mortality and increased survival and quality of life [43]. Prevention strategies can bring relevant benefits at lower costs relative to most treatment options provided that their cost-effectiveness value is almost always below the accepted societal WTP [44].

    For the Do CHANGE 2, we selected a WTP threshold of €15,000/QALY for the three countries, not corresponding to the value recommended by local Health Technology Assessment guidelines. The WTP threshold is lower in all cases. We set a lower WTP threshold in order to avoid the concerns mentioned above and to fit the results of the technology, and considering comparisons with other preventive interventions.

    Data Collection and Analysis

    The questionnaires, as defined in the study protocol, were loaded into the web tool LimeSurvey [45] and collected by local research assistants. Data from the medical devices (built-in electrocardiogram monitor, blood pressure meter, and weight scale) were collected through the Careportal. The data generated by the wearable devices (Fitbit and Beddit) were continuously monitored and integrated through the Vire app. Information regarding resource consumption (eg, hospitalization costs) was collected by local research assistants from the local electronic medical records.

    IBM SPSS Statistics for Windows Version 21.0 (IBM Corp) was used to perform the statistical analyses of the effectiveness study, and R (R Core Team 2018) and RStudio (RStudio Team 2016) were used to calculate the transition probabilities and utilities. We used the MAFEIP tool to perform the cost-effectiveness analysis.


    Study Participants

    Figure 3 shows the overall flow chart of participant recruitment in the Do CHANGE 2 project. Of the 4540 patients assessed for eligibility at all three sites, 238 were enrolled in the study (120 in the intervention group [Do CHANGE] and 118 in the control group).

    Owing to relevant differences in patient recruitment strategies between sites, most patients screened in Spain and the Netherlands agreed to participate, whereas many patients in Taiwan refused to participate. Based on retrospective investigation, it appeared that all inclusion criteria in Taiwan were checked after the patients were approached for participation. Hence, the number of patients that were approached appeared to be much higher. In Spain and the Netherlands, patients who fulfilled the basic inclusion criteria were offered to participate. If the same strategy was applied in Taiwan, the refusal rate would have been much lower. Both Spain and the Netherlands met the target number of participants as defined in the project plan, whereas Taiwan did not reach the planned number of participants.

    Eighteen patients dropped out before the end of the 6-month follow-up, demonstrating a very high adherence rate. Owing to the personalized nature of the intervention (eg, relevant behavioral prompts and personalized feedback), we expected the adherence to be high. We believe that the nature of the 3-month intervention, where blended and personalized care was provided, combined with the monitoring devices after that period contributed to the high adherence. Patients were engaged in their health management, and therefore, they might be more willing to proceed with monitoring.

    Of the 114 participants who were in the program for at least 3 months, 72 (60%) claimed to have carried out all the nudges provided by the DSD and 86 (72%) reckoned the program was useful. Reasons for not adhering to the program were mainly having no time (8/84, 6.7%), not feeling like it (2/84, 1.7%), and falling ill (2/84, 1.7%). One of the participants who quit the program disclosed that being confronted with the illness on a daily basis became too stressful. Moreover, in some cases, the confrontation for some partners to deal with the illness of their husband or wife caused anxiety.

    Table 1 summarizes the main demographic characteristics of the three participating countries. None of the variables collected showed relevant systematic differences between study conditions at baseline, except for Spain, with participants in the Do CHANGE 2 group being younger (Do CHANGE 2 group vs TAU group: mean 53.8 years, SD 15.8 years vs mean 67.4 years, SD 7.5 years), having a higher education (mean 14.5 years, SD 6.3 years vs mean 9.1 years, SD 5.5 years), and showing a higher employed proportion (17/37, 45.9% vs 4/37, 10.8%). Multimedia Appendix 2 presents the clinical characteristics of the study sample, medication, and psychological symptoms. The only significant difference at baseline was observed in psychotropic medication for participants in the TAU group in Spain (P=.03), which is consistent with the population in the control group being older.

    Figure 3. Flow chart of participant recruitment (aggregated numbers for Spain, the Netherlands, and Taiwan).
    View this figure
    Table 1. Demographic baseline characteristics of the total sample (N=238).
    View this table

    Model Input

    The model input for the MAFEIP tool included data regarding the health states (and transition probabilities) of study participants, the costs associated with each study group, and the utility estimate. Table 2 summarizes the distribution of study participants across the MAFEIP health states at study start, as well as the transition probabilities between these states, computed by considering data recorded at month 3. Table 3 summarizes the total health care and societal costs for each group and each state. The detailed amounts for each type of health care and societal cost are provided in Multimedia Appendix 3. The specific costs associated with the implementation of the Do CHANGE environment are presented in Table 4. The utility values calculated from the EQ-5D-3L scores and the estimated utility computed by adding the initial common measure are described in Table 5. No systematic differences were observed between study conditions at baseline. The utility values for the whole study sample in Spain, the Netherlands, and Taiwan were 0.897, 0.842, and 0.854, respectively.

    Table 2. Frequency and percentage of patients across the various health states (N=207).
    View this table
    Table 3. Total health care and societal costs for each of the study groups (N=207).
    View this table
    Table 4. Costs associated with the implementation of the Do CHANGE 2 intervention (N=92).
    View this table
    Table 5. Calculation of utility (N=207).
    View this table


    After cleaning the data, 207 participants were included in the cost-effectiveness analysis (92 in the intervention group and 115 in the control group). The Do CHANGE 2 intervention was less costly in Spain (incremental cost was −€2514.90) and more costly in the Netherlands and Taiwan (incremental costs were €1373.59 and €1062.54, respectively). Figure 4 shows the cost-effectiveness plane for the three countries. The cost-effectiveness plane plots the incremental cost of the intervention on the y-axis and the incremental health outcome (measured in QALYs) on the x-axis. The diagonal line represents the WTP per additional QALY gained, which is the maximum amount that the society is willing to give in exchange for a better quality of life. Different thresholds may also be selected. Depending on the location of the ICER point in this plane, one would be able to interpret whether an intervention is cost-effective. When the ICER point is within the lower-right quadrant, it means the intervention is accepted (it is more effective and cheaper), and when it is within the upper-left quadrant, it means that the intervention is not accepted (it is less effective and more expensive). If the ICER point lies in the other two quadrants, then the intervention may or may not be accepted depending on the ICER and WTP threshold values.

    Figure 4. Cost-effectiveness plane for the Do CHANGE intervention in Spain, the Netherlands, and Taiwan. The dotted line shows the willingness-to-pay threshold of €15,000 per QALY. Do CHANGE: Do Cardiac Health: Advanced New Generation Ecosystem; ICER: incremental cost-effectiveness ratio; QALY: quality-adjusted life-year; WTP: willingness to pay.
    View this figure

    Compared with usual care, the effectiveness of the Do CHANGE 2 program in terms of QALY gains was slightly higher in the Netherlands (incremental effect of 0.011) and lower in Spain and Taiwan (incremental effects of −0.134 and −0.094, respectively). Even though the Do CHANGE program was more effective than usual care in the Netherlands, the relative costs for gained utility (€124,489.27 per QALY) were too high to accept this intervention. Taken together, the Do CHANGE intervention would only be accepted in Spain, where it would help save €18,769.05 per QALY.

    We also calculated the incremental cost and health-related quality of life for every age-gender combination in the specified target population. The data are presented in Multimedia Appendix 4.


    Principal Findings

    In this cost-effectiveness analysis of an ICT-based intervention to change the health behavior of patients with CVD (the Do CHANGE program) assessed in a multicenter RCT, we found that the ICER strongly varied depending on the country where the intervention was applied. The Do CHANGE 2 program was slightly more effective than usual care in the Netherlands only, albeit at an incremental cost too high to accept the intervention at the selected WTP threshold (€15,000 per QALY). The same intervention was less effective but less costly than usual care in Spain. In Taiwan, the intervention resulted in the dominated option (less effective and more expensive). Therefore, implementation of the Do CHANGE 2 intervention is only recommended in Spain, where it could allow saving financial costs taking into account the costs and effects of the intervention. We further tested the results with higher WTP thresholds (ie, €30,000 per QALY), with results remaining in the same line.

    Contextualization With Previous Work

    There is a large body of evidence showing that ICT solutions, including mobile-based telemonitoring, improve the quality and outcomes of care in patients with CVD [46]. Unfortunately, the cost-effectiveness assessment is often disregarded, and many studies reporting cost information do not meet a quality standard high enough to determine the cost-effectiveness or cost-utility of the intervention [46,47].

    Regardless of the quality in reporting of individual studies, evidence on the cost-effectiveness of ICT-based lifestyle interventions is rather controversial, and many authors have acknowledged difficulties in drawing strong conclusions in this regard [48-50]. Overall, cost-effectiveness evaluations of secondary and tertiary prevention strategies for patients with CVD are challenged by the multiple factors influencing the outcomes and costs, such as baseline cardiovascular risk, the cost of drugs or other interventions, reimbursement procedures, and implementation of preventive strategies [1]. In the case of telemedicine approaches, it has been recognized that cost-effectiveness depends largely on local aspects of the individual service (and care as usual) being evaluated, and a service may be highly cost-effective in one context but highly ineffective when transferred to another context [47]. This was the case in our analysis, which yielded controversial results regarding the cost-effectiveness of the intervention in the different countries involved. Importantly, the success of an ICT-based lifestyle intervention strongly depends on the willingness of individuals to adopt the intervention, which is likely to be associated with cultural constraints and, therefore, to be country specific.

    These differences were particularly pervasive between the Netherlands and Taiwan, where the cost-effectiveness planes showed an almost opposite profile, although the same intervention was implemented. We associate this situation with the majority of patients recruited in Taiwan having hypertension as the primary diagnosis and medical consultations involving health care professionals (ie, physicians) in the Netherlands being too expensive for them to devote time to a prevention intervention that could perhaps be conducted by nurses. This finding supports the need to evaluate the cost-effectiveness of these types of interventions within each context in order to provide the various stakeholders with evidence to understand the financial consequences of scaling up ICT solutions for health care.


    The main limitation of our trial was the sample size, which was constrained by budget restrictions. The discrepancy between the target and the actual sample size was mainly due to technical difficulties in recruiting participants, who had to enroll in the trial for a minimum duration of 6 months. The low sample size might have constrained the representativeness of the results and made them more sensitive to biases associated with patients with extreme behaviors (ie, outliers). Actually, the extremely low number of participants within, for instance, the states “baseline” and “progressive disease stage 1,” might explain the unrealistic utilities of these patients before the intervention (eg, 0.962 and 1, respectively, for the control group), which were considerably higher than the average reported in larger RCTs involving HF (utility 0.84) [51]. Another example associated with this limitation is the relevant differences for patients allocated to the intervention group in Spain, who were younger and had higher education.

    Second, an acknowledgment must be made regarding the limitation associated with the heterogeneous characteristics of the study sample for the primary diagnosis and cultural setting (ie, Spain, the Netherlands, and Taiwan), which may have contributed to the heterogeneous results across countries.

    Unlike other cost-effectiveness analyses, we did not consider the contribution of medication to the health care costs. Although this might have increased the accuracy of the absolute costs, from a clinical point of view, it is unrealistic that the prescribed medicines would change throughout a 3-month time lapse. Since our main interest was assessing the change in costs rather than describing the actual values, we considered that it was more appropriate to exclude this concept from the cost-effectiveness analysis.

    Finally, although RCTs are considered the gold standard for assessing cost-effectiveness, some authors criticize that they might miss information regarding how the intervention fits into routine practice [15].


    Our results suggest that the Do CHANGE 2 environment may help reduce health care costs associated with the management of patients with CVD in certain settings. However, changing health behavior and assessing the impact of this change on health care and societal costs remain big challenges. In line with previous research in this field, our assessment does not allow drawing strong conclusions in this regard. Irrespective of the specific cost-effectiveness of the Do CHANGE 2 program, our results highlight the high heterogeneity that ICT-based interventions might show depending on the country where they are implemented and stress the need for assessing each intervention in all areas before scaling up implementation.


    The continuous development of the MAFEIP tool is funded by the European Commission’s Horizon 2020 program under the projects WE4AHA (grant number: 769705) and DigitalHealthEurope (grant number: 826353). The Do CHANGE study was funded by the European Commission’s Horizon 2020 program (grant number: 463735). The funder had no role in data collection or analysis, the decision to publish, or the preparation of the manuscript. The Do CHANGE team received funding for Research and Innovation from the European Union for the project. Within the consortium, two small- and medium-sized enterprises (Docobo and Do Something Different) and one start-up (Onmi) were financially supported to develop their products. The authors would like to acknowledge all the patients who were willing to participate in this study within the three countries and were willing to share their data, thus making this work possible. Moreover, the authors would like to thank all students and health care professionals for their support in recruitment and data management. The authors also want to acknowledge the inestimable work conducted by the Do CHANGE consortium members ( Lastly, the authors would like to thank Gerard Carot-Sans (PhD) for providing medical writing assistance during the preparation of the manuscript and Leah Bührmann (MSc) for her help with SPSS analysis.

    Authors' Contributions

    JPJ, JWMGW, MH, EB, and DVB contributed to the study design and data collection. JPJ, MW, and DVB conducted the statistical analyses. JPJ drafted the manuscript. All authors critically revised and approved the final version of the manuscript. FLV and FF supervised the study.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Do CHANGE leaflet.

    PDF File (Adobe PDF File), 796 KB

    Multimedia Appendix 2

    Clinical characteristics of the study sample, medication, and psychological symptoms.

    PDF File (Adobe PDF File), 179 KB

    Multimedia Appendix 3

    Health care and societal costs.

    XLSX File (Microsoft Excel File), 17 KB

    Multimedia Appendix 4

    Incremental costs and effects (age and gender specific) per pilot site.

    PDF File (Adobe PDF File), 674 KB


    1. Piepoli M, Hoes A, Agewall S, Albus C, Brotons C, Catapano A, ESC Scientific Document Group. 2016 European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts)Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur Heart J 2016 Aug 01;37(29):2315-2381 [FREE Full text] [CrossRef] [Medline]
    2. Epidemiology of cardiovascular disease. Nature Reviews Cardiology. 2019 Feb 28.   URL: [accessed 2019-07-09]
    3. Moltu C, Stefansen J, Svisdahl M, Veseth M. Negotiating the coresearcher mandate - service users' experiences of doing collaborative research on mental health. Disabil Rehabil 2012;34(19):1608-1616. [CrossRef] [Medline]
    4. Broers ER, Gavidia G, Wetzels M, Ribas V, Ayoola I, Piera-Jimenez J, et al. Usefulness of a Lifestyle Intervention in Patients With Cardiovascular Disease. Am J Cardiol 2020 Feb 01;125(3):370-375 [FREE Full text] [CrossRef] [Medline]
    5. Charlson ME, Wells MT, Peterson JC, Boutin-Foster C, Ogedegbe GO, Mancuso CA, et al. Mediators and moderators of behavior change in patients with chronic cardiopulmonary disease: the impact of positive affect and self-affirmation. Transl Behav Med 2014 Mar;4(1):7-17 [FREE Full text] [CrossRef] [Medline]
    6. Riegel B, Moser D, Buck H, Dickson V, Dunbar S, Lee C, American Heart Association Council on CardiovascularStroke Nursing; Council on Peripheral Vascular Disease;Council on Quality of CareOutcomes Research. Self-Care for the Prevention and Management of Cardiovascular Disease and Stroke: A Scientific Statement for Healthcare Professionals From the American Heart Association. J Am Heart Assoc 2017 Aug 31;6(9) [FREE Full text] [CrossRef] [Medline]
    7. Omboni S. Connected Health in Hypertension Management. Front Cardiovasc Med 2019;6:76 [FREE Full text] [CrossRef] [Medline]
    8. Changizi M, Kaveh MH. Effectiveness of the mHealth technology in improvement of healthy behaviors in an elderly population-a systematic review. Mhealth 2017;3:51 [FREE Full text] [CrossRef] [Medline]
    9. Brunetti ND, Scalvini S, Acquistapace F, Parati G, Volterrani M, Fedele F, et al. Telemedicine for cardiovascular disease continuum: A position paper from the Italian Society of Cardiology Working Group on Telecardiology and Informatics. Int J Cardiol 2015 Apr 01;184:452-458. [CrossRef] [Medline]
    10. Franklin NC, Lavie CJ, Arena RA. Personal health technology: A new era in cardiovascular disease prevention. Postgrad Med 2015 Mar;127(2):150-158 [FREE Full text] [CrossRef] [Medline]
    11. Fernandez-Luque L, Aupetit M, Palotti J, Singh M, Fadlelbari A, Baggag A. Health Lifestyle Data-Driven Applications Using Pervasive Computing. In: Big Data, Big Challenges: A Healthcare Perspective. Lecture Notes in Bioengineering. Switzerland: Springer; 2019:115-126.
    12. McConnachie A, Walker A, Robertson M, Marchbank L, Peacock J, Packard CJ, et al. Long-term impact on healthcare resource utilization of statin treatment, and its cost effectiveness in the primary prevention of cardiovascular disease: a record linkage study. Eur Heart J 2014 Feb;35(5):290-298 [FREE Full text] [CrossRef] [Medline]
    13. Mistry H, Morris S, Dyer M, Kotseva K, Wood D, Buxton M, EUROACTION study group. Cost-effectiveness of a European preventive cardiology programme in primary care: a Markov modelling approach. BMJ Open 2012;2(5):e001029 [FREE Full text] [CrossRef] [Medline]
    14. Ramsey SD, McIntosh M, Etzioni R, Urban N. SIMULATION MODELING OF OUTCOMES AND COST EFFECTIVENESS. Hematology/Oncology Clinics of North America 2000 Aug;14(4):925-938. [CrossRef]
    15. Bergmo T. How to Measure Costs and Benefits of eHealth Interventions: An Overview of Methods and Frameworks. J Med Internet Res 2015 Nov 09;17(11):e254 [FREE Full text] [CrossRef] [Medline]
    16. Habibović M, Broers E, Piera-Jimenez J, Wetzels M, Ayoola I, Denollet J, et al. Enhancing Lifestyle Change in Cardiac Patients Through the Do CHANGE System ("Do Cardiac Health: Advanced New Generation Ecosystem"): Randomized Controlled Trial Protocol. JMIR Res Protoc 2018 Feb 08;7(2):e40 [FREE Full text] [CrossRef] [Medline]
    17. Do CHANGE.   URL: [accessed 2019-12-01]
    18. Fletcher B, Hanson J, Page N, Pine K. FIT – Do Something Different. Swiss Journal of Psychology 2011 Jan;70(1):25-34. [CrossRef]
    19. Pine K, Fletcher B. Time to shift brain channels to bring about effective changes in health behaviour. Perspect Public Health 2014 Jan;134(1):16-17. [CrossRef] [Medline]
    20. Broers ER, Kop WJ, Denollet J, Widdershoven J, Wetzels M, Ayoola I, et al. A Personalized eHealth Intervention for Lifestyle Changes in Patients With Cardiovascular Disease: Randomized Controlled Trial. J Med Internet Res 2020 May 22;22(5):e14570 [FREE Full text] [CrossRef] [Medline]
    21. Pfaeffli Dale L, Dobson R, Whittaker R, Maddison R. The effectiveness of mobile-health behaviour change interventions for cardiovascular disease self-management: A systematic review. Eur J Prev Cardiol 2016 May;23(8):801-817. [CrossRef] [Medline]
    22. Duckworth RA. Evolution of Personality: Developmental Constraints on Behavioral Flexibility. The Auk 2010 Oct;127(4):752-758. [CrossRef]
    23. Broers ER, Widdershoven J, Denollet J, Lodder P, Kop WJ, Wetzels M, et al. Personalized eHealth Program for Life-style Change. Psychosomatic Medicine 2020;82(4):409-419. [CrossRef]
    24. Ayoola I. Advanced cardiac-health transformation ecosystem ACTivatE: contributions to the development of a health ecosystem for integrated cardiovascular disease management and an approach to changing behaviour. The Netherlands: Technische Universiteit Eindhoven; Sep 26, 2018:1-263.
    25. Monitoring and Assessment Framework for the European Innovation Partnership on Active and Healthy Ageing. European Comission.   URL: [accessed 2019-12-01]
    26. Alhambra-Borrás T, Durá-Ferrandis E, Ferrando-García M. Effectiveness and Estimation of Cost-Effectiveness of a Group-Based Multicomponent Physical Exercise Programme on Risk of Falling and Frailty in Community-Dwelling Older Adults. Int J Environ Res Public Health 2019 Jun 13;16(12) [FREE Full text] [CrossRef] [Medline]
    27. Position statement on use of the EQ-5D-5L valuation set for England. National Institute for Health and Care Excellence.   URL: https:/​/www.​​about/​what-we-do/​our-programmes/​nice-guidance/​technology-appraisal-guidance/​eq-5d-5l [accessed 2019-12-01]
    28. Fuchs FD, Whelton PK. High Blood Pressure and Cardiovascular Disease. Hypertension 2020 Feb;75(2):285-292. [CrossRef]
    29. Whelton P, Carey R, Aronow W, Casey DE, Collins KJ, Dennison Himmelfarb C, et al. 2017 ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA Guideline for the Prevention, Detection, Evaluation, and Management of High Blood Pressure in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol 2018 May 15;71(19):e127-e248 [FREE Full text] [CrossRef] [Medline]
    30. Eurostat - Microdata. European Commission. 2017.   URL: [accessed 2019-12-01]
    31. Cheng K, Lu C, Chung Y, Huang M, Shen H, Chen H, et al. ICU service in Taiwan. J Intensive Care 2014;2(1):8 [FREE Full text] [CrossRef] [Medline]
    32. Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med 2001 Jul;33(5):337-343. [CrossRef] [Medline]
    33. Badia X, Roset M, Herdman M, Kind P. A Comparison of United Kingdom and Spanish General Population Time Trade-off Values for EQ-5D Health States. Med Decis Making 2016 Jul 02;21(1):7-16. [CrossRef]
    34. Lamers LM, McDonnell J, Stalmeier PF, Krabbe PF, Busschbach JJ. The Dutch tariff: results and arguments for an effective design for national EQ-5D valuation studies. Health Econ 2006 Oct;15(10):1121-1132. [CrossRef] [Medline]
    35. Lee H, Hung M, Hu F, Chang Y, Hsieh C, Wang J. Estimating quality weights for EQ-5D (EuroQol-5 dimensions) health states with the time trade-off method in Taiwan. J Formos Med Assoc 2013 Nov;112(11):699-706 [FREE Full text] [CrossRef] [Medline]
    36. Mora-Ripoll R, Gilabert-Perramon A, Oliva-Moreno J, Puig-Junoy J. Guidance For Economic Evaluation and Budget Impact Analysis For Pharmaceuticals in Catalonia (Spain). Value Health 2014 Nov;17(7):A447 [FREE Full text] [CrossRef] [Medline]
    37. Rovira J, Antoñanzas F. Economic analysis of health technologies and programmes. A Spanish proposal for methodological standardisation. Pharmacoeconomics 1995 Sep;8(3):245-252. [CrossRef] [Medline]
    38. López-Bastida J, Oliva J, Antoñanzas F, García-Altés A, Gisbert R, Mar J, et al. Spanish recommendations on economic evaluation of health technologies. Eur J Health Econ 2010 Oct;11(5):513-520. [CrossRef] [Medline]
    39. PE Guidelines Around The World: The Netherlands. International Society for Pharmacoeconomics and Outcomes Research (ISPOR). 2017.   URL: [accessed 2019-12-01]
    40. Demarteau N, Tang C, Chen H, Chen C, Van Kriekinge G. Cost-effectiveness analysis of the bivalent compared with the quadrivalent human papillomavirus vaccines in Taiwan. Value Health 2012;15(5):622-631 [FREE Full text] [CrossRef] [Medline]
    41. Saidi O, O'Flaherty M, Zoghlami N, Malouche D, Capewell S, Critchley J, et al. Comparing Strategies to Prevent Stroke and Ischemic Heart Disease in the Tunisian Population: Markov Modeling Approach Using a Comprehensive Sensitivity Analysis Algorithm. Comput Math Methods Med 2019;2019:2123079 [FREE Full text] [CrossRef] [Medline]
    42. Padula W, Chen H, Phelps C. PRM39 - Is the Choice of Willingness-to-Pay Threshold in Cost-Utility Analysis Endogenous to the Resulting Value of the Technology? Value in Health 2018 Oct;21:S362. [CrossRef]
    43. Lloyd-Jones DM, Leip EP, Larson MG, D’Agostino RB, Beiser A, Wilson PW, et al. Prediction of Lifetime Risk for Cardiovascular Disease by Risk Factor Burden at 50 Years of Age. Circulation 2006 Feb 14;113(6):791-798. [CrossRef]
    44. Eze-Nliam CM, Zhang Z, Weiss SA, Weintraub WS. Cost-effectiveness Assessment of Cardiac Interventions: Determining a Socially Acceptable Cost Threshold. Interv Cardiol (Lond) 2014;6(1):45-55 [FREE Full text] [CrossRef] [Medline]
    45. Lime Survey.   URL: [accessed 2019-12-01]
    46. Piette JD, List J, Rana GK, Townsend W, Striplin D, Heisler M. Mobile Health Devices as Tools for Worldwide Cardiovascular Risk Reduction and Disease Management. Circulation 2015 Nov 24;132(21):2012-2027 [FREE Full text] [CrossRef] [Medline]
    47. Whitten P, Mair F, Haycox A, May C, Williams T, Hellmich S. Systematic review of cost effectiveness studies of telemedicine interventions. BMJ 2002 Jun 15;324(7351):1434-1437 [FREE Full text] [CrossRef] [Medline]
    48. Young LA, Buse JB, Weaver MA, Vu MB, Mitchell CM, Blakeney T, Monitor Trial Group. Glucose Self-monitoring in Non-Insulin-Treated Patients With Type 2 Diabetes in Primary Care Settings: A Randomized Trial. JAMA Intern Med 2017 Jul 01;177(7):920-929 [FREE Full text] [CrossRef] [Medline]
    49. Greving JP, Kaasjager HA, Vernooij JW, Hovens MM, Wierdsma J, Grandjean HM, et al. Cost-effectiveness of a nurse-led internet-based vascular risk factor management programme: economic evaluation alongside a randomised controlled clinical trial. BMJ Open 2015 May 20;5(5):e007128 [FREE Full text] [CrossRef] [Medline]
    50. Monahan M, Jowett S, Nickless A, Franssen M, Grant S, Greenfield S, et al. Cost-Effectiveness of Telemonitoring and Self-Monitoring of Blood Pressure for Antihypertensive Titration in Primary Care (TASMINH4). Hypertension 2019 Jun;73(6):1231-1239. [CrossRef]
    51. Lee D, Wilson K, Akehurst R, Cowie MR, Zannad F, Krum H, Eplerenone in Mild Patients Hospitalization And Survival Study in Heart Failure (EMPHASIS-HF) Study. Cost-effectiveness of eplerenone in patients with systolic heart failure and mild symptoms. Heart 2014 Nov;100(21):1681-1687 [FREE Full text] [CrossRef] [Medline]


    BP: blood pressure
    CVD: cardiovascular disease
    DBP: diastolic blood pressure
    DSD: Do Something Different
    Do CHANGE: Do Cardiac Health: Advanced New Generation Ecosystem
    EQ-5D: EuroQol five dimensions
    HF: heart failure
    ICER: incremental cost-effectiveness ratio
    ICT: information and communication technology
    MAFEIP: Monitoring and Assessment Framework for the European Innovation Partnership on Active and Healthy Ageing
    QALY: quality-adjusted life-year
    RCT: randomized controlled trial
    SBP: systolic blood pressure
    TAU: treatment as usual
    WTP: willingness to pay

    Edited by G Eysenbach; submitted 09.12.19; peer-reviewed by C Vis, T Bergmo, M Alrige; comments to author 24.01.20; revised version received 20.03.20; accepted 03.06.20; published 28.07.20

    ©Jordi Piera-Jiménez, Marjolein Winters, Eva Broers, Damià Valero-Bover, Mirela Habibovic, Jos W M G Widdershoven, Frans Folkvord, Francisco Lupiáñez-Villanueva. Originally published in the Journal of Medical Internet Research (, 28.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.