Published on 26.09.19 in Vol 21, No 9 (2019): September
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/14956, first published Jun 06, 2019.
Digital Health Transformation of Integrated Care in Europe: Overarching Analysis of 17 Integrated Care Programs
Background: Digital health tools comprise a wide range of technologies to support health processes. The potential of these technologies to effectively support health care transformation is widely accepted. However, wide scale implementation is uneven among countries and regions. Identification of common factors facilitating and hampering the implementation process may be useful for future policy recommendations.
Objective: The aim of this study was to analyze the implementation of digital health tools to support health care and social care services, as well as to facilitate the longitudinal assessment of these services, in 17 selected integrated chronic care (ICC) programs from 8 European countries.
Methods: A program analysis based on thick descriptions—including document examinations and semistructured interviews with relevant stakeholders—of ICC programs in Austria, Croatia, Germany, Hungary, the Netherlands, Norway, Spain, and the United Kingdom was performed. A total of 233 stakeholders (ie, professionals, providers, patients, carers, and policymakers) were interviewed from November 2014 to September 2016. The overarching analysis focused on the use of digital health tools and program assessment strategies.
Results: Supporting digital health tools are implemented in all countries, but different levels of maturity were observed among the programs. Only few ICC programs have well-established strategies for a comprehensive longitudinal assessment. There is a strong relationship between maturity of digital health and proper evaluation strategies of integrated care.
Conclusions: Notwithstanding the heterogeneity of the results across countries, most programs aim to evolve toward a digital transformation of integrated care, including implementation of comprehensive assessment strategies. It is widely accepted that the evolution of digital health tools alongside clear policies toward their adoption will facilitate regional uptake and scale-up of services with embedded digital health tools.
J Med Internet Res 2019;21(9):e14956
Digital health (eHealth) tools have been proposed to improve access to health care services, enhance care co-ordination and integration, enable self-management, support decision-making, enable monitoring, perform risk analysis, and facilitate proactive interventions . It is within this context that the European Commission has defined eHealth as follows:
The use of Information and Communication Technologies in health products, services and processes, combined with organizational change in health care systems and new skills, in order to improve health of citizens, efficiency and productivity in health care delivery, and the economic and social value of health 
The implementation of digital health tools has constituted an area of major research and innovation in the past years [, - ]. For example, the impact of electronic health records on health care quality has been explored by Campanella et al who showed improvement in health care quality in terms of guideline adherence and time efficiency while reducing medication errors [ ]. In parallel, most countries have recognized the value of patient portals [ - ]. The use of health monitoring devices in the general population is also increasing, with a broad spectrum of sophistication, including examples like integration between artificial intelligence and monitoring in a single device (ie, cardiac rhythm analysis) [ ]. Moreover, the use of open-source algorithms for subject-specific as well as population-based risk prediction has been reported [ , ]. Some of these technologies are already in use for the management of chronic multimorbid patients [ - ].
It is currently accepted that eHealth tools can be particularly useful to support Integrated Chronic Care (ICC) programs for patients with multimorbidity, that is, co-occurrence of 2 or more chronic disorders within 1 individual . Insights from the ICARE4EU project [ ] concluded that eHealth improves care integration and management processes, but the project identified that inadequate funding mechanisms, poor interoperability, and inadequate technological support represent major barriers for adoption of technologically supported ICC. In fact, it is acknowledged that the takeoff of digital health tools to support ICC is progressing rather slowly. Also, regulatory aspects are still a concern [ - ] to achieve a proper balance between preservation of individual privacy and the need for health data sharing [ ], as well as the increasing demand for health data analytics.
The Conceptual Framework
The Sustainable intEgrated care modeLs for multimorbidity: delivery, Financing, and performancE (SELFIE) Horizon 2020 project  aims to produce evidence and applicable policy advice on ICC programs for people with multimorbidity. Within the project aims, the SELFIE conceptual framework [ ] was developed. It comprises 6 core components of integrated care systems adapted from the World Health Organization, namely: (1) Service delivery, (2) Leadership and governance, (3) Workforce, (4) Financing, (5) Technologies and medical products, and (6) Information and research.
This paper focuses on 2 elements out of the 6 components of the SELFIE conceptual framework , which refer to the enabling role of digital health tools (Technologies and medical products) and assessment of ICC programs (Information and research). For each of these 2 components, the 3 levels of the SELFIE conceptual framework (micro, meso, and macro) were taken into account. The micro level is where the individual with multimorbidity interacts with care professionals and informal caregivers. The meso level relates to the organizational level and the institutional setup of providers. Finally, the macro level includes legislations, governance, policies, and system-wide changes at the national and international level.
For Technologies and medical products, the SELFIE framework stresses the need for digital health tools to be widely available and user-friendly to provide robust support to the care processes. At a micro level, the use of technology (eg, electronic medical records [EMR) and patient portals) can be a facilitator of collaborative care if tailored to the needs of the patient with multimorbidity. At a meso level, a shared information system (eg, EMR including shared care plans) among multiple providers and care settings can greatly facilitate communication, person-centeredness, personalized care, and care co-ordination. Finally, at a macro level, nationwide and international policies that foster technological development and innovation most likely would benefit from both implementation and continuous assessment of ICC programs for multimorbidity.
For Information and research, the project stresses the successful use of collected data from digital health tools for a 3-fold objective, namely: (1) Population health management; (2) Enhanced subject-specific health care delivery; and (3) Comprehensive assessment of ICC programs. At a micro level, currently collected individual-level data (eg, patient journey record) can effectively be used in the care process for individual risk prediction. At a meso level, shared information systems may further be used for service selection both at individual and group level (eg, triage systems and clinical predictive modelling) to strengthen the evidence base of complex integrated care interventions, as well as to develop indicators particularly relevant for the care of patients with multimorbidity. Alongside data ownership at a meso level, privacy and data protection legislation is an important consideration at macro level.
It is well accepted that the existence of an important gap between the way in which the role of digital tools is understood and the effective uptake of digital transformation by the different stakeholders in the health care systems at European level. To enable the real implementation and scale-up of the digital health tools with all their potential, we undertook this overarching analysis. Our study aims to synthesize the experiences, views, and opinions (including barriers and enabling factors) of the stakeholders and their impact on the care process, as well as the role and desirable future developments of digital health tools, to foster transformation of health care systems toward sustainability by enhancing management of patients with multimorbidity. A second aim is to characterize the different programs with respect to maturity of their supporting digital health tools and the level of assessment of the ICC program.
Here, we present the results of an overarching analysis of the Thick Descriptions  of 17 promising ICC programs selected by the SELFIE project across 8 European countries: Austria, Croatia, Germany, Hungary, the Netherlands, Norway, Spain, and the United Kingdom. Within the overarching analysis, we focus on the different aspects of implementation of digital health tools supporting services and facilitating assessment strategies. This will lead to future directions defining how digital support can contribute to scale-up and evaluation of integrated care services. These services should focus on patient-centered health care provision with dynamic evaluation of technology-enabled integrated care programs, without compromising patient privacy.
To select the 17 ICC programs, each country participating in the SELFIE project  applied a search strategy using the findings from an international scoping review, national publications on previous and on-going programs and projects, and consultation with national experts and networks. Details on the process of selection of the programs, as well as the list of the 17 selected programs per country, are reported in . The 17 programs were grouped into 4 categories: (1) population health management programs (n=5); (2) frail elderly programs (n=6); (3) programs for individuals at the end-of-life and oncology patients (n=3); and (4) programs for vulnerable individuals who face problems in multiple life domains, like health, housing, and financial problems (n=3).
Procedure and Data Collection
The Thick Description, a qualitative empirical research method, was used in SELFIE to gain a deep understanding of the ICC programs from the different stakeholders’ point of view [, ]. The method undertaken included two different approaches: (1) Study of a variety of documents about each of the 17 ICC programs (ie, official and contractual documents, documents related to past evaluations, and factsheets from each ICC); and (2) interviews conducted with all relevant stakeholders (ie, program managers, initiators, representative of sponsor or payer organizations, health care professionals, informal caregivers, and patients or patient representatives). As described earlier, we concentrate on the two information technology-related dimensions in this paper.
Each partner-country interviewers underwent specific training on how to conduct and analyze the semistructured interviews to ensure uniform procedures. A total of 233 stakeholders were interviewed from November 2014 to September 2016 (seefor more detailed information on stakeholder composition per country). The interviews were audio recorded and transcribed verbatim from the audio file by either the interviewer or an independent research transcriber. The resulting transcripts were analyzed by members of the local SELFIE teams using Mayring qualitative content analysis [ ]. The quotations which were used in the thick descriptions were edited into readable forms and translated into English. The transcripts were not returned to participants for correction.
When writing this manuscript, we adhered to the COnsolidated criteria for REporting Qualitative research . All information retrieved from the document analysis (including the stakeholders´ interviews) was processed according to the country-specific ethics statement listed under the subheading: Ethics Approval and Consent to Participate. The Thick Descriptions of the 17 ICC programs studied can be found on the SELFIE website [ ].
The first author did a thematic analysis  on the Thick Descriptions of the 2 components of the SELFIE conceptual framework referring to the enabling role of digital health tools (Technologies and medical products) and assessment strategies of the ICC programs (Information and research). He then discussed findings with all other coauthors. For each of these 2 components, this secondary analysis of the Thick Descriptions considered the 3 levels of the SELFIE conceptual framework: micro, meso, and macro. As detailed in , a 3-level grading system (+ to +++) was developed and used, under the criteria of the coauthors (JR and IC), to score maturity of the 17 ICC programs for each of the 2 components assessed in this research. Finally, the maturity of each of the 2 components of the SELFIE conceptual framework is summarized as the average of the maturity at micro, meso, and macro level.
|Technologies and medical products|
|Digital health tools (micro)|
|Electronic medical records—EMR||+|
|Personal health records at program level||++|
|Personal health records at regional level||+++|
|Organizational interoperability (meso)|
|Health information exchange||+|
|Shared case management systemsb||+++|
|Digital transformation policies (macro)|
|Only addressing EMR||+|
|Several initiatives at program level||++|
|National and regional strategic plans||+++|
|Information and research|
|Evaluation strategies (micro)|
|Full assessment with published papers||+++|
|Risk assessment (meso)|
|Clinical knowledge or rule-based||+|
|Clinical predictive modelling tools||++|
|Multilevel predictive modelling toolsc||+++|
|Research and innovation policies (macro)|
|Strong co-ordination with EUd programs||+++|
aShared electronic medical records among health care providers.
bShared case management systems among health care providers to support integrated care pathways.
cPredictive modelling tools that combine information from various data sources, for example, clinical, population-based, biological, and patient-reported.
dEU: European Union.
Ethics Approval and Consent to Participate
Letters of Medical Ethics Approval of study protocols, questionnaires, and informed consent forms were sent and approved by the European Commission as a Deliverable of the SELFIE project.
Austria: Letter from Institute for Advanced Studies (IHS) declaring that ethical approval is not necessary for the evaluation of the two Austrian Integrated Care programs, October 3, 2017.
Croatia: Statement from the Agency for Quality and Accreditation in Health Care and Social Welfare declaring that the two evaluation studies are not within the scope of work of Croatian Central Ethics Committee, August 28, 2017, with reference to Official Gazette No 121/07 and No 25/15.
Germany, Gesundes Kinzigtal: Letter from the Ethical Committee, Technische Universität Berlin, declaring that the research is ethically acceptable (Ref: ST_02_20170620, August 15, 2017).
Germany, Casaplus: Letter from the Ethical Committee, Technische Universität Berlin, declaring that the research is ethically acceptable (Ref: ST_01_20170428, August 4, 2017).
Hungary, OnkoNetwork: The research plan has been authorized under approval No IG/03092–000/2016 by the Director General of Moritz Kaposi General Hospital, based on the positive opinion of the Institutional Research Committee and the responsible person of the Hospital for data protection. The Institutional Ethics Committee of the hospital double-checked the research and publication plan and confirmed that no ethical concerns were emerging related to this research and to the publication of findings (October 10, 2018, Ref: IKEB_IG_04125-000_2018).
Hungary, Palliative Care Consult Service: Letter from the Medical Research Council (Tudomanyos es Kutatasetikai Bizottsag, ETT TUKEB) declaring that the research is granted with Professional-Ethical Approval (Ref: 18632–4/2017/EKU, 24–4-2017).
The Netherlands, Proactive Primary Care Approach for Frail Elderly (U-PROFIT): Letter from the Medical Ethical Committee (MEC) Erasmus Medical Center Rotterdam declaring that the research is exempt from the Medical Research Involving Human Subjects Act (Dutch acronym: WMO; Ref: MEC-2017-402, July 25, 2017).
The Netherlands, Care Chain Frail Elderly (CCFE): Letter from the Medical Ethical Committee (MEC) Erasmus Medical Center Rotterdam declaring that the research is exempt from the Medical Research Involving Human Subjects Act (Dutch acronym: WMO; Ref: MEC-2014.558, December 18, 2014).
The Netherlands, Better Together in Amsterdam North (BSiN): Letter from the Medical Ethical Committee (MEC) of the Free University Medical Centre declaring that the research is exempt from the Medical Research Involving Human Subjects Act (Dutch acronym: WMO; Ref: MEC-2017-121, March 10, 2017).
Norway, Learning Network for Whole, Co-ordinated and Safe Pathways: Letter from the Regional Committees for Medical and Health Research Ethics-West (Komité for medisinsk og helsefaglig forskningsetikk -REK vest) declaring that the research is ethically approved (Ref: 2017/632/REK vest, March 28, 2017).
Norway, Medically Assisted Rehabilitation Bergen: Letter from The Regional Committees for Medical and Health Research Ethics-West (Komité formedisinsk og helsefaglig forskningsetikk -REK vest) declaring that the research is ethically approved (2017/944/REK vest, June 21, 2017).
Spain, Barcelona-Esquerra (AISBE): Letter from Clinic Research Ethical Committee (Comitè Ètic d’Investigació Clinica—CEIC) of the Clinic Hospital of Barcelona (Ref: CIF-G-08431173, Reg. HCB 2017/0451, June 14, 2017).
Spain, Badalona Serveis Assistencials (BSA): Letter from Clinic Research Ethical Committee (Comitè Ètic d’Investigació Clinica—CEIC) of the Clinic Hospital of Barcelona (Ref: CIF-G-08431173, Reg. HCB 2017/0453, June 14, 2017).
United Kingdom: Approval was granted by the National Health Service (NHS) Health Research Authority Research Ethics Committee (SELFIE REF: 16/WM/0295; CLASSIC REF: 14/NW/0206, June 23, 2016).
All participants provided written informed consent before participation.
provides a high-level description of results for the 2 components of the SELFIE conceptual framework considered in this study. In this table, main features of each ICC program (third and fourth columns) are provided per type of ICC program (first column) and country (second column). Extended results for the 17 ICC programs are reported in the text below. As a summary of the results, displays an average of the 3-level maturity grading criteria stated in for each of the 17 ICC programs, according to the micro, meso, and macro levels of the SELFIE conceptual framework.
Technologies and Medical Products
The overarching analysis provided the following valuable insights on the implementation and exploitation of digital health tools to support ICC.
Digital Health Tools (Micro)
All 17 ICC programs have at least partial implementation of EMR and they are planning to enhance implementation of EMR in the future.
However, specific personal health records to enhance patient engagement are not considered in programs like Health Network Tennengau (HNT), Casaplus, OnkoNetwork (ON), Palliative Care Consulting Service (PCSS), Better together in Amsterdam North (BSiN), and Medically Assisted Rehabilitation (MAR). In such programs, digital information exchange between care provider and patient are either not considered or telephone is still the dominant tool for communications. Nevertheless, the use of personal health records has been key to support various telemonitoring services for patient self-management in programs like in GeroS (eKarton). Likewise, South Somerset Symphony Program (SSSP) and Salford Integrated Care Program (SICP), both from the United Kingdom, stress the role of digital health tools (ie, Patients Know Best) to support telemonitoring, albeit suffering from some implementation problems:
Tele-dermatology and we’re piloting it [...] the GP will take a photograph and email it and get a decision, they’re not doing suspected cancers obviously, but rashes. Yeah, we’ve done it
We've also got telehealth, that support. So we've got patients who are on telehealth in their homes, and each morning, the intensivists review the telehealth and see if there's any flags, like, if somebody is on [...] I'm trying to think. If somebody is on some sort of medication that they need to, you know, where fluid balance is an issue, if they've lost six pounds in weight that might flag some medication change. So they get them to weigh themselves, do their blood pressure, and so on. So, telehealth has been hugely supportive, actually, at keeping patients at home
|Country||Program||Technologies and medical products||Information and research|
aHNT: Health Network Tennengau.
bSMC: Sociomedical Centre Liebenau.
cGK: Gesundes Kinzigtal.
dAISBE: Area Integral de Salut Barcelona-Esquerra.
eBSA: Badalona Serveis Assistencials.
fPCS: Palliative Care System.
hPCCS: Palliative Care Consulting Service.
iBSiN: Better together in Amsterdam North.
jCCFE: Care Chain Frail Elderly.
kU-PROFIT: Proactive Primary Care Approach for Frail Elderly.
lLN: learning network.
mMAR: Medically Assisted Rehabilitation.
nSICP: Salford Integrated Care Program.
oSSSP: South Somerset Symphony Program.
It is of note that the availability of personal health records at regional level, which is the case with the programs Area Integral de Salut Barcelona-Esquerra (AISBE) and Badalona Serveis Assistencials (BSA) (La Meva Salut) , generates additional potential to foster collaborative work at micro level.
Organizational Interoperability (Meso)
Most of the programs use secure networks for health information exchange between hospitals and general practitioners, but with a broad spectrum of maturity. For example, the Casaplus program implemented a specific Web-based platform to support regular communication between case managers and nursing professionals only, but not primary care and the hospital. On the other hand, the health information exchange network used in HNT function only 1 way (Hospital to community). A potentially more mature example can be seen in SICP, which has implemented a single patient record accessible to the professionals of the case management multidisciplinary team and the emergency medicine professionals. Their ultimate goal is for the platform to be accessible by primary, secondary, and community care organizations in the Salford area. In a minority of the programs (eg, ON and PCSS), data transfer across various IT platforms of providers are manually performed by program administrators. All in all, most ICC programs indicate the determinant positive role of the existing regional digital health tools for health information exchange across health care tiers, which facilitates information sharing among heterogeneous providers, as seen in this example from the Proactive Primary Care Approach for Frail Elderly (U-PROFIT) program in the Netherlands:
We are working with a vulnerable population, frail in general, and it is important for them to avoid going from one place to another and visiting different service providers and collecting different forms [...] or duplicate papers because you have to present this paper here and this same paper over there [...] I think this is an important progress for the population
A step forward in terms of organizational interoperability, the computerization of health and social care records via a shared EMR among health care tiers, is at the heart of some ICC programs, such as the GeroS program. In line with organizational interoperability, the Care Chain Frail Elderly (CCFE) program focuses on structuring care and stimulating communication between all chain partners in primary care at various access levels with one another, thanks to an additional digital health tool (Care2U) that is used on top of the existing information systems to access the individual care plan and exchange information. However, although there has been much effort by the governments of these countries to have a shared EMR in place, this has not yet been fully successful, mostly due to data privacy issues. Last but not least, the AISBE program aims to consolidate a shared case management system  on top of the existing regional shared EMR, aiming to support the regional deployment of adaptive case management processes.
Digital Transformation Policies (Macro)
As all 17 ICC programs have at least partial implementation of EMR, all national and regional policies aim to expand the implementation of EMR in the future. However, in most program countries, the use and scope of digital health tools depends on several initiatives at program level, which serve as pilot sites for the nation and region wide rollout of digital health tools. This is the case, for example, in the Austrian programs (HNT and Sociomedical Centre Liebenau [SMC]), which are part of the electronic health files, the most comprehensive eHealth initiative in Austria. Our research has only been able to identify national and regional strategic plans for deployment of eHealth in Croatia, Spain, Hungary, the Netherlands, Norway, and the United Kingdom. The largest digital transformation policy being the Whole System Demonstrator pilots in the United Kingdom, which is a strategy proposed by the Department of Health in England to focus on health and social care for people with long-term needs, emphasizing the use of advanced assistive technologies including telehealth and telecare. It has demonstrated a slight reduction in mortality and emergency admission rates but was not demonstrated to be more cost effective than usual care [, ].
Under the auspices of the Norwegian Directorate of Health, a Care Journal has been recently established for all citizens (voluntary); this is an electronic tool comprising selected and important health data that are accessible for the citizen and health personnel for the whole health care sector in Norway (including the 2 programs analyzed in this paper). Another example is the Catalan Health Plan , which prioritizes the improvement and transformation of the health system and health care organization through the intensive introduction of emerging digital health technologies.
Information and Research
As summarized in the, the overarching analysis provided the following valuable insights on the assessment strategies of the 17 technology-enabled ICC programs.
Evaluation Strategies (Micro)
This research has shown that in some ICC programs, no comprehensive evaluation has been carried out so far (SMC, Palliative Care System, ON, and Learning network), but is planned to be performed.
However, most ICC programs have been subject to partial monitoring and/or preliminary evaluation (ie, HNT, GeroS, Palliative Care Consulting Service, CCFE, BSiN, MAR, SICP, and SSSP), involving mainly descriptive data analysis over well-defined outcome measures of interest or key performance indicators. Specifically, the SSSP program includes:
Number of bed days, average length of stay, 30 day readmission, avoidable emergency admissions, precautionary emergency admissions, patients admitted multiple times, excess bed days, avoidable A&E attendances, confidence to my own health, received enough support to help self-managed long-term conditions, have a written care plan, care plan regularly reviewed, patient access to GP and nurse, online services, GP referrals, mental wellbeing, the Warwick-Edinburgh Mental Wellbeing scale, patient activation measure [PAM], patient satisfaction experience, and number of contacts made.
Only some programs report full scientific assessment (ie, Casaplus , Gesundes Kinzigtal [ , ], U-PROFIT [ , ], and AISBE [ - ]). These programs have been evaluated using randomized controlled trials as well as pre-post evaluation with propensity score matching methods, following the Triple Aim outcomes [ , ]:
...was the number of hospital admissions reduced? How did they experience the effects of care (the insured person, the environment, the relatives)? Were the per capita costs reduced?
Risk Assessment Strategies (Meso)
Patient management purely based on clinical criteria (professional training, knowledge, instinct, and experience) or combined with rules-based clinical management  (thresholds for certain parameters defining pre-established decision criteria) constitutes current health professional practice in most ICC programs.
In contrast, the regular use of subject-specific predictive modelling tools for clinical decision support (predictive modelling establishing relationships between sets of variables and outcomes generated using statistical or machine learning tools) is still in its infancy, despite the fact that it seems a natural step toward customization of care to patient’s needs. Clinical predictive modelling tools are only reported to be used in some ICC programs (ie, SICP, SSSP, U-PROFIT, Gesundes Kinzigtal, and Casaplus) for individual risk assessment (which can be considered within the micro level). Within the U-PROFIT program, available data in the general practitioner EMR system are used by the U-PRIM software to screen frail patients of 60 years and older in every participating practice . SICP and SSSP programs in the United Kingdom use a well-known patient-level risk predictive tool, PARS [ ] and the Combined Predictive Model [ ], to identify those patients that require the most care and support and to assess the risk of patients having unplanned hospital admissions within a 12-month period. It is used to some extent, but more trust is placed in clinical judgement in many cases:
...so we’d looked at some of the higher risk patients that were identified by the Combined Predictive Model and PARS (Patient at Risk Score) exactly the same, because I’ve done that before for the unscheduled care, and we looked at that; and what you find is the high risk people that are identified by this risk stratification models that are promoted nationally, is that the only data that’s easy to count is the hospital data, is the Hospital Episode Statistics of your hospital episodic statistics and stuff.
Similarly, Casaplus uses a clinical predictive modelling tool to identify patients in high risk for hospital admissions within the next 12 months.
The use of clinical predictive modelling tools for population-based risk assessment is only reported in BSA and AISBE (Catalonia, ES). Since 2011, the Catalan Health Surveillance system collects detailed information on health care usage for the entire population of Catalonia , the region in which AISBE and BSA operate. It includes information on hospitalization, primary care visits, emergency department visits, skilled nursing facilities, palliative care and the mental health services, information on pharmacy prescription and expenditure, and a registry on the billing record also encompassing outpatient visits to specialists, home hospitalization, medical transportation (urgent and nonurgent), ambulatory rehabilitation, respiratory therapies, and dialysis. This information is used for provider payment purposes. Also, external audits are performed periodically to ensure the quality and reliability of the data. The Catalan Health Surveillance System is used to update, on a 6-month basis, the regional population-based health risk assessment tool, (the Adjusted Morbidity Groups) that generates the health risk strata pyramid of the general population of Catalonia [ , ].
Furthermore, AISBE is adopting a holistic approach that fosters inclusion of covariates from multilevel data sources, namely Multilevel Predictive Modelling: (1) clinical, (2) informal care; (3) biological research; and (4) outcomes from population-health risk predictive modelling (eg, the Adjusted Morbidity Groups), resulting in enhanced patient-based stratification and optimization of service selection. This approach aims to pave the way toward personalized medicine, provided that access to the multilevel data sources is granted. However, most legal frameworks on data privacy of the 17 ICC programs depend on the ongoing implementation of the European Union Data Protection Directive 95/46/EC to make the concept of multilevel predictive modelling operational.
Research and Innovation Policies (Macro)
A majority of the 17 ICC programs are part of incipient research and innovation initiatives constantly being implemented in practice, both bottom-up and top-down, using several, sometimes consecutive, project-budgets but without sustainable structural funding.
However, Croatian, Dutch, German, Norway, and Spanish programs are aligned with consolidated research and innovation programs at state and regional level. For example, the Gesundes Kinzigtal program in Germany has been extensively evaluated in terms of prevalence of multimorbidity, polypharmacy, proportion of generic drugs, prevalence of problematic drug prescriptions, prevalence of fractures among patients diagnosed with osteoporosis, quality of services, and overall health care costs . Another example is the Research Council of Norway, commissioned by the Ministry of Health and Care Services to carry out a research-based evaluation of the Co-ordination Reform. The Research Council has conducted a research program tailored at integrated care from 2012 to 2015.
Furthermore, the strong co-ordination of most programs from Norway, the Netherlands, Hungary, Spain, Croatia, and Germany with different European research and innovation initiatives under the umbrella of H2020 , EIP-AHA [ ], EIT Health [ ], and/or RIS3 [ ], as well as other specific research and innovation actions, should contribute to cross-fertilization among health care, research, and innovation.
The overarching analysis allowed us to assess the use of digital health tools to support the care process in the 17 ICC programs on the 2 specific aspects analyzed: Technologies & Medical products and Information & Research. As most of the ICC programs are pilot experiences in terms of nation and region wide rollout of digital health tools, this analysis was useful to learn from them regarding requirements for a successful large-scale implementation elsewhere.
Acknowledging that the 17 ICC programs are highly heterogeneous regarding the use and impact of digital health tools, the main findings are summarized below.
Electronic Medical Records
Each program studied shows at least partial implementation of EMR, and all of them have plans in place for a future mature implementation of EMR.
Personal Health Records
The use of personal health records to support telemonitoring services for patient self-management is not in place in most ICC programs, for which telephone is still the dominant means of communication.
Health Information Exchange Platforms
Most programs reported on the potential of secure health information exchange across providers to facilitate organizational interoperability for deployment of ICC by facilitating information sharing among heterogeneous providers and avoid generating additional burden of double-registration to health professionals. However, the maturity of implementation is currently rather poor. Moreover, the need for technological tools, on top of health information exchange platforms, supporting collaborative work across health care tiers to foster implementation of shared case management [, ] was stressed by programs like AISBE.
Digital Transformation Policies
The overarching analysis highlighted the lack of well-defined macro-level policies, with effective operational implementation plans, in the health care systems in which most ICC programs operate. Often, the use and scope of digital health tools depends on local or regional initiatives of individual providers involved in the ICC programs.
Health Data Analytics and Evaluation Strategies
Most programs systematically collect well-defined outcome measures to feed program-specific evaluation strategies, ranging from descriptive data analysis, comparison of trial and control groups, as well as pre- and postmeasuring. Still, some ICC programs recognize barriers for assessment such as a lack of financing, poor research capacity, concerns on data security, and misuse of data. This study clearly shows the need for formulation of structured and comprehensive evaluation and monitoring strategies for ICC including formulation of key performance indicators extensively shared across countries. Moreover, the Quadruple Aim approach [, ] (ie, the Triple Aim approach plus the health care professionals experience) should serve to standardize the evaluation across European Union sites.
Health Risk Assessment
Just a few of the ICC programs report on the use of clinical predictive modelling tools, and even less ICC programs claim the use of population-based health-risk prediction tools.
Research and Innovation Policies
The majority of the 17 ICC programs (either bottom-up or top-down initiatives) are often based on project-specific budgets without well-defined, operational policies and, consequently, without sustainable structural funding. Implementing the above technological innovations frequently requires hardware and software upgrades. The costs of initial rollout and training of staff also need to be considered and weighed against the likely benefits.
We acknowledge some limitation in our study such as the inherent limitations of the methodological approach adopted. Also, as the study conclusions relate only to programs based in Europe, worldwide representativeness of the study results cannot be assumed.
Comparison With Previous Work and Future Directions
A recent report by the European Union on Integrated Care maturity , including the evaluation of Information and eHealth tools, concluded that the level of maturity in Germany, Denmark, Belgium, Italy, Spain, Greece, Sweden, and Iceland scored higher in comparison to their peers in Estonia, the Netherlands, Poland, and Bulgaria. The most mature countries are Denmark, followed by Spain, Germany, and Iceland at the same level. Finally, Sweden, Belgium, Italy, and Greece were in intermediate level. This is line with our findings, except for the 3 case studies in the Netherlands, which in our study were more mature than Dates et al [ ] suggests. It is of note that differences between the study by Dates et al [ ] and this study might be explained by using different tools as well as the selection of 3 promising cases in the Netherlands which actually use eHealth tools. The European Union on Integrated Care maturity report applies an interactive tool developed by SIROCCO (Scaling Integrated Care in Context) to assess the maturity of ICC programs on a global level using different aspects (one of them being eHealth); whereas in our study, we explicitly focus on different aspects of digital health tools and assessment strategies solely. Also, the previous report is based on a review of the literature on integrated care policies and strategies, whereas our study adds the point of view from the different stakeholders directly involved in generation and implementation of these policies and strategies.
Integrated care programs for chronic patients involve complex interventions for heterogeneous populations; therefore, proper articulation of digital health tools and the different components of the evaluation process are still unmet needs that markedly hinder comparability and scale-up. The overarching analysis of the 17 ICC programs conducted in this study allowed us to identify the following potential areas for future developments:
Refinement of assessment methodologies of large-scale deployment and adoption of ICC programs, likely based on implementation research approaches [- ], are needed. We understand that assessment should adopt the classical three-dimensional approach including outcomes, processes, and structures [ ]. Moreover, usual health outcome variables (ie, mortality, hospital readmissions, etc.) should be ideally expanded [ ] considering the Quadruple Aim approach [ , ]. The approach requires the collection of patient-reported outcomes and experience data (PROMS and PREMS) on a regular basis.
The concept of adaptive case management explored in AISBE [, ] should be made operational. Conventional health information systems rely on the management of clinical episodes with a disease-oriented approach and only very rarely incorporate the required process logics to support continuity of care with a patient-centered approach.
Dynamic health-risk assessment taking into consideration both service commissioning (population-based health-risk predictive modelling) and subject-specific service selection involving optimal patient allocation in the health system (individual health-risk predictive modelling supporting decision support) should be addressed to improve outcomes [- ]. Ultimately, the application of holistic strategies for subject-specific risk prediction and stratification that incorporates multilevel determinants of health (eg, socioeconomic, lifestyle, behavioral, clinical, physiological, cellular, and omics information) emerges as a high priority goal to properly pave the way toward personalized medicine for complex chronic patients [ ]. Enhanced clinical predictive modelling, personalized diagnostic and treatment tools can contribute to the acceleration of transfer of scientific evidence to practice.
Development of pragmatic trials that incorporate real-life evidence from multilevel determinants of health may require implementation strategies, ideally using cloud computing environments, tackling privacy and regulatory constraints [, ]. Currently, the articulation of the main technical building blocks, that is, multilevel biomedical data integration, tools for clinical predictive modelling in the cloud and High-Performance Computing, as one integrated system is yet a largely unmet potential.
This overarching analysis informs the current implementation status of digital health tools for management of multimorbidity in the 17 promising ICC programs selected in SELFIE. Notwithstanding the heterogeneity of the results, most studied programs are progressively evolving their supporting digital health tools from pilot prototypes to full scale-up at regional and national level. However, the majority of programs have not yet undergone full evaluation and assessment strategies. Future directions which can enable of digital transformation are based on innovation at micro and meso level with full support from the macro level. Some strategic areas that can help toward this end are the following: (1) implementation of research strategies; (2) explore an adaptive case management approach; (3) further developments of health risk assessment; and (4) holistic implementation strategies using future, regulatory compliant, cloud computing environments.
The authors gratefully acknowledge the contribution of all other members of the SELFIE consortium and the linked third parties. Membership of the SELFIE consortium can be found on www.selfie2020.eu. The SELFIE project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement #634288. The content of this paper reflects only the SELFIE group’s views and the European Commission is not liable for any use that may be made of the information contained herein.
Conflicts of Interest
Essential and additional criteria for preliminary selection of Sustainable intEgrated care modeLs for multimorbidity: delivery, Financing, and performancE (SELFIE) programs.PDF File (Adobe PDF File), 223 KB
Stakeholders interviews per country.PDF File (Adobe PDF File), 241 KB
Summary of the overarching analysis of the 17 selected integrated chronic care programs.PDF File (Adobe PDF File), 184 KB
- Barbabella F, Melchiorre MG, Quattrini S, Papa R, Lamura G. In: Richardson E, van Ginneken E, editors. How Can eHealth Improve Care for People With Multimorbidity in Europe?. Copenhagen, Denmark: European Observatory on Health Systems and Policies; 2017.
- European Commission. 2012. eHealth Action Plan 2012-2020: Innovative Healthcare for the 21st century URL: https://ec.europa.eu/digital-single-market/en/news/ehealth-action-plan-2012-2020-innovative-healthcare-21st-century [accessed 2018-03-01]
- World Health Organization. 2019. eHealth Unit URL: https://www.who.int/ehealth/en/ [accessed 2019-02-04]
- Steinhubl SR, Topol EJ. Digital medicine, on its way to being just plain medicine. NPJ Digit Med 2018;1:20175 [FREE Full text] [CrossRef] [Medline]
- Lupiáñez-Villanueva F, Anastasiadou D, Codagnone C, Nuño-Solinís R, Garcia-Zapirain SM. Electronic health use in the European union and the effect of multimorbidity: cross-sectional survey. J Med Internet Res 2018 May 3;20(5):e165 [FREE Full text] [CrossRef] [Medline]
- Melchiorre MG, Papa R, Rijken M, van Ginneken E, Hujala A, Barbabella F. eHealth in integrated care programs for people with multimorbidity in Europe: insights from the ICARE4EU project. Health Policy 2018 Jan;122(1):53-63 [FREE Full text] [CrossRef] [Medline]
- Williams PA, Lovelock B, Cabarrus T, Harvey M. Improving digital hospital transformation: development of an outcomes-based infrastructure maturity assessment framework. JMIR Med Inform 2019 Jan 11;7(1):e12465 [FREE Full text] [CrossRef] [Medline]
- Campanella P, Lovato E, Marone C, Fallacara L, Mancuso A, Ricciardi W, et al. The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. Eur J Public Health 2016 Feb;26(1):60-64. [CrossRef] [Medline]
- Meskó B, Drobni Z, Bényei E, Gergely B, Győrffy Z. Digital health is a cultural transformation of traditional healthcare. Mhealth 2017;3:38 [FREE Full text] [CrossRef] [Medline]
- European Commission. 2014. Green Paper on Mobile Health ('mHealth') URL: https://ec.europa.eu/digital-agenda/en/news/green-paper-mobile-health-mhealth [accessed 2014-06-22] [WebCite Cache]
- Bhavnani SP, Narula J, Sengupta PP. Mobile technology and the digitization of healthcare. Eur Heart J 2016 May 7;37(18):1428-1438 [FREE Full text] [CrossRef] [Medline]
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019 Jan;25(1):44-56. [CrossRef] [Medline]
- Monterde D, Vela E, Clèries M, Grupo Colaborativo GMA. [Adjusted morbidity groups: a new multiple morbidity measurement of use in primary care]. Aten Primaria 2016 Dec;48(10):674-682 [FREE Full text] [CrossRef] [Medline]
- Dueñas-Espín I, Vela E, Pauws S, Bescos C, Cano I, Cleries M, et al. Proposals for enhanced health risk assessment and stratification in an integrated care scenario. BMJ Open 2016 Apr 15;6(4):e010301 [FREE Full text] [CrossRef] [Medline]
- Merck SF. Chronic disease and mobile technology: an innovative tool for clinicians. Nurs Forum 2017 Oct;52(4):298-305. [CrossRef] [Medline]
- Cho JH, Lee HC, Lim DJ, Kwon HS, Yoon KH. Mobile communication using a mobile phone with a glucometer for glucose control in type 2 patients with diabetes: as effective as an internet-based glucose monitoring system. J Telemed Telecare 2009;15(2):77-82. [CrossRef] [Medline]
- Logan AG, McIsaac WJ, Tisler A, Irvine MJ, Saunders A, Dunai A, et al. Mobile phone-based remote patient monitoring system for management of hypertension in diabetic patients. Am J Hypertens 2007 Sep;20(9):942-948. [CrossRef] [Medline]
- Whitehead L, Seaton P. The effectiveness of self-management mobile phone and tablet apps in long-term condition management: a systematic review. J Med Internet Res 2016 May 16;18(5):e97 [FREE Full text] [CrossRef] [Medline]
- Kim YJ, Rhee SY, Byun JK, Park SY, Hong SM, Chin SO, et al. A smartphone application significantly improved diabetes self-care activities with high user satisfaction. Diabetes Metab J 2015 Jun;39(3):207-217 [FREE Full text] [CrossRef] [Medline]
- Zulman DM, Jenchura EC, Cohen DM, Lewis ET, Houston TK, Asch SM. How can ehealth technology address challenges related to multimorbidity? Perspectives from patients with multiple chronic conditions. J Gen Intern Med 2015 Aug;30(8):1063-1070 [FREE Full text] [CrossRef] [Medline]
- National Institute for Health and Care Excellence. 2016. Multimorbidity: Clinical Assessment and Management URL: https://www.nice.org.uk/guidance/ng56/resources/multimorbidity-clinical-assessment-and-management-pdf-1837516654789 [accessed 2018-01-29]
- Vayena E, Haeusermann T, Adjekum A, Blasimme A. Digital health: meeting the ethical and policy challenges. Swiss Med Wkly 2018;148:w14571 [FREE Full text] [CrossRef] [Medline]
- Renner AT, Bobek J, Habl C, Anja J. European Commission. 2016. Study on Big Data in Public Health, Telemedicine and Healthcare URL: https://ec.europa.eu/digital-single-market/en/news/study-big-data-public-health-telemedicine-and-healthcare [accessed 2019-08-12]
- Adjekum A, Blasimme A, Vayena E. Elements of trust in digital health systems: scoping review. J Med Internet Res 2018 Dec 13;20(12):e11254 [FREE Full text] [CrossRef] [Medline]
- Schiza EC, Kyprianou TC, Petkov N, Schizas CN. Proposal for an ehealth based ecosystem serving national healthcare. IEEE J Biomed Health Inform 2019 May;23(3):1346-1357. [CrossRef] [Medline]
- SELFIE: Sustainable Integrated Care Models for Multi-Morbidity: Delivery, Financing and Performance. URL: http://www.selfie2020.eu/ [accessed 2018-04-02]
- Leijten LF, Struckmann V, van Ginneken E, Czypionka T, Kraus M, Reiss M, SELFIE Consortium. The SELFIE framework for integrated care for multi-morbidity: development and description. Health Policy 2018 Jan;122(1):12-22 [FREE Full text] [CrossRef] [Medline]
- Selfie 2020. 2018. Work Package 2: Executive summary URL: https://www.selfie2020.eu/wp-content/uploads/2016/12/SELFIE_WP2_Executive-summary.pdf [accessed 2017-12-03]
- Struckmann V, Leijten FR, van Ginneken E, Kraus M, Reiss M, Spranger A, SELFIE Consortium. Relevant models and elements of integrated care for multi-morbidity: results of a scoping review. Health Policy 2018 Jan;122(1):23-35 [FREE Full text] [CrossRef] [Medline]
- Pitter JG, Csanádi M, Szigeti A, Lukács G, Kovács A, Moizs M, et al. Planning, implementation and operation of a personalized patient management system for subjects with first suspect of cancer (OnkoNetwork): system description based on a qualitative study. BMC Health Serv Res 2019 Feb 21;19(1):131 [FREE Full text] [CrossRef] [Medline]
- Mayring P. Qualitative Inhaltsanalyse: Grundlagen und Techniken. Twelfth Edition. Weinheim, Germany: Beltz; 2015.
- Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357. [CrossRef] [Medline]
- Selfie 2020. 2018. SELFIE Case Studies URL: https://www.selfie2020.eu/publications/ [accessed 2018-03-01]
- Boyatzis RE. Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks, CA: Sage Publications; 1998.
- Modol JR. Navigating towards self-care: the Catalan public patient portal. In: Aanestad M, Grisot M, Hanseth O, Vassilakopoulou P, editors. Information Infrastructures within European Health Care. Cham, Germany: Springer; 2017.
- Cano I, Alonso A, Hernandez C, Burgos F, Barberan-Garcia A, Roldan J, et al. An adaptive case management system to support integrated care services: lessons learned from the NEXES project. J Biomed Inform 2015 Jun;55:11-22 [FREE Full text] [CrossRef] [Medline]
- Steventon A, Bardsley M, Billings J, Dixon J, Doll H, Hirani S, Whole System Demonstrator Evaluation Team. Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial. Br Med J 2012 Jun 21;344:e3874 [FREE Full text] [CrossRef] [Medline]
- Henderson C, Knapp M, Fernández JL, Beecham J, Hirani SP, Cartwright M, Whole System Demonstrator Evaluation Team. Cost effectiveness of telehealth for patients with long term conditions (Whole Systems Demonstrator telehealth questionnaire study): nested economic evaluation in a pragmatic, cluster randomised controlled trial. Br Med J 2013 Mar 20;346:f1035 [FREE Full text] [CrossRef] [Medline]
- Inici. Departament de Salut. 2016. Pla de Salut de Catalunya 2016-2020 URL: http://salutweb.gencat.cat/web/.content/_departament/pla-de-salut/Pla-de-salut-2016-2020/documents/Pla_salut_Catalunya_2016_2020.pdf [accessed 2017-11-03]
- DocPlayer. 2008. MedicalContact AG. 365 Tage Casaplus. Ein Präventionsprogramm feiert Geburtstag URL: https://docplayer.org/37867445-Medicalcontact-ag-365-tage-casaplus-ein-praeventionsprogramm-feiert-geburtstag.html [accessed 2018-11-04]
- Schulte T, Pimperl A. Pursuing the triple aim: a three-dimensional outcome evaluation of the integrated care project 'gesundes kinzigtal'? Int J Integr Care 2013 Oct 23;13(5):-. [CrossRef]
- Hildebrandt H, Hermann C, Knittel R, Richter-Reichhelm M, Siegel A, Witzenrath W. Gesundes kinzigtal integrated care: improving population health by a shared health gain approach and a shared savings contract. Int J Integr Care 2010;10(2):e046. [CrossRef] [Medline]
- Bleijenberg N, Drubbel I, Schuurmans MJ, Dam HT, Zuithoff NP, Numans ME, et al. Effectiveness of a proactive primary care program on preserving daily functioning of older people: a cluster randomized controlled trial. J Am Geriatr Soc 2016 Sep;64(9):1779-1788. [CrossRef] [Medline]
- Bleijenberg N, Drubbel I, Neslo RE, Schuurmans MJ, ten Dam VH, Numans ME, et al. Cost-effectiveness of a proactive primary care program for frail older people: a cluster-randomized controlled trial. J Am Med Dir Assoc 2017 Dec 1;18(12):1029-36.e3 [FREE Full text] [CrossRef] [Medline]
- Hernández C, Aibar J, Seijas N, Puig I, Alonso A, Garcia-Aymerich J, et al. Implementation of home hospitalization and early discharge as an integrated care service: a ten years pragmatic assessment. Int J Integr Care 2018 May 16;18(2):12 [FREE Full text] [CrossRef] [Medline]
- Hernández C, Alonso A, Garcia-Aymerich J, Grimsmo A, Vontetsianos T, Cuyàs FG, et al. Integrated care services: lessons learned from the deployment of the NEXES project. Int J Integr Care 2015;15:e006 [FREE Full text] [CrossRef] [Medline]
- Barberan-Garcia A, Vogiatzis I, Solberg HS, Vilaró J, Rodríguez DA, Garåsen HM, NEXES Consortium. Effects and barriers to deployment of telehealth wellness programs for chronic patients across 3 European countries. Respir Med 2014 Apr;108(4):628-637 [FREE Full text] [CrossRef] [Medline]
- South Carolina Hospital Association. 2012. A Guide to Measuring the Triple Aim: Population Health, Experience of Care, and Per Capita Cost URL: https://www.scha.org/files/ihiguidetomeasuringtripleaimwhitepaper2012.pdf [accessed 2017-11-05]
- Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood) 2008;27(3):759-769. [CrossRef] [Medline]
- Margolis CZ. Uses of clinical algorithms. J Am Med Assoc 1983 Feb 4;249(5):627-632. [CrossRef] [Medline]
- Drubbel I. Frailty Screening in Older Patients in Primary Care Using Routine Care Data. The Netherlands: Utrecht University; 2013.
- Billings J, Dixon J, Mijanovich T, Wennberg D. Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients. Br Med J 2006 Aug 12;333(7563):327 [FREE Full text] [CrossRef] [Medline]
- Wennberg D, Siegel M, Darin B, Billings J, Dixon J, Curry N. The King's Fund. 2006. Combined Predictive Model: Final Report URL: https://www.kingsfund.org.uk/sites/default/files/Combined_Predictive_Model_Final_Report.pdf [accessed 2018-12-04]
- Farré N, Vela E, Clèries M, Bustins M, Cainzos-Achirica M, Enjuanes C, et al. Medical resource use and expenditure in patients with chronic heart failure: a population-based analysis of 88 195 patients. Eur J Heart Fail 2016 Sep;18(9):1132-1140 [FREE Full text] [CrossRef] [Medline]
- European Commission. Horizon 2020 URL: https://ec.europa.eu/programmes/horizon2020/en [accessed 2017-06-01]
- European Commission. Innovation Union: Aims of the Innovation Union, State of Progress and Related Policy URL: http://ec.europa.eu/research/innovation-union/index_en.cfm?section=active-healthy-ageing [accessed 2017-06-19] [WebCite Cache]
- EIT Health. URL: https://www.eithealth.eu/ [accessed 2018-01-06]
- Smart Specialisation Platform - Europa EU. RIS3 Guide URL: http://s3platform.jrc.ec.europa.eu/s3-guide [accessed 2018-05-17]
- Herrmann C, Kurz M. Adaptive Case Management: Supporting Knowledge Intensive Processes with IT Systems. In: Proceedings of the International Conference on Subject-Oriented Business Process Management. 2011 Presented at: S-BPM ONE'11; September 29-30, 2011; Ingolstadt, Germany p. 80-97. [CrossRef]
- Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014;12(6):573-576 [FREE Full text] [CrossRef] [Medline]
- West CP. Physician well-being: expanding the triple aim. J Gen Intern Med 2016 May;31(5):458-459 [FREE Full text] [CrossRef] [Medline]
- Dates M, Lennox-Chhugani N, Sant H, Pereira A, Tedeschi M. European Commission. 2018. Health System Performance Assessment – Integrated Care Assessment (20157303 HSPA) URL: https://ec.europa.eu/health/sites/health/files/systems_performance_assessment/docs/2018_integratedcareassessment_en.pdf [accessed 2018-09-14]
- Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. Br Med J 2013 Nov 20;347:f6753. [CrossRef] [Medline]
- Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, StaRI Group. Standards for Reporting Implementation Studies (StaRI): explanation and elaboration document. BMJ Open 2017 Apr 3;7(4):e013318 [FREE Full text] [CrossRef] [Medline]
- Peters DH, Tran NT, Adam T. Implementation Research in Health: A Practical Guide. Geneva, Switzerland: World Health Organization; 2013.
- Donabedian A. The quality of care. How can it be assessed? J Am Med Assoc 1988;260(12):1743-1748. [CrossRef] [Medline]
- Sikka R, Morath JM, Leape L. The Quadruple Aim: care, health, cost and meaning in work. BMJ Qual Saf 2015 Oct;24(10):608-610. [CrossRef] [Medline]
- Hauder M, Pigat S, Matthes F. Research Challenges in Adaptive Case Management: A Literature Review. In: Proceedings of the 18th International Enterprise Distributed Object Computing Conference Workshops and Demonstrations. 2014 Presented at: EDOCW'14; September 1-2, 2014; Ulm, Germany. [CrossRef]
- Hu Z, Hao S, Jin B, Shin AY, Zhu C, Huang M, et al. Online prediction of health care utilization in the next six months based on electronic health record information: a cohort and validation study. J Med Internet Res 2015 Sep 22;17(9):e219 [FREE Full text] [CrossRef] [Medline]
- Cho I, Boo E, Chung E, Bates DW, Dykes P. Novel approach to inpatient fall risk prediction and its cross-site validation using time-variant data. J Med Internet Res 2019 Feb 19;21(2):e11505 [FREE Full text] [CrossRef] [Medline]
- Hao S, Jin B, Shin AY, Zhao Y, Zhu C, Li Z, et al. Risk prediction of emergency department revisit 30 days post discharge: a prospective study. PLoS One 2014;9(11):e112944 [FREE Full text] [CrossRef] [Medline]
- Cano I, Tenyi A, Vela E, Miralles F, Roca J. Perspectives on big data applications of health information. Curr Opin Syst Biol 2017 Jun;3:36-42. [CrossRef]
|AISBE: Area Integral de Salut Barcelona-Esquerra|
|BSA: Badalona Serveis Assistencials|
|BSiN: Better together in Amsterdam North|
|CCFE: Care Chain Frail Elderly|
|EMR: Electronic Medical Record|
|HNT: Health Network Tennengau (Gesundheitsnetzwerk Tennengau)|
|ICC: Integrated Chronic Care|
|MAR: Medically Assisted Rehabilitation|
|SELFIE: Sustainable intEgrated care modeLs for multimorbidity: delivery, Financing, and performancE|
|SICP: Salford Integrated Care Program|
|SMC: Sociomedical Centre Liebenau (Sozialmedizinisches Zentrum Liebenau)|
|SSSP: South Somerset Symphony Program|
|U-PROFIT: Proactive Primary Care Approach for Frail Elderly|
Edited by G Eysenbach; submitted 06.06.19; peer-reviewed by KC Wong, I Mircheva, K Goniewicz; comments to author 21.07.19; revised version received 30.07.19; accepted 30.07.19; published 26.09.19
©Erik Baltaxe, Thomas Czypionka, Markus Kraus, Miriam Reiss, Jan Erik Askildsen, Renata Grenković, Tord Skogedal Lindén, János György Pitter, Maureen Rutten-van Molken, Oscar Solans, Jonathan Stokes, Verena Struckmann, Josep Roca, Isaac Cano. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 26.09.2019
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.