This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Frail older people use emergency services extensively, and digital systems that monitor health remotely could be useful in reducing these visits by earlier detection of worsening health conditions.
We aimed to implement a system that produces alerts when the machine learning algorithm identifies a short-term risk for an emergency department (ED) visit and examine health interventions delivered after these alerts and users’ experience. This study highlights the feasibility of the general system and its performance in reducing ED visits. It also evaluates the accuracy of alerts’ prediction.
An uncontrolled multicenter trial was conducted in community-dwelling older adults receiving assistance from home aides (HAs). We implemented an eHealth system that produces an alert for a high risk of ED visits. After each home visit, the HAs completed a questionnaire on participants’ functional status, using a smartphone app, and the information was processed in real time by a previously developed machine learning algorithm that identifies patients at risk of an ED visit within 14 days. In case of risk, the eHealth system alerted a coordinating nurse who could then inform the family carer and the patient’s nurses or general practitioner. The primary outcomes were the rate of ED visits and the number of deaths after alert-triggered health interventions (ATHIs) and users’ experience with the eHealth system; the secondary outcome was the accuracy of the eHealth system in predicting ED visits.
We included 206 patients (mean age 85, SD 8 years; 161/206, 78% women) who received aid from 109 HAs, and the mean follow-up period was 10 months. The HAs monitored 2656 visits, which resulted in 405 alerts. Two ED visits were recorded following 131 alerts with an ATHI (2/131, 1.5%), whereas 36 ED visits were recorded following 274 alerts that did not result in an ATHI (36/274, 13.4%), corresponding to an odds ratio of 0.10 (95% IC 0.02-0.43;
The eHealth system has been successfully implemented, was appreciated by users, and produced relevant alerts. ATHIs were associated with a lower rate of ED visits, suggesting that the eHealth system might be effective in lowering the number of ED visits in this population.
clinicaltrials.gov NCT05221697; https://clinicaltrials.gov/ct2/show/NCT05221697.
The aging human population is increasing worldwide, and their health is characterized by high prevalence of chronic diseases and multimorbidity and a high vulnerability to acute diseases [
Patient (or family)-reported outcome measure (PROM) systems benefit patients with chronic diseases by improving quality of life, reducing mortality, reducing ED visits, and hospitalizations [
Today, attending physicians or nurses no longer have time and opportunity to regularly visit older people at home. HAs are key professionals in maintaining older adults at home. They have regular contact with them and can provide important information for decompensation prevention. The idea of this system is to optimize medical interventions when they are really necessary and to find alternatives via health recommendations or other interventions that do not require attending physicians when their presence is dispensable. It allows us to value HAs’ job, and it is based on their proximity with older adults to optimize the care pathway and avoid ED visits. Very few studies in the literature have analyzed the effectiveness of community-based interventions to prevent avoidable emergency hospitalizations of older individuals [
We have conducted a real-world pragmatic trial that included older adults receiving assistance from HAs. We aimed to implement a system that produces alerts when the machine learning algorithm identifies a short-term risk for an ED visit and examine health interventions required after these alerts and users’ experience. This study highlights the feasibility of the general system, the levers for compliance improvement, and its optimal effectiveness in reducing emergency hospitalizations among older people living at home.
This multicenter uncontrolled pragmatic trial (NCT05221697) was conducted with 3 home aid facilities participating in the study, located in 3 French cities: Marseille, Versailles, and Dinan. To be eligible, participants should be aged ≥75 years and living at home, receive the help of a social worker from these facilities, have seen their general practitioner within the last 12 months and had a mild or moderate level of dependency according to French national dependency tool, the AGGIR scale [
The intervention is summarized in
This questionnaire focused on functional and clinical autonomy (ie, activities of daily life), possible medical symptoms (eg, fatigue, falls, and pain), changes in behavior (eg, recognition and aggressiveness), and communication with the HA or their surroundings. This questionnaire is composed of very simple and easy-to-understand questions, giving a global view of the person’s condition. For each of the 23 questions, a yes/no answer was requested. Data recorded by HAs were sent in real time to a secure server to be analyzed by our machine learning algorithm, which predicted the risk level and displayed it on a web-based secure medical device called PRESAGE CARE, which is CE marked. A simplified diagram of the processing system and the description of data processing system can be found in
In the event of an alert, the coordinating nurse called the family caregiver to inquire about recent changes in the patient’s health condition and for doubt removal and could then decide to ask for a health intervention according to a health intervention model developed before the start of the study. In brief, this alert-triggered health intervention (ATHI) consisted of calling the patient’s nurse (if the patient had regular home visits of a nurse) or the patient’s general practitioner and informing them of a worsening of the patient’s functional status and a potential risk of an ED visit in the next few days according to the eHealth system algorithm. The ATHI was performed with the natural resources of the health system and not with the physicians or nurses employed in the study. No specific instruction or protocol was given to these health professionals, and they were free to make their own decisions. This model of ATHI had been presented and approved by the Agences Régionales de Santé of the regions involved in our study (
The application of the intervention protocol for alert management. ED: emergency department.
The primary outcomes were the rate of ED visits and the number of deaths recorded by the coordinating nurses, as well as the users’ experience with HAs and coordinating nurses recorded by the questionnaires. The eHealth system’s organizational outcomes in accordance with the guidelines for the evaluation of eHealth systems of the Haute Autorité de Santé, the French national health agency [
The secondary outcomes were to confirm the predictive capacity of the AI in real-world conditions. The diagnosis accuracy of the eHealth system alerts to predict ED visits was assessed by sensibility, specificity, positive and negative predictive values, and likelihood ratios. Accuracy analysis and reporting was conducted according to Standards for Reporting Diagnostics Accuracy Studies guidelines. Occurrences and dates of ED visits and hospitalizations were recorded by the HAs in each visit and by the home aid facility manager.
Continuous variables were described by means and SD, or medians and IQRs, if not normally distributed; categorical variables were described by relative frequencies. The 2-tailed
The research protocol was approved by the national French ethics committee for biomedical research, the Comité de Protection des Personnes, and the French Agency for the Safety of Health Products (2021-A02131-40–CPP 1-21-072 / 21.02093.000019).
Participants and the HAs and the home aid facilities’ managers were informed about the nature and purpose of the study and provided their written consent accordingly.
Among beneficiaries of the home aid facilities, 293 individuals were eligible, and 206/293 (71%) agreed to participate in the study and were included. Their mean age was 85 (SD 8) years, 161 of 206 (78%) participants were women, and 94 (45%) had a dependency level of GIR 3 or 4 (ie, moderate dependence level;
The mean follow-up period was 10 months with no patient loss during the trial. In total, 10 care managers of the home aid facilities (9 nurses and 1 pharmacist) and 109 HAs were involved in the study. From the 4753 home visits, 2656 (56%) were monitored by the app and provided inputs for the eHealth system (
Participants’ characteristics, activity, and the eHealth system; staff involved in their functioning; and alert-triggered health interventions.
Participants characteristics | Center 1 (n=67) | Center 2 (n=16) | Center 3 (n=123) | Total (N=206) | |||||
Age (years), mean (SD) | 86 (4) | 88 (6) | 86 (5) | 86 (5) | |||||
Gender (women), n (%) | 57 (86) | 13 (80) | 106 (86) | 176 (85) | |||||
Mild dependency (GIR 5 or 6), n (%) | 10 (15) | 3 (19) | 4 (3) | 17 (8) | |||||
Moderate dependency (GIR 3 or 4), n (%) | 23 (34) | 8 (50) | 63 (51) | 94 (46) | |||||
Severe dependency (GIR 1 or 2), n (%) | 0 (0) | 1 (6) | 13 (11) | 14 (7) | |||||
Unknown dependency level, n (%) | 34 (51) | 4 (25) | 43 (35) | 81 (39) | |||||
|
|||||||||
|
Home aides, n (%) | 46 (42) | 11 (10) | 52 (48) | 109 (100) | ||||
|
Care managers, n (%) | 6 (60) | 2 (20) | 2 (20) | 10 (100) | ||||
|
Visits monitored through the app, n (%) | 1130 (43) | 324 (12) | 1202 (45) | 2656 (100) | ||||
|
Compliance rate, % | 56.2 | 67·5 | 52.8 | 56.0 | ||||
|
Alerts, n (%) | 188 (46) | 47 (12) | 170 (42) | 405 (100) | ||||
|
Alert rate per monitored visits, % | 16.6 | 14.5 | 14.1 | 16.9 | ||||
|
|||||||||
|
Interventions, n (%) | 45 (34) | 46 (35) | 40 (31) | 131 (100) | ||||
|
Intervention rate per alerts, % | 23.9 | 97.9 | 23.5 | 32.3 |
The flowchart of alerts (index test), health interventions, and emergency department (ED) visits (reference standard).
Of 206 participants, 29 (14%) visited EDs during the study. Of these, 11 made 2 or more visits (up to 6 visits), and the total number of ED visits was 46; a total of 32 ED visits (for 19 people) were followed by hospitalization.
Hospitalization by direct admission without passing through an ED visit was recorded in 5 participants (
As a result of the 405 alerts generated by the eHealth system, 131 ATHI by health professionals were performed: 96 (73%) by nurses and 35 (27%) by physicians. After the 131 ATHI, only 2 ED visits (2/131, 1.5%) were recorded, whereas after the 272 alerts that did not result in a health intervention, we recorded 32 ED visits (13.2%), corresponding to an odds ratio of 0.10 (95% IC 0.02-0.43;
Five patients died during the study. All had alerts, 4 did not have ATHI and were hospitalized, and 1 had an ATHI (
Emergency department (ED) visits that occurred within 14 days of alerts generated by the eHealth system, according to the implementation of a health intervention triggered by the alerts.
Characteristics | ED visits (n=38) | No ED visits (n=367) | Odds ratio (95% CI) |
No alert-triggered health intervention, n (%) | 36 (13.1) | 238 (86.9) | Reference |
Alert-triggered health intervention, n (%) | 2 (1.5) | 129 (98.5) | 0.10 (0.02-0.43)a |
a
Association between intervention and death.
|
Alert-triggered health interventions | |||||
|
No (n=74) | Yes (n=132) |
|
|||
|
.04 | |||||
|
No (n=201) | 70 (94.6) | 131 (99.2) |
|
||
|
Yes (n=5) | 4 (5.4) | 1 (0.8) |
Users’ experience surveys were completed by 81 of 109 (72%) HAs involved in the study and 8 of 10 (80%) coordinating nurses. In terms of understanding the approach, 83% (90/109) of HAs reported that the screening questions were easy to understand. In terms of overall usability, the digital system was easy to use for 90% (73/81) of HAs, and the response time was acceptable for 89% (72/81) of them (
The eHealth system was also well perceived by the coordinating nurses (
The satisfaction of home aides about the eHealth system.
User’s experience with 8 coordinating nurses who received the alerts and completed the questionnaire.
During the study, 405 alerts (between 22 and 49 per month) were displayed, corresponding to 15.2% (405/2656) of the monitored visits. Of the 46 ED visits, 8 (17%) were not preceded by alerts in the previous 14 days and 38 (82%) were preceded by alerts within that time frame (
Contingency table for alerts generated by the eHealth system following home health aide visits and for emergency department visits occurring within 14 days of the alerts, and accuracy assessment.
Characteristics | Home aides’ visits with subsequent alerts (n=2656) | ||
|
Yes (n=405) | No (n=2251) |
|
Emergency department visits, n (%) | 38 (18.5) | 8 (0.3) | |
No emergency department visits, n (%) | 367 (81.5) | 2243 (99.7) |
Characteristics of alerts for predicting emergency department visits.
Home aides’ visits with subsequent alerts (n=2656) | Accuracy assessment (95% CI) |
Sensitivity, % (95% CI) | 83 (72-94) |
Specificity, % (95% CI) | 86 (85-87) |
Positive likelihood ratio | 5.87 (4.99-6.92) |
Negative likelihood ratio | 0.20 (0.11-0.38) |
Positive predictive value, % (95% CI) | 9.4 (6.5-12.2) |
Negative predictive value, % (95% CI) | 99.6 (99.3-99.9) |
In this intervention study, we successfully implemented an eHealth system based on HAs’ observations and a prediction algorithm that is capable of informing health care professionals of the risk of an ED visit in the next two weeks. In total, 109 HAs were involved in the study for 4753 visits. More than half of the visits were monitored. Alerts automatically displayed by this eHealth system accurately predicted emergency room visits, and 32% (131/405) of them were followed by interventions by the patients’ nurses or their general practitioners.
This eHealth system was well accepted and appreciated by HAs and their managers, and the accuracy was very good.
The evaluation of the accuracy of the alerts could have been biased by the transmission of the alert to a care manager. Nevertheless, it was shown that the interventions of the care managers allowed for a reduction in emergency hospitalizations, confirming the high predictive capacity of the system.
Balance between false negatives and false positives rates has been the subject of much reflection. In order to avoid a potential unnecessary ED visit (for a false positive, which is extremely rare), while avoiding as much as possible the loss of chances (false negative), the
The eHealth system reported in this study overcame the classic obstacles faced by such systems. First, the completion of the smartphone-based, customer-centered diagnostic tool was good, and 90% (72/81) of the HAs found it acceptable. This is in contrast to studies that have highlighted that barriers to the use of e-PROMs for caregivers or clinicians are primarily related to long completion time and poor usability [
Interestingly, in our study, the probability of an ED visit was very low after nurse or physician interventions following an alert, with a 10-fold decrease, compared to when alerts were not followed by such interventions. Even if this trial was not designed to examine such an outcome, this observation is very promising and prompts us to implement a controlled trial to document the effectiveness of the eHealth system. In addition, this device improves communication between professionals and promotes the empowerment of HAs. It responds to real public health issues for the prevention of the loss of autonomy in older people at home.
Our study faced some limitations. First, the study was conducted during the COVID-19 pandemic, which increased the risk of isolation for frail older adults and impacted primary care habits and HAs’ working conditions. In addition, the incidence of ED visits was lower compared to previous studies of participants with similar characteristics (13% vs 40%). This may be due to factors other than our intervention, such as reticence to attend EDs for fear of exposure to COVID-19 [
This study opens up broad prospects for optimizing the relevance of emergency visits for frail older adults. The predictive algorithm based on longitudinal observations of HAs could be improved by other types of input, such as patient’s clinical or biological records or measurements from connected devices. In addition, our approach could be applied to new target events in older adults in specific health contexts, including oncogeriatrics, cardiogeriatrics, or postsurgery. Clinical investigations are currently in progress and will allow for investigating the transferability of this system to certain clinical contexts. Finally, the system might evolve to a decision support system to help health professionals to optimize and personalize ATHI.
The eHealth system that we have successfully implemented offers an innovative approach to optimize the care of frail older adults. This approach is based on three paradigms: recording of the functional characteristics of daily life and their evolution over time, mobilization of nonprofessional health informants, and the use of a machine learning algorithm to monitor the level of individual risk and produce alerts that support health professional decisions for interventions. This means that multiple observers (not just social workers or nurses) could be trained to identify people at risk for ED visits. Such a predictive approach could form the basis for personalized health interventions that are designed to deliver early appropriate care and improve health outcomes.
The list of 23 items recorded by home care aides at each home visit and their completeness rates.
A simplified diagram of the processing system and the description of data processing system.
alert-triggered health intervention
emergency department
home aide
patient-reported outcome measure
The authors would like to thank the managers and the staff of the following home aid facilities as well as the care managers for their contribution to the study: Adar Provence, Arcade Assistance, Association Le Connétable, Centre Communale d’Actions Sociales de la ville du Chesnay-Rocquencourt, Communauté Professionnels de Territoire de Santé Marseille Itinéraire Santé, Intervenant Libéraux et Hospitaliers Unis pour le Patient, Pôle Autonomie Territorial Grand Versailles, and Plateforme Territoriale d’Appui Marseillaise.
This work is funded by the Agence Régionale de Santé Provence Alpes Cotes d’Azur and the Agence du Numérique en Santé. They did not play any role in the study except for the participation of Agence Régionale de Santé Provence Alpes Cotes d’Azur in the discussion about the ATHI.
The data that support the findings of this study are available from the corresponding author, JHV, upon reasonable request. Restrictions apply to the availability of the data collected by the home aids and the algorithm that was used in this study, which is the subject of a patent.
Each author met the International Committee of Medical Journal Editors criteria for authorship. JB, JHV, and FD conceptualized the study; JB, PV, MG, and SF conducted the acquisition of data; JB, JHV, FD, and PV contributed to the interpretation of the results; JB; JHV, and CHT were in charge of the formal analysis; JB, JHV, CHT, SM, FD, and PV drafted and revised the manuscript. All authors approved the final version of the manuscript.
JB received funding from Pfizer and Novartis for conferences and board participation. FD reports conflicts of interests with Chugai, Astra-Zeneca, Merck, Sivan, Takeda, Ipsen, Bristol Meyer Squibb, Viatris, Kelindi, and Hyperion where he is an invited speaker, and also serves on the advisory board at Sivan and Roche. The author holds stocks and shares with INeS, Kelindi, Hyperion and the institution receiver was Hyperion. JHV is the director of the company PRESAGE. CHT is employed by Présage care as a research and innovation coordinator. Other authors do not report any conflicts of interest.