This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Continuous positive airway pressure (CPAP) is an effective treatment for obstructive sleep apnea (OSA), but treatment compliance is often unsatisfactory.
The aim of this study was to assess the effectiveness and cost-effectiveness of an intelligent monitoring system for improving CPAP compliance.
This is a prospective, open label, parallel, randomized controlled trial including 60 newly diagnosed patients with OSA requiring CPAP (Apnea–Hypopnea Index [AHI] >15) from Lleida, Spain. Participants were randomized (1:1) to standard management or the MiSAOS intelligent monitoring system, involving (1) early compliance detection, thus providing measures of patient’s CPAP compliance from the very first days of usage; (2) machine learning–based prediction of midterm future CPAP compliance; and (3) rule-based recommendations for the patient (app) and care team. Clinical and anthropometric variables, daytime sleepiness, and quality of life were recorded at baseline and after 6 months, together with patient’s compliance, satisfaction, and health care costs.
Randomized patients had a mean age of 57 (SD 11) years, mean AHI of 50 (SD 27), and 13% (8/60) were women. Patients in the intervention arm had a mean (95% CI) of 1.14 (0.04-2.23) hours/day higher adjusted CPAP compliance than controls (
A machine learning–based intelligent monitoring system increased daily compliance, reported excellent patient satisfaction similar to that reported in usual care, and did not incur in a substantial increase in costs, thus proving cost-effectiveness. This study supports the implementation of intelligent eHealth frameworks for the management of patients with CPAP-treated OSA and confirms the value of patients’ empowerment in the management of chronic diseases.
ClinicalTrials.gov NCT03116958; https://clinicaltrials.gov/ct2/show/NCT03116958
Obstructive sleep apnea (OSA) is the most prevalent sleep-disordered breathing condition, affecting 15%-30% of adults in Western countries [
Nocturnal continuous positive airway pressure (CPAP), preventing upper airway collapse during sleep, is the treatment of choice for patients with symptomatic OSA [
So far, interventions tackling CPAP compliance, including novel educational and supportive or therapeutic strategies, have reported low to moderate evidence of success [
This is a prospective, open label, parallel, randomized controlled trial comparing the MiSAOS management model with care as usual for a duration of 6 months after CPAP prescription (ClinicalTrials.gov NCT03116958). The study was conducted from November 2016 to December 2017 in Lleida, Catalonia.
Eligible population included patients with OSA (AHI ≥15) being newly diagnosed in the sleep unit of University Hospital Santa Maria, Lleida, and requiring CPAP treatment according to the Spanish Respiratory Society (SEPAR) guidelines [
Accepting an α risk of .05 and a β risk of .2 in a 2-sided test, 29 patients per study arm were needed to recognize as statistically significant a difference in compliance greater than or equal to 1 hour/day. The common SD was assumed to be 1.35, based on previous research of the group.
Patients were recruited in the sleep unit and randomized (1:1) to receive 6 months of either MiSAOS or usual care management. Patients in the usual care arm were managed according to the SEPAR guidelines [
Similarly, patients in the MiSAOS arm were fitted with a mask, a CPAP device (AirSense 10; ResMed), and given a leaflet explaining its use. Patients received the same training sessions from the same personnel as in the usual care arm. However, these patients’ CPAP devices were equipped with mobile 2G (global systems for mobile/general packet radio service [GSM/GPRS]) technology capable of sending daily information on CPAP compliance, CPAPs, mask leaks, and residual respiratory events to the MiSAOS–Oxigen salud web database. In addition, patients in the MiSAOS arm had access to an integrated platform including a website [
Baseline information was collected by sleep unit personnel during recruitment, regardless of the study arm. This included age; gender; socioeconomic level; Epworth Sleepiness Scale (ESS) score; EuroQoL-5D quality of life (EQ-5D); lifestyle habits (tobacco and alcohol consumption); comorbidities; use of medications; weight; height; BMI; neck, waist, and hip circumference; and BP. Variables of the sleep study were also recorded and included registration time, sleep duration, AHI, and percentage of nighttime spent with an oxygen saturation less than 90%.
At 3 and 6 months all patients, regardless of the study arm, were visited at the sleep unit. Patients were checked about progress and compliance with therapy and any problems with their CPAP machine. During these visits we collected data on treatment compliance (number of hours/day), ESS score, OSA-related symptoms, EQ-5D, BP, and anthropometric variables. Additionally, data on CPAP, residual respiratory events and leaks, CPAP-related side effects (mask allergies and skin irritations, dry mouth, congestion, runny nose, sneezing, sinusitis, nosebleeds, and discomfort), overall satisfaction with the therapy (questionnaire), CPAP machine care and maintenance actions (ie, changes of mask), and the number of any additional visits and calls required by the patient during the follow-up were collected. Finally, costs for each component, use of services, and visits were computed based on standard prices of the CPAP provider and on Catalan Health Department official data (CVE-DOGC-A-13051031-2013) [
A
The primary and secondary analyses were performed on both the intention-to-treat (ITT) and per-protocol (PP) samples. The ITT sample included all the patients who were randomized. The PP sample excluded the patients who were lost during the follow-up period. Missing data were imputed using multiple imputation consisting of chained equations, for which 10 complete databases were checked. The R package “mice” was used for these calculations. All statistical analyses and data processing procedures were performed using R software, version 3.4.4 (The R Foundation).
This study was approved by the Ethics Committee of Hospital Arnau de Vilanova (CEIC-1283) and all patients provided written informed consent. This project was registered in ClinicalTrials.gov (registration number NCT03116958).
A total of 60 patients were randomized to receive either MiSAOS (intervention; n=30) or usual care (control; n=30) management, and up to 53 patients completed the study (
Study flowchart. CPAP: continuous positive airway pressure, ITT: intention-to-treat
Patients’ baseline characteristics (N=60).a
Characteristic | Control (n=30) | Intervention (n=30) | |
Gender (male), n (%) | 26 (87) | 26 (87) | .99 |
Age (years), mean (SD) | 58 (10) | 52 (12) | .04 |
Weight (kg), mean (SD) | 97 (19) | 101 (23) | .42 |
BMI (kg/m2), mean (SD) | 33.1 (6.4) | 34.7 (7.3) | .38 |
Systolic blood pressure (mmHg), mean (SD) | 138 (17) | 142 (20) | .42 |
Diastolic blood pressure (mmHg), median (range) | 87 (79-96) | 88 (81-95) | .74 |
Apnea–Hypopnea Index (events/hour) | 39 (25-71) | 53 (35-65) | .22 |
aData as per
Differences in primary and secondary outcomes of the trial according to an intention-to-treat analysis (N=60).a
Differences | Control (n=30), mean (SD) | Intervention (n=30), mean (SD) | Difference, mean (95% CI) | |||||
|
|
|
|
|||||
|
Compliance (hours/day) | 4.89 (2.30) | 5.79 (1.60) |
|
||||
|
Crude difference |
|
|
0.90 (–0.16 to 1.96) | ||||
|
Adjusted difference |
|
|
1.14 (0.04 to 2.23) | ||||
|
|
|
|
|||||
|
|
|
|
|
||||
|
|
Baseline | 10.9 (5.35) | 11.1 (5.35) |
|
|||
|
|
6 months | 4.90 (2.41) | 5.85 (3.91) |
|
|||
|
|
Change | –5.98 (4.42) | –5.22 (4.78) |
|
|||
|
|
Crude difference |
|
|
0.76 (–1.64 to 3.16) | |||
|
|
Adjusted difference |
|
|
1.05 (–0.51 to 2.61) | |||
|
|
|
|
|
||||
|
|
Baseline | 97.0 (18.6) | 101 (22.5) |
|
|||
|
|
6 months | 98.2 (20.2) | 100 (20.7) |
|
|||
|
|
Change | 1.26 (7.86) | –0.95 (7.91) |
|
|||
|
|
Crude difference |
|
|
–2.21 (–6.98 to 2.56) | |||
|
|
Adjusted difference |
|
|
–2.55 (–7.41 to 2.32) | |||
|
|
|
|
|
||||
|
|
Baseline | 33.3 (6.20) | 34.7 (7.17) |
|
|||
|
|
6 months | 34.2 (6.80) | 34.8 (6.32) |
|
|||
|
|
Change | 0.98 (3.26) | 0.14 (3.16) |
|
|||
|
|
Crude difference |
|
|
–0.84 (–2.95 to 1.27) | |||
|
|
Adjusted difference |
|
|
–0.82 (–2.97 to 1.32) | |||
|
|
|
|
|
||||
|
|
Baseline | 138 (17.0) | 142 (19.4) |
|
|||
|
|
6 months | 131 (12.7) | 138 (17.2) |
|
|||
|
|
Change | –7.02 (15.2) | –3.80 (12.7) |
|
|||
|
|
Crude difference |
|
|
3.22 (–5.03 to 11.47) | |||
|
|
Adjusted difference |
|
|
7.81 (0.57 to 15.05) | |||
|
|
|
|
|
||||
|
|
Baseline | 87.7 (13.5) | 90.3 (12.5) |
|
|||
|
|
6 months | 81.6 (8.84) | 86.8 (9.23) |
|
|||
|
|
Change | –6.13 (11.4) | –3.52 (10.6) |
|
|||
|
|
Crude difference |
|
|
2.61 (–4.04 to 9.27) | |||
|
|
Adjusted difference |
|
|
4.52 (–0.65 to 9.69) | |||
|
|
|
|
|
||||
|
|
Baseline | 0.84 (0.22) | 0.85 (0.17) |
|
|||
|
|
6 months | 0.80 (0.19) | 0.86 (0.20) |
|
|||
|
|
Change | –0.04 (0.17) | 0.00 (0.18) | ||||
|
|
Crude difference |
|
|
0.05 (–0.05 to 0.15) | |||
|
|
Adjusted difference |
|
|
0.03 (–0.06 to 0.13) | |||
|
|
|
|
|
||||
|
|
Baseline | 4.93 (3.41) | 4.63 (3.55) |
|
|||
|
|
6 months | 7.35 (1.71) | 8.03 (1.32) |
|
|||
|
|
Change | 2.42 (2.87) | 3.40 (3.65) |
|
|||
|
|
Crude difference |
|
|
0.98 (–0.72 to 2.69) | |||
|
|
Adjusted difference |
|
|
0.51 (–0.3 to 1.33) |
aOrdinary least-squares linear models adjusted by age and baseline value.
bESS: Epworth Sleepiness Scale.
cEQ-5D: EuroQoL-5D quality of life.
dHUI: health utility index
eVAS: visual analog scale.
Patients’ satisfaction with the management of their illness was excellent in both study groups (
Overall patients’ satisfaction and satisfaction with MiSAOS (N=45).a
Users’ satisfaction | Control (n=26) | Intervention (n=19) | |||||||||||
|
|||||||||||||
|
|
||||||||||||
|
|
Agrees/strongly agrees, n (%) | 26 (100) | 19 (100) |
|
||||||||
|
|
Overall score (1-7), mean (SD) | 6.38 (0.80) | 6.53 (0.61) | .51 | ||||||||
|
|
||||||||||||
|
|
Agrees/strongly agrees, n (%) | 26 (100) | 19 (100) |
|
||||||||
|
|
Overall score (1-7), mean (SD) | 6.62 (0.57) | 6.53 (0.61) | .62 | ||||||||
|
|
||||||||||||
|
|
Agrees/strongly agrees, n (%) | 26 (100) | 18 (95) |
|
||||||||
|
|
Overall score (1-7), mean (SD) | 6.62 (0.64) | 6.21 (1.62) | .31 | ||||||||
|
|||||||||||||
|
|
||||||||||||
|
|
Agrees/strongly agrees, n (%) |
|
17 (100) |
|
||||||||
|
|
Overall score (1-7), mean (SD) |
|
6.53 (0.72) |
|
||||||||
|
|
||||||||||||
|
|
Agrees/strongly agrees, n (%) |
|
16 (94) |
|
||||||||
|
|
Overall score (1-7), mean (SD) |
|
6.41 (0.87) |
|
||||||||
|
|
||||||||||||
|
|
Agrees/strongly agrees, n (%) |
|
15 (88) |
|
||||||||
|
|
Overall score (1-7), mean (SD) |
|
6.35 (1.17) |
|
aThe overall satisfaction questionnaire was answered by 26 controls and 19 intervention participants. The satisfaction with MiSAOS questionnaire was answered by 17 participants.
Within-trial intervention and follow-up costs (average cost per randomized patient; N=60).
Concept | Control (n=30), €a/patient, mean (SD) | Intervention (n=30), €/patient, mean (SD) | Difference, mean (95% CI) | ||||
|
|
|
|
||||
|
2G (GSMc/GPRSd) daily data transfer | 0 (0) | 41.5 (0) | –41.5 (—) | |||
|
Activation and maintenance | 0 (0) | 8 (0) | –8 (—) | |||
|
|
|
|
||||
|
Sleep unit visits and consultationse | 41 (0) | 0 (0) | 41 (—) | |||
|
CPAPf provider visits and consultationsg | 9.7 (8.9) | 10.0 (10.8) | –0.33 (–5.5 to 4.8) | |||
|
|
|
|
||||
|
ResMed Mirage Quattro | 12.5 (28.4) | 10.0 (32.6) | 2.5 (–13.3 to 18.3) | |||
|
ResMed Mirage FX | 0.8 (4.4) | 3.2 (8.3) | –2.4 (–5.9 to 1.1) | |||
|
ResMed Mirage Micro | 0 (0) | 1.1 (4.1) | –1.07 (–2.6 to 0.4) | |||
|
ResMed Swift FX | 1.5 (8.2) | 4.5 (13.7) | –3 (–8.9 to 2.9) | |||
|
ResMed Airfit P10 (without head-gear) | 0 (0) | 1.3 (7.3) | –1.33 (–4.1 to 1.4) | |||
|
Philips Respironics Nuance gel | 0 (0) | 2.45 (13.4) | –2.45 (–7.5 to 2.6) | |||
|
ResMed Airfit F10 | 15.0 (30.5) | 10.0 (25.9) | 5 (–9.6 to 19.6) | |||
|
ResMed Airfit P10 (with head-gear) | 1.50 (8.22) | 0 (0) | 1.5 (–1.6 to 4.6) | |||
|
SleepNet IQ | 2.20 (8.86) | 0 (0) | 2.2 (–1.1 to 5.5) | |||
|
SleepNet Ascend | 0 (0) | 0.8 (4.4) | –0.8 (–2.4 to 0.8) | |||
|
Philips Respironics Amara View | 5.00 (15.3) | 3.3 (18.3) | 1.66 (–7.0 to 10.4) | |||
|
Philips Respironics Comfort Gel Blue | 1.00 (5.48) | 0 (0) | 1 (–1.0 to 3.0) | |||
|
Total | 90.2 (53.1) | 96.2 (62.1) | –6.0 (–35.9 to 23.9) |
a€1 = US $1.17.
bEstimated costs supplied by the CPAP provider: 2G (GSM/GPRS) daily data transfer (€83 [US $97.32]/year); activation and maintenance (€16 [US $18.76]/year).
cGSM: global systems for mobile.
dGPRS: general packet radio service.
eNot including the baseline, 3-month, and 6-month visits, as all patients did them regardless of study arm. Costs based on the Catalan Institute of Health (CVE-DOGC-A-13051031-2013): sleep unit visits and consultations (€41 [US $48.07]/contact).
fCPAP: continuous positive airway pressure.
gCommercial costs supplied by the CPAP provider: CPAP provider visits and consultations (€10 [US $11.73]/contact).
hCommercial costs supplied by the CPAP provider: ResMed Mirage Quattro (€75 [US $87.94]/unit); ResMed Mirage FX (€24 [US $28.14]/unit); ResMed Mirage Micro (€16 [US $18.76]/unit); ResMed Swift FX (€45 [US $52.76]/unit); ResMed Airfit P10 (without headgear) (€40 [US $46.90]/unit); Philips Respironics Nuance gel (€73.5 [US $86.18]/unit); ResMed Airfit F10 (€75 [US $87.94]/unit); ResMed Airfit P10 (with head-gear) (€45 [US $52.76]/unit); SleepNet IQ (€22 [US $25.80]/unit); SleepNet Ascend (€24 [US $28.14]/unit); Philips Respironics Amara View (€50 [US $58.63]/unit); Philips Respironics Comfort Gel Blue (€30 [US $35.18]/unit).
Cost-effectiveness analysis based on treatment compliance (CPAP hours/day) and total costs for each arm, performed using a bootstrap probabilistic sensitivity analysis. CPAP: continuous positive airway pressure.
This study is the first randomized controlled clinical trial assessing the effectiveness and cost-effectiveness of a machine learning–based intelligent monitoring system aiming to improve CPAP compliance in patients with OSA. The MiSAOS intelligent monitoring system, based on early compliance detection, compliance prediction, and rule-based recommendations, was compared with usual care in the region of Lleida, showing a mean increase of 1.14 hours in daily compliance with no substantial differences in direct costs and an excellent patient satisfaction. This novel management system proved to be cost-effective and thus a viable option for the management of patients with OSA treated with CPAP.
This study has several strengths, including the (1) use of the same CPAP devices in both study arms; (2) use of an intelligent monitoring system model, based on early compliance detection, machine learning–based compliance prediction, and rule-based recommendations; (3) inclusion of continuous patient feedback through an app; (4) measurement of a broad range of effect measures (ie, compliance, changes in symptoms, and changes in quality of life); (5) assessment of patient comfort and satisfaction; and (6) inclusion of cost and cost-effectiveness analyses. Nevertheless, there are also some limitations to be acknowledged: (1) the slight infraestimation of the required number of study participants limited the statistical power of some of the between-arm comparisons, although this did not affect the results on the primary outcome and cost-effectiveness analysis; (2) the assessment of patient satisfaction was performed using a nonvalidated questionnaire; (3) the exclusion of patients with severe chronic pathologies and other dyssomnias or parasomnias could limit the generalizability of our results, although the included patients would be the ideal target for eHealth interventions as more complex patients could require a close follow-up in the sleep units; (4) the results of cost analyses are highly dependent on the characteristics of the health care setting in which they are conducted and, thus, extrapolation of the results to different settings should be done cautiously; and (5) the follow-up period does not allow the extrapolation of results to the long term.
Patients experiencing the MiSAOS intelligent monitoring system showed a mean increase of 1.14 hours in daily CPAP compliance when compared with patients in usual care. This result is more positive than the mean (95% CI) increase of 0.54 (0.29-0.79) hours reported by Aardoom et al [
The impact of the CPAP treatment on secondary outcomes in the MiSAOS intervention was very similar to that achieved in usual care and reports in previous literature [
Patient’s comfort and satisfaction are key drivers of compliance with CPAP treatment in the long term [
A key aspect of any new management strategy is the cost of the intervention and its cost-effectiveness. In this study, the analysis of costs and cost-effectiveness showed that the MiSAOS intervention had an overall cost similar to that of usual care while providing better results in terms of treatment compliance, thus demonstrating cost-effectiveness. This result is in contrast to previous cost-effectiveness trials of telemonitoring interventions for CPAP-treated patients in Spain, where cost-effectiveness was demonstrated because of an overall reduction in costs and no significant differences in effectiveness were found [
As previously stated, the main barriers for the large-scale implementation of a novel management intervention are its costs and cost-effectiveness. In the optimal scenario, a novel management strategy should be cheaper than usual care while providing better results. The MiSAOS model has shown the potential to generate better results than usual care in terms of compliance. However, it was not cheaper than usual care. It is worth mentioning that a big proportion of the intervention’s cost corresponded to the use of a 2G (GSM/GPRS) system for daily CPAP compliance data transfer. This technology could be easily replaced by a secure wireless connection to the patients’ home Wi-Fi network, which would represent a huge saving and further boost cost-effectiveness. Even in rural areas such as Lleida, this scenario is rapidly becoming a reality and most homes have a suitable Wi-Fi network.
The use of a machine learning–based intelligent monitoring system increased daily compliance, reported an excellent patient satisfaction similar to that reported in usual care, and did not incur in a substantial increase in costs, thus proving cost-effectiveness. This study supports the implementation of intelligent eHealth frameworks for the management of patients with OSA treated by CPAP and confirms the value of patients’ empowerment in the management of chronic diseases.
CONSORT-eHealth checklist (V 1.6.1).
Screenshots of the MiSAOS app showing from left to right and top to bottom: (i) loading screen with the app’s name; (ii) last week’s overall CPAP treatment performance summary, including an overall numeric score (0-10) together with a summary smiley, average compliance in hours, average air leaks, average residual apnea-hypopnea index (AHI), specific compliances for the last 7 days, and a summary text including reinforcements and tips to improve the overall rating in upcoming weeks (in this case, a positive reinforcement message for and a tip regarding air-leaks and hours of use); (iii) last week’s overall CPAP treatment performance summary with focus on air-leaks, including last 14 days air-leak information and additional details; (iv) patient and device’s summary information including technical details on the CPAP treatment characteristics; (v) main achievements summary, including a ranking of the patient’s performance compared to other CPAP users in the region and challenges’ progression such as good compliance streaks; and, (vi) sample information and training screen, in this case explaining the basics of obstructive sleep apnea pathophysiology.
Screenshots of the MiSAOS website showing from top to bottom: (i) registry of events and actions taken in relation to a given patient, including date, action details and comments; (ii) example of data collection for the feed of the predictive algorithms; (iii) sample prediction provided by MiSAOS intelligent algorithms, in this case predicting 6-month compliance based on early compliance and information such as the one collected in the previous screenshot; and, (iv) example of available training material tackling the most common issues and doubts of patients (in addition to videos there are also manuals, FAQs, and tips).
Differences in primary and secondary outcomes of the trial according to a per-protocol analysis.
Apnea–Hypopnea Index
blood pressure
continuous positive airway pressure
Epworth Sleepiness Scale
intention-to-treat
obstructive sleep apnea
per protocol
Spanish Respiratory Society
This work is part of the myOSA project (RTC-2014-3138-1), funded by the Spanish Ministry of Economy, Industry and Competitiveness (Ministerio de Economía, Industria y Competitividad) and Agencia Estatal de Investigación, under the framework “Retos-Colaboración”, State Scientific and Technical Research and Innovation Plan 2013-2016. The work was cofunded by the European Regional Development Fund (ERDF), “A way to make Europe”. JdB acknowledges receiving financial support from the Catalan Health Department (Pla Estratègic de Recerca i Innovació en Salut [PERIS] 2016: SLT002/16/00364) and Instituto de Salud Carlos III (ISCIII; Miguel Servet 2019: CP19/00108), co-funded by the European Social Fund (ESF), “Investing in your future”. Funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
CT, EV, JB, FB, and JdB participated in the conceptualization of project. CT, AL, LP, RV, and AC conducted data collection. CT, IB and AM-M participated in data curation. IB conducted all statistical analyses. CT, IB, and JdB wrote the original draft of the manuscript. All authors reviewed the final manuscript. JB and FB secured funding for the project.
None declared.