This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Several technologies have been proposed to support the reduction of insomnia complaints. A user-centered assessment of these technologies could provide insight into underlying factors related to treatment adherence.
Gaining insight into adherence to technology-mediated insomnia treatment as a solid base for improving those adherence rates by applying adherence-enhancing strategies.
Adherence to technology-mediated sleep products was studied in three ways. First, a meta-analysis was performed to investigate adherence rates in technology-mediated insomnia therapy. Several databases were queried for technology-mediated insomnia treatments. After inclusion and exclusion steps, data from 18 studies were retrieved and aggregated to find an average adherence rate. Next, 15 semistructured interviews about sleep-support technologies were conducted to investigate perceived adherence. Lastly, several scenarios were written about the usage of a virtual sleep coach that could support adherence rates. The scenarios were discussed in six different focus groups consisting of potential users (n=15), sleep experts (n=7), and coaches (n=9).
From the meta-analysis, average treatment adherence appeared to be approximately 52% (95% CI 43%-61%) for technology-mediated insomnia treatments. This means that, on average, half of the treatment exercises were not executed, suggesting there is a substantial need for adherence and room for improvement in this area. However, the users in the interviews believed they adhered quite well to their sleep products. Users mentioned relying on personal commitment (ie, willpower) for therapy adherence. Participants of the focus groups reconfirmed their belief in the effectiveness of personal commitment, which they regarded as more effective than adherence-enhancing strategies.
Although adherence rates for insomnia interventions indicate extensive room for improvement, users might not consider adherence to be a problem; they believe willpower to be an effective adherence strategy. A virtual coach should be able to cope with this “adherence bias” and persuade users to accept adherence-enhancing strategies, such as reminders, compliments, and community building.
People who suffer from insomnia have difficulties with initiating sleep, maintaining sleep, or early-morning awakenings, and this sleep disturbance significantly impairs their daily functioning [
Although the consequences of insomnia may be severe and prevalence is substantial, only a few people seek treatment [
Although CBT-I is effective, there is a lack of knowledge and accessibility regarding this type of therapy [
The World Health Organization (WHO) recognizes the importance of adherence to health regimes in general. They stated, “Adherence is a primary determinant of the effectiveness of treatment” [
Various authors, for example, Beun [
The studies in this paper are conducted in the context of the Sleepcare project [
In order to determine whether a certain outcome is related to a treatment, adherence rates must be measured. Otherwise, it cannot be claimed that the outcome was caused by the intervention [
First, it is important to distinguish between at least two concepts:
After analyzing reported adherence rates to technology-mediated sleep treatment in the literature, the next step was to study coachees’ reasons why they do or do not adhere to technology-mediated sleep interventions. To do so, interviews were conducted with people who (had) used a sleep product. The first step was to identify a sample of technology-mediated sleep products. The most familiar sleep product is probably the alarm clock. Besides alarms, there are many other sleep-supporting technologies on the market. For example, relaxation-supporting technologies, sleep-measuring apps and devices, and computerized therapies.
A limitation of the interviews from Study II, as will be discussed in more detail in the Results section, was that they were restricted to existing products, and did not include reflections on what might technically be possible regarding adherence-enhancing strategies. During the interviews, it also proved to be difficult for participants to think of additional functionality that could improve their adherence. To address the limitations of the interviews, focus groups were organized to discuss adherence-enhancing strategies of a to-be-developed sleep coach. The aim of study III was to gain insight into coachees’ attitudes and beliefs toward these adherence-enhancing strategies, for which focus groups are particularly suited [
The meta-analysis was primarily performed to answer the question "How well do coachees adhere to technology-mediated insomnia interventions and diagnostic tools?
Of the 18 included studies in this meta-analysis, 12 studies (67%) focused on CBT-I (
Characteristics of included studies.
First author | Condition | Number of people | Number of females/males | Mean age | Sleep problem severity, |
Oosterhuis [ |
Intervention | 400 | 63% female | 55 | N/Aa |
Rybarczak [ |
Intervention | 14 | 22/16 | 68 | PSQIb, 9.5 |
|
CBTc | 11 |
|
|
PSQI, 11.9 |
|
Control | 13 |
|
|
PSQI, 9.9 |
Ström [ |
Intervention | 54 | 71/38 | 44 | ISId, 18.08 |
|
Waiting list | 55 |
|
|
ISI, 18.11 |
Suzuki [ |
Intervention | 21 | 16/25 | 40 | N/A |
|
Waiting list | 22 |
|
|
|
Ritterband [ |
Intervention | 22 | 34/10 | N/A | ISI, ≥8 |
|
Waiting list | 23 |
|
|
ISI, ≥8 |
Van Straten [ |
Intervention | 126 | 163/84 | 52 | 72% rated SQe<6/10 |
|
Waiting list | 121 |
|
|
68% rated SQ<6/10 |
Vincent [ |
Intervention | 59 | 79/39 | N/A | N/A |
|
Waiting list | 59 |
|
|
|
Riley [ |
Intervention 1 | 24 | 52/38 | 49 | ISI, 8-14 (25 people) |
|
Intervention 2 | 33 |
|
|
ISI, 15-21 (53 people) |
|
SMMTf | 33 |
|
|
ISI, 22-28 (12 people) |
Lancee [ |
CCBT-Ig | 216 | 520/103 | 52 | Sleep-50, ≥19 |
|
CBT-Ih | 202 |
|
|
|
|
Waiting list | 205 |
|
|
|
Ritterband [ |
Intervention | 14 | 24/4 | 57 | ISI, 17.1 |
|
Waiting list | 14 |
|
|
ISI, 15.9 |
Espie [ |
Intervention | 55 | 120/44 | 49 | Met DSM-5i criteria |
|
TAUj | 54 |
|
|
Met DSM-5 criteria |
|
IRTk | 55 |
|
|
Met DSM-5 criteria |
Haimov [ |
Cognitive training (CogniFit) | 34 | 29/22 | 72 | Met AASMl criteria |
|
Active controlm | 17 |
|
|
Met AASM criteria |
Lancee [ |
Low depression | 198 | 316/163 | 47 | ISI, 16.73 |
|
Mild depression | 182 |
|
|
ISI, 18.63 |
|
High depression | 99 |
|
|
ISI, 20.69 |
|
|
|
|
|
Average ISI, 18.72 |
Lancee [ |
With support | 129 | 197/65 | 48 | ISI, 16.95 |
|
Without support | 133 |
|
|
ISI, 17.32 |
Lawson [ |
Intervention | 36 | 21/5 | 34 | N/A |
Van Straten [ |
Intervention | 59 | 83/35 | 49 | PSQI, 12.4 |
|
Waiting list | 59 |
|
|
PSQI, 11.7 |
Holmqvist [ |
Intervention | 39 | 55/18 | N/A | ISI, 18.72 |
|
CBT-I | 34 |
|
|
ISI, 18.50 |
Lipschitz [ |
Intervention | 37 | 27/10 | 37 | ISI, 7.24 |
aNot applicable (N/A)
bPittsburgh Sleep Quality Index (PSQI)
cCognitive behavioral therapy (CBT)
dInsomnia Severity Index (ISI)
eSleep quality (SQ)
fSelf-monitoring minimal treatment (SMMT)
gComputerized cognitive behavioral therapy for insomnia (CCBT-I)
hCognitive behavioral therapy for insomnia (CBT-I)
iDiagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5)
jTreatment as usual (TAU)
kImagery relief therapy (placebo) (IRT)
lAmerican Academy of Sleep Medicine (AASM)
mActive control consisted of word and paint training
Inclusion and exclusion criteria for papers in the meta-analysis.
Description of included studies.
First author | Intervention | Delivery | Treatment |
Follow-up length | Posta | Follow |
Adherence |
Oosterhuis [ |
SEc, SHd, CThe, RXf | TVg | 8 weeks | 4.5 months | Qh | Q | N/Ai |
Rybarczyk [ |
RX, SCj, SRk, CTh, SH | Audiotape | 6 weeks | 4 months | Q&Dl | Q&D | N/A |
Ström [ |
CBT-Im | Internet | 5 weeks | 9 months | Q&D | D | N/A |
Suzuki [ |
CBT-I | Internet | 2 weeks | 3 weeks | Q | Q | N/A |
Ritterband [ |
CBT-I | Internet | 9 weeks | 6 months | Q&D | Q | N/A |
Van Straten [ |
CBT-I | TV | 6 weeks | None | Q&D | N/A | Self-report |
Vincent [ |
CBT-I | Internet | 5 weeks | 4 weeks | Q&D | Q&D | Self-report |
Riley [ |
ASSn/CBT-I | Device | 6 weeks | 6 weeks | Q&D | Q&D | N/A |
Lancee [ |
CBT-I | Internet | 6 weeks | 4 weeks |
|
Q&D | Self-report |
Ritterband [ |
CBT-I | Internet | 6-9 weeks | None | Q&D | N/A | Log |
Espie [ |
CBT-I | Internet | 6 weeks | 8 weeks | N/A | Q&D | Log |
Haimov [ |
CTro | PCp | 8 weeks | None | Q&D | N/A | N/A |
Lancee [ |
CBT-I | Internet | 6 weeks | 4 weeks | N/A | Q&D | Self-report |
Lancee [ |
CBT-I | Internet | 6 weeks | 6 months | Q&D | Q&D | Log |
Lawson [ |
ASS | Appq | 7 days | None | Q | N/A | Log |
Van Straten [ |
CBT-I | Internet | 6 weeks | 3 months | Q&D | Q&D | Log |
Holmqvist [ |
CBT-I | Internet | 6 weeks | 8 weeks | Q&D | Q&D | N/A |
Lipschitz [ |
MBBr | Internet | 3 days | 1 week | Q | Q | Self-report |
aPostintervention measurement instrument
bFollow-up measurement instrument
cSleep education (SE)
dSleep hygiene (SH)
eCognitive therapy (CTh)
fRelaxation (RX)
gTelevision (TV)
hQuestionnaire (Q)
iNot applicable (N/A)
jStimulus control (SC)
kSleep restriction (SR)
lSleep diary (D)
mCognitive behavioural therapy for insomnia (CBT-I)
nActive sleep sampling (ASS) device
oCognitive training (CTr) (CogniFit)
pPersonal computer (PC)
qMobile phone app (app)
rMind-body bridging (MBB)
In order to establish a purposive sample of users across sleep products, various sleep products were categorized. Based on their background knowledge and a media scan, the authors generated a list of 54 technologies over the course of a few months. This composed list was supplemented with apps because the goal of the Sleepcare project is to design a virtual sleep coach on a mobile phone. The first 25 Android apps and 25 iPhone apps found in Google Play and the iTunes store with the search word "sleep" on November 19, 2012, were added to the product list. A total of 7 apps were unrelated to sleep—3 games, 2 hypnosis apps, 1 unlock, 1 music timer—and were therefore discarded, resulting in a list of 97 sleep products. The categorization made in this paper aims to be simple and objective. Sleep products were categorized based on their goal and the medium used.
After identifying the categories of existing sleep products, the next step was to learn more about the users’ usage and adherence to the sleep products. Interviews were conducted with people who used a sleep product in each of the largest product-medium combinations (eg, apps that help people fall asleep).
A graph showing the relationship between the goal of sleep products and the medium used. The size of each bubble indicates how many products of the 97 identified sleep products belong to that category.
People registered as participants at the Sleepcare project website [
Besides adherence, the interviews covered other topics to gain insight regarding users’ experiences with sleep products. Therefore, the semistructured interviews included both adherence-related questions and questions regarding the factors of the Unified Theory of Acceptance and Use of Technology (UTAUT) model [
The first author (CH) performed the data analysis following the phases of thematic analysis as described by Braun and Clarke [
The envisioned coach would use different adherence-enhancing strategies during the entire coaching process. For example, different roles (eg, motivator and educator) could be played by different virtual characters to increase the effect of the to-be-developed sleep coach (ie, split-persona effect) [
A total of 12 scenarios and 72 claims (see
The scenarios and claims were discussed in six focus groups to evaluate the adherence-enhancing strategies. Two groups consisted of potential users, two groups consisted of coaches, and a further two groups consisted of sleep experts. Demographics of the Dutch participants can be found in
The analysis was an iterative process of developing codes and themes in line with thematic analysis [
Demographics of the participants per focus group.
Focus groups | Participants, |
Age in years, |
Number of participants with a PSQIa>5, n (%) | Expertise |
Potential users 1 | 8 (38) | 35 (12) | 3 (38) | N/Ab |
Potential users 2 | 7 (71) | 48 (9) | 5 (71) | N/A |
Coaches 1 | 4 (75) | 51 (8) | N/A | 4 coaches (relationships, |
Coaches 2 | 5 (80) | 50 (6) | N/A | 4 coaches (lifestyle, career), |
Sleep experts 1 | 3 (67) | 50 (18) | N/A | 1 psychologist, 1 therapist, |
Sleep experts 2 | 4 (75) | 47 (14) | N/A | 3 researchers, |
aPittsburgh Sleep Quality Index (PSQI).
bNot applicable (N/A).
All analyses were completed with the Comprehensive Meta-Analysis statistical package, version 3, and were based on the random-effects model. In the analyses, a distinction was made between experimental compliance and treatment adherence. All studies reported experimental compliance, and most of them (10) also reported treatment adherence (see
Treatment adherence was reported in various ways, which can roughly be classified into two groups, namely self-reports and logs. Self-reports refer to questions in which participants were asked how well they adhered to the exercises. The five studies that used self-reports found that 41% (95% CI 36%-46%) of the participants met the adherence criteria set in that study. Logs refer to reports that show how many sessions were completed. A total of 5 studies used logs and found that 64% (95% CI 44%-79%) of the participants completed all sessions. If these two kinds of measures are taken together, an average treatment adherence of 52% (95% CI 43%-61%) is reached with reported adherence ranging from 28% [
In
Lastly, the relationship between treatment adherence and the effect size of the individual treatments was explored.
Mean compliance and adherence rates and their 95% CIs. post: posttreatment measurement; follow: follow-up measurement; Qs: questionnaires; Ds: diaries; self-report: self-reported adherence with questions; logs: automatically logged behavior.
Statistics of the meta-regression of adherence and effect size of the individual treatments.
Statistics meta-regression | Coefficient | Standard error | 95% CI |
|
|
Intercept | 0.74 | 0.20 | 0.35-1.13 | 3.69 | <.001 |
Adherence | 0.79 | 0.35 | 0.10-1.47 | 2.25 | .03 |
Meta-regression of adherence on effect size of treatments. The circles represent the individual studies [
The three main categories related to adherence are usage, effectiveness, and adherence (see
Usage
Intention
Two reasons for usage
Overcome sleep problems
Interest in the product
When used
Sleep trackers, alarms, relaxation: used in the evenings
Sleep coaches: varying usage times
Effectiveness
Per product type
Therapy-related products: no noticeable effect
Alarms: ambiguous effect, wake-up is okay, but not waking up better
Adherence
Keep using
Therapy-related products: personal attitude
Consumer products: need functionality
Not using
Consumer products
No need for functionality (anymore)
Product does not work
Forget to use product
Two initial reasons for using a product emerged from the interviews. First, all interviewees used the product to overcome some of their sleeping troubles. Interviewees wanted to wake up better, initiate or maintain sleep, and/or increase insight into their sleep. Second, some interviewees used a product because they thought the product in itself was interesting. Above all, this holds for the sleep-tracking apps. Most products—alarms, automatic sleep trackers, and relaxation support—were used in the evening before going to sleep. Most of those products, however, were not used on the weekend. Sleep coach usage varied, depending on the kind of assignments included in the product (eg, diary, relaxation, sleep hygiene exercise, bedtime scheduling).
Participants' quotes regarding reasons to start using sleep products were as follows (translated from Dutch):
The reason was that in my opinion I was awake too often, and too long. I could not fall asleep anymore.
Friends of mine had the app and I wanted to try it as well.
Participants' quotes regarding usage of sleep products were as follows (translated from Dutch):
I turn it on in the evening when I am lying in bed and want to go to sleep
Every day I needed to get up I used it, but on the weekends, for example, when I don’t need to get up I didn’t do anything with the app.
Actually, I did it whenever it suited me [about filling in a sleep diary].
One of the initial arguments for using a product was to overcome some kind of sleeping problem. However, the online sleep therapies were not perceived as having an effect on the interviewees’ sleep problems. Additionally, interviewees mentioned that it was hard to determine if the therapy improved sleep in the long term because they tried several things. Nevertheless, most interviewees took some advice that worked for them and continued applying it. Furthermore, sleep tracking apps as well as online sleep coaches provided the interviewees with more insight into their sleep and habits. Products that wake interviewees up (ie, smart alarm apps and wake-up lights) were assessed ambiguously. Both types of products did what was expected of them, namely wake the interviewee up. However, the effect of waking up better with the product was doubted. Moreover, smart alarms did not seem to fit into interviewees’ daily lives (see quote below from interview #4, sleep tracking app).
Participants' quotes regarding the effectiveness of sleep products were as follows (translated from Dutch):
The goal to sleep better was not reached
I have no clue if it helped, because it is going better at the moment, but I did other things in that same time period.
Also getting out of bed when I am awake for more than 30 minutes. The advice has helped, yes.
It measures the sleep debt that you are building up, that was effective.
It did what it supposed to do, wake me up.
Still not very well, but it became a little bit better, a little bit more pleasant
Problem of the app [smart alarm] is that you do not know what time you will wake up exactly. If I have an appointment somewhere I need an hour to get ready. If you do not know how late your alarm will go, it is hard to plan.
In general, interviewees perceived their own usage as sufficient. Interviewees especially perceived their own personal attitude, beliefs, and willpower as important for adherence. These personal characteristics were regarded as particularly important for adherence to therapy-related products. The usage of consumer products (eg, an alarm clock) was continued, because the interviewees needed the functionality. The main arguments for not using a consumer product were (1) no perceived need for the product, (2) a perceived lack of effectiveness, and (3) the interviewee forgot to use the product.
Participants' quotes regarding the satisfaction about adherence of sleep products were as follows (translated from Dutch):
It went well. I cannot remember not doing the exercises.
[about doing the exercises everyday] Well, that went ok.
I use it 3 or 4 times a week, depending on my needs.
Participants' quotes regarding the effect of personal attitude, beliefs, and willpower on adherence of sleep products were as follows (translated from Dutch):
I tried to keep myself to it as much as possible, and of course I missed a day now and then, but I tried really hard
You cannot just resign and accept your sleep problem.
I was really motivated, so that makes a difference.
[What dragged you through it?] My will. I intended to do it. I started it and I wanted to get a grip on my sleep problem, so I had to follow through. [So your own determination?] Yes, without discipline you will not succeed.
I felt like, I started it, so I should finish it.
You have to be serious about it. It is a therapy that you really have to complete, otherwise it will not have an effect. So, you have to believe in it.
If you do not recognize the need to change, you should not start it.
Participants' quotes regarding reasons for using and not using sleep products were as follows (translated from Dutch):
You have to set an alarm, anyway.
During the holidays there is no need for an alarm.
I did not have the impression that the app could change my sleeping pattern.
I simply forgot it.
The most obvious emerging themes in the focus groups were
Users being in control
Control increases commitment and motivation
Doing it for own sake
Phrase that was strongly believed in was "I do it for my own sake"
Motivation: three conflicting ideas
If coach is downloaded, then the user is motivated
Downloading does not imply motivated usage
Motivation can arise while using
Adherence-enhancing strategies
Awarding points for progress
Not seen as appropriate for sleep coach; however, awarding points can work against own expectations
Giving compliments
Not too often, not for nonsignificant actions
Should contain context, and vary over time
Providing reminders
Should not be necessary; however, they are practical
Reminders are perceived as positive when set by the users
Provide rationale: two types of people
Type 1: first experience exercise, then explanation
Type 2: first explanation, then perform exercise
I am not the only one
Provide a forum, stories from others, amount of app users, statistics
Potential users, coaches, and sleep experts agreed that the users should be in control. Different arguments were given. The coaches and sleep experts mainly argued that giving the user more control increases commitment and motivation. The potential users argued that they use the sleep coach for their own sake, so they want to be in control themselves. Another argument was that not being in control could lead to irritation. Aspects that participants believed the users should be able to control were the following: reminders, amount of information given by the app, scheduling exercises, decisions about motivation level, sharing therapy progress, sharing the outcome of questionnaires, and parameters shown in sleep diary overview.
The other interesting theme was
The claims underlying the envisioned usage scenarios stated that users should be motivated before they start sleep treatment, otherwise the probability of dropping out would be too high. The focus groups with the sleep experts manifested three different ideas about motivation. Some of the sleep experts argued that people will be motivated at least a little bit when they have downloaded the app, since that requires some effort. On the other hand, it was also argued that someone could show interest in the sleep coach, but he or she would not necessarily be motivated to use the sleep coach. Third, it was argued that motivation could arise during different phases of a therapy; for example, after someone performs an exercise and experiences its effects. In that situation, users would not need to be highly motivated at the beginning of the therapy.
Several adherence-enhancing strategies and ideas to increase motivation were scripted in the scenarios (eg, awarding points, compliments, reminders) and are discussed below.
In general, participants reacted adversely to the idea of awarding points as described in the scenarios, mainly because the sleep coach was seen as a serious program for adults. Furthermore, it was believed that a point system is not appropriate for sleep exercises, but more for workout programs. Nevertheless, a few participants spoke up and said that they liked the idea of points. A few stories came up about how awarded points motivated participants in other domains against their own expectations. Thus, points might improve adherence, despite users’ initial reluctance.
Furthermore, both the coaches and the potential users made negative remarks about the compliments. In principle, both groups thought compliments could enhance a user’s experience, but compliments should not be given too often, or for nonsignificant actions. They argued that compliments should contain context and should vary over time. Otherwise, compliments would not increase motivation.
Reminders were embraced, as long as users are in control of those reminders. The users wanted to set the reminders themselves, because sometimes "you just forget to do something." On the other hand, some users stated they do not need reminders, since they are using the sleep coach for their own sake. Besides that, they argued that they are adults, are motivated, and have self-discipline. Both the coaches and the sleep experts agreed with those potential users and thought that reminders should not be necessary. However, from a practical point of view, they understood that people sometimes do forget to do therapy exercises.
Other ideas to improve motivation mentioned by the participants were as follows: provide a rationale, show statistics, decrease the feeling of being alone, positive feedback, taking small steps, choosing your own coach, demanding a small investment before starting, and showing how much effort users have already invested.
According to the sleep experts, rationales for doing an exercise should be given before users start an exercise. However, the potential users and coaches mentioned there are two types of people: people who want to know how and why things are the way they are, and people who just want to experience an exercise and afterward gain an understanding of that exercise.
Secondly, different ideas were offered to ensure that users do not feel as if they are the only ones suffering from sleep problems. Ideas included a forum (suggested by users and coaches), reading stories from peers (suggested by coaches), and a measure that indicates how many people are using the app (suggested by sleep experts). The idea was that decreasing the feeling of being the only one with sleep problems could increase the motivation of users to adhere to the sleep therapy.
The meta-analysis of adherence rates found a mean experimental compliance of at least 70%, except for the follow-up diaries. Filling out a diary every day for a full week a few months after the intervention requires quite some effort, which might explain a lower adherence rate (58%) to follow-up diaries than to the other experimental compliance measures. The average self-reported treatment adherence was 41%, whereas the average logged adherence was higher at 64%. This is surprising because the self-reported adherence was less "strict" than the logged adherence; for instance, users were categorized as adherent when they reported doing an exercise a certain number of times (eg, more than 4 times a week), while the logged adherence rate was based on doing all exercises. The average treatment adherence rate (logged and self-reported, combined) was 52%. Although self-reports and logs are not exactly the same, they both measure adherence and are similar enough to be combined. Nevertheless, this general adherence rate of 52% should be interpreted carefully.
Furthermore, this meta-analysis confirmed that treatment adherence is positively related to treatment effect when it comes to technology-mediated insomnia treatment. Moreover, this analysis showed that experimental compliance and treatment adherence are not related. In other words, the percentage of participants who filled out questionnaires after the intervention was not found to be an indication of how well people adhered to the treatment. Therefore, it seems important to distinguish between experimental compliance and treatment adherence.
The quality of the individual studies was not assessed using a predefined algorithm, which might be a limitation. However, the included studies were all published in peer-reviewed journals and proceedings, which warrant an acceptable level of quality. Besides, Glass and colleagues argue that all studies should be included [
The aim of the interviews was to gain more insight into the reasons why coachees adhere to technology-mediated sleep products. Surprisingly, interviewees were quite satisfied with their own usage, which departs from the average 52% adherence rate found in the meta-analysis. The reasons why people started using a product were either out of interest or to overcome sleep problems. However, the products’ effectiveness was doubted by the interviewees and was given as a reason to stop using a product. In interviewees’ own opinions, they continued to use consumer products because they needed the functionality, whereas they adhered to therapy-related products because of their own attitudes, beliefs, and willpower. Previous research has also identified functionality as an important determinant for adherence in online sleep treatment [
Furthermore, it seemed challenging for interviewees to identify adherence-enhancing strategies in the products. It was also difficult for them to come up with an answer to the question of what could be added to the product to help them continue to use the product.
Focus groups were organized to discuss adherence-enhancing strategies. In addition to motivation,
The interviews and focus groups both revealed that people strongly believe willpower is an effective adherence strategy. Participants believed that their personal attitudes, beliefs, and motivation would ensure that they stick to their intentions of using a product. This result should be interpreted with caution because of three phenomena. First of all, sleep deprivation increases ego depletion [
Apart from relying on willpower for adherence, aversion to adherence-enhancing strategies emerged during the focus groups. Therefore, when designers implement adherence-enhancing strategies they should not assume that users would initially agree with the usefulness of these strategies.
Various design principles for a virtual sleep coach can be adopted from the interviews and focus groups. The first design principle covers functionality. During the first usage phase, the sleep coach should immediately tickle users’ interest, for example, by providing automatic sleep tracking. In the interviews, it appeared that interest made coachees start using products. Next, the sleep coach can provide an already-needed functionality (eg, an alarm clock). According to the interviews, a needed functionality ensures that users keep using a product. Lastly, reminders need to be a part of the sleep coach. Reminders make sure that users do not simply forget to adhere to the coach. Both the participants in the interviews and focus groups indicated that sometimes they just forget to use a product. Participants in the focus groups showed a positive attitude toward reminders as long as the users were in control over the reminders. Therefore, including reminders in a sleep coach would be a good first step in future research to increase adherence.
A second design principle could be to withhold adherence support at the start of the intervention (ie, to postpone possible help by a virtual sleep coach). In this way, the coachees are acknowledged and respected as serious, motivated, and autonomous adults. Coachees can prove that they adhere to the assignments of the sleep coach; however, the virtual coach can detect when coachees fail to do their assignments, and then offer support. This support can take different forms (reminders, compliments, awarding points, etc) and can be varied over time based on the needs of the coachee.
A third design principle that can be applied is explaining why willpower does not guarantee success. After such an explanation, the understanding of the added value and acceptance of adherence-enhancing strategies might increase. On top of that, users could be given the control over the employment of adherence-enhancing strategies.
In the authors’ opinion, the most important overall design principle is balance. Coachees should not feel overwhelmed with adherence-enhancing strategies, but appreciate some occasional support. Personalization of the virtual sleep coach can ensure that the perfect balance is reached for each and every user. For example, some users might need and appreciate reminders for filling out a sleep diary every day, while other users are more likely to forget to do their relaxation exercises.
Lastly, we want to stress that studies should measure and report treatment adherence, and make a distinction between experimental compliance and treatment adherence. It is important that future studies measure and report adherence rates, since it is only by the adherence measure that it can be established whether the treatment actually induces the observed outcome. The frequently made statement that adherence is important for the outcome of a treatment [
In order to review the quality of our research, it is helpful to know what we did to take care of the credibility, transferability, dependability, and confirmability of our studies [
In conclusion, treatment adherence seems important for the effectiveness of technology-mediated insomnia treatments. Individuals expect that they will adhere well to such treatments and would not gain much from adherence-enhancing strategies. They believe willpower is an effective adherence strategy. The 52% average treatment adherence reported in this paper, however, suggests that there is room for improvement. A virtual coach should be able to cope with this “adherence bias,” and persuade users to accept adherence-enhancing strategies (eg, reminders, compliments, and community building). Future research is needed to test the four derived design principles for a virtual coach, which might help to realize a substantial improvement.
Studies included in the meta-analysis.
Precontemplation scenario.
Claims in Dutch and English.
Short summary of personas.
General notes about the meta-analysis.
Results of the meta-analysis.
American Academy of Sleep Medicine
active sleep sampling
cognitive behavioral therapy
cognitive behavioral therapy for insomnia
computerized cognitive behavioral therapy for insomnia
cognitive therapy
cognitive training
sleep diary
Diagnostic and Statistical Manual of Mental Disorders, 5th Edition
imagery relief therapy (placebo)
Insomnia Severity Index
mind-body bridging
not applicable
Nationaal Initiatief Hersenen en Cognitie
Netherlands Organisation for Scientific Research
personal computer
personal digital assistant
Pittsburgh Sleep Quality Index
questionnaire
relaxation
stimulus control
sleep education
sleep hygiene
self-monitoring minimal treatment
sleep restriction
sleep quality
Dutch Technology Foundation
treatment as usual
television
Unified Theory of Acceptance and Use of Technology
World Health Organization
This research was supported by Philips and the Dutch Technology Foundation (STW), Nationaal Initiatief Hersenen en Cognitie (NIHC) under the Partnership program Healthy Lifestyle Solutions, which is partly financed by the Netherlands Organisation for Scientific Research (NWO).
None declared.