Published on in Vol 17, No 9 (2015): September

Adherence to Technology-Mediated Insomnia Treatment: A Meta-Analysis, Interviews, and Focus Groups

Adherence to Technology-Mediated Insomnia Treatment: A Meta-Analysis, Interviews, and Focus Groups

Adherence to Technology-Mediated Insomnia Treatment: A Meta-Analysis, Interviews, and Focus Groups

Original Paper

1Interactive Intelligence, Delft University of Technology, Delft, Netherlands

2Department of Clinical Psychology, University of Amsterdam, Amsterdam, Netherlands

3Interaction Technology, Utrecht University, Utrecht, Netherlands

Corresponding Author:

Corine Horsch, MSc

Interactive Intelligence

Delft University of Technology

EWI HB, 12th Floor

Mekelweg 4

Delft, 2628 CD


Phone: 31 152784145

Fax:31 152787141


Background: Several technologies have been proposed to support the reduction of insomnia complaints. A user-centered assessment of these technologies could provide insight into underlying factors related to treatment adherence.

Objective: Gaining insight into adherence to technology-mediated insomnia treatment as a solid base for improving those adherence rates by applying adherence-enhancing strategies.

Methods: Adherence to technology-mediated sleep products was studied in three ways. First, a meta-analysis was performed to investigate adherence rates in technology-mediated insomnia therapy. Several databases were queried for technology-mediated insomnia treatments. After inclusion and exclusion steps, data from 18 studies were retrieved and aggregated to find an average adherence rate. Next, 15 semistructured interviews about sleep-support technologies were conducted to investigate perceived adherence. Lastly, several scenarios were written about the usage of a virtual sleep coach that could support adherence rates. The scenarios were discussed in six different focus groups consisting of potential users (n=15), sleep experts (n=7), and coaches (n=9).

Results: From the meta-analysis, average treatment adherence appeared to be approximately 52% (95% CI 43%-61%) for technology-mediated insomnia treatments. This means that, on average, half of the treatment exercises were not executed, suggesting there is a substantial need for adherence and room for improvement in this area. However, the users in the interviews believed they adhered quite well to their sleep products. Users mentioned relying on personal commitment (ie, willpower) for therapy adherence. Participants of the focus groups reconfirmed their belief in the effectiveness of personal commitment, which they regarded as more effective than adherence-enhancing strategies.

Conclusions: Although adherence rates for insomnia interventions indicate extensive room for improvement, users might not consider adherence to be a problem; they believe willpower to be an effective adherence strategy. A virtual coach should be able to cope with this “adherence bias” and persuade users to accept adherence-enhancing strategies, such as reminders, compliments, and community building.

J Med Internet Res 2015;17(9):e214




People who suffer from insomnia have difficulties with initiating sleep, maintaining sleep, or early-morning awakenings, and this sleep disturbance significantly impairs their daily functioning [1]. Having insomnia may lead to personal suffering, such as feeling tired after a night’s sleep, reduced quality of life, and vulnerability to depression [2,3]. In addition, insomnia leads to societal costs that might include reduced productivity and more sick leave from work [2,4]. A review of the literature showed that about 9% to 15% of the western adult population suffers from insomnia symptoms and the daytime consequences thereof [5].

Although the consequences of insomnia may be severe and prevalence is substantial, only a few people seek treatment [6-8]. When help is sought, insomnia is most commonly treated with pharmacotherapy [7]. However, cognitive behavioral therapy for insomnia (CBT-I) is preferable, because CBT-I is equally effective in the short term and has more beneficial long-term effects than pharmacotherapy [9-11]. Generally, CBT-I consists of weekly sessions in which the focus lies on one or more of the following exercises: sleep restriction, stimulus control, relaxation, cognitive strategies, and sleep hygiene [12].

Although CBT-I is effective, there is a lack of knowledge and accessibility regarding this type of therapy [13]. General practitioners are often not aware of the existence of CBT-I, and neither is the general public [13]. In addition, there are too few sleep therapists to help all people with insomnia [14]. In order to increase the availability and accessibility of CBT-I, Espie et al [15] suggested a stepped model with Internet-based treatment as a first option. A meta-analysis about computerized CBT-I (CCBT-I) concluded that this therapy is a moderately effective self-help intervention for insomnia [16]. Nonetheless, adherence to insomnia and other technology-mediated treatments is often mentioned as a serious problem [17-19].

The World Health Organization (WHO) recognizes the importance of adherence to health regimes in general. They stated, “Adherence is a primary determinant of the effectiveness of treatment” [20]. In agreement with the WHO statement, Gould and Clum [21] found—in their meta-analysis of self-help treatments—that better adherence to a treatment improves the treatment effectiveness. They found that the effect size was three times higher for studies that had 75% to 100% adherence than for studies with adherence rates lower than 75%. The impact of adherence on treatment outcomes therefore warrants further investigation into how we could enhance adherence within an intervention in the context of insomnia therapy.

Various authors, for example, Beun [17] and Donkin et al [18], mention that treatment adherence is a problem for cognitive behavioral therapy (CBT) in general. Reports about adherence to various Internet-based interventions show mixed results. For example, Eysenbach [19] gives a few examples in his “law of attrition” of Internet-based interventions with adherence rates ranging from 1% to 35%. Interestingly, a meta-analysis about CCBT-I reported an average adherence rate of 78% for the six studies they included [16]. However, they did not make a distinction between treatment adherence and experimental compliance, that is, the proportion of the experimental assessments, such as questionnaires, that are completed. Thus, decisive conclusions on the exact adherence rates cannot be made.

The studies in this paper are conducted in the context of the Sleepcare project [22,23], which aims at the development of a virtual sleep coach that delivers personalized, automated sleep therapy via a mobile phone. A key challenge of this e-coach is to provide therapy support in such a way that the coachees really adhere to the regimen of the personal therapy plan. In this paper, we use the generic term coachee—instead of client, patient, user, etc—to refer to both patients and nonpatients who seek help to address their health issues. The first step in the development of a virtual sleep coach that meets this adherence challenge is the analysis of current adherence rates, current adherence-enhancing strategies, and coachees’ willingness to accept those strategies. Therefore, we conducted a meta-analysis about adherence rates in technology-mediated sleep interventions; interviewed coachees about their adherence to existing sleep-supporting technology; and discussed adherence-enhancing strategies in a to-be-developed virtual sleep coach among focus groups with potential users, sleep experts, and coaches. This complementary analysis approach provided new insights on how a virtual coach can support coachees to adhere to sleep therapy (ie, the needs and constraints).

Study I: Meta-Analysis Adherence Rates

In order to determine whether a certain outcome is related to a treatment, adherence rates must be measured. Otherwise, it cannot be claimed that the outcome was caused by the intervention [21]. Capturing adherence data is relatively easy in technology-mediated interventions [18]. However, as there is currently no standard adherence measure [18,24,25], various measures are used. A review [18] of adherence in e-therapies found the following adherence measures: number of log-ins, completed modules, number of visits/posts to a forum, pages viewed/printed, and self-reported measures. Other measures that have been suggested are the usage time of the technology [26] and reports by a spouse or related others [24]. Different measures have different advantages and disadvantages. For example, time spent using the technology is an objective measure. However, time spent is presumably influenced by cognitive ability, reading speed, familiarity with the technology, etc [18]. Therefore, time spent does not necessarily represent treatment adherence. Moreover, there is a difference in passively using material (ie, reading, listening, watching) and actively applying this material (ie, performing the exercises) [21].

First, it is important to distinguish between at least two concepts: treatment adherence and experiment compliance. Treatment adherence refers to the extent a coachee processes and applies the content of the treatment (as provided by the coach), whereas experiment compliance refers to the coachees’ completion of the experimental assessments. Other researchers have also made this distinction. For example, Christensen and colleagues [26] respectively use the terms adherence (experience content) and dropout (research trial protocol), whereas Hebert and colleagues [27] respectively call it nonusage attrition and study attrition. Treatment adherence and experiment compliance might be related, but to our knowledge no information about this relationship has been reported in the literature.

Study II: User Adherence to Existing Sleep-Supporting Technology

After analyzing reported adherence rates to technology-mediated sleep treatment in the literature, the next step was to study coachees’ reasons why they do or do not adhere to technology-mediated sleep interventions. To do so, interviews were conducted with people who (had) used a sleep product. The first step was to identify a sample of technology-mediated sleep products. The most familiar sleep product is probably the alarm clock. Besides alarms, there are many other sleep-supporting technologies on the market. For example, relaxation-supporting technologies, sleep-measuring apps and devices, and computerized therapies.

Study III: Focus Group Discussions—The Envisioned Sleep Coach

A limitation of the interviews from Study II, as will be discussed in more detail in the Results section, was that they were restricted to existing products, and did not include reflections on what might technically be possible regarding adherence-enhancing strategies. During the interviews, it also proved to be difficult for participants to think of additional functionality that could improve their adherence. To address the limitations of the interviews, focus groups were organized to discuss adherence-enhancing strategies of a to-be-developed sleep coach. The aim of study III was to gain insight into coachees’ attitudes and beliefs toward these adherence-enhancing strategies, for which focus groups are particularly suited [28].

Study I


The meta-analysis was primarily performed to answer the question "How well do coachees adhere to technology-mediated insomnia interventions and diagnostic tools?" and, secondly, to answer the question "How does adherence relate to treatment outcome?" Various databases were queried—Web of Science, Scopus, PubMed, and PsychINFO—on July 8 and 14, 2014, to find studies that investigate insomnia regimes mediated by technology. The used query was: insomnia and Internet-treatment, Internet-delivered, Internet-based, Internet-administered, Internet intervention, computerize, online treatment, Web application, Web-based, virtual, virtual reality, mass media intervention, smartphone, mobile phone, mobile technology, text message, handheld, or PDA (personal digital assistant). In addition, the references from recent meta-analyses, and systematic reviews on self-help and computerized insomnia therapy [16,29,30] were screened for potentially relevant publications. Together, this resulted in 448 unique papers of which the abstracts were read and examined (by the first author, CH) for meeting the following exclusion criteria: no main focus on insomnia, no technology involved, treatment that does not include assignments at home, no experiment, or targeted at children. Studies on children were excluded because children’s sleep problems often differ from those of adults. Besides, children’s bedtimes are partly controlled by the parents. Therefore, interventions targeted at children have other characteristics than interventions for adults and were excluded. A total of 56 papers were read completely and the inclusion of those papers was discussed between the first and second author (CH and JL).

Figure 1 shows the flow diagram for inclusion and exclusion criteria, resulting in 21 papers from which data was retrieved. Due to a lack of reported adherence data in 3 of the papers, only 18 papers were used in the analysis. The papers selected for this meta-analysis can be found in Multimedia Appendix 1.

Description of Included Studies

Of the 18 included studies in this meta-analysis, 12 studies (67%) focused on CBT-I (Table 1 and Table 2). Oosterhuis and Klip [31] and Rybarczyk and colleagues [32] did include most of the CBT-I exercises in their intervention. Out of the 18 studies, 2 (11%) focused on sleep tracking by using an active sleep sampling device. The active sleep sampling device used by Riley and colleagues [33] mainly supported sleep restriction and stimulus control. The other standard CBT-I exercises were explained in an additional manual. Lawson and colleagues [34] used an active sleep sampling device inspired by Riley’s device. They developed an active sleep sampling mobile phone app which focused on sleep tracking, but did not include the other CBT-I components. Lipschitz and colleagues [35] also developed a mobile phone app, offering sleep-focused, mind-body bridging exercises. The most important assumption of mind-body bridging for sleep is that the mind needs to be rested to sleep well. Haimov and Shatil [36] studied whether providing cognitive training, such as a memory game, affects sleep.

Table 1. Characteristics of included studies.
First authorConditionNumber of peopleNumber of females/malesMean ageSleep problem severity,
measure, score
Oosterhuis [31]Intervention40063% female55N/Aa
Rybarczak [32]Intervention1422/1668PSQIb, 9.5


PSQI, 11.9


PSQI, 9.9
Ström [37]Intervention5471/3844ISId, 18.08

Waiting list55

ISI, 18.11
Suzuki [38]Intervention2116/2540N/A

Waiting list22

Ritterband [39]Intervention2234/10N/AISI, ≥8

Waiting list23

ISI, ≥8
Van Straten [40]Intervention126163/845272% rated SQe<6/10

Waiting list121

68% rated SQ<6/10
Vincent [41]Intervention5979/39N/AN/A

Waiting list59

Riley [33]Intervention 12452/3849ISI, 8-14 (25 people)

Intervention 233

ISI, 15-21 (53 people)


ISI, 22-28 (12 people)
Lancee [42]CCBT-Ig216520/10352Sleep-50, ≥19


Waiting list205

Ritterband [43]Intervention1424/457ISI, 17.1

Waiting list14

ISI, 15.9
Espie [15]Intervention55120/4449Met DSM-5i criteria


Met DSM-5 criteria


Met DSM-5 criteria
Haimov [36]Cognitive training (CogniFit)3429/2272Met AASMl criteria

Active controlm17

Met AASM criteria
Lancee [44]Low depression198316/16347ISI, 16.73

Mild depression182

ISI, 18.63

High depression99

ISI, 20.69

Average ISI, 18.72
Lancee [45]With support129197/6548ISI, 16.95

Without support133

ISI, 17.32
Lawson [34]Intervention3621/534N/A
Van Straten [46]Intervention5983/3549PSQI, 12.4

Waiting list59

PSQI, 11.7
Holmqvist [47]Intervention3955/18N/AISI, 18.72


ISI, 18.50
Lipschitz [35]Intervention3727/1037ISI, 7.24

aNot applicable (N/A)

bPittsburgh Sleep Quality Index (PSQI)

cCognitive behavioral therapy (CBT)

dInsomnia Severity Index (ISI)

eSleep quality (SQ)

fSelf-monitoring minimal treatment (SMMT)

gComputerized cognitive behavioral therapy for insomnia (CCBT-I)

hCognitive behavioral therapy for insomnia (CBT-I)

iDiagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5)

jTreatment as usual (TAU)

kImagery relief therapy (placebo) (IRT)

lAmerican Academy of Sleep Medicine (AASM)

mActive control consisted of word and paint training

Figure 1. Inclusion and exclusion criteria for papers in the meta-analysis.
View this figure
Table 2. Description of included studies.
First authorInterventionDeliveryTreatment
Follow-up lengthPostaFollow
Oosterhuis [31]SEc, SHd, CThe, RXfTVg8 weeks4.5 monthsQhQN/Ai
Rybarczyk [32]RX, SCj, SRk, CTh, SHAudiotape6 weeks4 monthsQ&DlQ&DN/A
Ström [37]CBT-ImInternet5 weeks9 monthsQ&DDN/A
Suzuki [38]CBT-IInternet2 weeks3 weeksQQN/A
Ritterband [39]CBT-IInternet9 weeks6 monthsQ&DQN/A
Van Straten [40]CBT-ITV6 weeksNoneQ&DN/ASelf-report
Vincent [41]CBT-IInternet5 weeks4 weeksQ&DQ&DSelf-report
Riley [33]ASSn/CBT-IDevice6 weeks6 weeksQ&DQ&DN/A
Lancee [42]CBT-IInternet6 weeks4 weeks
Ritterband [43]CBT-IInternet6-9 weeksNoneQ&DN/ALog
Espie [15]CBT-IInternet6 weeks8 weeksN/AQ&DLog
Haimov [36]CTroPCp8 weeksNoneQ&DN/AN/A
Lancee [44]CBT-IInternet6 weeks4 weeksN/AQ&DSelf-report
Lancee [45]CBT-IInternet6 weeks6 monthsQ&DQ&DLog
Lawson [34]ASSAppq7 daysNoneQN/ALog
Van Straten [46]CBT-IInternet6 weeks3 monthsQ&DQ&DLog
Holmqvist [47]CBT-IInternet6 weeks8 weeksQ&DQ&DN/A
Lipschitz [35]MBBrInternet3 days1 weekQQSelf-report

aPostintervention measurement instrument

bFollow-up measurement instrument

cSleep education (SE)

dSleep hygiene (SH)

eCognitive therapy (CTh)

fRelaxation (RX)

gTelevision (TV)

hQuestionnaire (Q)

iNot applicable (N/A)

jStimulus control (SC)

kSleep restriction (SR)

lSleep diary (D)

mCognitive behavioural therapy for insomnia (CBT-I)

nActive sleep sampling (ASS) device

oCognitive training (CTr) (CogniFit)

pPersonal computer (PC)

qMobile phone app (app)

rMind-body bridging (MBB)

Study II

Participant Selection

In order to establish a purposive sample of users across sleep products, various sleep products were categorized. Based on their background knowledge and a media scan, the authors generated a list of 54 technologies over the course of a few months. This composed list was supplemented with apps because the goal of the Sleepcare project is to design a virtual sleep coach on a mobile phone. The first 25 Android apps and 25 iPhone apps found in Google Play and the iTunes store with the search word "sleep" on November 19, 2012, were added to the product list. A total of 7 apps were unrelated to sleep—3 games, 2 hypnosis apps, 1 unlock, 1 music timer—and were therefore discarded, resulting in a list of 97 sleep products. The categorization made in this paper aims to be simple and objective. Sleep products were categorized based on their goal and the medium used. Figure 2 shows the distribution of the products across the two dimensions: goal and medium. The size of the bubbles shows how many products belong to the intersections of the categories.

After identifying the categories of existing sleep products, the next step was to learn more about the users’ usage and adherence to the sleep products. Interviews were conducted with people who used a sleep product in each of the largest product-medium combinations (eg, apps that help people fall asleep).

Figure 2. A graph showing the relationship between the goal of sleep products and the medium used. The size of each bubble indicates how many products of the 97 identified sleep products belong to that category.
View this figure

People registered as participants at the Sleepcare project website [48] were invited to participate in the interviews if they had ever used a technology-mediated sleep product. In addition, two sleep therapists were asked to invite people who used sleep coaching products, as none of the respondents to the call used a sleep coaching product. A total of 15 Dutch persons agreed to be interviewed—6 (40%) females and 9 (60%) males—their ages ranging from 22 to 65 years (mean 37.5, SD 14.8). The mean Pittsburgh Sleep Quality Index (PSQI) [49] score was 8.0 (SD 4.0), with 12 out of the 15 (80%) interviewees having a score above 5, which is the threshold for poor sleep quality classification.


Besides adherence, the interviews covered other topics to gain insight regarding users’ experiences with sleep products. Therefore, the semistructured interviews included both adherence-related questions and questions regarding the factors of the Unified Theory of Acceptance and Use of Technology (UTAUT) model [50]. The described results of the interviews in this paper, however, will only include adherence-related topics. The interviews were conducted in person, by Skype, or by telephone by the first author. The audio of the interviews was recorded. The study was approved by the Human Research Ethics Committee of Delft University of Technology.


The first author (CH) performed the data analysis following the phases of thematic analysis as described by Braun and Clarke [51]. The first author familiarized herself with the data (phase 1) by conducting and transcribing the interviews, and reading the transcripts. While reading, the initial codes were generated bottom-up (phase 2). The first author coded the transcriptions and iteratively generated hierarchical codes and themes (phase 3). Short summaries of the codes and themes related to adherence were written down. The first and last author (CH and WB, respectively) discussed these summaries (phase 4) to form three final adherence-related categories (phase 5). In addition, an independent researcher applied the coding scheme to one of the interviews in order to minimize the threats to confirmability (known as objectivity in quantitative research). The independent coder confirmed the applicability and usefulness of the codes.

Study III


The envisioned coach would use different adherence-enhancing strategies during the entire coaching process. For example, different roles (eg, motivator and educator) could be played by different virtual characters to increase the effect of the to-be-developed sleep coach (ie, split-persona effect) [52]. Around 25 strategies were allocated to the coach ranging from strategies involving others (eg, peers or family members), helping with planning (eg, setting goals and making commitments), and gaming strategies (eg, earning points and taking a quiz). These adherence-enhancing strategies were scripted explicitly in the scenarios in order to discuss them in the focus groups.


A total of 12 scenarios and 72 claims (see Multimedia Appendices 2 and 3) were written to evaluate the adherence-enhancing strategies. Scenarios consisted of stories about people and their activities, goals, and motivations regarding a system [53]. Claims stated important design decisions (eg, about the adherence-enhancing strategies) that needed to be evaluated in the focus group. Furthermore, three fictitious people varying in age, gender, family situation, and readiness-to-change were created to act in the scenarios (see Multimedia Appendix 4 for these personas).

Procedure and Participants

The scenarios and claims were discussed in six focus groups to evaluate the adherence-enhancing strategies. Two groups consisted of potential users, two groups consisted of coaches, and a further two groups consisted of sleep experts. Demographics of the Dutch participants can be found in Table 3. Each session lasted 2 hours and included a general introduction, an introduction round of the participants, and approximately four animated videos that represented the different scenarios. After watching one video, the participants were asked to individually rate their agreement with the claims on a 7-point Likert scale. Subsequently, participants were asked in turn to react to the claims and discuss their ideas. The sessions were videotaped for later analysis. The study was approved by the Human Research Ethics Committee of Delft University of Technology.


The analysis was an iterative process of developing codes and themes in line with thematic analysis [51]. For that, the videotapes of the sessions were transcribed and summarized by the first author (CH). During that recapitulation, several codes emerged and an initial coding scheme of 12 codes was created. The first author coded the summaries according to this scheme. Additionally, a second coder, independent of the project, coded a sample of the summaries—48 of the 86 claims (56%). The second coder suggested eight additional codes. The two coders came together to discuss the coding scheme and agreed on a new scheme of 15 codes. The coding was improved (with this new scheme) by both coders, and within the sample a Cohen’s kappa of .80 was reached. Next, the first author wrote short resumes per theme, making use of quotes.

Table 3. Demographics of the participants per focus group.
Focus groupsParticipants,
n (% female)
Age in years,
mean, (SD)
Number of participants with a PSQIa>5, n (%)Expertise
Potential users 18 (38)35 (12)3 (38)N/Ab
Potential users 27 (71)48 (9)5 (71)N/A
Coaches 14 (75)51 (8)N/A4 coaches (relationships,
lifestyle, didactical)
Coaches 25 (80)50 (6)N/A4 coaches (lifestyle, career),
1 psychologist
Sleep experts 13 (67)50 (18)N/A1 psychologist, 1 therapist,
1 doctor
Sleep experts 24 (75)47 (14)N/A3 researchers,
1 psychologist

aPittsburgh Sleep Quality Index (PSQI).

bNot applicable (N/A).

Study I

All analyses were completed with the Comprehensive Meta-Analysis statistical package, version 3, and were based on the random-effects model. In the analyses, a distinction was made between experimental compliance and treatment adherence. All studies reported experimental compliance, and most of them (10) also reported treatment adherence (see Table 1 and Table 2). Experimental compliance was typically determined based on the completion of questionnaires and sleep diaries that were part of the study protocol; for more information see Multimedia Appendix 5. Immediately after the intervention (ie, postmeasures), the experimental compliance for questionnaires was 78% (95% CI 70%-85%), and for sleep diaries 71% (95% CI 65%-77%). At the follow-up assessments, experimental compliance to questionnaires was 72% (95% CI 69%-76%), while for diaries it was 58% (95% CI 52%-64%). These aggregated numbers are displayed in Figure 3. In Multimedia Appendix 6, individual numbers per study, aggregated rates, heterogeneity statistics, and publication bias tests can be found. Generally, the analyses indicated a substantial heterogeneity in the data, which supports the choice for a random-effects model. The shapes of the funnel plots and the Egger test did not suggest a significant publication bias.

Treatment adherence was reported in various ways, which can roughly be classified into two groups, namely self-reports and logs. Self-reports refer to questions in which participants were asked how well they adhered to the exercises. The five studies that used self-reports found that 41% (95% CI 36%-46%) of the participants met the adherence criteria set in that study. Logs refer to reports that show how many sessions were completed. A total of 5 studies used logs and found that 64% (95% CI 44%-79%) of the participants completed all sessions. If these two kinds of measures are taken together, an average treatment adherence of 52% (95% CI 43%-61%) is reached with reported adherence ranging from 28% [34] to 100% [35] across the 10 studies.

In Figure 3 the compliance and adherence rates and their 95% confidence intervals are shown; as can be seen, the self-reported treatment adherence is significantly different from the experimental compliance rates (nonoverlapping confidence intervals). Furthermore, two meta-regressions were run with studies that reported both experimental compliance and treatment adherence in order to discover a possible relationship between these two measures (experimental compliance and treatment adherence): one meta-regression with experimental compliance to postquestionnaires as the explanatory variable and logged treatment adherence as the outcome variable, and the other meta-regression with experimental compliance to follow-up questionnaires as the explanatory variable and self-reported treatment adherence as the outcome variable. These variables were chosen because most data were available for these combinations of variables. Both analyses did not reveal significant relationships between experimental compliance and treatment adherence (both had P>.05).

Lastly, the relationship between treatment adherence and the effect size of the individual treatments was explored. Multimedia Appendix 6, Figure 4, and Table 4 show the results of the meta-regression analysis. The analysis revealed a significant model (Qmodel=5.05, df=1, P=.03), with a coefficient of 0.79 (Z=2.25, P=.03) for adherence. In other words, treatment adherence and treatment effect are positively correlated. For example, if adherence increases with 0.30 (30%), this would coincide with a 0.24 increase in effect size (Hedge’s g) of the treatment, which is an increase of a small effect size of 0.20. The analysis also found that 75.4% (I2=75.4, Q=48.87, df=12) of the total variance in effect size could be explained by the variation between the studies. Of this 75.4%, 40% (R2=.407, T2total=.059, T2unexplained=.035) could be explained by treatment adherence.

Figure 3. Mean compliance and adherence rates and their 95% CIs. post: posttreatment measurement; follow: follow-up measurement; Qs: questionnaires; Ds: diaries; self-report: self-reported adherence with questions; logs: automatically logged behavior.
View this figure
Table 4. Statistics of the meta-regression of adherence and effect size of the individual treatments.
Statistics meta-regressionCoefficientStandard error95% CIZP (2-sided)
Figure 4. Meta-regression of adherence on effect size of treatments. The circles represent the individual studies [15-46]. The circle size indicates the weight of the study. The effect size is given in standard difference in means.
View this figure

Study II


The three main categories related to adherence are usage, effectiveness, and adherence (see Textbox 1).

Main themes mentioned by participants in the interviews.


  • Intention
  • Two reasons for usage
    • Overcome sleep problems
    • Interest in the product
  • When used
    • Sleep trackers, alarms, relaxation: used in the evenings
    • Sleep coaches: varying usage times


  • Per product type
    • Therapy-related products: no noticeable effect
    • Alarms: ambiguous effect, wake-up is okay, but not waking up better


  • Keep using
    • Therapy-related products: personal attitude
    • Consumer products: need functionality
  • Not using
    • Consumer products
      • No need for functionality (anymore)
      • Product does not work
      • Forget to use product
Textbox 1. Main themes mentioned by participants in the interviews.

Two initial reasons for using a product emerged from the interviews. First, all interviewees used the product to overcome some of their sleeping troubles. Interviewees wanted to wake up better, initiate or maintain sleep, and/or increase insight into their sleep. Second, some interviewees used a product because they thought the product in itself was interesting. Above all, this holds for the sleep-tracking apps. Most products—alarms, automatic sleep trackers, and relaxation support—were used in the evening before going to sleep. Most of those products, however, were not used on the weekend. Sleep coach usage varied, depending on the kind of assignments included in the product (eg, diary, relaxation, sleep hygiene exercise, bedtime scheduling).

Participants' quotes regarding reasons to start using sleep products were as follows (translated from Dutch):

The reason was that in my opinion I was awake too often, and too long. I could not fall asleep anymore.
[Interview #11, online sleep therapy]
Friends of mine had the app and I wanted to try it as well.
[Interview #4, sleep tracking app]

Participants' quotes regarding usage of sleep products were as follows (translated from Dutch):

I turn it on in the evening when I am lying in bed and want to go to sleep
[Interview #1, relaxation app]
Every day I needed to get up I used it, but on the weekends, for example, when I don’t need to get up I didn’t do anything with the app.
[Interview #7, sleep tracking app]
Actually, I did it whenever it suited me [about filling in a sleep diary].
[Interview #10, online sleep therapy]

One of the initial arguments for using a product was to overcome some kind of sleeping problem. However, the online sleep therapies were not perceived as having an effect on the interviewees’ sleep problems. Additionally, interviewees mentioned that it was hard to determine if the therapy improved sleep in the long term because they tried several things. Nevertheless, most interviewees took some advice that worked for them and continued applying it. Furthermore, sleep tracking apps as well as online sleep coaches provided the interviewees with more insight into their sleep and habits. Products that wake interviewees up (ie, smart alarm apps and wake-up lights) were assessed ambiguously. Both types of products did what was expected of them, namely wake the interviewee up. However, the effect of waking up better with the product was doubted. Moreover, smart alarms did not seem to fit into interviewees’ daily lives (see quote below from interview #4, sleep tracking app).

Participants' quotes regarding the effectiveness of sleep products were as follows (translated from Dutch):

The goal to sleep better was not reached
[Interview #12, online sleep intervention]
I have no clue if it helped, because it is going better at the moment, but I did other things in that same time period.
[Interview #10, online sleep therapy]
Also getting out of bed when I am awake for more than 30 minutes. The advice has helped, yes.
[Interview #10, online sleep therapy]
It measures the sleep debt that you are building up, that was effective.
[Interview #6, sleep tracking app]
It did what it supposed to do, wake me up.
[Interview #7, sleep tracking app]
Still not very well, but it became a little bit better, a little bit more pleasant
[Interview #5, sunrise alarm]
Problem of the app [smart alarm] is that you do not know what time you will wake up exactly. If I have an appointment somewhere I need an hour to get ready. If you do not know how late your alarm will go, it is hard to plan.
[Interview #4, sleep tracking app]

In general, interviewees perceived their own usage as sufficient. Interviewees especially perceived their own personal attitude, beliefs, and willpower as important for adherence. These personal characteristics were regarded as particularly important for adherence to therapy-related products. The usage of consumer products (eg, an alarm clock) was continued, because the interviewees needed the functionality. The main arguments for not using a consumer product were (1) no perceived need for the product, (2) a perceived lack of effectiveness, and (3) the interviewee forgot to use the product.

Participants' quotes regarding the satisfaction about adherence of sleep products were as follows (translated from Dutch):

It went well. I cannot remember not doing the exercises.
[Interview #13, online sleep therapy]
[about doing the exercises everyday] Well, that went ok.
[Interview #10, online sleep therapy]
I use it 3 or 4 times a week, depending on my needs.
[Interview #1, relaxation app]

Participants' quotes regarding the effect of personal attitude, beliefs, and willpower on adherence of sleep products were as follows (translated from Dutch):

I tried to keep myself to it as much as possible, and of course I missed a day now and then, but I tried really hard
[Interview #10, online sleep therapy]
You cannot just resign and accept your sleep problem.
[Interview #10, online sleep therapy]
I was really motivated, so that makes a difference.
[Interview #10, online sleep therapy]
[What dragged you through it?] My will. I intended to do it. I started it and I wanted to get a grip on my sleep problem, so I had to follow through. [So your own determination?] Yes, without discipline you will not succeed.
[Interview #11, online sleep therapy]
I felt like, I started it, so I should finish it.
[Interview #12, online sleep intervention]
You have to be serious about it. It is a therapy that you really have to complete, otherwise it will not have an effect. So, you have to believe in it.
[Interview #13, online sleep therapy]
If you do not recognize the need to change, you should not start it.
[Interview #13, online sleep therapy]

Participants' quotes regarding reasons for using and not using sleep products were as follows (translated from Dutch):

You have to set an alarm, anyway.
[Interview #6, sleep tracking app]
During the holidays there is no need for an alarm.
[Interview #5, sunrise alarm]
I did not have the impression that the app could change my sleeping pattern.
[Interview #6, sleep tracking app]
I simply forgot it.
[Interview #4, sleep tracking app]

Study III


The most obvious emerging themes in the focus groups were users in control and doing it for your own sake. In general, participants believed in the personal strengths and willpower of users to adhere to the proposed sleep coach. Furthermore, the adherence-enhancing strategies and motivation were discussed. See Textbox 2 for an overview of the results.

Main themes mentioned in the focus groups with potential users, coaches, and sleep experts.

Users being in control

  • Control increases commitment and motivation

Doing it for own sake

  • Phrase that was strongly believed in was "I do it for my own sake"

Motivation: three conflicting ideas

  • If coach is downloaded, then the user is motivated
  • Downloading does not imply motivated usage
  • Motivation can arise while using

Adherence-enhancing strategies

  • Awarding points for progress
  • Not seen as appropriate for sleep coach; however, awarding points can work against own expectations
  • Giving compliments
    • Not too often, not for nonsignificant actions
    • Should contain context, and vary over time
  • Providing reminders
    • Should not be necessary; however, they are practical
    • Reminders are perceived as positive when set by the users
  • Provide rationale: two types of people
    • Type 1: first experience exercise, then explanation
    • Type 2: first explanation, then perform exercise
  • I am not the only one
    • Provide a forum, stories from others, amount of app users, statistics
Textbox 2. Main themes mentioned in the focus groups with potential users, coaches, and sleep experts.
Users in Control

Potential users, coaches, and sleep experts agreed that the users should be in control. Different arguments were given. The coaches and sleep experts mainly argued that giving the user more control increases commitment and motivation. The potential users argued that they use the sleep coach for their own sake, so they want to be in control themselves. Another argument was that not being in control could lead to irritation. Aspects that participants believed the users should be able to control were the following: reminders, amount of information given by the app, scheduling exercises, decisions about motivation level, sharing therapy progress, sharing the outcome of questionnaires, and parameters shown in sleep diary overview.

Doing it for Your Own Sake

The other interesting theme was doing it for your own sake. In one scenario, there was an example exercise which entails making a list of people who can help you. In general, this exercise met resistance by the potential users. The idea that you have to solve your problems yourself was dominant. Users would feel ashamed to ask for support, and they believed the virtual sleep coach should help them. On the other hand, the coaches stated that thinking about social resources, such as family and peers, could really help people. The coaches mentioned that coachees usually consult a coach exactly because they try to solve their problems themselves, instead of asking their social resources for help. One potential user shared that only informing other people about her sleep problems and therapy already helped her a lot, even without asking for support. Nonetheless, the general mind-set was that people use such a coach for their own sake, and that they are and should be able to take responsibility for their own adherence.


The claims underlying the envisioned usage scenarios stated that users should be motivated before they start sleep treatment, otherwise the probability of dropping out would be too high. The focus groups with the sleep experts manifested three different ideas about motivation. Some of the sleep experts argued that people will be motivated at least a little bit when they have downloaded the app, since that requires some effort. On the other hand, it was also argued that someone could show interest in the sleep coach, but he or she would not necessarily be motivated to use the sleep coach. Third, it was argued that motivation could arise during different phases of a therapy; for example, after someone performs an exercise and experiences its effects. In that situation, users would not need to be highly motivated at the beginning of the therapy.

Adherence-Enhancing Strategies

Several adherence-enhancing strategies and ideas to increase motivation were scripted in the scenarios (eg, awarding points, compliments, reminders) and are discussed below.

Awarding Points for Progress

In general, participants reacted adversely to the idea of awarding points as described in the scenarios, mainly because the sleep coach was seen as a serious program for adults. Furthermore, it was believed that a point system is not appropriate for sleep exercises, but more for workout programs. Nevertheless, a few participants spoke up and said that they liked the idea of points. A few stories came up about how awarded points motivated participants in other domains against their own expectations. Thus, points might improve adherence, despite users’ initial reluctance.


Furthermore, both the coaches and the potential users made negative remarks about the compliments. In principle, both groups thought compliments could enhance a user’s experience, but compliments should not be given too often, or for nonsignificant actions. They argued that compliments should contain context and should vary over time. Otherwise, compliments would not increase motivation.


Reminders were embraced, as long as users are in control of those reminders. The users wanted to set the reminders themselves, because sometimes "you just forget to do something." On the other hand, some users stated they do not need reminders, since they are using the sleep coach for their own sake. Besides that, they argued that they are adults, are motivated, and have self-discipline. Both the coaches and the sleep experts agreed with those potential users and thought that reminders should not be necessary. However, from a practical point of view, they understood that people sometimes do forget to do therapy exercises.

Ideas Generated by the Participants

Other ideas to improve motivation mentioned by the participants were as follows: provide a rationale, show statistics, decrease the feeling of being alone, positive feedback, taking small steps, choosing your own coach, demanding a small investment before starting, and showing how much effort users have already invested.

According to the sleep experts, rationales for doing an exercise should be given before users start an exercise. However, the potential users and coaches mentioned there are two types of people: people who want to know how and why things are the way they are, and people who just want to experience an exercise and afterward gain an understanding of that exercise.

Secondly, different ideas were offered to ensure that users do not feel as if they are the only ones suffering from sleep problems. Ideas included a forum (suggested by users and coaches), reading stories from peers (suggested by coaches), and a measure that indicates how many people are using the app (suggested by sleep experts). The idea was that decreasing the feeling of being the only one with sleep problems could increase the motivation of users to adhere to the sleep therapy.

Study I

The meta-analysis of adherence rates found a mean experimental compliance of at least 70%, except for the follow-up diaries. Filling out a diary every day for a full week a few months after the intervention requires quite some effort, which might explain a lower adherence rate (58%) to follow-up diaries than to the other experimental compliance measures. The average self-reported treatment adherence was 41%, whereas the average logged adherence was higher at 64%. This is surprising because the self-reported adherence was less "strict" than the logged adherence; for instance, users were categorized as adherent when they reported doing an exercise a certain number of times (eg, more than 4 times a week), while the logged adherence rate was based on doing all exercises. The average treatment adherence rate (logged and self-reported, combined) was 52%. Although self-reports and logs are not exactly the same, they both measure adherence and are similar enough to be combined. Nevertheless, this general adherence rate of 52% should be interpreted carefully.

Furthermore, this meta-analysis confirmed that treatment adherence is positively related to treatment effect when it comes to technology-mediated insomnia treatment. Moreover, this analysis showed that experimental compliance and treatment adherence are not related. In other words, the percentage of participants who filled out questionnaires after the intervention was not found to be an indication of how well people adhered to the treatment. Therefore, it seems important to distinguish between experimental compliance and treatment adherence.

The quality of the individual studies was not assessed using a predefined algorithm, which might be a limitation. However, the included studies were all published in peer-reviewed journals and proceedings, which warrant an acceptable level of quality. Besides, Glass and colleagues argue that all studies should be included [54]. According to them, all studies should be reviewed in context with each other regarding the topics at issue, not necessarily regarding the overall quality of each study. Since adherence is the main focus of this paper, instead of examining a possible relationship between general study quality and adherence [54], the methodological differences of measuring adherence were reviewed by differentiating between experimental compliance and treatment adherence, and self-reported and logged adherence.

Study II

The aim of the interviews was to gain more insight into the reasons why coachees adhere to technology-mediated sleep products. Surprisingly, interviewees were quite satisfied with their own usage, which departs from the average 52% adherence rate found in the meta-analysis. The reasons why people started using a product were either out of interest or to overcome sleep problems. However, the products’ effectiveness was doubted by the interviewees and was given as a reason to stop using a product. In interviewees’ own opinions, they continued to use consumer products because they needed the functionality, whereas they adhered to therapy-related products because of their own attitudes, beliefs, and willpower. Previous research has also identified functionality as an important determinant for adherence in online sleep treatment [55]. Reasons for nonadherence were as follows: no need for the functionality, lack of effectiveness, or just forgetfulness.

Furthermore, it seemed challenging for interviewees to identify adherence-enhancing strategies in the products. It was also difficult for them to come up with an answer to the question of what could be added to the product to help them continue to use the product.

Study III

Focus groups were organized to discuss adherence-enhancing strategies. In addition to motivation, users in control and the awareness to do it for your own sake proved to be important for adherence. The focus groups provided insights into the up- and downsides of adherence strategies, such as awarding points, compliments, reminders, and community building.

General Discussion

Positive Attitudes Toward Adherence

The interviews and focus groups both revealed that people strongly believe willpower is an effective adherence strategy. Participants believed that their personal attitudes, beliefs, and motivation would ensure that they stick to their intentions of using a product. This result should be interpreted with caution because of three phenomena. First of all, sleep deprivation increases ego depletion [56]. In other words, when people are tired their willpower decreases and it will become more difficult to adhere to anything, including a virtual sleep coach. Second, the interviewees attributed their adherence to their own commitment and attitude, while nonadherence was attributed to malfunctioning of the product. This result should also be interpreted with caution because this phenomenon is in accordance with the self-serving bias. The self-serving bias states that successes are attributed to internal factors, while failure is attributed to external factors [57]. Therefore, the "good" adherence rates in the interviews were attributed to the interviewees' own willpower. Third, the participants in the focus groups were quite optimistic about their anticipated future adherence. Being optimistic about oneself and the future is one of the most robust biases (optimism bias) in psychology [58,59]. Several explanations for this unrealistic optimism has been offered, for example, ignoring everything that could go wrong [58], putting too much weight on current intentions [60], or having too much faith in willpower for future events [61]. These three phenomena provide reasons for treating participants’ optimism toward adherence with caution.

Aversion to Adherence-Enhancing Strategies

Apart from relying on willpower for adherence, aversion to adherence-enhancing strategies emerged during the focus groups. Therefore, when designers implement adherence-enhancing strategies they should not assume that users would initially agree with the usefulness of these strategies.

Various design principles for a virtual sleep coach can be adopted from the interviews and focus groups. The first design principle covers functionality. During the first usage phase, the sleep coach should immediately tickle users’ interest, for example, by providing automatic sleep tracking. In the interviews, it appeared that interest made coachees start using products. Next, the sleep coach can provide an already-needed functionality (eg, an alarm clock). According to the interviews, a needed functionality ensures that users keep using a product. Lastly, reminders need to be a part of the sleep coach. Reminders make sure that users do not simply forget to adhere to the coach. Both the participants in the interviews and focus groups indicated that sometimes they just forget to use a product. Participants in the focus groups showed a positive attitude toward reminders as long as the users were in control over the reminders. Therefore, including reminders in a sleep coach would be a good first step in future research to increase adherence.

A second design principle could be to withhold adherence support at the start of the intervention (ie, to postpone possible help by a virtual sleep coach). In this way, the coachees are acknowledged and respected as serious, motivated, and autonomous adults. Coachees can prove that they adhere to the assignments of the sleep coach; however, the virtual coach can detect when coachees fail to do their assignments, and then offer support. This support can take different forms (reminders, compliments, awarding points, etc) and can be varied over time based on the needs of the coachee.

A third design principle that can be applied is explaining why willpower does not guarantee success. After such an explanation, the understanding of the added value and acceptance of adherence-enhancing strategies might increase. On top of that, users could be given the control over the employment of adherence-enhancing strategies.

In the authors’ opinion, the most important overall design principle is balance. Coachees should not feel overwhelmed with adherence-enhancing strategies, but appreciate some occasional support. Personalization of the virtual sleep coach can ensure that the perfect balance is reached for each and every user. For example, some users might need and appreciate reminders for filling out a sleep diary every day, while other users are more likely to forget to do their relaxation exercises.

Measuring Adherence

Lastly, we want to stress that studies should measure and report treatment adherence, and make a distinction between experimental compliance and treatment adherence. It is important that future studies measure and report adherence rates, since it is only by the adherence measure that it can be established whether the treatment actually induces the observed outcome. The frequently made statement that adherence is important for the outcome of a treatment [17-19] seems to be supported by the findings of the meta-regression between effectiveness and treatment adherence. As a correlational analysis does not provide insight into the direction of a causal relationship, it remains unclear how effectiveness and adherence influence each other. Nevertheless, if coachees do not follow the treatment protocol (ie, adherence rates close to zero), the outcome could be attributed to other things outside the intervention [21], for example, to the waiting-list effect. Furthermore, it is important to make a distinction between experimental compliance and treatment adherence, since these seem to be two distinct constructs as the meta-analysis found no correlations. An earlier meta-analysis about the effectiveness of CCBT-I found a rather good "adherence rate" of 78% [16]. However, this rate would be considered as experimental compliance according to the definition used in this paper. Similar experimental compliance rates—79%, 72%, 70%, and 57%—were found by the meta-analysis, although treatment adherence was significantly lower. The average self-reported treatment adherence was 42%, whereas the logged treatment adherence was 64%. Although no significant difference between these two measures was found, it is important to consider how adherence is defined and measured. A study [62] that compared a paper diary with an electronic diary found a tremendous difference between self-reported adherence (90.5%) and logged adherence (10.9%) for a paper diary. Lastly, the question remains whether adherence in experimental settings resembles adherence in nonexperimental real-life settings. It could be that adherence rates in experiments are higher than in real-life situations. One possible explanation is the sunk-cost fallacy [63]. To illustrate, experiments demand more from participants regarding (pre-) measurements and participants might therefore be more committed to the intervention. When starting a treatment, they have already invested more time (ie, the sunk cost) compared to patients in nonexperimental settings, and are therefore less likely to drop out.

Research Quality

In order to review the quality of our research, it is helpful to know what we did to take care of the credibility, transferability, dependability, and confirmability of our studies [64]. Firstly, threats to all four concepts were minimalized by utilizing three different research methods—meta-analysis, interviews, and focus groups. Furthermore, the credibility of our findings is also consolidated by data source triangulation—literature, current users, potential users, coaches, and sleep experts. Additionally, honesty from our informants was reinforced by stating there are no right or wrong answers, and by allowing them the possibility to withdraw at any moment. We also had regular debriefing sessions between the executors and supervisors in order to strengthen credibility. The level of transferability to other application fields can only be judged by the readers, since they have the knowledge of these other domains [65]. Furthermore, future work can be done to replicate these findings in other fields. Transferability and dependability assessments are supported by descriptions of the research methods and Multimedia Appendices. Lastly, confirmability was addressed by audit trials and the second coders.


In conclusion, treatment adherence seems important for the effectiveness of technology-mediated insomnia treatments. Individuals expect that they will adhere well to such treatments and would not gain much from adherence-enhancing strategies. They believe willpower is an effective adherence strategy. The 52% average treatment adherence reported in this paper, however, suggests that there is room for improvement. A virtual coach should be able to cope with this “adherence bias,” and persuade users to accept adherence-enhancing strategies (eg, reminders, compliments, and community building). Future research is needed to test the four derived design principles for a virtual coach, which might help to realize a substantial improvement.


This research was supported by Philips and the Dutch Technology Foundation (STW), Nationaal Initiatief Hersenen en Cognitie (NIHC) under the Partnership program Healthy Lifestyle Solutions, which is partly financed by the Netherlands Organisation for Scientific Research (NWO).

Conflicts of Interest

None declared.

Multimedia Appendix 1

Studies included in the meta-analysis.

PDF File (Adobe PDF File), 6KB

Multimedia Appendix 2

Precontemplation scenario.

PDF File (Adobe PDF File), 8KB

Multimedia Appendix 3

Claims in Dutch and English.

PDF File (Adobe PDF File), 50KB

Multimedia Appendix 4

Short summary of personas.

PDF File (Adobe PDF File), 2KB

Multimedia Appendix 5

General notes about the meta-analysis.

PDF File (Adobe PDF File), 4KB

Multimedia Appendix 6

Results of the meta-analysis.

PDF File (Adobe PDF File), 456KB

  1. American Psychiatric Association, editor. Diagnostic and Statistical Manual of Mental Disorders. 5th edition (DSM-5). Arlington, VA: American Psychiatric Association Publishing; 2013.
  2. Rosekind MR, Gregory KB. Insomnia risks and costs: health, safety, and quality of life. Am J Manag Care 2010 Aug;16(8):617-626 [FREE Full text] [Medline]
  3. Baglioni C, Battagliese G, Feige B, Spiegelhalder K, Nissen C, Voderholzer U, et al. Insomnia as a predictor of depression: a meta-analytic evaluation of longitudinal epidemiological studies. J Affect Disord 2011 Dec;135(1-3):10-19. [CrossRef] [Medline]
  4. Daley M, Morin CM, LeBlanc M, Grégoire J, Savard J. The economic burden of insomnia: direct and indirect costs for individuals with insomnia syndrome, insomnia symptoms, and good sleepers. Sleep 2009 Jan;32(1):55-64 [FREE Full text] [Medline]
  5. Ohayon MM. Epidemiology of insomnia: what we know and what we still need to learn. Sleep Med Rev 2002 Apr;6(2):97-111. [Medline]
  6. Ancoli-Israel S, Roth T. Characteristics of insomnia in the United States: results of the 1991 National Sleep Foundation Survey. I. Sleep 1999 May 1;22 Suppl 2:S347-S353. [Medline]
  7. Benca RM. Diagnosis and treatment of chronic insomnia: a review. Psychiatr Serv 2005 Mar;56(3):332-343. [CrossRef] [Medline]
  8. Morin CM, LeBlanc M, Daley M, Gregoire JP, Mérette C. Epidemiology of insomnia: prevalence, self-help treatments, consultations, and determinants of help-seeking behaviors. Sleep Med 2006 Mar;7(2):123-130. [CrossRef] [Medline]
  9. Morin CM, Gaulier B, Barry T, Kowatch RA. Patients' acceptance of psychological and pharmacological therapies for insomnia. Sleep 1992 Aug;15(4):302-305. [Medline]
  10. Vincent N, Lionberg C. Treatment preference and patient satisfaction in chronic insomnia. Sleep 2001 Jun 15;24(4):411-417. [Medline]
  11. Perlis ML, Smith MT, Cacialli DO, Nowakowski S, Orff H. On the comparability of pharmacotherapy and behavior therapy for chronic insomnia. Commentary and implications. J Psychosom Res 2003 Jan;54(1):51-59. [Medline]
  12. Morin CM, Espie CA. Insomnia: A Clinical Guide to Assessment and Treatment. New York, NY: Kluwer Academic/Plenum Publishers; 2003.
  13. Morin CM. Insomnia: Psychological Assessment and Management. New York, NY: Guilford Press; 1993.
  14. Morin CM, Beaulieu-Bonneau S, LeBlanc M, Savard J. Self-help treatment for insomnia: a randomized controlled trial. Sleep 2005 Oct;28(10):1319-1327. [Medline]
  15. Espie CA, Kyle SD, Williams C, Ong JC, Douglas NJ, Hames P, et al. A randomized, placebo-controlled trial of online cognitive behavioral therapy for chronic insomnia disorder delivered via an automated media-rich web application. Sleep 2012 Jun;35(6):769-781 [FREE Full text] [CrossRef] [Medline]
  16. Cheng SK, Dizon J. Computerised cognitive behavioural therapy for insomnia: a systematic review and meta-analysis. Psychother Psychosom 2012;81(4):206-216. [CrossRef] [Medline]
  17. Beun RJ. Persuasive strategies in mobile insomnia therapy: alignment, adaptation, and motivational support. Pers Ubiquit Comput 2012 Jul 8;17(6):1187-1195. [CrossRef]
  18. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]
  19. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
  20. Adherence to Long-Term Therapies: Evidence for Action. Geneva, Switzerland: World Health Organization; 2003.   URL: [accessed 2015-08-30] [WebCite Cache]
  21. Gould RA, Clum GA. A meta-analysis of self-help treatment approaches. Clin Psychol Rev 1993 Jan;13(2):169-186. [CrossRef]
  22. Beun R, Griffioen-Both F, Ahn R, Fitrianie S, Lancee J. Modeling interaction in automated e-coaching: a case from insomnia therapy. In: Proceedings of the Sixth International Conference on Advanced Cognitive Technologies and Applications. 2014 Presented at: Sixth International Conference on Advanced Cognitive Technologies and Applications; May 25-29, 2014; Venice, Italy.
  23. Horsch C, Brinkman W, van Eijk R, Neerincx M. Towards the usage of persuasive strategies in a virtual sleep coach. In: Proceedings of the UK HCI 2012 Workshop on People, Computers & Psychotherapy.: BISL; 2012 Presented at: UK HCI 2012 Workshop on People, Computers & Psychotherapy; September 10, 2012; Birmingham, UK.
  24. Edinger JD, Means MK. Cognitive-behavioral therapy for primary insomnia. Clin Psychol Rev 2005 Jul;25(5):539-558. [CrossRef] [Medline]
  25. Vermeire E, Hearnshaw H, Van Royen P, Denekens J. Patient adherence to treatment: three decades of research. A comprehensive review. J Clin Pharm Ther 2001 Oct;26(5):331-342. [Medline]
  26. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
  27. Hebert EA, Vincent N, Lewycky S, Walsh K. Attrition and adherence in the online treatment of chronic insomnia. Behav Sleep Med 2010;8(3):141-150. [CrossRef] [Medline]
  28. Kuniavsky M. Observing the User Experience: A Practitioner's Guide to User Research. San Francisco, CA: Morgan Kaufmann Publishers; 2003.
  29. van Straten A, Cuijpers P. Self-help therapy for insomnia: a meta-analysis. Sleep Med Rev 2009 Feb;13(1):61-71. [CrossRef] [Medline]
  30. Ho FY, Chung K, Yeung W, Ng TH, Kwan K, Yung K, et al. Self-help cognitive-behavioral therapy for insomnia: a meta-analysis of randomized controlled trials. Sleep Med Rev 2015 Feb;19:17-28. [CrossRef] [Medline]
  31. Oosterhuis A, Klip EC. The treatment of insomnia through mass media, the results of a televised behavioral training programme. Soc Sci Med 1997 Oct;45(8):1223-1229. [Medline]
  32. Rybarczyk B, Lopez M, Benson R, Alsten C, Stepanski E. Efficacy of two behavioral treatment programs for comorbid geriatric insomnia. Psychol Aging 2002 Jun;17(2):288-298. [Medline]
  33. Riley WT, Mihm P, Behar A, Morin CM. A computer device to deliver behavioral interventions for insomnia. Behav Sleep Med 2010;8(1):2-15. [CrossRef] [Medline]
  34. Lawson S, Jamison-Powell S, Garbett A, Linehan C, Kucharczyk E, Verbaan S, et al. Validating a mobile phone application for the everyday, unobtrusive, objective measurement of sleep. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM; 2013 Presented at: SIGCHI Conference on Human Factors in Computing Systems; April 27-May 2, 2013; Paris, France p. 2497-2506.
  35. Lipschitz DL, Landward R, Nakamura Y. An exploratory study of an online mind-body program for poor sleepers in a community sample. Eur J Integr Med 2014 Feb;6(1):48-55. [CrossRef]
  36. Haimov I, Shatil E. Cognitive training improves sleep quality and cognitive function among older adults with insomnia. PLoS One 2013;8(4):e61390 [FREE Full text] [CrossRef] [Medline]
  37. Ström L, Pettersson R, Andersson G. Internet-based treatment for insomnia: a controlled evaluation. J Consult Clin Psychol 2004 Feb;72(1):113-120. [CrossRef] [Medline]
  38. Suzuki E, Tsuchiya M, Hirokawa K, Taniguchi T, Mitsuhashi T, Kawakami N. Evaluation of an internet-based self-help program for better quality of sleep among Japanese workers: a randomized controlled trial. J Occup Health 2008;50(5):387-399 [FREE Full text] [Medline]
  39. Ritterband LM, Thorndike FP, Gonder-Frederick LA, Magee JC, Bailey ET, Saylor DK, et al. Efficacy of an Internet-based behavioral intervention for adults with insomnia. Arch Gen Psychiatry 2009 Jul;66(7):692-698 [FREE Full text] [CrossRef] [Medline]
  40. van Straten A, Cuijpers P, Smit F, Spermon M, Verbeek I. Self-help treatment for insomnia through television and book: a randomized trial. Patient Educ Couns 2009 Jan;74(1):29-34. [CrossRef] [Medline]
  41. Vincent N, Lewycky S. Logging on for better sleep: RCT of the effectiveness of online treatment for insomnia. Sleep 2009 Jun;32(6):807-815 [FREE Full text] [Medline]
  42. Lancee J, van den Bout J, van Straten A, Spoormaker VI. Internet-delivered or mailed self-help treatment for insomnia?: a randomized waiting-list controlled trial. Behav Res Ther 2012 Jan;50(1):22-29. [CrossRef] [Medline]
  43. Ritterband LM, Bailey ET, Thorndike FP, Lord HR, Farrell-Carnahan L, Baum LD. Initial evaluation of an Internet intervention to improve the sleep of cancer survivors with insomnia. Psychooncology 2012 Jul;21(7):695-705 [FREE Full text] [CrossRef] [Medline]
  44. Lancee J, van den Bout J, van Straten A, Spoormaker VI. Baseline depression levels do not affect efficacy of cognitive-behavioral self-help treatment for insomnia. Depress Anxiety 2013 Feb;30(2):149-156. [CrossRef] [Medline]
  45. Lancee J, van den Bout J, Sorbi MJ, van Straten A. Motivational support provided via email improves the effectiveness of internet-delivered self-help treatment for insomnia: a randomized trial. Behav Res Ther 2013 Dec;51(12):797-805. [CrossRef] [Medline]
  46. van Straten A, Emmelkamp J, de Wit J, Lancee J, Andersson G, van Someren EJ, et al. Guided Internet-delivered cognitive behavioural treatment for insomnia: a randomized trial. Psychol Med 2014 May;44(7):1521-1532. [CrossRef] [Medline]
  47. Holmqvist M, Vincent N, Walsh K. Web- vs. telehealth-based delivery of cognitive behavioral therapy for insomnia: a randomized controlled trial. Sleep Med 2014 Feb;15(2):187-195. [CrossRef] [Medline]
  48. Sleepcare.   URL: [accessed 2014-11-17] [WebCite Cache]
  49. Buysse DJ, Ancoli-Israel S, Edinger JD, Lichstein KL, Morin CM. Recommendations for a standard research assessment of insomnia. Sleep 2006 Sep;29(9):1155-1173. [Medline]
  50. Venkatesh V, Morris M, Davis G, Davis F. User acceptance of information technology: toward a unified view. MIS Quarterly 2003 Sep;27(3):425-478. [CrossRef]
  51. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
  52. Baylor A, Ebbers S. The pedagogical agent split-persona effect: when two agents are better than one. In: Proceedings of EdMedia: World Conference on Educational Media and Technology. Waynesville, NC: Association for the Advancement of Computing in Education (AACE); 2003 Presented at: EdMedia: World Conference on Educational Media and Technology; 2003; Honolulu, HI p. 459-462.
  53. Carroll J. Five reasons for scenario-based design. Interact Comput 2000 Sep;13(1):43-60. [CrossRef]
  54. Glass GV, McGaw B, Smith ML. Meta-Analysis in Social Research. Beverly Hills, CA: Sage Publications; 1981.
  55. Middlemass J, Davy Z, Cavanagh K, Linehan C, Morgan K, Lawson S, et al. Integrating online communities and social networks with computerised treatment for insomnia: a qualitative study. Br J Gen Pract 2012 Dec;62(605):e840-e850 [FREE Full text] [CrossRef] [Medline]
  56. Barnes CM, Schaubroeck J, Huth M, Ghumman S. Lack of sleep and unethical conduct. Organ Behav Hum Decis Process 2011 Jul;115(2):169-180. [CrossRef]
  57. Zuckerman M. Attribution of success and failure revisited, or: The motivational bias is alive and well in attribution theory. J Pers 1979 Jun;47(2):245-287. [CrossRef]
  58. Buehler R, Griffin D, Ross M. Exploring the "planning fallacy": Why people underestimate their task completion times. J Pers Soc Psychol 1994;67(3):366-381. [CrossRef]
  59. Weinstein ND. Unrealistic optimism about future life events. J Pers Soc Psychol 1980;39(5):806-820. [CrossRef]
  60. Koehler DJ, Poon CS. Self-predictions overweight strength of current intentions. J Exp Soc Psychol 2006 Jul;42(4):517-524. [CrossRef]
  61. Helzer EG, Gilovich T. Whatever is willed will be: a temporal asymmetry in attributions to will. Pers Soc Psychol Bull 2012 Oct;38(10):1235-1246. [CrossRef] [Medline]
  62. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient compliance with paper and electronic diaries. Control Clin Trials 2003 Apr;24(2):182-199. [Medline]
  63. Arkes HR, Blumer C. The psychology of sunk cost. Organ Behav Hum Decis Process 1985 Feb;35(1):124-140. [CrossRef]
  64. Lincoln YS, Guba EG. Naturalistic Inquiry. Beverly Hills, CA: Sage Publications; 1985.
  65. Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Education for Information 2004;22(2):63-75.

AASM: American Academy of Sleep Medicine
ASS: active sleep sampling
CBT: cognitive behavioral therapy
CBT-I: cognitive behavioral therapy for insomnia
CCBT-I: computerized cognitive behavioral therapy for insomnia
CTh: cognitive therapy
CTr: cognitive training
D: sleep diary
DSM-5: Diagnostic and Statistical Manual of Mental Disorders, 5th Edition
IRT: imagery relief therapy (placebo)
ISI: Insomnia Severity Index
MBB: mind-body bridging
N/A: not applicable
NIHC: Nationaal Initiatief Hersenen en Cognitie
NWO: Netherlands Organisation for Scientific Research
PC: personal computer
PDA: personal digital assistant
PSQI: Pittsburgh Sleep Quality Index
Q: questionnaire
RX: relaxation
SC: stimulus control
SE: sleep education
SH: sleep hygiene
SMMT: self-monitoring minimal treatment
SR: sleep restriction
SQ: sleep quality
STW: Dutch Technology Foundation
TAU: treatment as usual
TV: television
UTAUT: Unified Theory of Acceptance and Use of Technology
WHO: World Health Organization

Edited by G Eysenbach; submitted 06.12.14; peer-reviewed by J Dizon, N Siriwardena; comments to author 21.04.15; revised version received 03.06.15; accepted 24.06.15; published 04.09.15


©Corine Horsch, Jaap Lancee, Robbert Jan Beun, Mark A Neerincx, Willem-Paul Brinkman. Originally published in the Journal of Medical Internet Research (, 04.09.2015.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.