Published on 05.12.11 in Vol 13, No 4 (2011): Oct-Dec
Effects of eHealth Interventions on Medication Adherence: A Systematic Review of the Literature
Background: Since medication nonadherence is considered to be an important health risk, numerous interventions to improve adherence have been developed. During the past decade, the use of Internet-based interventions to improve medication adherence has increased rapidly. Internet interventions have the potential advantage of tailoring the interventions to the needs and situation of the patient.
Objective: The main aim of this systematic review was to investigate which tailored Internet interventions are effective in improving medication adherence.
Methods: We undertook comprehensive literature searches in PubMed, PsycINFO, EMBASE, CINAHL, and Communication Abstracts, following the guidelines of the Cochrane Collaboration. The methodological quality of the randomized controlled trials and clinical controlled trials and methods for measuring adherence were independently reviewed by two researchers.
Results: A total of 13 studies met the inclusion criteria. All included Internet interventions clearly used moderately or highly sophisticated computer-tailored methods. Data synthesis revealed that there is evidence for the effectiveness of Internet interventions in improving medication adherence: 5 studies (3 high-quality studies and 2 low-quality studies) showed a significant effect on adherence; 6 other studies (4 high-quality studies and 2 low-quality studies) reported a moderate effect on adherence; and 2 studies (1 high-quality study and 1 low-quality study) showed no effect on patients’ adherence. However, most studies used self-reported measurements to assess adherence, which is generally perceived as a low-quality measurement. In addition, we did not find a clear relationship between the quality of the studies or the level of sophistication of message tailoring and the effectiveness of the intervention. This might be explained by the great difference in study designs and the way of measuring adherence, which makes results difficult to compare. There was also large variation in the measured interval between baseline and follow-up measurements.
Conclusion: This review shows promising results on the effectiveness of Internet interventions to enhance patients’ adherence to prescribed long-term medications. Although there is evidence according to the data synthesis, the results must be interpreted with caution due to low-quality adherence measurements. Future studies using high-quality measurements to assess medication adherence are recommended to establish more robust evidence for the effectiveness of eHealth interventions on medication adherence.
J Med Internet Res 2011;13(4):e103
Recent reports of the World Health Organization and the National Institute for Health and Clinical Excellence reveal that 30%–50% of patients with chronic illnesses do not adhere to prescribed medication . Other studies also show that rates of nonadherence are very high and depend on the type of disease. The highest adherence rates are found for patients with human immunodeficiency virus infection, while diabetes patients have the lowest rate [ ]. As such, medication nonadherence can be considered an important health care problem. This is especially true for patients with a chronic illness because medication adherence is a crucial factor in the effectiveness of a therapy [ ]. Consequently, many patient-centered interventions are developed to improve adherence, and the impact of the Internet in the development of these interventions is increasing. It is therefore important to understand how these interventions work and to know whether they are effective in improving adherence. To our knowledge, no recent review has studied the effectiveness of patient-centered Internet interventions on patients’ medication adherence. Therefore, we conducted a systematic literature study in which we reviewed evidence from studies on Internet interventions that were developed to assist patients in their medication management. The purpose of our study was fourfold: first, to gain insight into the current stage of development of these interventions; second, to assess the included studies for their effectiveness on medication adherence; third, to investigate to what degree adherence is determined by the characteristics of the intervention; and fourth, to investigate whether there is a relationship between the characteristics of the study and the reported effectiveness of the interventions.
Different terms are used in the literature to describe the concept of adherence—for example, compliance, adherence, and persistence. They have all been used to indicate that the patient is using the medication following the prescribed regimen. These terms differ in exact meaning. In this paper, we use the term adherence. Adherence is defined as the extent to which the patient’s behavior matches the agreed recommendations of the prescriber . According to this definition, nonadherence is a wide concept that varies from missing an occasional dose to never taking the prescribed medications [ ]. Patients have different reasons for being nonadherent. These different reasons have something in common: the patient does not execute the treatment plan and does not persist. Execution is a continuous process where the actual dosing history corresponds to the ideal doses [ , ].
To improve adherence and develop target interventions, it is important to address the specific reasons why a patient is not able or willing to execute the treatment plan. From this perspective, interventions should be personalized or tailored to address individual needs and beliefs. The definition of tailoring describes the features that make tailored health messages different from other approaches: “It is assessment-based and as a result the message can be individual-focused” . In other words, tailoring is based on gathering and assessing personal data related to health outcomes or several determinants in order to determine the most effective strategy to meet that person’s needs [ ]. With these characteristics, a tailored message is able to provide personal feedback, commands greater attention, is processed more deeply, and is perceived as more likable by patients than a general message [ , ]. Because of these possibilities, tailored health messages are also more likely than generic information to be read, remembered, and viewed as personally relevant [ , ].
Computer technologies can be used to tailor health messages to the personal situation of the patient and might therefore contribute significantly to the development of tailored message strategies. The Internet is potentially a powerful medium for delivering those tailored messages. The management of a chronic disease should be personalized to an individual because the person is ultimately responsible for the success of the intervention . The technology provides an opportunity to tailor the information in several formats and modalities, which enhances the user’s experience of the material and will result in a better understanding [ , ]. Moreover, Internet interventions have the advantage that they can provide interactive and responsive programs [ ]. These interventions can provide effective data and information provision and retrieval. The advantages of tailored message strategies can contribute to the incorporation of interactive and continued self-monitoring, feedback, and information exchange, which play an increasingly important role in changing patients’ behavior.
For this review, we used the guidelines of the Cochrane Collaboration to assess the studies on their internal validity and to summarize the existing evidence about Internet interventions to improve medication adherence in patients. The Cochrane Collaboration method is described in more detail in the Cochrane Handbook for Systematic Reviews of Interventions .
We included a study when the following inclusion criteria were met: (1) the study described a patient-centered Internet intervention, (2) the study described an intervention for patients who use prescribed medication for a chronic condition, (3) at least one of the outcome measures was adherence, (4) the study was quantitative, and (5) the study was published in either the English or Dutch language.
We conducted a systematic literature search to identify articles containing information about the effect of Internet interventions on medication adherence. Comprehensive literature searches were undertaken in the databases PubMed, PsycINFO, EMBASE, CINAHL, and Communication Abstracts.
The search strategies used the following keywords: (medication therapy management OR medication adherence OR patient compliance OR self-care) AND (internet) AND (intervention study OR randomized controlled trial OR clinical controlled trial).
We then continued with the snowball method by looking for references in publications, especially those of the included studies and reviews on interventions to promote adherence. The search was conducted in June 2010. Since Internet intervention is a relatively new topic, no time limits were applied.
Application of the search strategy to the specified databases resulted in a total of 620 hits (). In total, we selected 13 studies from these results.
Reference Manager version 11.0 (Thomson Reuters, New York, NY, USA) was used to manage the citations. Duplications were logged, leaving 520 unique results (see). On the basis of title and abstract, two researchers (pairs of AL, MV, LvD, JvW) independently selected studies for inclusion. If the study seemed to meet the inclusion criteria or if there were doubts about the inclusion, the full text of the article was obtained. Based on the full articles two reviewers independently reviewed whether these studies fit all the inclusion criteria. Disagreements were solved by discussions between the two researchers. For a more detailed description of the excluded studies see .
Assessment of Methodological Quality
The methodological quality of included randomized controlled trials (RCTs) and clinical controlled trials (CCTs) was independently reviewed by two researchers (AL and JvW) using the list from the Cochrane Collaboration Back Review Group . The list consists of 11 criteria for internal validity, namely:
- 3 criteria regarding selection bias: whether (a) randomization was adequate, (b) treatment allocation was concealed, and (c) groups were similar at baseline regarding the most important indicators,
- 4 criteria for performance bias: whether (d) patients were blinded to the intervention, (e) care provider was blinded to the intervention, (g) co-interventions were avoided, and (h) compliance with the intervention was acceptable,
- 2 criteria regarding attrition bias: whether (i) the dropout rate after baseline was acceptable, and (k) the analysis included an intention-to-treat analysis, and
- 2 criteria for detection bias: whether (f) the outcome assessor was blinded to the intervention, and (j) outcome assessments in all groups were similar.
For each included study, all criteria were scored as “yes,” “no,” or “unclear.” All unclear scores were later rated as “no.” Studies were rated as high quality (HQ) when at least 6 of the 11 criteria for internal validity were met. Otherwise, studies were considered of low quality (LQ). Disagreements were discussed until consensus was reached. If disagreement or indistinctness persisted a third reviewer (LvD) was consulted.
In addition, two researchers (AL and LvD) independently assessed the quality of the methods for measuring adherence to a medical regimen. A standard method to assess adherence does not exist and every method has its limits [, ]. In clinical trials, adherence can be measured based on, for example, interviews, diary, questionnaire-based self-reporting, prescription refills, pill counts, or electronic monitoring [ , ]. We categorized the measurements in high- and low-quality adherence assessments based on previous findings concerning the objectivity of these measurements [ - ]. In this review electronic monitoring and physiological/biomedical measures are defined as high-quality adherence assessment. These measurements are considered the most objective standard [ ]. Previous research has shown that data from pill counts and electronic monitoring are strongly correlated [ ]. Yet others consider pill counts not to be accurate [ , ]. In addition, meta-analyses have shown that self-reported adherence is also strongly correlated with electronic monitoring [ , ]. Like pill counts, the accuracy of self-reported measurements is debatable. Some argue that self-reports may be an accurate measurement for measuring adherence [ , ], while others state that the use of self-reported measurements is not an accurate method [ , , , ]. Taking all arguments into account, we considered self-measurements, such as questionnaires, pill counts, prescription refills, interviews, and diaries, to be most subjective for measuring adherence [ ]. We therefore considered these measurements low-quality adherence assessment. However, if two or more different low-quality adherence measurements were used in the same study, such as a combination of questionnaires and prescription refills, the method was considered high-quality adherence assessment.
One researcher (AL) documented the following characteristics of the included studies: (1) method (type of study), (2) participants (total number of participants, sex per group, mean age per group, type of disease), (3) intervention (name of experimental intervention, name of control condition, period, number of times/minutes per week), (4) outcome measures (type of outcome measures, time of measurement), (5) results (short description), and (6) author’s conclusion.
Due to diversity in the features of the interventions and the methods used to measure adherence, it was not possible to pool the data. Therefore, we conducted a best-evidence synthesis (see) based on [ ] and adapted by a Dutch study [ ].
The best-evidence synthesis was conducted by attributing various levels of evidence to the effectiveness of the interventions. The synthesis takes into account the design, methodological quality, and outcomes of the studies.shows that at least 1 HQ RCT or 2 HQ CCTs were needed to establish robust evidence for the effectiveness of an intervention.
We conducted a sensitivity analysis to indentify how sensitive the results of the best-evidence syntheses were to changes in the way the study quality was assessed. For the sensitivity analysis, the best-evidence synthesis was repeated in two different ways, using the following principles: (1) LQ studies were excluded, (2) studies were rated as HQ if they met at least 4 of the 11 criteria of internal validity instead of 6.
We then compared the results of the sensitivity analysis with the results of the best-evidence synthesis and described the sensitivity of the results [, ].
Study effectiveness was categorized as significant effect on adherence, moderate effect on adherence, and no effect on adherence. We defined a study effect as moderate if the authors reported a positive effect of the intervention on adherence but there were limitations, such as the following: improvement of adherence was found only in a subgroup of the intervention group; adherence was measured indirectly (eg, the study drew conclusions about the use of beta-agonist indicating that adherence was improved); or the significance of the results to medication adherence was not tested, but the authors used convincing arguments to explain the effectiveness of the intervention (see results section for explanation per study).
Intervention: Tailoring Level of Sophistication of the Website
Tailored Internet interventions differ in how they deliver their message . The difference is based on the sophistication of the way the message is tailored. We categorized the interventions in being low, moderate, or high in sophistication. Some interventions involve a form of online assessments (low sophistication), and others use online assessments, tailored feedback, and content matching (moderate sophistication). The third group of interventions provides instant feedback and a complex tailored health program with several tools and activities that would enable patients to achieve their health goals (high sophistication) (see ) [ ].
The main characteristics of the included studies are presented in[ - ] and further described below (for a more detailed description of the included studies see ).
|Study; method||Interventiona||Participants; sex; mean age||Adherence measurement; |
timing of measuring
|Artinian ; RCTb||Web-based monitoring system; tailored content; nature of expert/therapist contact||N = 18 (17 males; mean age 68 years); intervention group n = 9, control group n = 9||Pill counts; baseline and 3 months||Medication compliance rate was 94% for the monitor group as measured by the monitor system|
|Jan ; RCTb||Blue Angel for Asthma Kids variability; tailored content; nature of expert/therapist contact||N = 164, intervention group n = 88 (35 males; mean age 10.9 years); control group n = 76 (28 males; mean age 9.9 years)||Self-reported at baseline and 12 weeks||The Blue Angel for Asthma Kids has the potential for improving asthma outcome compared with conventional treatment over a period of 12 weeks|
|Chan ; RCTb||Customized educational and monitoring Web site; tailored content; nature of expert/therapist contact||N = 120; intervention group n = 60 (37 males; mean age 10.2 years); control group n = 60 (38 males; mean age 9.0 years)||Computerized prescription refill record at baseline, 26 weeks, and 52 weeks||No difference in adherence between groups|
|Chan ; RCTb||Customized educational and monitoring Web site; tailored content; nature of expert/therapist contact||N = 10; intervention group n = 5 (1 male; mean age 6.6 years); control group n = 5 (4 males; mean age 8.7 years)||Self-reported asthma diary and computerized prescription refill record at 90 days and 180 days||After the intervention, the use of beta-agonist decreased, which is an indication of better adherence|
|Joseph ; RCTb||Web-based asthma management program; tailored content; user control||N = 314 (36.6% male; mean age 15.3 years); intervention group n = 162; control group n = 52||Self-reported at baseline and 12 months||Positive changes in controller medication adherence were seen|
|Ross ; RCTb||SPPARO (System Providing Access to Records Online); tailored content; nature of expert/therapist contact||N = 104; intervention group n = 54 (80% male; mean age 57 years); control group n = 50 (74% male; mean age 55 years)||Self-reported at baseline, 6 months, and 12 months||Providing patients access to an online medical record improved adherence|
|Cherry ; prospective design||Telemedicine diabetes disease management program; tailored content; nature of expert/therapist contact||Intervention group n = 169 (39 males; mean age 53 years); historical group (usual care)||Self-reported||Outcomes offer encouraging evidence that telemedicine technology coupled with daily remote monitoring may improve appropriate use of health care services|
|Guendelman ; RCTb||Health Buddy, an interactive device connected to a home telephone; tailored content; nature of expert/therapist contact||N = 134; intervention group (40 males; mean age 12.2 years); control group (37 males; mean age 12.0 years)||Self-reported at baselines, 6 weeks, and 12 weeks||Patients were more likely to take their asthma medication without additional reminders|
|DeVito Dabbs ; RCTb||Pocket Personal Assistant for Tracking Health (PATH); tailored content; nature of expert/therapist contact||N = 30; intervention group n = 15 (60% male; mean age 55 years); control group n = 15 (60% male; mean age 57 years)||Self-reported at baseline and 2 months||Patients who received the PATH were more likely to show high adherence to the medical regimen|
|Van der Meer ; RCTb||Internet-based self-management program; customized health program; user control||N = 200; intervention group n = 101 (29% male; mean age 36 years); control group n = 99 (29% male; mean age 37 years)||Self-reported at baseline, 3 months, and 6 months||After 3 months asthma control improved|
|Van der Meer ; RCTb||Internet-based self-management program; customized health program; user control||N = 200; intervention group n = 111 (28 males; mean age 36 years); control group n = 89 (28 males; mean age 36.6 years)||Self-reported at baseline, 3 months, and 1 year||Weekly self-monitoring leads to improved asthma control in patients with partly and uncontrolled asthma at baseline and tailors asthma medication to individual patients’ needs|
|Dilorio ; Survey||WebEase; customized health program; user control||N = 35 (40% male; mean age 37.5 years)||Self-reported at baseline and 6 weeks||Participants showed some improvement in adherence following the program|
|Dew ; prospective design||Website including skills workshops, discussion group, ask an expert, question and answer, health tips, recourses, and references; customized health program; nature of expert/therapist contact||N = 64; intervention group n = 24 (18 males; mean age 45.8 years); control group n = 40 (30 males; mean age 57.5 years)||Self-reported at baseline and 4 months||The intervention appeared to be weakly associated with medical compliance change|
a Sophistication of tailoring classification based on.
b Randomized controlled trial.
Methodological Quality: Assessment of Internal Validity
For this review 10 RCTs were included, and 9 of them were assessed on their internal validity ([ - ]). We included 1 RCT with no data on our primary outcome variable (ie, medication adherence) for the control group [ ]. This means that for this tenth study, we could not assess validity criteria. Moreover, we reviewed 2 prospective cohort designs and 1 survey. A total of 7 RCTs met 6 or more of the 11 validity criteria and therefore qualified as HQ studies.
|Study||Validity criteriaa met||Study qualityb||Quality measurement |
|Randomized clinical trials|
|Artinian ||Not applicablec||Low||Low|
|Jan ||a, b, c, d, i, j||High||Low|
|Chan ||a, b, f, i, j||Low||Low|
|Chan ||a, b, c, f, hd, i, j||High||High|
|Joseph ||a, b, c, d, e, h, i, j||High||Low|
|Ross ||a, b, c, d, e, he, i, j, k||High||Low|
|Guendelman ||a, b, d, i, j||Low||Low|
|DeVito Dabbs ||a, b, c, e, i, j||High||Low|
|Van der Meer ||a, b, c, i, j, k||High||Low|
|Van der Meer ||a, c, hf, i, j, k||High||Low|
|Prospective design/clinical trial or cohort design|
a a: randomization adequate; b: treatment allocation concealed; c: groups similar at baseline regarding most important indicators; d: patients blinded to intervention; e: care provider blinded to intervention; f: outcome assessor blinded to intervention; g: co-interventions avoided; h: compliance with intervention acceptable; i: dropout rate after baseline acceptable; j: outcome assessed similarly in all groups; k: intention-to-treat analysis included.
b That is, 6 of 11 validity criteria were met.
c No data on medication adherence for the control group and therefore judged as low quality.
d Compliance was acceptable in the first interval (<90 days).
e Compliance was acceptable in the first interval (6 months).
f Compliance was acceptable in the first interval (3 months).
Intervention: Tailoring Level of Sophistication of the Website
All Internet interventions reported computer-tailoring methods. Interventions were categorized as having low sophistication (online assessments), moderate sophistication (online assessments, tailored feedback, and content matching), and high sophistication (a more complex tailored health program) (see) [ ].
Online Assessment and Feedback
Online assessment and feedback are used in interventions with single-incident computer-assisted risk or health assessments. For example, feedback is emailed to the patients or provided online . In addition, these interventions are brief and usually done once at the beginning of the intervention. None of the reviewed studies used online assessment and feedback.
With tailored content a program provides (1) tailored text messages composed in a unique way according to how patients respond to certain questions, or (2) restricted access to content sections per patient . Tailored content was used by 9 of the studies we reviewed; 1 study [ ], Blue Angel for Asthma Kids, conducted an Internet-based interactive asthma educational and monitoring program in which patients were able to complete an electronic diary, record symptoms and need for rescue medication, and upload their videos when they were using their inhaler. Based on these outcomes, the program comprised both an action plan with a warning system and a written treatment plan. A similar customized educational and monitoring Website for patients with asthma was conducted in 2 studies [ , ], and 1 study [ ] tested the asthma management program Puff City. The program used tailoring to alter behavior through individualized health messages based on the patients’ beliefs, attitudes, and personal barriers to change or maintain the behavior. Another form of a tailored website, System Providing Access to Records Online (SPPARO), was examined in 1 study [ ]. This website provided the medical record, an educational guide, and a message system for patients. Moreover, patients could contact the health provider by email. The telemedicine diabetes disease management program Health Buddy was tested in 2 studies [ , ]. However, the modality used in the diabetes program was different from the previously mentioned monitoring programs. Patients answered personalized questions that enabled them to monitor their disease symptoms, medication adherence, and disease knowledge by pressing buttons for response. The 2 studies using the Health Buddy differed in the intensity of the feedback. One study [ ] tested a medication compliance device. Data and answers to questions were recorded by the device and uploaded daily to a central server. Based on these answers health providers were able to monitor the patients, provide advice, and update the treatment regimens in the Med-eMonitor devices. One study [ ] also tested a handheld device, Pocket Personal Assistant for Tracking Health (PATH), developed for patients after lung transplantation to record health data, review data trends, and report their condition changes to the transplant team. The device included decision-support programs to promote self-care behaviors.
Customized Health Programs
Interventions that provide not only tailored content but also individualized instructions for meeting certain health goals, self-management goals, or goal-setting activities are so-called customized health programs , used by 4 of the included studies. Of these, 2 studies [ , ] tested the effects of an Internet-based self-management program for asthma patients. This website allowed monitoring through the website, text messages, use of an Internet-based treatment plan, online education, and the possibility to communicate with the health provider. The intervention WebEase [ ] consisted of three modules that were designed to assess an individual’s status related to self-management practices and create a plan for change or to maintain the behavior. The modules in WebEase required the patient to answer questions related to these topics. Feedback was provided based on these responses. Patients entered data into MyLog, which is a screen for recording data about medication-taking behavior, stress, etc. In addition, the intervention included a knowledge component and a discussion board. This means that each patient was directed to another path [ ]. Another study [ ] tested a customized health program where patients chose which components of the website they wanted to use. The website included a home page, posttransplant skills workshops, discussion groups, “ask an expert,” question-and-answer possibility, healthy-living tips, resources, and reference library. The way the patients used the website was based more on voluntary participation than in the study that used MyLog [ ].
and show for each study which method for delivering the tailored message was used (see column 2).
|Study||Study quality||Sophistication of tailoring||Quality measurement |
|Short-term effectiveness |
|DeVito Dabbs ||High||Moderate||Low||++|
a ++ = significant effect on medication adherence; + = moderate effect on medication adherence; – = no effect on medication adherence.
|Study||Study quality||Sophistication of tailoring||Quality measurement |
|Long-term effectiveness |
|Van der Meer ||High||High||Low||+|
|Van der Meer ||High||High||Low||+|
a ++ = significant effect on medication adherence; + = moderate effect on medication adherence; – = no effect on medication adherence.
Role of Health Providers
Interventions also differ in the type and extent of health provider involvement. User control allows individuals to take a major role in managing their own care, whereas in expert control an expert or therapist takes a more directive role .
Only 4 of the interventions were based on user control and 9 interventions used contact with the health provider. A Web-based asthma management program was developed in 1 study . The program used tailoring to alter behavior through individualized health messages based on the user’s beliefs, attitudes, and personal barriers to change. The health provider did not interfere. Three studies were also user based with treatment algorithms to give feedback [ - ].
In contrast, in the Blue Angel for Asthma Kids, a customized educational and monitoring website site providing secure email contact between patients and their therapist, the therapist had a more directive role . Like the Blue Angel for Asthma Kids, SPPARO included a messaging system that made it possible to exchange secure messages with the health provider [ ]. The intervention developed in 2 studies [ , ] consisted of a case manager who reviewed the data, sent emails about the peak flow, inhaler technique, and symptoms, and forwarded them the website. Patients (the virtual group as well as the office-based group) had access to their case manager 24 hours a day, 7 days a week. One study [ ] conducted a website including skills workshops, discussion group, ask an expert, question and answer, health tips, recourses, and references. In this case, the role of health providers was to provide the possibility for a patient to ask an expert.
Several interventions used a special device to exchange information between patients and health care providers. The Med-eMonitor recorded data and answers to questions that could be uploaded by the health provider. Based on these outcomes, the health provider provided advice and updated the treatment regimens . The PATH [ ] recorded data and provided a tailored decision-support program and email contact with the health provider. The Health Buddy device in 2 studies [ , ] is like the Med-eMonitor and PATH based on the feedback and participation of the health provider, but there is a difference in the intensity of participation of the health provider. In 1 study [ ] the health provider contacted the patient only when necessary, while in the other 2 studies [ , ] the patient received feedback instantly after sending a question.
Our analysis to examine the extent to which medication adherence is determined by different tailoring levels revealed no clear relationship between the intervention’s level of sophisticated tailoring and the extent to which the intervention was effective ().
Summary of Effects on Adherence
We found 5 studies with a significant effect on adherence (3 HQ studies and 2 LQ studies) [, , , , ]. The first study [ ] concluded that the intervention had a positive and significant effect on use of the inhaled corticosteroid and that this effect significantly differed from the baseline. In addition, the second study [ ] found positive changes in controller medication adherence. The third study [ ] tested the WebEase intervention and found a significant effect on adherence. The fourth study [ ] found that patients were more likely to take their asthma medication when they used the Health Buddy. In the fifth study [ ], patients who received the PATH intervention were more likely to adhere to their medical regimen.
A moderate effect on adherence was reported in 6 studies (4 HQ studies and 2 LQ studies). The SPPARO proved to be feasible and improved general adherence. Adherence to medication showed a similar trend but these results did not reach significance . The second study [ ] concluded that, after the intervention, the use of beta-agonist decreased, which is an indication of better adherence. The third study [ ] reported that medication compliance improved from 65% at pretest to 94% at posttest, but the difference was not statistically tested. The medication compliance rate in the fourth study [ ] was 94% for the monitor group as measured by the monitor system. However, because there was no pretest and the data of the control group were not available, it is unknown whether the results were significant as compared with the pretest or the control group. The intervention of the last 2 studies [ , ] improved adherence in patients with partly and uncontrolled asthma at baseline. The authors concluded that the intervention was most effective in improving adherence for patients with partly or uncontrolled asthma at baseline.
No significant results on patients’ adherence were found in 2 studies (1 HQ and 1 LQ) [, ].
Assessment of Adherence Measurements
Regarding the measurement of adherence, the 13 studies we reviewed showed a large variability of methods: 12 studies used a low-quality measurement to assess adherence and 1 used a combination of these methods (ie, a high-quality measurement to assess adherence) ().
Low Quality of Adherence Measurement
In 10 studies, self-reported scales were used to obtain the adherence rate. Although 5 studies [, , , , ] used self-reports to measure adherence, they did not describe what kind of instrument they used. One study [ ] used self-reported data by asking questions regarding adherence during the initial interview. Reports of therapist and patients were compared. In addition, 1 study [ ] used a self-developed medication adherence survey on the Health Buddy appliance. The other 3 studies chose existing, valid, self-reported adherence scales. One study [ ] used a combination of the Morisky scale and the General Adherence Scale from the Medical Outcomes Study, and 1 study [ ] used the Health Habits Assessment, a self-reported scale to measure adherence. One study [ ] used the self-report USCF Adherence Questionnaire and the Antiretroviral General Adherence Scale. Finally, 2 studies used measurements such as counting pills [ ], and 1 study [ ] used a computerized prescription refill record (after evaluation of the pilot study in which completing the diary turned out to be time consuming and inconvenient; see [ ]).
High Quality of Adherence Measurement
One study used a combination of methods. This study  used a diary in combination with a computerized prescription refill record.
shows the results of the assessment of the internal validity and the quality of adherence measurement.
Relation Between Quality of Adherence Measurement and Effectiveness
Our investigation of the relationship between the quality of the adherence measurement and the effectiveness of the interventions revealed no clear relationship (there was only 1 study using a high-quality method to assess adherence), although self-reported adherence measurements seemed to result more often in significant effects than did pill counts and pharmacist adherence measurements (and ). Of the 10 studies using self-reports (low-quality adherence measurement), 5 reported a significant effect of the intervention on adherence [ , , , , ], 4 a moderate effect [ , , , ], and 1 no effect [ ]. From the 2 studies in which pharmacist data or pill counting was used, 1 reported a moderate effect [ ] and 1 no effect [ ]. The 1 study that used a combination of methods to measure adherence [ ] found a moderate effect on adherence.
Relation Between Interval of Adherence Measurement and Effectiveness
There was no clear relationship between the timing of the adherence measurements and the effectiveness of the intervention. The intervals between baseline and follow-up measurements differed between projects. Short-term adherence (ie, within 6 months) was measured in 6 studies. The first study  showed that WebEase improved adherence 6 weeks after baseline. Patients who used PATH were more likely to show higher adherence than the control group after an interval of 8 weeks after baseline [ ]. The third study [ ] found a significant effect on adherence after 12 weeks, and the fourth study [ ] reported an improvement in adherence after 12 weeks. The fifth study [ ] found an adherence rate of 94% in the experimental group after 12 weeks. Because of the lack of adherence data for the control group and the lack of a pretest, the effects on adherence could not be established. The sixth study [ ] examined the proportion of nonadherent patients in both an intervention and a control group after 16 weeks. That study’s authors found uniformly small and nonsignificant differences between the control and intervention groups. However, they found an important difference within the intervention group. Subgroup differences appeared when the intensity of using parts of the intervention was related to the effectiveness. For example, patients who used the Managing Medical Regimen Workshop more often or intensely appeared to be more adherent than those using the intervention less often or intensely [ ].
Long-term adherence was measured in 7 studies—that is, adherence with an interval of 6 months or longer, mostly of 1 year or more. Two studies [, ] reported a moderate effect in their pilot on adherence after 6 months. In 2 studies [ , ], they found a moderate [ ] and significant [ ] improvement in adherence after 1 year. Two studies [ , ] found a moderate effect on adherence after 1 year. This means that all of the included studies using an interval of 6 months or longer showed an effect (significant or moderate) on long-term adherence. One study [ ] did not find an effect on adherence after 1 year.
and give an overview of the methodological quality of the studies, the level of sophistication of each intervention, the quality of measurement of adherence, and an overview of the short-term and long-term effects.
Using the principles of the best-evidence synthesis (see), taking into account the design, methodological quality, and outcomes of the studies, the following conclusions can be drawn. In total, 7 studies were considered HQ. We found 3 HQ studies [ , , ] and 2 LQ studies [ , ] that had a significant effect on enhancing medication adherence and that met 6 of 11 criteria. This means that there is evidence that tailored Internet interventions are successful in improving medication adherence.
The sensitivity analysis showed the same results as the best-evidence synthesis. The results remained the same when the analysis was repeated with the 6 LQ studies excluded (ie, taking only the 7 HQ studies into account). Moreover, when studies were rated to be HQ if 4 instead of 6 criteria of interval validity were met, results stayed the same.
First, our objective was to gain insight into the current state of the use of Internet interventions to improve medication adherence. Results of this review indicate that this is still a new field. This is visible in the differences in interventions with respect to crucial aspects such as the level of sophisticated tailoring and the role of health care providers. Despite the differences, it is remarkable that none of the interventions used a low level of tailoring and the majority (9 of 13) provided the opportunity to contact a health provider.
Second, the studies were assessed on their effectiveness on medication adherence. There is evidence that Internet interventions can improve medication adherence. This evidence comes from 3 HQ studies and 2 LQ studies, finding significant results on medication adherence.
Third, we wanted to investigate to what degree adherence is determined by the characteristics of the intervention. All interventions discussed in this review used tailored methods and used a moderately or highly sophisticated tailored intervention. These types of health programs, especially customized health programs, are more complex, generally long-term, allowing the patients to access the programs several times , and are considered appropriate for difficult-to-influence behaviors. We did not find a clear relationship between how sophisticated the tailoring of the intervention was and the extent to which the intervention appeared to be effective, possibly due to the various methods that were used.
Last, we wanted to investigate whether there is a relationship between the characteristics of a study and the reported effectiveness of the interventions. We found that there was variation not only in the level of tailoring, but also in the measurement of adherence, the timing of measuring adherence, and the intensity of the intervention. The included studies used self-reporting measurements (ie, interviews, diary, self-reporting via questionnaires) or pill counts, or prescription refills, or a combination. No study used electronic monitoring, which is perceived as a high-quality method for assessing adherence . Of the 13 studies we reviewed, 7 measured long-term adherence, using an interval of 6 months or longer. There is no clear evidence that the duration of the intervention is related to the effectiveness of the intervention. Nevertheless, of the 7 studies measuring long-term adherence, 1 HQ study showed positive effects and 4 HQ studies and 1 LQ study showed moderate effects on adherence. This indicates that long-term interventions are promising. However, more research in this field is needed.
There is evidence that Internet interventions can be effective in improving medication adherence. The evidence comes from 3 HQ studies. However, the results should be interpreted with caution. Self-reported scales were used in 10 studies, which is considered a low-quality adherence measurement: 5 reported a significant effect of the intervention on adherence, 4 a moderate effect, and 1 no effect. Self-measurements can contribute to overestimating of the effects of interventions . This could be explained by the possibility that patients may forget that they missed a dose. Biases that appear most prominent in estimating adherence by the patient from structured questionnaires are social desirability and social approval [ , ]. In other words, studies relying on self-reporting may have a tendency to err on the optimistic side when it comes to adherence, certainly compared with more objective pill-counting studies. However, it must be noted that anonymous self-report questionnaires are found to be significantly correlated with electronic monitoring [ ] and virologic response [ ], considered more objective methods. Research also shows that using specific strategies, such as ensuring patients that their responses will be kept confidential [ ] or stratifying patients according to their socially desirable response [ ], improves the prediction of adherence by self-reports. This indicates that self-reports are not useless, but future research should examine more strategies to reinforce accurate reporting by patients [ ]. On the other hand, we included RCTs and, consequently, self-reported adherence can be expected to be overestimated in both treatment arms. Thus, the intervention effect (ie, difference between intervention and control group) was not necessarily overestimated. Additionally, a distinction can be made between valid self-report measurements and measurements that are not. If self-reported measurements are used, for instance because this is a cost-effective method, using validated measurements is recommended.
Electronic monitoring or observation is considered to be more accurate. It electronically records the time and date of the actual dosing events. Because every single method has its limitations, the best approach is to use multiple assessment techniques concurrently, as a way to improve the accuracy of adherence assessment . One study in our review [ ] used self-reported diary and prescription refills to measure adherence and made a distinction in interpreting the results. They used a self-reported diary to assess how the patients used their inhaler, based on the idea that when patients are not using their inhaler according to the health providers’ advice, they can be considered nonadherent. In addition, they used prescription refills to measure how many refills the patient was obtaining. In line with this method of measuring adherence, the optimal approach could be suggested to be a combination of self-reports and more objective measurements [ ]. In addition, every single measurement needs a different interpretive approach because it has different relationships to clinical outcomes [ ].
Of the studies we reviewed, 6 measured short-term adherence that varied from 6 weeks to 4 months: 4 of them were found highly effective and 1 moderately effective regarding adherence. The question is whether adherence improved in the long term, because the period was too short to measure persistence. According to an international expert forum on adherence , it is not easy to identify adherent and nonadherent patients beforehand. There is a large body of evidence dominated by reports identifying factors that are predictive or associated with nonadherence. Adherence could be seen as a dynamic behavior that is determined or influenced by unrelated factors that fluctuate and change over time [ ]. As an adherent patient can become nonadherent over time, the importance of time (ie, persistence) has been emphasized [ ]. The quality of execution of the treatment plan can influence persistence. Factors such as perceptions of treatment outcomes, beneficial effects, and adverse effects can influence the quality of execution over time. Therefore, conducting interventions that address long-term adherence and overcome reasons why a patient is not able or not willing to adhere are recommended.
The search method was top-down in that we relied on existing databases and search terms. This approach has the possibility of missing important articles due to miscoding of search terms. A bottom-up strategy is more time consuming but has the advantage of being more comprehensive.
Monitoring adherence to optimize effects and minimize nonadherence could be time consuming. Computers, however, are very good at collecting data concerning the monitoring of adherence. Internet interventions can be tailored, collect data, and monitor adherence. In addition, based on this systematic literature review, there is evidence that tailored Internet interventions can be an effective method to improve medication adherence. This means that Web-based interventions can be effective at increasing medication adherence among chronically ill patients. Health providers, who want to enhance patients’ adherence, are encouraged to use tailored websites or reminder systems. They could use these interventions in addition to their everyday work.
Implications for Research
Because we did not find a clear relationship between the effectiveness and the degree of tailoring, we recommend that future studies should be conducted with variation in the level of sophistication of tailoring to further test which characteristics of the tailored messages have the most positive effects on adherence. Moreover, website compliance is often not completely reported. While the frequency in which the patient used the website is often reported, studies do not describe how exactly patients used the website. Therefore, it is difficult to compare the results of different interventions, because the way patients use the website can have implications for the effectiveness of the website.
With more than 40 million people using the Internet for a variety of purposes, health communication programs in the future are more likely to be delivered online . These types of interventions especially have the potential to address difficult-to-change behaviors such as adherence. This review shows promising results on the effectiveness of tailored Internet interventions to enhance medication adherence of chronically ill patients. There is evidence that these interventions can enhance adherence. But it remains a relatively new field, and studies using more objective measurements to assess adherence are recommended.
The authors acknowledge the support of 2Comply. 2Comply had no role in the design and the collection, analysis and interpretation of the data, nor in the writing of this manuscript.
Conflicts of Interest
JvW and LvD designed the study and acquired funding. MV conducted comprehensive literature searches through the databases. Two researchers pairs (AL/JvW and MV/LvD) performed a first selection for inclusion based on the abstracts. After having obtained full text of articles that seemed to meet the inclusion criteria, the same researchers pairs independently reviewed whether these studies fulfilled all the inclusion criteria. The methodological quality of included RCT’s and CCT’s were independently reviewed by AL and JvW. In addition, AL and LvD independently assessed the quality of the methods for measuring adherence to a medical regime. AL conducted the ‘Best Evidence Synthesis’ and drafted the manuscript. MV, LvD, ES and JvW have been involved in critically revising the manuscript. All authors have read and approved the final manuscript.
Multimedia Appendix 1
Detailed description of the excluded studies.PDF file (Adobe PDF File), 127 KB
Multimedia Appendix 2
Detailed description of the characteristics of the included studies.PDF file (Adobe PDF File), 155 KB
- Sabaté E. Adherence to Long-Term Therapies: Evidence for Action. Geneva: World Health Organization; 2003.
- DiMatteo MR. Variations in patients' adherence to medical recommendations: a quantitative review of 50 years of research. Med Care 2004 Mar;42(3):200-209. [Medline]
- Kane SV, Robinson A. Review article: understanding adherence to medication in ulcerative colitis - innovative thinking and evolving concepts. Aliment Pharmacol Ther 2010 Nov;32(9):1051-1058. [CrossRef] [Medline]
- Wroe AL. Intentional and unintentional nonadherence: a study of decision making. J Behav Med 2002 Aug;25(4):355-372. [Medline]
- Urquhart J, Vrijens B. New findings about patient adherence to prescribed drug dosing regimes: an introduction to pharmionics. Eur J Hosp Pharm Sci 2005;11(5):103-106.
- Kreuter MW, Farrell D, Olevitch L, Brenna L. Tailoring Health Messages: Customizing Communication With Computer Technology. Mahwah, NJ: Lawrence Erlbaum Associates; 2000.
- Lustria ML, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns 2009 Feb;74(2):156-173. [CrossRef] [Medline]
- Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull 2007 Jul;133(4):673-693. [CrossRef] [Medline]
- Kreuter MW, Wray RJ. Tailored and targeted health communication: strategies for enhancing information relevance. Am J Health Behav 2003;27 Suppl 3:S227-S232. [Medline]
- Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change outcomes. J Med Internet Res 2004 Nov 10;6(4):e40 [FREE Full text] [CrossRef] [Medline]
- Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions, version 5.1.0.: The Cochrane Collaboration; 2011 Mar. URL: http://www.cochrane-handbook.org/ [accessed 2011-10-07] [WebCite Cache]
- van Tulder M, Furlan A, Bombardier C, Bouter L, Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976) 2003 Jun 15;28(12):1290-1299. [CrossRef] [Medline]
- Sluijs E, van Dulmen S. van DijkL, Ridder D, Heerink R, Bensing J. Patient Adherence to Medical Treatment: A Meta Review. Utrecht: NIVEL; 2006.
- Wetzels GE, Nelemans PJ, Schouten JS, van Wijk BL, Prins MH. All that glisters is not gold: a comparison of electronic monitoring versus filled prescriptions--an observational study. BMC Health Serv Res 2006;6:8 [FREE Full text] [CrossRef] [Medline]
- Farmer KC. Methods for measuring and monitoring medication regimen adherence in clinical trials and clinical practice. Clin Ther 1999 Jun;21(6):1074-90; discussion 1073. [CrossRef] [Medline]
- Velligan DI, Wang M, Diamond P, Glahn DC, Castillo D, Bendle S, et al. Relationships among subjective and objective measures of adherence to oral antipsychotic medications. Psychiatr Serv 2007 Sep;58(9):1187-1192 [FREE Full text] [CrossRef] [Medline]
- Pullar T, Kumar S, Tindall H, Feely M. Time to stop counting the tablets? Clin Pharmacol Ther 1989 Aug;46(2):163-168. [Medline]
- Rudd P, Byyny RL, Zachary V, LoVerde ME, Titus C, Mitchell WD, et al. The natural history of medication compliance in a drug trial: limitations of pill counts. Clin Pharmacol Ther 1989 Aug;46(2):169-176. [Medline]
- Shi L, Liu J, Fonseca V, Walker P, Kalsekar A, Pawaskar M. Correlation between adherence rates measured by MEMS and self-reported questionnaires: a meta-analysis. Health Qual Life Outcomes 2010;8:99 [FREE Full text] [CrossRef] [Medline]
- Shi L, Liu J, Koleva Y, Fonseca V, Kalsekar A, Pawaskar M. Concordance of adherence measurement using self-reported adherence questionnaires and medication monitoring devices. Pharmacoeconomics 2010;28(12):1097-1107. [CrossRef] [Medline]
- Grymonpre RE, Didur CD, Montgomery PR, Sitar DS. Pill count, self-report, and pharmacy claims data to measure medication adherence in the elderly. Ann Pharmacother 1998;32(7-8):749-754. [Medline]
- DiMatteo MR, Giordani PJ, Lepper HS, Croghan TW. Patient adherence and medical treatment outcomes: a meta-analysis. Med Care 2002 Sep;40(9):794-811. [CrossRef] [Medline]
- Sajatovic M, Velligan DI, Weiden PJ, Valenstein MA, Ogedegbe G. Measurement of psychiatric treatment adherence. J Psychosom Res 2010 Dec;69(6):591-599. [CrossRef] [Medline]
- McMahon JH, Jordan MR, Kelley K, Bertagnolio S, Hong SY, Wanke CA, et al. Pharmacy adherence measures to assess adherence to antiretroviral therapy: review of the literature and implications for treatment monitoring. Clin Infect Dis 2011 Feb 15;52(4):493-506. [CrossRef] [Medline]
- Steultjens EMJ, Dekker J, Bouter LM. van de Nes JCM, Cardol M, van den Ende CHM. Occupational Therapy for Multiple Sclerosis (Protocol for a Cochrane Review). Oxford: Update Software; 2002.
- Verkaik R, van Weert JC, Francke AL. The effects of psychosocial methods on depressed, aggressive and apathetic behaviors of people with dementia: a systematic review. Int J Geriatr Psychiatry 2005 Apr;20(4):301-314. [CrossRef] [Medline]
- Artinian NT, Harden JK, Kronenberg MW, Vander Wal JS, Daher E, Stephens Q, et al. Pilot study of a Web-based compliance monitoring device for patients with congestive heart failure. Heart Lung 2003 Aug;32(4):226-233. [Medline]
- Jan RL, Wang JY, Huang MC, Tseng SM, Su HJ, Liu LF. An internet-based interactive telemonitoring system for improving childhood asthma outcomes in Taiwan. Telemed J E Health 2007 Jun;13(3):257-268. [CrossRef] [Medline]
- Chan DS, Callahan CW, Hatch-Pigott VB, Lawless A, Proffitt HL, Manning NE, et al. Internet-based home monitoring and education of children with asthma is comparable to ideal office-based care: results of a 1-year asthma in-home monitoring trial. Pediatrics 2007 Mar;119(3):569-578 [FREE Full text] [CrossRef] [Medline]
- Chan DS, Callahan CW, Sheets SJ, Moreno CN, Malone FJ. An Internet-based store-and-forward video home telehealth system for improving asthma outcomes in children. Am J Health Syst Pharm 2003 Oct 1;60(19):1976-1981. [Medline]
- Joseph CL, Peterson E, Havstad S, Johnson CC, Hoerauf S, Stringer S, Asthma in Adolescents Research Team. A web-based, tailored asthma management program for urban African-American high school students. Am J Respir Crit Care Med 2007 May 1;175(9):888-895 [FREE Full text] [CrossRef] [Medline]
- Ross SE, Moore LA, Earnest MA, Wittevrongel L, Lin CT. Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial. J Med Internet Res 2004 May 14;6(2):e12 [FREE Full text] [CrossRef] [Medline]
- Cherry JC, Moffatt TP, Rodriguez C, Dryden K. Diabetes disease management program for an indigent population empowered by telemedicine technology. Diabetes Technol Ther 2002;4(6):783-791. [Medline]
- Guendelman S, Meade K, Benson M, Chen YQ, Samuels S. Improving asthma outcomes and self-management behaviors of inner-city children: a randomized trial of the Health Buddy interactive device and an asthma diary. Arch Pediatr Adolesc Med 2002 Feb;156(2):114-120 [FREE Full text] [Medline]
- DeVito Dabbs A, Dew MA, Myers B, Begey A, Hawkins R, Ren D, et al. Evaluation of a hand-held, computer-based intervention to promote early self-care behaviors after lung transplant. Clin Transplant 2009;23(4):537-545. [CrossRef] [Medline]
- van der Meer V, Bakker MJ, van den Hout WB, Rabe KF, Sterk PJ, Kievit J, SMASHING (Self-Management in Asthma Supported by Hospitals‚ ICT‚ Nurses and General Practitioners) Study Group. Internet-based self-management plus education compared with usual care in asthma: a randomized trial. Ann Intern Med 2009 Jul 21;151(2):110-120. [Medline]
- van der Meer V, van Stel HF, Bakker MJ, Roldaan AC, Assendelft WJ, Sterk PJ, SMASHING (Self-Management of Asthma Supported by Hospitals‚ ICT‚ Nurses and General practitioners) Study Group. Weekly self-monitoring and treatment adjustment benefit patients with partly controlled and uncontrolled asthma: an analysis of the SMASHING study. Respir Res 2010;11:74 [FREE Full text] [CrossRef] [Medline]
- Dilorio C, Escoffery C, McCarty F, Yeager KA, Henry TR, Koganti A, et al. Evaluation of WebEase: an epilepsy self-management Web site. Health Educ Res 2009 Apr;24(2):185-197 [FREE Full text] [CrossRef] [Medline]
- Dew MA, Goycoolea JM, Harris RC, Lee A, Zomak R, Dunbar-Jacob J, et al. An internet-based intervention to improve psychosocial outcomes in heart transplant recipients and family caregivers: development and evaluation. J Heart Lung Transplant 2004 Jun;23(6):745-758. [Medline]
- Nieuwkerk PT, Oort FJ. Self-reported adherence to antiretroviral therapy for HIV-1 infection and virologic treatment response: a meta-analysis. J Acquir Immune Defic Syndr 2005 Apr 1;38(4):445-448. [Medline]
- Hugen PW, Langebeek N, Burger DM, Zomer B, van Leusen R, Schuurman R, et al. Assessment of adherence to HIV protease inhibitors: comparison and combination of various methods, including MEMS (electronic monitoring), patient and nurse report, and therapeutic drug monitoring. J Acquir Immune Defic Syndr 2002 Jul 1;30(3):324-334. [Medline]
- Nieuwkerk PT, de Boer-van der Kolk IM, Prins JM, Locadia M, Sprangers MA. Self-reported adherence is more predictive of virological treatment response among patients with a lower tendency towards socially desirable responding. Antivir Ther 2010;15(6):913-916. [CrossRef] [Medline]
- Drotar D, Richard KA, Burgess E, Levi R, Nobile C, Seti A, et al. Adherence to treatment in childhood chronic illness: issues and recommendations to enhance pediatric research and training. In: Drotar D, editor. Promoting Adherence to Treatment in Childhood Chronic Illness: Concepts, Methods and Interventions. Mahwah, NJ: Erlbaum; 2002.
- Reynolds NR. Adherence to antiretroviral therapies: state of the science. Curr HIV Res 2004 Jul;2(3):207-214. [Medline]
|CCT: clinical controlled trial|
|HQ: high quality (study)|
|LQ: low quality (study)|
|PATH: Pocket Personal Assistant for Tracking Health|
|RCT: randomized controlled trial|
|SPPARO: System Providing Access to Records Online|
Edited by G Eysenbach; submitted 24.01.11; peer-reviewed by P Nieuwkerk, C Stepnowsky; comments to author 25.04.11; revised version received 19.07.11; accepted 19.07.11; published 05.12.11
©Annemiek J Linn, Marcia Vervloet, Liset van Dijk, Edith G Smit, Julia CM Van Weert. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 05.12.2011.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.