Advertisement: Preregister now for the Medicine 2.0 Congress
Effectiveness of Web-based Interventions on Patient Empowerment: A Systematic Review and Meta-analysis
David Samoocha1,2,3, MSc; David J Bruinvels1,2,3, PhD, MD; Nieke A Elbers1,3, MSc; Johannes R Anema1,2, PhD, MD; Allard J van der Beek1,2, PhD
1Department of Public and Occupational Health, The EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, Netherlands
2Research Center for Insurance Medicine AMC-UWV-VUmc, Amsterdam, Netherlands
3IGER, the Amsterdam Interdisciplinary Center of Law and Health, Amsterdam, Netherlands
Department of Public and Occupational Health
The EMGO Institute for Health and Care Research
VU University Medical Center
Van der Boechorststraat 7
Amsterdam, 1081 BT
Phone: 31 20 444 1710
Fax: 31 20 444 8387
Background: Patient empowerment is growing in popularity and application. Due to the increasing possibilities of the Internet and eHealth, many initiatives that are aimed at empowering patients are delivered online.
Objective: Our objective was to evaluate whether Web-based interventions are effective in increasing patient empowerment compared with usual care or face-to-face interventions.
Methods: We performed a systematic review by searching the MEDLINE, EMBASE, and PsycINFO databases from January 1985 to January 2009 for relevant citations. From the 7096 unique citations retrieved from the search strategy, we included 14 randomized controlled trials (RCTs) that met all inclusion criteria. Pairs of review authors assessed the methodological quality of the obtained studies using the Downs and Black checklist. A meta-analysis was performed on studies that measured comparable outcomes. The GRADE approach was used to determine the level of evidence for each outcome.
Results: In comparison with usual care or no care, Web-based interventions had a significant positive effect on empowerment measured with the Diabetes Empowerment Scale (2 studies, standardized mean difference [SMD] = 0.61, 95% confidence interval [CI] 0.29 - 0.94]), on self-efficacy measured with disease-specific self-efficacy scales (9 studies, SMD = 0.23, 95% CI 0.12 - 0.33), and on mastery measured with the Pearlin Mastery Scale (1 study, mean difference [MD] = 2.95, 95% CI 1.66 - 4.24). No effects were found for self-efficacy measured with general self-efficacy scales (3 studies, SMD = 0.05, 95% CI -0.25 to 0.35) or for self-esteem measured with the Rosenberg Self-Esteem Scale (1 study, MD = -0.38, 95% CI -2.45 to 1.69). Furthermore, when comparing Web-based interventions with face-to-face deliveries of the same interventions, no significant (beneficial or harmful) effects were found for mastery (1 study, MD = 1.20, 95% CI -1.73 to 4.13) and self-esteem (1 study, MD = -0.10, 95% CI -0.45 to 0.25).
Conclusions: Web-based interventions showed positive effects on empowerment measured with the Diabetes Empowerment Scale, disease-specific self-efficacy scales and the Pearlin Mastery Scale. Because of the low quality of evidence we found, the results should be interpreted with caution. The clinical relevance of the findings can be questioned because the significant effects we found were, in general, small.
(J Med Internet Res 2010;12(2):e23)
Patient empowerment; Internet; eHealth
Patient empowerment refers to the enhanced ability of patients to actively understand and influence their own health status . Patient empowerment focuses on control in individuals’ experience of health, disease, and illness, as well as the roles of health care organizations, communities, and the broader health care system .
Since its introduction to health care in the 1970s , the popularity of the idea of patient empowerment has emerged in the context of several significant societal trends, such as a growth in health care consumerism and, as a result, the need for governments to reduce health care costs . In this case, patient empowerment potentially could be used to justify cost-cutting in which part of the responsibility for care is transferred to individual citizens . Furthermore, increased patient activism and organization has led to more focus on patient empowerment initiatives . Revealingly, and in line with these tendencies, the World Health Organization (WHO) has described patient empowerment as a “prerequisite for health” and “a proactive partnership and patient self-care strategy to improve health outcomes and quality of life among the chronically ill .”
During the last decades, the focus on empowerment resulted in many initiatives to increase patient empowerment. In general, strategies to increase patient empowerment have tended to address two aspects of patients’ experience of illness: (1) disease management and (2) relationships with health care providers . An increasing number of studies have been conducted in which approaches to increasing patient empowerment have been evaluated. These approaches have varied from patient self-management programs [9-16], to promoting patient involvement in treatment decision-making [17,18], to facilitating the physician-patient interaction [19,20]. Most of these interventions have taken place face-to-face or in facilitator-led group sessions.
Although some face-to-face or group session interventions to increase patient empowerment have been found to be effective, for example, in decreasing depression  or in job retention among the chronically ill , it is believed that the real opportunities for patient empowerment started with the rise of the Internet and eHealth . In recent years, the number of Internet users has increased considerably, and the Internet is being employed more frequently to locate information on health and health care delivery . The latest studies have shown that, among all Internet users, an estimated 58% consult the Web for health purposes . Because of this increased use of the Internet and its huge potential for delivering patient education and management programs, the Internet may have a revolutionary role in retooling the health care industry . Some scientific evidence already exists on the effectiveness of the Internet to improve health outcomes [26,27], increase specified knowledge [28,29], achieve behavior change [30,31], and increase participation in health care . Therefore, in recent years, a growing number of interventions aimed at patient empowerment are, not surprisingly, delivered online.
In this systematic review, we investigated whether these Web-based interventions were effective in increasing patient empowerment compared with usual care or face-to-face interventions.
Types of Studies
Only randomized controlled trials (RCT), quasi-randomized controlled trials, before-and-after studies, and interrupted time series analyses were included.
Types of Participants
Studies in which the intervention was aimed at patients or clients with a medical problem were included. Studies that included children and adolescents less than 18 years of age were excluded to create more homogeneity in the study population.
Types of Intervention
Studies in which the treatment consisted of a Web-based intervention were included. Web-based interventions were defined as all interactive Web applications accessed via the Internet or an intranet. Furthermore, we excluded studies if the intervention did not contain any aspects of health education or intention to change health-related behavior.
Types of Control Intervention
Studies were included only if the control intervention consisted of either usual care or a face-to-face intervention.
Types of Outcome Measures
Studies that measured empowerment or an empowerment-related component were included. Given the absence of a generally accepted definition of empowerment and conflicting views on how to measure empowerment, we decided to initially include concepts that are often linked to empowerment. Examples of these are self-efficacy, mastery, self-control, self-esteem, perceived control, perceived competence, or involvement in the decision-making process .
No language restriction was applied.
Publications were retrieved by a search of the following electronic databases:
- MEDLINE (l985 to January 2009)
- EMBASE (1985 to January 2009)
- PsycINFO (1985 to January 2009)
Detailed search strategies are presented in the Multimedia Appendix. Briefly, we combined two search concepts, the first consisting of the outcome measure (eg, “empowerment” or “self-efficacy”) and the second of the intervention (eg, “Internet” or “website”). Various synonyms were used for each concept. We chose a sensitive search strategy so that we would not miss any potentially relevant publications.
Study Identification and Data Abstraction
Citations and brief records identified by the search strategy were downloaded electronically to the bibliographic management package Reference Manager 11 (Thomson Reuters, Carlsbad, CA, USA). The study selection was completed in three steps. In step 1, two reviewers (authors DS and DB) independently screened the titles, keywords, and abstracts of the studies obtained by the search strategy to determine if they met the inclusion criteria. When inclusion or exclusion of a study could not be based on the screening of the title, keywords, and abstract, in step 2, the full article was retrieved and checked for inclusion. This was again, done by two reviewers (authors DS and NE). A consensus meeting with a third reviewer (DB) was arranged to sort out disagreements between reviewers. In step 3, we searched the reference lists of the included studies to find additional publications. Additionally, from all citations that were initially identified by the search strategy, we checked all systematic reviews concerning Web-based interventions and searched in the reference lists of these reviews to find additional publications that met our inclusion criteria.
Two reviewers (DS and NE) extracted the data using a data extraction form that included information on study design, randomization level, population, follow-up period, description of the intervention and control group treatments, and data on relevant outcomes. If certain studies did not report sufficient information on the outcomes, missing data (for example, standard deviations) were calculated according to guidelines in the Cochrane Reviewers’ Handbook . If it was not possible to calculate missing data, authors of the studies were contacted and additional information on the missing data was requested.
We evaluated the quality of individual studies using the Downs and Black quality assessment method, which is a list of 27 criteria to evaluate both randomized and nonrandomized trials . This quality assessment scale (QAS) assesses study reporting, external validity, and internal validity (ie, bias and confounding), and has been ranked in the top six quality assessment scales suitable for use in systematic reviews [36,37]. As has been done in other reviews using the Downs and Black scale [38,39], the tool was modified slightly for use in this review. Specifically, the scoring for question 27 dealing with statistical power was simplified to a choice of awarding either 1 point or 0 points depending on whether there was sufficient power to detect a clinically important effect. The criterion was that to detect a 10% difference, at least 87 subjects had completed follow-up in both the intervention and control groups of the study, assuming power of .90 and alpha of .05. The maximum score for item 5, reporting of principal confounders was 2. Downs and Black score ranges were grouped into the following 4 quality levels: excellent (26 to 28), good (20 to 25), fair (15 to 19), and poor (less than 14).
Two reviewers (DS and DB) independently assessed the quality of the included studies. A consensus method was used to resolve disagreement.
Data Analysis and the GRADE Approach
Analyses of this review were based on the outcome measure, that is, empowerment or an empowerment-related outcome. For studies that were comparable with respect to the control intervention and the outcome, results were pooled using meta-analyses. In these analyses, we included final measurements of continuous data. We were able to do so, because all included studies in this review were RCTs or quasi-RCTs, and no study reported significant baseline differences between the intervention and the control group. Since there were many different scales or instruments used to measure the same outcome, we could not use weighted mean differences and therefore calculated standardized mean differences (SMD). SMDs obtained from the meta-analyses were then reexpressed to a familiar instrument, using a so-called back-translation technique. In this technique, an instrument is selected from a study included in the meta-analysis that is representative of the population and at a low risk of bias, and then the standard deviation of the outcome measure of the control group of this study (the end of study mean) is multiplied by the pooled SMD.
We chose a random-effects meta-analysis because it was considered a more appropriate model to combine the results of the included studies, which were clinically and methodologically diverse . Additionally, publication bias was tested by inspecting the funnel plot on outcomes that were measured in more than 8 studies. A funnel plot is a scatterplot of treatment effect against a measure of study size. An asymmetric funnel indicates a relationship between treatment effect and study size, suggesting a possibility of publication bias. For all analyses Review Manager software (version 5.0) was used .
We present the overall quality of the evidence using the GRADE approach as recommended by the Cochrane Handbook for Systematic Reviews of Interventions  because of its many advantages . That is, for each specific outcome, the quality of the evidence was based on five factors: (1) limitations of the study design or the potential for bias across all studies that measure that particular outcome, (2) consistency of results, (3) directness (generalizability), (4) precision (sufficient data), and (5) the potential for publication bias. The overall quality was considered to be high if multiple RCTs with a low risk of bias provided consistent, generalizable results for the outcome. The quality of evidence was downgraded by one level if one of the factors described above was not met. Likewise, if two or three factors were not met, we downgraded the level of evidence by two or three levels, respectively. Thus, the GRADE approach resulted in four levels of quality of evidence: high, moderate, low, and very low. In the case of only one study measuring an outcome, the data were considered to be “sparse,” and subsequently the evidence was labeled as “low quality evidence.” If only one study was present for a given comparison, the results are described in the text. GRADEprofiler software (version 3.2) was used .
From all databases combined, we identified a total of 7676 titles: 1823 in MEDLINE, 3540 in EMBASE and 2313 in PsychINFO. After exclusion of duplicates, DS and DB reviewed the 7096 titles and abstracts. Figure 1 shows a flow diagram of the reviewed studies.
[view this figure]
|Figure 1. Study selection|
We selected 289 publications for retrieval of full text versions. From these 289 studies, initially 18 publications met our inclusion criteria: these were publications reporting results of 16 RCTs and 2 quasi-RCTs. The main reason for excluding studies was based on the outcome criteria: the majority of the studies did not measure empowerment or an empowerment-related outcome. From the 18 studies that initially met our inclusion criteria, 2 publications of Man et al [44,45] reported on the same study and were handled as one RCT. The publication of Kukafka et al  was excluded because standard deviations of the outcome (self-efficacy) were not reported and were not retrieved after contacting the author. The study by Wangberg  did not have an appropriate contrast for the website intervention. Instead, this study compared two study groups that both used tailored Internet interventions. After closer examination, the study by Robinson  was excluded because it did not meet the patient criteria, and the study by Williams et al  was excluded because the intervention in this study was not solely Web-based. The additional reference search resulted in one extra publication  that met our inclusion criteria. Because this publication by Warmerdam et al contained a description of a study protocol, we contacted the authors and received relevant unpublished data. With this study included, the total number of studies included in this review was 14 [45,50-62].
Table 1 shows the characteristics of the included studies.
[view this table]
|Table 1. Characteristics of the included studies|
Design of the Included Studies
Of the 14 included studies, 13 were RCTs, while the study that was reported in two publications [44,45] was a quasi-RCT. Most studies compared two study groups: an intervention group in which the treatment was a Web-based intervention and a control group receiving usual care or no care. Exceptions were the studies of Nguyen et al  and Gollings et al  in which comparisons were made between a Web-based intervention and a face-to-face intervention. Furthermore, the study by Man et al  consisted of four intervention arms: a control group, a computer-assisted training program, an online training program, and a therapist-administered training program. For this review, we compared the online training program with the control group. Also, the study by Warmerdam et al  included three interventions arms: a Web-based cognitive behavior program, a web-based problem-solving program, and a control group.
The number of participants varied from 15  to 958 . Studies differed regarding patient groups. For example, various studies included infertility patients, patients with post-traumatic stress disorder, patients with diabetes, or back pain patients. The mean dropout rate was 23% (SD 11%) after an average follow-up duration of 19 weeks (SD 15 weeks). Participants’ compliance with the intervention was not clearly described in many of the included studies.
Empowerment was explicitly measured in only two studies. Both of these used the Diabetes Empowerment Scale (DES) . Although the DES is meant to measure diabetes-related empowerment, the study of Hill et al  showed that the DES can be adapted to other diseases. All other included studies (one or more) measured empowerment-related outcomes: 9 studies measured disease-specific self-efficacy, 3 measured general self-efficacy, 2 measured mastery, and 2 measured self-esteem.
The quality of the included studies varied. According to the calculated quality assessment score (QAS), none of the studies were rated as being of poor quality, 6 studies were rated fair, 7 were rated good, and 1 was rated excellent. The mean QAS for the included studies was 19.6. Studies scored particularly poor on the following items: patient blinding (11 of 14), blinding of the outcome assessor (12 of 14), failure to adjust for confounding factors in the analysis (11 of 14), bias due to losses of patients to follow-up (9 of 14), and insufficient power to detect outcomes that are clinically important (6 of 14). Furthermore, in 9 of the 14 studies the randomization method and concealment were not described adequately. Because participants in Web-based research are not representative of the whole patient population (in this case through a selection process of only Internet users), the external validity of all studies was rated poor.
Effectiveness of Web-based Interventions (Web-based Interventions versus Usual Care)
Empowerment was measured in 2 studies with the Diabetes Empowerment Scale (DES). Homko et al  examined the effectiveness of an Internet-based telemedicine system that was aimed at self-management of underserved women with diabetes mellitus. Empowerment was assessed at baseline and at 37 weeks of gestation. The control group received paper logbooks, which served as a sham intervention, and in which women could monitor their blood glucose levels. In the study of Hill et al , the influence of an online intervention containing self-help support groups and Web-based educational tools on empowerment was examined among chronically ill women.
Because the study of Hill et al included only the 10-item “Setting and Achieving Goals” subscale from the DES, our comparison of the two studies was based on the results of this subscale alone.
Based on the GRADE approach, we downgraded the level of evidence two levels, that is, from high to low, on basis of the studies’ limitations and imprecision of the results (Table 2).
Therefore, based on 2 RCTs (combined n = 157) our results, shown in Figure 2, indicate low quality evidence that Web-based interventions had a significant positive effect on empowerment measured with the DES scale. Figure 2 shows that the SMD for these studies was 0.61 (95% CI 0.29 - 0.94). In Figure 2, the green squares indicate the individual study’s effect sizes, and the black diamond represents the pooled effect of the combined studies.
[view this figure]
|Figure 2. Comparison of Web-based interventions versus usual care for the outcome empowerment|
Of the 14 included studies, 9 studies provided sufficient data for calculation of an SMD for disease-specific self-efficacy outcomes. Cousineau et al  examined the effects of an online psycho-educational support program for infertile women. In this trial, where 190 females were recruited from US fertility centers, a trend was observed for improvement of self-efficacy levels among the women exposed to the online program compared with the controls (P = .07). Hirai et al  found that use of an Internet-based self-change program for traumatic event-related fear, distress, and maladaptive coping increased self-efficacy significantly (P < .01). In three large and relatively high quality studies (the average Downs and Black score was 21) by Lorig et al [56-58], disease-specific self-efficacy was significantly increased after use of online interventions compared with usual care. These studies contained interventions that dealt with management of heart-lung disease and type 2 diabetes (P = .06) , arthritis or fibromyalgia (P = .01) , and back pain (P = .02) . Although the duration of the interventions used in these studies was approximately 6 weeks, improvements in self-efficacy remained even after 1 year of follow-up. In a study by Man et al , in which people with acquired brain injury (ABI) were able to follow a tele-analogy-based problem-solving program, it was found that after 8 weeks, self-efficacy levels increased more among patients using the program compared with the controls, but this effect was not significant. Ross et al  found a trend in improvement of self-efficacy in patients with congestive heart failure who used a Web-based online medical record and educational guide, compared with patients who received usual care (P = .08). In a study of Tuil et al , patients who had to undergo in vitro fertilization (IVF) were empowered by an interactive website that contained health information, a medical record, and communication possibilities with fellow patients and a physician. No pre/post test differences in IVF-specific self-efficacy were found between the intervention and control group in this study. Finally, a pilot study conducted by Zutz et al  found a higher level of exercise-specific self-efficacy after use of an interactive website that was aimed at cardiac rehabilitation. Compared with changes in the control group, this intervention effect was not significant.
According to GRADE guidelines, we downgraded the level of evidence for this outcome by one level from high to moderate based on studies’ limitations (see Table 2).
Visual inspection of the funnel plot on this outcome indicated a possibility of publication bias. Figure 4 shows that small studies, as indicated by the high standard errors [SE] (y-axis), with relatively high effect sizes, as indicated by high SMDs (x-axis), were more present in this analysis than small studies with small positive or negative effects. A possible reason for this is that small and effective (pilot) studies are more likely to be published [45,54,62]. We did not, however, downgrade the quality of evidence on basis of the funnel plot because removing these small studies from the analysis did not result in a change of the pooled estimate.
Therefore, there was moderate quality evidence from 9 studies (combined n = 2402) that Web-based interventions had a significant positive effect on self-efficacy measured with disease-specific self-efficacy scales. Figure 3 shows that the SMD of these 9 studies was 0.23 (95% CI 0.12 - 0.33).
[view this figure]
|Figure 3. Comparison of Web-based versus usual care for the outcome disease-specific self-efficacy|
[view this figure]
|Figure 4. Funnel plot for comparison of Web-based interventions versus usual care for the outcome disease-specific self-efficacy|
In three studies, general self-efficacy was measured using general self-efficacy (GSE) scales [64,65]. In the study of Hill et al , no significant effect from the computer-based intervention was found on general self-efficacy. Likewise, in the studies of Tuil et al  and Zutz et al , the Web-based interventions did not have a significant impact on general self-efficacy levels.
We downgraded the level of evidence by two levels, from high to low, based on limitations in the studies and on basis of imprecision of the results (see Table 2).
Therefore, based on the GRADE approach, there was low quality evidence from 3 studies (combined n = 293) that there was no statistically significant difference between Web-based interventions and usual care in increasing general self-efficacy. The SMD of these 3 studies was 0.05 (95% CI -0.25 to 0.35). (See Figure 5).
[view this figure]
|Figure 5. Comparison of Web-based versus usual care for the outcome general self-efficacy|
One study used the Pearlin Mastery Scale  to measure the construct mastery, an outcome that is often linked to empowerment. In this RCT by Warmerdam et al , two independent Web-based interventions were tested as to their effectiveness to treat adults with depressive symptoms. The first intervention, which lasted 5 weeks, was a problem-solving treatment (PST) based on the idea that by solving people’s practical problems their depressive symptoms will improve. The second intervention, which lasted 12 weeks, was a cognitive behavioral therapy (CBT), aimed at changing people’s cognitive patterns to decrease symptoms of depression. Pre- and post treatment mastery levels between the intervention groups and a control group on a waiting list were compared. Both interventions were found to have had a significant effect on mastery, when compared with the control group. In our analysis, we combined the effects of the two interventions and compared them to the studies’ control group. At 12 weeks follow-up, average mastery levels for both interventions combined was 22.32 (SD 4.17), while the waiting list control group scored 19.37 (SD 3.75). The difference between the control group and the Web-based interventions was significant.
The mean difference [MD] was 2.95 (95% CI 1.66 - 4.24).
Because there was only one study available, according to GRADE there was low quality evidence (one study, n = 263) that Web-based interventions had a significant positive effect on mastery measured with the Pearlin Mastery scale.
Only one study included a measurement of the construct self-esteem, measured with the Rosenberg Self-esteem (RSE) Scale . In the study of Hill et al  no significant effects were found for the intervention on self-esteem. The MD was -0.38 (95% CI -2.45 to 1.69).
For this one study (n = 120), based on the GRADE approach, there was low quality evidence that there was no statistically significant difference between Web-based interventions and usual care in increasing self-esteem.
Summarized in Table 2 are the results of the GRADE approach to judging the quality of the evidence of the studies included in this review. Since all included studies were randomized trials, we started with a high quality of evidence and downgraded, if necessary, on basis of the 5 GRADE domains.
To be able to make clinical interpretations of the reported SMDs described above, we reexpressed the pooled SMDs into MDs, by using the technique of back-translation to a familiar instrument. This technique has been described in more detail elsewhere . Table 3 shows the absolute effects (MDs) on the outcomes for which SMDs were calculated, by using back-translation.
[view this table]
|Table 2. Overall judgment of quality of evidence using the GRADE approach|
[view this table]
|Table 3. Back-translation of SMDs into MDs by using a familiar instrument|
Web-based Interventions Versus Face-to-face Interventions
Nguyen et al  compared an Internet-based dyspnea self-management program for chronic obstructive pulmonary disease (COPD) patients with a face-to-face delivery of the same intervention. In this pilot study, 39 COPD patients were randomized into either one of the two intervention groups. At baseline and after a six-month follow-up period, mastery was measured. Results from this study indicated a slight but not significant advantage for the Internet-based delivery compared with the face-to-face intervention (MD = 1.20, 95% CI -1.73 - 4.13).
Because we found only one study (n = 50) in this category, there is low quality evidence that there is no statistically significant difference between Web-based interventions and face-to-face interventions in increasing mastery.
Gollings et al  compared an Internet and face-to-face delivery of a group body image and disordered eating intervention for women. In this 8-week program, participants were able to communicate either online or face-to-face with a therapist that moderated 8 step-by-step group sessions aimed at self-evaluation, managing social pressures and problem solving around weight, and shape and eating issues, for example. Before the intervention and at 8 weeks after the intervention, self-esteem was measured with the RSE in both groups. Although self-esteem improved after both interventions, no significant differences were found between the online and face-to-face delivery (MD = -0.10, 95% CI -0.45 to 0.25).
Again, because we found only one study (n = 39) in this category, there is low quality evidence that there is no statistically significant difference between Web-based interventions and face-to-face interventions in increasing self-esteem (MD = -0.10, 95% CI -0.45 to 0.25).
The Internet revolution and growing need for patient empowerment initiatives has resulted in many Web-based empowerment interventions that have been scientifically evaluated. With this systematic review we intended to gain more insight into the effectiveness of these interventions on empowerment or empowerment-related outcomes.
In this systematic review, 13 RCTs and 1 quasi-RCT were included. The included studies were clinically heterogeneous regarding included patients, duration and intensity of the intervention, duration of follow-up, and measured outcomes. Statistical pooling was considered to be appropriate in studies measuring the same outcome and comparing the same treatments (either Web-based vs usual care or Web-based vs face-to-face). This resulted in seven comparisons. Statistical pooling within these comparisons showed that Web-based interventions have a significant positive effect on empowerment measured with the DES (2 studies), self-efficacy measured with disease-specific self-efficacy instruments (9 studies), and mastery measured with the Pearlin Mastery Scale (1 study). No significant effects of Web-based interventions were found on self-efficacy measured with general self-efficacy scales (3 studies) and self-esteem measured with the Rosenberg self-esteem scale (1 study). When comparing Web-based interventions with face-to-face deliveries of the same interventions, no statistically significant effect was found in favor of either one of the two deliveries, when the outcome mastery (1 study) or self-esteem (1 study) was measured. Based on the GRADE approach, we found that the evidence for most of the findings described above is of low quality. This means that high quality future research is likely to have an effect on our confidence in the estimate of the effect. The main reason for the low quality of evidence was that many comparisons contained only one study. In the comparisons with more studies available, limitations in study design (lack of blinding, allocation of treatment, not taking into account loss to follow-up) and imprecision of the results, resulted in downgrading the level of evidence.
Although the results of this systematic review indicated that there is some evidence that Web-based interventions are effective in increasing certain empowerment or empowerment-related outcomes, the level of evidence for these effects is rather low, and the results should be interpreted with caution. The basis for the low evidence lies in several methodological issues. First, almost all included studies based their main conclusions on analysis of treatment rather than intention to treat. In this case, results are exposed to a high risk of bias, because characteristics from participants who comply with the treatment may differ from non-participants. This is especially the case in Web-based interventions, where it is known that selection bias is evident, that is, familiarity with the use of computers and the Internet leads to self selection in the use of these technologies . Results of the studies included in this review may thus overrate the effect of the interventions on the patient population as a whole. Moreover, compliance in Web-based research is often found to be low , and therefore it seems that Web-based interventions are not suitable for everyone. The issue of low compliance also increases the risk of bias. Other issues that also increase the risk of bias in the studies we found and that should be taken into account involve not adequately describing the randomization method and the lack of patient blinding.
Another concern is the likelihood of publication bias. In the comparison where disease-specific self-efficacy was the outcome, a funnel plot showed some evidence of bias due to publication of smaller and more effective studies or pilot studies [45,54,62]. In the other comparisons and for outcomes represented by only one RCT, many relatively small studies were found, and the choice to publish these studies may have been based on their effectiveness.
Despite these limitations, our meta-analysis included only RCTs or quasi RCTs, which gives our findings a greater robustness than would have been possible if other study designs had been included. Furthermore, by applying the GRADE approach to determine the level of evidence of the effect of an intervention on a set of relevant outcomes, we were able to draw balanced conclusions and give transparency on the basis of how this level of evidence was determined.
A final point involving the methodology of this review is our choice to statistically pool the results of some of the included studies. Even though pooling included studies that measured the same outcome, these studies were clinically heterogeneous with regard to types of patients, duration and intensity of intervention, and duration of follow-up measurements. It has been argued that in the face of this diversity one should not attempt to perform a meta-analysis . We, however, tried to obviate the clinical heterogeneity by pooling studies that measured the same outcome and by choosing a more conservative random-effects model in our meta-analyses. Also, we think that the general question asked by this review, that is, whether Web-based interventions are effective in increasing patient empowerment, could only be answered by including a broad spectrum of studies where Web-based interventions have been used to date for this purpose. As a result of this choice, caution is advised when interpreting this review in that the results may only apply to specific disease or to specific time frames (eg, short-term vs long-term effects).
Statistically significant effectiveness of Web-based interventions on disease-specific self-efficacy was found. On the other hand, no effects were found for general self-efficacy. Because general self-efficacy refers to a broad and stable sense of personal competence, it is possible that Web-based interventions aimed at a specific target patient population are too specific to influence a stable personal characteristic (ie, a trait). Therefore, it has been recommended that for the majority of applications, perceived self-efficacy should be conceptualized in a situation-specific manner . In line with this recommendation, a systematic review of Murray et al  found that among children suffering from a chronic disease, self-efficacy was more likely to improve after use of Interactive Health Applications compared with no intervention (SMD = 0.24, 95% CI 0.00 - 0.48).
Sustainability of the Effects
From the results of this review, little is known about the sustainability of the effects. In most cases we included data that was measured directly after participants were exposed to the intervention. The effects that are reported, therefore, reflect a direct effect of the intervention. On the other hand, in the studies of Lorig et al [56-58], which measured disease-specific self-efficacy, data was presented only at 6-month or 12-month follow-ups, while the intervention in these studies lasted 6 weeks. Because of the high number of participants included in these studies (total weight in the comparison is 67.1%) there are some signs that the effects of these interventions remain after a longer period of time.
We calculated SMDs in this review. This means that the effect sizes presented do not represent certain improvements on specific instruments. To be able to say something about the magnitude of the effect sizes we found, we used back translations of SMDs to a familiar instrument. The results of these back calculations are shown in Table 3. We should, however, realize that these transformations from SMDs to MDs are somewhat arbitrary and should be interpreted with caution. Nevertheless, we found that the statistically significant improvements in empowerment and empowerment-related outcomes were rather small effects: empowerment measured with the DES increased 6.2% after use of the Web-based interventions and self-efficacy measured with disease-specific self-efficacy scales improved 4.2% compared with usual or no care. What the direct or indirect impacts (on clinical outcomes, for example) of these improvements are remains unknown.
Implications for Clinicians
Based on this review, there is some evidence that the Internet can be an effective method to increase patient empowerment. The results from this review show that Web-based interventions can be effective in increasing empowerment among patients who are, for example, suffering from diabetes, depression, infertility, or arthritis. These findings are in line with the growing literature on the effectiveness of eHealth interventions in general, and on outcomes other than patient empowerment [68,72,73]. Clinicians who are interested in empowering their patients are encouraged to refer their patients to Internet empowerment websites, if available and appealing to the patient.
Implications for Research
The outcome empowerment usually refers to achieving self-efficacy, mastery, and control. Although many researchers underline that these constructs are closely related to the concept empowerment , still much is unclear about how empowerment is defined and how it should be measured. For example, recent work from Aujoulat et al  added that empowerment also includes a process of accepting relinquishment of control instead of solely gaining control. Either way, much work lies ahead for researchers in defining and conceptualizing the term empowerment. This will enable combining more unambiguous research outcomes and lead to better insight into the conditions under which, and the individuals for whom, Web-based interventions are effective and how effectiveness can be maximized. Furthermore, future Web-based RCTs aimed at patient empowerment should be conducted on large populations, include the intention-to-treat principle in their analysis  and, if applicable, use “sham” website interventions to blind participants from treatment in order to increase the quality of evidence in this field of research. In this perspective, to minimize the risk of bias, researchers are encouraged to consult quality assessment lists (such as the Downs and Black list used in this review) prior to conducting a trial.
We thank Ilse Jansma for her contribution in developing the search strategy and Piet Kostense for statistical support.
Conflicts of Interest
DS and DB contributed to protocol design, selection of studies, data extraction, statistical analysis, and data interpretation. NE contributed to the selection of studies and data extraction. DS wrote the manuscript, which was commented on by DB, NE, HA, and AB. All authors have seen and approved the final version of this paper.
Multimedia Appendix 1
Medline search strategy[PDF file (Adobe PDF File), 53 KB]
- Bruegel RB. Patient empowerment--a trend that matters. J AHIMA 1998 Sep;69(8):30-3; quiz 35-6. [Medline]
- Harris R, Veinot T. The Empowerment Model and Using E-Health to Distribute Information. Vancouver, BC: Action for Health, Simon Fraser University and the Vancouver Coastal Health Research Institute; 2004.
- Tomes N. Patient empowerment and the dilemmas of late-modern medicalisation. Lancet 2007 Feb 24;369(9562):698-700. [Medline] [CrossRef]
- Segal L. The importance of patient empowerment in health system reform. Health Policy 1998 Apr;44(1):31-44. [Medline] [CrossRef]
- Neuhauser D. The coming third health care revolution: personal empowerment. Qual Manag Health Care 2003;12(3):171-84; discussion 185-6. [Medline]
- Jones K, Baggott R, Allsop J. Influencing the national policy process: the role of health consumer groups. Health Expect 2004 Mar;7(1):18-28. [Medline] [CrossRef]
- Aymé S, Kole A, Groft S. Empowerment of patients: lessons from the rare diseases community. Lancet 2008 Jun 14;371(9629):2048-2051. [Medline] [CrossRef]
- Aujoulat I, d'Hoore W, Deccache A. Patient empowerment in theory and practice: polysemy or cacophony? Patient Educ Couns 2007 Apr;66(1):13-20. [Medline] [CrossRef]
- Anderson RM, Funnell MM, Butler PM, Arnold MS, Fitzgerald JT, Feste CC. Patient empowerment. Results of a randomized controlled trial. Diabetes Care 1995 Jul;18(7):943-949. [Medline]
- Elzen H, Slaets JPJ, Snijders TAB, Steverink N. Evaluation of the chronic disease self-management program (CDSMP) among chronically ill older people in the Netherlands. Soc Sci Med 2007 May;64(9):1832-1841. [Medline] [CrossRef]
- Foster G, Taylor SJC, Eldridge SE, Ramsay J, Griffiths CJ. Self-management education programmes by lay leaders for people with chronic conditions. Cochrane Database Syst Rev 2007(4):CD005108. [Medline] [CrossRef]
- Funnell MM, Nwankwo R, Gillard ML, Anderson RM, Tang TS. Implementing an empowerment-based diabetes self-management education program. Diabetes Educ 2005;31(1):53, 55-56, 61. [Medline]
- Lorig KR, Sobel DS, Ritter PL, Laurent D, Hobbs M. Effect of a self-management program on patients with chronic disease. Eff Clin Pract 2001;4(6):256-262 [FREE Full text] [Medline]
- Tsay SL, Hung LO. Empowerment of patients with end-stage renal disease--a randomized controlled trial. Int J Nurs Stud 2004 Jan;41(1):59-65. [Medline] [CrossRef]
- Viklund G, Ortqvist E, Wikblad K. Assessment of an empowerment education programme. A randomized study in teenagers with diabetes. Diabet Med 2007 May;24(5):550-556. [Medline] [CrossRef]
- Wong AL, Harker JO, Lau VP, Shatzel S, Port LH. Spanish Arthritis Empowerment Program: a dissemination and effectiveness study. Arthritis Rheum 2004 Jun 15;51(3):332-336 [FREE Full text] [Medline] [CrossRef]
- Davison BJ, Degner LF. Empowerment of men newly diagnosed with prostate cancer. Cancer Nurs 1997 Jun;20(3):187-196. [Medline] [CrossRef]
- Huby G, Brook JH, Thompson A, Tierney A. Capturing the concealed: Interprofessional practice and older patients' participation in decision-making about discharge after acute hospitalization. J Interprof Care 2007 Feb;21(1):55-67. [Medline] [CrossRef]
- Little P, Dorward M, Warner G, Moore M, Stephens K, Senior J, et al. Randomised controlled trial of effect of leaflets to empower patients in consultations in primary care. BMJ 2004 Feb 21;328(7437):441 [FREE Full text] [Medline] [CrossRef]
- McCann S, Weinman J. Empowering the patient in the consultation: a pilot study. Patient Educ Couns 1996 Apr;27(3):227-234. [Medline] [CrossRef]
- Anderson B. Collaborative care and motivational interviewing: improving depression outcomes through patient empowerment interventions. Am J Manag Care 2007 Nov;13(4 Suppl):S103-S106. [Medline]
- Varekamp I, Verbeek JHAM, van Dijk FJH. How can we help employees with chronic diseases to stay at work? A review of interventions aimed at job retention and based on an empowerment perspective. Int Arch Occup Environ Health 2006 Nov;80(2):87-97. [Medline] [CrossRef]
- Forkner-Dunn J. Internet-based patient self-care: the next generation of health care delivery. J Med Internet Res 2003 May 15;5(2):e8 [FREE Full text] [Medline] [CrossRef]
- Powell JA, Darvell M, Gray JAM. The doctor, the patient and the world-wide web: how the internet is changing healthcare. J R Soc Med 2003 Feb;96(2):74-76 [FREE Full text] [Medline] [CrossRef]
- Atkinson NL, Saperstein SL, Pleis J. Using the internet for health-related activities: findings from a national probability sample. J Med Internet Res 2009;11(1):e4 [FREE Full text] [Medline] [CrossRef]
- Clarke G, Eubanks D, Reid Ed, Kelleher C, O'Connor E, DeBar LL, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005;7(2):e16 [FREE Full text] [Medline] [CrossRef]
- Eysenbach G. The impact of the Internet on cancer outcomes. CA Cancer J Clin 2003;53(6):356-371 [FREE Full text] [Medline] [CrossRef]
- Homer C, Susskind O, Alpert HR, Owusu Ms, Schneider L, Rappaport LA, et al. An evaluation of an innovative multimedia educational software program for asthma management: report of a randomized, controlled trial. Pediatrics 2000 Jul;106(1 Pt 2):210-215. [Medline]
- Krishna S, Francisco BD, Balas EA, König P, Graff GR, Madsen RW. Internet-enabled interactive multimedia asthma education program: a randomized trial. Pediatrics 2003 Mar;111(3):503-510 [FREE Full text] [Medline] [CrossRef]
- Oenema A, Brug J, Lechner L. Web-based tailored nutrition education: results of a randomized controlled trial. Health Educ Res 2001 Dec;16(6):647-660 [FREE Full text] [Medline] [CrossRef]
- Wydra EW. The effectiveness of a self-care management interactive multimedia module. Oncol Nurs Forum 2001 Oct;28(9):1399-1407. [Medline]
- Gustafson DH, Hawkins R, Boberg E, Pingree S, Serlin RE, Graziano F, et al. Impact of a patient-centered, computer-based health information/support system. Am J Prev Med 1999 Jan;16(1):1-9. [Medline] [CrossRef]
- Zimmerman MA. Psychological empowerment: issues and illustrations. Am J Community Psychol 1995 Oct;23(5):581-599. [Medline] [CrossRef]
- Higgins J, Deeks JJ, Altman DG. Chapter 16: Special topics in statistics. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008): The Cochrane Collaboration; 2008. URL: http://www.cochrane-handbook.org/
- Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998 Jun;52(6):377-384 [FREE Full text] [Medline] [CrossRef]
- Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7(27):iii-x, 1-173 [FREE Full text] [Medline]
- Saunders LD, Soomro GM, Buckingham J, Jamtvedt G, Raina P. Assessing the methodological quality of nonrandomized intervention studies. West J Nurs Res 2003 Mar;25(2):223-237. [Medline] [CrossRef]
- Chudyk AM, Jutai JW, Petrella RJ, Speechley M. Systematic review of hip fracture rehabilitation practices in the elderly. Arch Phys Med Rehabil 2009 Feb;90(2):246-262. [Medline] [CrossRef]
- Hooper P, Jutai JW, Strong G, Russell-Minda E. Age-related macular degeneration and low-vision rehabilitation: a systematic review. Can J Ophthalmol 2008 Apr;43(2):180-187 [FREE Full text] [Medline] [CrossRef]
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986 Sep;7(3):177-188. [Medline] [CrossRef]
- The Nordic Cochrane Centre, The Cochrane Collaboration. 2008. Review Manager (RevMan) [computer software]. Version 5.0 URL: http://www.cc-ims.net/revman [WebCite Cache]
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008 Apr 26;336(7650):924-926. [Medline] [CrossRef]
- Brozek J, Oxman A, Schünemann H. GRADEpro [Computer program]. Version 3.2 for Windows. URL: http://www.cc-ims.net/revman/other-resources/gradepro [WebCite Cache]
- Man DWK, Soong WYL, Tam SF, Hui-Chan CWY. Self-efficacy outcomes of people with brain injury in cognitive skill training using different types of trainer-trainee interaction. Brain Inj 2006 Aug;20(9):959-970. [Medline] [CrossRef]
- Man DWK, Soong WYL, Tam SF, Hui-Chan CWY. A randomized clinical trial study on the effectiveness of a tele-analogy-based problem-solving programme for people with acquired brain injury (ABI). NeuroRehabilitation 2006;21(3):205-217. [Medline]
- Kukafka R, Lussier YA, Eng P, Patel VL, Cimino JJ. Web-based tailoring and its effect on self-efficacy: results from the MI-HEART randomized controlled trial. 2002 Presented at: Proc AMIA Symp p. 410-414. [Medline]
- Wangberg SC. An Internet-based diabetes self-care intervention tailored to self-efficacy. Health Educ Res 2008 Feb;23(1):170-179 [FREE Full text] [Medline] [CrossRef]
- Robinson TN. Community health behavior change through computer network health promotion: preliminary findings from Stanford Health-Net. Comput Methods Programs Biomed 1989;30(2-3):137-144. [Medline] [CrossRef]
- Williams GC, Lynch M, Glasgow RE. Computer-assisted intervention improves patient-centered diabetes care by increasing autonomy support. Health Psychol 2007 Nov;26(6):728-734. [Medline] [CrossRef]
- Warmerdam L, van Straten A, Cuijpers P. Internet-based treatment for adults with depressive symptoms: the protocol of a randomized controlled trial. BMC Psychiatry 2007;7(1):72 [FREE Full text] [Medline] [CrossRef]
- Cousineau TM, Green TC, Corsini E, Seibring A, Showstack MT, Applegarth L, et al. Online psychoeducational support for infertile women: a randomized controlled trial. Hum Reprod 2008 Mar;23(3):554-566 [FREE Full text] [Medline] [CrossRef]
- Gollings EK, Paxton SJ. Comparison of internet and face-to-face delivery of a group body image and disordered eating intervention for women: a pilot study. Eat Disord ;14(1):1-15. [Medline] [CrossRef]
- Hill W, Weinert C, Cudney S. Influence of a computer intervention on the psychological status of chronically ill rural women: preliminary results. Nurs Res 2006;55(1):34-42 [FREE Full text] [Medline] [CrossRef]
- Hirai M, Clum GA. An Internet-based self-change program for traumatic event related fear, distress, and maladaptive coping. J Trauma Stress 2005 Dec;18(6):631-636. [Medline] [CrossRef]
- Homko CJ, Santamore WP, Whiteman V, Bower M, Berger P, Geifman-Holtzman O, et al. Use of an internet-based telemedicine system to manage underserved women with gestational diabetes mellitus. Diabetes Technol Ther 2007 Jun;9(3):297-306. [Medline] [CrossRef]
- Lorig KR, Laurent DD, Deyo RA, Marnell ME, Minor MA, Ritter PL. Can a Back Pain E-mail Discussion Group improve health status and lower health care costs?: A randomized study. Arch Intern Med 2002 Apr 8;162(7):792-796 [FREE Full text] [Medline] [CrossRef]
- Lorig KR, Ritter PL, Laurent DD, Plant K. Internet-based chronic disease self-management: a randomized trial. Med Care 2006 Nov;44(11):964-971. [Medline] [CrossRef]
- Lorig KR, Ritter PL, Laurent DD, Plant K. The internet-based arthritis self-management program: a one-year randomized trial for patients with arthritis or fibromyalgia. Arthritis Rheum 2008 Jul 15;59(7):1009-1017 [FREE Full text] [Medline] [CrossRef]
- Nguyen HQ, Donesky-Cuenco D, Wolpin S, Reinke LF, Benditt JO, Paul SM, et al. Randomized controlled trial of an internet-based versus face-to-face dyspnea self-management program for patients with chronic obstructive pulmonary disease: pilot study. J Med Internet Res 2008;10(2):e9 [FREE Full text] [Medline] [CrossRef]
- Ross SE, Moore LA, Earnest MA, Wittevrongel L, Lin CT. Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial. J Med Internet Res 2004 May 14;6(2):e12 [FREE Full text] [Medline] [CrossRef]
- Tuil WS, Verhaak CM, Braat DDM, de Vries Robbé PF, Kremer JAM. Empowering patients undergoing in vitro fertilization by providing Internet access to medical data. Fertil Steril 2007 Aug;88(2):361-368. [Medline] [CrossRef]
- Zutz A, Ignaszewski A, Bates J, Lear SA. Utilization of the internet to deliver cardiac rehabilitation at a distance: a pilot study. Telemed J E Health 2007 Jun;13(3):323-330. [Medline] [CrossRef]
- Anderson RM, Funnell MM, Fitzgerald JT, Marrero DG. The Diabetes Empowerment Scale: a measure of psychosocial self-efficacy. Diabetes Care 2000 Jun;23(6):739-743 [FREE Full text] [Medline] [CrossRef]
- Schwarzer R, Jerusalem M. Generalized Self-Efficacy scale. In: Weinman J, Wright S, Johnston M, editors. Measures in Health Psychology: a User's Portfolio. Causal and Control Beliefs. Windsor, England: NFER-NELSON; 1995:35-37.
- Sherer M, Maddix J, Mercandante B, Prentice-Dunn S, Jacobs B, Rogers R. The Self-Efficacy Scale: Construction and validation. Psychological Reports 1982;51:663-671.
- Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav 1978 Mar;19(1):2-21. [Medline] [CrossRef]
- Rosenberg M. Society and the Adolescent Self-Image. Princeton, NJ: Princeton University Press; 1965.
- Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change outcomes. J Med Internet Res 2004 Nov 10;6(4):e40 [FREE Full text] [Medline] [CrossRef]
- Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [Medline] [CrossRef]
- Eysenbach G, Kummervold PE. "Is Cybermedicine Killing You?"--The story of a Cochrane disaster. J Med Internet Res 2005;7(2):e21 [FREE Full text] [Medline] [CrossRef]
- Bandura A. Self-efficacy: The Exercise of Control. New York, NY: Freeman; 1997.
- Murray E, Burns J, See TS, Lai R, Nazareth I. Interactive Health Communication Applications for people with chronic disease. Cochrane Database Syst Rev 2005(4):CD004274. [Medline] [CrossRef]
- Kuhl EA, Sears SF, Conti JB. Internet-based behavioral change and psychosocial care for patients with cardiovascular disease: a review of cardiac disease-specific applications. Heart Lung 2006;35(6):374-382. [Medline] [CrossRef]
- Aujoulat I, Marcolongo R, Bonadiman L, Deccache A. Reconsidering patient empowerment in chronic illness: a critique of models of self-efficacy and bodily control. Soc Sci Med 2008 Mar;66(5):1228-1239. [Medline] [CrossRef]
|ABI: acquired brain injury|
|CBT: cognitive behavioral therapy|
|COPD: chronic obstructive pulmonary disease|
|CRQ: Chronic Respiratory Questionnaire|
|DES: Diabetes Empowerment Scale|
|GSE: General Self-Efficacy Scale|
|ICSI: intracytoplasmic sperm injection|
|IVF: in vitro fertilization|
|MD: mean difference|
|PST: problem-solving treatment|
|QAS: quality assessment scale|
|RCT: randomized controlled trial|
|RSE: Rosenberg Self-esteem Scale|
|SMD: standardized mean difference|
|WHO: World Health Organization|
|Edited by G Eysenbach; submitted 06.07.09; peer-reviewed by R Scholten, T Cousineau, W Tuil; comments to author 14.10.09; revised version received 09.11.09; accepted 30.11.09; published 24.06.10|
Please cite as:
Samoocha D, Bruinvels DJ, Elbers NA, Anema JR, van der Beek AJ
Effectiveness of Web-based Interventions on Patient Empowerment: A Systematic Review and Meta-analysis
J Med Internet Res 2010;12(2):e23
END, compatible with Endnote
BibTeX, compatible with BibDesk, LaTeX
RIS, compatible with RefMan, Procite, Endnote, RefWorks
Add this article to your Mendeley library
Add this article to your CiteULike library
Copyright©David Samoocha, David J Bruinvels, Nieke A Elbers, Johannes R Anema, Allard J van der Beek. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.06.2010
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.