As the detailed protocol for this study has been published elsewhere [13], only summary information will be provided here (see Multimedia Appendix 1 for additional screenshots of the intervention). This study is the second phase of the substantive WALK 2.0 project and builds on an earlier RCT [15,16], which rigorously tested the efficacy of the interventions described here [11]. The Western Sydney University Human Research Ethics Committee granted ethical approval for this study (H8767). This trial has been registered at the Australian New Zealand Clinical Trials Registry (ACTRN12611000253909).

Adults 18 years of age or older spontaneously signing up for the freely available and Web-based 10,000 Steps program [17], which attracts over a 1000 new members per month [18], were asked during the registration process whether they wanted to participate in a research study from November 2012 to June 2014. The 10,000 Steps project has been funded since 2001 through Queensland Health, one of the Australian State Ministries of Health. The project is well known through media and marketing events in Australia, particularly in Queensland, with over 70% program awareness in population-based surveys. If potential participants agreed to participate in research, they received more information about the study, were screened online for eligibility, provided informed consent, and completed a brief baseline survey. Using a computer-generated algorithm, they were randomized to receive access to one of two intervention websites: a Web 1.0 intervention, which was the 10,000 Steps website they were originally signing up for, or a Web 2.0 intervention, which was the newly developed WALK 2.0 website. For technical reasons, participants were randomized before completing the baseline measures, however, they only gained access to intervention materials after completing the baseline assessment. Follow-up outcomes were assessed 3 and 12 months postbaseline using online questionnaires; participants were invited by email and received up to three reminders. All actions, from study invitation to completion, were fully automated with no interaction from the research team at any point. The research team also provided no instructions as to how the interventions should be used and there was no predefined intervention duration. However, even though the aim of reaching 10,000 steps a day was implicit, participants were provided with the Australian Physical Activity Guidelines and the websites were designed to encourage self-monitoring and interaction on a daily basis for as long as possible—participants had an option to receive a daily reminder to use the websites. No pedometers were provided. Exclusion criteria were the following: being under 18 years of age; seeking to participate in a 10,000 Steps Workplace Challenge; having been a participant in the WALK 2.0 RCT; and having a medical condition that prevents them from increasing physical activity, assessed through the Physical Activity Readiness Questionnaire [19].

 The trial was powered to detect a 4% between-group difference in the prevalence of sufficient physical activity, as defined by the Australian Physical Activity Guidelines, between the Web 1.0 and Web 2.0 groups. To achieve this aim with 80% power and an alpha level of .05, a minimum of 1034 participants per group were needed [13].

All statistical analyses were conducted using SAS version 9.4 (SAS Institute Inc). Differences between participants with complete and missing data were compared using t tests or Pearson chi-square tests. Analysis of covariance (ANCOVA) was used to test for differences between treatment groups at 3 months in physical activity, quality of life, and BMI; baseline physical activity, BMI, quality of life levels, and confounding variables were included in the models as covariates. Logistic regression was used to estimate between-group differences in the proportion of participants who achieved sufficient physical activity. Results are presented both for those with complete data at baseline and 3 months (completer analyses), as well as those with missing data, following intention-to-treat principles. Multiple imputation was applied to deal with missing data, under the missing-at-random assumption, using the chained equations method. Rubin’s method was used to pool the treatment effects using 25 imputed datasets, as the fraction of missing data was high [31]. To analyze between-group differences in website engagement and retention, t tests were used. A proportional hazards regression model was used to estimate between-group differences in time from randomization to nonusage; Kaplan-Meier estimates of the proportion remaining active (the survival distribution) are also presented [32]. Due to very small participant numbers, 12-month data were not included in any of the analyses. The significance level was set at P<.05.

A participant flowchart is provided in Figure 1. After automatically screening out ineligible people, 10,673 people were invited and 3480 indicated an interest in participating. After eligibility checks, providing informed consent, and website registration, 1328 people completed all baseline measures. Out of 1328 participants, 224 (16.87%) completed the 3-month assessment and 77 (5.80%) completed the 12-month assessment. Table 1 presents participant demographics. At baseline, the majority of participants were female (1095/1328, 82.45%), were 44 years of age or under (818/1328, 61.60%), were overweight or obese (849/1328, 63.93%), had a higher education (693/1328, 52.18%), were full-time employed (771/1328, 58.06%), had a professional or white-collar job (879/1328, 66.19%), and participated in sufficient physical activity (757/1328, 57.00%). There were no significant between-group differences in participant characteristics at baseline; however, several between-group differences were observed among those who completed the 3-month assessment and those who did not. There was significantly greater retention among those who were Web 1.0 group participants, male, aged 45 years or older, and not obese, as well as those having a higher education, professional occupation, and higher income.

The physical activity outcomes are presented in Table 2. In the intention-to-treat analysis, a significant difference between groups was found: participants randomized to the Web 2.0 group increased physical activity by 92.8 minutes per week more compared to those in the Web 1.0 group (P=.005; Cohen d effect size=0.29). In the completer analysis, participants in the Web 2.0 group increased physical activity by 56.6 minutes per week more compared to those in the Web 1.0 group, however, this difference was not significant (P=.20; Cohen d effect size=0.24).

At baseline, 57.00% (757/1328) of participants in both groups engaged in sufficient physical activity. At the 3-month time point, 77% (62/80) of Web 2.0 participants and 71.5% (103/144) of Web 1.0 participants engaged in sufficient physical activity. A significant between-group difference in favor of the Web 2.0 group was observed in the intention-to-treat analysis (Relative Risk [RR]=1.11, 95% CI 1.01-1.21; t_64.5=2.19, P=.03), but not in the completer analysis (RR=1.03, 95% CI 0.89-1.18; Z=0.36, P=.71).

There was no effect of the intervention on most quality of life variables (see Table 2), except for the physical functioning domain: a significant improvement (3.6 units) was observed in the Web 2.0 group compared to the Web 1.0 group (P<.001).

BMI reduced over time in both groups (see Table 2) and significant between-group differences were observed for both the intention-to-treat analysis (in favor of the Web 2.0 group: change in BMI=-1.03 kg/m², P=.002) and the completer analysis (in favor of the Web 1.0 group: change in BMI=-0.58 kg/m², P=.002).

Differences in terms of website usage are shown in Table 3. Participants used the websites on average 3 minutes per week and logged steps for approximately 10 days. There were no significant between-group differences with regard to time on site, entering steps without comments, and the SUS. However, total number of visits (9.0 [19.3] vs 5.7 [17.9], P=.002) and average number of visits (0.7 [1.6] vs 0.4 [1.4], P=.002) were significantly higher in the Web 2.0 group, as well as the time between first and last visit (3.6 [3.7] vs 2.2 [2.9], P<.001). Total days (0.7 [5.9] vs 0.2 [1.5], P=.03) and average days (0.05 [0.45] vs 0.01 [0.10], P=.02) with a step entry comment were also significantly higher in the Web 2.0 group. A Kaplan-Meier survival plot shows the proportion of participants that remained using the website for each week of the study (see Figure 2). Only 21.99% (292/1328) of participants were still using either website after 2 weeks (n=292 summed for both groups) and 6.55% (87/1328) after 10 weeks. The between-group difference in time-to-nonusage attrition was not statistically significant (Hazard Ratio=0.97, 95% CI 0.86-1.09; P=.57). Self-reported use and usefulness of the features available on both websites are reported in Table 4. More participants in the Web 2.0 group used features that were present, or similar, on both websites compared to the Web 1.0 group. Web 2.0 participants also rated the usefulness of these features more highly compared to Web 1.0 participants. Many participants in both groups did not use some of the interactive features or indicate that they were very useful.

The aim of this study was to compare physical activity behavior of individuals using a traditional Web 1.0 physical activity website with those using a more interactive and social Web 2.0 physical activity website in real-world settings. The primary hypothesis was confirmed: those in the Web 2.0 group did display significantly higher levels of physical activity at 3 months compared to those in the Web 1.0 group. The secondary hypotheses were partially confirmed: BMI significantly decreased, engagement was significantly higher in some variables (eg, website visits), and quality of life significantly improved in one variable (ie, physical functioning) in the Web 2.0 group compared to the Web 1.0 group at 3 months. However, there were no between-group differences for several other engagement variables (eg, logging steps and nonusage attrition) and most quality-of-life variables. This study was the first to demonstrate the importance of using Web 2.0 features in Web-based physical activity interventions in a real-life setting. These outcomes are strengthened by not finding between-group differences in website usability (ie, SUS score), indicating that outcomes were not influenced by factors such as user friendliness. Interestingly, the use and usability of features that were present, or similar, on both websites were higher in the Web 2.0 website compared to the Web 1.0 website. While, from this research, it is not possible to explain this difference given the similar SUS scores, it may in part explain why the Web 2.0 website performed better on several behavioral and engagement outcomes.

While there is an abundance of studies examining the effectiveness of physical activity promotion websites in controlled conditions [33], few studies have examined the use and effectiveness of physical activity websites in natural and real-life conditions. To our knowledge, only one other study has examined people who spontaneously signed up to an online physical activity intervention and where researchers had no direct contact with study participants: all processes were completed automatically [34]. Wanner et al conducted an RCT with two groups of participants that were actively recruited to be randomized to either a control group or an interactive, computer-tailored, physical activity website; however, they included a third group of spontaneous users from the same website. Significantly larger increases in physical activity were observed in the spontaneous users when compared to the actively recruited groups [34]. That study’s outcomes were similar to ours when comparing findings from our RCT with the ecological trial: physical activity increases reported in this ecological trial (+140 for Web 2.0 group; +40 for Web 1.0 group) are much higher than those observed in the RCT (+45.5 for Web 2.0 group; –1.0 for Web 1.0 group) [11]. The large differences observed in these studies indicate that it is not adequate to rely only on RCT outcomes when preparing Web-based physical activity interventions for dissemination and translation.

Poor engagement with Web-based interventions has often been reported; with regard to that aspect, this study is not unique [33,35]. The ecological trial and the RCT [11], however, were remarkably different in terms of engagement and retention outcomes. While the Web 2.0 group also had better engagement outcomes in the RCT, the overall engagement of participants was much lower in the ecological trial. For example, nonusage attrition in the RCT occurred on average after 35.5 and 25.5 weeks in the Web 2.0 and Web 1.0 groups, respectively, whereas it occurred after a mere 1.5 weeks for both groups in the ecological trial. The RCT website was visited, on average, 3.6 and 1.6 times per week for the Web 2.0 and Web 1.0 groups, respectively, but only 0.7 and 0.5 times per week, respectively, in the ecological trial. While participants in both trials were exposed to identical websites, there are factors that encourage engagement in RCTs, and factors that discourage engagement in ecological trials. In the RCT, participants were required to complete multiple face-to-face visits and they received phone calls, a pedometer, and multiple small financial incentives [11]. These strategies increased participant accountability and resulted in a 60% retention rate at 18 months, which is comparable to other studies in this field [6]. In order to keep the intervention implementation as natural as possible in the ecological trial, no such strategies, other than email reminders, were applied. However, in the ecological trial the process of asking 10,000 Steps visitors to participate in research and obtaining informed consent and baseline assessments may have deterred many participants, given the attrition at each step of the process (shown in Figure 1). These necessary steps may have been detrimental to their ongoing participation, as is demonstrated in a previous analysis of 16,948 spontaneous 10,000 Steps users who did not need to go through multiple screenings to be included in the analyses [18]. Among these 10,000 Steps users, nonusage attrition occurred after 4 weeks and the website was visited 2.4 times per week; these engagement outcomes are remarkably higher than those among ecological trial participants [18]. Finally, in another study, Wanner et al compared spontaneous users with trial participants and also reported large engagement differences: nonusage attrition occurred at 41 weeks in trial participants, but it was zero days in spontaneous users [36]. Collectively, these outcomes indicate that caution should be taken when interpreting engagement outcomes from both RCTs and ecological trials, as neither may be a good reflection of how participants engage with websites in reality. The outcomes further suggest that improved participant retention strategies are required that work well in ecologically valid circumstances [37].

The significant reduction of BMI in favor of the Web 2.0 group was surprising, as the intervention did not focus on weight loss, and the 3-month time frame is reasonably short. However, previous Web-based interventions have demonstrated weight loss, both in interventions focused on either weight loss or physical activity [38,39]. Further, few studies have examined how Web-based physical activity interventions can improve health-related quality of life. A meta-analysis of the impact of non-Web-based physical activity interventions on quality of life observed that the interventions only significantly changed the physical functioning domain of quality of life, and not other domains [40]. This is consistent with the outcomes of this study.

Women dominated participation in this study (82%), however, this was very comparable to other studies. For example, Anderson-Bill et al [41] conducted an online walking program and reported that 83% of participants were female and 75% of spontaneous website users were female in the study by Wanner et al [36]. Lower male participation is also commonly observed in RCTs [5,6] and this study demonstrates that the difficulty in attracting men to health behavior change interventions may even be greater in ecologically valid circumstances. Several significant differences were observed between those who participated at 3 months and those who did not. There was less attrition among those randomized to the Web 1.0 group (10,000 Steps). As all participants were originally signing up to participate in the 10,000 Steps program, it is plausible that many of those who were randomized to the Web 2.0 group (WALK 2.0) were disappointed with their allocation and dropped out for that reason. It is also possible that the increased complexity and interactivity of the Web 2.0 website resulted in higher dropout rates, though this is less likely as there was no significant difference in the SUS score between the websites. Older participants and men were less likely to drop out; this is similar to the study by Wanner et al, who found that among their spontaneous users, men and those of increasing age were more likely to repeatedly use their website [36]. It is also in line with a Web-based physical activity intervention study that found older participants spent more time on the website and changed behavior more than younger participants [42]. Finally, it is not surprising to see lower dropout rates among those with higher education, higher income, and a professional occupation, as it has been widely reported that people with a higher socioeconomic status are more amenable to participating in health behavior change interventions [43,44].

The major strengths of this study were the innovative ecological randomized nature of the study and the comparison of Web 1.0 and Web 2.0 intervention features. The large nonusage and study attrition, however, was an important limitation, making it problematic to analyze 12-month outcomes. This limitation, though, could be considered as a finding that is of interest, as it is a reflection of how Web-based interventions are being used in ecologically valid circumstances; it is not, per se, a reflection of poor study methodology. That said, several methodological limitations are inherent to ecological trials, such as the lack of a true control group and having to resort to less intrusive (ie, self-report) measures to assess outcomes. These methodological concerns can, however, be alleviated by comparing the study outcomes to those of an RCT that applied more rigorous methods [11]. Furthermore, all engagement and retention measures were assessed objectively through the use of website usage statistics. However, it was a limitation of this study that the interventions were only accessible via websites, as the technology landscape has rapidly changed since the study inception and it is now very common to also use mobile phone apps, which are more convenient to access than websites [18].

Web 2.0 intervention features appear to be more effective in increasing physical activity, decreasing BMI, and improving physical functioning (ie, quality-of-life domain) compared to Web 1.0 features in ecologically valid circumstances. While the social and interactive Web 2.0 features increased engagement compared to a traditional Web 1.0 website, website usage was low regardless, especially when compared to our previous RCT [11]. As such, more research is needed to increase our understanding of how people engage with Web-based interventions, both in controlled and ecologically valid circumstances, and how closely this engagement is related to actual behavior change (ie, what is the dose-response relationship?) [45]. Finally, the findings of this study are remarkable in how different they are from the findings observed in controlled conditions in terms of attrition, website usage, and behavior changes observed. This is important, because information obtained through RCTs may not translate well to real-world conditions. Yet almost all of our knowledge in this field is derived from RCTs and this has a major influence on how Web-based interventions are designed, what interventions are deemed effective, and what interventions are being disseminated and implemented. As such, there is an urgent need for more ecological trials and implementation studies.