This study was conducted in close collaboration with the Regional Public Health Service (RPHS) in the northern part of the Dutch region, Limburg. All people employed by the municipalities in this region were invited by the RPHS to participate in an Internet-delivered CT lifestyle program developed by our research group [41,42]. Invitations for this program were placed on the intranet of all municipalities and were also directly emailed to employees. The invitation contained a uniform resource locator (URL) that directed interested participants directly to the program. The program provided employees the opportunity to receive free-of-charge CT feedback about their current health behavior (physical activity, fruit and vegetable intake, alcohol consumption, and smoking) and assistance in changing these health behaviors.

Participants who logged in to the program to obtain CT advice were sent an email prompting them to re-use the program. Re-use was encouraged to allow participants to monitor their behavior change and to obtain iterative CT advice aimed at behavior change and relapse prevention. Furthermore, participants were offered an opportunity to obtain CT advice regarding an additional health behavior. Finally, re-use was stimulated to keep participants informed about new content added to the program website.

A 2×3 design was used to test the effect of 2 factors: prompt content (standard and standard+) and prompt timing (2, 4, and 6 weeks). The standard prompt (SP) contained a message reminding people about their previous visit to the program and invited them to re-use the program to monitor their progress and obtain additional feedback. The second version of the prompt (SP+) contained standard content complemented with a message alerting people to new content added to the intervention website. Participants received an email prompt after 2, 4, or 6 weeks. In addition to these 6 experimental conditions, a control condition was included, no prompt (NP). People allocated to the control condition did not receive an additional prompt and were only encouraged at baseline to re-use the program. To participate in the program, participants had to register by using a personal log-in code and password. Immediately after registration for the program, participants were randomly allocated to 1 of the 7 study conditions. Randomization occurred at the respondent level by means of a computer software randomization device. Data for the present study were collected from March to July 2011.

People who were allocated to the experimental conditions of the study received an email prompting them to re-use the program. Periodic prompts should be distinguished from reminders, which are also often used in Internet-delivered interventions. Sending reminders is a technique that is used to increase response rates in online data collection by proactively stimulating participation among nonresponders and is used to prevent drop-out attrition [21,43]. Periodic prompts, on the other hand, are used to boost re-use of the intervention content by approaching all participants and are used to prevent non-usage attrition [21]. All email prompts used in the present study contained standard content. This standard email opened with a personalized greeting and reminded people about their first visit to the program. Subsequently, people were invited to re-use the program to obtain information about their current health status and to monitor their progress. Participants were also given the opportunity to receive additional iterative health advice on the health behavior(s) selected at baseline or on a new behavior. Finally, to facilitate logging in to the program, the email also contained details about their personal log-in information (username and password). The email concluded with greetings from the research team and contact information. Half of all people in the experimental conditions received an email that also contained additional content (SP+). This additional content consisted of a preview of new information that was placed on the program website since they last visited it. This information referred to nutrition and provided examples of healthy food alternatives that were available for that current season (eg, spring/summer).

The CT program integrated established CT programs tested and proven to be effective in randomized controlled trials for increasing smoking cessation, promoting the intake of fruit and vegetables, increasing the level of physical activity, and reducing the consumption of alcohol [44-48]. The program used a dual approach to guide people toward behavior change. First, awareness of participants’ current health behavior status was increased by comparing their status to the Dutch public health guidelines set for these health behaviors, such as being moderately physically active for 30 minutes at least 5 days a week, eating 2 pieces of fruit per day, eating 200 grams of vegetables per day, not drinking more than 1 (women) or 2 (men) glasses of alcohol a day, and not smoking. Second, assistance was provided in changing participants’ health behavior by using CT modules available per behavior. The modules used a fixed, gradual approach consisting of 4 steps, guiding people toward behavior change based on the Integrated Model for exploring motivational and behavioral change (I-Change Model) [49]. Focus was on pros and cons of the desired behavior (step 1), the role of significant persons in the direct environment (step 2), preparatory plans assisting people to start changing their behavior (step 3), and coping plans to help them overcome difficult situations and prevent relapse (step 4). Within the modules, all health advice was adapted to the individual’s characteristics by considering demographic, behavioral, and cognitive characteristics [50-52]. The Internet-delivered CT lifestyle program is described in more detail elsewhere [41,42].

The program was embedded in a website designed for the current study. This website entailed general information considering a healthy lifestyle and the selected health behaviors. Furthermore, the website provided specific information regarding the project, contained a direct link to the CT program, and provided background information regarding the study and the research team. During the study, new information (eg, advice, supporting messages, recipes, and facts) was added to the website.

Participants in the experimental conditions (SP and SP+) received an email prompting them to re-use the intervention. Participants had to take 2 steps to re-use the program in order to self-monitor their level of behavior change and obtain additional CT advice. The first was clicking on the URL provided in the email prompt. Clicking on the URL was tracked by referral ID codes that were integrated in the URL. The second step was logging in to the CT program after arriving on the program website. Log-ins were objectively monitored during a 2- week period and log-in dates were compared to baseline dates to determine revisits.

To describe characteristics of program visitors, age, gender, and educational level of participants was assessed (1/low: no education or lower vocational school; 2/medium: secondary vocational school or high school; 3/high: higher professional education or university). Health behavior status consisted of information regarding the 5 key behaviors. Physical activity was measured by the Short Questionnaire to Assess Health-enhancing physical activity (SQUASH) [53] and guideline adherence was calculated following procedures used by Ainsworth et al (2000) [54]. Fruit consumption was measured by using a 4-item Food Frequency Questionnaire (FFQ) assessing weekly amount of fruit and fruit juice intake [55], whereas vegetable consumption was measured using a 4-item FFQ assessing the weekly amount of consumed boiled or baked vegetables as well as salad or raw vegetables [55]. The consumption of alcohol was measured by the Dutch Quantity-Frequency-Variability (QFV) questionnaire [56]. Finally, smoking status was assessed by asking participants whether they smoked, what they smoked (cigarettes, cigars, pipe tobacco), and how much they smoked per day (cigarettes) and per week (cigars/packets pipe tobacco) [57]. For each health behavior, a new variable was created to indicate whether participants met the Dutch guidelines provided for these behaviors (0=no; 1=yes).

General descriptive statistics were calculated to describe characteristics of program visitors, as well as main findings concerning adherence to the public health guidelines. Baseline differences between the intervention groups were calculated using the Chi-square test for dichotomous and categorical variables and 1-way analysis of variance (ANOVA) for continuous variables. Finally, logistic regression analyses were conducted. There were 2 dependent variables: whether participants clicked on the URL (0=no; 1=yes) and whether participants logged in to the CT program (0=no; 1=yes). Prompt content (dummy coded, SP=0; SP+=1) and prompt timing (dummy coded with 2 weeks as the reference category and 4 weeks as the reference category) and the interactions between these variables were used as predictors in the initial model of each dependent variable. Furthermore, because women and people with a higher educational level and age are more likely to use Internet-delivered lifestyle interventions [17,58], age, gender, and educational level were included in the models as possible covariates. An alpha of .05 was used to indicate statistical significance. All statistical analyses were done with the program SPSS 17.0 (SPSS Inc, Chicago, IL, USA).

There was no significant interaction between prompt content and prompt timing (Figure 1) with regard to clicking on the URL. Therefore, interaction terms were excluded from the remaining models and only main effects are reported.

Analyses of main effects indicated that there was a significant effect of prompt timing (χ² ₂ = 15.2, P = <.001). Participants who received an email prompt 2 weeks after their first visit, clicked on the URL significantly more often compared with participants that received the prompt after 4 weeks (odds ratio [OR] 3.069, 95% CI 1.392-6.765, P=.005) and after 6 weeks (OR 4.471, 95% CI 1.909-10.471, P=.001). There was no significant difference in reaction to the email prompt between participants receiving the prompts after 4 weeks, compared with participants receiving the prompts after 6 weeks. Also, no main effects of prompt content could be detected (see Table 1).

With regard to logging in to the program, no significant interaction between prompt content and prompt timing was found (Figure 2). Therefore, interaction terms were again excluded from the remaining models and only main effects are reported.

With regard to logging in to the program, analyses of main effects indicated that there was a significant effect of prompt timing (χ² ₂ = 16.5, P<.001). Participants receiving an email prompt 2 weeks after their first visit, logged in to the program significantly more often compared to participants receiving the prompt after 6 weeks (OR 16.356, 95% CI 2.071-129.196, P=.008). There was, however, no significant difference between participants receiving the prompts after 4 weeks compared with participants receiving the prompts after 2 or 6 weeks. With regard to prompt content, a trend was observed. Participants receiving a prompt with additional content were more likely to log in to the program compared to participants who received a prompt with standard content. However, this result was not statistically significant (OR 2.286, 95% CI 0.892-5.856, P=.09).

Finally, all experimental groups were compared with the control condition (NP) that did not receive an additional email prompt to boost revisits to the intervention. Results from this analysis revealed that participants who received an email prompt containing additional content after 2 weeks were significantly more likely to log in to the program (OR 6.059, 95% CI 1.195-30.726, P<.001).

The current study primarily focused on the effectiveness of using a single email prompt with regard to clicking on the URL and logging in to the program. Combined with results from our previous study [17], important information regarding the overall effectiveness of using a single email prompt as well as the optimal prompt content and timing was obtained. To our knowledge, this is one of the first studies that systematically studied optimal content and timing of a single email prompt used to boost re-use of a multiple CT lifestyle program offered through the Internet. Although these results suggest that sending a single email prompt is an effective strategy to boost revisits to interventions using multiple computer-tailoring, this is only a first step, and the findings should be interpreted keeping several limitations in mind. Firstly, the current study focused solely on sending an email prompt on 1 occasion, namely after 2, 4, or 6 weeks. However, achieving health behavior change is a complex and enduring process that requires continuous guidance [22,26]. Repeated visits to interventions using multiple computer-tailoring are imperative to ensure notification of the entire intervention content and involvement in its effective components [27,28] and must, therefore, be encouraged. Additional research should go one step further and focus on timing intervals and prompt frequency. Prompting for revisits shortly after a first visit indeed stimulated revisits to the program. However, repeatedly sending prompts at a certain interval to boost multiple revisits might have different or even reverse effects because responses to multiple prompts tend to gradually decline over time [62]. Secondly, effectiveness of the use of an email prompt largely depends on the degree to which the content is actually read and processed by receivers. Within the current study, no objective measure of the degree to which emails were read was obtained. Therefore, results on prompt content should be interpreted with caution. Future research should put additional effort in assessing the degree to which email content is actually read and processed by participants. Furthermore, additional research is needed to investigate the effects of using periodic prompts on behavior change and to investigate whether these effects remain present over a sustained period of time. Thirdly, the sample size for the present study was relatively small. As a result, the study might be relatively underpowered to study interaction effects and to indicate significant main effects. Furthermore, regression analyses resulted in several substantial confidence intervals. Although these results should be interpreted with caution, they still provide valuable information regarding the effectiveness of timing and content of a single email prompt as a strategy to boost program revisits. Fourthly, a related limitation is the representativeness of the study sample. This sample does not provide a good cross-section of the general Dutch population [63] because participants were mainly female, middle-aged, and medium to highly educated. This may limit possibilities for generalizing our results to the general population. However, previous studies repeatedly indicated that Internet-delivered CT programs tend to predominantly reach women and people who are older and higher educated [17,58,64-67]. As a consequence, the current sample corresponds to the subgroup of people known to be reached by Internet-delivered CT programs, although it does not represent the general population. Therefore, the obtained results are valuable in the context of CT lifestyle programs that are offered through the Internet.

We found that using relatively short prompt timing (2 weeks) resulted in more positive effects compared to using a longer time period. This effect not only referred to clicking on the URL, but also to the proportion of log-ins to the program. Although a trend was observed about the effectiveness of adding a preview of new website content to a standard prompt, this only referred to persuading people to log in to the program. The findings of this study underline the importance of sending an email prompt relatively shortly after a first visit to the program. Furthermore, it is important to further focus on prompt content because the addition of a peripheral cue may add to the prompt’s effectiveness.