To examine differences in dropout attrition and evaluation between participants with different educational levels, we used a convenience sample of participants from 7 Web-based computer-tailored intervention studies that were conducted in the past years (2010-2014) at the Department of Health Promotion of Maastricht University in the Netherlands.

The studies were RCTs to evaluate interventions that used computer-tailored techniques to improve diverse health behaviors. The study “Master Your Breath” (MYB) focused on increasing physical activity and smoking cessation among chronic obstructive pulmonary disease (COPD) patients and people at risk for COPD. The 3 studies “Stay Quit for You” (SQ4U), “Support to Quit” (STQ), and “Personal Advice in Stopping smoking” (PAS) focused on smoking cessation. “Weight in Balance” (WIB) aimed to prevent obesity by targeting physical activity and energy intake. The study “My Healthy Behavior” (MHB) targeted the following health behaviors: physical activity, fruit and vegetable consumption, alcohol intake, and smoking. The study “Alcohol-Everything within the Limits” (AEL) focused on moderate alcohol intake and is the only study that was not carried out in the Netherlands but in Germany.

All selected studies made use of the I-Change model [53,54], which postulates that the behavior change process has at least 3 phases: awareness, motivation, and action. The first factor is determined by factors such as behavioral awareness, knowledge, and risk perceptions. The second phase is determined by attitudes, social influence beliefs, and self-efficacy expectations, and results in a certain intention to perform a particular behavior. The third factor is determined by self-efficacy, action planning, skills, and barriers. The tailored feedback messages of the studies included in this paper have a strong focus on inter alia these determinants. A detailed description of these RCTs and the related publications can be found in Table 1.

In all 7 studies, educational level was assessed by asking participants about their highest completed level of education. In-line with national guidelines, educational level was categorized into 3 groups: lower (1=no education, primary or lower vocational school), middle (2=secondary vocational school or high school), and higher (3=higher professional education or university) educational level [66].

All studies included a process evaluation assessment to evaluate the intervention among participants within the intervention group. Participants were asked to evaluate the tailored feedback and the intervention. The process evaluation assessments of the 7 studies included had one item in common that asked participants to grade the intervention that they participated in: “Please evaluate the intervention with a school grade from 1 to 10” (10=highest grade, 1=lowest grade according to the Dutch school grading system; AEL: 15=highest grade, 1=lowest grade, which is in-line with the German school grading system).

To assess dropout attrition, participants who completed the baseline measurement but did not complete the follow-up measurement were characterized as dropouts (1=dropout; 0=completed follow-up). We assessed dropout attrition within differently educated participants for each follow-up measurement separately. Furthermore, we used the last available evaluation moment as predictor of dropout for the following measurement. Table 1 gives an overview of the specific follow-up moments per study.

All analyses were performed with SPSS 20.0 (IBM Corp, Armonk, NY, USA). Descriptive statistics were used to describe sample characteristics. Per study, a logistic regression analysis was conducted to examine if dropout rates differed for each educational level. To control for multiple testing, the Benjamini and Hochberg linear step-up method was used for each study [67,68]. With the use of an Excel template, the adjusted significance levels were calculated [69].

Differences between the educational levels with regard to evaluation of the Web-based computer-tailored interventions were analyzed by means of ANOVAs and Tukey honestly significant difference (HSD) tests. Control groups were excluded from analysis with regard to evaluation of the intervention because they could not evaluate it.

To be able to give a more general picture of whether lower and higher educated participants from the intervention groups evaluated Web-based computer-tailored interventions differently, the Exploratory Software for Confidence Intervals (ESCI) Excel template [70] was used for pooling the data by means of a meta-analysis (Table 1). The meta-analysis used a random effect model and gave an impression of the overall differences for intervention evaluation between lower and higher educated participants (ie, by subtracting the evaluation of the most different groups, the lower educated participants from the higher educated participants). In one study (MHB), the evaluation item was assessed at multiple follow-up measurements; in this case, we included only the last follow-up measurement [71].

Finally, logistic regression analyses were conducted to examine if dropout was predicted by evaluation in 4 of the 7 interventions among participants with different educational levels. We excluded the studies WIB, PAS, and MYB from this analysis because their evaluation assessment took place during the last follow-up measurement; therefore, it was not possible to assess evaluation as a predictor of dropout in these studies. To identify possible interaction effects, an interaction term of educational level and evaluation was used in the regression model. If this interaction term was significant, then analyses were conducted separately per educational level. Analyses were corrected for age and gender. A P value of .05 was used as the significance level for all analysis.

Table 2 shows the educational level, mean age, and gender distribution of the participants within the 7 selected studies at baseline.

Table 3 shows the results of the dropout analyses with regard to the educational level for each study, each follow-up measurement including the dropout rates, and study group in detail with high education as the reference group.

After correction for multiple testing, significantly higher dropout rates were found within 3 studies (MHB, STQ, WIB) among lower educated participants compared to higher educated ones. Furthermore, in these 3 studies, dropout attrition was also significantly higher among middle educated participants in comparison with higher educated participants. In 4 of 7 studies (AEL, MYB, PAS, SQ4U), no difference in dropout with regard to educational level was found.

Table 4 presents differences between the educational groups with regard to evaluation of the Web-based computer-tailored interventions in detail. In 3 of 7 studies (AEL, MHB, MYB), lower educated participants evaluated the intervention significantly higher compared to their counterparts. In one study (WIB), lower educated participants evaluated the intervention less positively compared to middle educated participants.

Furthermore, the meta-analysis of the 7 studies comparing evaluation of lower and higher educated participants indicated that participants with a lower educational level evaluated the interventions significantly more positively compared to highly educated participants (OR 0.28, 95% CI –0.54 to 0.04, P<.001) (see Figure 1). Nevertheless, the meta-analysis revealed presence of a moderate level of heterogeneity (I ²=66.10%) [74,75], which indicates variation across the studies.

For the 4 studies that evaluated the intervention before the follow-up measurements, a significant interaction between education and evaluation regarding dropout at follow-up was not found (Table 5). Only within the MHB study was a positive association found between the intervention evaluation and educational level. Participants with a middle educational level were more likely to dropout than participants with a higher educational level.

The first aim of this study was to evaluate whether participants with a lower educational level have higher dropout attrition from Web-based computer-tailored studies than participants with a medium or high educational level. In 3 of these studies, lower and middle educated participants dropped out more frequently compared to higher educated participants.

A possible explanation for the higher dropout rates may be that lower educated participants tend to use written health information more often [76] and spend less time online seeking health information [77,78]. It could be possible that they lose interest in the intervention sooner, which causes them to drop out of the study.

Also, the fact that lower educated people have an unhealthier lifestyle [30-32] might play a role in dropout. Due to tailoring, participants with an unhealthier lifestyle in multiple health behavior interventions received more recommendations to change their health behavior(s) and this has been found to decrease participants’ motivation to change [79]. It might be possible that lower educated participants started the intervention with the aim to change their health behavior, but that receiving information about extensive required changes decreased their self-efficacy to be able to change [80] and could subsequently have decreased their motivation to participate. Another explanation could be that lower educated participants might have been less likely to change their behavior and, thus, may have perceived the recommendations as less feasible, which caused them to drop out of the study [28,81,82]. This could have caused not only usage and nonusage attrition, but also dropout attrition because these 2 kinds of attrition seem to be related [46].

Moreover, lower educated people might be less familiar with Web-based computer-tailored interventions [83,84] and that might result in lower confidence in the effectiveness of those interventions (ie, lower perceived efficacy) and, in turn, could cause an increase in dropout [85,86]. Although these are reasons for nonusage attrition (not using the intervention), it seems convincing that this correlates with dropout attrition because participants who did not evaluate the intervention positively might have little interest in completing follow-up measurements [51].

All participants were asked to complete long questionnaires and received tailored feedback, which must be cognitively processed and requires intensive cognitive performance. Lower educated adults have been shown to have a lower level of health literacy [84,87,88]. They have more difficulties processing new information and this could cause ego depletion [89,90]. Ego depletion may, in turn, reduce the willingness to participate any longer within the study.

Some studies found that dropout attrition could be increased by sending reminders and prompts [78,91-93]. Further research is necessary to evaluate if this is also effective for people with different educational levels.

Against our initial expectation that lower educated participants might evaluate the interventions less positively, we found that lower educated participants evaluated the intervention in 3 of 7 studies more positively compared to their higher educated counterparts. This might be explained by the finding that highly educated people make more use of the Internet as health information resource, whereas these interventions might be newer and more interesting for lower educated people. A review supported this assumption because it shows that people with a high educational level may make more intensive use of several sources (eg, people form their social network, mass media, health professionals) to gain health-related information compared to lower educated people and they might read the received information more superficially [94]. This might result in less elaboration of the messages and a lower evaluation regarding the novelty of the messages. Due to the use of several resources, highly educated participants also rely less on online information and have lower levels of trust in them, which may negatively influence their evaluation of the intervention [95].

Evaluation was not a significant predictor of dropout at follow-up in any of our studies. This suggests that other factors must be important in explaining why participants did not return to the study for follow-up questionnaires. Dropout analysis performed within these studies has shown that a lower educational level, unhealthy lifestyle, low intention to change the behavior, and low self-efficacy were predictors of dropout [20,96,97].

First, the only item all studies had in common concerning the evaluation of the interventions was an overall grade participants assigned to the intervention. Although we can assume that this item gives an overall impression of participants’ evaluation, it might be that participants with different educational levels liked and disliked different aspects of the intervention (eg, layout, provided information, or personal relevance), which was not reflected in this overall grade. However, the evaluation of these different aspects was not equally assessed in all 7 studies. Furthermore, it is possible that participants who did not like the intervention dropped out before completing the evaluation item. In this study, we included only those participants that completed the evaluation item and assessed follow-up at the subsequent measurement.

Second, all interventions were based on the I-Change model and targeted the same social cognitive determinants to change behavior, which allows for comparing the 7 studies. However, a generalization of the results for other interventions must be done with caution because some interventions also used other theories, such as self-regulation theories, as a framework for the educational content.

Although the restricted number of studies is a limitation, including other Web-based tailored interventions might have resulted in even higher program heterogeneity and would have made comparisons even more difficult and results (partly) dependent on program characteristics. Also Wienert and Kuhlmann [98] have determined that tailored interventions are difficult to compare, whereas the interventions included in this study were comparable because they all were based on the same theoretical background (the I-Change model), all 7 studies provided tailored feedback on social cognitive determinants from this model and provided feedback, and all 7 studies included the same program evaluation item. The comparison of interventions using other tailoring techniques than the interventions described in this study is difficult and access to the original data at the individual level would be necessary for further research and adequate analysis [99].

One of the strengths of this study is the access to 7 datasets (at participant level), which allowed us to conduct the analysis with the original data. Second, all studies used at least one identical item to assess the study evaluation which enables us to compare these studies. Finally, all studies had a large number of participants ranging from 1149 to 5055, which makes our results meaningful.

This study showed that for 3 of 7 studies on computer-tailored interventions, participants with a lower educational level dropped out more often from follow-up measurements and tended to evaluate the interventions better compared to participants with a middle and higher educational level. However, the evaluation of the intervention did not predict participation or nonparticipation at follow-up. Based on our results, it is hard to say what other factors may play a role in dropout attrition from Web-based computer-tailored interventions. Further studies might evaluate different aspects of the intervention, besides only the participants’ grades, to find more relevant aspects of intervention evaluation.

Future studies should take high dropout among lower educated participants into consideration when developing strategies to decrease high dropout from Web-based computer-tailored interventions.