We promoted the open-access WISEteens landing page to recruit a convenience sample of potential participants using both online and offline strategies. As offline strategies, we developed print promotion materials (information leaflets and flyer cards) and distributed them in schools, youth-clubs, cafés, bars, stores, and adolescent specific events. We also used a three-fold online recruitment strategy with high rank of our websites’ domain in widely used search engines, advertisements via popular social media, and links on affiliated health promotion sites. To motivate study participants and enhance follow-up rates, we promoted and held a prize draw for tablet computers among participants who provided follow-up assessment.

The participants were anonymous throughout the study. At the first visit, they were asked to register, which required a user name, email address, and a password that did not contain their name. On the landing page, they could choose their respective language flag for a different language than pre-defined by browser options. After registration, respondents were screened for at-risk substance use. Those fulfilling the inclusion criteria then received study information including confidentiality, voluntariness of participation, and data security, as well as information about the randomization protocol. Participants were not blinded to random allocation. After informed online consent, the baseline assessment was completed, including those items that form the base for tailoring intervention content. Participants in the intervention condition received a login code to enable exit and re-entrance. Randomization was generated automatically by an online computer program without stratification. The envisioned number of participants was sufficient to ensure randomization integrity and a likely balanced distribution among the two parallel groups [55].

An adapted version of the 6-item CRAFFT (Car, Relax, Alone, Forget, Friends, Trouble) tool was used to screen for at-risk use of alcohol and other drugs (see Multimedia Appendix 2). This tool has proven criterion validity compared to other screening tools [56] and is recommended for identification of at-risk adolescents [17]. A CRAFFT score with at least two positive items was the criterion for study inclusion [57].

The WISEteens intervention relied on an interactive system to generate individually tailored content. All system-generated information was presented in small units that combined text and graphics (eg, photos and illustrative drawings) and directly referred to the participant’s statements assessed in the first place (eg, substance use, sex, weight, perceptions of normative drinking). Navigation through the program was designed as a dialogue between the user and a virtual expert with “gates” (ie, choice options) at the end of each page to permit varying degrees of approval or disapproval with page content. The system used these responses to introduce subsequent content on the next page.

Intervention content, dialogue tone, and style was based on MI [40] and consisted of the following 6 components (see Multimedia Appendix 3 for screenshots), with the first three applying to alcohol but not illegal drug use and polydrug use: (1) feedback for individual drinking patterns with information on associated health and developmental risks, (2) normative feedback to descriptive drinking norms about sex- and age-matched peer drinking levels using graphed comparative information, (3) feedback for blood alcohol concentration (BAC) and associated health and other risks (ie, traffic crash, unintended sex) for the peak drinking episode in the last 2 weeks, (4) importance and confidence rulers with a short summary and feedback to encourage change readiness and exploration of personal strengths, resources, and volitional strategies for goal attainment, (5) decisional balance for selection of personal costs and benefits of current substance use and a subsequent graphical display of comparative gains and losses of behavior change in a balance sheet to illustrate ambivalence, and (6) identification and selection of personal high-risk situations for substance use and provision of behavioral strategies, for example, to resist peer pressure (the assumed mechanisms for change are displayed in Multimedia Appendix 4; for a more detailed description, see [54]).

The WISEteens intervention was pilot-tested in two steps. First, 10 adolescents chose their preferred design concept among three options. A preliminary version with the preferred “look & feel” was then pre-tested by 37 other adolescents to ensure ease of registration and navigation use, comprehensibility of intervention content, satisfaction with layout and design, appropriateness of dialogue style (eg, avoiding judgmental and confronting language), overall satisfaction with the program, and time to complete baseline assessment and intervention. Furthermore, open feedback, technical problems, translational ambiguities, and other problems were documented and the program was adapted accordingly. Median for assessment and intervention was 15 minutes, ranging from 5-30 minutes. Most adolescents were satisfied or totally satisfied with the system, design, comprehensibility, and intervention dialogue. Table 1 provides a summary of the pilot-results. The control group received assessment only and was directed to a short information page on where to find help in case of urgent counseling or medical needs.

All study measures were administered anonymously and online via self-reports at baseline (t₀ before randomization) and at follow-up (t₁, 3 months after t₀) and required registration with a valid email address. Three months after completing the baseline assessment, participants were automatically invited to participate in the follow-up assessment and guided by an integrated hyperlink in the email invitation with one reminder email after 1 week.

Participants were asked for age, gender, country of residence (Sweden, Germany, Belgium, Czech Republic or other), current school attendance (yes/no), parental educational attainment (low, middle, high), their weight in kilograms for BAC-level feedback, descriptive norms [58, 59], and 12-months scores on the Alcohol Use Disorder Identification Test Consumption subscale (AUDIT-C [60]). To address baseline change motivation as a potential confounder, we included intention to reduce drinking (in the next 30 days) using a single item (“I intend to reduce my drinking during the next 30 days”, 1=“totally disagree” to 7=”totally agree”) and intention to abstain from illegal drug use (“I intend to abstain from taking illegal drugs during the next 30 days”, 1=“totally disagree” to 4=“totally agree”) [57], as a continuous measure for change readiness [61].

All outcome measures concerned use in the past 30 days. Change in alcohol use (frequency, frequency of binge drinking, and quantity) between the two assessments was the primary outcome and measured based on the three items of the AUDIT-C screening tool [60]. This measure provides a widely used and valid index sum score for problem alcohol use of adolescents [62]. The three indicators are drinking frequency (“How often did you have a drink containing alcohol?”; 0=“never” to 4=“four or more times a week ”), binge drinking frequency (“How often did you have 5 [4 for girls] or more drinks on one occasion, like during a party or on one night?”; 0=“never” to 4=“four or more times a week”), and drinking quantity (“How many drinks containing alcohol did you have on a typical day when you were drinking?”; 0=“one or two” to 4=“ten or more”). To assess the number of consumed drinks, we used a graphical display of various types of drinks with the indication to select the number of each drink per typical drinking occasion to account for national differences in typical standard drinks. Standard drinks were overall defined as containing 10-12 grams of pure ethanol, and responses were recoded to match the original 0-4 point scale. Using an index for drinking has two advantages over separate measures of alcohol use. First, it allows for modeling one drinking measure to increase the statistical power to detect intervention effects while maintaining several indicators of risky drinking. Second, due to the scale, characteristic outcome data can be analyzed as continuous data, which makes interpretation easier compared to discrete count drinking outcomes [63, 64]. We included the three separate outcomes (frequency, frequency of binge drinking, and quantity) and frequency of illegal drug use (1=“never” to 5=“four or more times a week” [65]) as secondary outcomes and past 30 days prevalence of illegal drug use and polydrug use as additional secondary outcomes. Because most drug use combinations in Europe include alcohol [10], we defined polydrug use as a dichotomous measure for use of alcohol and any illegal drug during the last 30 days, similar to previous studies [9].

Sample size calculation was based on the primary outcome with regard to effect sizes for alcohol use revealed by a recent review on Web-based interventions for young people [31]. According to results from similar studies, we expected a small effect size (Cohen’s d=0.2). To reach power of 80% at a type I error rate of 5% in a two-sided test and expecting a dropout rate of approximately 50% [27, 66], we aimed at N=400 per intervention condition [67]. Possible country dependent clustering effects were not included in the sample size calculation because the study was designed as an individual-based RCT and higher level effects from four clusters were not previsioned [68]. Nevertheless, possible higher order country effects were addressed in all further analyses as described below.

We first analyzed data on sample characteristics using t-tests (for metric data) and chi-square tests (for categorical data) to test for differences between intervention conditions. Next, we performed logistic regressions with completers (ie, those who provided valid follow-up data) versus dropouts as the binary dependent variable to test for possible attrition bias using all available sociodemographic and substance use variables as predictors. Intervention effects for primary and continuous secondary outcomes were tested using Linear Mixed Models (LMM) and binary secondary outcomes were analyzed using Generalized Linear Mixed Models. In all LMMs, we used change in outcome scores from baseline as the dependent variable, intervention condition as the only independent variable, baseline values as covariates (“fixed effects”), and country of residence as a single random effect (“random intercept”). This model controls for the correlation between baseline and follow-up outcome scores and does not require a repeated statement (ie, “time”) and no time x group interaction term to interpret intervention effects. Because we found no higher order effect for country of residence (primary outcome: Wald Z=.77, P=.441), we skipped the random effect and adjusted the analyses for country of residence and possible confounders (ie, variables that were not balanced between intervention and control group at baseline) and predictors for missing data as additional covariates, which resulted in improved model fit (delta Bayesian information criterion=-13.16) (see [69] for a similar approach). Binary secondary outcome analyses (prevalence illegal drug use and polydrug use) focused on follow-up outcome values rather than change scores. All analyses are based on a complete-case dataset and an intention-to-treat (ITT) sample with imputation of missing follow-up data based on expectation maximization (EM). Both results are relevant and commonly reported in Web-based interventions particularly when dropout is large [70]. EM is a single imputation method that was shown to outperform the multiple imputation module available in SPSS in eHealth studies with high dropout rates [66]. Given the huge dropout in this study, we cross-checked the ITT-outcome analysis using a full information maximum likelihood (FIML) estimator for missing follow-up data provided in the structural equation model software AMOS to reduce estimate bias for missing data [71] and increase the robustness of findings (see Multimedia Appendix 5). For all analyses, we report estimated marginal means (percentages) and Cohen’s d effect sizes. Distributions of outcomes (ie, skew and kurtosis) and missing-at-random requirements of missing data were checked prior to the main analyses. Results with a type I error rate of P<.05 in 2-sided tests were considered as statistically significant without adjustment for multiple comparisons but reporting of exact Pvalues [72]. All analyses other than in AMOS were performed using SPSS statistical software package version 22 [73].

The trial profile is shown in Figure 1. A total of 2673 participants logged on the WISEteens Web portal and participated in the initial screening. We excluded 655 (24.5%) from the study due to a negative CRAFFT screening. This resulted in 2018 (75.5%) adolescents who gave consent to participate in the study and started subsequent baseline assessment (t₀). A total of 569 (28.1%) dropped out during the baseline assessment leaving 1449 participants who completed baseline assessment and were randomized to either the intervention (N=715) or control group (N=734). In the intervention group, 453 (63.4%) completed the brief intervention as measured by a log file record whether the last page of the intervention has been visible to the user. A total of 211 adolescents participated in the follow-up assessment after 3 months, corresponding to a valid response rate of 14.5%. In this subsample, the completion rate for the brief intervention was higher than in the full randomized sample (82.4%).

In the randomized sample (ITT population), the mean age was 16.8 years (SD 0.74), nearly half of the participants were women and the majority were currently attending school. Most participants were recruited in the Czech Republic due to a more intense offline recruitment in this country, indicating that adjustment of country of residence as an additional covariate was required in subsequent analyses. Participants in the intervention group tended to have a higher rate of parental educational attainment compared to the control group as indicated by a near significant difference (P=.060). This variable was therefore adjusted as an additional covariate in subsequent analyses [68]. With the exception of binge drinking frequency (P=.048), there were no significant group differences with regard to demographic or assessment data at baseline (see Table 2). Importantly, there were no baseline differences in the intention to change current alcohol and illegal drug use among the groups. Distributions of all continuous outcome variables showed acceptable skew and kurtosis with values well below 1.0 for baseline and follow-up assessments [74]. Regarding characteristics of participants who provided data at follow-up, group comparisons revealed no significant differences in any assessed variable (see Table 3). Overall baseline group comparisons thus indicate that the randomization was successful and that the completer-only subsample appears largely similar to the randomized sample.

Response rates were very similar for the intervention group (15%) and the control group (14%). Logistic regression analyses with attrition at follow-up as the dependent variable and all demographic (country, parental educational attainment, sex, age) and substance use‒related variables (all primary and secondary outcomes) as predictors explained 8.4% of the total response variance (Nagelkerke’s R²). Corresponding odds ratios (OR) revealed country of residence as the only significant predictors for dropout. Response rates were significantly lower for participants from the Czech Republic (11%) than for those from Sweden (23%, 58/251; OR 2.42, 95% CI 1.52-3.85, P<.001) and Germany (27%, 31/146; OR 2.514, 95% CI 1.41-4.49; P<.001) but not Belgium (18%, 26/143; OR 1.46, 95% CI 0.76-2.80; P=.26). The analysis thus indicates no biased attrition based on the available variables except for country of residence, which was adjusted in all subsequent analyses as a relevant predictor of missing data [75]. Analogous attrition analyses for intervention completion (as indicated by a record whether a user has “seen” the last page of the intervention dialogue) indicated that 12.2% of the total response variance was predicted by all study variables, with significantly more completers being women (P=.007) and significant differences between participants depending on country of residence (P<.001) with the highest intervention completion in Germany (89.7%), followed by Belgium (71.4%), Sweden (69.8), and the Czech Republic (56.0%).

Tables 4 and 5 report the primary and secondary intervention outcomes of this trial at follow-up based on the non-imputed completer-sample and Tables 6 and 7 for the EM-imputed intention-to-treat-sample. All analyses concern substance use in the past 30 days and were adjusted for baseline scores, country of residence, and parental educational attainment.

Based on the non-imputed sample 3 months after the intervention, participants in the intervention group have reduced their drinking as indicated by reduced AUDIT-C based scores relative to baseline with an adjusted mean change of -0.85 (95% CI -1.49 to -0.26) while those in the control group slightly increased their drinking as indicated by a mean increase in drinking of 0.16 (95% CI -0.50 to 0.82). Adjusted mean differences between both groups were 1.02 (95% CI 0.25-1.79) and statistically significant (F _1,134=6.8, P=.010, d=.26). The corresponding between-group effect was smaller in the (imputed) ITT analysis due to significant reductions relative to baseline in the control group. However, the significant between-group effect (0.16, 95% CI 0.02-0.25) was maintained (F _1,1329=5.2, P=.022, d=.04). Additional analysis in AMOS based on an FIML estimation for missing outcome assessments confirmed these results (B=-0.72, β=-0.13, P=.046; see Multimedia Appendix 5).

We conducted identical analyses for the three drinking indicators separately. In the non-imputed sample, we found a significant between-group difference in drinking frequency of 0.25 (95% CI 0.02-0.50) in favor of the WISEteens group (F _1,134=4.4, P=.037, d=.15), which was not maintained in the analysis based on EM-imputation (F _1,1329=3.2, P=.073, d=.11) due to significant reductions relative to baseline in both groups (Ps<.001). We obtained a similar result for binge drinking frequency with a significant adjusted mean difference between groups of 0.31 (95% CI 0.01-0.61; F _1,121=4.2, P=.044, d=.16) in the non-imputed data analysis, which was not maintained in the imputed analysis (F _1,1329=2.3, P=130, d=.01). Additional FIML analysis for both outcomes revealed non-significant and similar results as in the imputed analysis (drinking frequency: B=-0.13, β=-0.08, P=.230 and binge drinking frequency: B=-0.25, β=-0.14, P=.059; see Multimedia Appendix 5). For drinking quantity, we found significant reductions relative to baseline for the intervention group in the non-imputed analysis of -0.39 (95% CI -0.72 to -0.06, P=.024), but these reductions were non-significantly different from the control group (between-group differences: 0.31, 95% CI -0.17 to 0.62; F _1,155=1.3, P=.257, d=.13). This effect was significant in the EM-imputed data-set (F _1,1329=5.3, P=.021, d=.05). However, when cross-checked using the FIML approach employed in AMOS, these differences were no longer significant (B=-0.22, β=-0.08, P=.209, see Multimedia Appendix 5).

Results for frequency and prevalence of illegal drug use as well as polydrug use prevalence are summarized in Tables 5 and 7. With regard to these outcomes, we found no statistically significant between-group effects in the non-imputed (Ps=.138-.311) and the EM-imputed (Ps=.363-.871) datasets. Although overall, both groups show numerical decreases between the measurements that were statistically significant in the intervention group for illegal drug use prevalence (P=.025) and polydrug use prevalence (P=.012) in the non-imputed analyses and statistically significant for all 3 outcomes (ie, frequency of illegal drug use and illegal drug use and polydrug prevalence, Ps<.001) in the imputed analyses.

The purpose of this study was to test the effectiveness of a fully automated Web-based screening and brief motivational intervention targeting adolescents with at-risk substance use in Europe.

We found that self-reported risk drinking as measured by a drinking index (ie, drinking frequency, frequency of binge drinking, and typical quantity of drinks) was significantly reduced for participants in the intervention group. The effect on the primary alcohol use outcome was consistent across imputation and non-imputation (“completers”) based analyses but accented in the non-imputed data analysis, even though statistical power was low for 3-month effects due to large loss to follow-up assessment. Secondary analyses using the three drinking indicators as separate outcomes revealed statistically significant mean differences at follow-up in favor of the WISEteens intervention group for drinking frequency and binge drinking frequency but not quantity when missing follow-up data was not imputed. In contrast, analyses using an EM-imputed dataset revealed drinking quantity as the only significant secondary effect. For illegal drug use or polydrug use, there were no significant intervention effects.

The effect sizes obtained in this study are small but match those summarized in recent systematic reviews for fully automated interventions for young adults [32] and meta-analyses for single session interventions [76, 77]. Moreover, they correspond with effect sizes reported for face-to-face brief interventions for youth who use alcohol and other drugs [17, 78, 79] and indicate that expected effects of MI-based interventions on substance use may indeed be small [41] but can be relevant when a large population can be reached. Overall, there are currently few Web-based interventions targeting adolescents, which limits direct comparisons to prior studies. However, our study contradicts results from one recent RCT that tested the effects of a similar intervention (What Do You Drink [WDYD]). This trial targeted drinking among young people (15-20 years) with low educational background in the Netherlands [80]. While WDYD and WISEteens are comparable in central characteristic (eg, age group, cultural context, single session fully automated delivery mode, intervention duration, applied theory, and outcome measures), there are a number of differences that could account for the divergent effects, such as the school-based study implementation and different follow-up times. Moreover, in the WDYD trial participants with an indication of severe problem drinking at baseline were excluded, while in our study about half of the participants were above the AUDIT-C risk cut-off of 5 points [62] at baseline. Severity of baseline drinking can influence effects of brief motivational interventions with stronger effects among subgroups of heavy drinkers [81].

WISEteens was not effective to address illegal drug use and polydrug use adequately. However, the number of participants with drug and polydrug use was rather low in our sample and meaningful between-group effects might have been undetected due to insufficient statistical power [37]. Note that there were notable decreases in the prevalence rates for polydrug use and illegal drug use in the intervention group, while there were no changes and even slight increases in the control group. There were also no spill-over effects from reduced problem drinking to other substance use, which suggests that effects on targeted outcomes may not translate to untargeted outcomes [82, 83].

In general, effects from comparable Web-based interventions for illegal drugs are typically smaller than for alcohol [32]. Moreover, the main hypothesis and focus of the intervention was on drinking. Some intervention elements, such as decisional balance, importance, and confidence ruler and advice for risk situations were available for alcohol per default and participants with other drug use were explicitly encouraged to take the exercises as templates for use of other substances. Thus, the lack of positive outcomes may also be the result of limited specific intervention content for drugs other than alcohol, which may be a limitation. This notwithstanding, the lack of significant effects in our study corresponds to previous trials among students that failed to promote positive behavior change [84, 85], although such interventions can in principle be effective in the general population [32, 86].

Our study is among the first to report on a targeted fully automated Web-based brief intervention among at-risk adolescents with excessive alcohol drinking and drug use in a randomized controlled trial. From a public health perspective, the significant effect on drinking is relevant. Notwithstanding the often dramatic consequences associated with illegal drug use, alcohol is the most frequently used psycho-active substance during adolescence, alcohol use disorders are among the most prevalent and costly mental disorders in industrialized countries, and prevention is a public health priority [87]. Although the main burden of alcohol-related diseases and injuries becomes apparent in adulthood, it is well documented that early at-risk alcohol use can lead to persistent problems [88,89]. Considering the magnitude of youth with at-risk alcohol use the need for effective targeted prevention may be particularly high in Europe. For example in Germany, hospital admissions due to acute alcohol intoxication have increased substantially in the past years, although the proportion of youth who drink alcohol is decreasing [7]. With this study, we provide initial support for the effectiveness of Web-based brief interventions for adolescents in Europe—an approach that has proven useful in other critical heavy drinking populations, such as college students [31, 33].

The WISEteens intervention was designed to reflect valid face-to-face motivational strategies in a Web-based format. We extended individual feedback techniques typically used in Web-based brief interventions by other MI techniques from face-to-face interventions, such as decisional balance, confidence and importance ruler, and provision of behavioral and regulatory strategies to resist peer pressure [38, 40, 90]. Moreover, we aimed to mimic MI principles by applying a carefully designed and pilot-tested motivational dialogue to reflect possibly “empathic” language, acknowledgement of ambivalent goals, needs for autonomy and self-directedness, and appreciation of change but not necessarily abstinence. Although our results cannot address this issue directly, we assume that this may have contributed to the favorable effects revealed in this study. Certainly we acknowledge the limits in delivering MI-consistent methods such as relational factors in a fully automated format [91]. However, arguably the ability of such interventions to deliver standardized intervention content on a large scale and at low cost may outweigh these limitations.

Finally, even though response to follow-up assessment was very low, screening and intervention were accessed and completed by a reasonably large number of teenagers, which may indicate acceptance and ecological validity in the target group. The intervention completion rate indicates acceptable user engagement in the “real world” [92] and a good balance between the required amount of program exposure and adherence requirements of a single session Web-based delivery format for adolescents [50, 93].

Our study has a number of limitations. First and foremost, results are limited by the higher than expected dropout rate for follow-up assessment, which is a frequent problem in Web-based trials [66, 70]. Dropout might be partly caused by invalid email addresses used by the participants and the fact that the system sent only one email reminder per participant [94]. Even though we detected no serious attrition bias, this may limit the validity of the study findings. Although in case of large dropout, any approach to missing data imputation such as the EM method employed in this study could be compromised. We approached this problem by cross-check analyses using an FIML approach to missing data estimation, which yielded similar and significant results for the primary outcome. Moreover, it must be noted that attrition was mainly a problem for the evaluation (ie, attrition took place between measurement points) and much less for intervention adherence (ie, intervention completion rates). The fact that we were able to identify intervention effects, even with the fewer than required number of participants needed to detect small effects in the evaluation, could be interpreted as an underestimation of intervention effects. This notwithstanding and although the study was performed in four European countries, the results should be taken only under careful consideration and follow-up should be translated to other countries or even to other regions or to other groups of adolescents in the four involved countries. The inconsistent results of the imputed and non-imputed analyses and substantial between-analyses deviations in obtained effect sizes serve as a quantitative indicator of uncertainty in these results due to the substantial amount of missing follow-up data.

Another limitation is reliance on self-reported data in this study, which are often associated with underreporting of alcohol drinking and other drug use [95]. Measures to avoid underreporting were assurance of and advice for confidentiality and a non-judgmental and non-confronting MI style employed throughout the intervention. Moreover in our study, the self-report was given anonymously and without personal contact, which may add to the reliability of self-reported data. Moreover, we relied on a primary outcome that was based on the AUDIT-C, which was developed as a screening tool for harmful alcohol use in adults. Given the adapted reference time of drinking in the past 30 days, it is difficult to interpret the practical significance of the measured changes in outcome scores in response to the intervention. Furthermore, participants were not blinded to the assigned interventions, which is a common limitation in Web-based trials [96].

Finally, given the focus on intervention effects in this study, we have not systematically included recruitment and reach in our analyses although these are important issues for estimating public health impact [97]. Rather, we aimed for a convenience sample from the general population by employing a pragmatic recruitment procedure. However, the incitement by lottery as an incentive for participation may have increased the reach to a higher level than can be expected in implementation outside a research project. While the self-selected nature of our sample again limits generalization, we consider the realistic setting of this trial a significant strength. In fact, apart from the evaluation requirements at baseline, the actual intervention program was equivalent to a potential real world application. We thus feel confident in stating that our study has realistic public health implications.

Web-based interventions to reduce adolescent at-risk substance use hold promise for accessibility and large scale dissemination but have rarely been tested in randomized controlled trials. Although limited by substantial dropout to follow-up assessment, our findings imply that young adolescents with excessive drinking can benefit from targeted interventions based on MI techniques and counseling style in a fully automated Web-based delivery mode. However, we found no effect on drug use, which calls for further research on effective intervention models, delivery modes, and recruitment strategies.