Using Ecological Momentary Assessment to Test the Effectiveness of a Web-Based Brief Alcohol Intervention Over Time Among Heavy-Drinking Students: Randomized Controlled Trial
Background: Web-based brief alcohol interventions are effective in reducing alcohol use among students when measured at limited follow-up time points. To date, no studies have tested Web-based brief alcohol intervention effectiveness over time by using a large number of measurements.
Objective: Testing whether the What Do You Drink (WDYD) Web-based brief alcohol intervention can sustain a reduction in alcohol use among heavy-drinking students aged 18-24 years at 1-, 3-, and 6-month follow-up intervals.
Methods: A purely Web-based, 2-arm, parallel-group randomized controlled trial applying an ecological momentary assessment approach with 30 weekly measurements was conducted in the Netherlands (2010-2011). Participants were recruited offline and online. A total of 907 participants were randomized into the experimental condition (n=456) including the single-session and fully automated WDYD intervention, or into the control condition (n=451) including assessment only. Weekly alcohol consumption and frequency of binge drinking were the self-assessed outcome measures.
Results: Attrition rates of the 907 participants were 110 (12.1%), 130 (14.3%), and 162 (17.9%) at 1-, 3-, and 6-month follow-up intervals, respectively. Latent growth curve analyses according to the intention-to-treat principle revealed that participants in the experimental condition had significantly lower weekly alcohol consumption compared to participants in the control condition that was sustained at 3-month follow-up (intercept=–2.60, P<.001; slope=0.16, P=.08). Additional linear regression analyses indicated that this intercept difference resulted from significantly higher levels of alcohol units per week for participants in the control condition compared to those in the experimental condition at 1-month (beta=–2.56, SE 0.74, Cohen’s d=0.20, P=.001), 3-month (beta=–1.76, SE 0.60, Cohen’s d=0.13, P=.003), and 6-month (beta=–1.21, SE 0.58, Cohen’s d=0.09, P=.04) follow-up intervals. Latent growth curve analyses further indicated that participants in the experimental condition had a significantly lower frequency of binge drinking compared to participants in the control condition that was sustained at 6-month follow-up (intercept=–0.14, P=.01; slope=0.004, P=.19). This intercept difference resulted from higher levels in this outcome for participants in the control condition relative to participants in the experimental condition at 1-month (beta=–1.15, SE 0.06, Cohen’s d=0.16, P=.01), 3-month (beta=–0.12, SE 0.05, Cohen’s d=0.09, P=.01), and 6-month (beta=–0.09, SE 0.05, Cohen’s d=0.03, P=.045) follow-up intervals.
Conclusions: The WDYD intervention was shown to be effective in preventing an increase in weekly alcohol consumption and frequency of binge drinking directly after the intervention. This effect was sustained 3 and 6 months after the intervention.
Trial Registration: Netherlands Trial Register NTR2665; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=2665 (Archived by WebCite at http://webcitation.org/6LuQVn12M).
J Med Internet Res 2014;16(1):e5)
Given the high prevalence and social and economic costs attributable to heavy drinking among young adults, there is an urgent need for adequate interventions [- ]. The widespread growth and availability of computer technology and the Internet has provided the opportunity to deliver interventions via the Web [ ], which is advantageous for young adults because it allows them to access information at a self-selected time and place while remaining anonymous [ ]. Web-based brief alcohol interventions have been found to be effective in reducing the quantity and frequency of alcohol use among heavy-drinking young adults and students [ - ]. However, despite the demonstrated effectiveness of these types of interventions, findings have to be interpreted with caution because of the way alcohol use and intervention effectiveness are assessed. First, alcohol use is typically assessed over relatively long recall periods (eg, 30 days) and participants are often asked to report the average number of alcohol units they consumed in a usual week. This can result in measurement errors because precise recall of alcohol use decreases after 2 or 3 days due to memory deficits [ ]. Second, the fluctuating nature of alcohol use among students because of calendar-specific events [ , ] is often overlooked because intervention effectiveness is measured at limited follow-up time points (ie, 1 short and several longer follow-ups of approximately 1-12 months after the intervention). The use of limited follow-up time points not only disregards important drinking events, but also increases the danger of drawing inaccurate conclusions about intervention effectiveness. In a previous study, we recognized the disadvantage of using only 2 follow-up time points (ie, 1 and 6 months) when testing the effectiveness of the What Do You Drink (WDYD) Web-based brief alcohol intervention. Our baseline assessment was completed during a usual drinking period without any remarkable events, whereas our first 1-month follow-up assessment coincided with carnival, a 4-day event associated with excessive drinking. The observed increase in alcohol use was likely the consequence of the selection of this particular week to test intervention effectiveness rather than the WDYD intervention, which aims to detect and reduce heavy drinking in young adults [ ]. The single-session and fully automated WDYD intervention was developed in collaboration with the Trimbos Institute (Netherlands Institute of Mental Health and Addiction) by using the intervention mapping protocol [ ]. Content is based on motivational interviewing principles [ ] and parts of the I-Change model [ ] in which knowledge, social norms, and self-efficacy are embedded as the most changeable determinants of behavior change [ ]. Despite the sound theoretical background of the WDYD intervention, no significant main effects on alcohol use were found when using 2 follow-up time points with short recall periods of 7 days [ ]. Third, individual changes in alcohol use and intervention effectiveness over time remain unnoticed when using limited follow-up time points. Information on these changes can be extremely valuable for determining the time at which the intervention effects have stopped and the time at which booster sessions may be needed to strengthen and/or extend intervention effects.
In evaluating intervention effectiveness, it appears to be important to use short recall periods to reduce measurement errors and to include a large number of measurements to consider the fluctuating nature of alcohol use over time and capture important drinking events. Higher precision in establishing intervention effectiveness can be achieved by means of ecological momentary assessment (EMA) and latent growth curve (LGC) modeling techniques. The ecological aspect of EMA is that data are collected in real-life settings at strategically selected moments in time . The momentary aspect of EMA implies that the assessment of alcohol use focuses on participants’ current or recent state. In addition, EMA is characterized by repeated and a large number of measurements over time and often used equivalent to experience sampling methods (ESM), a systematic way for participants to report on their ongoing alcohol use behavior [ , ]. LGC modeling techniques allow for estimation of average growth trajectories (ie, mean intercepts and slopes) of alcohol use over time as well as individual differences in these trajectories (ie, intercept and slope variances) [ , ]. The estimation of variances in growth trajectories increases the reliability of outcome measures. This is not possible with traditional statistical techniques that are often used to test intervention effectiveness, such as repeated-measures ANOVA [ ], because they only provide mean growth patterns and treat variances as error [ ]. Although the advantages of employing EMA and LGC modeling techniques are evident, most trials on Web-based brief alcohol interventions used long recall periods with limited follow-up time points and traditional techniques [ ] to test intervention effectiveness [ ]. To our knowledge, this is the first study to test whether a Web-based brief alcohol intervention can sustain a reduction in alcohol use among heavy-drinking students at 1-, 3-, and 6-month follow-up intervals. To test the effectiveness of the WDYD intervention over time, we conducted 30 weekly EMA measurements for 6 months through online surveys and LGC analyses to model individual change in weekly alcohol consumption and frequency of binge drinking at 1-, 3-, and 6-month follow-up intervals by condition. The rationale of reporting over 3 time periods was to gain insight into how long the intervention effects sustained and to limit the chance of reporting outlier trajectories. We hypothesized that participants in the experimental condition would reduce their alcohol use (intercept) compared to participants in the control condition directly after exposure to the WDYD intervention. Based on Web-based brief alcohol interventions that have produced long-term effects [ , ], it was hypothesized that the reduction would be sustained (slope) at the 6-month follow-up interval.
A 2-arm, parallel-group randomized controlled trial applying an EMA approach with 30 weekly EMA measurements was conducted online in the Netherlands (2010-2011) to test whether the WDYD intervention could sustain a reduction in alcohol use among heavy-drinking students at 1-, 3-, and 6-month follow-up intervals. This trial was purely Web-based because there were no face-to-face components in the intervention and for assessing the outcome measures.
Procedure and Participants
A convenience sampling strategy was used to recruit heavy-drinking students offline by distributing flyers at universities and higher professional education institutions (ie, universities of applied sciences) online by sending emails with information about the study from September to December 2010. The cover story was that students had to evaluate newly developed health education materials addressing alcohol use and that they had to judge these materials to reduce the risk of social desirability bias. Students were blinded to the aim of the study until the end of the EMA study. Interested students were referred to an email address and were sent a detailed description of the study by email. To be included in the study, students had to (1) be between ages 18 and 24 years, (2) report heavy drinking in the past 6 months, (3) be ready to change their alcohol use, (4) have daily access to the Internet (and be literate), and (5) sign an online informed consent form. Heavy drinking was defined as consuming more than 14 (females) or 21 (males) glasses of standard alcohol units per week and/or consuming 5 or more glasses of standard alcohol units per occasion at least 1 day per week . Students reporting a score of 20 or higher on the Alcohol Use Disorders Identification Test (AUDIT) [ ], and/or receiving treatment for alcohol-related problems were excluded from the study and advised to seek treatment because the WDYD intervention was developed to reduce heavy drinking rather than problem drinking.
A sample size of 908 participants was necessary to detect an increase in the percentage of participants showing low-risk drinking guidelines after 1 month of 42% in the experimental condition versus 31% in the control condition  with a 2-sided 5% significance level and a power of 80%, given an anticipated dropout rate of 30% after randomization. Students who met the inclusion criteria were randomly assigned to the WDYD intervention condition (n=456) or to the control condition (n=451) in blocks of 4 using a computerized random number generator by an independent researcher of the Behavioural Science Institute who could not influence or predict the randomization result. Participants were not blinded to randomization results. Randomization was stratified by sex and educational level before the baseline assessment in January 2011 [ ].
In total, 30 weekly EMA measurements were conducted online from January to August 2011, to assess outcome measures with 4 pretests and 26 posttests. After the 4 pretests in January, participants in the experimental condition were exposed to the WDYD intervention, whereas those in the control condition received assessment only. Directly after intervention exposure in the first week of February, participants in both conditions received the first posttest. One week after the intervention, all participants received weekly EMA posttest measurements for 6 months from February to August. EMA measurements were assessed on Monday mornings. All participants received an email with the instructions on the use of the survey, and they were asked to respond to the survey before midnight. Each survey took approximately 10 minutes to complete and contained identical questions about participants’ weekly alcohol consumption, frequency of binge drinking, and drinking refusal self-efficacy. In addition, extended surveys were administered at baseline assessment, immediately after the intervention, and at 1 and 6 months after the intervention. These extended surveys included additional questions concerning alcohol-related cognitions, cost-effectiveness, and problem drinking. Completion time of the extended surveys was approximately 20 minutes. Paper-and-pencil surveys with identical content were provided to participants in case they were unable to access the Internet. Participants who failed to complete the survey on Mondays received a short text message on their mobile phones on Tuesdays to remind them. Those who still did not complete the survey on Tuesdays were reminded by a telephone call on Wednesdays. On average, 11% (range 7%-17%) of the surveys were completed on Tuesdays and Wednesdays instead of Mondays. When participants completed at least 28 of 30 surveys, they received €100 as an incentive, as stated in the informed consent. Ethical approval was provided by the Ethical Committee of the Faculty of Social Sciences at Radboud University Nijmegen (ECG30062011). This trial is registered at the Netherlands Trial Register (NTR2665) as mentioned in the trial protocol .
Participants assigned to the experimental condition used the WDYD intervention. The first part of the WDYD intervention focuses on increasing the users’ awareness of the potential problems, consequences, and risks associated with their drinking behavior. It contains a home page and a screening test with personalized feedback delivered in a nonjudgmental, nonconfrontational, and nonaversive way (see). The screening test included participants’ self-reported name, sex, age, education level, weight, alcohol use, readiness to change alcohol use, average expenses for consumed alcohol beverages, and descriptive social norms. Personalized feedback consisted of advice about drinking according to low-risk drinking guidelines [ ], personal drinking profile (quantity-frequency consumed in past year), estimates of calorie intake, increases in weight, money expenses because of drinking, and a comparison of personal use rates with the national norms of same-sex peers to correct misperceptions of descriptive social norms (see ). The personalized feedback was based on the individuals’ personal situation, implying that the WDYD intervention was tailored. The second part of WDYD focused on setting and maintaining drinking goals (see ) and strengthening users’ drinking refusal self-efficacy to succeed and maintain drinking goals by providing tips to resist alcohol in different drinking situations (see ). Participants were able to track their progress through the WDYD intervention, which took approximately 20 minutes to complete. A full description of the WDYD intervention is given elsewhere [ ]. Participants assigned to the control condition received assessment only.
Weekly alcohol consumption and frequency of binge drinking were the self-assessed primary outcome measures through online surveys. Weekly EMA measurements were used to assess the primary outcome measures over time at 1-, 3-, and 6-month follow-up intervals.
Weekly alcohol consumption, defined as the mean number of glasses of standard alcohol units consumed in the previous 7 days, was assessed using the Dutch version of the Alcohol Weekly Recall . Participants were asked to indicate retrospectively the exact number, size, and type of alcohol beverage they consumed on each day of the previous 7 days. An overview of standard units for various beverages was provided to guarantee standardized responses. In total, 1.47% of the participants scored above 3 standard deviations of the sample mean of weekly alcohol consumption, but they were given a score exactly at 3 standard deviations above the sample mean of weekly alcohol consumption to retain outliers in the analyses (resulting range 0-109) [ ] and to handle outliers in accordance with our previous studies [ , ]. Binge drinking frequency, defined as the number of days in the previous week on which participants drank 5 or more glasses of standard alcohol units per occasion [ ], was assessed on an 8-point Likert scale ranging from 0=never to 7=every day.
Data were analyzed according the intent-to-treat (ITT) principle. Missing data were handled by using multiple imputations using the predictive mean matching method [, ]. Twenty imputed datasets were evaluated with P<.05 as the criterion for statistical significance by averaging the results (ie, pooling). First, descriptive analyses involving t tests and chi-square tests were conducted to explore whether the randomization resulted in a balanced distribution of participants’ demographic characteristics and alcohol use (ie, weekly alcohol consumption and frequency of binge drinking) across conditions at baseline assessment. Loss to follow-up was also examined with attrition analyses using 1-, 3-, and 6-month follow-up intervals as outcome measures and demographic characteristics, alcohol use, and condition status (intervention vs control) as predictors. Second, LGC analyses were conducted to model individual change in alcohol use over time by condition at 1-, 3-, and 6-month follow-up intervals. LGC analyses were conducted over 3 time periods to limit the chance of reporting outlier trajectories. The models without condition status and baseline levels of alcohol use were tested first. Subsequently, the growth curves were regressed on condition status for weekly alcohol consumption and frequency of binge drinking separately while adjusting for baseline levels of alcohol use. A random-effect parameter for educational institutions was not included in the models since variation in participants between institutions was expected to be limited because all participants needed to meet the inclusion criteria of the study.
Unstandardized intercepts, representing alcohol use directly after the intervention, and unstandardized slopes, representing the change of alcohol use over time, were reported. Global fit indexes were used to assess model fit for each construct: chi-square statistic, Comparative Fit Index (CFI), Tucker-Lewis Index (TLI) with a cut-off value of ≥0.90 and ≥0.95 for acceptable fit, and Root Mean Square Error of Approximation (RMSEA) with a cut-off value of ≤0.06 for acceptable fit . In parallel with the LGC analyses, linear regression analyses were conducted for weekly alcohol consumption and frequency of binge drinking at 1-, 3-, and 6-month follow-up intervals and presented as unadjusted and adjusted for baseline levels of alcohol use to provide additional specific tests of a difference between the conditions. For the linear regression analyses, unstandardized coefficients (betas), standard errors (SE), and Cohen’s d [ ] effect sizes were provided. All analyses were performed using Mplus version 6.0 [ ].
In total, 30 weekly EMA measurements were conducted with 4 pretests and 26 posttests. For the LGC and linear regression analyses, the 4 pretests were aggregated into a baseline score. The first posttest immediately after the intervention was excluded from both analyses because participants reported on outcome measures over the previous week, thereby making it impossible to observe direct intervention effects. Thus, for all analyses, 4 pretests and 25 posttests were used. Only for the additional linear regression analyses, aggregated scores were computed for the 1-month follow-up interval (posttests 1-4), the 3-month follow-up interval (posttests 1-12), and the 6-month follow-up interval (posttests 1-25).
illustrates the participant flow through the study following the Consolidated Standards of Reporting Trials (CONSORT) guidelines [ ] and the data collection with 30 weekly EMA measurements. Originally, 913 students were included in the study. However, 6 students did not fill in the baseline assessment and were excluded from the study. Finally, 907 participants were enrolled in the EMA study, randomized into the experimental condition (n=456, 50.3%) or control condition (n=451, 49.7%), and eligible for the ITT analyses. In total, 82.1% (745/907) completed the baseline assessment and all 25 EMA follow-ups.
Of the 907 participants, 547 (60.3%) were male, 667 (73.5%) attended university, and 194 (21.4%) were ready to reduce alcohol use in the near future at baseline assessment. The screening survey was administered between September and December 2010, whereas the baseline assessment was administered in January 2011, which might explain the lower rates of participant’s readiness to change alcohol use at baseline assessment. On average, participants were age 20.8 years (SD 1.7). At baseline assessment, participants reported to consume a mean 21.9 (SD 13.5) alcohol units per week and reported to have 1.8 (SD 1.0) occasions in which they drank 5 or more glasses of alcohol units per week (see). No significant differences emerged between conditions in demographic characteristics and outcome measures at baseline assessment (analyses not shown here).
Loss to Follow-Up
Because of lack of interest and time, the attrition rate at the 1-month follow-up interval was 12.1% (110/907; intervention: 62/456, 13.6%; control: 48/451, 10.6%), 14.3% at the 3-month follow-up interval (130/907; intervention: 72/456, 15.8%; control: 58/451, 12.9%), and 17.9% at 6-month follow-up interval (162/907; intervention: 87/456, 19.1%; control: 75/451, 16.6%). Attrition was not related to conditions at 1-, 3-, and 6-month follow-up intervals (χ21=1.9, P=.17; χ21=1.6, P=.21; and χ21=0.9, P=.34). Completers (those who completed the baseline assessment and all 25 EMA follow-ups, n=745) did not differ from noncompleters (n=162) with respect to demographic characteristics (ie, sex: χ21=0.3, P=.56; age: t902=–0.25, P=.80; education: χ21=1.9, P=.17; and readiness to change alcohol use: χ21=0.1, P=.73), and alcohol use (ie, weekly alcohol consumption: t903=0.32, P=.75; frequency of binge drinking: t903=–0.57, P=.57) at baseline assessment. The distribution of the missing values indicated that 87 of 907 participants (9.6%) did not complete the EMA study and that 75 of 907 participants (8.3%) nearly completed the survey (missing 1 or 2 of 30 EMA measurements).
The models for weekly alcohol consumption and frequency of binge drinking without condition status and baseline levels of alcohol use were tested first. The intercept and slope of weekly alcohol consumption were significant (intercept=23.7, P<.001; slope=–0.06, P=.002), indicating that participants consumed 23.7 alcohol units on average and gradually reduced their consumption throughout the 6-month study period (χ2320=1393.2, P<.001; CFI=0.90; TLI=0.91; RMSEA=0.06). For frequency of binge drinking, a significant intercept and slope was found (intercept=1.9, P<.001; slope=–0.01, P<.001), meaning that the average number of occasions in the previous week that participants had drunk 5 or more glasses of alcohol units was 1.89. Participants’ frequency of binge drinking slowly reduced throughout the 6-month study period (χ2320=904.2, P<.001; CFI=0.92; TLI=0.92; RMSEA=0.05). Next, condition status and baseline levels of alcohol were added to both models. The weekly alcohol consumption model provided an acceptable fit to the data at follow-up assessments, except for the RMSEA at the 1-month follow-up interval. Fit indexes for weekly alcohol consumption were χ29=107.3, P<.001, CFI=0.95, TLI=0.92, and RMSEA=0.11 at the 1-month follow-up interval, χ293=644.7, P<.001, CFI=0.91, TLI=0.91, and RMSEA=0.08 at the 3-month follow-up interval, and χ2366=1451.2, P<.001, CFI=0.91, TLI=0.91, and RMSEA=0.06 at the 6-month follow-up interval.
The LGC analyses revealed that participants in the experimental condition had a significantly lower weekly alcohol consumption compared to participants in the control condition directly after the intervention. The intercept difference in alcohol units between conditions sustained at the 3-month follow-up interval (intercept=–2.60, P<.001; slope=0.16, P=.08), but faded out over time resulting in a significant slope of the LCG at the 6-month follow-up interval (intercept=–2.18, P=.001; slope=0.08, P=.02) (seeand ). Linear regression analyses indicated that the intercept difference resulted from significantly higher levels of alcohol units per week for participants in the control condition compared to those in the experimental condition at 1-month (beta=–2.56; SE 0.74; Cohen’s d=0.20; P=.001), 3-month (beta=–1.76; SE 0.60; Cohen’s d=0.13; P=.003), and 6-month (beta=–1.21; SE 0.58; Cohen’s d=0.09; P=.04) follow-up intervals (see ).
The frequency of binge drinking model provided an acceptable fit at all 3 follow-up intervals. Fit indexes for frequency of binge drinking were χ29=42.3, P<.001, CFI=0.97, TLI=0.95, and RMSEA=0.06 at the 1-month follow-up interval, χ293=341.3, P<.001, CFI=0.93, TLI=0.93, and RMSEA=0.05 at the 3-month follow-up interval, and χ2366=956.9, P<.001, CFI=0.92, TLI=0.93, and RMSEA=0.04 at the 6-month follow-up interval. According to the LGC analyses, the frequency of binge drinking of participants in the experimental condition was significantly lower compared to participants in the control condition. The intercept difference in frequency of binge drinking was sustained at the 6-month follow-up interval (intercept=–0.14, P=.01; slope=0.004, P=.19) (see) and resulted from higher levels in this outcome for participants in the control condition relative to participants in the experimental condition at 1-month, (beta=–1.15; SE 0.06; Cohen’s d=0.16; P=.01), 3-month (beta=–0.12; SE 0.05; Cohen’s d=0.09; P=.01), and 6-month (beta=–0.09; SE 0.05; Cohen’s d=0.03; P=.045) follow-up intervals (see and ).
This study is the first to test whether a Web-based brief alcohol intervention can sustain a reduction in alcohol use among heavy-drinking students at 1-, 3-, and 6-month follow-up intervals by means of an EMA approach with 25 posttests. The WDYD intervention did not reduce weekly alcohol consumption and frequency of binge drinking of participants in the experimental condition compared to participants in the control condition. Instead, the WDYD intervention was shown to be effective in preventing an increase in weekly alcohol consumption and frequency of binge drinking directly after the intervention that was sustained at 3 and 6 months postintervention. Ideally, participants in the experimental condition should reduce their alcohol use and participants in the control condition should stabilize after intervention exposure. However, these results revealed that participants in the experimental condition stabilized, whereas participants in the control condition deteriorated by increasing their alcohol use. Calendar-specific events might explain the increase in alcohol use that occurred among participants in the control condition from the beginning of February. The alcohol use patterns of participants in the control condition were similar to the patterns of binge drinking among freshmen that increased from winter break mid-December to New Year’s Eve, subsequently decreased up to the end of January and then increased again to the end of Spring Break in mid-March . In the first week of February, participants in the experimental condition could benefit from the tips of the WDYD intervention to resist alcohol in different drinking situations. Exposure to the WDYD intervention might have led to an increase in drinking refusal self-efficacy, thereby making participants in the experimental condition less susceptible compared to participants in the control condition for calendar-specific events associated with elevated risk of excessive drinking [ ]. In addition, binge drinking primarily occurs when students are with friends inside their homes, and outside their homes in bars, at parties, on dates, or during socializing activities [ ]. It is reasonable to assume that students perceived the tips to resist alcohol as more relevant when they actually found themselves in drinking situations in which binge drinking occurs. This might explain the short-term preventive effect of the WDYD intervention for weekly alcohol consumption at 3 months postintervention and the long-term preventive effect of binge drinking frequency at 6 months postintervention.
In our previous study, we did not find significant main effects when we tested the effectiveness of the WDYD intervention at 1 and 6 months postintervention . However, by using EMA and LGC modeling techniques, overall significant intervention effects were generated for weekly alcohol consumption and frequency of binge drinking that were sustained 3 and 6 months postintervention. This finding stresses the importance of using a large number of measurements in combination with appropriate statistical techniques to obtain higher precision in intervention effectiveness and minimize the danger of inaccurate conclusions about intervention effectiveness when using limited follow-up time points. Moreover, the use of EMA enables one to examine whether intervention effectiveness on the treatment outcome varied over time and helps determine the time at which the intervention effects stopped and the time at which booster sessions are needed to strengthen and/or extend intervention effects.
Limitations and Strengths
Limitations of this study include the use of a large number of measurements by means of EMA, which might have affected the observed changes in the outcome measures by the act of assessing [- ], yet participants in both conditions received weekly posttest measurements. If there was assessment reactivity, it could lead to underestimates of the true intervention effect [ ]. Also, EMA could impose participant burden and reduce compliance because of the length of the survey entry, the frequency of responses, and the length of the study period [ ]. Nonetheless, noncompliance and attrition were low in the current study. It seemed to be important to provide a briefing about the study procedure before the study onset, use short and well-conducted surveys, and offer a monetary incentive after study completion. In addition, the use of EMA might even alleviate sample size requirements because it provides more refined outcome measures that are more sensitive to change, thereby making studies less difficult and less expensive to conduct [ ]. Additionally, the effect sizes of the WDYD intervention were small but comparable to those reported in other Web-based brief alcohol interventions [ , ]. Despite the small absolute differences in alcohol use between the conditions, the advantage lies in the inclusion of all the effects of the WDYD intervention over time across a far larger group of heavy drinkers with less serious alcohol-related problems resulting in a greater societal gain than reducing problem drinking among a smaller number of dependent drinkers, known as the prevention paradox, that is used to justify a population strategy of prevention [ ]. Further, the representativeness of the study sample might have been affected because of the convenience sampling strategy, although the majority of trials on Web-based brief alcohol interventions have used this type of sampling strategy (eg, [ ]) in which participants are selected based on availability. Moreover, contamination between conditions might have occurred if participants in the experimental condition shared the link of the WDYD intervention with participants in the control condition. Nonetheless, WDYD is not yet available online; thereby, it reduces the likelihood of contamination between conditions. Additionally, the EMA measurements relied on self-reported measures with 7-day recall, which still remains subject to measurement errors because data were not collected in the event and precise recall of alcohol use decreases after 2 or 3 days due to memory deficits [ ]. True in-the-event measures would be very difficult over long time periods. However, this is the first study using 30 weekly EMA measurements to assess outcome measures, thereby generating outcome measures that are much closer to the actual drinking behavior of individuals than any other trial on Web-based brief alcohol interventions in the current alcohol prevention literature. Another limitation is that the outcome measures of weekly alcohol consumption and frequency of binge drinking have not been validated for online use. However, these outcome measures have been validated in paper-based surveys [ , ]. In addition, research has shown that online survey data can be equal or superior to that of equivalent paper-based survey data [ ]. Furthermore, participants were not blinded to the assigned interventions, which is a common limitation in Web-based trials [ ]. Participants who are aware that they have been assigned to the experimental condition might have favorable expectations or increased apprehension and participants assigned to the control condition might feel deprived or relieved, which can affect their responses on the outcome measures. Finally, one should be careful in generalizing our findings to students who are not ready to change their alcohol use, individuals younger than 18 years, and those who do not attend or have not attended a college or university.
Strengths of this study included the online weekly EMA methodology for assessing alcohol use and intervention effectiveness over time while maintaining a high retention rate. First, the use of a large number of measurements over time by means of EMA enabled us to assess changes in alcohol use and intervention effectiveness over time while taking into account the fluctuating nature of alcohol use among students. Second, the coverage strategy of EMA minimized recall bias because of a relative short reference period (ie, 1 week), thereby generating more ecologically valid outcome measures of self-reported drinking behaviors . Third, the use of online surveys had the advantage over paper-and-pencil surveys because it reduced the likelihood of entry errors while improving cost-effectiveness [ ]. Fourth, the use of EMA in combination with LGC modeling techniques allowed diminishing statistical errors by generating overall intervention effects resulting in more reliable outcome measures and a higher precision in intervention effectiveness. In addition, the WDYD intervention is based upon the intervention mapping protocol, which is a sound framework for theoretical- and evidence-based development, implementation, and evaluation of effective behavior change interventions [ ]. Moreover, the WDYD intervention incorporated components (eg, personalized normative feedback) that are successful in reducing heavy drinking among student populations [ ].
The findings of the current study suggest that the WDYD intervention can prevent an increase in weekly alcohol consumption and frequency of binge drinking among heavy-drinking students that is sustained at 3 to 6 months postintervention. The collaboration with the Trimbos Institute can ensure an adequate large-scale implementation of the WDYD intervention by incorporating it in their materials and programs . In addition, the findings indicate the relevance of including a large number of measurements by means of EMA for assessing the outcome measures and evaluating the intervention effectiveness to obtain higher precision in future alcohol prevention trials. If a large number of measurements with extremely short reference periods (ie, 2 hours) are used to assess outcome measures, smartphones might be more beneficial than online surveys because they can capture data regardless of time and location of the participant [ ]. Moreover, future research should identify whether alcohol-related cognitions (eg, self-efficacy) account for the observed outcomes to help explain why Web-based brief alcohol interventions are effective in reducing or, in our case, preventing an increase in alcohol use among heavy-drinking students, especially considering that most Web-based brief alcohol interventions are designed to affect alcohol-related cognitions that determine heavy drinking in young adults [ , ].
The WDYD Web-based brief alcohol intervention was shown to be effective in preventing an increase in weekly alcohol consumption and frequency of binge drinking directly after the intervention among heavy-drinking students that was sustained at 3 and 6 months postintervention. Moreover, the findings emphasize the strengths of using EMA and statistical techniques, such as LGC, in testing the intervention effectiveness that would otherwise remain undetected.
The major funding agency ZonMw, The Netherlands Organization for Health Research and Development, supported this study (grant number 50-50110-96-682).
Conflicts of Interest
Multimedia Appendix 1
CONSORT-EHEALTH checklist V1.6.2 [PDF File (Adobe PDF File), 997KB ].
- Maggs JL, Williams LR, Lee CM. Ups and downs of alcohol use among first-year college students: Number of drinks, heavy drinking, and stumble and pass out drinking days. Addict Behav 2011 Mar;36(3):197-202 [FREE Full text] [CrossRef] [Medline]
- Ragsdale K, Porter JR, Mathews R, White A, Gore-Felton C, Mcgarvey EL. “Liquor before beer, you're in the clear”: binge drinking and other risk behaviours among fraternity/sorority members and their non-Greek peers. J Subst Use 2011;17:323-339. [CrossRef]
- Wicki M, Kuntsche E, Gmel G. Drinking at European universities? A review of students' alcohol use. Addict Behav 2010 Nov;35(11):913-924. [CrossRef] [Medline]
- White A, Kavanagh D, Stallman H, Klein B, Kay-Lambkin F, Proudfoot J, et al. Online alcohol interventions: a systematic review. J Med Internet Res 2010;12(5):e62 [FREE Full text] [CrossRef] [Medline]
- Voogt CV, Poelen EA, Kleinjan M, Lemmers LA, Engels RC. The effectiveness of the 'what do you drink' Web-based brief alcohol intervention in reducing heavy drinking among students: a two-arm parallel group randomized controlled trial. Alcohol Alcohol 2013;48(3):312-321. [CrossRef] [Medline]
- Bewick BM, Trusler K, Barkham M, Hill AJ, Cahill J, Mulhern B. The effectiveness of Web-based interventions designed to decrease alcohol consumption--a systematic review. Prev Med 2008 Jul;47(1):17-26. [CrossRef] [Medline]
- Hustad JT, Barnett NP, Borsari B, Jackson KM. Web-based alcohol prevention for incoming college students: a randomized controlled trial. Addict Behav 2010 Mar;35(3):183-189 [FREE Full text] [CrossRef] [Medline]
- Kypri K, Hallett J, Howat P, McManus A, Maycock B, Bowe S, et al. Randomized controlled trial of proactive Web-based alcohol screening and brief intervention for university students. Arch Intern Med 2009 Sep 14;169(16):1508-1514. [CrossRef] [Medline]
- Kypri K, McCambridge J, Vater T, Bowe SJ, Saunders JB, Cunningham JA, et al. Web-based alcohol intervention for Māori university students: double-blind, multi-site randomized controlled trial. Addiction 2013 Feb;108(2):331-338 [FREE Full text] [CrossRef] [Medline]
- Gmel G, Daeppen JB. Recall bias for seven-day recall measurement of alcohol consumption among emergency department patients: implications for case-crossover designs. J Stud Alcohol Drugs 2007 Mar;68(2):303-310. [Medline]
- Neighbors C, Atkins DC, Lewis MA, Lee CM, Kaysen D, Mittmann A, et al. Event-specific drinking among college students. Psychol Addict Behav 2011 Dec;25(4):702-707 [FREE Full text] [CrossRef] [Medline]
- Voogt CV, Poelen E, Kleinjan M, Lemmers L, Engels R. The development of a Web-based brief alcohol intervention in reducing heavy drinking among college students: An Intervention Mapping approach. Health Promot Int 2013 Mar 22:1-11. [CrossRef]
- Bartholomew K, Parcel G, Kok GS, Gottlieb N. Intervention Mapping: Designing Theory- and Evidence-Based Health Promotion Programs. Mountain View, CA: Mayfield; Oct 15, 2001.
- Miller WR, Rollnick S. Motivational Interviewing: Preparing People for Change (second edition). New York: Guilford Press; Apr 12, 2002.
- De Vries H, Dijkstra M, Kuhlman P. Self-efficacy: The third factor besides attitude and subjective norm as a predictor of behavioral intentions. Health Educ Res 1988;3:273-282. [CrossRef]
- Shiffman S. Ecological momentary assessment (EMA) in studies of substance use. Psychol Assess 2009 Dec;21(4):486-497 [FREE Full text] [CrossRef] [Medline]
- Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol 2008;4:1-32. [Medline]
- Trull TJ, Ebner-Priemer UW. Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol Assess 2009 Dec;21(4):457-462. [CrossRef] [Medline]
- Muthén B, Brown CH, Masyn K, Jo B, Khoo ST, Yang CC, et al. General growth mixture modeling for randomized preventive interventions. Biostatistics 2002 Dec;3(4):459-475 [FREE Full text] [CrossRef] [Medline]
- Muthén LK, Muthén B. Mplus User’s Guide (Sixth Edition). Los Angeles, CA: Muthén & Muthén; Apr 2010.
- Cunningham JA, Wild TC, Cordingley J, van Mierlo T, Humphreys K. A randomized controlled trial of an Internet-based intervention for alcohol abusers. Addiction 2009 Dec;104(12):2023-2032 [FREE Full text] [CrossRef] [Medline]
- Hardy SA, Thiels C. Using latent growth curve modeling in clinical treatment research: An example comparing guided self-change and cognitive behavioral therapy treatments for bulimia nervosa. Int J Clin Hlth Psyc 2009;9(1):51-71.
- Voogt CV, Poelen EA, Kleinjan M, Lemmers LA, Engels RC. Targeting young drinkers online: the effectiveness of a Web-based brief alcohol intervention in reducing heavy drinking among college students: study protocol of a two-arm parallel group randomized controlled trial. BMC Public Health 2011;11:231 [FREE Full text] [CrossRef] [Medline]
- Babor T, Higgins-Biddle JC, Saunders J, Monteiro MG. The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Care (second edition). Geneva: World Health Organization. Department of Mental Health and Substance Dependence; 2001:1-40.
- Boon B, Risselada A, Huiberts A, Riper H, Smit F. Curbing alcohol use in male adults through computer generated personalized advice: randomized controlled trial. J Med Internet Res 2011;13(2):e43 [FREE Full text] [CrossRef] [Medline]
- Gezondheidsraad. Richtlijnen voor goed voeding 2006 [Guidelines for Healthy Nutrition 2006]. Den Haag, the Netherlands: Gezondheidsraad; 2006.
- Lemmens P, Tan ES, Knibbe RA. Measuring quantity and frequency of drinking in a general population survey: a comparison of five indices. J Stud Alcohol 1992 Sep;53(5):476-486. [Medline]
- Tabachnick BG, Fidell LS. Using Multivariate Statistics (5th edition). Boston, MA: Allyn and Bacon; Mar 03, 2007.
- Voogt CV, Kuntsche E, Kleinjan M, Poelen E, Lemmers L, Engels R. Using ecological momentary assessment in testing the effectiveness of an alcohol intervention: A two-arm parallel group randomized controlled trial. PLOS ONE 2013 Nov;8(11):e78436. [CrossRef]
- Landerman LR, Land KC, Pieper CF. An empirical evaluation of the predictive mean matching method for imputing missing values. Socio Meth Res 1997;26(1):3-33. [CrossRef]
- Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling 1999;6(1):1-55. [CrossRef]
- Cohen J. Statistical Power Analysis for the Behavioural Sciences (2nd Edition). Hillsdale, NJ: Lawrence Erlbaum Associates; Jul 01, 1988.
- Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869 [FREE Full text] [Medline]
- Beets MW, Flay BR, Vuchinich S, Li KK, Acock A, Snyder FJ, Tobacco Etiology Research Network. Longitudinal patterns of binge drinking among first year college students with a history of tobacco use. Drug Alcohol Depend 2009 Jul 1;103(1-2):1-8. [CrossRef] [Medline]
- Demers A, Kairouz S, Adlaf EM, Gliksman L, Newton-Taylor B, Marchand A. Multilevel analysis of situational drinking among Canadian undergraduates. Soc Sci Med 2002 Aug;55(3):415-424. [Medline]
- Clifford PR, Maisto SA, Davis CM. Alcohol treatment research assessment exposure subject reactivity effects: part I. Alcohol use and related consequences. J Stud Alcohol Drugs 2007 Jul;68(4):519-528. [Medline]
- Kypri K, Langley JD, Saunders JB, Cashell-Smith ML. Assessment may conceal therapeutic benefit: findings from a randomized controlled trial for hazardous drinking. Addiction 2007 Jan;102(1):62-70. [CrossRef] [Medline]
- Maisto SA, Clifford PR, Davis CM. Alcohol treatment research assessment exposure subject reactivity effects: part II. Treatment engagement and involvement. J Stud Alcohol Drugs 2007 Jul;68(4):529-533. [Medline]
- McCambridge J. [Commentary] Research assessments: instruments of bias and brief interventions of the future? Addiction 2009 Aug;104(8):1311-1312. [CrossRef] [Medline]
- Carey KB, Scott-Sheldon LA, Elliott JC, Bolles JR, Carey MP. Computer-delivered interventions to reduce college student drinking: a meta-analysis. Addiction 2009 Nov;104(11):1807-1819 [FREE Full text] [CrossRef] [Medline]
- Rooke S, Thorsteinsson E, Karpin A, Copeland J, Allsop D. Computer-delivered interventions for alcohol and tobacco use: A meta-analysis. Addiction 2011;105(8):1381-1390. [CrossRef]
- Poikolainen K, Paljärvi T, Mäkelä P. Alcohol and the preventive paradox: serious harms and drinking patterns. Addiction 2007 Apr;102(4):571-578. [CrossRef] [Medline]
- Spijkerman R, Roek MA, Vermulst A, Lemmers L, Huiberts A, Engels RC. Effectiveness of a Web-based brief alcohol intervention and added value of normative feedback in reducing underage drinking: a randomized controlled trial. J Med Internet Res 2010;12(5):e65 [FREE Full text] [CrossRef] [Medline]
- Hibell B, Andersson B, Bjarnasson T, Ahlström S, Balakivera O, Kokkevi A, et al. The ESPAD Report 2003: Alcohol and other drug use among students in 35 European countries. Stockholm, Sweden: The Swedish Council for Information on Alcohol and other Drugs; 2004. URL: http://www.espad.org/Uploads/ESPAD_reports/2003/The_2003_ESPAD_report.pdf [accessed 2013-12-19] [WebCite Cache]
- Chang TZ, Vowles N. Strategies for improving data reliability for online surveys: A case study. IJECS 2013;4(1):121-130. [CrossRef]
- Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
- Miller ET, Neal DJ, Roberts LJ, Baer JS, Cressler SO, Metrik J, et al. Test-retest reliability of alcohol measures: is there a difference between Internet-based assessment and traditional methods? Psychol Addict Behav 2002 Mar;16(1):56-63. [Medline]
- Carey KB, Scott-Sheldon LA, Carey MP, DeMartini KS. Individual-level interventions to reduce college student drinking: a meta-analytic review. Addict Behav 2007 Nov;32(11):2469-2494 [FREE Full text] [CrossRef] [Medline]
- Burger C, Riemer V, Grafeneder J, Woisetschlager B, Vidovic D, Hergovich A. Reaching the mobile respondent determinants of high-level mobile phone use among a high-coverage group. Soc Sci Comput Rev 2009 Dec 02;28(3):336-349. [CrossRef]
- Cullum J, Armeli S, Tennen H. Drinking norm-behavior association over time using retrospective and daily measures. J Stud Alcohol Drugs 2010 Sep;71(5):769-777 [FREE Full text] [Medline]
- Danielsson AK, Wennberg P, Tengström A, Romelsjö A. Adolescent alcohol use trajectories: Predictors and subsequent problems. Addict Behav 2010 Sep;35(9):848-852. [CrossRef] [Medline]
|AUDIT: Alcohol Use Disorders Identification Test|
|CFI: Comparative Fit Index|
|CONSORT: Consolidated Standards of Reporting Trials|
|EMA: ecological momentary assessment|
|ESM: experience sampling methods|
|LGC: latent growth curve|
|RMSEA: Root Mean Square Error of Approximation|
|TLI: Tucker-Lewis Index|
|WDYD: What Do You Drink|
Edited by G Eysenbach; submitted 05.07.13; peer-reviewed by L Le Garjean, D Ho, J Merrill; comments to author 18.09.13; revised version received 18.10.13; accepted 04.11.13; published 08.01.14
©Carmen Voogt, Emmanuel Kuntsche, Marloes Kleinjan, Evelien Poelen, Rutger Engels. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.01.2014.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.