This is an openaccess article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Almost a decade ago, Sweden became the first country to implement a national system enabling student health care centers across all universities to routinely administer (via email) an electronic alcohol screening and brief intervention to their students. The Alcohol email assessment and feedback study dismantling effectiveness for university students (AMADEUS1) trial aimed to assess the effect of the student health care centers’ routine practices by exploiting the lack of any standard timing for the email invitation and by masking trial participation from students. The original analyses adopted the conventional null hypothesis framework, and the results were consistently in the expected direction. However, since for some tests the
The outcomes of the AMADEUS1 trial were derived from the first 3 items of the Alcohol Use Disorders Identification Test (AUDITC). The aim of this paper was to reanalyze the two primary outcomes of the AMADEUS1 trial (AUDITC scores and prevalence of risky drinking), using the same models used in the original publication but applying a Bayesian inference framework and interpretation.
The same regression models used in the original analysis were employed in this reanalysis (linear and logistic regression). Model parameters were given uniform priors. Markov chain Monte Carlo was used for Bayesian inference, and posterior probabilities were calculated for prespecified thresholds of interest.
Where the null hypothesis tests showed inconclusive results, the Bayesian analysis showed that offering an intervention at baseline was preferable compared to offering nothing. At followup, the probability of a lower AUDITC score among those who had been offered an intervention at baseline was greater than 95%, as was the case when comparing the prevalence of risky drinking.
The Bayesian analysis allows for a more consistent perspective of the data collected in the trial, since dichotomization of evidence is not looked for at some arbitrary threshold. Results are presented that represent the data collected in the trial rather than trying to make conclusions about the existence of a population effect. Thus, policy makers can think about the value of keeping the national system without having to navigate the treacherous landscape of statistical significance.
ISRCTN Registry ISRCTN28328154; http://www.isrctn.com/ISRCTN28328154
Alcohol consumption contributed to more than 4.5% of deaths globally in 2016 [
Early initiatives to use digital means of delivering alcohol interventions came in the form of electronic screening and brief interventions (eSBIs) [
Metaanalyses suggest that there exists a small positive effect of eSBIs on the amount of alcohol consumed weekly in the short term, with a Cohen
Almost a decade ago, Sweden became the first country to implement a national system enabling student health care centers across all universities to routinely administer eSBIs. The system, which is still routinely used today, sends an email to all university students with an invitation and a hyperlink to a 10item web questionnaire which is then followed by personal feedback and advice. At the time this system was introduced, there was some evidence of the effectiveness of eSBIs but there was a paucity for largescale, multisite, effectiveness trials of routine care systems.
The Alcohol email assessment and feedback study dismantling effectiveness for university students (AMADEUS1) trial [
During the autumn term of 2011, all students in semesters 1, 3 and 5 at two universities in Sweden (Linköping and Luleå) were included in the AMADEUS1 trial. Notably, students’ email addresses were randomized into 3 groups (Group 1, Group 2, and Group 3) prior to any invitation or contact with the students. Ethical concerns with the use of this type of masking was considered and approved by the Regional Ethical Committee in Linköping, Sweden (No 2010/29131). In a subsequent trial (the AMADEUS2 trial [
On September 5, 2011, Group 1 and Group 2 were sent an email from the student health care center with a hyperlink to a web questionnaire comprising 10 items which assessed their current alcohol consumption, masked as part of routine care. Group 1 was additionally told that they would also get feedback, which they received immediately after responding to the questionnaire. Group 2 was thanked for their participation and offered a hyperlink to a website with general information about alcohol, which was not believed to have any content helpful for supporting behavior change. Group 3 was not contacted at this time.
Three months after the initial email to Group 1 and Group 2, all three groups were sent identical emails with an invitation to participate in a webbased general lifestyle survey where 3 out of the 15 items were the first 3 items of the Alcohol Use Disorders Identification Test (AUDITC [
Outcomes of the AMADEUS1 trial were derived from the 3 AUDITC items in the general lifestyle survey. This reanalysis will focus on two primary outcomes: AUDITC scores and prevalence of risky drinking. In Sweden, risky drinkers are those who fulfil at least one of two criteria: (1) heavy episodic drinking of at least 4 (female) or 5 (male) standard drinks of alcohol on one occasion the past month; or (2) consuming more than 9 (female) or 14 (male) standard drinks of alcohol per week. One standard drink is defined as 12 grams of alcohol in Sweden.
The current goal is to reanalyze the two primary outcomes of the AMADEUS1 trial, using the same models used in the original publication but also using a Bayesian inference framework and interpretation.
In the original analysis of the AMADEUS1 trial, normal regression was used to contrast AUDITC scores (logtransformed) and logistic regression was used to contrast risky drinking. Both models were adjusted for baseline variables. In this Bayesian analysis, the same regression models were used, and uniform priors were applied to all model parameters. The full specifications of the Bayesian models can be seen in the following 2 equations. Separate analyses were done comparing Group 1 versus Group 3 and Group 2 versus Group 3. In all cases, Group 3 was considered the control group and Group 1 and Group 2 were considered intervention groups.
Equation 1:
Equation 2:
When contrasting AUDITC scores (Equation 1), the primary interest of the analyses was the regression coefficient for the group variable (α_{1}). A negative value for α_{1} suggests that the group which was randomized to receive an intervention (Group 1 and Group 2 respectively) had, on average, lower AUDITC scores at followup than the group which was randomized to the control setting (Group 3). Coefficients were back transformed prior to inspection. Informed by the original analysis, it was decided that thresholds of interest for which the marginal posterior distribution for α_{1} should be inspected were 0, –0.02, and –0.04. The thresholds were chosen to communicate whether offering an intervention is preferable to not doing so (the 0 threshold), and to indicate the magnitude of the difference between groups (–0.02 and –0.04).
When contrasting risky drinking (Equation 2), the primary interest was the regression coefficient for the group variable (β_{1}), that is the log of the odds ratio (OR) between the group which was randomized to an intervention (Group 1 and Group 2 respectively) and the group which was randomized to the control setting (Group 3). Coefficients were exponentiated before inspection, thus a value of β_{1} lower than 1 would suggest that the odds of risky drinking in the intervention group was lower than the odds in the control. Informed by the original analysis, it was decided that thresholds of interest for which the marginal posterior distribution for β_{1} should be inspected was 1, 0.9 and 0.8. Again, the thresholds were chosen to communicate whether offering an intervention is preferable to not doing so (the 1 threshold), and to indicate the magnitude of the difference between the groups (0.9 and 0.8).
Markov chain Monte Carlo was used for Bayesian inference (RStan version 2.16.2). For each model, 50,000 iterations were run with 25,000 warmup iterations in four chains. Inference for AUDITC scores (Equation 1) took 3.5 minutes, and for risky drinking (Equation 2) 5.5 minutes. All computations were done on a MacBook Pro (2017 model).
A total of 14,910 students were randomized into the 3 arms of the trial. In Group 1, 36.2% (1798/4969) of participants completed the eSBI, 32.6% (1621/4969) of Group 2 participants completed the alcohol screening questionnaire, and as previously discussed Group 3 was not contacted at this point. Approximately half of all students responded to the general lifestyle survey that was sent three months after randomization: 51.2% (2546/4969) in Group 1, 52.2% (2594/4969) in Group 2, and 53.7% (2669/4972) in Group 3.
The original analysis for the AMADEUS1 trial is presented in
As a reminder,
Original analysis of AUDITC and risky drinking at followup, comparing Group 1 versus 2 and Group 2 versus 3.
Categories  Group 1 (n=2546)  Group 2 (n=2594)  Group 3 (n=2669)  Group 1 versus 3  Group 2 versus 3  

Regression coefficient^{a}, 95% CI  Regression coefficient^{a}, 95% CI  
AUDITC^{b}  3.46 (3.09)^{c}  3.44 (3.17)^{c}  3.60 (3.14)^{c}  –0.032 (–0.066 to 0.003)  .07  –0.038 (–0.072 to –0.002)  .04 
Risky drinking^{d}  1136 (44.6)^{e}  1194 (46.0)^{e}  1288 (48.3)^{e}  0.85 (0.76 to 0.95)  .006  0.90 (0.81 to 1.01)  .08 
^{a}Linear coefficient for AUDITC scores (back transformed) and odds ratio for risky drinking (adjusted for sex, age, university, and semester).
^{b}AUDITC: Alcohol Use Disorders Identification Test.
^{c}Geometric mean (SD). Approximate standard deviation backcalculated from the logscale.
^{d}Risky drinking: heavy episodic drinking ≥1 a month or weekly consumption >14 for men and >9 for women (Swedish national guidelines).
^{e}n (%).
The computational result of a Bayesian analysis using Markov chain Monte Carlo uses samples from the posterior distribution of each parameter of interest. Histograms of these samples are shown in
Rather than just visually inspecting the histograms, the samples drawn during inference can be used to calculate probabilities of interest (
Samples from the posterior distribution of α_{1} in the AUDITC model when comparing Group 1 versus Group 3 (Equation 1, back transformed). AUDITC: Alcohol Use Disorders Identification Test.
Samples from the posterior distribution of α_{1} in the AUDITC model when comparing Group 2 versus Group 3 (Equation 1, back transformed). AUDITC: Alcohol Use Disorders Identification Test.
Samples from the posterior distribution of β_{1} in the risky drinking model when comparing Group 1 versus Group 3 (Equation 2, exponentiated).
Samples from the posterior distribution of β_{1} in the risky drinking model when comparing Group 2 versus Group 3 (Equation 2, exponentiated).
Bayesian analysis of AUDITC at followup comparing Group 1 versus 3 and Group 2 versus 3.

Group 1 versus 3  Group 2 versus 3  

Threshold 1  Threshold 2  Threshold 3  Threshold 1  Threshold 2  Threshold 3 
Regression coefficient^{a} (AUDITC^{b})  <0  <–0.02  <–0.04  <0  <–0.02  <–0.04 
Marginal posterior probability (%)  96.4  75.7  32.9  98.1  83.7  44.4 
^{a}Back transformed linear regression coefficient (model adjusted for sex, age, university, and semester).
^{b}AUDITC: Alcohol Use Disorders Identification Test.
Bayesian analysis of risky drinking at followup comparing Group 1 versus 3 and Group 2 versus 3.

Group 1 versus 3  Group 2 versus 3  

Threshold 1  Threshold 2  Threshold 3  Threshold 1  Threshold 2  Threshold 3 
Odds ratio^{a} (Risky drinking)  <1  <0.9  <0.8  <1  <0.9  <0.8 
Marginal posterior probability (%)  99.7  82.4  13.4  96.1  46.7  1.6 
^{a}Logistic regression coefficient in terms of odds ratios (model adjusted for sex, age, university, and semester).
When comparing the analysis done in a null hypothesis framework with one done within the Bayesian framework, it is important to remind oneself of what the quantities represent as the questions being asked and answered are different.
The null hypothesis testing approach aims to put forth evidence about the population value of a parameter (ie, the existence of an effect on the entire population). The
On the other hand, the Bayesian approach only concerns itself with the data at hand. It does not attempt to say anything about a population level effect, but instead calculates posterior distributions over model parameters. We can use these posterior distributions to calculate the probability of there being a difference between groups with respect to different trial outcomes.
When contrasting AUDITC scores (Equation 1) using the null hypothesis framework (
In the original report [
The Bayesian approach (
In
The Bayesian approach (
Clinically significant effect sizes are not universal, as they depend on the context in which the intervention can be offered and must be decided upon given cost, alternative interventions, ethical and practical concerns, and so on. One of the benefits of using a Bayesian approach is that we have access to a posterior distribution over the parameters of our model, which allows us to answer questions such as, “What is the probability that the effect is X or greater?” Therefore, we can evaluate the probability of clinically significant effect sizes in several different contexts. For instance, at the time of the AMADEUS1 trial, student health care centers in Sweden did not have any means of reaching the entire student population with a brief intervention, thus there were no alternative interventions to the eSBI on trial. In addition, there was very little cost involved in adopting the eSBI into routine practice.
The years to come after the AMADEUS1 trial saw many more trials of eSBIs, and as was mentioned earlier, metaanalyses suggest a small positive effect of eSBIs on the amount of alcohol consumed weekly in the short term.
The AMADEUS1 trial was unconventional in the sense that participants were randomized prior to being invited to the trial. This design allowed for a naturalistic study context and allowed for methodological advantages. However, participation rates were lower than would be expected in a more traditional setting where participants are randomized after registering interest in the trial (eg, only 36.2% [1798/4969] of participants allocated to Group 1 completed the eSBI). The overall followup rate was not remarkable at 52% (7764/14,910), which at the time was considered average for eHealth trials. Since missing at random cannot be guaranteed, effect sizes should be considered in the light that bias might have been introduced due to lower than ideal followup rates.
In the original publication of the AMADEUS1 trial, we summarized the main results as follows [
Since not all
In this Bayesian reanalysis, we may instead summarize our findings as: There is 96.4% probability that Group 1 had a lower AUDITC score on average than did Group 3, and there is a 99.7% probability that the prevalence of risky drinking was lower in Group 1 compared to Group 3 (and a further 82.4% probability that the OR was less than 0.9). This then allows us to go forth and inspect the posterior distributions at effect sizes that are clinically significant in different contexts and discuss whether the intervention should be adopted into routine practice.
Trace plots.
Alcohol email assessment and feedback study dismantling effectiveness for university students
Alcohol Use Disorders Identification Test
electronic health
electronic screening and brief intervention
odds ratio
MB owns a private company that develops and distributes evidencebased lifestyle interventions to be used in health care settings, including student health care centers.