The study design consisted of a two-arm randomized controlled trial (RCT) developed to assess the neuropsychological effects of mHealth applications in alcoholics. (The trial was registered retrospectively because our funding agency, the Portuguese National Science Foundation [FCT] does not require registration for psychological tests nor does our home country have a registration system.) Our sample size was estimated based on a priori power analysis according to commonly accepted standards in the field (see Statistical Procedures) and agreed to in advance with the clinical institution where the trial took place.

The method of patient assignment was based on simple randomization with a random number generator. The patients were randomly assigned to experimental and control groups in a specialized institution for treatment of alcohol dependence (see Multimedia Appendix 1 for the CONSORT eHealth Checklist). The trial was approved by the Ethics Committee of the research center where the authors were affiliated at the time (Centre for the Study of Cognitive and Learning Psychology [CEPCA]) and adhered to the principles of the World Medical Association’s Declaration of Helsinki.

Sixty-eight patients diagnosed with ADS according to DSM-IV criteria were recruited from a specialized institution for treatment of alcohol dependence, the Novo Rumo Clinic, São João de Deus Institute in the Lisbon region, Portugal, and were asked to participate in a study on the effects of their treatment on cognitive abilities. In the experimental condition, they were told that their treatment would include cognitive exercises. Due to a dropout rate of 20.6% (14/68 patients), the final sample consisted of 54 patients.

Only patients that scored higher than the cut-off values for their age (see Outcomes) on the Mini Mental State Examination (MMSE) [35] and with no clinical scores on the Symptoms Checklist Revised (SCL-90-R) [36] were included in the study.

Patients continued during the entire study their regular medication regimen consisting of anxiolytics, mostly Diazepam and Tiapride, which help minimize withdrawal symptoms, and vitamins. Each patient’s assistant psychiatrist guaranteed the stability of the medication regimens throughout the program.

Patients with dependency on substances other than alcohol or with a history of previous neurological disorders were excluded from the study. Patients were also screened for minimal computer literacy, but no patient was excluded due to this criterion (ie, all patients demonstrated the minimum literacy required).

After initial recruitment and screening, participants were randomly assigned to either the experimental group with mHealth SG-based cognitive stimulation plus treatment-as-usual (n=33) or to the control group (n=35), which received solely treatment-as-usual. The treatment-as-usual consisted of an alcohol-abstinence program adapted from the Minnesota Model (see [37] for a detailed description of this program) and lasted for an average period of 1 month.

Figure 1 illustrates the flow of participants in the experimental group throughout the protocol. Participants failing to complete training sessions within the assigned time-frame were considered to have dropped out and their data were not analyzed.

The trial took place over a period of 6 months, and no changes to the stimulus program were made during this period. Both the treatment and the assessments took place on location at the clinic where participants were recruited.

Each participant underwent two complete neuropsychological assessments, once after they had undergone screening and given their written informed consent to treatment and assessment, and again after at least 30 days. In the case of the treatment group, this was done after completing the intervention.

The intervention consisted of ten 60-minute sessions of cognitive stimulation with mobile technology using SG (2-3 sessions per week over the usual 4-6 week period of treatment). No institutional affiliations were presented in the eHealth media. The executive training exercises performed by participants in the experimental group were selected in order to develop cognitive abilities related to executive functioning. Each session started with a brief training period, when participants were able to (re)acquire interaction skills with the touchscreen devices. Participants accessed the exercises freely over the Internet, and their responses were registered using the input from the device’s touchscreen.

Therapists from the research and intervention team were involved in both the recruitment and the cognitive stimulation. These therapists were introduced to patients by in-house therapists and asked patients to participate in the study, explaining its benefits, duration, and demands on patients’ time and commitment. During the assessments, other therapists provided, explained, and collected the assessment forms. In the cognitive simulation sessions, the first group of therapists provided the mobile devices, launched the exercises, and explained how they worked to participants.

The mobile cognitive stimulation program consisted of several mobile applications developed to run on Android OS, which were adapted from the traditional paper-and-pencil exercise conceived for cognitive stimulation on patients with cognitive impairments, independently of the cause, but selected for their relevance to address the most common cognitive impairments in addicts. Cognitive stimulation in each session comprised attention, working memory, and logical reasoning exercises (see Textbox 1 for a more detailed description; these exercises (Figure 2) are available online [34]). The level of difficulty of each task increased progressively throughout the cognitive stimulation rationale. In the last session, the same neuropsychological tests used in the first assessment were again applied.

The hardware used to perform the exercises consisted of Samsung Galaxy 10.1ʺ tablets. The applications were developed using Unity 2.5, and their alpha and beta versions had been previously tested by a group of students.

All neuropsychological assessments were carried out with the pencil-and-paper forms of well-established cognitive tests. Global cognitive abilities were assessed with the MMSE [35], a brief screening test that has been validated for the Portuguese population [38]. The MMSE assesses diverse aspects of cognitive mental function, allowing an overall assessment of cognitive performance based on 30 items grouped into 6 categories: Orientation, Retention, Attention and Calculation, Language, and Visual-spatial abilities. The maximum score is 30 points and the current cut-off values for the Portuguese population were estimated according to age and education, specifically for people aged over 40 years: (1) 0-2 years of schooling—22, (2) 3-6 years of schooling—24, and (3) more than 7 years of schooling—27. MMSE was dictated on the basis of education.

Frontal brain functioning was evaluated with the Frontal Assessment Battery (FAB) [39]. The FAB assesses conceptualization, mental flexibility, motor programming, sensitivity to interference, inhibitory control, and environmental autonomy. The total score of the FAB (maximum 18 points) is estimated through the sum of each of the subtest scores. The FAB assesses 6 domains of frontal functioning through the ability to generate similarities (conceptualization), verbal fluency (mental flexibility), Luria’s motor series (motor programming), conflicting instructions (sensitivity to interference), go-no go paradigm (inhibitory control), and prehension behavior assessments (environmental autonomy).

Cognitive flexibility was measured with the WCST [40], which evaluates cognitive functions in several executive domains, namely the ability to develop and maintain appropriate strategies for problem solving and planning, and the ability to use environmental feedback to modify a cognitive response. The WCST assesses cognitive abilities such as abstraction, mental flexibility, and sustained attention. We focused on the overall index of performance through correct responses, categories, and perseverative errors. In the current study, we used the short version of the test (WCST-64).

Psychomotor processing speed and attentional abilities were estimated through the Color Trail Test (CTT) [41]. The CTT assesses focused and divided attention, sequencing, mental flexibility, visual search, and motor functions. The CTT consists of an A4 sheet containing circles with numbers printed in yellow and pink. Participants are asked to link the numbered circles in the right order, as fast as possible, and without lifting the pen. The CTT consists of two different forms (CTT1 and CTT2), which differ in difficulty. Two measures of performance were analyzed: errors and execution time.

The variables evaluated during the neuropsychological assessments were submitted to parametric statistical analyses. The dependent variables were based on the neuropsychological outcomes from the MMSE, FAB, WCST, and the CTT. Scores reported refer to before intervention (baseline) and after intervention (follow-up).

Sample and baseline characteristics were compared between the experimental and control groups with Student’s t test for independent samples for interval dependent variables and chi-square test for categorical dependent variables.

The statistical analyses of evaluation outcomes were based on generalized linear model procedures using type III Sum of Squares. A full factorial model (within-between interaction) was tested using repeated measures analysis of variance (ANOVA). The factorial design consisted of one within-subjects factor with 2 levels (baseline vs follow-up) and one between-subjects factor with 2 levels (experimental vs control group). Missing data imputation was completed using linear interpolation method. A significance level of .05 was adopted for all statistical procedures. Post-hoc effect sizes are eta-square in the ANOVA procedure, and within-between interactions were tested using simple effects.

Inferential statistics were carried out using IBM SPSS v.20. A priori power analysis was estimated with G*Power v.3.1 with Cohen’s f effect size for F tests [42].

The expected effect size for calculating the required sample size was 0.35 (medium effect) based on a power of 0.80 for a significance level of .05. According to these criteria, a total sample size of 68 patients would be required for this trial. This was the initial size of the sample recruited for this study.

Due to a dropout rate of 20.6% (14/68 patients), the final sample consisted of 54 patients diagnosed with alcohol dependence syndrome: 45 males and 9 females, mean age 45.37 years (SD 10.12) with an average of 10 years of formal education (SD 4.62). Of these, 26 patients had been assigned to the mobile health cognitive stimulation condition: 19 males and 7 females, mean age 45.50 years (SD 10.18) with an average of 10 years of education (SD 4.39). Twenty-eight patients were assigned to the control condition consisting of treatment-as-usual: 26 males and 2 females, mean age 45.25 years (SD 10.26) with an average of 10 years of formal education (SD 4.95). No statistically significant differences at the conventional P<.05 level were observed between groups for the demographic characteristics of gender (Table 1), age, and education (Table 2).

The statistical analyses for the neuropsychological data at baseline were focused on comparisons between groups for the overall score of the MMSE and the FAB, the correct responses, categories and perseverative errors in the WCST, and the mean errors and execution times in the CTT. The results revealed no statistically significant differences between the experimental and control group for neuropsychological outcomes at baseline (all Ps>.41) (Table 3).

Global cognitive function was estimated at baseline and follow-up assessment with the MMSE. The ANOVA on the overall score of the MMSE revealed a main effect of assessment (F _1,52=20.68, eta-square=0.41, P<.001), indicating an improvement of cognitive ability from baseline (mean 26.42, SE 0.38) to follow-up (mean 27.70, SE 0.26). No statistically significant main effect of group or interaction effect between factors (Fs<1) was found (Figure 3).

Frontal lobe functions were also evaluated under the same factorial design. The data from the FAB showed a significant interaction effect between factors (F _1,52=8.00, eta-square=0.16, P=.01). Simple effects analysis was performed to test differences between assessments on each group (experimental and control). These data indicated a significant increase in the FAB score from baseline (mean 13.89, SE 0.58) to follow-up (mean 15.50, SE 0.46) only in the experimental group, but not in the control group (mean 14.14, SE 0.56) and (mean 14.14, SE 0.45), respectively for baseline and follow-up (Figure 4).

Cognitive flexibility was measured on three different dimensions of the WCST: the correct responses, the number of categories, and the number of perseverative errors during the task. The ANOVA showed a main effect of assessment on the number of correct responses (F _1,52=15.10, eta-square=0.29, P<.001) and on the number of completed categories (F _1,52=4.94, eta-square=0.09, P=.03), indicating an improvement in performance of the WCST from baseline to follow-up assessment, but there was no interaction effect with the treatment factor (F<1). No significant effects (F<1) were reported for perseverative errors (Figure 5).

Speed of processing and attentional functioning were assessed respectively with execution time and error rate of the two forms of the CTT. We found a main effect of assessment for both error rate and execution time in the CTT1 and in the CTT2. A significant decrease in error rate (F _1,52=5.20, eta-square=0.10, P=.03) and execution time (F _1,52=4.26, eta-square=0.08, P=.04) was found in the CTT1 (Figure 6). The same pattern, but with a stronger effect, was found for the results of the CTT2, with a decrease in error rate (F _1,52=13.23, eta-square=0.25; P<.001) as well as execution time (F _1,52=14.41, eta-square=0.28, P<.001) (Figure 7). In both the CTT1 and the CTT2, there was no significant interaction (F<1) with the treatment factor.

The current experiment was designed to study the effectiveness of a cognitive stimulation program in alcohol abusers’ executive functioning with mobile SG applications. According to previous studies, alcohol dependence may impair information processing in the prefrontal areas, such as the dorsolateral prefrontal cortex and the anterior cingulate cortex, which are thought to be related to executive functioning in cognitive domains ranging from attention and working memory to higher-order functions of reasoning and decision making. In this study, our aim was to evaluate the effect of cognitive stimulation with mobile technology and serious games on the general cognitive ability of patients diagnosed with ADS.

To do this, we carried out an RCT on patients undergoing an alcohol rehabilitation treatment at a specialized institution for treatment of alcohol dependence. The overall MMSE scores at baseline suggest deficits in general cognitive ability of ADS patients, which concurs with previous studies highlighting the relationship between the alcohol abuse and cognitive impairments [4]. There were no differences at baseline between control and treatment groups, which indicates a successful random distribution of patients.

Our results indicate a general improvement of cognitive abilities in both patient groups, which reflects an effect of the withdrawal of the direct impact of alcohol on the brain. In support of the specific effectiveness of the cognitive stimulation, this effect was qualified with an interaction with the treatment factor in the FAB test, indicating a more pronounced increase of frontal cognitive abilities in patients subjected to mHealth SG approach, with improvements in frontal lobe function and executive functioning. However, there were no significant effects of the treatment on the MMSE and WCST, which measure more specific, not general, domains of frontal lobe functioning. These results are in agreement with those of other cognitive rehabilitation trials with addict populations, suggesting that the positive effects of neuropsychological rehabilitation in addicted populations are restricted to cognitive abilities related to frontal lobes [21].

The explanation for this is that both traditional exercises and SG applications focused mainly on cognitive abilities related to frontal lobe functioning, such as attention, working memory, decision making, and planning. The systematic and repeated stimulation of these functions may have had a more pronounced effect on those domains, but not on the others. Our results suggest that the neuropsychological benefits of our mHealth SG cognitive stimulation program in alcoholics may be limited to frontal lobe general functioning and that may arise from the response demands of the game tasks. That is, the processes required to meet the games’ tasks are most closely related to the processes that are captured by the FAB.

In addition, the overall feedback from the participants was positive. Qualitative comments were mainly related to the technological and innovative features of this approach and intrinsic aspects such as a positive motivation to pursue a goal in the tasks.

This study was not designed to test the relative effectiveness of mobile intervention with online stimulation and serious games against traditional pencil-and-paper cognitive stimulation exercises. It was simply to test if this mHealth solution could add effectiveness in cognitive rehabilitation to treatment-as-usual procedures. Further studies testing the relative effectiveness of SG vs traditional approaches are needed. Due to the characteristics of SG and of the clinical context, it would not have been possible to conduct a double blind study. However, the therapists conducting the assessment were not aware of which group (experimental vs control) the patients were enrolled in.

The increase in general cognitive function for both experimental and controls supports the beneficial role of existing alcohol treatment protocols, which are based on medication to help minimize withdrawal symptoms along with behavioral therapy mainly to help patients manage their stress levels during abstinence, and indicates that the neuropsychological effects of alcohol abuse on brain structure are reversible through rehabilitation. The effect of the trial is encouraging, suggesting an improvement specifically in frontal cognitive general functioning in alcoholics following an mHealth approach with SG in line with previous studies using cognitive stimulation with addicts in rehab.