A total of 182 participants were recruited from January to March 2013 across 4 outpatient mental health care clinics and institutions in the Netherlands and Belgium. Eligible parents and children were informed by their clinician about this study. In other cases, the patient organization provided information about the study to their members; these parents directly applied for the study. Once the clinician identified eligible parents and children, they received detailed written and verbal information about the study from the researcher. After signing informed consent, they were invited for a screening visit (performed by trained research assistants with MA in psychology) to verify inclusion and exclusion criteria. This resulted in a sample of 170 participants. Inclusion criteria were (1) a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) diagnosis of ADHD (confirmed by the Kiddie Schedule for Affective Disorders and Schizophrenia-Lifetime version [K-SADS] [44,45]), (2) aged between 8 and 12 years, (3) stable on pharmacological and/or psychological treatment for ADHD 8 weeks before baseline (determined by health care professionals on the basis of medication data and behavioral observation), (4) no initiation or change of pharmacological and/or psychological treatment for ADHD during the study period, (5) availability of a computer workstation at home with Internet and sound facilities, and (6) sufficient understanding of the Dutch language by the child and by at least one of the parents. ADHD severity was measured by the parent version of the Disruptive Behavior Disorder Rating Scale (DBDRS) [46,47] and children with common comorbid disorders of ADHD (eg, oppositional defiant disorder as measured by the DBDRS) could participate in the study. Exclusion criteria were (1) an estimated total Intelligent Quotient (IQ) lower than 80 (determined by vocabulary and block design subtests of the Wechsler Intelligence Scale for Children III [WISC-III] [36,48,49], (2) substance abuse problems (eg, drugs, alcohol), (3) conduct disorder, (4) autism spectrum disorder (both previously diagnosed by health care professionals), (5) comorbid acute psychiatric disorder (eg, depression, mania; confirmed by the K-SADS [44,45]), and (6) participation in a previous pilot study with a prototype of Plan-It Commander. Children with a severe physical disability (eg, blindness, deafness) or learning disability (eg, dyslexia) were also excluded on the basis of the child’s medical file and a standardized interview administered by phone to parents. Written informed consent was obtained from parents and children aged 12 years. All study procedures were approved in advance by the Erasmus (Dutch; MEC-2012-539) and Leuven (Belgian) Medical Ethical Committees.

This study used a 20-week multisite randomized controlled crossover open-label trial design (see Figure 1). The intervention was an online serious game called Plan-It Commander. Participants were randomized to a serious game intervention group (group 1) or a treatment-as-usual crossover group (group 2). Participants randomized to group 1 received a serious game intervention in addition to treatment as usual for the first 10 weeks and then received treatment as usual only for the next 10 weeks. Participants randomized to group 2 received treatment as usual for the first 10 weeks and crossed over to the serious game intervention in addition to treatment as usual for the subsequent 10 weeks. All participants in the study received treatment as usual and most participants (91.8%, 156/170) were on medication. Participants were instructed to play the serious game for a maximum of 65 minutes approximately 3 times per week. The game was programmed so that participants could not play more than 65 minutes in one 24-hour period to prevent excessive use of the game. The CONSORT EHEALTH checklist is presented as Multimedia Appendix 1

Randomization was carried out on a 1:1 ratio and based on a prespecified computer-generated randomization list. Allocation was stratified by study site and gender and arranged in permutated blocks. Group assignment was performed online using the next available number on the randomization list corresponding to the site and gender of the participant. It was not possible to blind participants to their treatment allocation. After screening and baseline assessment, parents received an email with the notification to which group (group 1 vs group 2) their child was allocated. Although all efforts were made to keep the investigator blind during baseline assessments, full blinding of researchers and teachers at the other assessment points could not be guaranteed because participants could spontaneously talk about the game during the study.

The serious game is an online adventure game (called Plan-It Commander) developed by health care professionals, researchers, and game experts in collaboration with parents and children with ADHD. In collaboration with a focus group of parents, the multidisciplinary game development team agreed on the game’s learning goals and play frequency/time. After each prototype build, usability tests were iteratively performed to examine whether children liked the game and understood how to use it and navigate within the game. User data were evaluated and incorporated in the design process for the final game format, which was examined in this study. Plan-It Commander was designed to improve domains of daily life functioning with a primary focus on time management, planning/organizing, and cooperation skills in children with ADHD. Unifying their knowledge and expertise resulted in a unique online learning environment in which principles of behavior therapy and game-based learning were combined [43]. Players had their own password and ID to log on to the Internet-based serious game from their home where they could access 2 game components: (1) a mission-guided game environment with minigames related to the learning goals of time management, planning/organizing, and cooperation skills and (2) a closed social community. The game was linked to a database in which data about play frequency and duration were registered from each participant.

Plan-It Commander is a mission-guided game divided into 10 different missions and several side missions (Figure 2). Missions guide the player’s behavior throughout the game as he or she follows the storyline and is asked to solve problems requiring specific skills. Central parts are the 3 minigames addressing time management, planning/organizing, and prosocial behavior that are embedded in the structure of the game. The first minigame is focused on teaching the player time estimation and time management skills. The second minigame is focused on enhancing planning skills; the player is taught to plan ahead and break down the total assignment into pieces. The third minigame focuses on enhancing prosocial behavior, teaching the player to help their team members and to cooperate with each other. In addition to the mission-guided game, players could access a closed social community (called “Space Club”) to stimulate prosocial behavior (eg, helping other players, giving compliments) (Figure 3). Players can ask for help or help other players through predefined messages and reward them with a thank you message. The player’s profile is presented within the community and shows an overview of his or her progression throughout the game. When a player completes certain “challenges” in the mission-guided game, an achievement is unlocked in the community. Every player has an overview of awarded achievements in the form of badges or medals in their profile within the community. By making progress in the game and reaching certain milestones, the player unlocks rewards in the community. Rewards can vary in form, such as papercraft models, desktop wallpapers, and music from the game. Players can see each other’s profiles and this generates competition between players. Details of the development and content of Plan-It Commander are described elsewhere [43].

Multi-informant (parent, teacher, and self-report) measures were administered at baseline (T0), at 10 weeks (T1), and at 10-week follow-up (T2). Parent and teacher reports were administered through online questionnaires. Questionnaires were administered to the children during face-to-face appointments at each assessment time point. At baseline, demographic information and children’s game experience were collected through parent reports. The parent reported on the game experience of their child as starter, amateur, experienced, or expert. For the primary outcomes, parents filled in the following questionnaires during the 3 assessment time points: (1) a time management questionnaire (Multimedia Appendix 2), (2) the subscale Plan/Organize of the Behavior Rating Inventory of Executive Function (BRIEF; parent version) [50,51], and (3) the subscale Cooperation of the Social Skills Rating System (SSRS; parent version) [52,53]. The time management questionnaire gave a more detailed insight into children’s behavior strategies used to improve their time management skills compared to other existing questionnaires (primarily focusing on time perception and/or coordination) and demonstrated good reliability (α=.85) in a pilot study (KCMB, unpublished data, 2016). Secondary outcomes consisted of parent, teacher, and self-reports. Parents filled in the subscale Working Memory of the BRIEF (parent version); the subscales Responsibility, Assertiveness, Self-Control, and Total of the SSRS (parent version); and the It’s About Time Questionnaire (IATQ; parent version) [54]. In addition, teachers were asked to fill in the time management questionnaire, the subscales Plan/Organize and Working Memory of the BRIEF (teacher version), and the SSRS (teacher version) to provide an indication of how the participant functioned at school. Further, we asked participants to fill in a self-efficacy questionnaire (Multimedia Appendix 3) [55]. After receiving the serious game, both parents and participants filled in a satisfaction questionnaire indicating general satisfaction with the serious game on a 10-point Likert scale. Table 1 includes a description of each measure.

The sample size was determined in advance by power calculations on the basis of previous pilot study descriptive results (mean, SD) on primary outcome measures, which indicated that 78 participants per group would give 87% power to detect differences of a medium effect size (at least 0.5) between groups (α=.05; 2-sided). In the current study, differences in baseline characteristics were tested with an independent samples t test or a chi-square test. For primary and secondary outcome measures, changes from baseline to 10 weeks (reflected by its difference scores) were compared between group 1 and group 2 with ANCOVAs, with baseline score as a covariate and gender and site as factors. To assess improvement during treatment within both groups, paired samples t tests were performed on primary and secondary outcome measures before and after playing the serious game. To assess whether effects were maintained after playing the game for 10 weeks within group 1, within-group comparisons of changes at 10 weeks versus 10-week follow-up were performed. Intention-to-treat analyses were used and included all randomized participants. Linear trend at point was used as an imputation method. All statistical analyses were performed using SPSS version 19.0 statistical software (IBM Corp, Armonk, NY, USA) and were 2-sided with a level of significance of α=.05. The significance level for primary outcome measures was adjusted on the basis of the Hochberg procedure [56]. Effect sizes were reported for all analyses using Cohen’s d [57].

A total of 170 participants met the inclusion criteria and participated in the study. Mean scores for primary and secondary outcome measures and characteristics of groups 1 and 2 did not differ significantly at baseline (see Table 2). Most participants (91.8%, 156/170) received medication as their treatment as usual. Medication use did not differ between 4 outpatient mental health care clinics and institutions (χ² ₃=3.7, P=.29).

At 10 weeks (T1), 152 of 170 participants (89.4%) completed the study and 139 of 170 (81.8%) completed at the 10-week follow-up (T2). At 10 weeks (T1), the dropout rate was higher in group 1 compared to group 2 (χ² ₁=8.0, P=.01). The dropout rate did not differ between the 2 study groups at 10-week follow-up (χ² ₁=2.5, P=.12). Participants who dropped out during the study period did not differ according to age (t ₁₆₈ =–1.34, P=.18), gender (χ² ₁=2.2, P=.13), ADHD subtype (χ² ₂=2.5, P=.29), or intelligence (t ₁₆₈ =–1.66, P=.10) compared to participants who did not drop out (for flow diagram see Figure 4). However, children who dropped out during the study (mean 45.06, SD 15.79) had higher ADHD severity scores compared to children who completed the study (mean 39.07, SD 14.16; t ₁₆₈=2.09, P=.04).

Participants played for a mean 19.04 (SD 9.61) days in the mission-guided game and a mean 11.20 (SD 8.55) days in the closed social community. Additionally, participants played the mission-guided game for a total duration of a mean12.56 (SD 6.57) hours and engaged with the closed social community for a mean 54.27 (SD 70.00) minutes. A difference was seen between group 1 (mean 13.53, SD 6.25) and group 2 (mean 11.53, SD 7.25) with regard to the amount of time playing the mission-guided game (t ₁₅₅=1.81, P=.07) but it did not meet statistical significance. There was a significant difference between group 1 (mean 12.61, SD 8.60) and group 2 (mean 9.70, SD 8.28) with regard to the number of days they engaged with the closed social community (t ₁₅₇=2.17, P=.03). With regard to the amount of time playing in the closed social community, there was a difference between group 1 (mean 1.04, SD 1.16) and group 2 (mean 0.44, SD 1.02; t ₁₅₆=1.82, P=.07), although this was not statistically significant. There were no differences between the 2 groups with regard to the number of days playing the mission-guided game. Both parents (mean 6.96, SD 1.40) and participants (mean 7.33, SD 1.87) reported moderate to high satisfaction with receiving the serious game intervention.

To test the hypothesis that participants playing the serious game would improve on primary and secondary outcome measures, differences between group 1 and group 2 from baseline to 10 weeks (T1; posttest) were evaluated with ANCOVAs (see Table 3). On the primary outcome measures, group 1 showed significantly greater improvements in parent-rated time management skills compared to participants in group 2. Group 1 also showed more improvement in parent-reported planning/organizing skills compared to group 2, although this did not meet statistical significance (P=.07). There were no differences concerning participants’ cooperation skills.

Regarding the secondary outcome measures, group 1 also improved significantly more than the group 2 on measures of parent-reported working memory and responsibility skills. Participants in group 1 showed greater improvements in participants’ time perception compared to group 2, although this did not meet statistical significance (P=.09). Teachers reported greater improvements in group 1 than group 2 on the measure of time management and working memory, although this latter effect did not meet statistical significance (P=.05). Finally, the same accounted for participants’ self-efficacy in which participants in group 1 showed greater improvements as compared to group 2 (P=.09), but it did not meet statistical significance. No differences were found on parent-rated total social skills (with subscales assertiveness and self-control) and teacher-rated total social skills and planning/organizing skills.

Within-group differences for group 2 were evaluated (see Table 4). While receiving treatment as usual for the first 10 weeks, participants improved significantly on parent-reported time management and teacher-reported social skills. After crossing over to the serious game intervention in addition to treatment as usual for the subsequent 10 weeks, significant improvements in outcomes of parent-reported time management, time perception, planning/organizing, working memory, and social skills (primarily cooperation and assertiveness) were found. Furthermore, significant improvements were demonstrated for all teacher-reported outcomes. Self-reported self-efficacy also significantly improved after receiving the intervention (see Table 4).

Within-group differences for group 1 were then evaluated (see Table 5). While playing the serious game intervention in addition to treatment as usual for the first 10 weeks, significant improvements in outcomes of parent-reported time management, time perception, planning/organizing, and social (primarily cooperation and responsibility) skills were found. Furthermore, significant improvements were demonstrated for participants’ time management, working memory, and social skills as reported by their teachers. Within-group effects showed significant improvement from 10 weeks to 10-week follow-up for parent-reported time management, working memory, time perception, and social skills (primarily cooperation, responsibility, and self-control). Furthermore, significant improvements were demonstrated for teacher-reported time management and working memory skills (see Table 5). This implies that most effects maintained or even further improved at 10-week follow-up.

While playing the serious game, adverse events were registered by the researcher and checked by a health care professional. Overall, there were 10 adverse events that could be related to the intervention that were reported by parents, teachers, or participants themselves. All adverse events were of mild (n=5) or moderate (n=5) severity, but this was no reason to discontinue study participation. Examples of adverse events were pain in the fingers, irritability, and headache. An adverse event was a reason to discontinue the study for only one known participant. This participant did not want to play the game anymore because he could not concentrate during his school activities. Sounds reminded him of the game and this consequently distracted and frustrated him. No serious adverse events were reported.

ADHD is a chronic health problem and previous studies have emphasized the need for efforts to treat impairments outside the therapy context and provide patients with greater autonomy [22,23,63]. The Internet-based serious game intervention in this study fulfills this need by addressing impairments associated with ADHD among school-aged children in the home and school context. Results demonstrated that parents as well as children were satisfied with their treatment. The current intervention was positioned as an adjunct to treatment as usual. No therapist or parent explicitly intervened during the game intervention. Furthermore, no additional rewards were given and no prompts to play the game regularly were explicitly provided outside the game. Given that young patients with ADHD have engagement and motivation issues in general, easy accessible interventions such as serious games can stimulate them to comanage their health care processes as part of the Chronic Care Model of Child Health and the World Health Organization Mental Health Action Plan 2013-2020 [22,23,64].

The current intervention is unique in its contribution to the adjunctive ADHD treatment repertoire because it differs from existing computerized neurocognitive training formats. Instead of requiring the repetition of executive function tasks normally presented in neurocognitive training format for children with ADHD, Plan-It Commander offers behavioral strategies (e.g., reinforcement, immediate performance feedback from a mentor, goal setting through missions, modeling, social support, and comparison) that increase functional outcomes within a relatively short period of time. Even more important is the fact that participants labeled as “clinically stable” by their clinicians still showed significant improvements in daily functioning. It is encouraging that significant results were obtained over and above medication effects. Future research could examine the effects of this serious game in a nonmedicated sample to disentangle its effects. Notably, participants with higher severity scores on ADHD symptoms were more likely to drop out from the study, which implies that we can only generalize our results to children with less severe ADHD symptoms, but this remains speculative because symptoms were within the normal range. Furthermore, future research should consider family factors (eg, social support network, socioeconomic status, parental ADHD) as well in contributing to study dropout.

The results of this study must be considered in the light of several limitations. Group 2 followed treatment as usual and did not use a nontherapeutic “placebo game.” Therefore, this study could be controlled for changes in time and effects of repeated measurements, but not for placebo effects. Further, parents were not rater-blinded and rater-blindness of teachers could not fully be guaranteed because children were free to report game experiences. Questionnaires to assess time management and self-efficacy were designed on theoretical basis and guidelines by Bandura [55]. Both instruments show good reliability. The time management questionnaire was developed because of a lack of instruments for this age group. This questionnaire was used previously in a randomized controlled pilot study (KCMB, unpublished data, 2016). Future research should evaluate the psychometric characteristics of these questionnaires in more detail.

The current randomized controlled study demonstrated that Plan-It Commander is an effective adjunctive Internet-based behavioral intervention for children with ADHD. It is a unique contribution to the literature on serious games because it showed that a serious game for ADHD, as an adjunct to treatment as usual, improves functional outcomes of time management as well as working memory and social responsibility. It fits the current interest in nonmedical treatment options for ADHD and stimulates young children to manage their impairments by offering an easy, accessible home treatment intervention. The findings contribute to scientific knowledge about the impact of serious game interventions on behavioral outcomes, Internet-based interventions for mental health that are consistent with the Chronic Care Model of Health, and innovative approaches to treating people coping with chronic mental health conditions.