Published on 18.09.17 in Vol 19, No 9 (2017): September
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/8148, first published Jun 04, 2017.
Video Game Intervention for Sexual Risk Reduction in Minority Adolescents: Randomized Controlled Trial
Background: Human immunodeficiency virus (HIV) disproportionately impacts minority youth. Interventions to decrease HIV sexual risk are needed.
Objective: We hypothesized that an engaging theory-based digital health intervention in the form of an interactive video game would improve sexual health outcomes in adolescents.
Methods: Participants aged 11 to 14 years from 12 community afterschool, school, and summer programs were randomized 1:1 to play up to 16 hours of an experimental video game or control video games over 6 weeks. Assessments were conducted at 6 weeks and at 3, 6, and 12 months. Primary outcome was delay of initiation of vaginal/anal intercourse. Secondary outcomes included sexual health attitudes, knowledge, and intentions. We examined outcomes by gender and age.
Results: A total of 333 participants were randomized to play the intervention (n=166) or control games (n=167): 295 (88.6%) were racial/ethnic minorities, 177 (53.2%) were boys, and the mean age was 12.9 (1.1) years. At 12 months, for the 258 (84.6%) participants with available data, 94.6% (122/129) in the intervention group versus 95.4% (123/129) in the control group delayed initiation of intercourse (relative risk=0.99, 95% CI 0.94-1.05, P=.77). Over 12 months, the intervention group demonstrated improved sexual health attitudes overall compared to the control group (least squares means [LS means] difference 0.37, 95% CI 0.01-0.72, P=.04). This improvement was observed in boys (LS means difference 0.67, P=.008), but not girls (LS means difference 0.06, P=.81), and in younger (LS means difference 0.71, P=.005), but not older participants (LS means difference 0.03, P=.92). The intervention group also demonstrated increased sexual health knowledge overall (LS means difference 1.13, 95% CI 0.64-1.61, P<.001), in girls (LS means difference 1.16, P=.001), boys (LS means difference 1.10, P=.001), younger (LS means difference 1.18, P=.001), and older (LS means difference=1.08, P=.002) participants. There were no differences in intentions to delay the initiation of intercourse between the two groups (LS means difference 0.10, P=.56).
Conclusions: An interactive video game intervention improves sexual health attitudes and knowledge in minority adolescents for at least 12 months.
Trial Registration: Clinicaltrials.gov NCT01666496; https://clinicaltrials.gov/ct2/show/NCT01666496 (Archived by WebCite at http://www.webcitation.org/6syumc9C0).
J Med Internet Res 2017;19(9):e314
- randomized controlled trial;
- human immunodeficiency virus;
- risk reduction;
- primary prevention
Human immunodeficiency virus (HIV) disease and sexually transmitted infections (STIs) significantly impact young people, with racial/ethnic minority youth disproportionately affected. Nearly 10,000 youth were diagnosed with HIV in 2014 in the United States, accounting for 22% of new infections . There is a considerable range in both the quantity and quality of education around sexual health, HIV, and acquired immune deficiency syndrome (AIDS) adolescents receive. Only 22 states and the District of Columbia require that public schools teach sex education. Nineteen states require that if sex education is taught, it must be factually accurate [ ]. Another 12 states require HIV/AIDS education. Although there are effective HIV and STI prevention programs, challenges in their implementation and fidelity exist. Service providers may lack access to programs [ ], selectively implement program components [ ], or never implement programs at all [ ]. Barriers to implementation include access to adequately trained providers, resource constraints, fidelity, and challenges of adapting an intervention from one population to another [ - ]. To address these issues, digital health interventions have demonstrated efficacy at influencing sexual health [ , ], featuring adaptable content for broader reach, with greater fidelity, at a potentially lower cost [ ].
Serious games, defined as video games for a primary purpose beyond pure entertainment , offer unique benefits in targeting health promotion and risk prevention [ ]. They have efficacy [ ] in areas ranging from depression to asthma to cancer [ , - ]. They also have considerable reach, with 97% of adolescents, including all racial/ethnic groups, playing video games with 50% playing for at least 1 hour or more per day [ ]. In a study assessing the impact of digital technologies on schools, more than 333,000 students in grades 6 to 12 reported wanting to use digital games for learning in school and reported games increased their engagement [ ]. Active participation through simulated role-playing [ , ] in video games allows individuals to acquire knowledge and practice behavioral skills in a highly accessible, portable, and engaging way [ , ].
Aim of This Study
Our objective was to test the efficacy of a digital health intervention in the form of a theory-driven interactive video game intervention compared to a set of control games on sexual risk behaviors, knowledge, attitudes, and intentions in a population of racial/ethnic minority adolescents. We hypothesized that a highly engaging theory-based video game intervention would have a greater impact on our outcomes of interest than a set of control games.
We conducted a randomized controlled trial in 12 urban community-based settings consisting of seven school-based afterschool programs, four independent afterschool programs, and one summer camp. A description of the development of the PlayForward: Elm City Stories (PlayForward) intervention and the trial design have been published [, ]. The Yale School of Medicine Human Investigation Committee approved the research.
Participant eligibility included ages 11 to 14 years, speaks English, able to provide assent and parental/legal guardian consent, and willing to play video games for up to 75 minutes twice weekly for 6 weeks. Although the primary outcome of this study was delay of initiation of sexual intercourse, we enrolled participants who had already initiated intercourse because we did not want potential participants to provide incorrect information regarding their sexual activities to gain or avoid access to the study. These participants (n=6) were not included in these analyses. Participants were provided with an age-appropriate study description and parents/legal guardians were informed that the study focused on promoting healthy behaviors and reducing risk in adolescents. If an adolescent or parent/guardian did not wish to participate, they were not included in the study. All data were collected from 2013 to 2015 and analyzed in 2016.
Enrolled participants were randomized in an unmasked fashion to the PlayForward group or to a set of 12 attention-time control video games with all participants playing the games on iPad tablets. A single randomization scheme was generated and written in TrialDB, a customizable Web-based clinical trials database system . After obtaining written youth assent and parental/legal guardian informed consent, eligible participants were assigned to play PlayForward or attention/control games in a 1:1 ratio using a computerized single randomization scheme. Randomization was stratified by gender and age group (11-12 years and 13-14 years), both predictors of the outcomes of interest [ , ]. Notably, gender differences have been identified as being important not only in terms of the trajectory of sexual behaviors in boys and girls during adolescence but also in terms of their response to interventions targeting sexual risk [ ]. Randomization was under the control of an investigator who was not involved with eligibility assessment. Study personnel accessed the computerized randomization system, retrieved the assignment, and notified participants of their group assignment.
Participants played PlayForward or a set of control games on the iPad. PlayForward is a two-dimensional, role-playing adventure video game [, - ] developed using a theoretical foundation [ , - ] about how individual choices made within the social environment of life impact both short-term and long-term goals ( ). These theories primarily focus on the individual’s decision making, yet account for the fact that the decisions are made within a certain social context. Delivered as an interactive video game, the intervention allows for the player to see how their individual decisions are influenced by their social surroundings. The game consists of approximately 16 hours of gameplay. The player’s goal is to acquire and practice skills to reduce risk behaviors and gain knowledge and healthier attitudes and intentions with the ultimate goal of HIV prevention. The game involves an interactive world where the player creates an Aspirational Avatar [ ] and “travels” through life, facing challenges and making decisions in the context of a series of narratives depicting common social situations whose outcomes bring different risks and benefits ( ). The game focuses on sexual health and risk and a range of risky behaviors including substance use, academic dishonesty, and unsafe driving. Five skill-based interactive “mini-games” ( and ) are combined with 12 story-based “challenges” comprising the overarching narratives.
The attention/time control games consisted of 12 video games such as Angry Birds, Dragonbox, and Subway Surfer. They were devoid of content relevant to our study goals and mirrored the number of sessions and length of gameplay in the experimental group . We considered a conventional prevention education control condition; however, there was no “gold standard” for teaching sexual education or HIV prevention in US schools [ ], therefore using HIV prevention/education materials as the control was not consistent with current “treatment as usual.” We opted for an attention/time control because our primary goal was to determine the efficacy of the video game intervention.
Participants played their assigned game(s) for two sessions per week, approximately 1 hour per session, for 6 weeks on-site at their program . Similarly, all assessments were conducted at the participant’s program and included data collected at baseline, 6 weeks (immediately after completion of gameplay), and at 3, 6, and 12 months.
The PlayForward game software records in-game data assessing intervention exposure and fidelity . In-game data documents player’s actions, time spent on each action, and overall performance providing a measure of exposure to each intervention component. Periods of inactivity (eg, player is away from the device) can be identified from activity logging data timestamps. Research personnel observed and documented both the PlayForward and the control group participants’ duration of gameplay.
The primary outcome was delay of initiation of sexual intercourse (defined as initiation of vaginal or anal intercourse) at 12 months post-baseline [- ]. On June 1, 2015, before any data analyses, the investigators and the Data and Safety Monitoring Board (DSMB) clarified that the primary outcome would be assessed at 12 months postrandomization. Secondary outcomes were sexual attitudes, sexual health knowledge, and sexual intentions. Items assessing the secondary outcomes were compiled from standard instruments and assessed for their internal consistency using Cronbach alpha: delay of the initiation of intercourse [ ], sexual attitudes [ ] (3 items, maximum score=12, Cronbach alpha=.90), sexual health knowledge [ , ] (15 items, maximum score=15, α=.68), and intentions to delay intercourse [ ] (four items, maximum score=16, Cronbach alpha=.84). Although the sexual health knowledge measure (which was piloted before the trial) had a slightly lower level of reliability than is considered acceptable (Cronbach alpha=.7), the items in this measure reflected content that was specifically built into the game, representing very good content validity, which is an important component of a knowledge test in terms of demonstrating that the items within the test are closely related to one another. In addition, although the Cronbach alpha was slightly lower than the .7 value, the result was not borderline, but considerable. Additional assessment data were collected but were not the focus of this paper [ ]. Outcomes were assessed at 6 weeks and at 3, 6, and 12 months after randomization. Data were collected face-to-face by research staff and entered into a Web-based database. As described elsewhere [ ], we collected data from participants individually; for assessments including sensitive data, participants filled out responses after questions were read to them. Measures were taken to ensure participants felt their responses were being kept confidential.
Power calculations were based on data from published studies [, ] and systematic reviews [ - ] evaluating youth HIV prevention interventions. These studies suggested a small to moderate effect size of interventions on delaying/preventing initiation of sexual intercourse. National data indicated that approximately 7% of youth reported vaginal or anal intercourse before age 13 years [ ] and the final sample size accounted for this figure. A sample size of 330 with 165 in each study condition, was estimated to detect a 15% difference in the proportion of participants achieving the primary outcome of delay of initiation of sexual intercourse (90% PlayForward vs 75% control groups), providing a power of 80% or greater to detect significant (two-tail alpha=.05) differences of this magnitude [ ]. This sample size also afforded adequate power (>90%) to detect small to moderate effects on secondary outcome measures. NQuery version 4.0 was used to estimate sample size. This sample size accounted for losses in primary outcome assessment and for enrollment of participants who at baseline had already initiated sexual intercourse.
The primary comparison evaluated the effect of PlayForward compared with the control video games on the delay of initiation of sexual intercourse measured at 12 months postrandomization and constructed as a binary outcome (delayed beyond 12 months/initiated before 12 months). The primary analysis was carried out as an intent-to-treat analysis and adjusted for gender and age (the randomization stratification variables). Sensitivity analyses were carried out with missing responses being assigned both as delay and not delay of initiation of intercourse.
Differences in scores in the secondary outcomes were compared between the two groups at the time points using longitudinal mixed-effects models. Changes in secondary outcome measures since baseline were assessed in repeated measures models (with unstructured covariance), with the assigned baseline values, study group, gender, age, study time point, and study group*time interactions used as covariates. Least squares (LS) means and standard error were plotted for each secondary outcome at each time point. Statistical analyses were done using SAS version 9.4 (SAS Institute Inc, Cary, NC, USA). All protocols were reviewed at intervals by a DSMB.
Description of Study Sample
A total of 333 participants were recruited and enrolled into the study between February 26, 2013 and May 16, 2014; 166 were assigned to the PlayForward intervention and 167 were assigned to the control condition (). A total of 166 participants were assigned to PlayForward (162/166, 97.6% initiated gameplay) and 167 to the control games (159/167, 95.2% initiated gameplay). Reasons for not initiating gameplay included participants’ inability to participate in the afterschool program due to transportation or medical issues. The PlayForward group played a median of 10.1 hours (interquartile range (IQR) 3.9) over 10 sessions for a median of 60.4 minutes/session (IQR 11.6). The control group played a median of 10.1 hours (IQR 3.9) over 10 sessions for a median of 61.4 minutes/session (IQR 10.9). Eighteen participants withdrew leaving 315 participants in active follow-up. A total of 269 (82.7%) completed the 6-week assessments, 267 (82.9%) completed the 3-month assessments, 253 (78.6%) completed the 6-month assessments, and 258 (81.6%) completed 12-month assessments. No demographic or clinical differences by intervention and control conditions were observed at baseline ( ).
The primary outcome was delay of initiation of sexual intercourse. Six participants (PlayForward: n=4; control: n=2) who had engaged in intercourse before baseline were removed from the primary analysis because they had already reached the primary outcome ().
Overall, the rates of initiating sexual intercourse were low in both groups. There were no differences in rates of delaying initiation of intercourse at each time point in the PlayForward versus the control groups. At 12 months, for the 258 of 304 (84.9%) participants from whom data were available, 122 of 129 (94.6%, 95% CI 89.1%-97.8%) in the PlayForward group versus 123 of 129 (95.4%, 95% CI 90.2%-98.3%) in the control group had delayed the initiation of intercourse (relative risk=0.99, 95% CI 0.94-1.05, P=.77). Sensitivity analysis suggested no difference in these proportions (missing=delay, P=.90; missing=no delay, P=.78).
Over the 12-month follow-up period, the PlayForward group demonstrated an improvement in attitudes about sexual health compared to the control group (LS means difference 0.37, 95% CI 0.01-0.72, P=.03) (). Treatment*age group and treatment*gender interactions were not significant (P=.06 and P=.09, respectively). The improvement in attitudes was not seen in girls (LS means difference 0.06, 95% CI –0.46 to 0.58, P=.81), but was seen in boys (LS means difference 0.67, 95% CI 0.18-1.16, P=.008). In addition, this improvement was only seen in the younger group of participants (age 11-12 years: LS means difference 0.71, 95% CI 0.21-1.20, P=.005) and not the older group of participants (age 13-14 years: LS means difference 0.03, 95% CI –0.49 to 0.54, P=.92).
Over the 12-month follow-up period, the PlayForward group demonstrated an increase in sexual health knowledge compared to the control group (LS means difference 1.13, 95% CI 0.64-1.61, P<.001) (). Both girls and boys in the PlayForward group demonstrated an increase in sexual health knowledge compared to the control group (girls: overall LS means difference 1.16, 95% CI 0.46-1.86, P=.001; boys: LS means difference 1.10, 95% CI 0.43-1.77, P=.001). Both younger (age 11-12 years) and older (age 13-14 years) participants in the PlayForward group demonstrated an increase in sexual health knowledge compared with the control group (younger: LS means difference 1.18, 95% CI 0.50-1.85, P=.001; older: LS means difference 1.08, 95% CI 0.39-1.78, P=.002).
Over the 12-month follow-up period, there were no differences in intentions to delay the initiation of intercourse between the two groups (LS means difference 0.10, 95% CI –0.23 to 0.43, P=.56) (). Similarly, there were no differences noted over the 12-month follow-up period by gender or by age group.
|Characteristics||Control (n=167)||PlayForward (n=166)||Total (N=333)|
|Gender, n (%)|
|Male||89 (53.3)||88 (53.0)||177 (53.2)|
|Female||78 (46.7)||78 (47.0)||156 (46.8)|
|Age (years), mean (SD)||12.9 (1.1)||12.9 (1.1)||12.9 (1.1)|
|Age group (years), n (%)|
|11||41 (24.6)||42 (25.3)||83 (24.9)|
|12||45 (26.9)||45 (27.1)||90 (27.0)|
|13||45 (26.9)||45 (27.1)||90 (27.0)|
|14||36 (21.6)||34 (20.5)||70 (21.0)|
|Race, n (%)|
|White||14 (8.6)||17 (10.4)||31 (9.5)|
|Black||65 (40.1)||70 (42.7)||135 (41.4)|
|Other||80 (49.4)||76 (46.3)||156 (47.9)|
|Unknown||3 (1.9)||1 (.6)||4 (1.2)|
|Ethnicity, n (%)|
|Hispanic||87 (55.1)||85 (55.9)||172 (55.5)|
|Non-Hispanic||71 (44.9)||67 (44.1)||138 (44.5)|
|Sexual health attitudes score, mean (SD)||10.1 (2.4)||10.1 (2.5)||10.1 (2.5)|
|Sexual health knowledge score, mean (SD)||6.5 (2.8)||6.1 (2.6)||6.3 (2.7)|
|Intentions to delay initiation of sex score, mean (SD)||14.9 (2.0)||14.6 (2.2)||14.8 (2.1)|
|Behavior||Control||PlayForward||Total, n (%)||Pc|
|n (%)||95% CIb||n (%)||95% CIb|
|Delay of initiation of sexual intercourse||165 (100)||162 (100)||327 (100)a|
|Delay of initiation of sexual intercourse||132 (97.8)||93.6-99.5||132 (98.5)||94.7-99.8||264 (98.1)|
|Initiation of sexual intercourse||3 (1.8)||2 (1.2)||5 (1.5)|
|Delay of initiation of sexual intercourse||133 (97.8)||93.7-99.54||127 (97.0)||92.4-99.2||260 (97.4)|
|Initiation of sexual intercourse||3 (1.8)||4 (2.5)||7 (2.1)|
|Delay of initiation of sexual intercourse||126 (97.7)||93.4-99.5||120 (96.8)||92.0-99.1||246 (97.2)|
|Initiation of sexual intercourse||3 (1.8)||4 (2.5)||7 (2.1)|
|Delay of initiation of sexual intercourse||123 (95.4)||90.2-98.3||122 (94.6)||89.1-97.8||245 (95.0)|
|Initiation of sexual intercourse||6 (3.6)||7 (4.3)||13 (4.0)|
aA total of six participants (control: n=2; PlayForward: n=4), who at baseline were identified (per primary outcome definition) as being engaged in sexual intercourse, were removed from the primary analysis because they had already reached the primary outcome.
bExact 95% confidence intervals are provided for the main outcome (delay of initiation of sexual intercourse).
cP values are based on a two-sided Fisher exact chi-square test.
In a cohort of community-based racial/ethnic minority adolescents, we found low rates of initiation of sexual intercourse over the 12-month follow-up in the PlayForward and control groups. Those who played PlayForward demonstrated greater improvement in attitudes around sexual health and greater increases in sexual health knowledge over 12 months than the control group.
This study is unique in that it is based in community settings targeting HIV risk in teens using engaging and novel methods that possess the potential for widespread dissemination and impact. To our knowledge, this is the first randomized controlled trial demonstrating the efficacy of a portable sexual health serious video game intervention impacting sexual attitudes and knowledge. The PlayForward game includes many components of programs that have been shown to be effective in reducing sexual risk in adolescents [, ], such as tailoring programs for the target population, using theory to guide program development, addressing more than just sexual risk, and targeting behaviors most amenable to change. PlayForward was developed with the input of the target audience allowing for its cultural appeal for this population. It incorporates social learning theory and self-efficacy [ , ] and principles from message framing [ ] grounded in prospect theory [ ]. It addresses myriad risk situations adolescents face, including but not limited to sex and a range of sexual risk behaviors and critical antecedents to those behaviors including attitudes, knowledge, and intentions, with the goal of impacting youth on a number of different levels. It was designed through an iterative approach that focused on our target audience and employed qualitative research methods as well as mixed methods to create and refine the final intervention [ ]. Although the theories that serve as the foundation for the game focus on individual reflective processes and the assessments measured these specific processes, the intervention does not disregard automatic processes. Throughout the game, individuals practice their decision-making skills within social contexts. In the narrative sections of the game, the players are challenged to explore the risk around them and learn how that risk influences their individual processes. Where the theory does not specifically target automatic processes, the gameplay does address these processes. Although the game is comprised of up to 16 hours of unique gameplay content, the goal was to expose players to 8 to 12 hours of the intervention given that this amount of time is consistent with existing HIV prevention interventions for adolescents. Athough the median amount of gameplay was 10 hours, many of the players completed the game. An earlier analysis examined the important question as to whether there is a relationship between exposure to different intervention components and study outcomes [ ]. This study revealed that it was not the duration of gameplay that seemed to be important in impacting outcomes, but the quality (ie, how well they played) of the participants’ performance in specific intervention components of the game.
Our findings are consistent with and complement findings from studies demonstrating the effectiveness of school-based interventions targeting sexual risk reduction in adolescents of similar ages and racial/ethnic backgrounds [, , ]. Notably, two of the studies had similar rates of sexual initiation at baseline (4-9%) [ , ] as this study, whereas the third study had considerably higher rates of initiation (23%), indicating it may have been a higher risk population [ ]. Similar to this study, one study demonstrated increased positive sexual health attitudes in boys only and increased HIV-related knowledge in boys and girls [ ]. This study’s intervention delayed sexual initiation and had an impact on other psychosocial determinants in boys, but had a limited effect for girls. The reason for our study’s finding of an impact on boys and not girls for sexual health attitudes is not entirely clear. One contributing factor could be that boys, compared to girls, reported feeling more connected to their character in the game (60% vs 39%, P=.01) and this connection could conceivably result in a greater impact on boys’ attitudes (unpublished data). Two school-based sexual risk interventions [ , ] demonstrated a delay of initiation of sex in middle-school students, although one did not show an effect in boys or in African-American students [ ]. All these studies [ , , ] required either trained facilitators and/or group discussions as part of the intervention and one study included an additional computer component [ ], as compared to this study in which the video game was a stand-alone intervention. In addition, they all followed participants for 24 months, with participants being older at follow-up, allowing for a potentially greater number of events of sexual initiation.
Although there have been long-standing efforts for comprehensive sex education, many challenges for school-based programs remain. Barriers to implementation include competing priorities, lack of parental and administration support, and lack of training [- ]. In one study, school staff reported confidence in discussing sexuality, but reported varying levels of support for comprehensive sex education from parents (42%), community leaders (53%), and school officials (50%) [ ]. In contrast, our qualitative study of key stakeholders implementing PlayForward in real-world settings demonstrated support for the game intervention’s potential role in sexual health education in schools [ ]. PlayForward offers potential implementation advantages over other school-based interventions because it does not involve intensive training requiring significant human and financial resources, and it provides a level of fidelity in the delivery of content. Importantly, the PlayForward intervention incorporates components of the operational guidance on comprehensive sexuality education promoted by international organizations [ ]. These include the provision of scientifically accurate information, a safe learning environment, participatory teaching approaches, strengthening adolescents’ skills in communication, decision making, and critical thinking, youth advocacy, and civic engagement in program design and cultural appropriateness, tailored for distinct subpopulations. Technology-based platforms such as the PlayForward video game intervention offer unique advantages because their content and graphics can be rapidly and inexpensively modified and updated, allowing for adaptation of the intervention for different populations and outcomes [ ]. Therefore, the PlayForward intervention responds to the call for rigorously evaluated interventions using technology for HIV prevention in adolescents [ ]. As the use of digital health interventions is expanding and holds great promise, it is important that they are subject to serious evaluation. Our use of a randomized controlled trial is justified by the ensured stability and engagement of the delivery vehicle of our intervention and that it was delivered with documented fidelity. Given the engagement of our target population in its development and the substantial exposure of our participants to the intervention over time, it has a considerable likelihood of having a clinically meaningful impact [ ].
Although we adhered to standards regarding evaluating behavioral interventions , some limitations of this study must be considered. A strength of this study is that the intervention was built on a solid foundation of well-established theories, but these theories apply most specifically to behavior change and may have less utility for a prevention intervention where the goal is for the behavior to remain the same. There is considerable precedent in the literature for grounding prevention interventions within a behavior change framework; however, in doing this we may not be accounting for factors that may be relevant to prevention but not behavior change. Notably, the low rate of initiation of sexual intercourse precluded us from determining PlayForward’s impact on the primary outcome at 12 months. These low rates are consistent with current national figures [ ] with 4% of high school students reporting sexual intercourse before age 13, increasing to 24% by ninth grade (approximately 15 years). Therefore, 12-month follow-up for our cohort may be too short to capture these events. Our outcomes were based on self-report, which can introduce biases in terms of disclosure of sensitive information. As described elsewhere [ ], we used data collection methods that optimized disclosure and the accuracy of self-reported data, and ensured privacy and confidentiality in this age group [ , ]. We chose to use face-to-face assessments as we were collecting large amounts of data (15 different assessment instruments) in a young teen cohort and wanted to ensure completion of the assessments. We were cognizant of issues around disclosure given that some of the assessments included sensitive data (sex- and substance use-related data). There are differing views in the literature examining paper versus computer-based assessments of sensitive data [ ], but research has found that paper versions elicited higher and more accurate rates of disclosure [ ], demonstrated more skipped items, and had no specific advantage to Web-based interviews [ ]. Although we studied a population of racial/ethnic minority adolescents who, as a group, are at increased risk for HIV/STIs, they were involved in structured programs and, therefore, the findings may not translate to other populations including those not engaged in afterschool or school-based programs [ ].
Despite these limitations and the low event rate of sexual initiation in our cohort, we demonstrated a compelling and persistent impact on attitudes and knowledge. There is evidence for a correlation between attitudes and behavior [- ], especially in the case of strong attitudes [ ]. One study examining survey data from over 1700 ninth graders analyzed the relationship between a number of variables including attitudes and specific outcomes, one being the behavior of abstaining from sexual intercourse [ ]. This study found that attitudes had the strongest relationship, not only with intentions (beta=.48) but, more importantly, with behavior (beta=–.34). This finding demonstrated a relationship with students who had high scores indicating more conservative attitudes about having sex being less likely to have sex (P=.01). Furthermore, one meta-analysis concluded that changing attitudes could produce a significant impact specifically on stemming the HIV epidemic [ ]. In addition to digital health interventions such as video games being effective, there remains a critical need for those interventions targeting sexual health to be accessible, adaptable, easily disseminated, and delivered with fidelity. Given the growing focus on the use of digital health (including sexual health [ ]) to increase access to and engagement with interventions, this study provides evidence for this approach. According to the 2016 National Education Technology Plan, “technology increasingly is being used to personalize learning and give students more choice over what and how they learn and at what pace, preparing them to organize and direct their own learning for the rest of their lives” and the focus is on “using technology to transform learning experiences with the goal of providing greater equity and accessibility” [ ].
Serious video games as digital health interventions offer the unique opportunity to increase the accessibility and reach of theory-driven and tested interventions. The PlayForward intervention demonstrated efficacy in improving sexual attitudes and knowledge in racial/ethnic minority adolescents that persisted up to 12 months. Future research should assess PlayForward’s efficacy in populations with higher rates of sexual initiation and its comparative efficacy, effectiveness, and implementation.
We acknowledge all authors who contributed significantly to this work. We also acknowledge our game developer partners at Schell Games; our research staff and students, particularly Erica Miller, Orli Florsheim, and Christina Mezes; Gail Slap, MD, MS and Peter Salovey, PhD for their input and support; and Alex Seropian and Noah Falstein for their guidance on the games aspects of this study. We also thank our Data and Safety Monitoring Board members. We offer special thanks to the afterschool and youth programs and their staff and all the adolescents that participated and made a major contribution to this study. The study was supported by grant R01HD062080 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development.
Conflicts of Interest
Multimedia Appendix 1
PlayForward homescreen.PNG File, 3MB
Multimedia Appendix 2
A Challenge Stack: 7 Minutes in Heaven.PNG File, 3MB
Multimedia Appendix 3
Know Sense mini-game.JPG File, 141KB
Multimedia Appendix 4
People Sense mini-game.PNG File, 1MB
Multimedia Appendix 5
CONSORT-EHEALTH checklist (v1.6.1).PDF File (Adobe PDF File), 768KB
- Centers for Disease Control and Prevention. 2015 Nov. HIV Surveillance Report, 2014 URL: https://www.cdc.gov/hiv/pdf/library/reports/surveillance/cdc-hiv-surveillance-report-us.pdf [accessed 2016-07-20] [WebCite Cache]
- National Conference of State Legislatures. State Policies on Sex Education in Schools. 2016 Dec 21 Presented at: National Conference of State Legislatures; May 5, 2015; Chicago, IL URL: http://www.ncsl.org/research/health/state-policies-on-sex-education-in-schools.aspx
- Collins C, Harshbarger C, Sawyer R, Hamdallah M. The diffusion of effective behavioral interventions project: development, implementation, and lessons learned. AIDS Educ Prev 2006 Aug;18(4 Suppl A):5-20. [CrossRef] [Medline]
- Galbraith JS, Stanton B, Boekeloo B, King W, Desmond S, Howard D, et al. Exploring implementation and fidelity of evidence-based behavioral interventions for HIV prevention: lessons learned from the focus on kids diffusion case study. Health Educ Behav 2009 Jun;36(3):532-549 [FREE Full text] [CrossRef] [Medline]
- Veniegas RC, Kao UH, Rosales R, Arellanes M. HIV prevention technology transfer: challenges and strategies in the real world. Am J Public Health 2009 Apr;99 Suppl 1:S124-S130 [FREE Full text] [CrossRef] [Medline]
- Owczarzak J, Dickson-Gomez J. Provider perspectives on evidence-based HIV prevention interventions: barriers and facilitators to implementation. AIDS Patient Care STDS 2011 Mar;25(3):171-179 [FREE Full text] [CrossRef] [Medline]
- Harshbarger C, Simmons G, Coelho H, Sloop K, Collins C. An empirical assessment of implementation, adaptation, and tailoring: the evaluation of CDC's National Diffusion of VOICES/VOCES. AIDS Educ Prev 2006 Aug;18(4 Suppl A):184-197. [CrossRef] [Medline]
- Rebchook GM, Kegeles SM, Huebner D. Translating research into practice: the dissemination and initial implementation of an evidence-based HIV prevention program. AIDS Educ Prev 2006 Aug;18(4 Suppl A):119-136. [CrossRef] [Medline]
- Bailey JV, Murray E, Rait G, Mercer CH, Morris RW, Peacock R, et al. Interactive computer-based interventions for sexual health promotion. Cochrane Database Syst Rev 2010(9):CD006483. [CrossRef] [Medline]
- Roberto AJ, Zimmerman RS, Carlyle KE, Abner EL, Cupp PK, Hansen GL. The effects of a computer-based pregnancy, STD, and HIV prevention intervention: a nine-school trial. Health Commun 2007;21(2):115-124. [CrossRef] [Medline]
- Barak A, Fisher WA. Toward an internet‐driven, theoretically‐based, innovative approach to sex education. J Sex Res 2001 Nov;38(4):324-332. [CrossRef]
- Abt C. Serious Games. New York: The Viking Press; 1970.
- Fiellin L, Hieftje K, Duncan L. Videogames, here for good. Pediatrics 2014 Nov;134(5):849-851 [FREE Full text] [CrossRef] [Medline]
- Hieftje K, Edelman EJ, Camenga DR, Fiellin LE. Electronic media-based health interventions promoting behavior change in youth: a systematic review. JAMA Pediatr 2013 Jun;167(6):574-580 [FREE Full text] [CrossRef] [Medline]
- Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MFG. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. BMJ 2012 Apr 19;344(apr18 3):e2598-e2598. [CrossRef]
- Lieberman D. Interactive video games for health promotion: effects on knowledge, self-efficacy, social support, and health. In: Street RL, Gold WR, Manning T, editors. Health Promotion and Interactive Technology: Theoretical Applications and Future Directions. Mahwah, NJ: Lawrence Erlbaum Associates; 1997:103-120.
- Lieberman DA. Management of chronic pediatric diseases with interactive health games: theory and research findings. J Ambul Care Manage 2001 Jan;24(1):26-38. [Medline]
- Krishna S, Francisco BD, Balas EA, König P, Graff GR, Madsen RW. Internet-enabled interactive multimedia asthma education program: a randomized trial. Pediatrics 2003 Mar;111(3):503-510. [Medline]
- Kato PM, Cole SW, Bradlyn AS, Pollock BH. A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics 2008 Aug;122(2):e305-e317 [FREE Full text] [CrossRef] [Medline]
- Lenhart A, Kahne J, Middaugh E, Macgill A, Evans C, Vitak J. Pew Research Center. 2008 Sep 16. Teens, video games, and civics URL: http://www.pewinternet.org/2008/09/16/teens-video-games-and-civics/ [accessed 2017-08-26] [WebCite Cache]
- Speak Up Digital Learning 24/7: Understanding Technology-Enhanced Learning in the Lives of Today's Students. Irvine, CA: Project Tomorrow; 2015. URL: http://www.tomorrow.org/speakup/pdfs/SU14StudentReport.pdf [accessed 2017-05-31] [WebCite Cache]
- Shaffer D, Squire K, Halverson R, Gee J. Video games and the future of learning. Phi Delta Kappan 2005;87:105-111 [FREE Full text]
- Lave J, Wenger E. Situated Learning: Legitimate Peripheral Participation. New York: Cambridge University Press; 1991.
- Street R, Rimal R. Health promotion and interactive technology: a conceptual foundation. In: Street R, Gold W, Manning T, editors. Health Promotion and Interactive Technology: Theoretical Applications and Future DirectionsI. Mahwah, NJ: Lawrence Erlbaum Associates; 1997:1-18.
- Hieftje K, Duncan L, Pendergrass T, Sawyer B, Fiellin L. Development of a HIV prevention videogame: lessons learned. Int J Serious Games 2016;3(2):83-90.
- Fiellin LE, Kyriakides TC, Hieftje KD, Pendergrass TM, Duncan LR, Dziura JD, et al. The design and implementation of a randomized controlled trial of a risk reduction and human immunodeficiency virus prevention videogame intervention in minority adolescents: PlayForward: Elm City Stories. Clin Trials 2016 Aug;13(4):400-408. [CrossRef] [Medline]
- Nadkarni PM, Brandt C, Frawley S, Sayward FG, Einbinder R, Zelterman D, et al. Managing attribute--value clinical trials data using the ACT/DB client-server database system. J Am Med Inform Assoc 1998;5(2):139-151 [FREE Full text] [Medline]
- Stanton BF, Li X, Ricardo I, Galbraith J, Feigelman S, Kaljee L. A randomized, controlled effectiveness trial of an AIDS prevention program for low-income African-American youths. Arch Pediatr Adolesc Med 1996 Apr;150(4):363-372. [Medline]
- Rotheram-Borus MJ, Song J, Gwadz M, Lee M, Van RR, Koopman C. Reductions in HIV risk among runaway youth. Prev Sci 2003 Sep;4(3):173-187. [Medline]
- Hieftje K, Duncan LR, Fiellin LE. Novel methods to collect meaningful data from adolescents for the development of health interventions. Health Promot Pract 2014 Sep;15(5):714-722 [FREE Full text] [CrossRef] [Medline]
- Hieftje K, Rosenthal M, Camenga D, Edelman E, Fiellin L. A qualitative study to inform the development of a videogame for adolescent human immunodeficiency virus prevention. Games Health J 2012;1(4):1-5 [FREE Full text] [CrossRef]
- Duncan LR, Hieftje KD, Culyba S, Fiellin LE. Game playbooks: tools to guide multidisciplinary teams in developing videogame-based behavior change interventions. Transl Behav Med 2014 Mar;4(1):108-116 [FREE Full text] [CrossRef] [Medline]
- Camenga DR, Hieftje KD, Fiellin LE, Edelman EJ, Rosenthal MS, Duncan LR. The use of message framing to promote sexual risk reduction in young adolescents: a pilot exploratory study. Health Educ Res 2014 Apr;29(2):360-366 [FREE Full text] [CrossRef] [Medline]
- Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 1977;84(2):191-215. [CrossRef]
- Bandura A. Self-Efficacy: The Exercise of Control. New York: Freeman; 1997.
- Rothman AJ, Salovey P. Shaping perceptions to motivate healthy behavior: the role of message framing. Psychol Bull 1997 Jan;121(1):3-19. [Medline]
- Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science 1981 Jan 30;211(4481):453-458. [Medline]
- Kazdin A. Research Design in Clinical Psychology. New York: Harper & Row; 1980.
- Montanaro E, Fiellin L, Fakhouri T, Kyriakides T, Duncan L. Using videogame apps to assess gains in adolescents' substance use knowledge: new opportunities for evaluating intervention exposure and content mastery. J Med Internet Res 2015 Oct 28;17(10):e245 [FREE Full text] [CrossRef] [Medline]
- Centers for Disease Control and Prevention. 2009. 2009 State and Local Youth Risk Behavior Survey URL: http://www.cdc.gov/HealthyYouth/yrbs/pdf/questionnaire/2009HighSchool.pdf [accessed 2008-11-16] [WebCite Cache]
- Centers for Disease Control and Prevention. 2010. 2009 Middle School Youth Risk Behavior Survey URL: http://www.cdc.gov/HealthyYouth/yrbs/pdf/questionnaire/2009MiddleSchool.pdf [accessed 2008-11-16] [WebCite Cache]
- Tortolero SR, Markham CM, Peskin MF, Shegog R, Addy RC, Escobar-Chaves SL, et al. It's Your Game: Keep It Real: delaying sexual behavior with an effective middle school program. J Adolesc Health 2010 Feb;46(2):169-179 [FREE Full text] [CrossRef] [Medline]
- Basen-Engquist K, Mâsse LC, Coyle K, Kirby D, Parcel GS, Banspach S, et al. Validity of scales measuring the psychosocial determinants of HIV/STD-related risk behavior in adolescents. Health Educ Res 1999 Feb;14(1):25-38. [Medline]
- Kelly JA, St Lawrence JS, Hood HV, Brasfield TL. An objective test of AIDS risk behavior knowledge: scale development, validation, and norms. J Behav Ther Exp Psychiatry 1989 Sep;20(3):227-234. [Medline]
- Jemmott L, Jemmott J, McCaffree K. The National Campaign to Prevent Teen and Unplanned Pregnancy. 2005. Making a Difference!: An abstinence approach to HIV/STD and teen pregnancy prevention URL: http://thenationalcampaign.org/effective-programs/making-difference-abstinence-based-approach-hivstd-and-teen-pregnancy-prevention [accessed 2017-08-26] [WebCite Cache]
- St Lawrence JS, Brasfield TL, Jefferson KW, Alleyne E, O'Bannon RE, Shirley A. Cognitive-behavioral intervention to reduce African American adolescents' risk for HIV infection. J Consult Clin Psychol 1995 Apr;63(2):221-237. [Medline]
- Clark L, Miller K, Nagy S, Avery J, Roth D, Liddon N, et al. Adult identity mentoring: reducing sexual risk for African-American seventh grade students. J Adolesc Health 2005 Oct;37(4):337. [CrossRef] [Medline]
- Ingram BL, Flannery D, Elkavich A, Rotheram-Borus MJ. Common processes in evidence-based adolescent HIV prevention programs. AIDS Behav 2008 May;12(3):374-383 [FREE Full text] [CrossRef] [Medline]
- Johnson BT, Carey MP, Marsh KL, Levin KD, Scott-Sheldon LAJ. Interventions to reduce sexual risk for the human immunodeficiency virus in adolescents, 1985-2000: a research synthesis. Arch Pediatr Adolesc Med 2003 Apr;157(4):381-388. [CrossRef] [Medline]
- Pedlow C, Carey M. HIV sexual risk-reduction interventions for youth: A review and methodological critique of randomized controlled trials. Behav Modif 2003;27:135-190 [FREE Full text]
- Eaton DK, Kann L, Kinchen S, Shanklin S, Ross J, Hawkins J, et al. Youth risk behavior surveillance--United States, 2007. MMWR Surveill Summ 2008 Jun 06;57(4):1-131 [FREE Full text] [Medline]
- Cohen J. Statistical Power Analysis for Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
- Goesling B, Colman S, Trenholm C, Terzian M, Moore K. Programs to reduce teen pregnancy, sexually transmitted infections, and associated sexual risk behaviors: a systematic review. J Adolesc Health 2014 May;54(5):499-507. [CrossRef] [Medline]
- Sales JM, Milhausen RR, Diclemente RJ. A decade in review: building on the experiences of past adolescent STI/HIV interventions to optimise future prevention efforts. Sex Transm Infect 2006 Dec;82(6):431-436 [FREE Full text] [CrossRef] [Medline]
- Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med 2016 Nov;51(5):833-842. [CrossRef] [Medline]
- Jemmott JB, Jemmott LS, Fong GT. Efficacy of a theory-based abstinence-only intervention over 24 months: a randomized controlled trial with young adolescents. Arch Pediatr Adolesc Med 2010 Feb;164(2):152-159 [FREE Full text] [CrossRef] [Medline]
- Coyle KK, Kirby DB, Marín BV, Gómez CA, Gregorich SE. Draw the line/respect the line: a randomized trial of a middle school intervention to reduce sexual risk behaviors. Am J Public Health 2004 May;94(5):843-851. [Medline]
- Fagen MC, Stacks JS, Hutter E, Syster L. Promoting implementation of a school district sexual health education policy through an academic-community partnership. Public Health Rep 2010;125(2):352-358 [FREE Full text] [CrossRef] [Medline]
- Alldred P, David M, Smith P. Teachers' views of teaching sex education: pedagogy and models of delivery. J Educ Enquiry 2003;4(1):80-96.
- Peskin M, Hernandez B, Markham C, Johnson K, Tyrell S, Addy R, et al. Sexual health education from the perspective of school staff: implications for adoption and implementation of effective programs in middle school. J Applied Res Children 2011;2(2):9-38.
- Donovan P. School-based sexuality education: the issues and challenges. Fam Plann Perspect 1998;30(4):188-193. [CrossRef]
- Pendergrass TM, Hieftje K, Crusto CA, Montanaro E, Fiellin LE. If we build it, will they come? A qualitative study of key stakeholder opinions on the implementation of a videogame intervention for risk reduction in adolescents. Games Health J 2016 Aug;5(4):279-285 [FREE Full text] [CrossRef] [Medline]
- UNFPA Operational Guidance for Comprehensive Sexuality Education: A Focus on Human Rights and Gender. New York: United Nations Population Fund; 2014. URL: http://www.unfpa.org/sites/default/files/pub-pdf/UNFPA_OperationalGuidance_WEB3.pdf [accessed 2016-05-02] [WebCite Cache]
- Oldenburg B, Taylor CB, O'Neil A, Cocker F, Cameron LD. Using new technologies to improve the prevention and management of chronic conditions in populations. Annu Rev Public Health 2015 Mar 18;36:483-505. [CrossRef] [Medline]
- Pettifor A, Bekker L, Hosek S, DiClemente R, Rosenberg M, Bull SS, et al. Preventing HIV among young people: research priorities for the future. J Acquir Immune Defic Syndr 2013 Jul;63 Suppl 2:S155-S160 [FREE Full text] [CrossRef] [Medline]
- Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med 2016 Nov;51(5):843-851. [CrossRef] [Medline]
- Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med 2008 Feb 19;148(4):295-309. [Medline]
- Centers for Disease Control and Prevention. 2016. Youth Risk Behavior Surveillance -- United States, 2015 URL: https://www.cdc.gov/mmwr/volumes/65/ss/ss6506a1.htm [accessed 2016-11-01] [WebCite Cache]
- Flanagan SM, Greenfield S, Coad J, Neilson S. An exploration of the data collection methods utilised with children, teenagers and young people (CTYPs). BMC Res Notes 2015 Mar 01;8:61 [FREE Full text] [CrossRef] [Medline]
- Wyrick DL, Bond L. Reducing sensitive survey response bias in research on adolescents: a comparison of web-based and paper-and-pencil administration. Am J Health Promot 2011;25(5):349-352. [CrossRef] [Medline]
- Beebe T, Harrison P, Park E, Evans J. The effects of data collection mode and disclosure on adolescent reporting of health behaviour. Soc Sci Comput Rev 2006;24(4):476-488 [FREE Full text]
- Afterschool Alliance. America After 3PM: Afterschool Programs in Demand. 2014. URL: http://www.afterschoolalliance.org/documents/AA3PM-2014/AA3PM_Key_Findings.pdf [accessed 2017-05-31] [WebCite Cache]
- Albarracin D, Durantini M, Earl A. Empirical and theoretical conclusions of an analysis of outcomes of HIV-prevention interventions. Curr Dir Psychol Sci 2006;15(2):73-78 [FREE Full text]
- Ajzen I, Fishbein M. The influence of attitudes on behavior. In: Albarracin D, Johnson BT, Zanna MP, editors. The Handbook of Attitudes. Mahwah, NJ: Lawrence Erlbaum Associates; 2005:173-221.
- Basen-Engquist K, Parcel G. Attitudes, norms, and self-efficacy: a model of adolescents' HIV-related sexual risk behavior. Health Educ Q 1992;19(2):263-277. [Medline]
- Holland RW, Verplanken B, Van Knippenberg A. On the nature of attitude-behavior relations: the strong guide, the weak follow. Eur J Soc Psychol 2002 Nov;32(6):869-876. [CrossRef]
- Albarracín D, Johnson BT, Fishbein M, Muellerleile PA. Theories of reasoned action and planned behavior as models of condom use: a meta-analysis. Psychol Bull 2001 Jan;127(1):142-161 [FREE Full text] [Medline]
- Allison S, Bauermeister JA, Bull S, Lightfoot M, Mustanski B, Shegog R, et al. The intersection of youth, technology, and new media with sexual health: moving the research agenda forward. J Adolesc Health 2012 Sep;51(3):207-212 [FREE Full text] [CrossRef] [Medline]
- US Department of Education, Office of Educational Technology. 2016 National Education Technology Plan. 2016. Future ready learning: reimagining the role of technology in education URL: https://tech.ed.gov/files/2015/12/NETP16.pdf [accessed 2017-05-31] [WebCite Cache]
|AIDS: acquired immune deficiency syndrome|
|DSMB: data and safety monitoring board|
|HIV: human immunodeficiency virus|
|LS: least squares|
|STI: sexually transmitted infection|
Edited by G Eysenbach; submitted 04.06.17; peer-reviewed by K Curtis, A Fitzgerald; comments to author 06.07.17; revised version received 09.08.17; accepted 14.08.17; published 18.09.17
©Lynn E Fiellin, Kimberly D Hieftje, Tyra M Pendergrass, Tassos C Kyriakides, Lindsay R Duncan, James D Dziura, Benjamin G Sawyer, Linda Mayes, Cindy A Crusto, Brian WC Forsyth, David A Fiellin. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 18.09.2017.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.