The study was performed in accordance with the Declaration of Helsinki. A study protocol detailing objectives, procedures, materials, and measures was submitted to the Ethics Committee of the Berlin Medical Association prior to data collection. The ethics committee ruled that the nature of the project does not require professional ethical and legal consultation (reference number: Eth-KB-25-017, dated August 29, 2025). Following ethical review, the study was registered at the German Clinical Trials Register (DRKS00037830). Before participation, individuals were informed about the study’s aims, procedures, and data security measures. They were explicitly notified that participation was voluntary and that they could withdraw at any time without negative consequences. All participants provided informed consent via checkboxes on the survey platform; for participants aged 15 years and younger, additional informed consent was obtained from a parent or legal guardian. Privacy and confidentiality of the participants’ data and identity were maintained throughout the entire study procedure. Participants were compensated by the market research institute responsible for data collection.

This study was conducted in two independent samples of teachers and adolescents from Germany. Participants were recruited from a large online panel with approximately 150,000 participants that are—in turn—recruited exclusively offline via telephone. The panel is representative of the German-speaking population in Germany, aged 14 years and older. A range of quality assurance protocols ensure the prevention of duplicate panel participation. For the teachers’ sample, a random sample from all panelists working as teachers at secondary or vocational schools was drawn (ie, no primary school teachers). Before study onset, teacher and school status were verified by the participants via screening questions. Adolescents (aged 14-17 years) were recruited either directly from the panel via existing panel data or through their parents using a random selection process (specifically the “last birthday method”) to ensure representativeness. Parents were asked to let their children complete the questionnaire independently to maintain data integrity.

For the teachers’ sample, the following inclusion criteria were determined: informed consent, male, female or diverse gender, age 18 years and older, currently working as a teacher in a secondary school, sufficient knowledge of the German language, and living in Germany. For the adolescents’ sample, inclusion criteria were informed consent, male, female or diverse gender, age between 14 and 17 years, sufficient knowledge of the German language, and living in Germany. There were no exclusion criteria apart from not fulfilling the inclusion criteria.

Participants in both samples were invited via e-mail with a maximum of 2 reminders sent.

An a-priori sample size calculation was computed using G*Power (version 3.1.9.7) [32] with the following parameters for the interaction effect in an analysis of variance, which was the primary outcome in this study (see Statistical Analysis): α=.05, effect size η²_p=.03 (f=0.176), power=.80. This resulted in a minimum sample size of 258 participants per sample. The small to medium effect size is based on previous studies of similar evaluations [17-19]. In order to ensure a sufficiently large number of cases even in the event of possible dropouts, we aimed to recruit approximately 500 individuals per sample. Data collection took place in October and November 2025.

The intervention evaluated here is a short interactive e-learning module, developed to enhance specific aspects of digital health literacy and to be used on mobile or larger devices. It takes approximately 10 minutes to complete. Short mobile interventions can be used as stand-alone interventions or to complement and foster topics already addressed in school lessons or other learning settings. This flexibility facilitates their implementation in blended-learning environments [33]. To enhance motivation and engagement of the users, the short mobile interventions incorporate elements of gamification [28-30], by using game design elements [34]. Specifically, the users win and lose points and get immediate feedback for their actions and decisions, and they are awarded a virtual trophy depending on their final score. These gamification elements can enhance motivation and engagement of the users to actively deal with the topics addressed in the modules [29,30]. The interventions combine interactive visual and written elements. They are complemented by optional sounds that can be deactivated and are not necessary for their successful completion, for example when used in larger classrooms. Several different short mobile interventions were recently developed, but only one was selected to be evaluated here in detail. This specific intervention focuses on how to recognize good graphs with the help of different criteria. Another intervention was presented to the control group that focuses on how to interpret average values in the context of health information (eg, arithmetic means). The interventions are structured in five parts. First, a short technical introduction and instructions are given to explain the structure and functions of the intervention, how to interact with it, and how the point system works. This technical introduction is followed by a topical introduction that is both target-group–oriented and problem-based. In the case of the short mobile intervention evaluated here, this topical introduction includes the presentation of two graphs that convey information about germs on different surfaces that the participants might encounter when scrolling through social media. After a brief introduction, the main part begins, in which they learn how to judge good graphs based on certain criteria, building the skillset to properly judge the two graphs from the introduction. This part consists of multiple exercises, has a progress bar, has points added and subtracted from a total score, and incorporates feedback elements to explain correct and incorrect responses. Five different learning objectives are pursued in this main part that are addressed in several different tasks, for example, multiple-choice questions, selection tasks, and slide bars:

The final part includes brief feedback. Participants are awarded one of three trophies (bronze, silver, or gold) or no trophy according to their overall score. If they have not reached more than half of the total number of points, they do not receive a trophy and are invited to repeat the short mobile intervention to foster their knowledge. Screenshots of the intervention used in the experimental group can be found in Figure 1.

This web-based study used a prospective randomized-controlled pre-post study design with one experimental group and an active control group. Group allocation was based on a computerized randomization procedure with an allocation ratio of 1:1. The randomization sequence was generated by the market research institute. Participants were blinded to the fact whether their respective mobile intervention was the experimental or the control intervention. At T1 (preassessment), all participants completed an online survey to obtain sociodemographic information, baseline levels of objective and subjective knowledge, and digital health literacy. Then, both groups conducted their respective short mobile intervention described above (see Intervention), which was directly integrated into the survey administration software. At T2 (postassessment), participants in both groups filled in the knowledge and digital health literacy questionnaires again. In addition, a number of questionnaires directly related to the intervention were assessed. These encompassed questions on quality, visual aesthetics, length, tempo, perceived difficulty, applicability in schools, implementation intentions, and motivation (for more details, refer to Measures).

Data were preprocessed using the statistical software SPSS (version 29.0.2.0; IBM Corp). All inferential analyses were then conducted with the statistical software JASP (version 0.19.1). Only participants with complete data (T1 and T2) were included in the analyses. To analyze the effectiveness of the intervention, mixed ANOVAs were computed with the between-subjects factor group (experimental group, control group) and the within-subjects factor time (T1, T2), separately for each dependent variable (total score subjective knowledge self-report, total score objective knowledge quiz, mean values for the DHLI “evaluating reliability” and “determining relevance” subscales) and each sample. If these resulted in significant interaction effects, additional 2-tailed t tests were computed for the direct comparison of each pair of factor levels. Effect sizes for the ANOVAs are reported as η²_p. Effect sizes for the post hoc t tests are reported as Cohen d. For all analyses, P values are reported two-sided, and α=.05 was used as the significance threshold. P values were adjusted using a Bonferroni-correction for the post hoc t tests. Internal consistency reliability coefficients for the overall and subscale scores were computed as Cronbach α. These can be interpreted with the following rule of thumb: >.90 excellent, >.80 good, >.70 acceptable, >.60 questionable, >.50 poor and <.50 unacceptable [40].

In the adolescents’ sample, 500 individuals (n=260 female, n=232 male, n=8 with diverse gender) aged 14-17 years (mean 15.46, SD 1.11 years) participated in the study. The adolescents studied at different types of schools with the biggest proportion (n=313) attending Gymnasium, the secondary school to acquire a university entrance qualification.

In the teachers’ sample, 603 individuals (n=322 female, n=281 male) aged 20-69 years (mean 51.48, SD 9.11 years) participated in the study. The biggest proportions of teachers (n=170) taught at Gymnasium or at vocational schools (n=136).

The number of participants across the study phases is presented in Figure 2. There were no significant differences regarding age (adolescents: t₄₉₈=0.65, d=0.06; P=.52; teachers: t₆₀₁=–0.65, d=–0.05; P=.52), nor gender distribution (adolescents: χ²₂=0.65, V=0.04; P=.72; teachers: χ²₁=0.002, V=0.002; P=.97) between the experimental and control groups in either sample.

Cronbach α indices of internal consistency for both samples and time points are in Table S1 in Multimedia Appendix 2. The indices for the objective knowledge scale were unacceptable in both samples at T1 (α=.48-.49), and poor to questionable at T2 (α=.54-.62). For the subjective knowledge scale excellent Cronbach α indices of internal consistency were obtained across both samples and time points (α=.92-.95). The DHLI scales reached acceptable to good indices (α=.70-.85), while the VisAWI and SEEQ scales reached good to excellent internal consistency indices (α=.80-.95).

This study evaluates an innovative, accessible, and gamified short intervention designed for mobile use (eg, on smartphones and tablets) and flexible deployment in educational settings such as schools. The effectiveness of the intervention was analyzed with a randomized-controlled pre-post approach with an active control group in 2 separate samples, that is, adolescents in secondary school (the main target group of the intervention) and teachers. Consistent with some—but not all—hypotheses, the results showed significant interaction effects (eg, stronger increases in the experimental groups) for objective knowledge, self-reported knowledge, and one facet of digital health literacy in the teacher sample. Among adolescents, a significant interaction effect was observed only for objective knowledge but not for the other three outcomes (including the primary subjective knowledge outcome).

For the main target group of the intervention, adolescents in secondary school, significantly higher increases could be observed in only one of four outcomes for the experimental group compared to the control group. Specifically, the experimental group’s objective knowledge increased from the baseline to the post-intervention assessment, while there was no such increase in the control group. This finding is consistent with the hypotheses and with prior findings demonstrating positive effects of (more comprehensive) digital health literacy interventions for adolescents in Germany [17-19] and other target groups [41-43]. Overall, our findings concerning objective knowledge underline the potential of using short mobile interventions for the promotion of aspects of digital health literacy. However, contrary to our hypothesis, this objective improvement did not translate into a significant increase in self-reported subjective knowledge among adolescents and was also not observed in the 2 digital health literacy subscales “evaluating reliability” and “determining relevance.” Interestingly, the control group also showed descriptive improvements in subjective knowledge and health literacy following their intervention, which likely explains the lack of statistically significant interactions. There are several potential reasons for this pattern of results: first, the control intervention, a short mobile intervention on a different topic (on average values), might have been too similar to the experimental intervention [44]. Specifically, the control intervention might have also promoted some aspects captured by the knowledge and digital health literacy scales, although it was initially designed to focus on other aspects of digital health literacy, namely numeracy in the context of health information (for example, understanding how the arithmetic mean can be interpreted in health contexts). For example, evaluating health information in graphs might implicitly require understanding averages which was promoted in the control group intervention. To prevent this overlap between groups, future trials should use active control groups with truly distinct intervention topics. Second, expectation, demand, and placebo effects might have inflated the improvements in the control group [45,46]. This is supported by the fact that the lack of statistical interaction effects only occurred in the self-reported outcomes and not the—arguably more objective—knowledge quiz [47]. Third, the scope of the experimental short mobile intervention—due to its intended brevity (10 minutes)—might have been too narrow to promote knowledge and digital health literacy with large effect sizes in this target group. A broad and comprehensive promotion of digital health literacy with mobile interventions might therefore only be achieved when used in blended-learning environments alongside other educational approaches or when multiple modules on similar topics are combined [33].

Evidence of the potential negative effects of smartphone use [48-50], particularly among adolescents, has sparked debate regarding the introduction of school-based smartphone bans [51]. However, restrictive device bans do not resolve these issues entirely, as mobile technology has become deeply integrated into everyday life. Consequently, educating adolescents on the responsible use of these devices is a vital strategy for enhancing both digital literacy and digital health literacy [5,52,53]. Importantly, the flexibility of the intervention evaluated in this study makes it applicable even in schools with comprehensive smartphone bans, for example, by being presented on school-supplied hardware, such as tablets or laptops.

In contrast to the adolescents, in the teachers’ sample, the effects were more robust. Specifically, statistically significant interaction effects were observed for both objective and subjective knowledge and the “evaluating reliability” subscale of digital health literacy (though not fully supported by the post hoc tests for the latter). The only outcome without any significant interaction effects was the “determining relevance” subscale of the DHLI. As for this subscale, no significant results could be obtained in either sample, it is possible that it did not adequately measure the specific dimensions of digital health literacy targeted by the intervention. Future development of intervention modules should more explicitly address the gap between the information presented and the user’s perceived personal applicability, as this is an integral aspect of digital health literacy [6,7]. The observation that subjective knowledge effects were more robust in the teachers’ sample than among adolescents might be explained by teachers estimating their baseline level of knowledge more realistically, whereas adolescents may have overestimated theirs [54,55]. This interpretation is corroborated by the convergence of qualitative findings for objective and subjective knowledge among teachers, a pattern not observed in the adolescent sample. Further evidence for this interpretation comes from the baseline data: whereas adolescents showed descriptively lower levels of objective knowledge than teachers, the difference between the samples’ subjective knowledge was notably smaller. Teachers might have had relevant prior knowledge and abstract reasoning skills [56] on which they could build to efficiently learn the contents of the mobile intervention. Another possible reason for the discrepancies between the adolescents’ and the teachers’ findings is that teachers were more motivated given their professional interest in the intervention. This is partly supported by the fact that the teachers reported higher levels of motivation and implementation intentions in the postintervention survey than adolescents.

Overall, the largely positive evaluation results in the teachers’ sample provide important evidence for the potential of the short mobile intervention to be used in an educational context. However, these results also indicate that the intervention works better for older and more educated individuals than for the main target group. This highlights that future revisions and newly developed similar interventions should aim for a tighter alignment with the primary target group’s baseline abilities.

Across both samples, the intervention was met with high overall acceptance and positive aesthetic evaluations, underscoring the value of high-quality design in digital learning tools [39]. In addition, the majority of the teacher and adolescent participants found the tempo, length, and difficulty of the intervention appropriate. While endorsement of the intervention for private use was only moderate in both samples, teachers reported high willingness to use the intervention in professional—that is, educational—contexts, for example, by recommending them to colleagues and students, and by implementing them in classes and project weeks. Furthermore, the majority of adolescents confirmed that such integration would enhance their motivation. Taken together, these descriptive findings highlight the aesthetic and educational quality of the intervention, supporting its potential for effective integration into secondary educational settings. The gamification elements incorporated in the intervention may have enhanced the motivation of the adolescents and strengthened the teachers’ implementation intentions [29,30,57]. Moreover, the intervention’s brevity, coupled with its alignment with German media, information, and digital literacy curricula, further underscores its broad applicability and feasibility within educational settings [20]. Hence, the results of this evaluation highlight that the short mobile interventions could be used to promote aspects of digital health literacy as required by national legislation in Germany, such as the German Social Code Book V [15,16].

A few limitations should be noted when interpreting the results of the study. First, as outlined above, the control group received an intervention highly similar to the experimental group intervention, introducing a risk of confounding. Second, because of the brevity of the intervention (which was intended to secure its flexible use in diverse school contexts), only a few aspects of digital health literacy were focused on here, namely data—and more specifically—graph literacy. While these are integral and often overlooked elements of digital health literacy education [23], the comprehensive promotion of (digital) health literacy in schools should more broadly cover all of its central elements [12]. This aim might be achieved by combining multiple e-learning modules, such as the examples presented in this research, with other didactic approaches, for example, educational videos [58,59] and classic classroom teaching materials such as worksheets and discussions. Third, the outcomes of the present evaluation partly relied on self-reports, which may have introduced biases due to impression management and limited introspection abilities [60]. Fourth, the reliability of the objective knowledge quiz was very low, possibly due to the difficulty of some items, the diversity of the content captured by a single scale, and the relatively strict scoring scheme. In the future, more thorough pretesting with individuals from the target group might help to improve the scale’s quality. Fifth, the short retest interval precludes the interpretation of long-term changes in digital health literacy and thus does not allow for testing the stability of the effects, which should be addressed in more comprehensive future studies. Sixth, the intervention was tested online in a setting that offered less control than in a laboratory-based experiment. Some aspects of school environments, such as distractions or technical problems, might not be well represented in this setting. In the future, the short mobile intervention should therefore also be tested in environments with higher ecological validity, such as classrooms. Finally, the DHLI instrument was adapted to match the contents of the evaluated intervention. Results are therefore not comparable to other studies using the same instrument.

Overall, the results highlight the potential of using short mobile interventions in schools to promote aspects of digital health literacy, for example, regarding knowledge on the interpretation of health-related graphs, thereby contributing to public health endeavors to promote digital health literacy. However, as not all relevant outcomes (including the primary outcome) showed statistically significant results, the intervention should be further optimized and refined in the future to secure more comprehensive knowledge and competence gains in adolescents. Schools should consider using short, gamified mobile interventions like the one tested here as feasible, low-cost tools to teach specific digital health literacy skills. However, to change students’ broader health literacy competencies and knowledge, these modules are likely best used as part of a sequenced curriculum, not as stand-alone solutions.