To answer the research questions, the study used a multimodal data collection process combining web analytics as well as self-reported demographic and UX measures.

Data collection took place over the 10-month period between May 1, 2024, and March 1, 2025. During this time period, the LONDI platform was advertised on social media platforms targeting users in regions in which the majority of the population are native German speakers (ie, Germany, Switzerland, and Austria). Visits to the LONDI platform were tracked, and data were also collected from voluntarily completed pop-up questionnaires. The data collected for this study enabled an extension of the previous evaluations in 2 ways. First, by assessing the newest platform version (ie, launched in 2024), and second, by incorporating additional state-of-the-art research methods, namely, reading time and chatbot engagement.

This study was part of a larger project (ie, LONDI), approved by the ethics committee at the university hospital of the Ludwig Maximilian University of Munich (approval number 22-0300 1 V). Participants who took part in the project’s controlled experimental studies provided written informed consent. This was not done for this study, due to its real-world setting. Namely, its participants were those who used the online platform of their own volition, rather than participating in a study done in an experimental setting. Given these circumstances, the ethics committee approved waiving informed consent for this study, provided that strict measures to ensure anonymity were adhered to. Specifically, users’ IP addresses were anonymized via masking by Matomo, in accordance with strict data protection laws, and the experimenters had no access to participants’ unmasked IP addresses or any other identifying data. Furthermore, when accessing the platform, all users were informed via a pop-up notice that the website uses cookies and similar technologies to track data and were given the option to accept or opt out. Compensation was not offered.

This study’s goal was to perform an assessment of a digital health platform, using both web analytics and UX measurements. Specifically, the following 4 RE-AIM framework dimensions were assessed: reach, adoption, implementation, and maintenance. In terms of reach and adoption, the study results revealed that the platform reached its target population of mental health professionals and that they plan to adopt it in their everyday practice. Moreover, users rated the platform’s help system above the benchmark average both in terms of its pragmatic and its hedonic qualities. In terms of implementation, the results revealed that users did not implement the platform as expected, as indicated by the average time they spent on it and by the low chatbot engagement rate. In terms of maintenance, the results revealed that platform use changed over time, predominantly indicated by the sharp increase in the number of visits. All in all, the results from all but one of the assessed RE-AIM dimensions, namely, implementation, reflected a positive public health contribution.

The assessed metrics for reach and adoption were, respectively, the number of mental health professionals among the LONDI users, and their intention to keep using its help system. Results for the reach dimension revealed that the proportion of both user groups of mental health professionals (ie, learning therapists and school psychologists) exceeded their proportion in the German population. In the evaluation study of the initial version of LONDI [23], it was also found that the platform reached targeted users. This study’s results demonstrate that the retention was sustained. This is promising, as there are other valuable sources in the context of LDs for German speakers that users could have reverted to [49]. Nevertheless, the number of learning therapists was more than double that of the school psychologists, even though their estimated number in the German population is comparable. Further marketing endeavors are required to ensure that the platform is advertised equally to all its user groups. The adoption results revealed that users intend to keep using the help system, and this was predicted by UX qualities (eg, ease of use). Users’ intention to reuse the help system increased compared to the first phase evaluation [23]. This points toward the potential positive contribution caused by revising the help system (ie, the design was simplified and instructional tutorials were created). Continued endeavors are required for technical maintenance of the help system, as well as regular updates of the featured diagnostic and intervention recommendations. Thus, results from both the reach and adoption metrics were positive, and further endeavors are needed to optimize and retain this.

The assessed metrics for implementation were the reading time of the pages intended for mental health professionals, as well as chatbot engagement. These metrics were not used in the first LONDI evaluation phase. Rather, they were chosen based on insights from a first phase study, which used popular metrics used in commercial websites [23]. For example, the first phase study used the conversion rate metric (ie, the percentage of website visitors who completed a desired goal) to assess the platform’s implementation. However, such metrics were found to be suboptimal in the context of digital health, as their interpretation is based on online marketing benchmarks for profit-driven websites [50]. In this study, the chosen implementation metrics assessed whether users spend enough time on the platform to read its content and whether they interact with its chatbot.

The reading time results revealed that the average time users spent on the pages intended for mental health professionals was lower than the minimum time required to read them. The time users spend on pages is an important indicator of users’ engagement with platform content, as longer sessions enable more in-depth reading [27]. Notably, in the first LONDI evaluation phase, a notable shift occurred over time, with more users accessing the platform via smartphones rather than desktop devices [23]. This was also the case for the revised LONDI version, with this study revealing that most users accessed the platform via smartphones. To optimize users’ smartphone experience, the design of the revised LONDI version was optimized for smartphone use. Additionally, in line with the recommendation to improve online readability by designing pages with low visual complexity [51], infographics were added to break up long text blocks. Nevertheless, on average, users did not spend enough time on the tracked pages to read their content. Paradoxically, despite the rise in average screen times in recent years, which would, in theory, allow users to have longer reading sessions, smartphones are associated with reduced reading durations [52]. It is possible that LONDI visitors used smartphones for superficial reading of the platform’s content (eg, only reading the headings). This is in line with other studies demonstrating that compared to printed texts, online texts are read in a quicker, shallower manner, leading to reduced reading comprehension [53-55]. Thus, the findings from this study could be taken as an indication of low user engagement. Alternatively, it is possible that users wanted to get a quick overview of the platform, with the intention of reading certain content in more detail at a later point.

Notably, in compliance with GDPR, users’ data were anonymous (users’ IPs were masked), making it impossible to distinguish new users from returning users. Previous studies have shown that returning users need less time to read digital content [56]. A future assessment could ask users to use individual trackable links, enabling a distinction between the use data of new vs returning users. Interestingly, the reading times of all the tracked pages, regardless of their word count, ranged between 12 and 18 seconds. A future assessment could also assess whether there is a ceiling effect (ie, the maximum amount of time users spent on pages) and whether this differs between new and returning users.

In addition to reading time, the other assessed implementation metric was chatbot engagement, revealing an engagement rate far below the range suggested by industry benchmarks [44]. However, this finding needs to be interpreted cautiously, as the comparison relied on industry benchmarks due to the lack of established standards for digital health chatbots. Moreover, the LONDI platform was not designed for profit, unlike many industry-based platforms. Thus, this study’s comparison with industry benchmarks should be viewed as exploratory. In line with the findings of Abd-Alrazaq et al [39], further research is required to establish standardized measures for chatbot engagement in the digital health context. Furthermore, in this study, a simple FAQ chatbot was chosen over a more complex chatbot, as its implementation was feasible within the study’s scope. The results revealed that chatbot engagement rates were far below the industry benchmark [44]. One possible reason for the low engagement could be that the users did not think the chatbot was useful, as it only offered predefined questions and answers, not allowing users to type their own questions. This is in line with the notion that, despite the spike in technological advancements, many online interactions are still clumsy and do not fulfill their intended purpose [1]. Further endeavors are required to provide a more complex chatbot, better simulating a conversation with a human agent. Another reason for the low engagement could be related to users’ trust in AI and their willingness to engage with chatbots. Most of the visits to LONDI occurred from Germany. Interestingly, a study by Chang et al [57] showed that compared with other nationalities (eg, Brazilians), Germans are less open to engaging with chatbots. Another reason for the low engagement could be that the chatbot was too hidden, namely, users did not notice the chat bubble icon they had to press for the chatbot to open. This was done in order not to aggravate the users by having the chatbot pop up whenever they use LONDI. This is in line with the literature showing that users prefer a “polite pop-up” that only opens when clicked on [58]. Thus, future research should qualitatively examine user reactions to a more complex, attention-grabbing chatbot.

The assessed metric for maintenance was the use comparison between the previous and the revised platform versions. One notable result was that the number of visits increased by roughly 81%. This is encouraging, as increasing user traffic increases a website’s impact [59]. Nevertheless, this finding should be interpreted cautiously, as it stands in contrast with the finding that, on average, users do not spend enough time on the platform to read its content. Taken together, these results suggest that while visits to the platform increased, many visits were brief and allowed only limited, surface-level engagement. Importantly, the growth in the number of visits was largely accounted for by social media campaigns (ie, the LONDI Instagram and Facebook accounts). This stresses the importance of instrumentalizing social media to attract visitors. Notably, this growth was entirely not accounted for by referrals from AI assistants. This could be the result of the period of data collection ending in early 2025. European-wide data show that adults used AI less frequently than adolescents and young adults in 2025 [60]. To examine if these changes occur in the future, further endeavors should be made to examine changes in AI assistant referrals over time. Moreover, the written content of the revised platform version does not differ from the content of the previous version. Rather, the versions differ in their design (ie, the new version is more suitable for smartphone use, features more infographics, and the help system’s design was simplified, and instructional tutorials were added). This stresses the importance of creating a user-friendly design to attract visitors.

Another notable result was the increase in the number of visits from the United States, Canada, and Ireland, even though the platform’s content is entirely in German. This is surprising, as the number of users from these countries exceeded the number of users from Austria and Switzerland, where German is an official language [61]. One explanation for this could be that users were using a virtual private network, which can make it seem as if users are using the platform from a different location [62]. Another explanation could be that the platform was read in different languages using machine translation (eg, the Google Translate browser extension). Machine translation is often favored over professional translation, the latter being much more resource-intensive and time-consuming [63]. However, machine translation is not suitable for the LONDI platform, as a lot of its content is tailored to the needs of German residents (eg, specific information on the German school system). Thus, further endeavors should be made to maintain the growth in user traffic and to obtain the resources to professionally translate and adapt its content to other languages.

This study is not without its limitations. One limitation is inherent in the use of web analytics. While web analytics offer valuable insights, it is not possible to use them to infer actual engagement with platform content or users’ reading comprehension. In particular, the results concerning reading times and chatbot engagement need to be interpreted with caution, especially since there are no standard benchmarks for chatbot engagement in the context of digital health. As the purpose of the assessment was to evaluate how users use the platform in a natural setting, the sample consisted of anonymous users using the platform of their own accord. Therefore, it was not possible to use this sample to gather qualitative data (eg, via interviews) or to follow up with individual users longitudinally. Nevertheless, the study did use other quantitative measures (ie, pop-up questionnaires) to collect user input. Moreover, as the study was part of a larger project, it is complemented by another preregistered study that uses qualitative measures [38]. A second limitation is the possibility of a self-selection bias. When participants voluntarily participate in a study, they are likely to have similar attitudes and traits, compromising sample diversity [64]. On the one hand, this was not a crucial point for this study, as its goal was to target a very specific population, namely, German speakers who have a special interest in supporting children with LDs. On the other hand, this limits the findings’ external validity. Therefore, further similar evaluations targeting different populations are needed, particularly for the interpretation of the RE-AIM adoption and reach dimensions. A third limitation is the smaller number of participants (n=160) used to evaluate the adoption dimension, likely due to the smaller number of participants completing the help system process and answering the respective questionnaire. While this limits this dimension’s generalizability, the other dimensions were measured with larger sample sizes (eg, N=37,133 for implementation), contributing to the current study’s strength. A fourth limitation is that the platform’s impact on children’s academic outcomes was not measured (ie, the RE-AIM effectiveness dimension). A recent systematic review has shown that psychoeducational LD interventions targeting adults, namely parents, are associated with improvements in children’s academic, behavioral, and social outcomes [65]. Thus, a future study, with a different methodological and analytical focus, could assess effectiveness by recruiting families and conducting premeasurements and postmeasurements.

While the influx of digital health brings many benefits, it also increases the risk of misinformation and misconceptions. Digital health tools developed by professionals and academic institutions can potentially combat these risks, but they must be rigorously assessed. This study assessed the LONDI platform, based on the following 4 dimensions of the RE-AIM framework: reach, adoption, implementation, and maintenance. Results from both web analytics and UX measurements indicated a positive public health contribution in all but the implementation dimension. Specifically, users did not spend enough time on pages to read their content and did not engage with the platform’s chatbot as expected. Standard benchmarks for engagement with digital health chatbots are needed. Further research is needed to verify if reading times depend on whether users are new or returning, if there is a ceiling effect for time spent on pages, and if a more complex chatbot will lead to more engagement. As digital health continues to expand, more assessment studies are essential to ensure these tools perform as intended.