Digital mental health tools continue to have a critical role in advancing the delivery of mental health care in clinical care settings [1-4]. In 2020, the World Health Organization released a global strategy on digital health innovation [5], which advocated for the opportunities and need to advance the implementation of digital health strategies and people-centric health systems fueled by digital health tools. In the United States, the American Medical Association [6] found that physician adoption of telehealth during the COVID-19 pandemic grew 5-fold, and the use of remote monitoring tools doubled due to improvements in patient outcomes and work efficiency.

As such, many organizations have advocated for the use of digital health tools in mental health care delivery [7-9]; for example, the American Psychiatric Association has released a set of tool kits designed to support telepsychiatry and clinician recommendation of digital tools in the clinical setting [10], while the Mental Health Commission of Canada has released a number of tool kits to support adoption within Canada, including one for implementing digital mental health innovations [4]. There are also some frameworks for characterizing various digital mental health tools and use cases where digital tools can be helpful to support care [4]. Finally, the National Health Service in England has released a mental health digital playbook that focuses on outlining clinical care pathways that embed the use of digital mental health tools, as well as the associated governance, policies and change management [11]. This playbook, among others, has led to the uptake of many digital mental health tools across health systems, including Big White Wall (now known as Togetherall) in Canada and the United Kingdom [12-14] and the Digital Opportunities for Outcomes in Recovery Services program in the United States [15-19].

However, there continues to be limited evidence on the outcomes and impact of digital mental health tools within clinical care settings in real-world environments [20]. A recent systematic review synthesized findings from 19 trials on digital mental health apps and determined that there is inconclusive evidence to suggest that these tools can be recommended as stand-alone interventions [21]. Many challenges have hindered the effective adoption and utility of the tools. While some are related to user-centered design and the adaptation of evidence-based principles and content within the platform, an emerging area of concern is suboptimal engagement with the tool by patients and families [22]. Several studies have found that continued use of the tool rapidly decreased after initial use [22-24]. The results of a systematic review that looked at user engagement for 7 apps for depression and anxiety showed that <42% of users stayed engaged and continued to use the tool beyond 4 weeks [25]. To ensure that the expected benefits are realized for the end user, it is essential that there is sufficient user engagement where appropriate and necessary.

User engagement has been a growing area of interest over the last few years, given the proliferation of tools and the recognition of this gap in the 2010s [22]. As a result, there have been numerous conceptualizations and characterizations of user engagement; for example, in 2008, O’Brien and Toms [26] characterized user engagement as a process that involves engagement and disengagement, and Perski et al [27] developed an integrative definition: “(1) the extent (e.g. amount, frequency, duration, depth) of usage and (2) a subjective experience characterised by attention, interest and affect.” Some work has also been conducted to characterize user engagement and to help visualize user engagement through use log data, Pham et al [28] developed a framework of metrics, while Pham et al [29] built an analytics platform. Likewise, Yaeger et al [30] looked at factors related to user engagement with a trauma recovery eHealth intervention using the Health Action Process Approach, while MacPhail et al [31] used the Health Action Process Approach to explore health behavior engagement in individuals with type 2 diabetes mellitus. Several researchers have since attempted to uncover factors related to user engagement in the context of facilitators and barriers [32], as well as neuropsychological [33] and persuasive design [34] frameworks. This process has resulted in a myriad of factors, from gamification and technical issues to personalization [32-34], alongside studies evaluating the impact of these interventions on boosting engagement [35].

However, to date, there have been very limited discussions on the factors that influence user engagement with digital tools used specifically in clinical care models and settings. While the majority of tools being developed are focused on self-help in the community [21], there is growing demand and interest from the clinical community in implementing digital tools in clinical care; for example, the Stepped Care Model 2.0 offered by the Mental Health Commission of Canada highlights how tools can be used to augment care being delivered for individuals across various care levels [36]. Given that the integration of digital tools requires careful consideration of the environment and the broader health system, there is a timely need to look at the factors relevant to user engagement with digital mental health tools used specifically in clinical care pathways and delivery.

One approach to addressing this issue is the application of a sociotechnical framework, which allows for the characterization and examination of complex environments to support the adoption and use of digital tools in clinical care settings [37]. In particular, it encourages the researcher to look at the interactions across components at the microlevel (eg, individual), mesolevel (eg, organizational), and macrolevel (eg, health system). The majority of digital health research has focused on factors related to end users or the tool itself [24]; as a result, it has yielded limited value in terms of understanding user engagement within the complex clinical environment. Applying a sociotechnical framework can help highlight the processes and workflows of the clinical environment as well as the broader policies of the organization and health system [38,39].

Frameworks that have been used to look at innovations in health care include the nonadoption, abandonment, scale-up, spread, and sustainability framework [40] and the sociotechnical framework developed by Singh and Sittig [41]. Greenhalgh et al [40] developed the nonadoption, abandonment, scale-up, spread, and sustainability framework, which focuses on examining features across the condition, technology, adopters, and organization, among others [40]. Similarly, the sociotechnical framework developed by Singh and Sittig [41] outlines 8 components that are focused across the micro-, meso-, and macrolevel factors. These eight components include (1) hardware and software computing infrastructure; (2) clinical content; (3) human-computer interface; (4) people; (5) workflow and communication; (6) internal organizational policies, procedures, and culture; (7) external rules, regulations, and pressures; and (8) system measurement and monitoring. Both frameworks have been used to examine various clinical innovations such as virtual care [38] and issues related to ransomware attacks [42]. In particular, they have been used to look at factors influencing the adoption of artificial intelligence in Canadian health care [43] and clinical handoff tools [44].

In this regard, this study aims to develop a comprehensive understanding of the factors related to user engagement with digital mental health tools in clinical care settings through the use of the sociotechnical framework developed by Singh and Sittig [41]. Obtaining a snapshot of the current evidence can help identify the current gaps in literature and inform the development of a comprehensive framework for assessing user engagement with digital mental health tools in clinical care contexts.

To identify the factors that influence user engagement with digital mental health tools in clinical care settings, we used a scoping review approach [45]. Given the exploratory nature and understanding of the factors related to user engagement, this approach was considered appropriate to identify a preliminary set of factors for further exploration [42]. The approach outlined by Arksey and O’Malley [45] and later refined by Levac et al [46] and Peters et al [47] was used. One of the authors (SK), a patient partner, was engaged in the development, implementation, and analysis of this scoping review.

The research questions (RQs) of the scoping review are as follows:

We used the sociotechnical framework developed by Sittig and Singh [41] to guide the synthesis of factors that influence user engagement with digital mental health tools in clinical care environments. This ensured a comprehensive overview of the factors across individual, organizational, and health system levels.

To identify articles on user engagement with digital mental health tools in clinical care contexts, a systematic search strategy was developed based on previous search strategies [32,48]. Relevant Medical Subject Headings (MeSH) terms and keywords related to user engagement, mental health, and digital health tools were applied across databases (Multimedia Appendix 1). The search was conducted on 5 databases—MEDLINE, Embase, PsycINFO, CINAHL, and Web of Science—in January 2022 without restrictions on date or study type. These databases are popular among health sciences researchers and were expected to index most of the literature published in this field. The search strategy was first developed in MEDLINE and was adapted to other databases. A research librarian was also consulted in the refinement of the search strategy. The search strategy was validated by confirming whether previously identified relevant articles (eg, the study by Hoffman et al [17]) were included in the search. The search was updated in October 2023 using the same strategy and approach.

The inclusion and exclusion criteria for the scoping review are outlined in Textbox 1. Eligible studies must examine a digital health tool (eg, mobile app, patient portal, or wearable) that primarily addresses a mental health or addiction issue. To ensure relevance to clinical care, the tool must include at least 1 patient-facing component (eg, app) and 1 clinician-facing component (eg, dashboard) as specified in the Mental Health Commission of Canada Toolkit for E-Mental Health Implementation [4]. Studies must explore the concept of user engagement as an objective, following the aforementioned definition by Perski et al [27]. Articles published in languages other than English were excluded for feasibility. Systematic and literature reviews were excluded, but their reference lists were examined. Non–peer-reviewed article types, such as theses and conference presentations, were also excluded.

Study selection was conducted by the first author (BL) in duplicate with 2 doctoral students (KD and RC) in health informatics using Covidence (Veritas Health Innovation Ltd). Deduplication was performed by identifying records with identical titles and publication years, with verification by a member of the project team (BL). Screening was carried out in 2 stages: an initial title and abstract review, followed by full-text screening of studies meeting the inclusion criteria. To ensure consistency, a pilot screening (n=100) of titles and abstracts was conducted with each of the two doctoral students (KD and RC). Any discrepancies were discussed and resolved by the 3 reviewers, and Cohen κ was used to assess interrater reliability [49]. A Cohen κ value of >0.70 was achieved in the pilot, after which we proceeded with the screening process. A similar process was followed for full-text screening, in that small pilot rounds were conducted by 2 doctoral students, and any discrepancies were discussed and resolved by the 3 reviewers.

The elements extracted for each RQ are outlined in Textbox 2. The article type, year of publication, and study objective were collected to understand the characteristics of the included articles. For RQ1, relevant information about the digital mental health tool, such as technology type, objective, and main features, were extracted. For RQ2—identifying the characteristics of the population—we collected information about the demographic characteristics of the users and access requirements for the tool. For RQ3, factors relevant to user engagement with digital mental health tools in clinical care contexts were extracted. Finally, for RQ4, information about the objective and the digital health delivery model was collected. KD and RC conducted a pilot extraction of 5 to 10 articles, and the extracted data were compared. On the basis of the feedback, the extraction table was iteratively refined (eg, by adding relevant elements).

Both quantitative and qualitative approaches were used to summarize and analyze the extracted data. Descriptive statistics (eg, mean and median) were used to analyze the characteristics of included studies. Descriptive statistics were also used to analyze article and population characteristics for RQ1 and RQ2. A thematic analysis [50] was conducted to categorize the factors from the literature that influence user engagement with digital mental health tools (RQ3) using the sociotechnical framework developed by Sittig and Singh [41]. A thematic analysis was also conducted to characterize the clinical digital health programs that were identified (RQ4). The findings were reported following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist [51] (Multimedia Appendix 2).

Study selection results were reported using the PRISMA-ScR diagram [51] (Figure 1). After deduplication, the search strategy yielded 11,503 records for title and abstract screening. Of the 417 articles reviewed for eligibility, 136 (32.6%) were included for analysis. Table 1 summarizes the characteristics of the included articles, and Multimedia Appendix 3 [52-177] provides details of the included articles.

In terms of the year of publication, 59.6% (81) of the 136 articles were published within the last 5 years. Publications on this topic originated from 18 countries. The top 4 countries were the United States (n=47, 34.6%), the United Kingdom (n=23, 16.9%), Australia (n=11, 8.1%), and Canada (n=11, 8.1%). A wide variety of study designs were used by the included articles, including cohort studies (n=35, 25.7%), feasibility or pilot studies (n=14, 10.3%), qualitative studies (n=43, 31.6%), and randomized controlled trials (19/136, 14%). With regard to how user engagement was examined, the majority of articles focused on user experience and interest (qualitative; 95/136, 69.9%) as opposed to the degree of use (quantitative; 55/136, 40.4%).

Of the 136 included articles, 113 (83.1%) disclosed the product or name the tool that was studied (Table 2). Commonly discussed tools in the included articles were FOCUS [60,68,96,113,117] and Apps4Intelligence [116,185], mobile apps for people with schizophrenia; Intellicare, a platform that focuses on navigation and recommendation of digital tools [84,106]; and Horyzons, a social therapy platform for first-episode psychosis [133,134]. In terms of tool type, the majority were websites (62/136, 45.6%) and mobile apps (49/136, 36%). Notably, 6 (10%) of the 62 websites were patient portals from health care organizations [132,158,159,171,173]. From a client-facing perspective, these digital tools offered a wide range of functionalities (Table 2). Specifically, among computerized interventions, resources, and applications (111/136, 81.6%), some examples include tools that were focused on delivering cognitive behavioral therapy [57,59,63-65,74,79,80,89,91,98,102,111,128,184] and educational workbooks or resources [54,56,66,123,125] for patients to complete. Regarding clinician involvement in the support and use of the tool, most provided support in a coaching (49/136, 36%) or comprehensive (53/136, 39%) manner [4]; for example, for the FOCUS app and Intellicare platform, clinicians would guide users on the use of the tool and develop a plan around how the tool should be used throughout the care journey [84,106,117]. Other tools, such as the BRAVE platform, required clinicians to use it in a more comprehensive manner to review results and to discuss and plan next steps in care through and with the platform [118].

With regard to the population of end users (Table 3), the majority of articles (101/136, 74.3%) examined tools that were developed for the adult population. Only a few articles (12/136, 8.8%) had a focus on younger adults (aged <18 years), while the study by Sheeran et al [55] evaluated a depression management program for individuals aged >65 years receiving home care services. Moreover, the target conditions of the tools varied significantly, with a little more than one-third of the articles (47/136, 34.6%) examining tools that supported >1 type of mental health condition. Regarding condition-specific tools, the most frequent conditions examined were depression (26/136, 19.1%) and schizophrenia spectrum and related psychotic disorders (20/136, 14.7%). Finally, regarding the duration of participation, 52.9% (72/136) of the studies reported how long participants were asked to use the tool. Of these 72 studies, 42 (58%) asked participants to use the tool for <3 months.

Some of the studies (20/136, 14.8%) provided a brief description of how digital mental health tools can be embedded within clinical programs [53,55,66,75,104,109,113,120,121,129, 143,146,154,155,160,163,168,177,178,182]; for example, Kemmeren et al [109] outlined a clinical workflow specifying when these tools should be introduced and used with patients, as well as the intended duration and use of the various modules within the blended cognitive behavioral therapy tool. In addition, these articles described training for clinicians on using the tool with patients and provided guidance on how to navigate and support patients in its use [55,104]. For the FOCUS app [113], information was provided on the frequency and approach for checking in with clients regarding the use of the digital tool.

To our knowledge, this is one of the first reviews to focus on identifying a preliminary, comprehensive framework of factors that influence user engagement with digital mental health tools in clinical care settings [24]. In this review, 26 factors were identified from 136 articles that spanned across the components of the sociotechnical framework developed by Sittig and Singh [41]. These factors illustrate how technology, relevant stakeholders, and environment can influence user engagement with digital mental health tools. Moreover, based on the studies that examined the clinical programs that embed digital mental health tools, there is increasing discussion on how these tools can be delivered in real-world environments.

As there are a growing number of studies looking at ways to address the ongoing challenges of user engagement [194-196], this work has contributed to a better understanding of the dynamics of, and contributors to, effective user engagement in a clinical environment across individual, organizational, and health system levels [37]. Given that the focus of this scoping review was on digital mental health tools in clinical care environments, this work will help inform the development of implementation approaches and strategies for ensuring effective engagement and integration of digital mental health tools in clinical care settings and models of care. In addition, as digital tools continue to proliferate in the clinical environment, engagement with these tools has become an extension of how patients engage with clinicians, receive care, and navigate the clinical environment [24,33,197]. Hence, engagement with digital tools can be a key factor in the overall success of treatment. The specific impact of this work on several key areas of user engagement is discussed in the following paragraphs.

First, as outlined in the Introduction section, a number of existing frameworks have explored the factors that influence user engagement with digital tools [27,32]. In this work, while many of the identified factors are consistent with the findings of previous reviews, the application of the sociotechnical framework developed by Sittig and Singh [41] was useful in 2 ways. Given the overarching goal of developing interventions to enhance effective user engagement, the sociotechnical framework helped to further break down the factors related to user engagement by focusing on digital mental health tools in clinical care settings; for example, technology-related factors were further characterized into factors that pertain to the content, the technology itself, and the human-computer interface [41]. Second, rather than providing a “laundry list” of factors within the categories, the sociotechnical framework helped to clarify and conceptualize the interplay of factors across its components. Thus, this work extends the broad scope by providing more specificity on how some of the factors would interact with each other through a sociotechnical lens. However, there were also challenges in applying the sociotechnical framework. Given that many of the factors are closely related, it was difficult to articulate the interconnectedness and interactions across the various components of the framework. Future work should further explore these nuances, including how these factors are connected and how factors specific to different end users (eg, patients and clinicians) interact.

The findings from this work also provide more insights into factors specific to the discipline and the implementation setting of these tools. Borghouts et al [32] conducted a similar scoping review but focused more broadly on digital mental health tools and identified 16 factors that span across the user, program, technology, and environment. While many of these factors align with those in this review, the relatively narrower focus of our review uncovered 2 additional facets specific to tools integrated within the clinical environment. First, the importance of support from clinicians was identified as a critical component. While many studies have looked at how clinician-guided approaches can help improve engagement with digital mental health tools [198], there remains a lack of guidance and best practices on how best to support patients throughout their care journey [138]. As such, it may be useful to develop best practices that is informed by a customer experience approach through user experience perspectives [199].

Moreover, as the tool will be embedded in the health care organization, it is critical to consider the workflows and internal processes. Establishing clear guidelines on the accountability and role of each member of the circle of care in the use of digital tools will ensure that end users understand what constitutes appropriate, safe, and effective use [20]; for example, for tools that are not actively monitored after hours, it is critical to reiterate through policy that patients and clinicians understand that the tool is not to be used during emergency crises [108]. Another important aspect of workflow and organizational policies is ensuring protected time for clinicians to use digital tools with patients. In the Canadian context, where physicians rely on a fee-for-service model, there is a need to establish remuneration policies so that clinicians can be compensated for the time and administrative burden associated with the use of these tools [115]. With clinician burnout related to digital tool use becoming a tremendous challenge in the last few years [200-204], there is a need to consider the impact of these tools on documentation and administrative burden. Thus, the findings from this review highlight the need to address engagement not only at the patient level but also at the clinician and system levels. In particular, investing in factors that support clinicians and the broader health system is equally important to ensure that all roles and key players can engage with the tool in a meaningful way, which can potentially be more cost-effective than focusing solely on patient-level factors [198,205,206]. Future work should capture how various organizations address the factors at these levels.

Finally, the findings from this work have emerging implications for mental health clinicians, administrators, developers, and researchers. It is also important to note that while these factors were developed from mental health literature, it is likely that many of these factors are also relevant for digital health in other clinical areas [197,198]. For clinicians, it would be useful to consider these factors in formulating a plan and encouraging patients in the use of digital mental health tools in clinical care delivery [207]. Developers can consider these factors throughout the software development lifecycle and consider how these factors should be integrated into the features and development (eg, push reminders) of digital mental health tools. Educators and administrators may consider leveraging these factors to develop appropriate guidelines and curricula to enhance the capacity and readiness of current and future health care professionals to use these tools in their practice [208]. Similarly, administrators should consider these findings in developing a conducive environment for enhancing engagement with digital health tools in clinical practice. Finally, researchers can build on these findings to develop guidance (eg, tool kits) for designing tools keeping user engagement in mind. In addition, interventional studies can examine the impact of these factors on user engagement with digital mental health tools in clinical care settings.

Several limitations should be considered when interpreting and applying the preliminary framework of factors from this literature review. First, given the nature of scoping reviews, the quality of the included articles was not assessed, and the literature search is not assumed to be exhaustive [46,51]. Second, this work primarily focused broadly on the patient population and, given the diverse characteristics of the populations included in this review, it was difficult to identify nuances and differences in impact across various sociodemographic and clinical populations.

In addition, no studies were identified that looked at user engagement with digital mental health tools from the perspectives of caregivers and family members, despite their importance in mental health care [209]. As studies such as that by Simões de Almeida [210] have shown, exploring the prominence of various factors among different populations may be of interest. Finally, the preliminary framework of factors was developed based on the academic literature and were not validated with subject matter experts. This will be addressed in a subsequent study.

Several future directions have been identified to further strengthen and validate the framework of factors for digital mental health tools in clinical care settings. First, given the large number of factors identified from the included articles, there is a strong need to validate and identify the impact of these factors on user engagement. A Delphi study will be conducted to validate and identify any factors not found in this literature review [211]. Second, as the majority of the included studies (49/136, 36.1%) focused on the use of mobile apps, there was limited insight into the use of emerging tools such as patient portals and wearables. Finally, it would be imperative to explore how these factors influence user engagement in different clinical contexts.

A total of 26 factors influencing user engagement with digital mental health tools within clinical care settings were identified from the academic literature. These factors spanned across the categories of the sociotechnical framework, highlighting the complexity and interconnectedness of the factors in influencing user engagement. Future work should focus on identifying the factors considered essential for influencing user engagement and conducting exploratory studies to examine the differential impact of these factors on user engagement.