Digital health interventions (DHIs) are revolutionizing health care delivery by providing personalized, private, and instantaneous solutions to end users. An umbrella term, a DHI is a mobile- or web-based digital solution that facilitates a diverse array of health interventions—be they emotional, decisional, or behavioral. A DHI is designed for a complex set of end users—those who use the intervention, including patients, caregivers, and health care professionals [1]. With the ever-increasing ubiquity of mobile health (mHealth) technologies, cost-effective scalable DHIs are emerging almost daily [2]. These solutions range from self-monitoring to diagnostic, educational, and behavior change–based interventions. At the heart of a DHI is the contextual adaptation [3] around the needs of end users, those who use the technology directly to impact their health [4], such as a patient with diabetes monitoring blood sugar on an mHealth app. This is an exciting yet challenging shift in health care design. It promises more autonomy to the end user through a bottom-up [5] design approach; however, it ruptures the top-down [6] traditional health care approach that positions experts as the central designers of interventions. This evolution marks a shift from visiting health facilities for health services toward the ubiquitous digital participation of end users and health experts. This has produced a new era of health literacy built around diagnostics, services, and self-help. This reciprocal, empowering approach is creating a more mutual understanding [7], evolving the dynamic [8] among health care professionals, engineers, designers, and end users in DHI co-design, ultimately shifting the balance of power toward the patient. Nonetheless, the promise of DHIs is tempered by the fact that 26% of apps are discarded after first use [9]. To address this concern, numerous design approaches such as user-centered and human-centered design, among others, are being used to varying degrees of success [10]. Each approach seeks to incorporate end users, from the primary stages of ideation to the final testing stages of production, gathered around multidisciplinary teams of digital and health care professionals. It is within this focus that we have examined the prominent plus points (ie, advancements), pain points (ie, challenges), and system-wide gaps to incorporate end users in co-design, issues that cut across the entire range of design approaches used in digital health at present.

Despite today’s digital sphere increasingly being one of digital natives, the path toward a fluid, organic digital health design space has been fraught with challenges. From a digital perspective, rapid Agile development methods evolve to garner end-user satisfaction in an evolving form of iterative qualitative evaluation. From a health care perspective, slow, safe expert-led studies produce large quantitative datasets toward validating patient involvement [11]. Digital health, in its nascent stages, sits somewhat uncomfortably between these two domains. From the macrolevel of policy down to the microlevel of co-design workshops, the very language of collaboration may differ. Hybridizing these 2 contrarian methodologies is challenging. The concept of building validation around the user or patient varies, depending on the given perspective. These challenges, along with others, have led to a multiplicity of design approaches, a disparity in conceptualizing the end user, and competing agendas [6,7,12,13] among DHI stakeholders that muddy the waters of digital health design. In this regard, we were inspired to conduct this systematic review on the basis of the initial analysis of our previous study titled, “The Challenges Toward Real-world Implementation of Digital Health Design Approaches” [10]. In our primary study, we found that irrespective of the domain, design methodology, or user definition (Multimedia Appendix 1), there are common pain points (ie, repeated points of conflict or displeasure resulting in a poor user experience [UX]) emerging that impact digital health at the domain level.

In addition, numerous studies [3,6,11,14,15] have noted the challenge to design frameworks that effectively define and implement “co-design” or “co-leadership” around end-user involvement in DHI design. On one hand, health stakeholders may lament the lack of alignment of UX design to health care delivery [9]; on the other hand, digital stakeholders may insist that user-centered design (UCD) is essential toward acceptability [16]. In both cases, each stakeholder is viewing efficacy through a different lens. This cuts to the heart of the matter of co-designing DHIs with end users; the overarching philosophy and methods used are starkly different. Therefore, there is a need to better understand the common pain points in this collaboration, which in turn may lead to a better perspective on why, when, and how to incorporate end users in the design of DHIs.

This study seeks to compile the core challenges to incorporate end users in the design of DHIs over the last decade. We sought to thematically analyze current approaches, to analyze evidence of pain points that presented a challenge or conflict in the incorporation of end users in the co-design process, and to explore gaps and opportunities toward a more organic co-design approach in digital health. This includes a broad stroke of studies, including quantitative, qualitative, mixed methods, and research through design. It features the perspectives of health care, psychology, engineering, development, and design in the production of mHealth, eHealth, and extended reality DHIs. This intentionally broad approach is reflective of the nascency of digital health as a field and the open-mindedness of this study, such that the divergence and convergence of methodologies and tools is viewed as a prosperous and necessary growing pain toward a more organic digital health approach to end users.

It is hoped that through critical analysis of these advancements, challenges, and gaps, synthesized findings from the following questions lead to an improved approach toward incorporating end users in DHI co-design:

This systematic review followed an a priori published protocol with detailed methods registered with PROSPERO (CRD42021238164) [17]. The review followed the Cochrane Handbook for Systematic Reviews [18] and was reported using the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols) guidelines [19].

We followed six stages in this systematic review: (1) literature search, (2) article selection, (3) data extraction, (4) quality appraisal, (5) data analysis, and (6) data synthesis.

Considering the complexity of DHIs, we used the PerSPEcTiF [20] guidelines for intervention (Table 1). We selected it due to its suitability for qualitative synthesis in the health care domain.

In line with this framework, studies were included in this review that (1) address research on DHIs; (2) focus on interaction and co-design with end users; (3) explain results such that uptake, effectiveness, satisfaction, and health outcomes are discernible, positively or negatively; and (4) describe actionable procedures for better DHI co-design.

We systematically searched 6 electronic databases: PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Institute of Electrical and Electronics Engineers Xplore, and Scopus. We selected these databases according to preliminary searches and consultations with experts and librarians in this field. Keywords related to DHIs from preliminary searches were compiled and refined. We adapted the search strategy as needed to return a breadth of papers without retrieving an unmanageably large number of irrelevant articles.

A list of the search terms used in this review were grouped into 3 clusters (Table 2). In addition, we have included all search strings for replicability (Multimedia Appendix 2).

Cluster 1 includes DHIs (owing to behavior change in health care resulting from DHIs). Cluster 2 includes UX (owing to engineering, development, and design in digital health). Cluster 3 includes methodologies and frameworks (owing to digital project facilitation and to health care intervention design and policy).

The inclusion and exclusion criteria were based on the peer-reviewed studies outlined in Textbox 1.

Due to the rapid innovation of digital health, only studies in the English language from January 1, 2015, to May 31, 2024, were included. Broadly put, the inclusion criteria included any population group that involved the critical analysis of end-user incorporation (ie, inclusive of terms such as patient, person, and human) in the co-design of a DHI. Studies that did not meet this criteria were excluded, including those that included end users but did not provide any critical analysis of the process; reviews and commentaries due to them being nonobservational or nonexperimental; single-user studies due to the inability to generalize; randomized controlled trials (RCTs) due to a lack of focus on the end-user integration in the design of a DHI; and nonhuman studies, which are impertinent to the design of a DHI.

All articles identified and selected from database searches were stored in the reference management software Paperpile (Paperpile LLC), which was used to eliminate duplicates and to tag and organize the research findings. The initial screening process involved 2 independent reviewers (AD and either IR or NB) who screened titles and abstracts of all studies for possible inclusion using an Excel (Microsoft Corp) spreadsheet (Multimedia Appendices 3 and 4) to ensure no studies met the exclusion criteria. After filtering results on the basis of the possible inclusion criteria, the second stage involved a full-text review of the remaining articles, which was conducted by 3 independent reviewers (AD and IR or AS). Upon scrutiny of the full-text selections, reviewers (AD and IR or AS) removed items that did not meet the inclusion criteria. In the third and final stage, reviewers (AD and IR or AS) met to cross-reference eligibility of the final inclusion list, as well as to settle any conflicts. This cross-reference involved the individual review of all items to white list them per the inclusion criteria. For items in conflict, reviewers (AD and IR or AS) simultaneously scanned items side by side to determine whether the items met the inclusion criteria. Since this was the third round of filtering, the majority of items met the inclusion criteria, so the determining factor in conflicts was to collectively review each item to see whether the studies actively involved end users in the design of the DHI and whether critical analysis of this was present in the results or discussion sections of the respective study. This final step removed any subjectivity and resolved conflict.

A data extraction template was created by one of the authors (AD) that provided a record of the study characteristics. The categorization of themes and subthemes emerged organically from the findings (Multimedia Appendix 5). The collection, extraction, and subsequent analysis and synthesis of data is outlined in Table 3.

Raw content (ie, quotes and reflective analysis) was extracted and collected in a research document (Multimedia Appendix 6). Data were extracted and analyzed relative to the goals of the research questions, following guidance from the Cochrane Handbook for Systematic Reviews of Interventions [18]. These data are synthesized and presented in narrative form in the Results section. Full characteristics of all included end-user studies are available in Multimedia Appendix 7.

The search strategy identified 13,961 studies. After removing duplicates (n=794), a total of 13,167 (94%) studies were screened for title and abstract. After the initial title and abstract screening phase, 12,678 (91%) studies were excluded, leaving 489 (4%) potentially relevant papers. In the second full-text screening phase, 318 (2%) studies were excluded. This resulted in the inclusion of 171 (1%) relevant studies in this systematic review. Reasons for exclusion included a lack of critical analysis of end-user involvement in digital health co-design and nonprimary research (eg, reviews and op-eds, RCTs, and single-user case studies). A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram (Figure 1) was used to record the details of the screening and selection process so that the study can be reproduced.

Before addressing our research questions, we present a record of characteristics detailing the diversity of digital health studies retrieved (Table 4).

Owing to the diversity of digital health, the 171 studies identified span 5 different research approaches. Most approaches were qualitative (107/171, 62.6%), followed by mixed methods (43/171, 25.1%), indicative of the social and emotional influence on DHIs. These studies were collected from 52 different journals across multiple research domains, with JMIR accounting for the majority (94/171, 55%).

The DHIs spanned 8 different solution types with mHealth (97/171, 56.7%) representing the majority. This development is in line with the growing dominance of mobile digital devices with up to 92% of internet traffic at present being mobile based [21].

These solutions were used in 15 different populations with patient-focused populations being the most prominent (91/171, 53.2%). There was considerable variation in the number of end users involved in a DHI co-design with 6 to10 (44/171, 25.7%) end users being the most prominent, followed by 11 to15 (28/171, 16.4%), 21 to 30 (25/171, 14.6%), and 1 to 5 (21/171, 12.3%) end users. Only 9 studies (5.3%) involved >50 end users.

As a guiding design methodology, UCD (84/171, 49.1%) was most prominent, followed by the broader approach of participatory design (58/171, 33.9%), then human-centered design (18/171, 10.5%). Other approaches such as patient-centered design, patient-based approach, and patient-led design were used sparingly.

Finally, in addition to tracking the design methodology, we tracked the project facilitation method used. The vast majority of studies did not specify a project facilitation method (133/171, 77.8%) that was used. Of those that did, Agile was used the most (24/171, 14%), followed by design thinking (10/171, 5.8%). Scrum, a subset of Agile, was next (9/171, 5.3%), while none specified other common industry methods such as Kanban, Extreme Programming, or Waterfall. This is particularly interesting when comparing how digital health prototypes are facilitated in academic research environments versus typical industry methods.

We present the aforementioned study characteristics to illustrate the diversity of investigating the incorporation of end users in co-design. As discussed, this rigorous approach sought to incorporate as many vantage points as possible to identify common advancements, challenges, and opportunities for improvement in the domain. A thematic analysis of the plus points (ie, advantages), pain points (ie, disadvantages), and challenges to incorporate end users in the co-design of DHIs (Figure 2) as addressed in the 3 research questions is presented later.

A brief definition of the resultant themes and subthemes is presented in Table 5. This is followed by an explanation of the results relevant to each of the 3 research questions in this study.

After assessing plus points and pain points in the first 2 research questions, our final question sought to understand the limitations and gaps that are challenging the advancement of end-user incorporation in the design of DHIs.

This review examined the advancements and challenges of co-designing DHIs with end users, highlighting key gaps and limitations that cut through a wide range of design methodologies. The burgeoning growth of the digital health industry is a major advancement in health care delivery, one that is seeing exciting developments and major investment. Participatory co-design approaches that promote collaboration and ideation around the end users, often in situ, are promoting positive behavior change by seeking to absorb a diverse stakeholder group into a more fluid digital health ethos. These iterative design approaches are often centered around UCD methodologies and other similar approaches (eg, human-, person-, and patient- centered approaches) that emphasizes a more holistic empathetic approach, positioning the end users as coequal in the design phase (Multimedia Appendix 1). Co-design is an exploratory process that experiences divergence and convergence. Through this disruptive process, the collective brain of a diverse stakeholder group is sought out in order to resolve conflicts and to amalgamate design needs. It is hoped that through these approaches, a common language and improved cultural context may form, leading to a better co-design outcome, one that provides both better efficacy and UX.

These promising developments are tempered by the sheer complexity of combining digital and health approaches into one funneled, organic design around the end user. The rapidity of co-design workshops, with their limited cohort size, has raised many questions about the validity and reliability of findings. While an RCT is the gold standard in health care validation, its timeframe often outlives the development of a digital solution [11]. In response to this, researchers are experimenting within the nascent stages of digital health design, porting a variety of research methodologies (ie, observational and experimental, whether qualitative or quantitative) and design methods (ie, Agile and behavior change, among others), mixing and matching concepts seeking out how best to validate DHIs with end users.

Exactly how this incorporation occurs is the flash point of conflict. Digital design perspectives that are heavily borrowed from the mainstream digital industry are rooted in UX approaches. Health design perspectives focus on long-term safety and efficacy (Multimedia Appendix 8). While digital approaches are suited to rapid innovation, pivoting changes in real-time, health approaches are rooted in rigor and safety, implementing change slowly with longitudinal data. The variance between qualitative and quantitative approaches, UX and behavior change, and efficacy and usability accentuates the downstream challenges that occur in ideation, testing, and implementation of DHIs. This culture clash calls for pragmatism. Hybridizing digital and health approaches such that the demarcation between them eventually dissolves is the apex of DHI co-design. To better achieve this, the digital health domain must become comfortable with the uncomfortable, allowing the tensions of hierarchical health approaches to collide with flat-management digital approaches. Finding common ground between complaints of digital UX approaches that oversimplify health concerns and health approaches that are bureaucratic and time intensive will be paramount in creating an equilibrium that blends competing agendas.

These 5 recommendations are responses to the challenges identified in this review, with an eye toward the emerging era of digital health citizenry, in which techno-sociality and bio-sociality are undemarcating the boundaries of digital health [187]. This “health citizen” agenda envisions digital health as more of a public good that provides tools, opportunities, responsibilities, and rights [183]. This transformation is built upon the prosumerist nature of today’s technologies, such that any given DHI may be an end-user experience of patients, clinicians, caregivers, therapists, and passive observers collectively, facilitated through algorithmically driven big data solutions (ie, artificial intelligence and machine learning) that are intuitive, customizable, and adaptable. The delivery of health care is increasingly blending government (eg, health records and policy), medical (eg, public and private sector solutions), and citizens (eg, end users) into one fluid infrastructure. This relationship is becoming increasingly reciprocal, more productive than facilitative in nature [187]. The potential for crowdsourcing data in tomorrow’s era of digital health seems boundless.

Needless to say, this vision needs to be brought to the forefront of the messier, creative stages of co-design workshops. The reliance on small qualitative datasets does not leverage the vastness of crowdsourcing end-user feedback that is equally capable of collecting quantifiable, demographically contextual statistics, while garnering emotional qualitative data in the form of asynchronous digital co-design platforms. If there is one word to define the evolution of digital health moving forward, it may be pragmatism. Leveraging the right tools to incorporate end users in the design of DHIs should mean removing any boundaries to the type of data collected. This means removing any allegiances to particular schools of thought on qualitative or quantitative, observational, or statistical, instead focusing on a pragmatic “patient activation” [187], approach to garner positive involvement from end users at scale. There is no need for competing interests methodologically; a combination of metrics that increases the understanding of digital health users only serves to strengthen the confidence of design solutions, reducing attrition rates and increasing validity and generalizability. The digital health citizen is envisioned not as a user, an expert, or a novice, but as an active participant in the health care ecosystem, one who constantly learns and adapts to their health care needs. It is fluid and proactive rather than static and receptive. Therefore, there is a clear need to move the incorporation of end users in DHI co-design from small participant workshops objective to a dynamic socio-digital platform of ideating, testing, and evaluating in real time, at scale. Looking forward, this approach involves a new outlook that leverages the vision of the digital health citizen methodology, with the pragmatism of Agile tools, and the rigors of health care metrics. Experimenting with processes and tools that expand the range of data collection may better identify end-user types, better triangulate preferences and challenges, and provide stronger integration with health policies and implementation objectives downstream. Refocusing co-design approaches within this lens serves to better use the emerging potential of big data, potentially increasing the user perspectives of all stakeholders across the digital health spectrum, without sacrificing the rapidity and innovation of the design environment.

This systematic review drew from a large swath of studies, using various methodologies and techniques to incorporate end users in DHIs. The strength of this approach was identifying a large number of studies for analysis and highlighting key global issues that digital health stakeholders face, including points of failure and challenges. We used the PerSPEcTiF framework due to its suitability to qualitative studies in health care. However, many of these studies were limited in the number of participants involved, bringing the validity and reliability of findings from these studies into question. The largely qualitative approach to studies selected, with vast variance in data collection approaches, did not allow for sensitivity analysis on the basis of measurement criteria (ie, setting, duration, and demographics).

A limitation also exists in the generalizability of findings toward specific scenarios (ie, mental health and patient aids) in which domain-specific usages weigh heavily on the context of end-user incorporation. We recommend further research to investigate these themes as subsets within specific digital health communities (ie, mental health and serious gaming) In addition, in many cases, studies did not provide perspectives on risk of harm when using approaches to incorporate end users. This issue requires critical analysis itself. With digital health being a hybrid domain, there is no standardized approach to quality assessment, leaving the potential for risk of bias. These factors complicate data extraction and interpretation. This may be improved by using a health-centric qualitative framework in such studies, such as Confidence in the Evidence from Reviews of Qualitative Research [188].

The challenge to incorporate end users in the co-design of DHIs is complicated by a variety of methodological influences on problem definition, testing approaches, and hoped outcomes. The porting of legacy Agile, behavioral, and analytical methods has often resulted in conflicts that weaken conclusions that can be drawn from the co-design. In this study, we sought to draw a line through the common pain points and gaps in the literature that are omnipresent regardless of the design approach. The subsequent recommendations stemming from this systematic review serve as guidelines that may help soothe the path toward the big data environment that is emerging through a socio-technical digital health citizen approach into the mid-21st century.