We performed an exploratory case study to identify which mechanisms enabled Gezonde zorg, Gezonde regio (GzGr) to successfully develop and implement CDSS in Dutch primary care [25]. We define mechanisms in the context of this paper as sequential patterns of actions that explain how change occurs, both between and within organizations in relation to technological interventions. These actions represent how GzGr created the conditions and behaviors needed to cope with challenges that occurred over time in the development and implementation of complex digital health solutions in complex settings. The sequential aspect emphasizes how actions build on previous actions or challenges over time, indicating that their order may be causally related to their final effect. To identify how, why, and when mechanisms were applied, we used qualitative process analysis by tracing actions and decisions over time [26-28]. While the mechanisms captured what drove change, our accompanying narrative illustrated how these processes unfolded in practice. This study was designed and reported in line with the COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist (Checklist 1) [29].

We researched “Healthy care, Healthy region” (in Dutch: Gezonde zorg, Gezonde regio [GzGr]). We identified GzGr as a unique case study via field exploration through researchers’ networks and via internet searches. Case selection was based on the following criteria: (1) successful regional implementation of a CDSS across multiple primary care practices, (2) full integration in primary care workflow, including IT integration; and (3) collaboration involving all relevant stakeholders in primary care.

The GzGr initiative involves a diverse range of organizations: general practitioner (GP) practices, regional primary care organizations, a regional hospital, a health care insurance firm, data processing firms, technology and content development firms (collectively referred to as technology partners), and advisory groups responsible for implementation support and technology evaluation [30]. Figure 2 provides a visual representation of the GzGr structure. GzGr developed multiple rule-based and data-driven CDSS to provide real-time alerts to GPs regarding patient health risks [31]. The steering committee, consisting of executives from key partner organizations, sets the strategic direction and oversees key decisions. The working group, consisting of health care purchasers and innovation managers from the care groups, manages day-to-day operations and the collaboration and advises the steering committee. The editorial committee consists of end users, technology developers, and representatives from the health care insurance firm and is responsible for identifying widely experienced problems and overseeing CDSS development.

We collected multiple sources of data, including interviews and documents, aiming for data triangulation to increase the validity of the data [32]. In total, we recruited and interviewed 15 unique respondents from December 2021 to September 2022 by JDHvW. Interviews were conducted until data saturation was reached, meaning that interviews confirmed findings rather than yielded new findings [33]. Data saturation determined our study size. We interviewed at least one participant from each partner organization involved in GzGr. An overview of participants is given in Multimedia Appendix 1. Our eligibility criteria for participants were their (1) active involvement in the operational development of the CDSS, by working at one of the technology partners, at a general practice or a care group (defined as an organization that coordinates and supports GPs with administrative burden); and/or (2) involvement in the strategic decision-making of GzGr, either through participation in the steering committee or by holding an executive mandate within an organization.

We sampled the respondents using snowball sampling and purposive sampling and used the networks of participants, partner organizations of GzGr, and researchers [34]. JDHvW contacted participants by email through a GzGr contact person, after which additional participants were identified by previously interviewed respondents. Although this approach introduced potential selection bias at first, snowball sampling helped recruit a diverse and representative group. No participants declined participation.

We followed a semistructured interview approach to examine which mechanisms were used by GzGr to ensure progress in development and implementation. We composed an interview topic guide based on the NASSS framework [12]. Topics were based on the following domains: intervention features, the expected value, adopter readiness, the broader context, adaptability over time, and the interplay among domains. We omitted the “condition” domain, as the developed CDSS were designed to apply across multiple clinical conditions. Furthermore, we asked about involved stakeholders, organizational characteristics, and the development process. We explored how GzGr addressed related challenges in these domains to identify underlying mechanisms, which formed the basis for identifying mechanisms.

Interviews were conducted via video call. Each interview lasted between 45 and 60 minutes, and the respondent(s) and JDHvW were present. The interviews were audio recorded and afterward transcribed verbatim. Transcripts were made available to respondents for review upon request. Moreover, we collected internal and external documents of the organizations via the participants and the internet, from September 2023 to January 2024. All documents were stored in Atlas.ti (ATLAS.ti Scientific Software Development GmbH) for analysis.

To identify mechanisms that facilitated overcoming challenges in the development and implementation of CDSS, our data analysis consisted of multiple steps, namely (1) composing a chronological overview; (2) thematic and inductive coding to identify mechanisms; (3) performing a member check; and (4) constructing a conceptualized narrative. We used Atlas.ti 9 to support the coding process. Further elaboration on the coding process and the codebook is given in Multimedia Appendix 2.

This study was conducted in accordance with ethical guidelines for qualitative research involving human participants. Ethical approval was obtained from the Research Ethics Review Committee of the Erasmus School of Health Policy & Management (reference number 21‐014), ensuring compliance with relevant regulations on data protection and participant rights.

Additionally, participants signed an informed consent form before participating in interviews, which stated the goal of the research, the researchers, and the interview; privacy measures (including pseudonymizing data); and confidentiality measures (raw data available upon reasonable request) and included consent for audio recording. No identifying information or images of individual participants are included in this manuscript or its supplementary materials. Participants did not receive any compensation.

In this section, we described the chronological sequence of the development and implementation challenges and actions. Table 2 summarizes the main challenges faced by GzGr and the resulting actions that led to the development and implementation of CDSS. In retrospect, we distinguished 3 periods, each with distinct challenges and focus. The chronological overview in Multimedia Appendix 3 is a more elaborated version of these challenges, actions, and their connection to the mechanism and provides a brief overview of the timeline.

In this first period, we explain how GzGr started their collaboration with a mission, formed a mission-driven coalition of the willing, and established principles for expanding the collaboration.

The GzGr collaboration was initiated by 3 senior influential individuals in health care. One was a senior executive at the largest regional health care insurer, securing long-term funding. The other two were executives from a primary care group and a regional hospital, providing managerial commitment. These individuals were considered influential in the region because they could allocate resources, influence organizational decisions, and mobilize support (financial and people). The aim was to provide more efficient care to maintain regional health care accessibility, with a focus on prevention, the use of routine data, and in line with shared savings funding models. In their view, delivering future-proof care required close collaboration between care organizations and funders to foster joint responsibility for a specific population. As an initial attempt toward operationalizing this mission, a research-focused technology partner explored the feasibility of population-based funding. However, it became evident that this organization lacked expertise in operationalizing the outcomes, posing a challenge for GzGr, leading GzGr to stop the collaboration.

Realizing the long-term mission to aim for more efficient care and involving partnering organizations was also experienced as challenging for GzGr; therefore, they decided to refocus their efforts on a more short-term, actionable mission: enhancing preventive care interventions in GP practices by developing and implementing CDSS. This system would leverage routine care data to support GPs in patient monitoring and the early detection of health risks.

To realize the development, GzGr invited 2 new individuals from organizations to join the collaboration: an innovation manager working at technology partner 1, contributing to clinical content and CDSS development, and a project manager from the regional network and consulting organization. According to respondents, the addition of these 2 organizations was key in translating GzGr’s broad mission into an actionable, concrete CDSS. Together, these 5 individuals brought extensive professional networks, complementary expertise, and executive backing from their organizations:

More importantly, respondents emphasized that these individuals consistently prioritized the shared, collaborative mission. At the same time, they ensured alignment with their own organizational interests, without aiming to maximize individual organizational gain. GzGr referred to the individuals as a “coalition of the willing,” marking the emergence of a mission-driven collaboration within GzGr:

To make progress, GzGr recognized the need for a data infrastructure to support CDSS implementation across GP practices and enable data integration and evaluation. This meant that additional technology partners were needed. Identifying technology partners that aligned with GzGr’s mission and collaborative approach proved challenging. Therefore, GzGr formed principles for organization selection and task allocation: willingness to experiment, previous experience in CDSS development, knowledge of legal and regulatory standards, possession of appropriate certifications, a willingness to work at a cost-plus model (ie, a pricing approach where providers are reimbursed based on actual costs plus a fixed margin), and agreeing upon the shared mission. These principles ensured that new participating organizations aligned with GzGr’s chosen direction, allowing flexibility in how the mission was executed but preserving its overall purpose, which prevented mission drift and delay in progress.

Through technology partner 1’s network, technology partners 2 and 3 joined. Together, GzGr began developing CDSS that could be implemented at scale. One of the first successful pilots was a data-driven CDSS for cardiovascular pharmacotherapy, identifying cost-effective treatment options with equal or better outcomes. Savings were shared between providers and insurers, illustrating an early shared savings model. Figure 3 provides a visual overview of the contributing organizations (squares), their roles, interaction (arrows), and an example of CDSS (circle). Data from multiple national registries are integrated by technology partners (double-headed arrow) and linked to GP electronic health records and the GP (single-headed arrow), where CDSS are embedded to support clinical decisions. Finally, CDSS are evaluated with insurers and academic partners. The figure also includes an example of a cardiovascular risk management rule, showing how data generated by GPs trigger actions for patient follow-up, supported by continuous evaluation and feedback between organizations.

From the outset, CDSS and infrastructure development were co-funded by health care insurers and technology partners, who believed in the potential of CDSS and reimbursed GPs for using CDSS, even in the absence of traditional scientific validation. Gaining internal support within the insurer required ongoing efforts from the individual involved in GzGr from the start, who actively used the shared mission to convince colleagues of the value of GzGr:

To increase the impact of the CDSS, GzGr sought to expand the collaboration with additional GP care groups in the region. However, not all care groups were willing to experiment and change their internal processes to accommodate the CDSS. Therefore, GzGr decided to prioritize GP practices with an “innovative mindset” and willingness to experiment. Additionally, not all collaborations within GzGr endured. A regional hospital withdrew after the departure of the executive initiating GzGr, which respondents linked to a misalignment between individual commitment and organizational priorities.

To conclude, GzGr operationalized their broad mission to enhance preventative care by developing and implementing CDSS. The first mechanism was used: enacting an interorganizational value model, which built on shared value and accountability. We identified two sequential patterns of actions, forming this mechanism: (1) forming a coalition of the willing that shared a common mission, combined their expertise, and mobilized resources; and (2) defining principles for organization selection to ensure mutual value, emphasizing innovation, compliance, cost-plus funding model, and mission alignment. Table 1 presents the sequential patterns of actions that form the mechanism.

In the second period, GzGr further developed the interorganizational value model by aligning on development principles, preventing disagreements during or after the CDSS development. Management, together with end users, developed these principles to ensure broad implementation, feasibility, and usability in multiple levels within the organizations. The principles for developing CDSS were as follows.

First, agreement was needed on the pricing structure and ownership of CDSS. For GPs and the insurer, it was not acceptable to invest their time and share sensitive data, while the technology partners would make high profits, own CDSS, and if prices would fluctuate. At the same time, the technology partners needed to work for profit to survive. Eventually, it was agreed that the technology partners were allowed to make a small profit based on a cost-plus price model and that CDSS would be free of property rights:

Second, GzGr fundamentally chose co-creation as the basis for CDSS development, ensuring that CDSS were driven by clinical problems that were prioritized by both GPs, practice nurses, and patients. While clinicians identified problems in practice related to delivering care, technology developers explored what was technically feasible, and funders assessed financial viability. This ensured the integration of the requirements from people, technology, and business. Initially, this human-centered, problem-driven approach challenged technology partners, who were used to a supplier-client model with predefined requests. Now, they had to share responsibility with end users in identifying clinical problems, requiring them to adapt their development processes:

Third, the GzGr organizations agreed that transparency of CDSS and its development was key. CDSS’ decision paths (even for AI, using logistic regression) could be followed by GPs, and the functionality was made explainable, allowing GPs to verify accuracy and build confidence in the use of CDSS. This transparency supports GPs in meeting legal requirements that state that health care professionals must be able to explain clinical decisions.

Fourth, aligning with legal, regulatory, and clinical guidelines was essential for GzGr. This meant that GPs retained control over clinical decision-making, with the ability to override CDSS recommendations. Although aligning with guidelines was costly, it ensured quality, scalability, and support from GPs:

Fifth, it was agreed that seamless workflow integration was essential by embedding CDSS in GP electronic health records to avoid system switching. The coalition of the willing convinced the largest GP information system developer (technology partner 4) to join, which made it possible that CDSSs could be integrated within their systems. Respondents noted that achieving technical integration was a significant but successfully managed challenge, as they had experienced how a nonintegrated system was rarely used:

While following these 5 principles, 7 CDSS were developed in the GP landscape and were used by 160 GP practices, reaching up to 550,000 patients. An additional 5 CDSS were under development and were tested in 1 to 5 practices, with a reach of 10,000 to 25,000 patients (in 2022; Textbox 1). The development of CDSS started in 2014 and is ongoing.

In conclusion, the organizations concretized the interorganizational value model by aligning early on developing principles (the sequential process): (1) agreements on ownership and pricing, (2) co-creation, (3) transparency and guideline alignment, and (4) embedding CDSS in daily workflows. These principles translated the coalition’s mission into daily practice, helping organizations navigate conflicts, build trust, and coordinate decisions while preserving their individual and regional value. Additionally, the mechanism of iterative, co-creative experimentation emerged with the first prototypes, driven by structured feedback loops between end users, developers, and decision-makers (the sequential process). The sequential patterns of actions that form these mechanisms are presented in Table 1.

After the first CDSS was developed, it became clear that the use of CDSS by GPs was lower than initially expected. In response, the focus shifted to integrating CDSS into daily practice and optimizing use, learning from previous experience, and iteratively adapting the strategy, with the same mission.

Initially, during implementation, primary care groups managed communication within their own organizations, but this proved challenging and ineffective. As a result, GzGr started developing materials, webinars, and introduced on-site implementation staff to strengthen end user skills and integration into workflows. Identifying relevant problems for CDSS also remained difficult, leading to the formation of an editorial committee, comprising technology partners, the insurer, and innovative GPs and assistants. They identified widely experienced implementation problems, composed solutions and an implementation plan, and promoted CDSS use among peers.

As the health care insurer and technology partners had substantially invested in the development and implementation, they wanted a guarantee that GPs used CDSS. Therefore, implementation outcomes (eg, CDSS usage rate and adoption of regional GP practices) were developed and agreed upon, obliging the GzGr organizations to act when algorithm usage rates dropped below a certain threshold:

Additionally, in this period, practice-oriented academic partners joined GzGr to facilitate research, testing, and evaluation, without losing momentum and to ensure a pragmatic process. The results demonstrated that CDSS contributed to time efficiency, improved care quality, and increased adoption by bringing a scientific perspective. The steering committee assessed implementation outcomes across people, technology, and business to ensure stakeholder value. Values were not only monetary; namely, for a GP, saving time was essential, and the health care insurer had to fulfill their duty of care.

GzGr used the implementation and scientific outcomes and user feedback for a continuous iterative learning and improvement experimentation process to refine both the CDSS and the collaboration. The process was as follows: every first version of CDSS is tested in 1 or 2 GP practices (serving between 2000 and 5000 patients). The first tests were done in 5 to 10 GP practices (serving between 10,000 and 25,000 patients). When a CDSS was approved by GzGr, it was implemented in all practices (serving more than 500,000 patients). This process led to adaptations in both the technology and the mission-driven collaboration. Early feedback enabled timely adjustments, strengthened collaboration, and increased support among end users. Respondents noted that this iterative process became a functioning “engine for innovation” within and between the development and implementation phase: creating the opportunity and confidence to develop other technologies and expand the collaboration with a hospital and more care groups.

To conclude, the mechanism “iterative co-creative experimentation” further developed as GzGr shifted from developing technology to embedding CDSS in daily practice. The sequential patterns of actions were (1) co-creation in structured feedback loops, ensuring that people, technology, and organizational perspectives directly informed further development; (2) GzGr took ownership of communication; and (3) technological adjustments were tied to measurable, scientific outcomes. Together, these sequential patterns of actions strengthened shared commitment to GzGr’s mission, while the interorganizational value model streamlined decision-making and maintained focus during iterative development and implementation cycles. Table 1 presents the sequential patterns of actions that shape the mechanisms.

We conducted a qualitative process analysis of our exploratory case study data to identify which mechanisms helped overcome challenges in the development and implementation of CDSS within GzGr, an interorganizational collaboration. This analysis shed light on sequential patterns of actions by identifying the mechanisms through which challenges were addressed across 3 subsequent periods. We identified 2 mechanisms: (1) enacting an interorganizational value model and (2) iterative, co-creative experimentation. The 3 periods show that the mechanisms reinforced each other and how the underlying actions built on each other over time. Iterative, co-creative experimentation refined the interorganizational value model. In turn, this shared value model made iterative experimentation more focused and effective.

We elaborate on how these mechanisms that explain CDSS can be developed and implemented in an interorganizational context and compare them to previous research.

To interpret the results of this study, we have reflected on several limitations in our research. First, the focus on a single case in the Netherlands restricts the generalizability of the findings to other contexts. CDSSs have been scaled to other organizations and are currently being expanded to additional regions. However, it remains uncertain to what extent the collaboration’s approach to CDSS development and implementation is dependent on context or how this compares to other countries. Further research is needed to validate these findings across different organizations, regions, and countries.

Second, reflecting on our use of qualitative data and a retrospective analysis, recalling bias and interpretative variation among respondents might exist. We mitigated this limitation by combining data sources (interviews, documents, and GzGr experts), making use of data triangulation. Moreover, our transdisciplinary research team enriched qualitative data analysis by incorporating diverse perspectives, including medical expertise, innovation ecosystems, organizational governance, design, and implementation. This multiperspective approach enhances the study’s applicability and relevance, despite its single-case focus. Additionally, the retrospective character of this research gave us the opportunity to compose a chronological overview. Due to this nature, the study has a descriptive rather than a predictive value, as formally no causality between actions, sequences of actions, and effects was assessed. Future research may provide predictive insights into how the timeline of the sequential patterns of actions and mechanisms unfolds during implementation.

Third, our identification of 2 mechanisms may reflect a degree of interpretation of the researchers, as is inherent to qualitative research. We addressed this by documenting our observations in a chronological overview table (Multimedia Appendix 3), performing member checks, and linking our findings to existing literature. These factors provide a thorough understanding of the mechanisms underlying CDSS development and implementation in an interorganizational context.

This research identified mechanisms that help interorganizational collaborations overcome challenges in the iterative development and implementation of CDSS. In this perspective, our study extends the determinant-level frameworks NASSS, Centre for eHealth and Wellbeing Research, and UK Medical Research Council by adding an interorganizational, process-level perspective that captures how collaboration unfolds across independent organizations. It shows how organizations, while maintaining their own interests, continuously realign around a shared goal through iterative steps to compose an interorganizational value model that sustains collective progress.

Our practical implications consist of 3 elements. First, to navigate complexity, the development and implementation of CDSS in interorganizational collaborations should involve multilevel stakeholders from the start of the collaboration, especially innovative and influential individuals who are willing to go beyond formal roles to advance shared goals and form a coalition of the willing. Second, an orchestrating actor, such as GzGr, is needed to facilitate communication, alignment, and technical support across individual organizations within interorganizational collaborations. Third, an iterative approach should be adopted, where feedback from multilevel stakeholders informs continuous technological and collaborative improvements that will support both adoption and long-term collaboration.

Future research should enhance the generalizability of our findings by comparing this case with other interorganizational collaborations to identify shared mechanisms. Additionally, further investigation is needed into the scalability of CDSS, distinguishing context-dependent elements. Finally, future research should explore how these mechanisms can be systematically applied to support adaptive implementation in research and practice.

Our study offered insights into the mechanisms for developing and implementing CDSS in an interorganizational collaboration in primary care. Our findings extend the determinant-level implementation frameworks for complex digital health interventions by offering process-level insights into sequential patterns of actions that show how change unfolds in complex, real-world settings. We identified two mechanisms: (1) enacting an interorganizational value model and (2) iterative, co-creative experimentation. These mechanisms supported both technological and organizational change, through the approach of integrating people, technology, and business aspects for all and across organizations. Through this approach, CDSS were generated and implemented via an iterative, co-creative experiment. This resulted in CDSS that were (financially) viable and that generated added value per individual stakeholder and organization. Additionally, CDSS could be embedded in the individual organizations as they fulfilled their values, needs, and interests. Therefore, CDSSs reached multilevel support and were well adopted.

Overall, our study enhances the understanding of how sequential patterns of actions enable digital health innovations to be embedded and sustained in an interorganizational collaboration. It highlights that successful implementation in complex systems depends on establishing shared principles early, forming the basis for an interorganizational value model. Simultaneously, an iterative collaboration spanning development and implementation is required. Throughout this process, continuous alignment of social, technological, and organizational dimensions is essential. We offered both practical and conceptual contributions to understanding how digital health interventions can be embedded and sustained in interorganizational collaboration.