To develop our pragmatic approach, we convened a multidisciplinary group representing clinicians (primary care, geriatrics, cardiology, critical care), psychologists; informaticists; implementation scientists; health services researchers; and experts in human-computer interaction, decision science, and behavioral economics. Members of the group were purposefully selected based on their relevant expertise. They were contacted electronically by emails and synchronous meetings to discuss the goals of the approach, framework selection, and processes. This group was tasked with iteratively developing a systematic approach to select nudge “types” and design nudge “forms” for clinician-facing CDS tools within an EHR. Here, we define nudge “types” based on the Messenger, Incentives, Norms, Defaults, Salience, Priming, Affect, Commitments, and Ego (MINDSPACE) categories (see framework selection, below) and “forms” as different ways the nudge types can be designed [39].

At the outset, we defined key characteristics and goals of this approach including that it needed to (1) be feasible and pragmatic; (2) be applicable across different types of CDS and clinical situations; (3) comprehensively consider the breadth of different types of nudges; (4) balance the potential to impact clinician behavior change with the risk of unintended consequences and resources and effort required to build the nudge and CDS; and (5) promote or foster equity, sustainability, and generalizability of the nudge and overarching CDS solution.

To facilitate comprehensive consideration of different types of nudges, we selected the MINDSPACE framework [39]. While other frameworks could be used, we selected MINDSPACE because it is commonly used [40] and is easy to use for diverse audiences, including those without extensive training in psychology or behavioral economics. MINDSPACE categorizes nudges into 9 types based on how they alter the choice architecture. The MINDSPACE nudge types [39] are described in Table 1 along with the theoretical logic behind the behavior the nudge seeks to address. Each nudge type can then be applied in different ways or forms for specific situations. For example, the nudge type of salience can take on the form of bolded text or a pop-up that causes an interruption. To understand the potential effectiveness of different nudge types and forms on changing behavior, we also used the nudge “ladder” [25]. The nudge ladder categorizes nudges on an effectiveness scale of 1-5 (1=strongest/most effective) in which default options are the strongest and simply providing information is the weakest or least likely to change behavior [25]. As described in Table 1, we applied the nudge ladder ratings to the 9 MINDSPACE nudge types.

While other implementation science frameworks could be used, we selected the Practical, Robust Implementation and Sustainability Model (PRISM) [41-43] because (1) it has been applied with user-centered design (UCD) and human-computer interaction principles to CDS [32], (2) it offers specific guidance on issues of sustainability and equity [41,42,44-46], (3) it includes the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) outcome measures, which can guide nudge design decisions and support pragmatic evaluation of nudges deployed, and (4) there are numerous tools and resources to assist diverse users with and without implementation science expertise through the process of applying the PRISM framework [41,42,47,48].

Use of a framework such as PRISM is key to systematically design nudges to align with the implementation context to optimize relevance, generalizability, sustainability, and equity [41,42]. An implementation science framework such as PRISM provides holistic guidance on how to assess and align complex multilevel contextual issues with the selection of nudge types and forms, along with consideration and measurement of a variety of implementation outcomes.

PRISM is illustrated in Figure 1 and includes the RE-AIM outcomes and 4 context domains that work together to enhance alignment among the context, intervention, and implementation strategies in order to maximize outcomes [43]. The PRISM context domains are (1) the characteristics of the organization and patient, (2) the perspectives of the organization and patient about the intervention including the complexity and evidence to support the intervention, (3) the implementation and sustainability infrastructure including resources available for initial and ongoing implementation, and (4) the external environment including clinical guidelines and policies or regulations.

Central to PRISM is consideration of the multilevel perspectives of organizational partners (eg, leaders, managers, staff) and patient partners (eg, family or caregiver, individual).

The multidisciplinary group iteratively developed this approach, and we applied it to design a CDS tool to improve guideline-concordant prescribing of a mineralocorticoid receptor antagonist (MRA) for patients with heart failure and reduced ejection fraction (HFrEF) [49]. Despite strong evidence that MRAs improve mortality and quality of life for patients with HFrEF, suboptimal prescribing remains problematic across health systems [50,51]. We selected this use case because it is widely relevant across health systems and it can be adapted to address other gaps in care.

The MRA CDS tool was designed to be implemented within primary care and cardiology clinics across the UCHealth system. UCHealth is a regional health system that includes 14 hospitals and more than 900 clinics in academic, urban, suburban, and rural settings across Colorado and parts of Wyoming and Nebraska. UCHealth uses one integrated EHR (Epic Systems).

This study was reviewed and approved by the Institutional Review Board. Patients and clinicians who participated in focus groups provided informed consent and were compensated for their time (patients were compensated $60 and providers were compensated $100). All local, national, regional, and international laws and regulations regarding the protection of personal information, privacy, and human rights were adhered to.

Through a series of iterative applications to the MRA case example and consensus-based discussion with the multidisciplinary group, we developed a 4-step process that can be used to systematically identify and prioritize nudges and that applies to a range of different types of nudges, CDS tools, and diverse clinical issues. These steps are outlined in Textbox 1. The four iterative steps are as follows: (1) engage partners for UCD, (2) develop a shared understanding of the nudge types, (3) determine the overarching CDS format, and (4) brainstorm and prioritize nudges to address each modifiable contextual issue.

These steps are iterative and intended to be adapted to align with the local resources and needs of various clinical scenarios and settings. Below, we describe these four steps and illustrate how this approach was applied to the use case of a CDS to improve prescribing of MRAs for HFrEF.

Step 1, engage partners for UCD, is a process that involves several sub-steps: assessment of the context, identification of key partners, statement of the problem the CDS will address, and identification of modifiable drivers of the clinical decision.

Aligned with implementation science principles of designing relevant, generalizable, sustainable, and equitable solutions, the first sub-step is using PRISM to assess the context of the local and external setting [52,53]. Key to assessing the context is multilevel partner engagement. PRISM can be used to identify the multilevel partners to engage, inform the input needed from the different partners, and promote equity through representation of all partner perspectives [48]. While CDS are clinician-facing tools, their recommendations should be aligned with the patient’s needs and preferences; therefore, the patient is an important partner perspective to capture. Use of an implementation science framework such as PRISM provides guidance on how to design for equity, sustainability, and scalability (generalizability) from the beginning [53].

Create a clear problem statement or evidence-based gap. This is informed by the contextual assessment and with input from your team. Set a team-based approach with diverse expertise and iterative engagement of multiple types of partners. It is ideal to have expertise in the areas of CDS best practices and technical build, the clinical area of interest, decision science, implementation science, and ethics, but all this expertise is not always available within usual operations. Such expertise may be represented within the “implementation team,” which leads the design activities or through the partner engagement process. Partners engaged should include clinician end users of the CDS and other levels of partners within the local setting who could influence the uptake of the CDS tool (eg, leaders, informatics governance, decision-makers) or who could be influenced by the CDS tool (eg, patients whose care is impacted). The nature of partner engagement can range from formal semi-structured interviews or focus groups to less formal meetings or even emails to meet the goals of the engagement while also fitting within available resources [20].

For the MRA example, the goal was to design a CDS tool to improve evidence-based MRA prescribing for patients with HFrEF in outpatient cardiology and primary care clinics. We convened an implementation team with expertise in the clinical situation, implementation science, clinical informatics, and decision science. We used PRISM with UCD strategies to guide our contextual assessment and partner engagement activities [32]. To understand end user characteristics and perspectives of the CDS solution, we conducted a series of clinician focus groups. Through these focus groups, we assessed contextual factors that influence prescribing, their workflows, and preferences for receiving information within such a CDS solution [54]. Patients with HFrEF were also engaged in focus groups to ensure the CDS recommendations were patient-centered [55]. To promote equity, we used purposeful sampling of providers and patients across different types of settings (eg, academic, rural, and community settings) to ensure representation of diverse perspectives, not just the average. We engaged internal health system operational and informatics leaders and governance groups. This engagement was iterative and occurred via informal one-on-one and standing group meetings to ensure alignment with strategic priorities, gain buy-in, and get approval to deploy the CDS.

Next in the UCD process is understanding modifiable drivers of the clinical decision. Findings from the local context should be iteratively considered and mixed with findings about the context of the external environment (eg, regulations, evidence-based recommendations, reimbursement issues) to understand facilitators and barriers that can influence clinician decision-making, both the initial feasibility and ongoing sustainability of maintaining the CDS tool, and the scalability or generalizability of the tool. Throughout this process, the best practices in CDS design [32], including evidence-based principles of human-computer interaction and UCD methods, provide more granular direction on assessing CDS design decisions that consider issues of socio-technical relationships.

For the MRA example, we quantitatively assessed patterns of prescribing MRAs, including characteristics of patients, to identify additional contextual drivers of prescribing [56]. We then compared our findings to externally published literature to understand generalizability and validate the issues identified. We considered key external issues including national quality benchmarks, evidence-based recommendations for HFrEF management, and best practices in CDS design.

Finally, in Step 1 of the UCD process, modifiable issues are identified that can be addressed by nudges. Not all of what is discovered in Step 1 will be modifiable or amenable to a nudge intervention; therefore, an important part is reviewing the contextual findings to create a clear list of modifiable issues that can be addressed by nudges. Partner input may be needed to determine which issues are modifiable. For example, if the implementation team does not have CDS technical or clinical expertise to consider viability of tangible solutions to contextual issues, these perspectives will need to be engaged. Findings from the contextual assessment in Step 1 that are not amenable to nudge intervention are revisited in later steps to inform the type of nudge selected and its design or form.

In the MRA CDS tool example, the ultimate goal was getting clinicians to prescribe an evidence-based MRA for patients with HFrEF. In Step 1, we identified the following contextual drivers of the decision to prescribe an MRA that were deemed tangible issues for nudges to address within a CDS: (1) remembering to prescribe an MRA during a patient encounter, (2) overcoming clinician misconceptions that high-normal serum potassium and low-normal renal function are barriers to MRA, and (3) time needed to determine what laboratory monitoring is needed to safely monitor an MRA after prescribing. We identified many other contextual barriers and facilitators that were not amenable to a nudge intervention and documented these to consider later. For example, providers felt patients often hesitated to start treatment due to cost or complexity, and while this was considered an important contextual consideration, we felt it was not amenable to a nudge intervention.

For Step 2, develop a shared understanding of the nudge types, it is useful to review the nudge types as a team. The focus should be on developing a shared understanding in language that resonates with the team of both the different nudge types and their strength of changing behavior based on the nudge ladder. To facilitate this process, we suggest using MINDSPACE and documenting shared definitions using relatable terminology along with ratings from the nudge ladder for each nudge type. Although MINDSPACE is relatively accessible to broad audiences, the jargon and categorization can still be interpreted in different ways, especially among multidisciplinary teams. We have found this to be an iterative process to come to consensus and, ideally, integrate multidisciplinary team perspectives.

In the MRA CDS tool example, we found that the original definitions of MINDSPACE were not always intuitive and that the differentiation between the categories was not always clear. Because the original MINDSPACE definitions were interpreted differently by different members of our team, this process of developing a shared “mental model” was key to developing a common vocabulary by which we could systematically consider applying the different nudge types. Together, we iteratively worked to create definitions that resonated with the unique perspectives of our multidisciplinary team, that clearly distinguished the different categories of nudge types, and that considered the strength of the nudge. For example, salience was defined as something that draws our attention because it seems novel or relevant, but our team amended this definition to define salience as “anything that draws our attention, not necessarily because it is novel or relevant.” Table 1 provides an example of how we documented our shared understanding of the MINDSPACE nudge types and their strength.

The overarching CDS tool itself is a nudge that can include additional embedded nudges. Step 3, determining the overarching CDS format, focuses on determining a format that addresses the ultimate intended behavior change (eg, appropriate prescribing), while the embedded nudges can address additional contextual drivers of the ultimate behavior change (eg, address informational needs). Deciding on the overarching CDS format can be key to guiding later decisions about what embedded nudges are feasible; therefore, Steps 3 and 4 are purposely distinct. For example, with some CDS formats, such as those embedded within a medication order, there may not be space or technical capacity to address additional contextual drivers of the ultimate behavior change. CDS within medication orders often have character limits and do not allow for functions such as hyperlinks, thus providing limited space to address contextual drivers.

Often, the overarching format of the CDS tool is broadly categorized as interruptive (eg, “pop up”) or passive. With passive CDS, either the user needs to seek out the CDS tool or the tool is presented in such a way that the user’s workflow is not interrupted (no hard stop). A decision on whether the overarching CDS tool is an interruptive or passive format is based on the severity and frequency of the targeted behavior to be changed, as well as organizational norms, priority of the issue, and end user preferences [20]. Interruptive CDS is considered a stronger nudge type based on the nudge ladder and is often reserved for higher risk clinical situations. Decisions about the overarching CDS format should be informed by traditional UCD principles and best practices in CDS design [32].

In the MRA example, an interruptive “pop-up” CDS was selected for the overarching format based on the severity of the clinical situation, input from clinicians, and perceived need for space to address additional contextual drivers of prescribing an MRA. Per MINDSPACE and the nudge ladder, this CDS format is a “salient” type, and the strength is “strong.” At our health system, interruptive CDS are reserved for situations in which the clinical severity warrants the interruption or when passive CDS are not feasible to address the need. A strong, salient nudge was deemed appropriate for our local context, and this format provided space to address other drivers of the behavior change.

Step 4 is to brainstorm and prioritize the nudges. To support divergent and creative thinking, this step is ideally completed independently by multiple members of the team who then share ideas for discussion and prioritization. In this step, each MINDSPACE nudge type defined in Step 3 is iteratively applied or mapped to each modifiable issue identified in Step 2 to define different ways or forms a given nudge type could be designed within the overarching CDS format determined in Step 3. There may be multiple ways a nudge type can be implemented for each modifiable issue, which should be explored. It is also possible there is not a feasible way to apply a given nudge type due to technical or other constraints. When this step is completed by individuals with technical knowledge and understanding of the clinical context, such decisions about feasibility may be more efficient and immediately evident.

In this step, we suggest using visual prototypes or mockups of how the different nudge types could address the different modifiable issues within the overarching CDS format. Mockups can be helpful to efficiently aid understanding and facilitate discussion among the team. In most cases, low-fidelity static prototypes or wireframes are sufficient to illustrate the nudge form and are conducive to rapid iterations.

For the MRA CDS tool example, two clinician informaticists familiar with the clinical situation and technical capability of CDS (KT and SZ) systematically and independently mapped the MINDSPACE nudge types to the modifiable issues identified in Step 2 by creating different forms for each nudge type. They mocked up each nudge form into low-fidelity, static prototypes of the overarching CDS format (interruptive CDS). The prototypes consisted of screenshots of other interruptive CDS that they edited using Microsoft PowerPoint. Based on their knowledge of technical feasibility and clinical situation, they did not identify any practical forms for applying the MINDSPACE nudge types for “Commitments” or “Incentives.” Although the team considered requiring clinicians to provide a free-text reason for deferring prescription that would then be saved to the patient’s EHR record, this was quickly deemed not feasible given technical limitations and thus was not mocked up into a prototype.

Using the mockups, the team then discusses the different forms for each modifiable contextual issue and rates them based on (1) anticipated impact on changing clinician behavior, (2) potential unintended consequences, and (3) technical feasibility to build. In this step, PRISM’s RE-AIM outcomes can prove useful in anticipating the potential impact of a nudge form on pragmatic outcomes of RE-AIM [57,58]. As with any technology, usability and user experience of a given nudge are important determinants to consider when anticipating impact on outcomes, especially those outcomes that are upstream of effectiveness, including clinician adoption and implementation fidelity. Considering the impact of a nudge form based on RE-AIM, including explicit discussion about potential unintended consequences, promotes safe and ethical use of nudges, fosters a culture of “do no harm,” and considers issues of equity and autonomy. RE-AIM can also assist in anticipating the magnitude of impact a nudge form can have on various outcomes including changing clinician behavior and assist in determining if the effort is worth the return [58].

It is important to assess technical feasibility as part of examining the overall effort required. Depending on team composition, ratings of technical feasibility may be reserved for a separate conversation with EHR technical staff (eg, analysts, builders) with this knowledge. A focused discussion regarding the ratio of “benefit to resources required” is important to design practical nudges that consider the impact within resource constraints [33]. For example, if a particular nudge form is anticipated to have low adoption and be moderately effective at changing behavior but will require a high degree of ongoing resource allocation, it may not be prioritized.

For the MRA example, the prototypes guided a discussion between the two clinician informaticists and two other members of the team, a social psychologist and a clinician with expertise in decision science. The team discussed and rated the prototypes based on impact, unintended consequences, and technical feasibility. This discussion and rating process occurred over a series of meetings in which other members of the broader team were consulted as questions arose. For example, the heart failure specialist was consulted when questions arose about the appropriateness of encouraging certain clinical decisions and the operational capacity to include a link for referral to cardiology within the CDS.

Throughout this prioritization process, the nudges with the greatest anticipated value (high impact, minimal or no unintended consequences, low resource burden) are selected and used to iteratively refine CDS mockups that integrate the different combinations of nudge forms within the overarching CDS format in ways that are deemed user-friendly. Principles of UCD and CDS design best practice need to be considered when designing the nudge forms and especially when thinking about how to combine them within the overarching CDS format to optimize human-computer interaction. To promote transparency and generalizability, it is important to document decisions and reasons for decisions. Ultimately, the CDS mockups resulting from this step are then iteratively tested and refined using traditional UCD approaches (eg, simulated scenarios and usability testing) to optimize usability and user experience.

Aligned with the goals of learning health systems, once a minimum viable product is defined, an evaluation plan is critical to ensuring positive impact and to iteratively refine the tool over time. This includes consideration of a study design and pragmatic outcome measurement. PRISM and PRISM’s RE-AIM outcome measures (Reach, Effectiveness, Adoption, Implementation, Maintenance) provide a framework for measuring both implementation and effectiveness outcomes iteratively over time along with the representativeness (equity) of the outcomes and important contextual drivers. More details of how to use PRISM and its RE-AIM outcomes to evaluate CDS tools within learning health systems along with examples can be found in other references [32,59].

For the MRA example, this process led to the development of three low-fidelity static prototypes of the CDS tool that integrated various combinations of the nudge types of salience, messenger, incentives, and defaults in different forms. These three prototypes were iteratively refined by sharing with the broader implementation team and then through a traditional UCD process with potential clinician end users. Examples of the three low-fidelity prototypes and the final CDS tool are available in Multimedia Appendix 1 with illustrative examples of the different nudge types. To evaluate the impact of the MRA CDS tool, we are currently comparing 2 versions of it in a 6-month, mixed methods randomized controlled trial across outpatient cardiology and primary care practices. We are conducting a mixed methods evaluation guided by PRISM and PRISM’s RE-AIM outcomes, including interviews of clinicians and quantitative EHR data.

In developing this approach, we initially met frequently, and some meetings were dedicated to trialing different approaches (eg, nominal group technique), but ultimately settled on the approach described here that involved two 1-hour meetings between 2 members of the team (KT and SZ) who rapidly prototyped and prioritized nudge designs, three 1-hour meetings for input from experts in social psychology and decision science (KT, DM, LS, and JM), and periodic asynchronous input via email from other members of the multidisciplinary team (DM, LA, CL, AH, and JM). It is also notable that we did not include a patient perspective in our multidisciplinary group to design the nudges, but we did engage patients early on in Step 1 to ensure the CDS were designed with their preferences in mind.

We describe a new systematic and pragmatic approach to selection and prioritization of nudges that can be used to comprehensively and systematically consider key issues in designing CDS to optimize feasibility, effectiveness, equity, and sustainability while minimizing potential for unintended consequences. Grounded by implementation science principles and approaches, this 4-step approach involves (1) engaging partners for UCD, (2) developing a shared understanding of the nudge types, (3) determining the overarching CDS format, and (4) brainstorming and prioritizing nudges to address each modifiable contextual issue. The approach prioritizes those applications of nudges with greatest potential to maximize the effectiveness of the intended behavior change, minimize unintended consequences, and align with the variable resource constraints of time and personnel available to design and build CDS tools in ways that are sustainable. We focused on clinician-facing nudges, recognizing they can also be patient-facing [51].

Our findings present one way of systematically and comprehensively selecting nudges for inclusion in CDS tools using an implementation science framework, PRISM [41]. We considered other approaches, including the nominal group technique, but found them to be cumbersome and less practical. The approach we present is intended to be pragmatic, iterative, and rapid within the context of a learning health system and should flex in ways that are relevant and meet the needs and resources or expertise available within a given health system. For example, not all health systems have access to experts in social psychology who have in-depth knowledge of behavioral economics, but this limitation should not preclude them from applying this approach. In other situations, a health system may need to expedite the design and implementation of a CDS solution to address a safety issue [20], and this systematic approach may need to be abbreviated without the luxury of having a series of discussions over time.

Despite growing application of nudges to CDS tools [28-31,60,61], there is a paucity of guidance on how to apply nudges, and recent calls have been made to refine such strategies [62,63]. One framework that others have used to leverage insights from behavioral economics when designing implementation strategies is Easy-Attractive-Social-Timely (EAST) [64,65]. The 4 domains of EAST—easy, attractive, social, and timely—can provide broad guidance when designing nudges for CDS but may not provide the specificity some CDS developers need, nor does EAST provide focused guidance on issues of equity, sustainability, or unintended consequences [64]. In contrast, the approach we describe provides more granular direction on how to select among the various types of nudges by leveraging the MINDSPACE framework [39] while also considering the broad implementation context via its use of the PRISM implementation science framework. Further, our integration of PRISM into this approach provides guidance on how to design nudges in ways that optimize equity and sustainability while minimizing potential unintended consequences of nudges [28-31,41-43,47,48].

When considering the research implications of our findings, this approach addresses an unmet need for guidance on how to systematically apply nudges to CDS tools in ways that allow for adaptation and tailoring to align with the local context, but that is also replicable. This pragmatic and rigorous approach has the potential to optimize the positive impact (eg, effectiveness, equity, sustainability) of CDS tools locally by ensuring relevance, while the systematic process and use of an implementation science framework simultaneously facilitates the scalability of CDS tools and their effect [3]. In using this approach to optimize CDS tools via nudges, it is hoped the well-known knowledge to practice gap can be decreased [66,67].

Aligned with the learning health system goals of being rapid and improving the quintuple aim, we encourage the use of this approach to create minimum viable products that are not anticipated to cause harm [68]. In other words, aim for good enough and not perfect. Technology and health care are rapidly changing, and aiming for perfect is likely an unachievable goal that will only stymie progress. The goal of this systematic approach to nudges is to aim for a CDS solution that is “good enough” while being confident that no harm will result and having a plan for continual improvement over time. This approach provides guidance on how to integrate methods and principles from behavioral economics with implementation science and traditional UCD principles to intentionally design nudges (and other behavioral economics strategies) for CDS that are equitable, effective, sustainable, and generalizable. This approach considers the complexity of how the overarching CDS itself is a nudge which can have layers of embedded nudges that must all work together and align with the dynamically changing context of health care locally and nationally to positively change behavior [69,70].

Table 2 outlines tips for success when using this approach, and Table 3 outlines some considerations for using this approach that surfaced when we developed this approach.

The approach we present to selecting nudges should be used as a guide and adapted as needed. When adapted, we encourage documenting and reporting how the process was adapted, to promote rigor and replicability. Those seeking to replicate this approach for their health system may also choose to use a different implementation science framework that is more familiar to them or that fits their situation better [71,72]. There are also other frameworks to categorize nudge types that could be used [25].

Limitations of this approach include the costs of personnel time incurred from multiple meetings; these are important to consider and balance when deciding how to apply this approach. Although our use of a multidisciplinary team-based approach is a strength, not all health systems will have access to the same types of expertise. In this paper, we describe how one health system applied the approach based on their available resources and expertise, which may not be generalizable for all health systems. However, this approach is intended to be pragmatic, and we encourage health systems to adapt it to fit within the available resources and expertise. Further, although this team-based approach may rely on consensus for decision-making, decisions are grounded in evidence-based principles of human-computer interaction and guided by principles of UCD and conceptual frameworks, namely MINDSPACE and PRISM from implementation science.

By using a systematic approach to selecting nudges and documenting and reporting reasons for decisions, the findings can be adapted and generalized to other health settings and clinical situations, advancing the goals of learning health systems to generate evidence that is both internally and externally valid [3]. Grounded in implementation science, this approach has the potential to improve the effectiveness, equity, and sustainability of CDS tools while minimizing the potential for unintended consequences. We hope that lessons learned using this and similar approaches will be used to make these approaches more accessible and useful to broad audiences to optimize the effectiveness of CDS and ultimately expedite the translation of evidence into practice. Future research is also needed to evaluate the impact of using this approach on CDS effectiveness.