A user-centered design approach based on UX/UI industry standards was followed to develop a decision tool to promote shared decision making [33-35]. User-centered design is an iterative, multistage design and evaluation approach that is driven and refined by user input and customizes the interface based upon an explicit understanding of users, tasks, and environments [36]. UX design refers to user experience, while UI design stands for user interface. Both elements are crucial to app development. UX/UI refers to different aspects of the design. UX design is more comprehensive than UI—encompassing user needs, values, abilities, and limitations as they relate to the user’s interaction with and perception of the design product. UI design focuses on ensuring that the graphical interface has elements that can be used, accessed, and understood based on user needs. UX/UI development was adopted with a goal of creating a tool that deviates from traditional CDS (eg, alerts and reminders).

UX/UI design elicits feedback and input from a multidisciplinary team, here including patients, emergency care clinicians (attending physicians, residents, physician assistants, and nurse practitioners), health services researchers, interaction design experts, and health systems information specialists (including a software system engineer and a computer programmer) to make incremental refinements to the prototypes. The development process involved successive iterations of the prototype within 4 UX/UI phases: (1) initial prototype development, (2) usability testing, (3) field testing, and (4) beta testing (Figure 1). Each phase continued until thematic saturation [37]. The initial prototype, including review and synthesis of the evidence and analysis of usual practice, has been previously described [16,19]. In the second phase, we performed formative usability evaluation in a simulated clinical environment using clinicians with standardized patients to maximize ease of use and clinical integration. Next, the prototype was field-tested by the research team with ED patients and, finally, the tool was beta-tested during clinical care by physician users.

Participants were patients and clinicians recruited from an urban, academic Level I trauma center ED with 103,000 patient visits per year and a satellite ED with 24,000 patient visits per year. Clinicians were recruited from the 48 attending physician faculty, 58 resident physicians, and 47 midlevel providers.

The study protocol was approved by the hospital institutional review board (IRB). All participants provided their verbal consent as specified by the IRB. Some portions of the evaluation were performed at an outside institution. The protocol was also approved by that institution’s IRB. Usability evaluation subjects were compensated for their time and travel with $100 gift cards. In beta testing, physicians were compensated for their time with $50 gift cards for each patient enrolled. Patients were not compensated in beta testing during their ED visit.

The initial results of the prototype were previously reported [16,19]. Cognitive task analysis (critical decision method interviews and 150 hours of direct observation in the ED of peer-nominated senior emergency physicians recognized for their skill in safely minimizing testing while maintaining patient safety and engagement) revealed 5 core domains: trust, anxiety, constraints, influence of others, and patient expectations [16].

The initial prototype followed a visual metaphor of design reminiscent of decision aids on paper cards [19]. After the patient filled out eligibility and questionnaire forms to autopopulate subjective components of the clinical decision rule, 3 sections followed. The first section centered around patient education (information about concussions, CT scans) to be used by the patient alone prior to the clinician’s evaluation and gave the patient the opportunity to flag concerns on a digital checklist. These concerns would later show up in the second section to be used by the clinician with the patient (screen capture of this section displayed in Figure 2). After completing a CDS checklist, the tool generated patient-specific risk estimates for pertinent outcomes and risk of cancer from a head CT. The final section involved a process of shared decision making in which patients and clinicians decided together whether to obtain a CT scan, to continue to be observed in the ED, or to go home.

Usability evaluation was conducted 3 times with 9 users. Observation revealed the tool required modification to facilitate conversation between the patient and clinician to be incorporated seamlessly into the clinical workflow [18,19]. Therefore, the initial user-centered design was augmented by interaction design using patient-centered and participatory decision aid development [18,19,41-44]. An interaction designer (MB) joined the research team [18,45,46]. Subsequent rounds involved rapid prototyping and low-fidelity wireframing.

This enhanced approach focused on tool usefulness (and lack of use by test subjects). Interview responses revealed users were not using the tool because the tool was overly prescriptive with too much text on the screen that interrupted or distracted from conversation with patients. Earlier prototypes were overdesigned, which forced clinicians to give more attention to the tool than the patient or to abandon the tool. Eliminations included the patient section with educational materials for patient review prior to the clinician’s evaluation (based on previous qualitative findings that patients come to the ED for a clinician’s expert evaluation) and a patient demographic survey and questionnaire about the injury. Revisions dramatically reduced the number of screen taps, checkboxes, and data entry. Furthermore, the Concerns section expanded to 6 boxes a patient could select to discuss (Figure 3). This minimalist version allowed clinicians to adapt the tool to their practice style and patient-specific education. It reassured patients by providing structure to the clinical conversation with cues (eg, How soon can I get back to work?). The tool was less prescriptive and increased the likelihood of implementation.

Field testing was conducted with 10 patients. Additional incremental revisions were made to the prototype. Observation and analysis of use in the ED context and application of ecological interface design principles distilled the workflow for the final Concussion or Brain Bleed app (Figure 4). This further elucidated important patient issues. The final app now supports the clinician’s decision and patient engagement and education around patient-specific risk about head injuries, CT imaging, counseling, and patient concerns.

Data entry was streamlined, and explicit user input was nearly eliminated. Grouping risk categories provides the clinician with the patient’s individualized risk assessment by a single tap of the screen (Figure 5). This efficient Canadian CT Head Rule display gives the clinician more time for risk communication with the patient.

The risk visualization format and content underwent revisions from the initial prototype through usability and field testing (Figure 6 a-d). The initial prototype used text-based risks (eg, clinically important brain injury) [2,19,47]. Later versions used pictographs, plain language, absolute risks with a constant denominator, and a color scheme to differentiate the 4 categories of patient-centered outcomes [42,44,48,49].

A key finding was how important it is to teach and emphasize that a concussion is not visible on CT. The tool evolved into helping patients understand specific recommendations and their implications. The Risk Discussion section offers plain language on the utility (or lack thereof in low-risk patients) of CT as well as cues to discuss concussion and the individual patient’s concerns (Figure 7).

Beta testing was conducted over 6 weeks with 4 attending emergency physicians in the care of 7 low-risk, minor head injury ED patients. The final Considerations section for low-risk patients was revised based on user feedback that it was too busy. Prior to beta testing, this section had a wall of text including a large inventory of sections that could be discussed at the clinician’s discretion. Beta testing revealed just a checklist with the option to expand sufficed. The section’s content remained relatively unchanged with the format converted to a checklist with single-tap dropdown options that provided more information (via hypertext) when specifically selected (Figure 8). Readability increased with limited distractions while remaining flexible to differing clinician practice styles and individual patient needs.

We developed a work-around for integration with EHR workflow using Epic (Epic Systems Corp) SmartPhrases (Multimedia Appendix 4). This charting tool allows clinicians to autopopulate text using shorthand. SmartPhrases allow rapid documentation of use of the Concussion or Brain Bleed app in the EHR.

A total of 16 successive iterations have resulted in a tool that integrates Canadian CT Head Rule CDS at point of care with a patient decision aid to promote conversation around individualized risk and patients’ specific concerns. Design, development, and formative evaluation were informed by the philosophy that technology can accelerate the provision of evidence-based care that is efficient and empathic, effectively reducing unnecessary care [17,19,24]. The user can traverse the app in its entirety in 3 to 5 screen taps. Concussion or Brain Bleed addresses the human factors that research demonstrates are critical for optimizing CT use in minor head injury by creating the time and space for conversation between patients and their clinician [16]. The app equips clinicians to foster trust and manage patient expectations in a fast-paced ED environment characterized by uncertainty and high emotions.

We elected a formative evaluation inclusive of usability evaluation—namely, the well-established practice-based, patient-centered, and participatory decision aid development process adapted for our computerized tool [18,19,41-44]. It allowed for problematic elements of the prototype to be rapidly identified and addressed and usability of unchanged elements to be tested by multiple users through existing and subsequent iterations. The final product is refined based on user input and represents the culmination of rigorous testing in simulations and real-world clinical encounters. Feedback from a multidisciplinary team has been incorporated with the express goal of practicality and usability. Our tool addresses multiple items across the dimensions of the International Patient Decision Aids Standards including the use of a systematic development process, presenting information on probabilities of outcomes and using the scientific literature on which content is based, conveyed with plain language [43]. Involving end-users and a variety of clinicians in both simulated and real-world clinical environments in an iterative process ensures that the format and information content of our tool is responsive to user preferences and the complexities of decision context.

CDS is most effective when it is part of the clinician’s workflow at the time and location of decision making [27,28]. Decision support strategies to date have focused on physician behavior [25-27]. By bringing CDS to the point of care and integrating it with a decision aid on a tablet computer shared by the patient and clinician, Concussion or Brain Bleed could improve both the quantity and quality of time at the beside. Patient decision aids have been established as an effective way to translate evidence-based care into practice [29,50]. Visualizing benefits and harms can lead to increased patient knowledge and involvement in decision making, greater satisfaction with the decision-making process, and optimal health outcomes consistent with patient values and preferences [29]. Patient involvement begins with the development of a partnership and includes participation in information exchange, deliberation, and decision making [51]. Patients—even older patients with more experience in a historically paternalistic mode—report great interest in getting involved in similar decision making again [52,53]. While paper-based patient decision aids are beneficial, they do not provide CDS for the clinician and are static in nature with regard to patient-specific estimates and concerns [18,29,44,50]. Tablet computers retain the portability and usability of a paper-based decision aid while also providing the customization and flexibility of computerized CDS with regard to patient-specific risk visualization. In developing a digital tool, we also have the additional benefit of having a database to collect, edit, store, and retrieve data generated by the tool and further reduce workflow burden through direct integration with EHR systems.

The fight to stem medical overuse will require the use of disruptive technologies—often innovative but simple, high-value solutions that can be widely adopted and easily used. In creating this patient-centered clinical decision support tool, we aim to decrease CT use for minor head injury. This tool combines evidence-based practices with patient engagement that establishes trust, empowers active participation, and addresses patient concerns and uncertainty about their condition at the point of care. It helps clinicians to determine who needs a CT and then helps patients to understand why. We hypothesize that, when implemented, the Concussion or Brain Bleed app will support—not hinder—the clinician-patient relationship, safely reduce CT use, and improve the patient experience of care.