We conducted interviews with software producers and users, that is, physicians, to identify design components for user-friendly clinical dashboards to enhance the quality of care [18]. By interviewing 2 stakeholder groups, we collected insights on 2 key perspectives (software producers vs users).

The recruitment of the interviewees followed a consecutive process: first, we recruited in our own network, and then, through a snowballing technique, we were able to contact software producers and users outside of our network. The conduction of interviews ended once thematic saturation was reached, that is, when no additional information was gained from further interviews [30]. The interviews were semistructured, and findings from a previous literature search (see Multimedia Appendix 1) guided the development of the interview questionnaire. Initially, AYB developed the interview guide, while IS, DK, and AG collaborated to refine it. To finalize the questionnaire, a pilot interview was conducted with a key opinion leader in this subject area. After the first 5 interviews, we iteratively tailored the interview structure to align with the evolving interview dynamics, facilitating a focused exploration of our research priorities. This adaptive approach involved selecting pertinent questions toward the gravitation points that seemed to emerge from our preestablished question catalog, thus augmenting the inductive explanatory insights.

All interviews were conducted by 2 authors (AYB and IS) to ensure that the same content was covered. Both interviewers were female, had received training in qualitative methods as part of their PhD studies, and had no previous relationship with any of the interviewees. All interviews took place between November 02, 2022, and May 15, 2023. Interviewees decided on the interview date and time.

Before each interview, 3 introductory slides were presented to all interviewees to ensure a common understanding of clinical dashboards, outline the underlying research questions, and clarify content-related questions. All interviews covered the following main areas: (1) general questions on dashboards and PROM usage, (2) questions about market penetration (software producers only), (3) usage of the dashboard, (4) dashboard development and data collection, (5) feature assessment, and (6) role of the patient (Multimedia Appendix 2).

We conducted the one-to-one interviews on the web (by Zoom [Zoom Video Communications] or Microsoft Teams) in English or German. During the interviews, both interviewers took notes. All interviews were audio-recorded and transcribed verbatim to perform detailed data analysis. We sent the transcripts back to the interviewees to receive approval and then anonymized all participant details. For the analysis of the interviews, we applied thematic coding to uncover potential patterns and themes within the data [31]. To identify relevant information for the design components, first, deductive coding was used to cross-validate the findings from the literature search. Second, inductive coding allowed us to identify and classify information not covered by the literature search. AYB and CW coded all interviews and conducted the analysis with Atlas.ti Windows (version 22) [32]. The study is reported in accordance with the Consolidated Criteria for Reporting Qualitative Research (COREQ) 32-item checklist to ensure transparency and reliability (Multimedia Appendix 3) [33].

This study was performed in line with the principles of the Declaration of Helsinki. The Ethics Committee of the University of St. Gallen was informed about this research and issued a letter of exemption.

In total, we conducted 16 interviews with 6 software producers (I1 to I6) and 10 users (I7 to I16) until thematic saturation was reached (male=10, female=6). All software producer interviewees were from different companies and had different positions. A total of 8 out of 10 user interviewees were department heads with different specialties, such as pneumology, oncology, pediatrics, or orthopedics. This allowed us to analyze distinctive needs for design components in clinical dashboards. One of the remaining user interviewees was a chief hospital innovation officer, and the other interviewee was a member of hospital quality management. These specific interviewees lacked a formal medical background. However, based on their employment within a hospital setting, they routinely interacted closely with diverse medical practitioners and, thus, can be assumed to have a profound understanding of physicians’ needs for clinical dashboards. The software producer interviews lasted between 29 and 45 minutes, whereas the user interviews had a duration of 20 to 36 minutes (Multimedia Appendix 4).

The interviews with software producers as well as users revealed that they perceived differentiation in design depending on disease (1-time acute vs chronic condition) as not essential. Therefore, we did not distinguish between the design requirements of different disease types but instead focused on the design principles for clinical dashboards incorporating PROMs in general.

Concerning the dashboard components, representatives of both interview groups mentioned that some patient information (such as patient photographs, demographic information, or contact details of other care team members) needs to be presented on the dashboard (9/16). An interviewee additionally emphasized that she expects the patient information to include,

Except for I1 and I5, all other software producers rated clinical data (eg, lab results or medication data) as meaningful information that must be included in the clinical dashboard. In contrast, users tended rather not to overfill the clinical dashboard with clinical data as this information might be available from another source, as a participant mentioned:

The free write-in design component did not resonate well in either of the 2 groups. A total of 6 interviewees replied that a free write-in space is not useful, whereas only 2 interviewees perceived this design component as very beneficial. Nevertheless, when having such a free write-in box included, users (3/10) wanted it as an additional source of information for some specific variables.

Software producers and users rated the past assessment PRO score as one of the most crucial design components in a clinical dashboard incorporating PROMs. From the software producer perspective, one mentioned:

Another participant had a similar perception:

Another well-perceived design component was the peer-group comparison. All software producer interviewees (6/6) agreed on the inclusion of this feature. The users (7/10) appreciated the inclusion, too. However, 3 users did not perceive an added value in the peer-group comparison due to uncertainties in interpreting the applied PRO (I9), the missing guidance on the relevant factors to compare different patient groups (I12), or a clear reasoning on why peer group comparison is useful (I15).

Although only 1 user emphasized applying PRO-related goals, overall health-related goals were discussed more controversially within both groups. Software producer I4 reported that their dashboard includes a component where the physician can develop overall health goals together with the patient and check goal achievement, whereas I6 stated that this is not part of their dashboard. From the user perspective, this design component could add value to the patient-physician communication, as the goal statement makes the aim or expectations of the patients explicit (I9 and I16).

Concerning alerts, software producers and users have different perceptions of whether alerts should be included in the dashboard. Software producers (5/6) mentioned that an alert function needs to be included. Generally, producers were indifferent regarding whether the alert is in real time (4/6) or only appears during the appointment (4/6). A participant mentioned, however, that liability for medical risks and errors is unclear when real-time alerts are used:

In contrast, 4 users preferred not to include alerts. I8 and I11 mentioned that this feature previously existed in their dashboard, but the acceptance was not high enough in their teams, which made them stop using it. Furthermore, I12 raised the issue that additional interpretations for physicians are required to ensure that they completely understand what the deterioration or improvement in a score means. I10 favored the alerts during the appointment to highlight critical factors and to facilitate comparison over time. I13 and I16 preferred real-time alerts allowing the treating physician to react directly to the patient’s issues.

Concerning the dashboard’s customizability, all interviewees agreed that some degree of customizability is required to meet different user needs. Although most users preferred customizability from a standard set (7/10), software producers still seemed undecided whether a clinical dashboard should be adapted to individual needs (2/6) or “off the shelf” (3/6). Nevertheless, software producers (5/6) agreed that scalability is only achievable in case clinical dashboards equipped with features defined in a standard set are provided.

Whereas the above-presented results were derived from the deductive procedure, the subsequent inductive procedure allowed us to identify additional aspects that were not explicitly discussed in the current literature. These aspects included, on the one hand, the added value of clinical dashboards incorporating PROMs and visualization possibilities of PROMs and, on the other hand, various barriers, reducing the uptake of the clinical dashboard. Furthermore, the interviewees also discussed the patient’s access to their data controversially.

Although the visualization of the PRO scores was not extensively discussed in the literature, we asked our interviewees about their preferences. Only 2 out of 6 software producers indicated that the index and dimensional scores are always provided in their dashboards. The remaining software producers elaborated on the visualization of their PRO scores. Contrarily, almost all users (9/10) mentioned their preferences for the visualization: 4/10 users preferred to have index and dimensional scores available, 3/10 users prioritized dimensional scores, whereas 2/10 users voted for the visualization of index scores. For instance, I7 voted for the visualization of dimensional scores by stating, “we must of course know the dimensionality and different aspects.” In contrast, an interviewee argued:

Concerning added value provided by the clinical dashboard, software producers emphasized the visualization of results (5/6), enhanced workflow efficiency (4/6), improved data comparability (3/6), and higher patient satisfaction (3/6). Users perceived the biggest advantages of a clinical dashboard in the visualization of results (8/10), enhanced workflow efficiency (6/10), and improved data comparability (5/10). Higher patient satisfaction (2/10) was not among the most often named advantages.

Potential barriers to implementing the full potential of clinical dashboards are interoperability between various systems for both interview groups (12/16) and the burdensome collection of data for users (6/10). The reasons for the burdensome collection were various. For instance, a participant highlighted organizational issues by stating:

Furthermore, a participant raised technical challenges in collecting PROs by expressing:

Furthermore, software producers mentioned legal consequences in case of displaying inadequate information (4/6) and the licensing of the PROM questionnaires (3/6) as potential barriers. Users rather perceived the lack of good data (4/10), legal consequences (3/10), and nonintuitive use (3/10) as additional barriers.

Except for 2 users (I7 and I14), all interviewees agreed that patients benefit from access to the information stored on the clinical dashboard. A total of 9 of the interviewees expect the patients to develop a better understanding and interpretation of their own health status. One software producer summarized their efforts as

Users highlighted similar scenarios, for instance:

This quote also represents the statement of I14, whereas I15 additionally emphasizes the positive correlation between an enhanced understanding of one’s own health status and increased treatment adherence. Especially the score visualization supports the patient’s understanding of their own health status (6/16). Still, users (7/10) acknowledged that more explanation of the meaning of the data is required to avoid patient’s misinterpretation of the data. Furthermore, 2 users claimed that physicians and patients need different levels of aggregation of available data concerning depth of analysis and interpretation complexity to avoid overwhelming patients. Finally, I6 and I8 stated they would refuse to share clinical data with patients as clinical outcomes might frighten patients or lead to misinterpretations, such as outlined by a user:

The derivation of deductive and inductive codes led to insights into how different design components are related to each other (Figure 1). To build clinical dashboards incorporating PROMs, not only the selected PROMs on the dashboard are crucial, but rather the complete construct of the clinical dashboard. In addition, generic dashboard components need to be respected as well. Furthermore, aspects such as awareness of barriers, patients’ perspectives, and the potential of added value need to be considered when reflecting on the adoption of clinical dashboards (Multimedia Appendix 6 contains a full description of all topics).

The qualitative research approach allowed for a more profound understanding of the design components for clinical dashboards incorporating generic and disease-specific PROMs. Our developed framework (Figure 1) displays that we found 16 design components based on a literature search and identified 9 additional design components through interviews with software producers and users. The interviews revealed that both interview groups had a similar perception of the use case of a clinical dashboard incorporating PROMs.

The use of a dashboard primarily strengthens patient-physician communication in outpatient settings. It should include a combination of disease-specific and generic PROMs. Physicians and other health care professionals were the most frequently mentioned key users. In addition, preferences for visualizing PROs, whether as index or dimensional scores, are subjective and dependent on individual inclinations. According to software producers and users, dashboards provide benefits, especially through the visualization of data and the enhancement of workflow efficiency. The main barriers to dashboard implementation were seen in the interoperability between various systems and the collection of data.

In addition, the interviews revealed that dashboard content should not vary across diseases, except by an adaptation of the disease-specific PROM. This project showed that not only design components directly displayed by the clinical dashboard incorporating PROMs but also other features, such as providing system support or the definition of key users, are relevant topics for a successful implementation.

Concerning the use of PROMs, studies show that it is common to use generic PROMs, particularly the EQ-5D, to track the health status of patients. Notably, researchers see the standardized format and content as a major benefit that facilitates its usability across different diseases and patient groups [34]. Furthermore, the multilingual questionnaire allows large-scale analyses [35,36]. Not only were disease-specific PROMs complementing generic PROMs, but their combined value was seen as more valuable than their sum, as in the case of the EQ-5D and the COPD (chronic obstructive pulmonary disease) Assessment Test (CAT) combination [37]. Interestingly, similar findings emerged from studies developing PROM dashboards for various disease areas. Baeksted et al [28] and Hassett et al [9] found it relevant to include the cancer care–specific PRO of Common Terminology Criteria for Adverse Events (CTCAE) in their dashboards, and Nicolas-Boluda et al [22] incorporated endometriosis-specific indicators into their dashboard. We found that disease-specific PROMs are preferred over generic PROMs by the interviewees. Nevertheless, a combined use of both PROMs was seen as beneficial, too.

Enhanced visualization of the patient’s PROs over time can facilitate the moderation role of the physician [12,19,38] in line with the growing demand for shared decision-making. We found a similar outcome as the interviewees perceived the purpose of a clinical dashboard incorporating PROMs in the improved communication between patient and physician. Furthermore, Desantis et al [12] found that using a clinical dashboard improves the workflow and communication of changes in the HRQoL between the patient and physician. Following this argument, interviewees emphasized that a clinical dashboard incorporating PROMs may also serve as a basis for better-informed decisions by physicians as various key information points are displayed in 1 report. Therefore, additional information besides the PRO data must be displayed to display a holistic picture of the patient’s general health conditions. Patient information (such as sociodemographic information or most recent health updates) and clinical data (such as lab results and medication lists) are required, which was previously highlighted in the literature [20,24,25] and also rated as essential by the interviewees in our study.

Furthermore, the interviewees promoted the design component “past assessment PRO score” and ranked it as one of the most important ones. Changes in the patient’s health status become apparent and traceable, and thus, enable the initiation of countermeasures immediately. Similar reasons are also provided in the literature [9,28]. Another well-perceived design component for analyzing PROs was the peer-group comparison. Interviewees claimed that this design component facilitates understanding whether a patient’s health status is above or below the norm. Hartzler et al [17] highlighted that peer-group comparison should match patients in age and treatment. In addition, Ragouzeos et al [27] advise indicating a “normal” range.

Design components where interviewees did not indicate a clear tendency were PRO-related and overall health goals and the function of alerts. Cronin et al [16] found that patients want to set and evaluate goals over time, which could be supported by a clinical dashboard. However, Liu et al [26] warned that setting goals might further pressure the patient. Furthermore, the challenge of including alerts is to decide on the appropriate alert level [39] and avoid “alert fatigue” of physicians [20].

Our study has 3 limitations. First, the interviewees were mainly from Western countries, especially Switzerland and Germany. A generalization of our results to other countries should, therefore, be carefully assessed. Second, the user interviewees were most often specialists, and we did not control for their years of experience. This research can be seen as a starting point in building guidance on what needs to be considered when developing clinical dashboards incorporating PROMs. Therefore, we wanted to include a broad range of opinions and needs on critical design components. Future research should consider various dashboard stakeholders from different countries with varying years of professional experience to further evaluate our proposed design components. Third, the design components were not built and evaluated in a real-world setting but only discussed during the interviews. We aimed to develop a theoretical framework of design components based on software producers’ capabilities and users’ needs. Therefore, future research approaches can build up on our findings to refine clinical dashboards and test them in clinical practice.

In our study, we identified 25 design components and consolidated them into a framework of 4 main concepts. This framework strives to work as a guiding approach where researchers and practitioners can orient themselves on components to be included when building clinical dashboards incorporating PROMs. Design components such as peer-group comparison or alert thresholds require further investigation in future research, as knowledge on determining appropriate peer groups or setting alerts based on PRO thresholds needs to be improved. In addition, future research efforts may test the real-world applicability of the proposed design components by either discussing them in focus groups or implementing them in a dashboard in clinical practice.