Interactive Digital Health Tools to Engage Patients and Caregivers in Discharge Preparation: Implementation Study

Background: Poor discharge preparation during hospitalization may lead to adverse events after discharge. Checklists and videos that systematically engage patients in preparing for discharge have the potential to improve safety, especially when integrated into clinician workflow via the electronic health record (EHR). Objective: This study aims to evaluate the implementation of a suite of digital health tools integrated with the EHR to engage hospitalized patients, caregivers, and their care team in preparing for discharge. Methods: We used the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to identify pertinent research questions related to implementation. We iteratively refined patient and clinician-facing intervention components using a participatory process involving end users and institutional stakeholders. The intervention was implemented at a large academic medical center from December 2017 to July 2018. Patients who agreed to participate were coached to watch a discharge video, complete a checklist assessing discharge readiness, and request postdischarge text messaging with a physician 24 to 48 hours before their expected discharge date, which was displayed via a patient portal and bedside display. Clinicians could view concerns reported by patients based on their checklist responses in real time via a safety dashboard integrated with the EHR and choose to open a secure messaging thread with the patient for up to 7 days after discharge. We used mixed methods to evaluate our implementation experience. Results: Of 752 patient admissions, 510 (67.8%) patients or caregivers participated: 416 (55.3%) watched the video and completed the checklist, and 94 (12.5%) completed the checklist alone. On average, 4.24 concerns were reported per each of the 510 checklist submissions, most commonly about medications (664/2164, 30.7%) and follow-up (656/2164, 30.3%). Of the 510 completed checklists, a member of the care team accessed the safety dashboard to view 210 (41.2%) patient-reported concerns. For 422 patient admissions where postdischarge messaging was available, 141 (33.4%) patients requested this service; of these, a physician initiated secure messaging for 3 (2.1%) discharges. Most patient survey participants perceived that the intervention promoted self-management and communication with their care team. Patient interview participants endorsed gaps in communication with their care team and thought that the video and checklist would be useful closer toward discharge. Clinicians participating in focus groups perceived the value for patients but suggested that low awareness and variable workflow regarding the intervention, lack of technical optimization, and inconsistent clinician leadership limited the use of clinician-facing components. Conclusions: A suite of EHR-integrated digital health tools to engage patients, caregivers, and clinicians in discharge preparation during hospitalization was feasible, acceptable, and valuable; however, important challenges were identified during implementation. We offer strategies to address implementation barriers and promote adoption of these tools. J Med Internet Res 2020 | vol. 22 | iss. 4 | e15573 | p. 1 http://www.jmir.org/2020/4/e15573/ (page number not for citation purposes) Fuller et al JOURNAL OF MEDICAL INTERNET RESEARCH


Introduction
The transition from hospital to ambulatory care is a vulnerable time for patients and stressful for their caregivers: new treatments have been initiated, conditions require monitoring, and the plan is in flux. Approximately 19% to 28% of patients experience preventable adverse events after discharge, many due to suboptimal monitoring of conditions, medication errors or nonadherence, and failure to execute the recovery plan [1][2][3][4][5][6]. During hospitalization, discharge planning is often initiated late, and input from patients regarding their preparedness is frequently lacking, which may lead to delays and dissatisfaction [7]. After discharge, patients report problems related to follow-up, medications, and self-care; have unanswered questions that could have easily been addressed before discharge [8]; and often feel more relieved than burdened when readmitted [9]. Lack of patient engagement during the process of discharge preparation may contribute to avoidable adverse events and costly readmissions [10], particularly those that occur early after hospitalization [11,12].
To date, efforts to enhance and standardize discharge practices have typically targeted clinicians [13,14]; interventions directed at patients provide an opportunity to improve patient understanding, self-management, and postdischarge outcomes [15]. National agencies (eg, Agency for Healthcare Research and Quality [AHRQ], Centers for Medicare and Medicaid Services) are attempting to engage patients and caregivers more broadly by offering access to discharge preparation materials that include checklists for patients [16,17]. Few institutions have determined how best to operationalize these tools for patients. Digital health technology could be leveraged to more proactively engage patients, caregivers, and clinicians during the process of discharge preparation [18][19][20][21]; however, currently available patient-facing digital health tools such as patient portals have gaps in functionality with regard to assessing discharge readiness, are not well integrated with the electronic health record (EHR), and present challenges when used during hospitalization [18,[21][22][23]. Although it is technically feasible to administer a discharge checklist through a patient portal or mobile device [24,25], hospitals lack knowledge about the potential for adoption and perceived utility of these tools for patients and clinicians in a real-world clinical setting, as well as potential barriers for sustaining the intervention from an organizational perspective.
To address this knowledge gap, we designed and developed an interactive patient-centered discharge toolkit (PDTK), a suite of EHR-integrated digital health tools that enabled patients to self-assess and communicate discharge preparedness to their care team and request secure text messaging with a hospital physician after discharge as part of a project funded by the AHRQ. Guided by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework, we conducted a mixed methods study to describe the use, adoption, and perceived utility of the PDTK, as well as the key challenges encountered during implementation [26].

Overview
We used RE-AIM (Table 1), a framework designed to address issues related to the implementation of health services and clinical informatics research [26], to identify research questions to evaluate the feasibility and acceptability of the PDTK (Table  2) in a real-world clinical setting. Specifically, we employed a variety of quantitative and qualitative methods to assess the Reach and potential for Adoption, while identifying barriers to Implementation and strategies to Maintain the PDTK for hospitalized general medicine patients. Effectiveness of the intervention on outcomes will be evaluated in future studies. Table 1. Reach, Effectiveness, Adoption, Implementation, and Maintenance framework: research questions and methods of analysis by dimension.

Methods Dimension
Reach Main results Descriptive analysis of patients approached, and enrolled, including reasons for declining How many patients participate and why do they choose to decline?

• •
Main results Descriptive analysis of patient characteristics and hospitalization metrics from administrative databases, and whether they did or did not submit a checklist, or watch the video What types of patients use the patientfacing PDTK a components?
• Table 3 Effectiveness Future study Interviews at discharge to assess proportion of patients with Patient Activation Measure scores >55 (level 3 or 4) Does the PDTK activate patients at discharge?
Future study Medical record review and phone interviews (30 days after discharge) to determine the proportion of patients with ≥1 unscheduled emergency department visit or readmission Will the PDTK favorably impact health care resource utilization after discharge?
Main results Descriptive analysis of patients' responses to checklist items Can a checklist identify patients' discharge concerns?

Setting and Participants
The PDTK study (Clinicaltrials.gov NCT03116074) was approved by the Partners' institutional review board and was conducted at Brigham and Women's Hospital in Boston, MA.
As part of the PSLL, we integrated a bedside display for patients and clinicians, a patient portal for patients and caregivers, and a safety dashboard for clinicians into our EHR environment (Epic Systems, Inc, Verona, WI) [29,30]. These applications used enterprise data services to obtain clinical data from the EHR in real time [19,[30][31][32][33]. This EHR-integrated digital health infrastructure (Figure 1) served as a platform on which to incorporate enhancements for the independently funded PDTK study. The key focus of the PSLL was to use systems engineering and human factors methods to design, develop, and implement tools to prevent harm in the hospital (eg, falls, catheter-associated urinary tract infections) [29]. The goal of the PDTK study was to improve safety during transitions out of the hospital by designing, developing, and implementing enhancements to the PSLL infrastructure (Table 2) based on clinician end-user requirements and organizational priorities. The PDTK comprised enhancements to each of the 3 components of the PSLL technical infrastructure: bedside display, patient portal, and safety dashboard (see Multimedia Appendix 1).
Each of the 3 study units was codirected by a physician and nurse pair and staffed by its own group of nurses. Patients admitted to each unit were cared for by 1 of 2 geographically localized general medicine teams comprising residents or physician assistants and a supervising hospitalist attending physician [34]. These medical teams rotated approximately every 2 weeks. A few off-service patients (ie, admitted to a service other than general medicine) were admitted to these units under the care of a different attending physician.
Any English-and Spanish-speaking patient admitted to these units under the general medicine service was eligible to participate by using any of the patient-facing PDTK components (checklist, video, and secure messaging). For patients who did not have the capacity to consent (as determined by a member of the care team), a caregiver (a designated health care proxy) could participate on their behalf. Patient or caregiver participants were offered access to an acute care patient portal on either personal devices or study-issued mobile devices (iPad Air, Apple, Inc, Cupertino, CA) as part of the concurrent PSLL study [19]. Any clinician (nurse, resident, physician assistant, and attending) caring for a general medicine patient admitted to these units was eligible to participate: all clinicians had access to EHR-integrated digital health infrastructure as part of the concurrent PSLL study and could therefore access the clinician-facing PDTK components (safety dashboard and secure messaging).

Iterative Refinement of Intervention Components
In previous work, we engaged patient advisors, clinical stakeholders, information system professionals, and quality and safety leaders to identify gaps in discharge processes [19,35]. For this study, during the design and development phase, we conducted informal workflow observations on study units and interviews with stakeholders to identify end-user requirements for addressing these gaps by engaging patients and clinicians in discharge preparation while aligning with key organizational priorities: engaging patients to improve patient satisfaction, improving expected discharge date (EDD) documentation in the EHR, and reducing 30-day hospital readmissions [31]. For example, improving EDD accuracy-defined by our institution as the percent of final EDD entries equal to the actual discharge date (ADD)-was an organizational priority for improving operational throughput. Thus, to ensure timeliness of checklist completion and review of checklist responses by the care team, we enhanced the EHR-integrated patient portal, bedside display, and safety dashboard to improve the visibility of the EDD for both patients and clinicians ( Figure 1). We presumed that the likelihood of checklist submission by patients and review by clinicians would be dependent on where patients were in their hospital course as well as their currently documented EDD.
As in the concurrent PSLL study [27,29,36], we applied user-centered design principles to refine patient-and clinician-facing intervention components ( Figure 1) to ensure that we addressed end-user needs [31]. For the discharge checklist ( Figure 2, left), our goal was to improve structure and organization, validate content, and clarify wording and utility. Key refinements were identified through multiple iterations of the original checklist within our research team (in part based on our experience with a transitions study funded by the Patient-Centered Outcomes Research Institute [37]), 2 sessions with our hospital's patient and family advisory council, and a short pilot in which we administered a paper-based prototype to a convenience sample of 10 hospitalized patients and requested feedback. On the basis of the feedback from unit nurses and patient advisors, we also created a video to help hospitalized patients understand the purpose of completing the checklist to prepare for discharge. To develop the discharge video ( Figure 2, right), we adapted a method previously demonstrated to improve patients' understanding of their medical condition and care plan [38]. Finally, we determined that patients would need to watch the video and complete the checklist via one of several workflows: using the patient portal on a hospital-issued mobile device, using their own mobile device, or having research staff coach patients or caregivers to complete the checklist and then submit responses on their behalf. Next, we made discharge process-based enhancements to the PSLL technical infrastructure integrated with our EHR to ensure that patient-reported information from the checklist would be communicated to the care team as the EDD approached. On the basis of the feedback from patients and clinical unit leadership, we confirmed that patients would want their care team to have access to the checklist responses sufficiently before actual discharge to allow time for the care team to review and address any issues (see Enrollment below). Thus, we developed a checklist submission and review process ( Figure 3) to ensure that checklist responses submitted by patients would be visible for clinicians to review in the EHR in real time.
The discharge checklist ( Figure 2, left), originally created by Coleman [24], was adapted for our institution [4], and further refined into a four-domain, 16-item discharge checklist. Questions were simplified, reordered, and separated into four domains: My Understanding, My Medications, My Self-Care Management, and My Follow-up. Dichotomous checklist responses (Yes/No) were determined to be too strict (ie, patients were uncertain of their response and did not want to check yes or no); thus, a third option (Unsure) and a box for a free-text response was added. A caregiver version to be completed by health care proxies, as well as a Spanish version (approved by our hospital's interpreter services) was created to make the checklist more inclusive of patients who lacked capacity or who did not speak English as their primary language (Multimedia Appendix 2).
The discharge video (Figure 2, right) was developed by creating an English version of a script in which a clinician guided the viewer through each checklist domain. This script was translated into Spanish and approved by our hospital's interpreter services. A mobile device (iPad Air, Apple, Inc) was used to film English and Spanish versions. Video editing software (iMovie, Apple, Inc) was used to produce the videos. The videos were uploaded onto a video-hosting site (YouTube, LLC); hyperlinks to these videos were incorporated into the patient portal and REDCap (Research Electronic Data Capture, Nashville, TN; see Checklist Submission section).

Checklist Submission
The two options available for patients to complete and submit the checklist were as follows: the patient portal or a Web-based REDCap survey [19,39]. REDCap submissions could be completed by patients on a mobile device (via a hyperlink emailed to the patient), or by study staff who would submit the responses on their behalf. Checklist responses submitted via the patient portal were made visible in the safety dashboard via a shared database. Checklist responses submitted via REDCap were routed to the safety dashboard using the REDCap application programming interface and matched to the corresponding patient using key identifiers (medical record number and admission date). In either case, checklist responses were displayed on the safety dashboard for clinicians within 5 min of submission.

Checklist Review
A new discharge column in the safety dashboard displayed key data elements (medical and nonmedical barriers, EDD) from the EHR (Epic Systems, Inc, Verona, WI) using enterprise Web services [27]. EHR data were transformed into clinical decision support using color-coded flags: red=action needed; yellow=risky state; green=guideline compliant; gray=not applicable. For example, a red flag appeared if the current date was past the EDD, no EDD was entered, or the current date was within 24 hours of the EDD, but no checklist had been submitted. A checklist icon appeared on the safety dashboard on successful submission. Checklist items with a no or unsure response were briefly summarized as a keyword on the patient-detail view of the safety dashboard (eg, Meds: Access/Adherence) with a hover-over displaying the specific item answered by the patient (eg, I understand how to get my medications and will take them as prescribed after I leave the hospital). Patients were permitted to enter their mobile phone number during checklist submission to request secure messaging (Imprivata, Lexington, MA) with a hospital physician (attending or senior resident) for up to 7 days after discharge [19,27,40]. A link appeared in the safety dashboard for the clinician to initiate a message thread with the patient.

Patient and Caregiver Enrollment
The intervention went live in December 2017, starting with a 1-month wash-in period in which we debugged various technical components. During business days, recruitment lists were created in which the care unit and patient approach order were randomized to minimize confounding (eg, always starting enrollment on the top floor of the hospital and ending at the bottom). All patients admitted to intervention units for at least 24 hours with EDDs within 24-48 hours were eligible. Research assistants asked the nurses caring for these patients if they were appropriate to approach, temporarily excluding any patient who was nonverbal, incapacitated, or behaviorally not safe. When an eligible patient was confirmed as not capable of participating, the research assistant attempted to identify a caregiver (a designated health care proxy). Patients and caregivers who were not available for recruitment (eg, off unit getting a test) were reapproached later that day or on subsequent days if they still met eligibility criteria. Research assistants then asked eligible patients (or caregivers) to watch the video and complete the checklist (including entering their mobile phone number to request postdischarge messaging) via the patient portal or REDCap workflow. A patient was considered enrolled on successful submission of the checklist.

Measurements and Data Collection
We used mixed methods to evaluate our implementation experience per each RE-AIM dimension (Table 1). Specifically, we used quantitative methods to measure usage and perceived utility of intervention components (Reach, Adoption, and Implementation). We used qualitative methods to assess barriers to and facilitators of Implementation and Maintenance from the patient and clinician perspective.

Usage of Intervention Components (Quantitative)
We captured the number of times the video was watched, the checklist was submitted, and postdischarge messaging was requested by patients. We captured the number and type of patient-reported concerns for each checklist submitted, as well as the number of times clinicians accessed the discharge column on the safety dashboard to view patient-reported concerns and click on the link to initiate postdischarge messaging.

Patient Surveys (Quantitative)
We previously reported usability of the patient portal, which included a discharge module with an earlier, noninteractive version of the checklist [19]. To better understand perceived utility of the PDTK, we asked a convenience sample of enrolled patients to participate in a survey guiding them through each intervention component. The 2-part survey, based on prior work [32], asked participants to rate their perceived readiness for discharge and willingness to self-assess discharge preparedness on a 5-point Likert scale. After watching the video, completing the checklist, and viewing how clinicians could visualize checklist responses in the safety dashboard, participants rated statements about their confidence that the intervention would facilitate self-identification and communication of discharge concerns to their care team.

Patient Interviews (Qualitative)
We verbally consented and conducted semistructured interviews with a convenience sample of English-speaking patients within 24-48 hours of anticipated discharge. Care team members (nurse, physician assistant, and resident or attending) were asked to identify patients within this time frame who were present in their rooms and would be amenable to participating in a brief interview. To reduce sampling bias, potential participants were selected to create a diverse sample based on age, gender, and reason for hospitalization. To minimize selection bias, patients were assured that their participation would not influence their care team's medical decisions, and care teams were not told which patients agreed to participate. All participants completed the checklist and watched the video either before or at the time of the interview. Study staff (DP and NP) trained in qualitative research methods conducted the interviews using a semistructured interview guide that explored (1) patient experiences completing the checklist and viewing the video, (2) addressal of discharge concerns by the care team, and (3) pros and cons of using these tools. Interviews were digitally recorded, transcribed, and reviewed for accuracy, and conducted until thematic saturation was achieved [41].

Clinician Focus Groups (Qualitative)
After the study was completed, we conducted focus groups with physicians, physician assistants, and nurses to assess implementation barriers and facilitators until thematic saturation. Using a structured guide, we asked about EDD entry via the EHR, perception of the checklist workflow, usage of the discharge column of the safety dashboard to review checklist responses and initiate postdischarge messaging, and awareness of patient-facing components (bedside display and patient portal). Focus group discussions were digitally recorded, transcribed, and reviewed for accuracy.

Statistical and Qualitative Analyses
Descriptive statistics were used to report patient demographic and administrative data, quantify patient-reported concerns, calculate the frequency of tool use by patients and clinicians, and quantify survey data. We calculated the proportion of patients (or caregivers) completing the checklist via the patient portal or REDCap. All qualitative data collected from patient and clinician participants were transcribed, openly coded, and analyzed using the constant comparative method [42]. Two researchers (DP and NP) independently coded all transcripts line by line using Word (Microsoft, Inc, Redmond, WA). The code structure was revised as needed to capture novel concepts, adapt, and merge existing concepts; transcripts were coded with each iteration of the codebook, with any discrepancies resolved during consensus meetings. This process was repeated until no novel concepts were identified, at which point, the 2 researchers again independently applied the final code structure to all transcripts. Key themes were identified in a final group consensus meeting with the study staff. Key implementation barriers were identified by study staff via a group consensus approach based on quantitative and qualitative data.

Results
Of 752 patient-admissions, the patient (or caregiver) watched the video and completed the checklist in 416 (55.3%), and the patient/caregiver completed the checklist alone in 94 (12.5%).
Research assistants made 313 attempts at approaching patients in the remaining 242 patient-admissions; however, the patient was unavailable (126), not appropriate per nurse (97), declined to participate (41), did not speak English or Spanish and no caregiver was available (33), did not respond by email when reminded (8), or encountered technical issues (8). The demographic characteristics of the 67.8% (510/752) patient-admissions (480 unique patients) in which a checklist was submitted and the 32.2% (242/752) patient-admissions (238 unique patients) in which the checklist was not submitted are reported in Table 3. In general, those who did not submit a checklist were older, more often Hispanic and non-English speaking, less often privately insured, had higher diagnosis-related group (DRG) weights and longer lengths of stay, and were typically discharged to a destination other than home. Usage of each PDTK component for the 510 patient-admissions in which a checklist was submitted is reported in Table 4. Although the patient was enrolled in the acute care patient portal in 173 (33.9%) of the 510 patient-admissions, the patient portal was used to submit the checklist in 53 (10.4%); the remainder were submitted via REDCap. The checklist was submitted once and 2 or more times in 492 and 18 patient-admissions, respectively. The median (IQR 25,75 ) days from initial checklist submission to EDD and ADD were 1 (1,2) and 2 (1,5), respectively. On average, 4.24 concerns were reported for each checklist submitted: the most commonly entered concerns by patients were about medications (664/2164, 30.68%) and follow-up (656/2164, 30.31%). The EDD was accurate in 307 (60.2%) of the 510 patient-admissions.
Of the 20 patient experience survey participants, 13 (65%) felt well prepared for discharge, and 16 (80%) stated that they would be willing to self-assess discharge preparedness via a checklist. After viewing how the PDTK components functioned, all (100%) completed the checklist, reporting an average of 5.1 concerns (17 understanding of the plan, 30 medications, 21 self-care management, and 34 follow-up); and 7 (35%) requested secure messaging after discharge; 13 (65%) felt that the checklist facilitated self-identification of potential issues before discharge; 15 (75%) believed that their care team would become aware of these issues via the safety dashboard; 10 (50%) felt more confident about what to do to prevent issues after leaving, and 15 (75%) felt confident that they could quickly communicate with a hospital physician via secure messaging should an issue arise postdischarge.
Of the 20 patients approached for semistructured interviews, 12 participated: 7 (58%) were male, 10 (83%) were white, the median age was 70.5 years, and 8 had public insurance. The most common reason for declining to participate was feeling unwell or tired. We identified two overarching themes about discharge preparation: (1) (2) despite perceived communication gaps, patients were confident that their care team would address all of their questions and concerns before discharge (eg, I knew everything was going to be done and things were going to be taken care of, but really, I didn't feel informed, I really didn't). We also identified key themes regarding patient experiences using the checklist and video components of the PDTK (Table 5).
In total, 22 clinicians (8 physicians, 6 physician assistants, 8 nurses; mean age 36.9 years; 14 (14/22, 64%) female) participated in 1 of 3 focus groups from which we identified 3 major themes regarding the safety dashboard component of the PDTK (Table 5): low awareness and variable workflow, lack of optimization, and inconsistent leadership. c Denominator reflects number of patient admissions in which postdischarge messaging was available.  The checklist and video increased understanding of self-care needs and follow-up plans and promoted patient engagement and empowerment in the discharge process.
Patient utility dependent on the timing of administration • • "Well, it was a little unclear given that we're not about to leave. It's hard to report on the process because it hasn't actually happened yet." [Patient] The checklist and video were most useful when administered close to discharge but before a detailed discussion of discharge preparation by a care team member.

Safety dashboard
Low awareness, variable workflow Although clinicians were generally aware, checklist answers were variably viewed on the safety dashboard.
• "When it first rolled out there was a lot of information about it and then it just dropped off, and then the usage dropped off…" [Clinician] • Reinforcement and reminders to use the safety dashboard to review patient-reported discharge concerns were variable. Usage was dependent on senior-level clinician leadership (attending or senior resident). a EDD: expected discharge date.

Principal Findings
We used the RE-AIM framework to evaluate the feasibility and acceptability of a suite of EHR-integrated digital health tools to engage patients, caregivers, and clinicians in discharge preparation. Most patients agreed to watch the discharge video and complete the checklist to self-assess discharge preparedness when coached, and we did not encounter significant technical difficulty in our approach. The patient-facing tools were perceived to be valuable by both patients and clinicians, and most patient-reported concerns submitted via the checklist related to medications and follow-up. Clinician use of the safety dashboard discharge column to view these concerns was modest, mostly due to workflow challenges. A large percentage of patients requested postdischarge messaging, but very few clinicians opted in. Themes identified from our qualitative analysis suggest that timing of administration, additional workflow integration, optimization, and leadership are necessary to promote a more robust adoption of these tools.
We attribute the high rate of patient participation to flexible Web-based workflows and facilitation by research assistants.
First, the iterative process to develop and refine the checklist and video incorporated feedback from patient advisers and institutional stakeholders, resulting in a product that was relevant and understandable to patients. Next, research assistants functioned as discharge advocates for study participants, guiding them through the process of viewing the video, and completing the checklist. This encouraged participation from patients who might otherwise have not been comfortable or motivated to do so independently. In addition, we included a video component and incorporated a checklist submission process that did not depend on the acute care patient portal: REDCap's Web-based workflow was useful as a mobile app prototype and also circumvented key barriers to patient portal enrollment and use during hospitalization, such as patients' dislike of having a separate log-in for the acute care patient portal [19,36]. In contrast, the direct hyperlink to the discharge checklist in REDCap offered more streamlined access (eg, no log-in was needed) for patients, and facilitation by research staff mitigated the perceived burden of submitting the checklist electronically on their own.
Although most patient participants perceived that the intervention would facilitate communication regarding discharge concerns to their care team, often this did not occur because of low uptake of the intervention by clinicians; we attribute this to low awareness of the intervention, inconsistent understanding of its purpose and how to use it, and lack of specificity of patient-reported concerns viewable on the safety dashboard. Specifically, although the safety dashboard discharge column logic was vetted by institutional stakeholders to align with hospital priorities (improving EDD accuracy), clinicians often misinterpreted safety dashboard flag colors (eg, green did not signify safe for discharge) and had a different understanding of responsibility for updating the EDD in the EHR because of inconsistent processes. In addition, if clinicians did not access the detailed view for the patient or click on the flag, they did not see the full text of the flag and what it meant, only its color [43]. Finally, variable use of the discharge column by physicians likely led to poor awareness of the link to initiate a secure messaging thread when requested by patients, and many were resistant to using this feature altogether.
As one of the first reported attempts at engaging patients, caregivers, and clinicians in discharge preparation using a suite of EHR-integrated digital health tools, findings from our implementation study, guided by the RE-AIM framework, offer several instructive lessons (Table 6). First, the optimal timing of when to complete the checklist is paramount. If completed too early, patients perceive less utility, anticipating that the care team would eventually address their concerns. If completed too late, concerns identified from the patients' perspective are less likely to be communicated to clinicians, leading to potential deficiencies or delays in addressing them. Second, dashboard flag changes were not linked to relevant EHR data elements because of competing workflows (eg, a newly created process for documenting and escalating discharge barriers to hospital leadership) and technical limitations (eg, lack of a Web service to retrieve readmission risk scores from the EHR). Rather than linking flag changes to EDD documentation and timing, using the safety dashboard to review patient-reported concerns about discharge might be more clinically meaningful in the context of patient-specific readmission risk scores; this would be more consistent with the overall intent of the safety dashboard as a tool for proactively identifying patients at risk for harm [27,44], Third, improving EDD accuracy via this type of intervention is more likely to be achieved if the responsibility for updating EDD resides with clinicians rather than unit clerks and is clear to all parties. Patients and clinicians will then have more confidence in the EDD displayed on the patient portal, bedside display, and safety dashboard. Still, it is noteworthy that EDD accuracy for enrolled patients (307/510, 60.2%) was marginally higher than the EDD accuracy rate for general medicine (965/1702, 56.70%) as a whole. Finally, secure messaging after discharge clearly requires a better understanding of factors predicting whether clinicians will use this feature and how to incentivize its use. Propose enhancements based on lessons learned from concurrent QI efforts to explain how the use of a checklist can prepare patients for postdischarge care (increasing patient satisfaction, reducing readmission rates) while maintaining or reducing the length of stay (by proactively identifying and overcoming barriers to timely discharge) Poor specificity of patient-reported concerns viewed in the dashboard • Provide a link to discharge checklist questions and patient's responses • Link patient-reported concerns to specific clinical actions (eg, if poor understanding of the main diagnosis, update after visit summary with condition-specific educational materials)

Secure postdischarge messaging
Physician resistance • Frame the initiation of secure messaging thread as an opt-in process • Align with value-based incentives for clinical services (readmissions) • Communicate success stories from early adopters to assuage fears (eg, excessive text messages from patients) Managing patient expectations about whether physicians will initiate secure messaging • Educate patients about the opt-in process for attendings • Encourage patients to request attendings to use this feature for clearly defined reasons (eg, concern about obtaining a key medication)

Limitations
Our study has several limitations. First, it was conducted for general medicine patients at a single institution without a control group; therefore, we could not evaluate the impact of our intervention on clinical outcomes. Similarly, our qualitative analyses were performed on convenience samples of participants, which may limit the generalizability of our findings. Second, we used research assistants to coach patients in submitting the checklist; although dedicated discharge advocates are becoming increasingly common, many institutions lack sufficient personnel [45]. In most hospitals, nursing staff could serve patients in this capacity as they are often the first to identify concerns reported by patients preparing for discharge. Third, we identified disparities among those who submitted and those who did not submit a checklist. Clearly, additional work is needed to address disparities in underrepresented groups to fully evaluate the utility of this intervention in a broader population. Finally, this was a hospital-centric intervention-we did not engage primary care physicians. Although our efforts at postdischarge messaging attempted to bridge the transition from inpatient to ambulatory care, the secure messaging vendor used in this study did not offer the ability to communicate with multiple care team members simultaneously, as we previously described [40]. Nonetheless, seamless communication with key ambulatory clinicians is important during the immediate postdischarge period [46,47].

Conclusions
We believe that EHR-integrated digital health tools such as those we described will become increasingly useful as part of an institutional strategy to engage patients, caregivers, and clinicians in improving discharge safety if they simultaneously address key hospital priorities (eg, improving EDD accuracy and mitigating readmission risk). Currently, we are making further enhancements to the intervention components and their implementation. For example, we are reconfiguring the dashboard discharge column logic to more clearly identify patients at high risk for readmission, which should provide context for the types of concerns patients report after completing the checklist. Exploratory features, such as secure messaging with patients, clearly require further investigation to better characterize patient and clinician perceptions of its value and appropriate use after discharge. However, we believe that many of these features will become increasingly utilized to comply with new regulations (eg, Caregiver Advise, Record, Enable [CARE] Act) [48]. Finally, we plan to conduct rigorously designed studies to evaluate the impact of the PDTK on key outcomes during transitions, such as patient activation at discharge, postdischarge health care resource utilization, and hospital readmissions.