The PubMed, CINAHL, and Embase databases were searched for articles published between 2005 and 2017 using keywords related to patient engagement, electronic health records, patient portals, and their associated subject headings in each database: the full search terms for each database are provided in Multimedia Appendix 1.

Figure 1 shows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram for the systematic search and selection process. Inclusion criteria for articles were (1) written in English, (2) hospital inpatient setting, and (3) patient portals tethered to a hospital EMR. The initial combined database search produced 703 articles, and an additional 16 were identified by scanning their reference lists. After eliminating duplicates, the article abstracts were independently reviewed by three authors to identify articles that did not meet the inclusion criteria. This led to 617 articles being excluded. Full-text screening was conducted for the remaining 102 articles, leading to the identification of 62 articles that did not meet the inclusion criteria. At each stage, the authors met to reconcile, by consensus, any disagreements about article inclusion. An independent coder also coded the 102 articles for inclusion/exclusion using our criteria, and interrater agreement was high (Cohen kappa=.75). In cases of disagreement, we opted to include the article if it addressed a potentially important policy issue (eg, privacy issues, rural/urban divide). To ensure we included as many up-to-date papers as possible, we periodically conducted database searches for new articles after the initial search. This step was performed by just one author because by this stage, the authors had established a well-developed understanding of the inclusion/exclusion criteria. This periodic update, up to August 2018, identified another 18 articles for inclusion.

The included articles were assessed for quality. Two authors independently scored each article’s quality using the most recent version of A Measurement Tool to Assess Systematic Reviews (AMSTAR 2) [21] for review articles and the Quality Assessment Tool for Studies with Diverse Designs (QATSDD) [22] for qualitative, quantitative, and mixed empirical studies. We classified the overall AMSTAR 2– or QATSDD-derived score for each paper on a 5-point scale (very low, low, medium, high, very high), thus establishing the qualities of all articles on one scale. The ratings by the authors and the independent coder were highly correlated (r=.81). A third coauthor then independently reviewed and reconciled the scores. We assigned valence ratings to each article to characterize the overall findings in each article regarding patient portals as positive, negative, or mixed [23].

We analyzed the information extracted from the included articles by categorizing the themes related to the implementation of patient portals into inputs, processes, and outputs. The inputs are the material (eg, hardware and software) and nonmaterial (eg, leadership) components that facilitate or impair the establishment or use of the portal. Processes include the interactions of the users with the portal. Outputs comprise the results of the implementation or the use of the portal. Through the analysis, we identified 14 themes within these three categories, shown in Textbox 1.

Details of the 58 articles included in this review are given in Multimedia Appendices 2 and 3. The majority of the articles (41) are indexed in all three databases; 13 articles are indexed in two databases, and only 4 are in indexed in a single database. The PubMed and Embase databases each index 56 articles while CINAHL indexes 41 articles. Nineteen articles described qualitative studies, 18 quantitative studies, and 9 mixed method studies, while 12 were reviews. Our quality assessment placed 19 articles in the high- or very high–quality categories, 21 in the medium category, and 18 in the low- or very low–quality categories. Twenty-nine articles were assigned mixed valence, 16 articles were assigned positive valence, and 2 were assigned negative valence. Valence was not applicable or could not be drawn in 11 articles (ie, those that examined portals but that were not focused on evaluating them in any way).

There was a spread of articles addressing themes over the inputs, processes and outputs categories (see Multimedia Appendix 3). However, only one-third of the articles (19/58) addressed the full spectrum of categories.

Textbox 2 shows that there was corroboration among studies for some findings (eg, information) but contradicting findings in some themes (eg, health outcomes). We did not observe trends in the findings that were related to study quality or design (ie, there was no association between study quality or study design and reported findings).

Forty articles addressed themes in the inputs category. We identified 22 articles that addressed portal design. In an iterative design approach, feedback from patients, including requests for electronic games and other functions, was used to refine designs. Enhancements included links to medical education, artificial intelligence techniques, and disease-specific design.

As shown in Textbox 2, poor portal designs caused usability difficulties for some patients. Those difficulties were among the barriers to portal use. Textbox 2 also shows that more barriers to portal use than facilitators were identified in the reviewed literature. Various methods were used to train patients and staff to use the portals. However, training methods were not optimal [5,33,44]. For example, in one study, patient training was delivered via an 11-minute video that was not well received, with less than a third (26.3%) watching the entire video [33]. In another study among health care staff, doctors had the lowest confidence in a patient portal, and they doubted their training was sufficient to allow them to effectively use it [5].

Organizational factors (leadership, staff support, and key decisions, etc) was the least addressed theme. However, the findings summarized in Textbox 2 indicate that organizational factors are likely the most crucial in determining whether or how patient portals are implemented.

Thirty-six articles addressed themes categorized as processes. Five articles addressed portal adoption while use was addressed in 18 articles (see Multimedia Appendix 3). Sociodemographic characteristics of patients such as race, age, gender, level of education, and social status were predictors of both portal adoption and use [4,6,33,48]. In addition to sociodemographic factors, there was higher portal use among patients with greater disease severity (eg, advanced cancer) [4], but at any given time, patient condition influenced portal use as described in Textbox 2.

Seventeen articles addressed information and 20 addressed communication. The articles suggested that patients wanted unlimited access to their EMRs, medical education in layman’s terms [24,41], and nonclinical information [19,24,30].

Patient-provider communication, usually in the form of secure messages, is a key feature of inpatient portals [26]. Despite expressions of interest in this feature, actual use was low [30,44]; in a pediatric study with 296 parents participating, only about 6% sent messages to health care teams via the inpatient portal [30]. Most portals enabled patients to send messages to a single mailbox that was accessed by staff members on duty, but some patients wanted to communicate with specific staff members [33]. Similar to adoption and use, sociodemographic factors were predictors of which patients used the messaging feature (see Textbox 2).

The outputs category was addressed by 46 articles, and 24 articles addressed patient engagement. Results of patient engagement were mixed: portals in some studies did not cause statistically significant improvement, but patients in other studies reported that portals enabled better engagement in their care.

User perceptions was the most commonly addressed theme across all categories (35 articles), and Textbox 2 shows that users perceived some functions of portals (eg, access to information) as inadequate but perceived other functions (eg, communication) as very useful. Despite dissatisfaction and concerns with some aspects of the portals, users’ perceptions of the patient portal concept as a whole were mostly positive. For example, in one study, 90% of participants reported overall satisfaction with the portal, 89% thought that the portal reduced errors, and 94% agreed that the portal improved care delivery [30]. In another study, 84% of participants described a patient portal as useful and 90% reported that they would recommend it to their peers [32]. Health care staff appreciated the reasons for implementing patient portals and their own roles and responsibilities in the process, but they stated that they would like to receive sufficient training [5].

Twenty articles addressed the association between patient portals and health outcomes, such as medical errors, readmissions, and mortality. Results were mixed as some studies did not show positive associations between portal use and health outcomes [2,4,20]. For example, a retrospective study of cancer patients found no association between portal adoption or use and adverse events [4] and another retrospective study found no association between portal use and 30-day readmission or inpatient mortality [20]. However, in a third study, active portal use significantly improved glycated hemoglobin levels [54]. Importantly, patient portals in several studies facilitated discovery of EMR errors by patients, particularly medication errors [24,26,30,54-56].

Finally, 7 articles addressed the benefits of patient portals, and Textbox 2 shows that portals offers a wide range of benefits for patient care.

This systematic review examined 58 articles studying inpatient portals. Although there was overlap in the themes reflected in these studies, there was also significant variation in the setting, patient population, software, outcomes assessed, and study methodology, making it hard to come to a definitive conclusion on whether inpatient portals are beneficial. This is further shown by the higher number of included studies that were judged to relay mixed valence than those with positive valence. However, more studies relayed positive than negative valence, showing that patient portals may be beneficial for health care. We discuss below the patient portal input, process and output factors that contribute to this assessment, and the areas of research that need focus in order to improve patient care. Figure 2 provides a visual summary.

When assessing common themes within the inputs category, portal design, usability, and barriers were more widely covered than user training and organizational factors. Many articles showed that good portal design is crucial for usability and adoption by patients. While most studies involved users in the design process, more needs to be done to overcome design-related barriers, particularly for people with low health literacy [6,19,24]. Increasing the use of enabling technology such as voice-commanded digital assistants, artificial intelligence, and natural language processing could make the systems cater to a wider range of audiences. Designing platforms so that they can be displayed on users’ personal mobile devices could be another enabler of portal adoption.

Patients’ concerns regarding privacy and security of their medical information [34,40] are particularly relevant in cases where caregivers need access to patients’ EMRs. Discussion of privacy issues in cases of caregiver access to patient medical information in the literature is limited. In the absence of formal policy for caregiver access, patients may opt to disclose personal log-in details to their caregivers, which is not recommended [26].

The results of user training showed that reliance on a one-size-fits-all approach may not be effective for educating users because of varying preferences. Training could be enhanced by providing information that directly addresses common patient concerns (eg, information security) and health care staff concerns (eg, workflow changes).

Information on the best timing to deliver training material to patients is lacking in the literature. While staff training can be scheduled ahead of deploying a patient portal, patients may only encounter a portal upon hospitalization. Furthermore, the severity of their condition may limit their ability to focus on or understand the training material. Innovation may be needed to inform patients about portal services prior to hospitalization. For example, patients could sign up for outpatient portals which would be similar in design to the inpatient portals to ensure seamless transition between the two portals. Hospitals could also collaborate with medical insurance providers to make training material available to potential patients already signing up for hospital insurance.

Associations between patient sociodemographic characteristics and portal use [4,6,33,48] indicate that patients who are most vulnerable (eg, those with low health literacy or seriously ill) would be least likely to benefit from patient portals. Recruitment and participation bias when engaging participants in user testing mean that actual use by patients may not reflect the use in the initial testing phases. Therefore, health care organizations may need to conduct multiple studies to iteratively address factors that influence portal use within the communities they serve.

The varying preference among patients for level of access to EMRs [6,18,24,26,30] and staff concerns with unlimited patient access to sensitive information (information that could cause anxiety for patients) [24,41,49] present challenges for health care organizations. Health care organizations could fulfill these diverse information needs by flagging sensitive information and warning patients that despite having access to the information, they can opt to receive it in person from a health care team member. Some hospitals already have guidelines for releasing sensitive information to outpatient portals [26], and those guidelines could be adapted to the inpatient setting.

Health care organizations may view patient requests of nonclinical information and functions such as electronic games [18,19,24,30] as fringe requests that could raise developmental costs. However, granting these requests may improve portal adoption and use and overall patient satisfaction. Also, patients have requested background information on their health care staff, but staff perceptions about the disclosure of such information to their patients is not known [19,24]. Further research is needed to uncover staff perceptions in this regard and explore ways of implementing this service.

Portal-based patient-provider communication is potentially beneficial but may also be disruptive [26]. Although staff concerns about potential interruptions caused by constant patient messaging did not materialize [45], in practice, staff members may be overwhelmed by messages at any time. Structured messaging may be a solution that ensures patients communicate only important and relevant information [26]. If health care organizations decide to enforce structured messaging, they should prioritize patient safety and therefore avoid restrictions that could prevent patients reporting genuine concerns.

The literature explored some, but not all, potential outputs of patient portal implementations. Most of the studies assessed implementation of patient portals using interim outcomes such as user perceptions, and few studies addressed important objective outcomes such as length of stay, morbidity, or mortality [2,4,20,54].

Some studies showed no association between portal use and health outcomes such as readmission, adverse events, or mortality [2,4,20]. However, a number of those studies drew their conclusions from retrospective analysis of portal adoption and use data only [4,20]—adoption and frequency of use alone do not provide sufficient information about effective portal use, which may affect outcomes.

Increasing patient engagement is a goal of patient portals, but the engagement of health care staff is also important since they are likely to be approached by the patients with portal-related queries [5,44]. Nurses and doctors should have sufficient knowledge to answer basic questions or appropriately escalate complex questions (eg, to information technology support). Also, nurses may be required to respond to patient-generated messages within the portal. Similarly, doctors’ perceptions of portals are also important, as they may use a portal to communicate with patients [5] and therefore need to be confident with its functions. Theories in health care information technology suggest that user perceptions can predict the acceptance and use of new technologies [58-60]. Therefore, it is essential for hospitals to ensure positive staff attitudes toward patient portals through effective staff training and technical support and incorporating staff needs in portal design and workflows.

Several studies reported that patient portals facilitate patient discovery of errors in EMRs [24,26,30,54-56]. Discussion in the literature of how patients could notify health care providers of such errors is lacking. Further research is needed to establish how patient-discovered errors are reported and to identify optimal reporting methods.

Research in the evaluation of patient portals is also currently limited. Standardized evaluation frameworks and measures are needed to enable better comparisons of patient portal implementation and outcomes in the future.

While an extensive search was undertaken, the majority of the included studies were conducted at single locations, used outcome measures that were not comparable to those used in other studies, and had small sample sizes. That means results of those studies may not be generalizable to other population groups. Also, a number of the studies were conducted in controlled settings, such as closed-door observations and interviews, which would not be representative of hospital settings. The absence of standardized evaluation tools means the results could not be compared or synthesized, and we were thus limited to providing a descriptive summary of findings only. Most studies that addressed user perceptions or patient-reported results depended on the opinions of those who completed end-of-study questionnaires or interviews; such results could be biased as they may lack feedback from participants who felt uneasy about giving negative feedback. Finally, our conceptual framework enabled us to group the findings gleaned from the included articles, although there was some overlap in the categories caused by interdependence in some of the themes.

The review results suggest that the available evidence for inpatient portals is currently immature. Standardized outcomes assessment and more high-quality studies with objective outcomes (length of stay, mortality, and morbidity) are required to fully understand the impact of such portals.