JMIR Publications

Journal of Medical Internet Research

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 23.06.15 in Vol 17, No 6 (2015): June

This paper is in the following e-collection/theme issue:

    Review

    Patient Portals and Patient Engagement: A State of the Science Review

    School of Nursing, University of Pittsburgh, Pittsburgh, PA, United States

    Corresponding Author:

    Taya Irizarry, BSN, MSN

    School of Nursing

    University of Pittsburgh

    336 Victoria Building

    3500 Victoria Street

    Pittsburgh, PA, 15261

    United States

    Phone: 1 412 624 5314

    Fax:1 412 383 7227

    Email:


    ABSTRACT

    Background: Patient portals (ie, electronic personal health records tethered to institutional electronic health records) are recognized as a promising mechanism to support greater patient engagement, yet questions remain about how health care leaders, policy makers, and designers can encourage adoption of patient portals and what factors might contribute to sustained utilization.

    Objective: The purposes of this state of the science review are to (1) present the definition, background, and how current literature addresses the encouragement and support of patient engagement through the patient portal, and (2) provide a summary of future directions for patient portal research and development to meaningfully impact patient engagement.

    Methods: We reviewed literature from 2006 through 2014 in PubMed, Ovid Medline, and PsycInfo using the search terms “patient portal” OR “personal health record” OR “electronic personal health record”. Final inclusion criterion dictated that studies report on the patient experience and/or ways that patients may be supported to make competent health care decisions and act on those decisions using patient portal functionality.

    Results: We found 120 studies that met the inclusion criteria. Based on the research questions, explicit and implicit aims of the studies, and related measures addressed, the studies were grouped into five major topics (patient adoption, provider endorsement, health literacy, usability, and utility). We discuss the findings and conclusions of studies that address the five topical areas.

    Conclusions: Current research has demonstrated that patients’ interest and ability to use patient portals is strongly influenced by personal factors such age, ethnicity, education level, health literacy, health status, and role as a caregiver. Health care delivery factors, mainly provider endorsement and patient portal usability also contribute to patient’s ability to engage through and with the patient portal. Future directions of research should focus on identifying specific populations and contextual considerations that would benefit most from a greater degree of patient engagement through a patient portal. Ultimately, adoption by patients and endorsement by providers will come when existing patient portal features align with patients’ and providers’ information needs and functionality.

    J Med Internet Res 2015;17(6):e148

    doi:10.2196/jmir.4255

    KEYWORDS



    Introduction

    Patient Engagement and Patient Portals

    Patient engagement has been identified as an essential dimension of the multifaceted solution to the cost/quality crisis in US health care. The patient-centric definition of patient engagement by the Agency for Healthcare Research and Quality (AHRQ) is “the involvement in their own care by individuals (and others they designate to engage on their behalf), with the goal that they make competent, well-informed decisions about their health and health care and take action to support those decisions” [1]. AHRQ also defines patient engagement from a systems perspective as “a set of behaviors by patients, family members, and health professionals and a set of organizational policies and procedures that foster both the inclusion of patients and family members as active members of the health care team and collaborative partnerships with providers and provider organizations” [1].

    Currently, there is an increasing awareness of health care system’s responsibility to provide easily accessible ways for patients to be engaged in their own care by creating effective partnerships that lead to the patient’s ability to make competent and well-informed decisions [2]. While an electronic personal health record (ePHR) tethered to an electronic health record (EHR), also known as a patient portal, is currently recognized as a promising mechanism to support greater patient engagement, questions remain about how health care leaders, policy makers, and designers can encourage adoption by both providers and patients and what factors might contribute to sustained utilization.

    Definition and Background of Patient Portals

    An ePHR that directly links, or is “tethered”, to an EHR is most commonly referred to as a patient portal. In general, patient information from the EHR such as the problem list, allergies, and lab test results populate the patient portal. In some instances, patients may enter data to populate the EHR. In contrast, an untethered ePHR is under the control of the patient. This means an individual manually enters all information or grants permission for the information to be transferred to the ePHR, from a specific source like a laboratory or pharmacy, and determines who will have access. Thus, the value of an untethered ePHR is determined by a person’s willingness to manage and maintain their ePHR information. Because there is little that health care organizations can do to initiate patient engagement using an untethered ePHR, this literature review is focused exclusively on the patient portal, directly linked to an EHR.

    Patient portals were introduced and adopted by a few large health care organizations in the late 1990s (eg, MyChart at the Palo Alto Medical Foundation and Indivo at Boston Children’s Hospital) [3,4]. However, patient portals did not gain widespread use until 2006 when several initiatives coincided, including the launch of ePHRs by Microsoft and Google, the awarding of Centers for Medicare and Medicaid Services (CMS) contracts to private firms to conduct feasibility studies of ePHRs using existing claims data from Medicare programs, and Blue Cross and Blue Shield Association and America’s Health Insurance Plans’ announcement to develop data-sharing programs that would ultimately support ePHR development [5]. These initiatives also coincided with the broad social movement towards adoption and daily use of powerful information and communication sharing tools such as smartphones and social media, illustrating the readiness of the general population to embrace technology in a new socially interactive way.

    The current principal driver of patient portal development is the meaningful use (MU) criteria of the CMS EHR incentive program [6]. Features mandated by MU that directly relate to patient portal functionality include providing (1) a clinical summary to the patient after each visit, (2) secure messaging (SM) between patient and provider, (3) ability to view, download, and transmit personal health record data, (4) patient specific education, (5) patient reminders for preventative services, and (6) medication reconciliation [7]. While these criteria clearly outline tasks and goals, they do little to reflect the value proposition to the end users (patients and providers) or the steps required to engage patients in a sustained and relevant way. Therefore, an aim of this review was to explore the current research addressing the encouragement and support of patient engagement through the patient portal.


    Methods

    Search Strategy

    Due to the advances in technology and consumer readiness in the mid-2000s, the review was limited to recent literature to better reflect current trends in design, functionality, and perceived user readiness of patient portals. We reviewed literature from 2006 through 2014 in PubMed, Ovid Medline, and PsycInfo using the search terms “patient portal” OR “personal health record” OR “electronic personal health record”. Bibliographies and the literature reviews from these sources were used to identify additional studies [8,9]. Initial inclusion criteria were (1) original, peer-reviewed, qualitative, and quantitative research of tethered ePHRs or patient portals, (2) English language, and (3) available in full text. The final inclusion criterion was that the studies reported on the patient experience and/or ways that patients may be supported to make competent health care decisions and act on those decisions using patient portal functionality. Studies were not targeted to any particular patient subgroup, disease, or clinical setting.

    Of the 440 articles identified by the search, 176 were excluded based on title and abstract. Further review based on the final inclusion criterion resulted in 120 articles, which were reviewed in depth (see Multimedia Appendix 1 for summaries of each). Excluded articles focused on the provider perspective only, technicalities of patient portal implementation (eg, policy issues, safety, security), implications for Health Information Exchange, economics impacts, or the utility of patient portal data for research purposes (see Figure 1).

    Figure 1. Literature review flow chart.
    View this figure

    Results

    Overview

    We grouped the studies into five major topics based on the research questions, explicit and implicit aims of the studies, and related measures addressed. The topics identified included patient adoption, provider endorsement, health literacy, usability, and utility (Table 1). Of the 120 articles that were reviewed, 66 (55.0%) were non-experimental descriptive, 26 (21.7%) were qualitative or mixed-methods, 14 (11.7%) were randomized controlled trials, 10 were pilot studies or case reports (8.3%), and 4 were cohort studies (3.3%) (Table 2). Only 11 articles explicitly identified a guiding theoretical framework, with the Chronic Care Model being the most common among them. The year 2011 was a turning point in the number of published articles, which coincides with the initiation of CMS EHR incentives program. The topical areas that showed the greatest increase in volume were patient adoption and utility (Table 1). See Multimedia Appendix 1 for a brief description of each article and the topical areas addressed. The following section describes each topical area and discusses relevant implications for research, development, and implementation of patient portals.

    Table 1. Summary of articles on categories of patient portals for patient engagement.
    View this table
    Table 2. Levels of evidence adapted from Melnyk & Fineout-Overholt, 2005.
    View this table

    Patient Adoption

    Before a patient portal can serve as a tool for individuals to become more engaged and involved in their own care, patients must first adopt it. CMS 2014 stage 2 MU regulations define adoption in terms of institutional reporting for reimbursement and require that 5% of the institutions’ patient population (1) download or view electronic health information and (2) use secure electronic messages (eg, email) [6]. However, in our review, various operational definitions of adoption were used. For example, many observational studies used usage data of the initial login to the patient portal site to represent adoption; others used data from surveys about patients’ intention to use the portal. Several randomized controlled trials (RCTs) used rates of patient portal intervention adherence to study protocol to define adoption, and for some of these trials, those who completed the studies were considered adopters; in others, adoption was defined as the frequency of intervention use.

    Of the 62 articles [5,10-70] that focused on or described patient portal adoption as part of the report, six RCTs included detailed descriptions of intervention group participants who completed the study (and therefore were considered adopters) in comparison to those who did not. We found 12 qualitative or mixed-method studies that collected data about adoption from patients through focus groups or semistructured interviews; 21 studies focused on interest and barriers to adoption for specific populations or patient portal functions (eg, elderly, safety-net, human immunodeficiency populations, secure messaging, prescription refills).

    The term “digital divide” is often used to describe major potential barriers to access of electronic tools such as a patient portal and refers to disparities among subgroups based on access to the Internet and computer literacy. However, this term does not encompass the many other factors that may contribute to adoption such as language barriers, age, race and ethnicity, social economic status, and level of patient activation [32,50,54,71]. Several studies examining adoption have shown that ethnic minorities (African American, Latino, Asian) and patients who are younger (under 35 years), healthier, and less educated were less likely to adopt patient portals [15,55,72]; however, results are mixed regarding gender differences [50,63]. People with disabilities and chronic conditions, frequent users of health care services, and caregivers of elderly parents or children tend to have the most interest in patient portals [28,50,62,73]. Other important factors of patient portal adoption include provider acceptance and promotion, and usability of the patient portal interface including ease of registration, navigation, and perceived privacy and security [18-20,74].

    Provider Endorsement

    Provider endorsement and continued engagement with the patient portal have been identified as important factors in a patient’s decision to adopt and continue to use the patient portal functions to achieve and sustain anticipated positive outcomes [19,75]. Of the 8 articles that addressed physician endorsement [12,19,34,76-80], 5 studies were qualitative or mixed-method studies, and one RCT included a retrospective survey of physicians’ use and satisfaction.

    Four of the studies sought to capture attitudes of clinicians towards patient portals prior to having firsthand experience interacting with them. Prior to actual use of patient portals, clinicians expressed concerns related to patient engagement including: the potential for inducing patient anxiety regarding test results; the accuracy of patient entered data; the potential liability for tracking and acting on critical clinical information, such as blood glucose levels and blood pressure readings; implications for changes in the patient-provider relationship; and the anticipated increased workload [34,77,78,81].

    Retrospective studies showed that the pre-portal concerns regarding patient anxiety about test results were not justified as demonstrated by numerous patients who found the test result feature one of the most useful [82]. In addition, while perceived increases in workload and duration of clinic visits varied among studies, clinicians believed patients were more interested in participating in their care and found that verifying the additional information in the patient portal provided during face-to-face visits was helpful, thus eliminating the accuracy concern [19]. Overall, the workflow of individual providers and the health care team as a whole, including nurses, pharmacists, support staff, and physicians, must be adapted in order to incorporate patient portal functionality, and the patient engagement it allows, into the delivery of preventative services and illness management processes [45].

    Health Literacy

    The definition of health literacy developed for the National Library of Medicine and used by the Healthy People 2010 initiative is “the degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions” [83]. Of the 16 studies that specifically addressed health literacy [11,14,30,40,42,64,65,74,84-91], the majority included self-reported health literacy measures via survey questions or open-ended questions; only Noblin et al (42) and Taha et al [91] included validated health literacy measures. Four studies [64,85,88,91] identified conceptual knowledge, numeracy, and computer skills as particularly important literacy factors that contributed to successful patient engagement via a patient portal.

    Noblin et al [42] found that 65% of participants who intended to adopt the outpatient clinic’s patient portal had a higher eHealth literacy score than those who were not interested in patient portal adoption. Taha et al [91] results indicated that if health texts involved numeric concepts, users encountered problems, even if they were considered to have “adequate” health literacy. These studies underscore the importance of evaluating health literacy and health numeracy separately in order to identify specific risk factors and design flaws that could impact patient comprehension and ultimately jeopardize the accuracy of patient input and interpretation of results.

    Four studies directly addressed the impact of health literacy of intended users on the successful completion of specific tasks [64,84,88,92]. Results showed that patients responded better when medical jargon and abbreviations were translated into “patient friendly” language. These results echo Haggstrom et al [85] and Monkman & Kushniruk’s [88] findings of the dangers of low health and computer literacy to safe and effective use of patient portals.

    Schnipper et al [92] and Sox et al [84] revealed that, despite patient involvement in early design and testing of patient portals, subsequent scenario-based usability testing uncovered navigation difficulties primarily due to the unfamiliarity with complex medical language and confusion of how and when to correct identified errors. Monkman & Kushniruk [88] suggest that including health literacy assessments in usability testing of consumer health information systems, such as patient portals, would inform the design of systems for better navigation, data input, and conceptual understanding of health information included throughout the patient portal.

    Monkman & Kushniruk [88] also proposed a specific heuristic for health literacy whose purpose is to identify and categorize when clinical information within the patient portal would most likely be misunderstood by a layperson who does not possess a health care background. This study, along with several other qualitative studies showed that specific health topics (eg, medications, lab results, and allergies) required extra attention to designing with health literacy considerations in mind [45,89,93]. Proposed navigation and aiding tools that increased patients’ ability to understand their personal health information more fully include integrating links to definitions of terms and detailed explanations, using movies and illustrations, substituting lay language for medical terminology and using graphs to track trending data, such as blood pressure and blood glucose levels [84,85,94].

    Usability

    Usability testing is the term used to describe the assessment of how easy a user interface is to operate. The word “usability” also refers to methods for improving ease of use during the design process [95]. One such method is heuristic evaluation, a method of testing a preliminary prototype by examining the interface and judging its compliance with recognized usability principles (ie, “heuristics”). Further iterative usability testing is accomplished using a series of prototypes and participatory scenario-based and “think-aloud” sessions with intended users in order to redesign the interface and workflows to better match user needs and preferences. Early usability testing, and its role in patient portal design, is important because it directly impacts whether or not a patient can easily adopt a patient portal. It also impacts the ability of the user to successfully navigate portal functions, accurately input information, and comprehend the information presented, ultimately contributing to its usefulness as a tool for patient engagement.

    Of the 20 studies that addressed usability of patient portals, 6 performed some form of heuristic and usability testing with objective observation and various forms of “think aloud” sessions [25,84,85,92,94,96]. Only Schnipper et al [92] included usability testing of both the clinician and patient interfaces. The remaining 14 studies assessed users’ subjective satisfaction and ease of use with questionnaires and/or interviews to evaluate overall adoption and utilization [11,38,45,47,48,64,65,73,82,88,89,91,97,98].

    Schnipper et al [92] addressed the needs of both end users (ie, clinicians and patients) in the usability testing of a medication management module embedded within the patient portal. The study highlighted the need for end user-specific interfaces and functionality in order to make the user experience easier and more efficient, thus demonstrating its value and promoting sustained use. For patients, this meant striking a balance between free-text, structured, and coded data fields in order to leverage the usefulness of patient-entered data without confusing or overwhelming patients. For example, drop-down menus and scrolls bars were found to be less confusing and more efficient than dynamic text boxes that would react to the word being typed when inputting data, such as medications and allergies. In the case of clinicians, this meant integrating the clinician side of the application with their workflow so that clinicians could verify and correct patient-entered data while simultaneously facilitating the flow of that data into the EHR.

    Much of the literature surrounding usability confirms that adoption and sustained use of technology are directly related to ease of navigation and the perceived usefulness of the available information [99]. While nearly all the patient portal usability studies that used subjective assessments showed positive results for ease of use and satisfaction, the in-depth objective usability studies were more effective at uncovering a variety of barriers to safe and effective use.

    Utility

    Utility refers to the availability of needed features. Utility and usability are equally important and together determine whether something is useful [99]; 76 studies focused in some way on patient portal utility [5,12,13,15,19,22,23,25-27,30,34,37,41,44, 47,52,53,56,57,59,60,64,65,69,70,79,82,84-87,89,90,92,96,98,100-137]. The majority of descriptive, qualitative, and mixed-method studies focused on eliciting patient preferences for specific functions. Patients preferred functions that offered convenience, such as an easy way to contact and communicate with providers, order prescription refills, and access multiple family medical records. Easy-to-read, printer-friendly summaries were also viewed as helpful for sharing information with family members and providers who did not have patient portal access. The top two patient portal qualities that were deemed most utilitarian for patients were personalization and collaborative communication between patients and providers [67,138].

    Personalization

    While numerous descriptive and qualitative studies attest to the desire for personalized patient portal functionality, there is little research about what kind of personalization would lead to greater patient engagement. Currently, the greatest research focus is on chronic disease medication management and preventative services. Only 3 RCTs specifically tested the efficacy of patient-tailored interventions [13,30,90]. Grant et al [13] provided patient-tailored decision support and enabled the patient to author a “Diabetes Care Plan” for electronic submission to the physician prior to upcoming appointments. This intervention led to increases in pre-visit use of the patient portal and increased rates of diabetes-related medication adjustment at 12 months. Krist et al [62] provided a personally tailored list of prevention recommendations and found that at 16 months, 1 in 4 users were up-to-date on all preventive services—nearly double that of non-users. Sequist et al [30] sent personalized electronic messages that included (1) alerts for overdue health screenings and information on screening options, (2) a mechanism for patients to submit requests to schedule screening examinations, and (3) a link to a Web-based tool for patients to assess their personal risk of colorectal cancer. Findings showed that screening rates were significantly higher at 1 month for patients who received electronic messages than for those who did not, but the difference was no longer significant at 4 months.

    Collaborative Communication

    Collaborative communication refers to the ability for patients and providers to share timely and pertinent information, enabling patients to participate as active members of the care team beyond the hospital or clinic setting. SM and medication reconciliation are the two most common patient portal functions that offer the opportunity for such communication. Both functions also pose the greatest potential changes to provider workflow and overall impact on the patient-provider relationship.

    For example, the difficulty aligning information management tools with current provider workflow and care delivery priorities was highlighted in a study of an interactive medication reconciliation module that emailed primary care physicians when a patient added or changed information [106]. Results showed that patients were willing and able to annotate their medication list, offering the most up-to-date and complete information, but email notifications were ineffective at prompting providers to update the EHR medication list outside of a clinic visit [106]. Thus, while the notion of designing patient portals to support patient involvement in their care, such as opportunities for their participation in medication reconciliation, shows promise, their effectiveness will depend on the ability to better incorporate these functions into provider workflow and delivery of care.

    Other implications of electronic forms of communication via a patient portal are the potential to improve efficiency by way of substituting SM for face-to-face encounters and using SM reminders to decrease missed appointments and promote timely preventative care. However, research shows mixed results leading researchers to believe that the relationship between SM and utilization is more complex than the simple substitution of online for in-person care suggests. For example, while an earlier study at Kaiser Permanente showed a decrease in face-to-face encounters after the initiation of SM [22], a subsequent study in a different Kaiser region showed the opposite effect [115]. A study done at the Mayo Clinic, aimed at clarifying this discrepancy, focused on frequency of messages, long-term use, and importance of SM among certain subgroups [121], which showed neither an increase nor decrease in face-to-face provider visits with the use of SM.

    SM is also being used as a one-way communication tool to deliver reminders for preventative care and appointments. A 2011 study at seven Duke medical clinics showed that email reminders, in combination with scheduling functionality within the patient portal, demonstrated significant declines in “no-shows” [27]. A meta-analysis and systematic review by Guy et al [139] demonstrated a substantial increase in the likelihood of attending clinic appointments when patients received SM reminders. Perhaps the most encouraging results with SM were the large reduction in missed appointments among historically disadvantaged groups, such as Medicaid recipients, the uninsured, and black patients [27].

    SM reminders via email have also been shown to be generally successful at encouraging higher rates of preventative services use. For example, a multi-practice randomized controlled trial showed improvement in the rates of certain preventive screenings and vaccinations, but preventative services as a whole were not impacted [113]. Findings suggest that SM reminders are most effective when they are tailored to the population and context, thus targeting specific goals such as herpes zoster vaccinations for older adults, or pediatric preventative care visit reminders for parents [119,129].


    Discussion

    Principal Findings

    The current principal driver of patient portal development is CMS and Medicaid EHR incentive program meaningful use (MU) criteria [6]. While MU criteria clearly outline requirements of basic functionality and targeted adoption rates, they do not delineate the steps or features required to engage patients in a sustained and relevant way. Presently there is no clear definition of patient portal adoption beyond the minimum use requirements outlined in the MU criteria. However, in order for health care institutions to track the success of patient portals in terms of patient engagement, a multi-dimensional definition of portal adoption should include both motivating factors for initiation and use over time A definition of this kind would inform a set of universal quality and efficiency reporting measures beyond the current minimal MU criteria to include more relevant patient engagement data.

    Current research has demonstrated that patients’ interest and ability to use patient portals is strongly influenced by personal factors such age, ethnicity, education level, health literacy, health status, and role as a caregiver. Health care delivery factors, mainly provider endorsement and patient portal usability, also contribute to patients’ ability to engage through and with the patient portal.

    While health literacy has been identified as an important factor in the successful use of patient portals, few studies have used validated health literacy measures, making it difficult for future research to build on the findings. Research demonstrates that specific aspects of health literacy, mainly numeracy and familiarity with medical terminology, greatly impact the ability of patients to accurately input data and interpret the information provided in the patient portal. The direct relationship between health literacy and effective use of the patient portal supports the argument for the use of specific health literacy heuristics as part of overall usability testing.

    Research also demonstrates that objective testing (as opposed to solely subjective) should also be a part of patient portal usability testing. Although objective usability testing is expensive and time consuming, studies demonstrate the need for continued work in this area in order to ensure patient portal interfaces promote patient comprehension and data entry accuracy. The promotion of content accuracy and patient comprehension impacts the overall usefulness of the information for both patients and providers.

    The perceived usefulness of patient portals from the providers’ perspectives cannot be underestimated. Provider endorsement is one of the most influential factors impacting patients’ initial adoption, as well as its continued use as a tool for collaborative communication [20]. Yet, current research demonstrates the difficulty in aligning information management tools, such as the patient portal, with current provider workflow and care delivery priorities.

    While current development and research is focused on demonstrating feasibility and efficiency of medication reconciliation and SM reminders, the research has revealed roadblocks to successful implementation rooted in the lack of provider workflow adaptations A greater understanding of the essential adjustments in provider workflow, including potential changes in the roles and responsibilities of the care team overall, is necessary in order to translate findings into practice. Few studies have focused on exploring how patient portal use should unfold within the context of the patient-provider interaction, or how it might impact the overall organization and workflow of the health care team including potential liability concerns, reimbursement, and relationships with patients.

    Ultimately, successful implementation requires health care institutions to invest time and resources to systematically assess the health needs of their specific patient and caregiver populations, their individual stages of readiness to adopt a patient portal, and the types of assistance needed to do so [140]. Ideally, interactive sites would collect information on individuals’ health, health behaviors and personal goals, and assess health literacy and functional ability, which would then inform the adaptation of the patient portal to accommodate the needs of the individual and/or what additional or alternative resources may be useful [2]. Such adaptations include personalized content and tailored data presentations specifically designed to enhance interpretation and comprehension of key personal health concerns and timely and pertinent action steps.

    In addition, external environmental and contextual factors, such as distance between patient and clinic, and complexity and trajectory of health concerns, may impact which form of access is preferred for a specific person, provider, location, and situation. Future directions of research should focus on identifying specific populations and contextual considerations that would benefit most from a greater degree of patient engagement through a patient portal. This information could then lead to the creation of health care service policies that promote the use of a patient portal by both providers and patients within the most appropriate settings.

    Conclusions

    If institutions are to engage patients via the patient portal in a way that encourages them to become active members of the care team, support their competence in making health-related decisions, and help them to act on those decisions, institutional leaders must consider the contributing factors that impact efficacy and sustained use of patient portals. According to this review, these factors include attention to the topical areas of patient adoption, provider endorsement, health literacy, usability, and utility. Ultimately, adoption by patients and endorsement by providers will come when existing patient portal features align with patients’ and providers’ information needs and functionality. Conceptualizing patient portals as a dynamic component of the patient-provider relationship and health care delivery system as a synergetic whole, rather than an isolated repository of information or a set of disconnected functions meant to collect patient data for provider use, may help to inform future research, improve patient portal design, and efforts to promote adoption and effectiveness.

    Acknowledgments

    TI was funded on T32: Technology Research in Chronic and Critical Illness (2NR008857, Erlen & DeVito Dabbs, Directors). The funding agency had no role in the preparation, review, or approval of the manuscript. The opinions, results, and conclusions reported in this paper are those of the authors and are independent of the funding source.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    A brief description of each article and the topical areas it addresses.

    PDF File (Adobe PDF File), 306KB

    References

    1. Rockville MD, Maurer M, Dardess P, Carman KL, Frazier K, Smeeding L. Guide to Patient and Family Engagement: Environmental Scan Report. 2012.   URL: http://www.ahrq.gov/research/findings/final-reports/ptfamilyscan/ptfamilyscan.pdf [accessed 2015-01-17] [WebCite Cache]
    2. Kupchunas WR. Personal health record: new opportunity for patient education. Orthop Nurs 2007;26(3):185-91; quiz 192. [CrossRef] [Medline]
    3. Halamka JD, Mandl KD, Tang PC. Early experiences with personal health records. J Am Med Inform Assoc 2008;15(1):1-7 [FREE Full text] [CrossRef] [Medline]
    4. Mandl KD, Simons WW, Crawford WCR, Abbett JM. Indivo: a personally controlled health record for health information exchange and communication. BMC Med Inform Decis Mak 2007;7:25 [FREE Full text] [CrossRef] [Medline]
    5. Weitzman ER, Kaci L, Mandl KD. Acceptability of a personally controlled health record in a community-based setting: implications for policy and design. J Med Internet Res 2009;11(2):e14 [FREE Full text] [CrossRef] [Medline]
    6. Wright A, Feblowitz J, Samal L, McCoy AB, Sittig DF. The Medicare Electronic Health Record Incentive Program: provider performance on core and menu measures. Health Serv Res 2014 Feb;49(1 Pt 2):325-346 [FREE Full text] [CrossRef] [Medline]
    7. Ahern DK, Woods SS, Lightowler MC, Finley SW, Houston TK. Promise of and potential for patient-facing technologies to enable meaningful use. Am J Prev Med 2011 May;40(5 Suppl 2):S162-S172. [CrossRef] [Medline]
    8. Kim J, Jung H, Bates DW. History and Trends of "Personal Health Record" Research in PubMed. Healthc Inform Res 2011 Mar;17(1):3-17 [FREE Full text] [CrossRef] [Medline]
    9. Goldzweig CL, Orshansky G, Paige NM, Towfigh AA, Haggstrom DA, Miake-Lye I, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013 Nov 19;159(10):677-687. [CrossRef] [Medline]
    10. Weingart SN, Rind D, Tofias Z, Sands DZ. Who uses the patient internet portal? The PatientSite experience. J Am Med Inform Assoc 2006;13(1):91-95 [FREE Full text] [CrossRef] [Medline]
    11. Keselman A, Slaughter L, Smith CA, Kim H, Divita G, Browne A, et al. Towards consumer-friendly PHRs: patients' experience with reviewing their health records. AMIA Annu Symp Proc 2007:399-403 [FREE Full text] [Medline]
    12. Wald JS, Burk K, Gardner K, Feygin R, Nelson E, Epstein M, et al. Sharing electronic laboratory results in a patient portal--a feasibility pilot. Stud Health Technol Inform 2007;129(Pt 1):18-22. [Medline]
    13. Grant RW, Wald JS, Schnipper JL, Gandhi TK, Poon EG, Orav EJ, et al. Practice-linked online personal health records for type 2 diabetes mellitus: a randomized controlled trial. Arch Intern Med 2008 Sep 8;168(16):1776-1782 [FREE Full text] [CrossRef] [Medline]
    14. Greenhalgh T, Wood GW, Bratan T, Stramer K, Hinder S. Patients' attitudes to the summary care record and HealthSpace: qualitative study. BMJ 2008 Jun 7;336(7656):1290-1295 [FREE Full text] [CrossRef] [Medline]
    15. Lafky DB, Horan TA. Prospective personal health record use among different user groups: results of a multi-wave study. In: Proceedings of the 41st Annual Hawaii International Conference on System Sciences.: IEEE; 2008 Presented at: 41st Annual International Conference on System Sciences; Jan. 7-10, 2008; Hawaii. [CrossRef]
    16. Zickmund SL, Hess R, Bryce CL, McTigue K, Olshansky E, Fitzgerald K, et al. Interest in the use of computerized patient portals: role of the provider-patient relationship. J Gen Intern Med 2008 Jan;23 Suppl 1:20-26 [FREE Full text] [CrossRef] [Medline]
    17. Chen C, Garrido T, Chock D, Okawa G, Liang L. The Kaiser Permanente Electronic Health Record: transforming and streamlining modalities of care. Health Aff (Millwood) 2009 Apr;28(2):323-333 [FREE Full text] [CrossRef] [Medline]
    18. Roblin DW, Houston TK, Allison JJ, Joski PJ, Becker ER. Disparities in use of a personal health record in a managed care organization. J Am Med Inform Assoc 2009;16(5):683-689 [FREE Full text] [CrossRef] [Medline]
    19. Wald JS, Businger A, Gandhi TK, Grant RW, Poon EG, Schnipper JL, et al. Implementing practice-linked pre-visit electronic journals in primary care: patient and physician use and satisfaction. J Am Med Inform Assoc 2010;17(5):502-506 [FREE Full text] [CrossRef] [Medline]
    20. Wald JS. Variations in patient portal adoption in four primary care practices. AMIA Annu Symp Proc 2010;2010:837-841 [FREE Full text] [Medline]
    21. Wen KY, Kreps G, Zhu F, Miller S. Consumers' perceptions about and use of the internet for personal health records and health information exchange: analysis of the 2007 Health Information National Trends Survey. J Med Internet Res 2010;12(4):e73 [FREE Full text] [CrossRef] [Medline]
    22. Zhou YY, Kanter MH, Wang JJ, Garrido T. Improved quality at Kaiser Permanente through e-mail between physicians and patients. Health Aff (Millwood) 2010 Jul;29(7):1370-1375 [FREE Full text] [CrossRef] [Medline]
    23. Nazi KM. Veterans' voices: use of the American Customer Satisfaction Index (ACSI) Survey to identify My HealtheVet personal health record users' characteristics, needs, and preferences. J Am Med Inform Assoc 2010;17(2):203-211 [FREE Full text] [CrossRef] [Medline]
    24. Kahn JS, Hilton JF, Van NT, Leasure S, Bryant KM, Hare CB, et al. Personal health records in a public hospital: experience at the HIV/AIDS clinic at San Francisco General Hospital. J Am Med Inform Assoc 2010;17(2):224-228 [FREE Full text] [CrossRef] [Medline]
    25. Greenhalgh T, Hinder S, Stramer K, Bratan T, Russell J. Adoption, non-adoption, and abandonment of a personal electronic health record: case study of HealthSpace. BMJ 2010;341:c5814 [FREE Full text] [Medline]
    26. Ancker JS, Barrón Y, Rockoff ML, Hauser D, Pichardo M, Szerencsy A, et al. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med 2011 Oct;26(10):1117-1123 [FREE Full text] [CrossRef] [Medline]
    27. Horvath M, Levy J, L'Engle P, Carlson B, Ahmad A, Ferranti J. Impact of health portal enrollment with email reminders on adherence to clinic appointments: a pilot study. J Med Internet Res 2011;13(2):e41 [FREE Full text] [CrossRef] [Medline]
    28. Goel MS, Brown TL, Williams A, Cooper AJ, Hasnain-Wynia R, Baker DW. Patient reported barriers to enrolling in a patient portal. J Am Med Inform Assoc 2011 Dec;18 Suppl 1:i8-12 [FREE Full text] [CrossRef] [Medline]
    29. Goel MS, Brown TL, Williams A, Hasnain-Wynia R, Thompson JA, Baker DW. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med 2011 Oct;26(10):1112-1116 [FREE Full text] [CrossRef] [Medline]
    30. Sequist TD, Zaslavsky AM, Colditz GA, Ayanian JZ. Electronic patient messages to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med 2011 Apr 11;171(7):636-641 [FREE Full text] [CrossRef] [Medline]
    31. Zulman DM, Nazi KM, Turvey CL, Wagner TH, Woods SS, An LC. Patient interest in sharing personal health record information: a web-based survey. Ann Intern Med 2011 Dec 20;155(12):805-810. [CrossRef] [Medline]
    32. Sarkar U, Karter AJ, Liu JY, Adler NE, Nguyen R, López A, et al. Social disparities in internet patient portal use in diabetes: evidence that the digital divide extends beyond access. J Am Med Inform Assoc 2011 May 1;18(3):318-321 [FREE Full text] [CrossRef] [Medline]
    33. North F, Hanna BK, Crane SJ, Smith SA, Tulledge-Scheitel SM, Stroebel RJ. Patient portal doldrums: does an exam room promotional video during an office visit increase patient portal registrations and portal use? J Am Med Inform Assoc 2011 Dec;18 Suppl 1:i24-i27 [FREE Full text] [CrossRef] [Medline]
    34. McInnes DK, Solomon JL, Bokhour BG, Asch SM, Ross D, Nazi KM, et al. Use of electronic personal health record systems to encourage HIV screening: an exploratory study of patient and provider perspectives. BMC Res Notes 2011;4:295 [FREE Full text] [CrossRef] [Medline]
    35. Levy MA, Giuse DA, Eck C, Holder G, Lippard G, Cartwright J, et al. Integrated information systems for electronic chemotherapy medication administration. J Oncol Pract 2011 Jul;7(4):226-230 [FREE Full text] [CrossRef] [Medline]
    36. Jung C, Padman R, Shevchik G, Paone S. Who are portal users vs. early e-Visit adopters? A preliminary analysis. AMIA Annu Symp Proc 2011;2011:1070-1079 [FREE Full text] [Medline]
    37. Nagykaldi Z, Aspy CB, Chou A, Mold JW. Impact of a Wellness Portal on the delivery of patient-centered preventive care. J Am Board Fam Med 2012 Apr;25(2):158-167 [FREE Full text] [CrossRef] [Medline]
    38. Nielsen AS, Halamka JD, Kinkel RP. Internet portal use in an academic multiple sclerosis center. J Am Med Inform Assoc 2012 Feb;19(1):128-133 [FREE Full text] [CrossRef] [Medline]
    39. Tenforde M, Nowacki A, Jain A, Hickner J. The association between personal health record use and diabetes quality measures. J Gen Intern Med 2012 Apr;27(4):420-424 [FREE Full text] [CrossRef] [Medline]
    40. Johnson AJ, Easterling D, Nelson R, Chen MY, Frankel RM. Access to radiologic reports via a patient portal: clinical simulations to investigate patient preferences. J Am Coll Radiol 2012 Apr;9(4):256-263. [CrossRef] [Medline]
    41. Rosenbloom ST, Daniels TL, Talbot TR, McClain T, Hennes R, Stenner S, et al. Triaging patients at risk of influenza using a patient portal. J Am Med Inform Assoc 2012;19(4):549-554 [FREE Full text] [CrossRef] [Medline]
    42. Noblin AM, Wan TTH, Fottler M. The impact of health literacy on a patient's decision to adopt a personal health record. Perspect Health Inf Manag 2012;9:1-13 [FREE Full text] [Medline]
    43. Wakefield DS, Kruse RL, Wakefield BJ, Koopman RJ, Keplinger LE, Canfield SM, et al. Consistency of patient preferences about a secure Internet-based patient communications portal: contemplating, enrolling, and using. Am J Med Qual 2012;27(6):494-502. [CrossRef] [Medline]
    44. Wagner PJ, Dias J, Howard S, Kintziger KW, Hudson MF, Seol YH, et al. Personal health records and hypertension control: a randomized trial. J Am Med Inform Assoc 2012 Aug;19(4):626-634 [FREE Full text] [CrossRef] [Medline]
    45. Urowitz S, Wiljer D, Dupak K, Kuehner Z, Leonard K, Lovrics E, et al. Improving diabetes management with a patient portal: a qualitative study of diabetes self-management portal. J Med Internet Res 2012;14(6):e158 [FREE Full text] [CrossRef] [Medline]
    46. Tsai J, Rosenheck RA. Use of the internet and an online personal health record system by US veterans: comparison of Veterans Affairs mental health service users and other veterans nationally. J Am Med Inform Assoc 2012;19(6):1089-1094 [FREE Full text] [CrossRef] [Medline]
    47. Emani S, Yamin CK, Peters E, Karson AS, Lipsitz SR, Wald JS, et al. Patient perceptions of a personal health record: a test of the diffusion of innovation model. J Med Internet Res 2012;14(6):e150 [FREE Full text] [CrossRef] [Medline]
    48. Ahlers-Schmidt CR, Nguyen M. Parent intention to use a patient portal as related to their children following a facilitated demonstration. Telemed J E Health 2013 Dec;19(12):979-981. [CrossRef] [Medline]
    49. Schickedanz A, Huang D, Lopez A, Cheung E, Lyles CR, Bodenheimer T, et al. Access, interest, and attitudes toward electronic communication for health care among patients in the medical safety net. J Gen Intern Med 2013 Jul;28(7):914-920 [FREE Full text] [CrossRef] [Medline]
    50. Sanders MR, Winters P, Fortuna RJ, Mendoza M, Berliant M, Clark L, et al. Internet access and patient portal readiness among patients in a group of inner-city safety-net practices. J Ambul Care Manage 2013;36(3):251-259. [CrossRef] [Medline]
    51. Ronda MCM, Dijkhorst-Oei LT, Gorter KJ, Beulens JWJ, Rutten GEHM. Differences between diabetes patients who are interested or not in the use of a patient Web portal. Diabetes Technol Ther 2013 Jul;15(7):556-563 [FREE Full text] [CrossRef] [Medline]
    52. Osborn CY, Mayberry LS, Wallston KA, Johnson KB, Elasy TA. Understanding patient portal use: implications for medication management. J Med Internet Res 2013;15(7):e133 [FREE Full text] [CrossRef] [Medline]
    53. Nazi KM, Hogan TP, McInnes DK, Woods SS, Graham G. Evaluating patient access to Electronic Health Records: results from a survey of veterans. Med Care 2013 Mar;51(3 Suppl 1):S52-S56. [CrossRef] [Medline]
    54. Lyles CR, Sarkar U, Ralston JD, Adler N, Schillinger D, Moffet HH, et al. Patient-provider communication and trust in relation to use of an online patient portal among diabetes patients: The Diabetes and Aging Study. J Am Med Inform Assoc 2013;20(6):1128-1131 [FREE Full text] [CrossRef] [Medline]
    55. Ketterer T, West DW, Sanders VP, Hossain J, Kondo MC, Sharif I. Correlates of patient portal enrollment and activation in primary care pediatrics. Acad Pediatr 2013;13(3):264-271. [CrossRef] [Medline]
    56. Kerns JW, Krist AH, Longo DR, Kuzel AJ, Woolf SH. How patients want to engage with their personal health record: a qualitative study. BMJ Open 2013;3(7):1-10 [FREE Full text] [CrossRef] [Medline]
    57. Hanberger L, Ludvigsson J, Nordfeldt S. Use of a web 2.0 portal to improve education and communication in young patients with families: randomized controlled trial. J Med Internet Res 2013;15(8):e175 [FREE Full text] [CrossRef] [Medline]
    58. Butler JM, Carter M, Hayden C, Gibson B, Weir C, Snow L, et al. Understanding adoption of a personal health record in rural health care clinics: revealing barriers and facilitators of adoption including attributions about potential patient portal users and self-reported characteristics of early adopting users. AMIA Annu Symp Proc 2013;2013:152-161 [FREE Full text] [Medline]
    59. Gu Y, Day K. Propensity of people with long-term conditions to use personal health records. Stud Health Technol Inform 2013;188:46-51. [Medline]
    60. Gerber DE, Laccetti AL, Chen B, Yan J, Cai J, Gates S, et al. Predictors and intensity of online access to electronic medical records among patients with cancer. J Oncol Pract 2014 Sep;10(5):e307-e312 [FREE Full text] [CrossRef] [Medline]
    61. Kondylakis H, Kazantzaki E, Koumakis L, Genitsaridi I, Marias K, Gorini A, et al. Development of interactive empowerment services in support of personalised medicine. Ecancermedicalscience 2014;8:400 [FREE Full text] [CrossRef] [Medline]
    62. Krist AH, Woolf SH, Bello GA, Sabo RT, Longo DR, Kashiri P, et al. Engaging primary care patients to use a patient-centered personal health record. Ann Fam Med 2014;12(5):418-426 [FREE Full text] [CrossRef] [Medline]
    63. Jung C, Padman R. Virtualized healthcare delivery: understanding users and their usage patterns of online medical consultations. Int J Med Inform 2014 Dec;83(12):901-914. [CrossRef] [Medline]
    64. Haun JN, Lind JD, Shimada SL, Martin TL, Gosline RM, Antinori N, et al. Evaluating user experiences of the secure messaging tool on the Veterans Affairs' patient portal system. J Med Internet Res 2014;16(3):e75 [FREE Full text] [CrossRef] [Medline]
    65. van der Vaart R, Drossaert CHC, Taal E, Drossaers-Bakker KW, Vonkeman HE, van de Laar MAFJ. Impact of patient-accessible electronic medical records in rheumatology: use, satisfaction and effects on empowerment among patients. BMC Musculoskelet Disord 2014;15:102 [FREE Full text] [CrossRef] [Medline]
    66. Ancker JS, Silver M, Kaushal R. Rapid growth in use of personal health records in New York, 2012-2013. J Gen Intern Med 2014 Jun;29(6):850-854. [CrossRef] [Medline]
    67. Abramson EL, Patel V, Edwards A, Kaushal R. Consumer perspectives on personal health records: a 4-community study. Am J Manag Care 2014 Apr;20(4):287-296 [FREE Full text] [Medline]
    68. Phelps RG, Taylor J, Simpson K, Samuel J, Turner AN. Patients' continuing use of an online health record: a quantitative evaluation of 14,000 patient years of access data. J Med Internet Res 2014;16(10):e241 [FREE Full text] [CrossRef] [Medline]
    69. Neuner J, Fedders M, Caravella M, Bradford L, Schapira M. Meaningful use and the patient portal: patient enrollment, use, and satisfaction with patient portals at a later-adopting center. Am J Med Qual 2015;30(2):105-113. [CrossRef] [Medline]
    70. Riippa I, Linna M, Rönkkö I. The effect of a patient portal with electronic messaging on patient activation among chronically ill patients: controlled before-and-after study. J Med Internet Res 2014;16(11):e257 [FREE Full text] [CrossRef] [Medline]
    71. Ancker JS, Osorio SN, Cheriff A, Cole CL, Silver M, Kaushal R. Patient activation and use of an electronic patient portal. Inform Health Soc Care 2014 Apr 30:1-13. [CrossRef] [Medline]
    72. Lober WB, Zierler B, Herbaugh A, Shinstrom SE, Stolyar A, Kim EH, et al. Barriers to the use of a personal health record by an elderly population. AMIA Annu Symp Proc 2006:514-518 [FREE Full text] [Medline]
    73. Taha J, Czaja SJ, Sharit J, Morrow DG. Factors affecting usage of a personal health record (PHR) to manage health. Psychol Aging 2013 Dec;28(4):1124-1139. [CrossRef] [Medline]
    74. Sarkar U, Karter AJ, Liu JY, Adler NE, Nguyen R, Lopez A, et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010;15 Suppl 2:183-196 [FREE Full text] [CrossRef] [Medline]
    75. Logue MD, Effken JA. Modeling factors that influence personal health records adoption. Comput Inform Nurs 2012 Jul;30(7):354-362. [CrossRef] [Medline]
    76. Wynia MK, Torres GW, Lemieux J. Many physicians are willing to use patients' electronic personal health records, but doctors differ by location, gender, and practice. Health Aff (Millwood) 2011 Feb;30(2):266-273 [FREE Full text] [CrossRef] [Medline]
    77. Nazi KM. The personal health record paradox: health care professionals' perspectives and the information ecology of personal health record systems in organizational and clinical settings. J Med Internet Res 2013;15(4):e70 [FREE Full text] [CrossRef] [Medline]
    78. Yau GL, Williams AS, Brown JB. Family physicians' perspectives on personal health records: qualitative study. Can Fam Physician 2011 May;57(5):e178-e184 [FREE Full text] [Medline]
    79. van der Vaart R, Drossaert CHC, Taal E, van de Laar MAFJ. Giving rheumatology patients online home access to their electronic medical record (EMR): advantages, drawbacks and preconditions according to care providers. Rheumatol Int 2013 Sep;33(9):2405-2410 [FREE Full text] [CrossRef] [Medline]
    80. Urowitz S, Wiljer D, Apatu E, Eysenbach G, Delenardo C, Harth T, et al. Is Canada ready for patient accessible electronic health records? A national scan. BMC Med Inform Decis Mak 2008;8:33 [FREE Full text] [CrossRef] [Medline]
    81. Witry MJ, Doucette WR, Daly JM, Levy BT, Chrischilles EA. Family physician perceptions of personal health records. Perspect Health Inf Manag 2010;7:1d [FREE Full text] [Medline]
    82. Pai HH, Lau F, Barnett J, Jones S. Meeting the health information needs of prostate cancer patients using personal health records. Curr Oncol 2013 Dec;20(6):e561-e569 [FREE Full text] [CrossRef] [Medline]
    83. Selden CR, Zorn M, Ratzan SC, Parker RM. National Library of Medicine Current Bibliographies in Medicine: Health Literacy. Washington, DC: U.S. Government Printing Office; 2000. US Department of Health and Human Services. Healthy People 2010. Introduction   URL: http://chirr.nlm.nih.gov/health-literacy.php [accessed 2015-06-02] [WebCite Cache]
    84. Sox CM, Gribbons WM, Loring BA, Mandl KD, Batista R, Porter SC. Patient-centered design of an information management module for a personally controlled health record. J Med Internet Res 2010;12(3):e36 [FREE Full text] [CrossRef] [Medline]
    85. Haggstrom DA, Saleem JJ, Russ AL, Jones J, Russell SA, Chumbler NR. Lessons learned from usability testing of the VA's personal health record. J Am Med Inform Assoc 2011 Dec;18 Suppl 1:i13-i17 [FREE Full text] [CrossRef] [Medline]
    86. Krist AH, Peele E, Woolf SH, Rothemich SF, Loomis JF, Longo DR, et al. Designing a patient-centered personal health record to promote preventive care. BMC Med Inform Decis Mak 2011;11:73 [FREE Full text] [CrossRef] [Medline]
    87. Woods SS, Schwartz E, Tuepker A, Press NA, Nazi KM, Turvey CL, et al. Patient experiences with full electronic access to health records and clinical notes through the My HealtheVet Personal Health Record Pilot: qualitative study. J Med Internet Res 2013;15(3):e65 [FREE Full text] [CrossRef] [Medline]
    88. Monkman H, Kushniruk A. Applying usability methods to identify health literacy issues: an example using a Personal Health Record. Stud Health Technol Inform 2013;183:179-185. [Medline]
    89. Zarcadoolas C, Vaughon WL, Czaja SJ, Levy J, Rockoff ML. Consumers' perceptions of patient-accessible electronic medical records. J Med Internet Res 2013;15(8):e168 [FREE Full text] [CrossRef] [Medline]
    90. Krist AH, Woolf SH, Rothemich SF, Johnson RE, Peele JE, Cunningham TD, et al. Interactive preventive health record to enhance delivery of recommended care: a randomized trial. Ann Fam Med 2012;10(4):312-319 [FREE Full text] [CrossRef] [Medline]
    91. Taha J, Sharit J, Czaja SJ. The impact of numeracy ability and technology skills on older adults' performance of health management tasks using a patient portal. J Appl Gerontol 2014 Jun;33(4):416-436. [CrossRef] [Medline]
    92. Schnipper JL, Gandhi TK, Wald JS, Grant RW, Poon EG, Volk LA, et al. Design and implementation of a web-based patient portal linked to an electronic health record designed to improve medication safety: the Patient Gateway medications module. Inform Prim Care 2008;16(2):147-155. [Medline]
    93. Fuji KT, Abbott AA, Galt KA. Personal health record design: qualitative exploration of issues inhibiting optimal use. Diabetes Care 2014;37(1):e13-e14 [FREE Full text] [CrossRef] [Medline]
    94. Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc 2009;16(5):660-669 [FREE Full text] [CrossRef] [Medline]
    95. Kuziemsky CE, Kushniruk A. Context mediated usability testing. Stud Health Technol Inform 2014;205:905-909. [Medline]
    96. Do NV, Barnhill R, Heermann-Do KA, Salzman KL, Gimbel RW. The military health system's personal health record pilot with Microsoft HealthVault and Google Health. J Am Med Inform Assoc 2011;18(2):118-124 [FREE Full text] [CrossRef] [Medline]
    97. Gaskin GL, Longhurst CA, Slayton R, Das AK. Sociotechnical Challenges of Developing an Interoperable Personal Health Record: Lessons Learned. Appl Clin Inform 2011;2(4):406-419 [FREE Full text] [CrossRef] [Medline]
    98. Ennis L, Robotham D, Denis M, Pandit N, Newton D, Rose D, et al. Collaborative development of an electronic Personal Health Record for people with severe and enduring mental health problems. BMC Psychiatry 2014;14(1):305 [FREE Full text] [CrossRef] [Medline]
    99. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004 Feb;37(1):56-76. [CrossRef] [Medline]
    100. Tuil WS, ten Hoopen AJ, Braat DDM, de Vries RPF, Kremer JAM. Patient-centred care: using online personal medical records in IVF practice. Hum Reprod 2006 Nov;21(11):2955-2959 [FREE Full text] [CrossRef] [Medline]
    101. Grant RW, Wald JS, Poon EG, Schnipper JL, Gandhi TK, Volk LA, et al. Design and implementation of a web-based patient portal linked to an ambulatory care electronic health record: patient gateway for diabetes collaborative care. Diabetes Technol Ther 2006 Oct;8(5):576-586 [FREE Full text] [CrossRef] [Medline]
    102. Hess R, Bryce CL, Paone S, Fischer G, McTigue KM, Olshansky E, et al. Exploring challenges and potentials of personal health records in diabetes self-management: implementation and initial assessment. Telemed J E Health 2007 Oct;13(5):509-517. [CrossRef] [Medline]
    103. Kim EH, Stolyar A, Lober WB, Herbaugh AL, Shinstrom SE, Zierler BK, et al. Usage patterns of a personal health record by elderly and disabled users. AMIA Annu Symp Proc 2007:409-413 [FREE Full text] [Medline]
    104. Bergman DA, Brown NL, Wilson S. Teen use of a patient portal: a qualitative study of parent and teen attitudes. Perspect Health Inf Manag 2008;5:13 [FREE Full text] [Medline]
    105. Green BB, Cook AJ, Ralston JD, Fishman PA, Catz SL, Carlson J, et al. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. JAMA 2008 Jun 25;299(24):2857-2867 [FREE Full text] [CrossRef] [Medline]
    106. Staroselsky M, Volk LA, Tsurikova R, Newmark LP, Lippincott M, Litvak I, et al. An effort to improve electronic health record medication list accuracy between visits: patients' and physicians' response. Int J Med Inform 2008 Mar;77(3):153-160. [CrossRef] [Medline]
    107. Wald JS, Grant RW, Schnipper JL, Gandhi TK, Poon EG, Businger AC, et al. Survey analysis of patient experience using a practice-linked PHR for type 2 diabetes mellitus. AMIA Annu Symp Proc 2009;2009:678-682 [FREE Full text] [Medline]
    108. Leveille SG, Huang A, Tsai SB, Allen M, Weingart SN, Iezzoni LI. Health coaching via an internet portal for primary care patients with chronic conditions: a randomized controlled trial. Med Care 2009 Jan;47(1):41-47. [CrossRef] [Medline]
    109. Kim EH, Stolyar A, Lober WB, Herbaugh AL, Shinstrom SE, Zierler BK, et al. Challenges to using an electronic personal health record by a low-income elderly population. J Med Internet Res 2009;11(4):e44 [FREE Full text] [CrossRef] [Medline]
    110. Wiljer D, Leonard KJ, Urowitz S, Apatu E, Massey C, Quartey NK, et al. The anxious wait: assessing the impact of patient accessible EHRs for breast cancer patients. BMC Med Inform Decis Mak 2010;10:46 [FREE Full text] [CrossRef] [Medline]
    111. Reti SR, Feldman HJ, Ross SE, Safran C. Improving personal health records for patient-centered care. J Am Med Inform Assoc 2010;17(2):192-195 [FREE Full text] [CrossRef] [Medline]
    112. Shaw RJ, Ferranti J. Patient-provider internet portals--patient outcomes and use. Comput Inform Nurs 2011 Dec;29(12):714-8; quiz 719. [CrossRef] [Medline]
    113. Wright A, Poon EG, Wald J, Feblowitz J, Pang JE, Schnipper JL, et al. Randomized controlled trial of health maintenance reminders provided directly to patients through an electronic PHR. J Gen Intern Med 2012 Jan;27(1):85-92 [FREE Full text] [CrossRef] [Medline]
    114. Schnipper JL, Gandhi TK, Wald JS, Grant RW, Poon EG, Volk LA, et al. Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial. J Am Med Inform Assoc 2012;19(5):728-734 [FREE Full text] [CrossRef] [Medline]
    115. Palen TE, Ross C, Powers JD, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. JAMA 2012 Nov 21;308(19):2012-2019. [CrossRef] [Medline]
    116. Schnipper JL, Gandhi TK, Wald JS, Grant RW, Poon EG, Volk LA, et al. Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial. J Am Med Inform Assoc 2012;19(5):728-734 [FREE Full text] [CrossRef] [Medline]
    117. Wade-Vuturo AE, Mayberry LS, Osborn CY. Secure messaging and diabetes management: experiences and perspectives of patient portal users. J Am Med Inform Assoc 2013 May 1;20(3):519-525 [FREE Full text] [CrossRef] [Medline]
    118. Sarkar U, Lyles CR, Parker MM, Allen J, Nguyen R, Moffet HH, et al. Use of the refill function through an online patient portal is associated with improved adherence to statins in an integrated health system. Med Care 2014 Mar;52(3):194-201 [FREE Full text] [CrossRef] [Medline]
    119. Otsuka SH, Tayal NH, Porter K, Embi PJ, Beatty SJ. Improving herpes zoster vaccination rates through use of a clinical pharmacist and a personal health record. Am J Med 2013 Sep;126(9):832.e1-832.e6 [FREE Full text] [CrossRef] [Medline]
    120. North F, Crane SJ, Stroebel RJ, Cha SS, Edell ES, Tulledge-Scheitel SM. Patient-generated secure messages and eVisits on a patient portal: are patients at risk? J Am Med Inform Assoc 2013;20(6):1143-1149 [FREE Full text] [CrossRef] [Medline]
    121. North F, Crane SJ, Chaudhry R, Ebbert JO, Ytterberg K, Tulledge-Scheitel SM, et al. Impact of patient portal secure messages and electronic visits on adult primary care office visits. Telemed J E Health 2014 Mar;20(3):192-198 [FREE Full text] [CrossRef] [Medline]
    122. Murray MF, Giovanni MA, Klinger E, George E, Marinacci L, Getty G, et al. Comparing electronic health record portals to obtain patient-entered family health history in primary care. J Gen Intern Med 2013 Dec;28(12):1558-1564 [FREE Full text] [CrossRef] [Medline]
    123. Heyworth L, Paquin AM, Clark J, Kamenker V, Stewart M, Martin T, et al. Engaging patients in medication reconciliation via a patient portal following hospital discharge. J Am Med Inform Assoc 2014 Feb;21(e1):e157-e162 [FREE Full text] [CrossRef] [Medline]
    124. Britto MT, Hesse EA, Kamdar OJ, Munafo JK. Parents' perceptions of a patient portal for managing their child's chronic illness. J Pediatr 2013 Jul;163(1):280-1.e1. [CrossRef] [Medline]
    125. Aberger EW, Migliozzi D, Follick MJ, Malick T, Ahern DK. Enhancing patient engagement and blood pressure management for renal transplant recipients via home electronic monitoring and web-enabled collaborative care. Telemed J E Health 2014 Sep;20(9):850-854. [CrossRef] [Medline]
    126. Barron J, Bedra M, Wood J, Finkelstein J. Exploring three perspectives on feasibility of a patient portal for older adults. Stud Health Technol Inform 2014;202:181-184. [Medline]
    127. Borbolla D, Del FG, Taliercio V, Otero C, Campos F, Martinez M, et al. Integrating personalized health information from MedlinePlus in a patient portal. Stud Health Technol Inform 2014;205:348-352. [Medline]
    128. Wright A, Feblowitz J, Maloney FL, Henkin S, Ramelson H, Feltman J, et al. Increasing patient engagement: patients' responses to viewing problem lists online. Appl Clin Inform 2014;5(4):930-942. [CrossRef] [Medline]
    129. Tom JO, Chen C, Zhou YY. Personal health record use and association with immunizations and well-child care visits recommendations. J Pediatr 2014 Jan;164(1):112-117. [CrossRef] [Medline]
    130. Nazi KM, Turvey CL, Klein DM, Hogan TP, Woods SS. VA OpenNotes: exploring the experiences of early patient adopters with access to clinical notes. J Am Med Inform Assoc 2015 Mar;22(2):380-389. [CrossRef] [Medline]
    131. Lau M, Campbell H, Tang T, Thompson DJS, Elliott T. Impact of patient use of an online patient portal on diabetes outcomes. Can J Diabetes 2014 Feb;38(1):17-21. [CrossRef] [Medline]
    132. Hess R, Fischer GS, Sullivan SM, Dong X, Weimer M, Zeith C, et al. Patterns of response to patient-centered decision support through a personal health record. Telemed J E Health 2014 Nov;20(11):984-989. [CrossRef] [Medline]
    133. Hall MJ, Herda MM, Handorf EA, Rybak CC, Keleher CA, Siemon M, et al. Direct-to-patient disclosure of results of mismatch repair screening for Lynch syndrome via electronic personal health record: a feasibility study. Genet Med 2014 Nov;16(11):854-861. [CrossRef] [Medline]
    134. Druss BG, Ji X, Glick G, von Esenwein SA. Randomized trial of an electronic personal health record for patients with serious mental illnesses. Am J Psychiatry 2014 Mar;171(3):360-368. [CrossRef] [Medline]
    135. Crotty BH, Tamrat Y, Mostaghimi A, Safran C, Landon BE. Patient-to-physician messaging: volume nearly tripled as more patients joined system, but per capita rate plateaued. Health Aff (Millwood) 2014 Oct;33(10):1817-1822. [CrossRef] [Medline]
    136. Chrischilles EA, Hourcade JP, Doucette W, Eichmann D, Gryzlak B, Lorentzen R, et al. Personal health records: a randomized trial of effects on elder medication safety. J Am Med Inform Assoc 2014;21(4):679-686. [CrossRef] [Medline]
    137. Cabitza F, Simone C, De MG. User-driven prioritization of features for a prospective InterPersonal Health Record: perceptions from the Italian context. Comput Biol Med 2015 Apr;59:202-210. [CrossRef] [Medline]
    138. Hargreaves JS. Will electronic personal health records benefit providers and patients in rural America? Telemed J E Health 2010 Mar;16(2):167-176. [CrossRef] [Medline]
    139. Guy R, Hocking J, Wand H, Stott S, Ali H, Kaldor J. How effective are short message service reminders at increasing clinic attendance? A meta-analysis and systematic review. Health Serv Res 2012 Apr;47(2):614-632 [FREE Full text] [CrossRef] [Medline]
    140. Simmons LA, Wolever RQ, Bechard EM, Snyderman R. Patient engagement as a risk factor in personalized health care: a systematic review of the literature on chronic disease. Genome Med 2014;6(2):16 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    AHRQ: Agency for Health care Research and Quality
    ePHR: electronic personal health record
    EHR: electronic health record
    CMS: Centers for Medicare and Medicaid Services
    MU: meaningful use
    SM: secure messaging


    Edited by G Eysenbach; submitted 20.01.15; peer-reviewed by D Amante, A Benis; comments to author 05.02.15; revised version received 10.03.15; accepted 03.04.15; published 23.06.15

    ©Taya Irizarry, Annette DeVito Dabbs, Christine R Curran. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 23.06.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.